Gathering quantization result on ResNet18 & MobileNetV1

gathering quantization results for ResNet18 and MobileNetV2 using Post-Training Quantization (PTQ)

added StatusWork in Progress label

assigned to @noamzerah

added to epic &15

created branch 74-gathering-quantization-result-on-resnet18-mobilenetv2 to address this issue

8 Bit Quantization Results for ResNet18

Method	Accuracy (%)
Baseline	71.0
Fake Quantized (without single shift)	70.6
Fake Quantized (with single shift)	68.8
True Quantized INT32 (without single shift)	70.3
True Quantized INT32 (with single shift)	68.7

Why "true" quantized int32? You should set dynamic to int8!

Also you shouldn't have an error between Fake and True quantize.

The overflow error #75 shouldn't affect you sicne you are in int32. But still I think you have an issue...

When I say "true quantized INT32," I'm referring to the current PTQ implementation, where all types are cast to INT32. This is done to test the execution of integer-based kernels with real bit-shifts, as the full conversion to INT8 is not yet available.

Regarding the slight accuracy drop between Fake and True Quantization, it primarily comes from the execution of certain kernels that are inherently less precise in integer arithmetic compared to floating point. For example, Global Average Pooling involves a division operation, which can introduce a loss of precision when performed in integer format.

Ok, what is the scope of this issue?

Do you plan on adding this table to the documentation? If so you need to make a MR in aidge not in quantization!

Do you plan on testing inference with int8 kernels?

Yes, I plan to add the results to the documentation later. For now, I’m using this issue to keep track of the results that are already available.

As for INT8 inference, I intend to test the models as soon as the feature becomes available.

@noamzerah as discussed in this moning meeting you should also benchmark MobileNetv1! :)

changed title from Gathering quantization result on ResNet18 & MobileNetV2 to Gathering quantization result on ResNet18 & MobileNetV1