Add support for QLinearConv

The goal of this issue is to add support for QLinearConv Operator https://github.com/onnx/onnx/blob/main/docs/Operators.md#QLinearConv

To do so, I propose to add a function to fuse all Conv + Scaling operator into a MetaOperator QLinearConv. This function will be left to the user to call depending on how he wants to export his quantized network. (Exporting each operator idependantly or using QOp from ONNX)

Then we will need to add the support for the export of this new MetaOperator.

TODO:

import

• Add support in import node for MetaOperators QLinearConv, QuantizeLinear and DequantizeLinear
• Perform inferences in the exported model and onnx model and verify results (testing with multiple ORT Session options may be needed)

export

• Add/fuse QuantizeLinear - Add DequantizeLinear Meraoperators
• Fuse into QLinearConv Metaoperator
• Modify graph into ONNX's quantized format
• Add support in export node for a MetaOperators QLinearConv, QuantizeLinear and DequantizeLinear
• Perform inferences in the exported model and aidge model and verify results (testing with multiple ORT Session options may be needed)
• Method to freeze weights and biases to be able to fold into quantized producers
• Method for the removal of quantize and dequantize operators (they will cancel each other out)

verifications

• compare imported metaoperators and exported metaoperators (after recipe)
• correct export of imported quantized ONNX model

Note: Due to future removal of Scaling node the regex to use in aidge_core.fuse_to_metaops() may change but this is a minor change

added Feature 🚀 label

assigned to @lucaslopez

marked the checklist item Create a simple network with one Conv as completed

marked the checklist item Quantize this network with random data as completed

marked the checklist item Fuse Conv and Scaling using the aidge_core.fuse_to_metaops() function into a MetaOperator QLinearConv as completed

added StatusWork in Progress label

changed the description

marked the checklist item Add support in export node for a MetaOperator QLinearConv as completed

changed the description

Can this also be done for Gemm + Scaling being transformed to a QLinearMatMul MetaOperator ?

Hello @julienl

Yes it will be done next, we are currently experimenting with the QLinearConv and trying to understand the ONNX formalism

Cheers,

Cyril

Okay, thank you for the update

We have an issue with the export to QLinearConv, in current Aidge PTQ, scaling factor is hidden inside the weights and so we cannot export it right away.

@noamzerah and @bhalimi are gonna work on changing Aidge PTQ method to expose the scaling factor.

mentioned in merge request !90 (merged)

mentioned in merge request aidge_backend_cpu!106 (merged)

mentioned in merge request aidge_core!251 (merged)

mentioned in issue aidge_quantization#47 (closed)

assigned to @noamzerah

marked the checklist item Perform inferences in the exported model and aidge model as completed

marked the checklist item Perform inferences in the exported model and aidge model as incomplete

added #48 as child task

mentioned in merge request aidge_quantization!33 (merged)

closed

mentioned in issue aidge_core#230 (closed)

reopened

I would like to highlight that the implementation of this method has also led to a slight performance improvement when using the single shift approximation. In the previous approach, the values of the producers' tensors were directly scaled and then rounded. During the application of the single shift, we would rescale the values from the rounded result and subsequently round them again. The accumulation of these two rounding steps resulted in a greater loss of precision. In the current solution, by introducing scaling nodes and utilizing a single rounding step, we are able to "stack" the different coefficients within the scaling node, thus reducing the rounding to only one step.

mentioned in issue aidge#246

mentioned in merge request aidge_quantization!41

mentioned in merge request !112

changed the description

assigned to @cmoineau

unassigned @noamzerah

mentioned in issue #61