[Feat] Add quantized exports for Resnet18 and LeNet

Context

This MR aims to enable the quantized (int8) export of LeNet and Resnet18.

It brings a lot of changes to clean and update the CPP Aidge export with the latest features.

Refactor

• Operators register files are splitted from one global operators.py file to several ones, each matching the actual implementation of one given kernel.
• Some utils functions are added to simplify the export file :
  • cpp_fuse_to_metaops() : Fuse the model nodes into the metaoperators corresponding to the operators supported by the export.
  • set_nodes_names() : Rename the newly created nodes based on their type, position in the graph and number of iterations of this specific layer, to ensure readability once exported.
  • set_nodes_datatypes() : Datatypes have to be set manually after the quantization step.
  • read_log_file() : May be useful to compare intermediate results...

New features

General improvements

• Now possible to ignore some producers during export phase, as some of them are not relevant (for instance, the producers holding scaling coefficients, which are handled separately).
• Change the kernels_to_copy export node system as the old system had some limitations. Indeed, it was not possible to chose a destination path different from the source path. Also the copied kernel would be automatically included in the fwd file, sometime leading to unused includes. The new system is based on a list of dict holding, for each kernel file, the source and destination paths as well as a boolean to indicate if the kernel needs to be included in the forward file.
• Add an aidge_cmp feature which allows to compare the aidge_export_cpp and aidge_backend_cpu intermediate tensor results for the exported model.

New models examples

• Add LeNet generation file (create_lenet.py) which create, train and export a LeNet model.
• Add a lenet.py file describing the different steps to perform an export with the aidge_export_cpp module.
• Add a resnet18.py file describing the different steps to perform an export with the aidge_export_cpp module.
• Add these example scripts to the CI

Quantized Export

• Handle the scaling nodes added by the PTQ and transmit the shift and coef parameters to the generated export.
• Add scaling functions (SingleShift and FixedPoint) to the export.
• Add a aidge_cmp option which exports the log_outputs (intermediate tensors generated by the aidge_backend_cpu) as reference and allow data comparison at inference time.
• Now possible to export quantized LeNet using --dtype int8 while running the lenet.py file.
• Now possible to export quantized Resnet18 using --dtype int8 while running the resnet18.py file.

Related Issues

[ResNet18] Export Cpp INT8 not supported

TODO

• LeNet Export fp32
• LeNet Export int8
• Resnet18 Export fp32
• Resnet18 Export int8

aidge_export_cpp/kernels/activation.hpp

 #ifndef __AIDGE_EXPORT_CPP_KERNELS_ACTIVATION__
 #define __AIDGE_EXPORT_CPP_KERNELS_ACTIVATION__
 
-#include <type_traits>
-#include "network/typedefs.hpp"
-#include "network/utils.hpp"
-#include "kernels/rescaling.hpp"
-
-template<typename Output_T, typename T,
-         typename std::enable_if<std::is_floating_point<T>::value>::type* = nullptr>
-__attribute__((always_inline)) inline
-Output_T saturate (T value, int32_t /*sat*/)
-{
-    return value;
-}
-
-template<typename Output_T, typename T,
-         typename std::enable_if<!std::is_floating_point<T>::value>::type* = nullptr>
-__attribute__((always_inline)) inline
-Output_T saturate (T value, uint32_t sat)
-{
-    if (std::is_unsigned<Output_T>::value) {
-        return clamp(value, T(0), (T(1) << sat) - 1);
-    } else {
-        return clamp(value, -(T(1) << (sat - 1)), (T(1) << (sat - 1)) - 1);
-    }
-}
-
-template<typename Output_T,
-         typename Sum_T,
-         typename Rescaling_T>
-__attribute__((always_inline)) inline
-Output_T activation_forward_value (Sum_T weightedSum,
-                                   int output,
-                                   ActivationFunction_T func,
-                                   const Rescaling_T& __restrict rescaling)
-{
-    switch(func) {
-        case Linear:
-        case Saturation: {
-            break;
-        }
-        case Rectifier: {
-            if(weightedSum <= 0) weightedSum = 0;
-            break;
-        }
-        default:
-            // Unsupported activation function
-            break;
-    }
-
-    // Value fixed here for now but it should be generated by
-    // the export module or determined by the type of Output_T
-    // For now only works for int8_t and uint8_t
-    const uint32_t NB_BITS = 8;
-    return saturate<Output_T>(rescaling(weightedSum, output), NB_BITS);
-}
-
+#include "network/activation_utils.hpp"
+#include "network/rescaling_utils.hpp"
 
 template<int NB_DATA,
          ActivationFunction_T ACTIVATION,