feat: Add missing operators for AIDGE model benchmarking

Context

I am working on benchmarking AIDGE models by fully creating them in AIDGE, setting their weights randomly, and exporting them to C++. The goal is to verify if the outputs in C++ match the forward pass results in AIDGE. You can find it here

To achieve this, I have added several missing operators necessary for running models such as:

• SqueezeNet
• ResNet18 / ResNet50
• MLP for MNIST
• MobileNet

These operators are now available in the export pipeline to ensure proper benchmarking.

Modified Files

• Operators & Kernels

  • aidge_export_cpp/kernels/batchnorm.hpp: Added missing batch normalization operator.
  • aidge_export_cpp/kernels/concat.hpp: Added concatenation operator.
  • aidge_export_cpp/kernels/pad.hpp: Added padding operator.
  • aidge_export_cpp/kernels/pooling.hpp: Added AvgPooling2D.
  • aidge_export_cpp/kernels/softmax.hpp: Added softmax operator.

• Python Script

  • aidge_export_cpp/operators.py: Updated operator handling to include new additions.

• Configuration Templates

  • aidge_export_cpp/templates/configuration/*: Added configurations for batchnorm, concat, pad, and softmax.

• Kernel Forward Templates

  • aidge_export_cpp/templates/kernel_forward/*: Added missing forward implementations for batchnorm, concat, pad, and softmax.

• Unit Tests

  • aidge_export_cpp/unit_tests/test_export.py: Updated tests to validate newly added operators.

---

Detailed Major Modifications

• Implemented missing operators to support model export and validation in C++.
• Ensured each new operator follows the existing API.
• Updated test cases to validate correctness of the exported model execution.
• Added the possibility of setting the weights inside the model randomly or with a default value to stay deterministic for the unit tests

TODO

• Add some of the missing args of Pad2D. Do add assertion for now that attribute for padding is constant fill (i.e.do not support other attributes such as mirror etc...).

aidge_export_cpp/kernels/batchnorm.hpp

@@ -2,16 +2,18 @@
 #define __AIDGE_EXPORT_CPP_KERNELS_BATCHNORM__
 
 #include "network/typedefs.hpp"
-#include "kernels/rescaling.hpp"
+#include "kernels/activation.hpp"
+
 #include <math.h>
 
 // WARNING: this kernel only works for 32-bits floating point values
 
-template<int NB_OUTPUTS,
+template<int NB_BATCHES, int NB_OUTPUTS,
          int OUTPUTS_HEIGHT, int OUTPUTS_WIDTH,
          ActivationFunction_T ACTIVATION,
          typename Input_T, typename Output_T,
-         typename Param_T>
+         typename Param_T,
+         typename Rescaling_T>
 __attribute__((always_inline)) inline
 void batchnorm_forward (
     const Input_T* __restrict inputs,
@@ -20,18 +22,22 @@ void batchnorm_forward (
     const Param_T* __restrict variances,
     const Param_T* __restrict means,
     const Param_T* __restrict scales,
-    const double epsilon)
+    const double epsilon,
+    const Rescaling_T& __restrict rescaling)
 {
-    for (unsigned int output = 0; output < NB_OUTPUTS; ++output) {
-        const Output_T var = sqrt(variances[output] + epsilon);
+    for (unsigned int batch = 0; batch < NB_BATCHES; ++batch) {
+        for (unsigned int output = 0; output < NB_OUTPUTS; ++output) {
+            // If the variance is 0, we need to avoid division by 0
+            Output_T var = sqrt(variances[output] > 0.0 ? variances[output] + epsilon : epsilon);
 
-        for (int oy = 0; oy < OUTPUTS_HEIGHT; ++oy) {
-            for (int ox = 0; ox < OUTPUTS_WIDTH; ++ox) {
-                const int outputOffset = OUTPUTS_HEIGHT * oy + ox;
+            for (int oy = 0; oy < OUTPUTS_HEIGHT; ++oy) {
+                for (int ox = 0; ox < OUTPUTS_WIDTH; ++ox) {
+                    const int outputOffset = batch * OUTPUTS_WIDTH * OUTPUTS_HEIGHT * NB_OUTPUTS + output * OUTPUTS_WIDTH * OUTPUTS_HEIGHT + OUTPUTS_WIDTH * oy + ox;
 
-                const Output_T normalized = (inputs[outputOffset + output] - means[output]) / var;
-                const Output_T sAs = scales[output] * normalized + biases[output];
-                outputs[outputOffset + output] = sat<Output_T>(sAs, output, ACTIVATION, NoScaling);
+                    const Output_T normalized = (inputs[outputOffset] - means[output]) / var;
+                    const Output_T sAs = scales[output] * normalized + biases[output];
+                    outputs[outputOffset] = activation_forward_value<Output_T>(sAs, output, ACTIVATION, rescaling);
+                }
             }
         }
     }