Initial version of hybrid C++/Python static analysis

This MR builds on !159 (closed) ideas to provide a native C++ and Python interface to static analysis. The main features are:

• Base C++ classes OperatorStats and StaticAnalysis, binded and extendable in Python;
• Registry for Operator static analysis is in C++, making it possible to use and extend it in Python while accessing some base methods in C++ (requirement for implementing graph transformations heuristics based on static analysis in C++);
• summary() function similar to Keras;
• Fix possible segfault in DynamicAttributes::dict() method;
• Added a Python simplify_graph() recipe, that currently simplifies DinoV2.

TODO

• Correctly compute number of parameters. This is trivial for Conv and FC, because they have a Param input category, but how to handle other operators like MatMul or Add? I struggle to find a simple solution here! Solution is to count the direct input Producers of nodes as well as internal Producers (in case of meta-ops). Known limitation: if the input Producer is in the GraphView, it will be counted as parameters.

TODO later:

• Add all operators support (only a dozen for now) too much work for this MR. Unsupported operators are clearly indicated in the plot results (label marked in red).

Usage example

from aidge_core.static_analysis import StaticAnalysisExt

stats = StaticAnalysisExt(model)
stats.summary()
stats.log_nb_params("stats.png", title="Nb params per operator")
stats.log_nb_mac_ops("stats_macs.png", log_scale=True)

--------------------------------------------------------------------------------
                        Layer (type)               Output Shape         Param #
================================================================================
                       (FusedConv#0)          [1, 32, 112, 112]             896
                       (FusedConv#1)          [1, 32, 112, 112]             320
                       (FusedConv#2)          [1, 64, 112, 112]            2112
                       (FusedConv#3)            [1, 64, 56, 56]             640
                       (FusedConv#4)           [1, 128, 56, 56]            8320
                       (FusedConv#5)           [1, 128, 56, 56]            1280
                       (FusedConv#6)           [1, 128, 56, 56]           16512
                       (FusedConv#7)           [1, 128, 28, 28]            1280
                       (FusedConv#8)           [1, 256, 28, 28]           33024
                       (FusedConv#9)           [1, 256, 28, 28]            2560
                      (FusedConv#10)           [1, 256, 28, 28]           65792
                      (FusedConv#11)           [1, 256, 14, 14]            2560
                      (FusedConv#12)           [1, 512, 14, 14]          131584
                      (FusedConv#13)           [1, 512, 14, 14]            5120
                      (FusedConv#14)           [1, 512, 14, 14]          262656
                      (FusedConv#15)           [1, 512, 14, 14]            5120
                      (FusedConv#16)           [1, 512, 14, 14]          262656
                      (FusedConv#17)           [1, 512, 14, 14]            5120
                      (FusedConv#18)           [1, 512, 14, 14]          262656
                      (FusedConv#19)           [1, 512, 14, 14]            5120
                      (FusedConv#20)           [1, 512, 14, 14]          262656
                      (FusedConv#21)           [1, 512, 14, 14]            5120
                      (FusedConv#22)           [1, 512, 14, 14]          262656
                      (FusedConv#23)             [1, 512, 7, 7]            5120
                      (FusedConv#24)            [1, 1024, 7, 7]          525312
                      (FusedConv#25)            [1, 1024, 7, 7]           10240
                      (FusedConv#26)            [1, 1024, 7, 7]         1049600
================================================================================
Total params: 3196032
--------------------------------------------------------------------------------
Input size (MB): 0
Forward/backward pass size (MB): 0.60113525390625
Params size (MB): 12.19189453125
Estimated Total Size (MB): 12.79302978515625
--------------------------------------------------------------------------------

aidge_core/simplify_graph.py

+import numpy as np
+import aidge_core
+
+def simplify_graph(graph: aidge_core.GraphView):
+    """
+    Simplify a graph loaded from ONNX.
+
+    :param graph: The GraphView to simplify.
+    :type graph: aidge_core.GraphView
+    """
+
+    def check_constant_producer(value):
+        def _check_constant_producer(node):
+            out = node.get_operator().get_output(0)
+            return (len(out) == 1 and np.isclose(out[0], value))
+        return _check_constant_producer
+
+    gm = aidge_core.SinglePassGraphMatching(graph)
+    gm.add_node_lambda("Constant_sqrt2", check_constant_producer(np.sqrt(2)))
+    gm.add_node_lambda("Constant_1", check_constant_producer(1))
+    gm.add_node_lambda("Constant_0_5", check_constant_producer(0.5))
+
+    # Linear [from PyTorch ONNX]
+    aidge_core.fuse_to_metaops(gm, "MatMul-*>Add", "Linear")
+
+    # LayerNorm [from PyTorch ONNX]
+    aidge_core.fuse_to_metaops(gm, "ReduceMean-*>Sub#1~>(Pow#1->ReduceMean-*>Add#1->Sqrt)-*>Div#1-*>Mul#1-*>Add#2;"
+                                   "Sub#1~*>Div#1;"
+                                   "Pow#1<1~Producer;"
+                                   "Add#1<*~Producer;"
+                                   "Mul#1<*~Producer;"
+                                   "Add#2<*~Producer;"
+                                   "Sub#1~>$", "LayerNorm")
+
+    # ScaledDotProductAttention [from PyTorch ONNX]
+    aidge_core.fuse_to_metaops(gm, "MatMul->Div#1->Softmax-*>MatMul;"
+                                   "Div#1<1~Producer", "ScaledDotProductAttention")
+
+    # MultiHeadAttention [from PyTorch ONNX]
+    aidge_core.fuse_to_metaops(gm, "ScaledDotProductAttention#1->Transpose->Reshape#1->Linear;"
+                                   "Reshape#1<1~Producer;"
+                                   "ScaledDotProductAttention#1<0-(Transpose<-Reshape#2<-Add#1);"
+                                   "ScaledDotProductAttention#1<1-(Transpose<-Reshape#3<-Add#2);"
+                                   "ScaledDotProductAttention#1<2-(Transpose<-Reshape#4<-Add#3);"
+                                   "Reshape#2<1~Producer;"
+                                   "Add#1<*-0-Split#1;"
+                                   "Add#2<*-1-Split#1;"
+                                   "Add#3<*-2-Split#1;"
+                                   "Split#1<-MatMul;"
+                                   "Split#1<1~Producer", "MultiHeadAttention")
+
+    # GeLU [from PyTorch ONNX]
+    aidge_core.fuse_to_metaops(gm, "Div#1->Erf->Add#1-*>Mul->Mul#2;"
+                                   "Div#1<1~Producer[Constant_sqrt2];"
+                                   "Add#1<*~Producer[Constant_1];"
+                                   "Mul#2<*~Producer[Constant_0_5]", "GeLU")