Handling of Int64 variables when exporting model to Arm Cortex-M

Problem description

Some operators have input/output data types not supported by the platform. This is the case for the Producer, Gather, Slice and Reshape operators, whose inputs/outputs are/can be of type Int64.

For the Gather operator, the second input is called "indices" and is of type Int64. In what follows, we assume that it is constant in the model and produced by a producer. When exporting the model, we encounter the following situation:

• The value of "indices" will be exported as a parameter of type "const int64_t" in the "dnn/include/parameters" folder (according to the default conversion table defined in "aidge_core/export_utils/data_conversion.py"), but will not be used later in "dnn/src/forward.cpp".
• The value of "indices" is also converted and exported as a parameter of type "const int" according to the instructions defined in "/templates/configuration/gather.jinja". This parameter will ultimately be provided as an argument to the kernel method "aidge_gather_chw_float32" when it is called in "dnn/src/forward.cpp".

This way of handling Int64 values ​​will not work if the value changes during inference on the platform. Furthermore, the conversion from Int64 to Int32 is done behind the scene, which is not good.

changed the description

@cmoineau, @axelfarr @olivierbichler any thought ?

The issue seems to point at different problems:

1. Gather indice input is saved in the config and not as a producer which may be a problem if it is not a constant
2. The indice is cast in the jinja template from 64 -> 32 bits

Regarding the first issue. I think @axelfarr made a function to ignore certain producers which could remove the unused input.

For the second one, I don't really see it as an issue due to the nature of indice (if we encounter an indice > 2^32 I am worried ^^) IMO we could even store this as a INT8 by default or a INT32 if the value is above 256 ...

Maybe we could make two kernels of reshape one if the "indice" input is linked to a producer (constant case). And one if it is another node. This would allow us to have a more performant kernel and a generic one in case the indice cannot be computed statically...

For the rest of the issue I am not sure to understand if you can explain @oantoni?

For the second issue, we should simply use the right type, int64_t.

For the first one, it is a bit complicated, two scenario are actually possible:

• All dimensions are known at export generation, meaning a forward_dims() can be run successfully on the model before export. In this case, dimension parameters are exported as constant values, and the corresponding Producer pointer are not even used, but exist nevertheless...
• Dimensions are not known at export time, and... the export won't work! Only static graphs are currently exportable, because the memory is statically allocated in an export, thanks to the memory manager. Unknown dimensions would mean that the memory manager would not be able to allocate a fixed amount of memory at compile time, and would require the export to handle dynamic memory allocation.

There is currently no plan to support dynamic memory allocation in export.