[export_cpp] Identity node reserves a redundant space for the input into the memory block
Hi !
in short
So in the context of using Tiling on a model's encoder (cf aidge_onnx#70 (moved) ),
my graph's first node is an Identity node followed by multiple Slice nodes branches in parallel.
The issue happens on microcontroller:
- I allocate memory for my input outside the
forwardlibrary generated by AIDGE - then inside the
forwardlibrary, the local memory blockmemhas allocated space for the Identity node to copy my input.
I don't want my input vector be allocated twice in memory !
here is how it happens
- The local memory block
memhas allocated space for the Identity node to copy my input inforward.cpp
// Memory block
static unsigned char mem[18112];
- Then at the begining of
model_forwad, we haveIDENTITY_input_0==my_input_pointer_somewhere_else_in_ramandIDENTITY_output_0==memso I getIDENTITY_input_0 != IDENTITY_output_0:
void model_forward (const float* IDENTITY_input_0,float** node_softmax_tiled_output_0_ptr)
{
float* IDENTITY_output_0 = (float*) (mem + IDENTITY_OUTPUT_0_MEM_OFFSET); // ==mem
identity_forward<IDENTITY_NB_ELTS>
(IDENTITY_input_0, IDENTITY_output_0);
- Then in
identity.hpp, as(inputs == outputs)is false, it makes a copy of inputs in outputs :
// If inputs and outputs pointers are the same, the memory manager has already optimized this function so it is a no-op !
if (inputs == outputs)
return;
// A identity in c++ world should equal to a Noop
// We only need to copy the input buffer to the output
for (size_t m = 0; m < M; ++m) {
outputs[m] = inputs[m];
}
Bad fix by removing Identity node
So I can solve this by removing the Identity Node from the graph, but this is weird because now the graph has X inputs that for a unique input.
Question
How to cleanly handle a model that starts with branches from the input ?
- Should we modify Identity's implementation so it does not allocate memory in the
memblock when Identity is a first node ? How ?
Edited by Louis Lerbourg