Skip to content

[export_cpp] Identity node reserves a redundant space for the input into the memory block

Hi !

in short

So in the context of using Tiling on a model's encoder (cf aidge_onnx#70 (moved) ),

my graph's first node is an Identity node followed by multiple Slice nodes branches in parallel.

The issue happens on microcontroller:

  • I allocate memory for my input outside the forward library generated by AIDGE
  • then inside the forward library, the local memory block mem has allocated space for the Identity node to copy my input.

I don't want my input vector be allocated twice in memory !

here is how it happens

  • The local memory block mem has allocated space for the Identity node to copy my input in forward.cpp
// Memory block
static unsigned char mem[18112];
  • Then at the begining of model_forwad , we haveIDENTITY_input_0==my_input_pointer_somewhere_else_in_ram and IDENTITY_output_0==mem so I get IDENTITY_input_0 != IDENTITY_output_0:
void model_forward (const float* IDENTITY_input_0,float** node_softmax_tiled_output_0_ptr)
{
    float* IDENTITY_output_0 = (float*) (mem + IDENTITY_OUTPUT_0_MEM_OFFSET); // ==mem

    identity_forward<IDENTITY_NB_ELTS>
                     (IDENTITY_input_0, IDENTITY_output_0);
  • Then in identity.hpp , as (inputs == outputs)is false, it makes a copy of inputs in outputs :
// If inputs and outputs pointers are the same, the memory manager has already optimized this function so it is a no-op !
    if (inputs == outputs)
        return;

    // A identity in c++ world should equal to a Noop
    // We only need to copy the input buffer to the output
    for (size_t m = 0; m < M; ++m) {
        outputs[m] = inputs[m];
    }

Bad fix by removing Identity node

So I can solve this by removing the Identity Node from the graph, but this is weird because now the graph has X inputs that for a unique input.

Question

How to cleanly handle a model that starts with branches from the input ?

  • Should we modify Identity's implementation so it does not allocate memory in the mem block when Identity is a first node ? How ?
Edited by Louis Lerbourg