AdaptToBackend generate to much transpose

• aidge_core: a3d2762c

Problem description

In earlier versions of aidge_core, with the nhwc format supported for convolution, the “[cpu/recipes] AdaptToBackend” test worked, even forwardDims and generate_scheduling. Now, since fwdDims and generateSchduling are commented. The “MR” I did on Transpose (or perhaps other MRs on core) broke this test.

As you see, the graph generated after AdaptToBackend contains unnecessary Transpose like the one between Conv and ReLU.

And the dims (or dataFormat) used are not the right ones during the forward_Dims .

[DEBUG] - Trying to forward dims of node  (of type Conv2D)
[ERROR] - Wrong input size ([1, 18, 18, 8]) for Conv operator. Expected dims are [x, 8, x, x].
[DEBUG] - Unable to forward dimensions for node  (of type Conv2D)

My approach to understanding where this problem comes from will be to :

• Check if without my last MR this test passes
• Check with DataFormat NCHW and not Default
• Add more test bc Adapt_To_Backend's have only three big tests (with mr !336 )
• If not, try modifying getAdaptation in OperatorImpl.cpp

Warning: Displaying this diagram might cause performance issues on this page.

Display

%%{init: {'flowchart': { 'curve': 'monotoneY'}, 'fontFamily': 'Verdana' } }%%
flowchart TB

Producer_0("dataProvider<br/><sub><em>(Producer#0)</em></sub>"):::producerCls_rootCls
Producer_2("conv1_w<br/><sub><em>(Producer#2)</em></sub>"):::producerCls
Producer_1("conv1_b<br/><sub><em>(Producer#1)</em></sub>"):::producerCls
ReLU_0("relu1<br/><sub><em>(ReLU#0)</em></sub>")
Producer_4("conv2_w<br/><sub><em>(Producer#4)</em></sub>"):::producerCls
Producer_3("conv2_b<br/><sub><em>(Producer#3)</em></sub>"):::producerCls
ReLU_1("relu2<br/><sub><em>(ReLU#1)</em></sub>")
Producer_6("conv3_w<br/><sub><em>(Producer#6)</em></sub>"):::producerCls
Producer_5("conv3_b<br/><sub><em>(Producer#5)</em></sub>"):::producerCls
Conv2D_0("<em>Conv2D#0</em>")
Transpose_0("<em>Transpose#0</em>")
Transpose_2("<em>Transpose#2</em>")
Transpose_1("<em>Transpose#1</em>")
Conv2D_1("<em>Conv2D#1</em>")
Transpose_3("<em>Transpose#3</em>")
Transpose_5("<em>Transpose#5</em>")
Transpose_4("<em>Transpose#4</em>")
Conv2D_2("<em>Conv2D#2</em>")
Transpose_6("<em>Transpose#6</em>")
Transpose_8("<em>Transpose#8</em>")
Transpose_7("<em>Transpose#7</em>")
Producer_0-->|"0 [1, 3, 22, 22] Float32<br/>&darr;<br/>0"|Transpose_0
Producer_2-->|"0 [4, 3, 3, 3] Float32<br/>&darr;<br/>0"|Transpose_2
Producer_1-->|"0 [4] Float32<br/>&darr;<br/>2"|Conv2D_0
ReLU_0-->|"0 [1, 4, 20, 20] Float32<br/>&darr;<br/>0"|Transpose_3
Producer_4-->|"0 [8, 4, 3, 3] Float32<br/>&darr;<br/>0"|Transpose_5
Producer_3-->|"0 [8] Float32<br/>&darr;<br/>2"|Conv2D_1
ReLU_1-->|"0 [1, 8, 18, 18] Float32<br/>&darr;<br/>0"|Transpose_6
Producer_6-->|"0 [10, 8, 1, 1] Float32<br/>&darr;<br/>0"|Transpose_8
Producer_5-->|"0 [10] Float32<br/>&darr;<br/>2"|Conv2D_2
Conv2D_0-->|"0 [] Float32<br/>&darr;<br/>0"|Transpose_1
Transpose_0-->|"0<br/>&darr;<br/>0"|Conv2D_0
Transpose_2-->|"0<br/>&darr;<br/>1"|Conv2D_0
Transpose_1-->|"0<br/>&darr;<br/>0"|ReLU_0
Conv2D_1-->|"0<br/>&darr;<br/>0"|Transpose_4
Transpose_3-->|"0<br/>&darr;<br/>0"|Conv2D_1
Transpose_5-->|"0<br/>&darr;<br/>1"|Conv2D_1
Transpose_4-->|"0<br/>&darr;<br/>0"|ReLU_1
Conv2D_2-->|"0<br/>&darr;<br/>0"|Transpose_7
Transpose_6-->|"0<br/>&darr;<br/>0"|Conv2D_2
Transpose_8-->|"0<br/>&darr;<br/>1"|Conv2D_2
Transpose_7--->|"0<br/>&darr;"|output0((out#0)):::outputCls
classDef inputCls fill:#afa
classDef outputCls fill:#ffa
classDef externalCls fill:#ccc
classDef producerCls fill:#ccf
classDef genericCls fill:#f9f9ff,stroke-width:1px,stroke-dasharray: 5 5
classDef metaCls stroke-width:5px
classDef rootCls stroke:#f00
classDef producerCls_rootCls stroke:#f00,fill:#ccf
classDef genericCls_rootCls stroke:#f00,fill:#f9f9ff,stroke-width:1px,stroke-dasharray: 5 5
classDef metaCls_rootCls stroke:#f00,stroke-width:5px

Reproducible example code

You can use the test ("[cpu/recipes] AdaptToBackend", "[AdaptToBackend][recipes]") and uncomment theses lines :

    REQUIRE(g1->forwardDims());
    g1->save("adapttobackend_after_forwarddims");

    SequentialScheduler sched(g1);
    sched.forward();

And comment these line in Transpose.cpp :

for (; i < outputDimsOrder().size(); ++i) {
            AIDGE_ASSERT(i == outputDimsOrder()[i],
                "Permutation vector ({}) must be the identity above the input tensor rank ({}).", outputDimsOrder(), getInput(0)->dims());
        }

added Bug 🐛 Help Wanted LanguageC++ PriorityMedium Regression 📉 labels

assigned to @wboussella

@olivierbichler @cmoineau If you have any recommendations, i'm interested: )

marked the checklist item Check if without my last MR this test passes as completed

marked the checklist item Check with DataFormat NCHW and not Default as completed

I am unsure what the issue is and the "reproductible code" doesn't seems that reproductible. Could you just create a script with a Conv -> ReLU and adaptToBackend that generate this behavior?

If I understand correctly, your issue is that adaptToBackend generate a Transpose when the input of the conv was already in the right data format. What is weird is that previous conv seems to be fine so why does it happens to a random conv in the middle of the graph?

Are you sure the right data format are sets?

Any news?

The error occurs in the unit test already available in the core. I'm doing all the unit tests asked, but I'm also trying to fix a SIGSEGV at the same time. I'll update you soon.

Update: I think I've found the place where my bug was generated, on @pineapple's commit 8947fddd. Or even my unit tests have been changed on Test_Conv_Op.cpp two commits down: d83ddf49 I'm trying to see if I can adapt the code even with @pineapple 's commits

marked the checklist item Add more test bc Adapt_To_Backend's have only three big tests (with mr !336 ) as completed