Idea: any operator that just reorganizes data only has to call copy() from input to output. It is therefore possible to record all the copy() calls and automatically deduce the backward() function of such operator!
In this MR, I take the example of the Transpose operator to illustrate the concept. It would work the same for any operator with only data copy (Reshape, Slice, Concat, Memorize, Pop, Cast, Move, Gather, Squeeze, Unsqueeze...). Change for these operators would be absolutely trivial. Transpose is probably the worst example btw because it only copies element by element which would generate a very large copy call list!
Backward coding would essentially be free for these operators, on any backend! It would work for user's defined operators as well, provided that the user uses the OperatorCopyTapeImpl::copy() function in its forward().
Another idea would be to automatically generate the compute kernels in export based on the copy list. Also, it might be possible to simplify and merge several operators of this type (data copy only) to generate a single optimized custom operator, usable in Aidge and also in exports.
Solve issue with chuncks clearing. Use hooks? => wait for a decision regarding hooks refactoring.
Proof of concept of generic simplification with the mergeCopyTape() recipe (remove 3 Transpose that together form the identity);
Proof of concept of compute kernel generation;
Recognize patterns to replace several operator by one known?
