Skip to content

Draft: Copytape operator implementation

Olivier BICHLER requested to merge copytape into dev

💡 Idea: any operator that just reorganizes data only has to call copy() from input to output. It is therefore possible to record all the copy() calls and automatically deduce the backward() function of such operator!

In this MR, I take the example of the Transpose operator to illustrate the concept. It would work the same for any operator with only data copy (Reshape, Slice, Concat, Memorize, Pop, Cast, Move, Gather, Squeeze, Unsqueeze...). Change for these operators would be absolutely trivial. Transpose is probably the worst example btw because it only copies element by element which would generate a very large copy call list!

Backward coding would essentially be free for these operators, on any backend! It would work for user's defined operators as well, provided that the user uses the OperatorCopyTapeImpl::copy() function in its forward().

Edited by Olivier BICHLER

Merge request reports