Draft: Copytape operator implementation
copy()
from input to output. It is therefore possible to record all the copy()
calls and automatically deduce the backward()
function of such operator!
In this MR, I take the example of the Transpose
operator to illustrate the concept. It would work the same for any operator with only data copy (Reshape
, Slice
, Concat
, Memorize
, Pop
, Cast
, Move
, Gather
, Squeeze
, Unsqueeze
...). Change for these operators would be absolutely trivial. Transpose
is probably the worst example btw because it only copies element by element which would generate a very large copy call list!
Backward coding would essentially be free for these operators, on any backend! It would work for user's defined operators as well, provided that the user uses the OperatorCopyTapeImpl::copy()
function in its forward()
.