Add Convert operator (a.k.a. Transmitter)
This MR is currently built on top of !53 (merged), which should be merged soon.
This MR is now ready to be merged (once !53 (merged) is merged) and with the corresponding MR in backend_cpu and backend_cuda.
What is in this MR?
- Added many conversion functions to Tensor, which is now on par with what was implemented in N2D2 (but much better and much simpler!);
- Updated TensorImpl interface in order to provide everything needed;
- Added documentation for TensorImpl interface;
- Generic implementation of a Cast and Move operators (does not require specific backend implementation);
- Cast is intended to be compatible with ONNX Cast operator;
- Recipe for automatic Cast/Move nodes insertion;
- Adapted FuseBatchNorm recipe to work for any type;
- Introduce the notion of "device", in order to handle for example multi-GPUs;
- Added half float (float16) support.
Principle
The implementation strategy is essentially summarized here.
To do
-
Add unit tests => tested in CPU and CUDA backends; -
Clarify what to do when changing backend (copy-convert data or not?) => added an extra argument that defaults to true.
To do later
- Adapt more operator implementations to handle implicit type casting and device change (without requiring explicit
Cast
/Move
operator in the graph).
Edited by Olivier BICHLER