Added Tanh and Sigmoid operators necessary for vanilla LSTM;
Added a new Pop operator that generate a sequence from a Producer;
Added getConnectedGraphView() to build a GraphView from connected nodes;
ProducerOp and GenericOp have a default implementation now, allowing them to be used for scheduling;
Improved various error messages useful for debugging (there is still work to do!);
Much improved display in GraphView and Scheduler:
Consistent nodes naming everywhere;
New GraphView::getRankedNodes() and GraphView::getRankedNodesName() methods;
Use the light-weight C++ {fmt} formatting library, from which the C++20 std::format is based on.
Invalid registrar now displays the value of the missing key!
Handle multi-output ONNX nodes.
Bonus
Ported MemoryManager from N2D2 to Aidge;
Added SequentialScheduler::generateMemory() that does the same as old N2D2::CPP_DeepNetExport::generateMemory(), but in a much more general way (only the specific handling of Concat operator, that added a lot of complexity in N2D2, is missing).
TODO
Improve forwardDims();
Provide a more comprehensive recurrent example (RNN or LSTM);
Test with actual values.
Import LSTM model with ONNX.
TODO later
I feel this MR may become too big and take too much time to integrate the following things:
Test and run LSTM ONNX import with actual values.
We don't have a simple model to test. Model from PulseAudition has still some operators missing...
...but it is correctly imported for now, give it a try:
Implement an ONNX import able to translate Loop using the proposed Memorize mechanism;
Normally, LSTM and GRU should be exported with the corresponding ONNX operators. That was not always the case in the past, but a LSTM or GRU exported as a Loop in ONNX is considered as an error in PyTorch and Tensorflow. We therefore have a lack of actual use case for now.
The Memorize Operator does not store intermediate outputs right now (this will be needed for learning).
To be seeing when basic learning mechanisms will be in place.
