Skip to content

[Fix] Rework the Gradient Clearing routine

Benjamin Halimi requested to merge grad_rework into dev

Description

Now that all the gradient computations are set to accumulative mode (as explained in aidge_backend_cuda!58 (merged)) we need to adapt the way the gradients are cleared. Indeed, after each pass, the gradient values would accumulate, and the model would diverge, if we dont change the way we reset the gradients.

By rewriting resetGrad(), this MR ensure that all the output gradients of all nodes are now cleared.

When it comes to MetaOperators, we need to dive into their MicroGraphs to ensure that all the tensors are taken into consideration. A simple answer to this concern is to make resetGrad() recursive.

Modified files : Optimizer.hpp, Optimizer.cpp, pybind_optimizer.cpp

Merge request reports

Loading