Add backward functions for ReLU, Sigmoid and Tanh

Context

Add some backward functions that were missing.

Detailed modifications

• Use a protype for backward function that may fit all the needs : input, output and grad_output are needed to compute grad_input in the general case (input was missing, output is kept to save computation). This may lead to unused parameters, but it is probably not an issue ?

• Add missing backward functions for Sigmoid and Tanh.

• Use input instead of output for testing the sign in ReLU (both codes should give same results, but the use of input is preferred).

TODO

For this merge request :

• Fell free to adapt the proposed modifications to match code requirements.
• Check if my understanding of the way the forward methods should work is correct.

For future :

• Check the forward function for Sqrt operator that seems not to be correct.
• Homogenize the way forward and backward are coded for all available operators.

mentioned in issue aidge#111 (closed)

changed target branch from main to dev

added Feature LanguageC++ labels

Apparently, unused variables generate errors. How can I help to fix errors ?

added 1 commit

• c022fba3 - Add backward functions for ReLU, Sigmoid and Tanh

Compare with previous version

added 1 commit

• 33d46ff3 - Add backward functions for ReLU, Sigmoid and Tanh

Compare with previous version

added 1 commit

• 1ab9eb04 - Add backward functions for ReLU, Sigmoid and Tanh

Compare with previous version

requested review from @olivierbichler

approved this merge request

@oantoni Your MR is approved and ready to be merged. In order to be able to merge, you first need to sign the Eclipse ECA (see https://api.eclipse.org/git/eca/status/b3af0b9a6c4191e37e3060ea799fc082/ui).

Thanks for MR approval. Eclipse ECA has been signed.

mentioned in commit 3988b2f4

merged

assigned to @oantoni

changed milestone to %aidge - May 2024

mentioned in merge request !93 (merged)