Skip to content

BCE loss: fix numerical issues

Olivier Antoni requested to merge oantoni/aidge_learning:BCE_loss into dev

Context

With previous implementation, numerical instabilities were encountered for the following reason: (1 + Epsilon) - p = 0 for p = 1 and Epsilon = 1e-12f.

Detailed major modifications

Numerical instabilities are solved by computing: (1 - p) + Epsilon = Epsilon for p = 1.

This implementation is inspired from PyTorch (false gradient computation instead of exact derivative).

Merge request reports

Loading