Fix the NaN bug (!6) · Merge requests · Eclipse Projects / aidge / aidge_quantization

Benjamin Halimi requested to merge bhalimi/aidge_quantization:DevPTQ into main May 13, 2024

Previous version of the PTQ implementation could result in a model producing NaN values. This was due to a poor memory initialization obtained by doing a multiplication by zero. Thus, rescaleTensor(tensor, 0) is now replaced by fillTensor(tensor, 0) which solves the problem.

Fix the NaN bug

Merge request reports