Skip to content

Fix the PTQ NaN bug

Benjamin Halimi requested to merge fix_nan into dev

Description

After the PTQ, some architectures were producing NaN outputs when benched.

It was caused by a datatype issue during the CLE step : values were manipulated as floats instead of doubles.

Note that the upcoming improvements branch will completely rework the handling of the data types.

Modified files : CLE.cpp and CLE.hpp

Merge request reports

Loading