Skip to content
Snippets Groups Projects

Fix the NaN bug

Merged Benjamin Halimi requested to merge bhalimi/aidge_quantization:DevPTQ into main

Previous version of the PTQ implementation could result in a model producing NaN values. This was due to a poor memory initialization obtained by doing a multiplication by zero. Thus, rescaleTensor(tensor, 0) is now replaced by fillTensor(tensor, 0) which solves the problem.

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
Please register or sign in to reply
Loading