Skip to content
Snippets Groups Projects

Enhancement : Quantizer only PTQ

Merged Benjamin Halimi requested to merge quantizer_only_ptq into dev

Description

Current PTQ relies on a system of tags, and the combination of multiple Meta-Operators to achieve the quantization of a graphView.

While it is functionally working, we argue that it lacks of a general organization, and that the current approach eventually leads to over-complicated code.

In this MR, we propose to only rely on a single meta-operator, the Quantizer, to handle all the graph modifications of the PTQ.

The idea is to initialize the Quantizer to only contain a Mul operator, that performs the scaling operation, and to progressively enrich the meta-operator with the Rounding, Cliping, Cast and BitShift operations.

TODO

  • create the new quantizer MetaOperator
  • modify the quantizer insertions functions :
  • insertProducerScalingNodes()
  • insertResidualScalingNodes()
  • insertScalingNodes()
  • modify the normalization functions :
  • normalizeParameters()
  • normalizeActivations()
  • modify the quantization function
  • modify the single-shift related functions
  • modify the Cross-Layer Equalization
  • modify the Constant-Folding
  • modify the "real integer cast" function
  • update the unit test
Edited by Benjamin Halimi

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
Please register or sign in to reply
Loading