Enhancement : Quantizer only PTQ
Description
Current PTQ relies on a system of tags, and the combination of multiple Meta-Operators to achieve the quantization of a graphView.
While it is functionally working, we argue that it lacks of a general organization, and that the current approach eventually leads to over-complicated code.
In this MR, we propose to only rely on a single meta-operator, the Quantizer
, to handle all the graph modifications of the PTQ.
The idea is to initialize the Quantizer
to only contain a Mul
operator, that performs the scaling operation, and to progressively enrich the meta-operator with the Rounding
, Cliping
, Cast
and BitShift
operations.
TODO
-
create the new quantizer MetaOperator -
modify the quantizer insertions functions : - insertProducerScalingNodes()
- insertResidualScalingNodes()
- insertScalingNodes()
-
modify the normalization functions : - normalizeParameters()
- normalizeActivations()
-
modify the quantization function -
modify the single-shift related functions -
modify the Cross-Layer Equalization -
modify the Constant-Folding -
modify the "real integer cast" function -
update the unit test
Edited by Benjamin Halimi
Merge request reports
Activity
Filter activity
Please register or sign in to reply