Support for 8-bit quantization with TensorRT
Overview
This merge request implements support for 8-bit quantization with TensorRT.
Changes Made
Added BatchStream.hpp and IInt8EntropyCalibrator.hpp. These additions enable the processing and fetching of calibration data necessary for TensorRT's 8-bit quantization.
Implemented calibration methods to:
- Select a pre-existing calibration table.
- Specify the folder containing calibration data.
TODO
- Add other calibrators and allow users to implement/use their own.
- Assist users in transforming their data into usable calibration data.
Edited by Nathan Thoumine
Merge request reports
Activity
Filter activity
added 1 commit
- d13f356f - Add BatchStream.hpp and IInt8EntrpyCalibrator.hpp, and modify graph.cpp for...
added 1 commit
- c06f34c4 - Refactor Int8EntropyCalibrator class to match noexcept specifications in...
added 1 commit
- 2ea9df3e - Refactor Int8EntropyCalibrator and BatchStream class to use member variable...
added 1 commit
- e0af522d - Fix includes in Graph.cpp and BatchSteam.hpp
added 1 commit
- 7273399a - Add option to choose calibration folder and cache file
added 1 commit
- ce0618ab - Set the number of calibration files based on the number of files in the calibration folder
added 1 commit
- cca44cf0 - Add data.info file in calibration folder to provide input dimensions for BatchStream
added 1 commit
- 3f1301a1 - Fix file name to exclude from nbCalibFiles count
added 1 commit
- 7bb96060 - Correct data loading in _fileBatch array within BatchStream.hpp
added 1 commit
- 56b98f59 - Raise an error if the calibration file is smaller than the expected input size
requested review from @cmoineau and @vtemplier
added AddFeature StatusReview Ready labels
changed milestone to %v0.2.0
Please register or sign in to reply