Support for 8-bit quantization with TensorRT
Overview
This merge request implements support for 8-bit quantization with TensorRT.
Changes Made
Added BatchStream.hpp
and IInt8EntropyCalibrator.hpp
. These additions enable the processing and fetching of calibration data necessary for TensorRT's 8-bit quantization.
Implemented calibration methods to:
- Select a pre-existing calibration table.
- Specify the folder containing calibration data.
TODO
- Add other calibrators and allow users to implement/use their own.
- Assist users in transforming their data into usable calibration data.
Edited by Nathan Thoumine