Quantization Aware Training using the SAT method

Context

A more sophisticated method for performing the QAT is the SAT framework. In this method, the weights are quantized using the DoReFa nodes, and the activations are quantized using the CG-PACT node. DoReFa requires a TanhClamp node to work. Also a ScaleAdjust node ensure that the flowing gradient are properly scaled.

Key steps

• test the SAT operators (cpu)
• test the SAT operators (cuda)
• implement the quantizers insertion and init routines
• benchmark the SAT QAT on the MNIST (cpu)
• benchmark the SAT QAT on the CIFAR-10 (cuda)
• benchmark the SAT QAT on the ImageNet (cuda)

RESULTS

TODO