Draft: Dev ptq

TODO

• Normalize layers
  • Get weight max value for each layer (Fc/Conv)-> alpha
  • Divide weights by alpha
  • Divide bias by alpha
  • Add PerOutputChannel normalization
    • TODO : add steps to do this ...
• Normalize activation
  • Normalize stimuli between [0;1]
    • or [-1,1]
  • Forward on validation dataset
  • Get max value of activation for each layer -> beta
    • Develop hook sytsem
    • Develop get max activation hook
  • Add scaling factor beta to the activation
  • Add scaling factor beta to the bias
  • Rescale activation by parent bias scaling
• Quantization
  • Input: multiply by (2^n-1)-1 (singed) or (2^n-1) (unsigned)
  • Weights: Multiply by (2^n-1)-1 + round + store as a signed integer of nbbits size
  • Biases: Multiply by (2^n-1)-1 + Multiply by (2^n-1)-1 (singed) or (2^n-1) (unsigned) + store as a signed integer of nbbits size
  • Activation scaling:
    • Input scaling: (2^n-1)-1 + Multiply by (2^n-1)-1 (singed) or (2^n-1) (unsigned)
    • Output scaling: (2^n-1)-1 (singed) or (2^n-1) (unsigned)
    • Activation sclaing : divide by (input scaling divided bu output_scaling)
• Activaiton clipping
  • generate histogram of outputs for each layer (using validation dataset)
  • MSE
    • TODO : add steps to do this
  • KL-Divergence
    • TODO : add steps to do this
• Weights clipping
  • TODO : add steps to do this
• Better scaling operation
  • Fixed-point scaling
  • Single shift scaling
  • Double shift scaling
• Bind method in Python
• LeNet integration test
• Refactor OutputRange hook to use Tensor Getter/Setter
• Add docstring to OutputRange hook
• Refactor hook system to not use registrar ?

setup.py

@@ -62,11 +62,11 @@ class CMakeBuild(build_ext):
 
         os.chdir(str(build_temp))
 
-        # Impose to use the executable of the python 
+        # Impose to use the executable of the python
         # used to launch setup.py to setup PythonInterp
         param_py = "-DPYTHON_EXECUTABLE=" + sys.executable
-        
-        install_path = f"{build_temp}/install" if "AIDGE_INSTALL" not in os.environ else os.environ["AIDGE_INSTALL"]
+
+        install_path = os.path.join(sys.prefix, "lib", "libAidge")  if "AIDGE_INSTALL" not in os.environ else os.environ["AIDGE_INSTALL"]
 
         self.spawn(['cmake', str(cwd), param_py, '-DTEST=OFF', f'-DCMAKE_INSTALL_PREFIX:PATH={install_path}'])
         if not self.dry_run:
@@ -83,11 +83,11 @@ class CMakeBuild(build_ext):
             for file in files:
                 if file.endswith('.so') and (root != str(aidge_package.absolute())):
                     currentFile=os.path.join(root, file)
-                    shutil.copy(currentFile, str(aidge_package.absolute())) 
+                    shutil.copy(currentFile, str(aidge_package.absolute()))
 
         # Copy version.txt in aidge_package
         os.chdir(os.path.dirname(__file__))
-        shutil.copy("version.txt", str(aidge_package.absolute()))    
+        shutil.copy("version.txt", str(aidge_package.absolute()))
 
 
 if __name__ == '__main__':