In the context of implementing three operators for the Quantization of Vision Transformers (ShiftMax, ShiftGELU and I-LayerNorm). Here is the first Thales contribution with the implementation of the forward of ShiftMax and ShiftGELU. In aidge_backend_cuda, we implement the kernel cuda for the forward of SHiftMax and ShiftGELU. We didn't make any modifications on the existing file (except cuda.hpp where we include our file).
Add files
src/operator/ShiftGELUImpl.cpp and include/aidge/backend/cuda/operator/ShiftGELUImpl.hpp : Implementation of the forward for ShiftGELU
src/operator/ShiftGELUImpl_CUDA_kernels.cu and include/aidge/backend/cuda/operator/ShiftGELUImpl_CUDA_kernels.hpp : Implementation of the kernel cuda for ShiftGELU
src/operator/ShiftMaxImpl.cpp and include/aidge/backend/cuda/operator/ShiftMaxImpl.hpp : Implementation of the forward for ShiftMax
src/operator/ShiftMaxImpl_CUDA_kernels.cu and include/aidge/backend/cuda/operator/ShiftMaxImpl_CUDA_kernels.hpp : Implementation of the kernel cuda for ShiftMax
unit_tests/Test_ShiftGELUImpl.cpp : Unitary test for showing how work the operator ShiftGELU
unit_tests/Test_ShiftMaxImpl.cpp : Unitary test for showing how work the operator ShiftMax
TODO
The functions that launch the Kernel (ShiftGELULaunchKernel and ShiftMaxLaunchKernel) are not yet templated. We didn't fix the error yet.
