CUDA libraries are too big for pypi

context

currently we have issues bc cuda package ships with the whole cuda library that weights approx 400Mo and this exceeds the pip package size limit of 100 Mo.

Hence deploy jobs of aidge_backend_cuda fails: 6971fe6c

First mentionned here : aidge#79 (comment 2831786)

Solutions

The official solution is to dynamically load cuda library.

• Here is an extensive tutorial on how to do that: https://www.devxperiences.com/pzwp1/2023/05/17/c-dynamic-loading-of-a-shared-library-also-create-and-load-one-on-your-own/
• stack overflow example : https://stackoverflow.com/questions/68977087/c-how-to-use-dlopen-in-c

another less costful option

delete repairwheel command from cibuildwheel to avoid having auditwheel pickupd cuda & cublad libraries and add them to the .whl generated package.

added StatusWork in Progress label

assigned to @gregkub

marked this issue as blocking aidge#67

changed the description

changed the description

mentioned in commit ad5e8e04

mentioned in merge request !43 (closed)

changed title from Dynamically load cuda / cudnn libraries to CUDA libraries are too big for pypi

changed the description

The less costful option (delete repairwheel command from cibuildwheel ) doesn't work : librarie are not found by the package when importing the pckg built inncibuildwheel.

λ ldd aidge_backend_cuda.cpython-38-x86_64-linux-gnu.so
        linux-vdso.so.1 (0x00007ffca939f000)
        libcublas.so.11 => not found
        libcublasLt.so.11 => not found
        libcudnn.so.9 => /lib/x86_64-linux-gnu/libcudnn.so.9 (0x00007b88ebe00000)
        libcudart.so.11.0 => not found
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007b88ec97a000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007b88ec975000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007b88ec970000)
        libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007b88eba00000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007b88ec319000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007b88ec950000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007b88eb600000)
        /lib64/ld-linux-x86-64.so.2 (0x00007b88ec9a3000)

We can see that the linker is looking for libraries with specific versions that do not exist in my computer :

cublas :

λ plocate cublas.so
/home/greg/.local/lib/python3.8/site-packages/nvidia/cublas/lib/libcublas.so.12
/usr/local/cuda-12.5/targets/x86_64-linux/lib/libcublas.so
/usr/local/cuda-12.5/targets/x86_64-linux/lib/libcublas.so.12
/usr/local/cuda-12.5/targets/x86_64-linux/lib/libcublas.so.12.5.2.13
/usr/local/cuda-12.5/targets/x86_64-linux/lib/stubs/libcublas.so

cublasLT :

λ plocate libcublasLt.so
/home/greg/.local/lib/python3.8/site-packages/nvidia/cublas/lib/libcublasLt.so.12
/usr/local/cuda-12.5/targets/x86_64-linux/lib/libcublasLt.so
/usr/local/cuda-12.5/targets/x86_64-linux/lib/libcublasLt.so.12
/usr/local/cuda-12.5/targets/x86_64-linux/lib/libcublasLt.so.12.5.2.13
/usr/local/cuda-12.5/targets/x86_64-linux/lib/stubs/libcublasLt.so

cudart :

λ plocate libcudart.so
/home/greg/.local/lib/python3.8/site-packages/nvidia/cuda_runtime/lib/libcudart.so.12
/usr/local/cuda-12.5/targets/x86_64-linux/lib/libcudart.so
/usr/local/cuda-12.5/targets/x86_64-linux/lib/libcudart.so.12
/usr/local/cuda-12.5/targets/x86_64-linux/lib/libcudart.so.12.5.39

I will try to run cibuildwheel with cudart versions that stick to my computer and see if it works. If so, specific vesions of cuda, cudart & cublas will be required to install packages.

mentioned in commit d5650609

mentioned in merge request !46 (merged)

mentioned in commit 3a007361

mentioned in commit aabf23df

closed with merge request !46 (merged)

closed with commit d5650609

reopened

This issue sadly isn't done.

Since d5650609 solution was to delete the auditwheel repaircall. However this did not solve the issue at all the repair stage does not only bundles dependencies but also changes a <pkg>_..._linux....whl pkg into a <pkg>_..._manylinux....whl. And This pipeline thaught me that simple linux packages are not authorized on pypi.

Currently I'm trying to NOT bundle nvidia in the library. Simply unzipping the wheel & deleting the library wouldn't do the trick because library names are MODIFIED to avoid shadowing. I would need to change the path of the linked libraries.

cibuildwheel documentation

If your build is linking into a native library dependency, you can build/install that in CIBW_BEFORE_ALL. However, on Linux, Mac (and Windows if you're using delvewheel), the library that you install will be bundled into the wheel in the repair step. So take care to ensure that

added to epic &13

mentioned in commit f0f50823

mentioned in merge request !56 (merged)

mentioned in commit 4ae09e2e

mentioned in commit b35a2076

mentioned in commit aa334ba7

closed with commit b35a2076

closed with merge request !56 (merged)

mentioned in commit f6c974bd

mentioned in commit 7ff699d0

mentioned in commit 1908686b

added Compilation ⚙ label