Using `-Xfatbin=-compress-all` for kernel compilation

Not sure how relevant this is for how CuPy compiles things, but something we have been exploring in RAPIDS that may be of interest is building with -Xfatbin=-compress-all. In some cases this can make for quite dramatic reductions in compiled kernel sizes. For an example please see PR ( https://github.com/rapidsai/cudf/pull/7583 ). Again not sure if this is relevant for how CuPy is building things, but just wanted to mention it in case it is.

[kmaehashi]

Thanks for sharing this! I tried this locally (my environment uses export CUPY_NVCC_GENERATE_CODE=arch=compute_60,code=sm_60;arch=compute_70,code=sm_70;arch=compute_80,code=sm_80) and measured du -csh cupy cupyx cupy_backends:

• with compression: 95MB
• without compression: 102MB

In CuPy it seems the effect is limited because nvcc is only used for a few modules (cub/thrust/random).

Moreover, after compressing files into a wheel (ie ZIP), the compressed one was 262 KB larger.

[leofang]

It’s another 15-25M for the cub module and I didn’t check other modules. I don’t see the build time increased, and the runtime overhead for decompression is negligible based on an internal discussion (though likely outdated). For reference, all RAPIDS binaries are built with this option.

Though, to be fair, it does not always help. For example, with this feature I see cub inflates to 190M+, and the compression only brings it down to 170M+, not a significant change.