Hard-coded RPATH in libraries

When I run pip install cocotb==1.4.0rc1 in one conda environment, it builds and installs a wheel and saves the wheel in a cache. Now, when I run the same command in a new conda environment, it reuses the wheel. The problem is that the .so files have a hard-coded RPATH to the first environment, so when that env is deleted, cocotb breaks in the other env.

From what I can tell, conda solves this problem for its packages: https://docs.conda.io/projects/conda-build/en/latest/resources/make-relocatable.html

[mtikekar]

Yes, all extension modules built with setuptools/distutils will have this rpath. The difference is that they are being loaded by python which has already loaded libpython. So these extension modules don’t really care for having the right rpath.

You can check by building some simple extension modules like https://github.com/starnight/python-c-extension. This is the compile command it runs:

gcc -pthread -shared -B /home/mtikekar/.conda/envs/foo/compiler_compat -L/home/mtikekar/.conda/envs/foo/lib \
-Wl,-rpath=/home/mtikekar/.conda/envs/foo/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.7/bind.o \
build/temp.linux-x86_64-3.7/libmypy.o -o build/lib.linux-x86_64-3.7/helloworld.cpython-37m-x86_64-linux-gnu.so

You can see the -Wl,-rpath but no -lpython.

For cocotb, libcocotb is loaded by the simulator which then has to load libpython. That is source of the problem here.

[ktbarrett]

I opened a discussion on pypa’s discourse to maybe get some more experienced eyes on the problem.