Support fixing top-level extension modules

Potential duplication of GH-12, GH-15, GH-22, GH-45, GH-63 and possibly many others. I open this to discuss the strategy to fix this quite common issue.

Rationale

delocate-listdeps works perfectly for these wheels, while delocate-wheel fails silently. This is really confusing to the users. It’d be a lot better if any detected missing lib can be added to the wheel.

Proposal

auditwheel support top-level extension modules by copying the shared libraries to {package-name}.libs and I suggest delocate follow that convention. With package-name being canonicalized to snake_case, this should never cause name collision with the package ending with libs. For example

$ unzip palace-0.1.5-cp38-cp38-manylinux2014_x86_64.whl 
Archive:  palace-0.1.5-cp38-cp38-manylinux2014_x86_64.whl
  inflating: palace.cpython-38-x86_64-linux-gnu.so  
  inflating: palace-0.1.5.dist-info/top_level.txt  
  inflating: palace-0.1.5.dist-info/RECORD  
  inflating: palace-0.1.5.dist-info/METADATA  
  inflating: palace-0.1.5.dist-info/LICENSE  
  inflating: palace-0.1.5.dist-info/WHEEL  
  inflating: palace.libs/libopus-abf1ee53.so.0.3.0  
  inflating: palace.libs/libvorbisfile-c5d289a9.so.3.3.5  
  inflating: palace.libs/libalure2-9f93a6b1.so  
  inflating: palace.libs/libvorbis-02341991.so.0.4.6  
  inflating: palace.libs/libFLAC-b0904aee.so.8.3.0  
  inflating: palace.libs/libgsm-ed01cacd.so.1.0.12  
  inflating: palace.libs/libopusfile-a90fc0fd.so.0.3.0  
  inflating: palace.libs/libopenal-ce7fd2e6.so.1.20.1  
  inflating: palace.libs/libvorbisenc-e32e3ab5.so.2.0.9  
  inflating: palace.libs/libsndfile-7c9b3dac.so.1.0.25  
  inflating: palace.libs/libogg-24eaa32b.so.0.8.0  

I’m looking forward to see your response on this. Happy labor day BTW 🎆

[McSinyx]

In our project, we work around it by running the following script:

#!/bin/sh
set -ex
cd $(mktemp -d)
unzip $abspath_to_wheel
delocate-path -L $package_name.dylibs .
wheel=$(basename $abspath_to_wheel)
zip -r $wheel *
mkdir -p $output_dir
mv $wheel $output_dir
tempdir=$(pwd)
cd -
rm -rf $tempdir

The actual setup is available at McSinyx/palace@5406740. Thank you for providing the lower-level utility for hacks like this to be possible, though I still think it’d be nice if this is supported by delocate-wheel.

[sarnold]

I’m in the same boat as @bsolomon1124 and I ended up using this fork: https://github.com/Chia-Network/delocate/commits/master which merged #39 and adds a (force) file insertion option. Since this seems to work (and builds macos wheels with the right bundled libs directory) I’m wondering what’s still holding this up? If I actually had a Mac I would certainly poke at this more and make sure the tests work, etc, but I don’t have a Mac (or any “extra” time for that matter). Thanks! (my example is here: https://github.com/freepn/py-re2/actions/runs/487166636)