Policy: name native libs after SONAME

[This is an issue from multiple other places: https://github.com/conda-forge/libpng-feedstock/pull/4#issuecomment-222751729, https://github.com/conda-forge/libpng-feedstock/issues/3#issuecomment-222777245 and https://github.com/conda-forge/conda-forge.github.io/pull/155#issuecomment-222988756]

If the library is well maintained, the SONAME (which usually includes the major version: libc.so.6) changes if the lib introduces any incompatible changes (aka a app build against a lower version would break if dynamically linked against the newer one -> API/ABI changes as opposed to API additions).

On linux (at least debian), the package which contains the library has to change each time the SONAME changes:

https://www.debian.org/doc/debian-policy/ch-sharedlibs.html#s-sharedlibs-runtime

The run-time shared library must be placed in a package whose name changes whenever the SONAME of the shared library changes. This allows several versions of the shared library to be installed at the same time, allowing installation of the new version of the shared library without immediately breaking binaries that depend on the old version. […] Every time the shared library ABI changes in a way that may break binaries linked against older versions of the shared library, the SONAME of the library and the corresponding name for the binary package containing the runtime shared library should change. Normally, this means the SONAME should change any time an interface is removed from the shared library or the signature of an interface (the number of parameters or the types of parameters that it takes, for example) is changed. This practice is vital to allowing clean upgrades from older versions of the package and clean transitions between the old ABI and new ABI without having to upgrade every affected package simultaneously.

Currently the native packages all use generic names (aka libpng instead of libpng16 or zlib instead of zlib1). Following the same pattern as debian has two main advantages:

• you don’t need to use < <next major version> pins, >= <min version> is enough. If the library breaks, the SONAME changes which means a different package name is used. Currently, any packages which miss the < <new major version> pin can be installed with an newer version of the library, but would break.
• Two incompatible versions of the packages can be installed at the same time. Currently only one version can be installed which might end in a recompile nightmare when conda-forge would update to libpng 17 which would then contain libpng17.dll or libpng17.so.17 -> All packages which link against the old libpng would need to be recompiled to get a higher dependency

Library packages should also include a test for the filename so that the build breaks if the soname/filename changes.

[ChrisBarker-NOAA]

well, I’ve had a lot of issues trying to upgrade the GNOME project to use newer libs, and conda-forge wherever possible:

https://github.com/NOAA-ORR-ERD/PyGnome/blob/master/conda_requirements.txt

note that I think I need to pin all those version, rather than >=.

But here’s what I get with it as it is:

clean python2 conda environment:

$ conda config --get channels –add channels ‘defaults’ # lowest priority –add channels ‘NOAA-ORR-ERD’ –add channels ‘conda-forge’ # highest priority

$ conda install --file conda_requirements.txt

Hey! that just worked! Maybe there’s been a change since a couple days ago – yippie!

[mingwandroid]

On a case-by-case basis, for specific pain points with libraries that are small enough and that follow semantic versioning I don’t object, but …

What @janschulz is talking about here is referred to as ‘semantic versioning’. Only a small percentage of software projects follow semantic versioning (and attempting to apply this strategy to those that do not is pointless), many of those that use this scheme do it wrong and as soon as you depend on anything that does it wrong you can’t do it right yourself either (see Qt as an example of a project that tries hard to do it right; does anyone here believe that you can use Qt 5.6 on an executable that was built against Qt 5.5? IMHO there is not a chance that it will work. I’d be surprised if an application linked against Qt 5.6.0 will run against Qt 5.6.2 without a rebuild).

It requires patching a huge amount of software and that requires a good level of knowledge of the build system used in each case.

Conda packages contain more than shared libraries so attempting to allow more than one of each to be installed in an environment at the same time will lead to the last one installed writing all those other files. Headers (and on Windows, import libraries) will not match the binaries, files in /etc will not be of the right format wrt the shared library, man pages will not match up. Split packages might help here, but only to the extent that in that case, the files would not have been installed, unless conda learns about the concept of conflicts.

I would rather either keep things as they are (as a general principle) or see something less up to chance implemented, such are forming a checksum of the interface definition of each shared library and renaming that library according to the checksum.

IMHO it’s not worth the effort, That Debian attempt it doesn’t convince me that it works well in general.

The compatibility problem here is that conda defaults is Fedora and conda-forge is Ubuntu (feel free to pick two different distros), and we’re trying to mix software from two different distributions and not really anything to do with SONAMEs.