4.0.34 manylinux wheels include old libodbc.so and libltdl.so libraries

Environment

• Python: 3.8
• pyodbc: 4.0.34
• OS: Xubuntu 20.04
• DB: n/a
• driver: n/a

Issue

With 4.0.34 we started building manylinux wheel files. Unfortunately, they include rather old versions of libodbc.so and libltdl.so that override the system versions, leading to errors as described in #1079 and #1081.

pyodbc 4.0.34

(pyodbc_test) gord@vbox-Xubu-20-04-a:~$ pip install pyodbc
Collecting pyodbc
  Downloading pyodbc-4.0.34-cp38-cp38-manylinux_2_24_x86_64.whl (475 kB)
  …
Successfully installed pyodbc-4.0.34
(pyodbc_test) gord@vbox-Xubu-20-04-a:~$ python
Python 3.8.10 (default, Jun 22 2022, 20:18:18) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyodbc
>>> pyodbc.__file__
'/home/gord/.virtualenvs/pyodbc_test/lib/python3.8/site-packages/pyodbc.cpython-38-x86_64-linux-gnu.so'
>>> quit()
(pyodbc_test) gord@vbox-Xubu-20-04-a:~$ ldd /home/gord/.virtualenvs/pyodbc_test/lib/python3.8/site-packages/pyodbc.cpython-38-x86_64-linux-gnu.so
	linux-vdso.so.1 (0x00007ffda9f7a000)
	libodbc-2003e41d.so.2.0.0 => /home/gord/.virtualenvs/pyodbc_test/lib/python3.8/site-packages/pyodbc.libs/libodbc-2003e41d.so.2.0.0 (0x00007fc4d0931000)
	libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fc4d073d000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fc4d05ee000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fc4d05d3000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fc4d05b0000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fc4d03be000)
	libltdl-738907ff.so.7 => /home/gord/.virtualenvs/pyodbc_test/lib/python3.8/site-packages/pyodbc.libs/libltdl-738907ff.so.7 (0x00007fc4d01b1000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fc4d0dd1000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fc4d01ab000)

Note:

libodbc-2003e41d.so.2.0.0 => /home/gord/.virtualenvs/pyodbc_test/lib/python3.8/site-packages/pyodbc.libs/libodbc-2003e41d.so.2.0.0 (0x00007fc4d0931000)

libltdl-738907ff.so.7 => /home/gord/.virtualenvs/pyodbc_test/lib/python3.8/site-packages/pyodbc.libs/libltdl-738907ff.so.7 (0x00007fc4d01b1000)

pyodbc 4.0.32

(pyodbc_test) gord@vbox-Xubu-20-04-a:~$ pip install pyodbc==4.0.32
Collecting pyodbc==4.0.32
  Downloading pyodbc-4.0.32.tar.gz (280 kB)
  …
Successfully installed pyodbc-4.0.32
(pyodbc_test) gord@vbox-Xubu-20-04-a:~$ python
Python 3.8.10 (default, Jun 22 2022, 20:18:18) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyodbc
>>> pyodbc.__file__
'/home/gord/.virtualenvs/pyodbc_test/lib/python3.8/site-packages/pyodbc.cpython-38-x86_64-linux-gnu.so'
>>> quit()
(pyodbc_test) gord@vbox-Xubu-20-04-a:~$ ldd /home/gord/.virtualenvs/pyodbc_test/lib/python3.8/site-packages/pyodbc.cpython-38-x86_64-linux-gnu.so
	linux-vdso.so.1 (0x00007ffe10340000)
	libodbc.so.2 => /lib/x86_64-linux-gnu/libodbc.so.2 (0x00007f15f08b5000)
	libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f15f06d3000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f15f0584000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f15f0569000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f15f0377000)
	libltdl.so.7 => /lib/x86_64-linux-gnu/libltdl.so.7 (0x00007f15f036c000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f15f0347000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f15f0966000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f15f0341000)

Note:

libodbc.so.2 => /lib/x86_64-linux-gnu/libodbc.so.2 (0x00007f15f08b5000)

libltdl.so.7 => /lib/x86_64-linux-gnu/libltdl.so.7 (0x00007f15f036c000)

Action Needed

Build wheels without including those libraries.

Workarounds for the time being

Pin pyodbc at version 4.0.32, or

pip install pyodbc --no-binary pyodbc

[jfmontanaro]

Just chipping in my $0.02 here: As a consumer of pyodbc, it would be slightly more convenient for me if it included all the binaries it needs rather than requiring me to compile them. Mostly this is just because it would simplify Docker builds, which is primarily where I use it.

Compiling packages from source in a Docker build is annoying, because if you don’t want your final image to be bloated with a bunch of build dependencies that are completely unnecessary for the running application, you have to:

1. Install build tools
2. Build from source
3. Uninstall build tools
4. Remove a bunch of package-manager caches, etc.

So in this case, the “install pyodbc” portion of my (Debian-based) Dockerfile looks like this:

RUN apt-get install -y build-essential unixodbc unixodbc-dev && \
    pip install pyodbc && \
    apt-get purge -y build-essential unixodbc-dev && \
    apt-get autoremove -y

Which is somewhat complex and time-consuming.

However Docker does cache build layers, meaning this cost doesn’t have to be paid on every build, so it’s not completely the end of the world. I would rate this as “mildly inconvenient” at worst. Just wanted to chime in since it’s under discussion.

[v-chojas]

Linux wheels - Debian 9 or CentOS 7 (or both)?

CentOS 7 is the older one, so you will get wider compatibility if you build on it. In general, backwards-compatibility means that binaries built on an older system will work on a newer one, but not vice-versa; on the other hand, in contrast to Windows, it is much more difficult to do the opposite.