Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
M1 Macbook: unixODBC + FreeTDS working, but pyodbc hangs
See original GitHub issueFirst, let me say thank you to the maintainers of this package. I appreciate all the support you’ve given to users like me.
Issue
On my M1 Macbook Pro, I’ve followed the steps carefully outlined in the helpful guide: Connecting to SQL Server from Mac OSX. I have unixODBC and FreeTDS successfully installed. I can use isql and sqlcmd to successfully connect to & query a SQL Server database on my network. Now, I’d like to use pyodbc to accomplish the same thing.
sqlcmd Working As Expected:
$ sqlcmd -S sqlserver.mylocal.network,1433 -E -Q 'SELECT @@VERSION'
Microsoft SQL Server 2014 (SP3-GDR) (KB4583463) - 12.0.6164.21 (X64)
Nov 1 2020 04:25:14
Copyright (c) Microsoft Corporation
Standard Edition (64-bit) on Windows NT 6.3 <X64> (Build 9600: ) (Hypervisor)
isql Working As Expected:
$ isql MYMSSQL
+---------------------------------------+
| Connected! |
| |
| sql-statement |
| help [tablename] |
| echo [string] |
| quit |
| |
+---------------------------------------+
SQL>
pyodbc Not Working As Expected:
# test.py
import pyodbc
server = "sqlserver.mylocal.network"
port = "1433"
db = "mydb"
driver_str = "ODBC Driver 17 for SQL Server"
conn_str = (
f"Driver={{{driver_str}}};"
f"Server={server},{port};"
f"Database={db};"
"Trusted_Connection=yes;"
)
print(f"{conn_str=}")
conn = pyodbc.connect(conn_str)
print("Connected")
conn.close()
$ python3 test.py
conn_str='Driver={ODBC Driver 17 for SQL Server};Server=sqlserver.mylocal.network,1433;Database=mydb;Trusted_Connection=yes;'
# Hangs here. Never connects.
Using freetds.conf to create a DSN at least gives an immediate error message:
# test_with_freetds_dsn.py
import pyodbc
# the DSN value should be the name of the entry in odbc.ini, not freetds.conf
cnxn = pyodbc.connect("DSN=MYMSSQL;")
crsr = cnxn.cursor()
rows = crsr.execute("select @@VERSION").fetchall()
print(rows)
crsr.close()
cnxn.close()
$ python3 test_with_freetds_dsn.py
Traceback (most recent call last):
File "/Users/edrogers/repos/pyodbc-test/test_with_freetds_dsn.py", line 6, in <module>
cnxn = pyodbc.connect("DSN=MYMSSQL;")
pyodbc.Error: ('01000', '[01000] [FreeTDS][SQL Server]Unknown host machine name. (20013) (SQLDriverConnect)')
Things checked so far:
- I’ve ensured that my python version, installed by pyenv, is an arm64 build, matching my device architecture
- I’ve installed OpenSSL version 1.1.1o and ensured that it’s pointed to by each of
sqlcmd,isql, and my python version - I’ve built installed the
pyodbcpackage from source in my virtual environment - I also tried editing my
odbcinst.inifile to include:
[ODBC]
Trace=Yes
TraceFile=/dev/stdout
and I get verbose chatter during the connection process for sqlcmd and isql. None of that same chatter comes when using pyodbc though, which I find surprising.
Similarly, I can set the environment variable DYLD_PRINT_LIBRARIES=1 before calling either sqlcmd or isql to get an extensive list of libraries used by each. This environment variable doesn’t generate any similar listing when using pyodbc.
Let me know if there are any debug flags or environment variables I can set to allow further investigation. The silent hang from pyodbc is quite puzzling.
Environment Details & Diagnostic Commands w/ Output:
Python
- Version:
3.9.11(installed withpyenv) - Architecture:
arm64
$ python3 -c "import platform; print(platform.uname())"
uname_result(system='Darwin', node='Ed-Rogers.local', release='21.3.0', version='Darwin Kernel Version 21.3.0: Wed Jan 5 21:37:58 PST 2022; root:xnu-8019.80.24~20/RELEASE_ARM64_T6000', machine='arm64')
$ python3 -c "import platform; print(platform.architecture()[0])"
64bit
- OpenSSL Version:
1.1.1o
$ python3 -c "import ssl; print(ssl.OPENSSL_VERSION)"
OpenSSL 1.1.1o 3 May 2022
pyodbc
- Version:
4.0.32(installed from source onto my M1 Macbook following these instructions)
OS:
MacOS Monterey Version 12.2.1
$ uname -a
Darwin Ed-Rogers.local 21.3.0 Darwin Kernel Version 21.3.0: Wed Jan 5 21:37:58 PST 2022; root:xnu-8019.80.24~20/RELEASE_ARM64_T6000 arm64
OpenSSL:
$ openssl version
OpenSSL 1.1.1o 3 May 2022
UnixODBC & FreeTDS Drivers:
- unixODBC Version: 2.3.11
- FreeTDS Version: 1.3.10
- sqlcmd Version: 17.9.0001.1 Linux
$ odbcinst -j
unixODBC 2.3.11
DRIVERS............: /opt/homebrew/etc/odbcinst.ini
SYSTEM DATA SOURCES: /opt/homebrew/etc/odbc.ini
FILE DATA SOURCES..: /opt/homebrew/etc/ODBCDataSources
USER DATA SOURCES..: /Users/edrogers/.odbc.ini
SQLULEN Size.......: 8
SQLLEN Size........: 8
SQLSETPOSIROW Size.: 8
$ DYLD_PRINT_LIBRARIES=1 isql MYMSSQL 2>&1 | grep odbc
dyld[79381]: <3C74CDB4-A94B-3BDD-864E-ED4AD16EDD52> /opt/homebrew/Cellar/unixodbc/2.3.11/bin/isql
dyld[79381]: <567F0218-30B5-32A0-8400-BCCAEB9E27C0> /opt/homebrew/Cellar/unixodbc/2.3.11/lib/libodbc.2.dylib
dyld[79381]: <37095933-050A-35DB-A64C-90C28D9281D0> /opt/homebrew/Cellar/freetds/1.3.10/lib/libtdsodbc.0.so
dyld[79381]: <DA536636-6C8E-3000-817D-CC0D45EB9CCB> /opt/homebrew/Cellar/unixodbc/2.3.11/lib/libodbcinst.2.dylib
$ sqlcmd "-?" | grep Version
Version 17.9.0001.1 Linux
Target DB:
Microsoft SQL Server 2014
Sorry for the gigantic write-up. I’ve been investigating this on my own for a while. Hopefully I’ve missed something obvious.
Issue Analytics
- State:
- Created a year ago
- Comments:7 (3 by maintainers)
Top Results From Across the Web
Top Related Medium Post
Top Related StackOverflow Question
Troubleshoot Live Code
Top Related Reddit Thread
Top Related Hackernoon Post
Top Related Tweet
Top Related Dev.to Post
Top Related Hashnode Post
I’ve resolved this issue. @v-chojas, thank you so much for your guidance and help troubleshooting.
The problem was likely that I had problematic environment variables set before installing my python version with pyenv. (In particular, a Medium post I mentioned above, titled “Installing pyodbc and unixodbc for Apple Silicon” led me astray by recommending iODBC options be set for my environment variables.) I arrived at this diagnosis by searching for ODBC in my python build library directory.
Identifying unwanted iODBC flags in the python build:
The output ended up showing the problematic
-liodbcand-liodbcinstflags in my build.So, the solution was to simply uninstall my python version with pyenv, set my preferred environment variables, and reinstall my python with pyenv.
Rebuild python without iODBC flags in the environment:
Re-running the find & grep combo above showed that I no longer had references to
-liodbcor-liodbcinstin my python build. Next, creating a virtualenv and building pyODBC from source (usingpython3 setup.py buildandpython3 setup.py install) gave me a package with no references to iODBC.Verify that pyODBC is now linked solely to unixODBC as its DM:
Now that IODBC is not referenced by pyODBC, everything works as expected.
Thank you again!
Thanks for this guidance. Carefully inspecting the output when I build from source, I do see
-liodbcand-liodbcinstsneaking in. Not sure how to point pyODBC to the correct dylibs, though.Building from source with the recommended environment variables exported for Mac [1] [2]:
It’s the last line that includes the offending references to iODBC. To help in parsing it, I’ll rebuild, grabbing only the last line, and inserting line-breaks and enumeration, for clarity:
In the line-break delimited version above, I can see on lines 5 & 12 the problematic references to iODBC. Looking at how the setup.py constructs the
ext_modulessettings kwarg on the darwin platform, it’s unclear to me how the iODBC references are sneaking in there.FWIW, if I use
odbc_config, (a CLI I didn’t know about until reading the linux portion of thesetup.pysource code), I see output that looks more in-line with my expectations, as well as to what’s in lines 16 & 17 above:Any suggestions on things to try next?