OS X iODBC queries raise syntax error in python 3.5.0

This might be related to #57 and #73.

When using python 3.5.0, execute() always raises a pyodbc error complaining about a syntax error. With the FileMaker driver, this is pyodbc.ProgrammingError, but with the SQLite driver this is pyodbc.Error. I first encountered this problem with the FileMaker iODBC driver. To make sure it wasn’t (just) the driver, I’ve also tested against “Actual Open Source Databases” iODBC drivers for SQLite. I encounter a similar syntax error problem in both cases.

I’m using pyodbc 3.0.10. I have virtual environments set up for python 2.7.5 (stock OS X 10.9) and python 3.5.0 (homebrew). I only encounter this problem with python 3.5.0.

I believe pyodbc is built against iODBC 3.52.10 in both cases, but I’m not sure how to verify that. I created a homebrew receipt to install iodbc.

👍 Python 2.7.5, “FileMaker” driver:

>>> import pydobc
>>> cnxn = pyodbc.connect('DSN=MyDatabase;UID=username;PWD=password;")
>>> cnxn.cursor().execute('select * from FileMaker_Tables').fetchone()
('TableOccurence', 1065210.0, 'BaseTable', 'MyDatabase', 12.0)

👎 Python 3.5.0, “FileMaker” driver:

>>> import pyodbc
>>> cnxn = pyodbc.connect("DSN=MyDatabase;UID=username;PWD=password;")
>>> The following statement always works in FileMaker, regardless of schema.
... cnxn.cursor().execute('''select * from FileMaker_Tables''')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
pyodbc.ProgrammingError: ('42000', '[42000] [FileMaker][FileMaker] FQL0001/(1:1): There is an error in the syntax of the query. (8310) (SQLExecDirectW)')

>>> # A more typical query
>>> cnxn.cursor().execute('''select "id" from "test_table" ''')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
pyodbc.ProgrammingError: ('42000', '[42000] [FileMaker][FileMaker] FQL0001/(1:1): There is an error in the syntax of the query. (8310) (SQLExecDirectW)')

👍 Python 2.7.5, “Actual Open Source Databases” driver:

Python 2.7.5 (default, Mar  9 2014, 22:15:05) 
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyodbc
>>> cnxn = pyodbc.connect('DSN=MyDatabase;UID=username;PWD=password')
>>> cnxn.cursor().execute('select * from action').fetchone()
(1, 71427167, '66486446233213756711660861965240535626013997849', datetime.date(2014, 1, 10), 1)

👎 Python 3.5.0, “Actual Open Source Databases” driver:

Python 3.5.0 (default, Sep 23 2015, 04:41:33) 
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyodbc
>>> cnxn = pyodbc.connect('DSN=MyDatabase;UID=username;pwd=password')
>>> cnxn.cursor().execute('select * from action').fetchone()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
pyodbc.Error: ('HY000', '[HY000] [Actual][SQLite] near "s": syntax error (0) (SQLExecDirectW)')

My guess is that this is an encoding problem, that the iODBC driver manager is expecting some 8-bit encoding and it’s receiving something like UTF-16.

My goal here is to access FileMaker, which is why I’m using the iODBC driver manager. My understanding is that I cannot use the FileMaker driver with unixODBC, so I’m stuck with iODBC.

[raphaelvannson]

Same issue attempting to connect to an Aster database from Mac OS X on python3.4.

Looking into the doc, I actually found how to set the encoding settings. This solved the issue.

Make sure to have a look at the ‘Connecting to Databases’ section of the wiki

In my case these instructions resolved my problem.

Thank you for all the good work on the doc (and the driver) @johnchristopherjones !

Environment

$ uname -a
Darwin MUSRV186016-382 14.5.0 Darwin Kernel Version 14.5.0: Tue Apr 11 16:12:42 PDT 2017; root:xnu-2782.50.9.2.3~1/RELEASE_X86_64 x86_64

$ isql --version
unixODBC 2.3.4

$ python3 --version
Python 3.4.4

Symptoms

Using isql (command line client)

No issues here.

$ isql -v 'mydb'
+---------------------------------------+
| Connected!                            |
|                                       |
| sql-statement                         |
| help [tablename]                      |
| quit                                  |
|                                       |
+---------------------------------------+
SQL> select tablename, schemaname from NC_SYSTEM.nc_all_tables t, NC_SYSTEM.nc_all_schemas s where t.schemaid=s.schemaid
...
...
...
SQLRowCount returns -1
141 rows fetched
SQL>

Using pyodbc (in jupyter notebook)

import os
import pyodbc

conn = pyodbc.connect('dsn=mydb')
cursor = conn.cursor()

# This should list all the tables in the database
cursor.execute('select tablename, schemaname from NC_SYSTEM.nc_all_tables t, NC_SYSTEM.nc_all_schemas s where t.schemaid=s.schemaid')

---------------------------------------------------------------------------
ProgrammingError                          Traceback (most recent call last)
<ipython-input-20-59d076043433> in <module>()
----> 1 cursor.execute('select tablename, schemaname from NC_SYSTEM.nc_all_tables t, NC_SYSTEM.nc_all_schemas s where t.schemaid=s.schemaid')

ProgrammingError: ('42000', '[42000] [AsterData][nCluster] (34) ERROR: syntax error at or near "���������������������������������������������������������d".  (34) (SQLExecDirectW)')

cursor.close()
conn.close()

Resolution

Add the proper encoding settings to the connection.

import os
import pyodbc

conn = pyodbc.connect('dsn=mydb')
conn.setdecoding(pyodbc.SQL_CHAR, encoding='utf-8')
conn.setdecoding(pyodbc.SQL_WCHAR, encoding='utf-8')
conn.setdecoding(pyodbc.SQL_WMETADATA, encoding='utf-32le')
conn.setencoding(encoding='utf-8')

cursor = conn.cursor()

# This should list all the tables in the database
cursor.execute('select tablename, schemaname from NC_SYSTEM.nc_all_tables t, NC_SYSTEM.nc_all_schemas s where t.schemaid=s.schemaid')

<pyodbc.Cursor at 0x10948dc30>

cursor.close()
conn.close()

[macfreek]

Thanks @raphaelvannson, I had the same issue on macOS with unixODBC, Python 3.6 and FileMakerPro.

@mkleehammer Is there a way to set these encodings by default? It seems that the problem is slightly more widespread, and will affect new users.