insert with autoincrement fails with sqlalchemy.exc.ArgumentError

What did I do

Create a table with an autoincrement column as in your README . Then fetch the table into a new MetaData object and insert, relying on autoincrement / no value for that column.

What did I expect

A row to be created, with the autoincrement column filled from the sequence

Error

sqlalchemy.exc.ArgumentError: Column type DECIMAL(38, 0) on column ‘sequencetable.id’ is not compatible with autoincrement=True

Reproduce

metadata = MetaData()
t1 = Table('sequencetable', metadata,
    Column('id', Integer, Sequence('id_seq'), primary_key=True),
    Column('data', String(39))
)

t1.create(engine)

### this works
connection.execute(t1.insert(), [ {'data': 'test_insert'}])

select_stmt_t1 = select([t1])
connection.execute(select_stmt_t1).fetchall()

metadata2 = MetaData()
t2 = Table('sequencetable', metadata2, autoload=True, autoload_with=engine)

### this fails 
connection.execute(t2.insert(), [ {'data': 'multi_insert_1'}, {'data': 'multi_insert_2'} ])
connection.execute(t2.insert(), [ {'data': 'test_insert'}])

### this works
seq = Sequence('id_seq')
nextid = connection.execute(seq)
connection.execute(t2.insert(), [ {'id': nextid, 'data': 'test_insert'}])

Remark: The above code stopped working for me now. When I create the table, it fails with

sqlalchemy.exc.ProgrammingError: (snowflake.connector.errors.ProgrammingError) 090106 (22000): Cannot perform CREATE SEQUENCE. This session does not have a current schema. Call 'USE SCHEMA', or use a qualified name.
[SQL: CREATE SEQUENCE id_seq]

As a workaround I will create and use the sequence explicitly.

While you are at it, maybe correct the autoinrecment typo in your tests.

Version info

Python 3.7.4 Darwin-18.7.0-x86_64-i386-64bit

Package                    Version  
-------------------------- ---------
asn1crypto                 1.2.0    
azure-common               1.1.23   
azure-storage-blob         2.1.0    
azure-storage-common       2.1.0    
boto3                      1.9.253  
botocore                   1.12.253 
certifi                    2019.9.11
cffi                       1.13.1   
chardet                    3.0.4    
cryptography               2.8      
docutils                   0.15.2   
future                     0.18.1   
idna                       2.8      
ijson                      2.5.1    
jmespath                   0.9.4    
oscrypto                   1.1.0    
pip                        19.3.1   
pycparser                  2.19     
pycryptodomex              3.9.0    
PyJWT                      1.7.1    
pyOpenSSL                  19.0.0   
python-dateutil            2.8.0    
pytz                       2019.3   
requests                   2.22.0   
s3transfer                 0.2.1    
setuptools                 41.4.0   
six                        1.12.0   
snowflake-connector-python 2.0.2    
snowflake-sqlalchemy       1.1.16   
SQLAlchemy                 1.3.10   
urllib3                    1.25.6   
wheel                      0.33.6   

[ynux]

I agree, autoincrement=True gets ignored. In that case, the insert throws yet another error: snowflake.connector.errors.ProgrammingError: 100072 (22000): NULL result in a non-nullable column

[sfc-gh-aling]

hey folks, apologize for the late response and thanks for your patience.

we’ve investigated the issue found the existing code was not doing correctly in two aspects:

1. The first issue is that when we create a column with Sequence, the default value was not set correctly, it should not be set using “autoincrement” which failed to connect the column with the sequence. - to fix the issue, we should use “default seq.nextval” to set as the column’s default value. - for more context please check doc here for “autoincrement” and “default” keyword: https://docs.snowflake.com/en/sql-reference/sql/create-table.html#optional-parameters
2. The second issue is caused by type affinity check, specifically sqlalchemy is checking whether DECIMAL(NUMBER in snowflake) is a subclass of INTEGER (this is what user hit). - In snowflake, INTEGER is synonymous of NUMERIC (DECIMAL), see doc here: https://docs.snowflake.com/en/sql-reference/data-types-numeric.html, when we create a sqlalchemy Integer in snowflake, a NUMBER(38, 0) (DECIMAL is sqlalchemy) is created. - to fix the issue, we need to customize the affinity check of DECIMAL in which case DECIMAL(38,0) is affinity of INTEGER.

We’ve fixed the aforementioned issues in the PR https://github.com/snowflakedb/snowflake-sqlalchemy/pull/297 which shall be included in our next release, will keep you informed once the package is out.