SNOW-638841: `int`s returned as `Decimal`s when upgrading from version 1.3.4 to 1.4.0

Please answer these questions before submitting your issue. Thanks!

1. What version of Python are you using?

3.10.5

1. What operating system and processor architecture are you using?

macOS-12.4-x86_64-i386-64bit

1. What are the component versions in the environment (pip freeze)?

Running conda list returns:

# Name                    Version                   Build  Channel
abseil-cpp                20210324.2           he49afe7_0    conda-forge
appnope                   0.1.3              pyhd8ed1ab_0    conda-forge
arrow-cpp                 7.0.0           py310hf221cfc_8_cpu    conda-forge
asn1crypto                1.5.1              pyhd8ed1ab_0    conda-forge
asttokens                 2.0.5              pyhd8ed1ab_0    conda-forge
aws-c-cal                 0.5.11               hd2e2f4b_0    conda-forge
aws-c-common              0.6.2                h0d85af4_0    conda-forge
aws-c-event-stream        0.2.7               hb9330a7_13    conda-forge
aws-c-io                  0.10.5               h35aa462_0    conda-forge
aws-checksums             0.1.11               h0010a65_7    conda-forge
aws-sdk-cpp               1.8.186              h766a74d_3    conda-forge
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                        py_2    conda-forge
backports.functools_lru_cache 1.6.4              pyhd8ed1ab_0    conda-forge
brotlipy                  0.7.0           py310h1961e1f_1004    conda-forge
bzip2                     1.0.8                h0d85af4_4    conda-forge
c-ares                    1.18.1               h0d85af4_0    conda-forge
ca-certificates           2022.6.15            h033912b_0    conda-forge
certifi                   2022.6.15       py310h2ec42d9_0    conda-forge
cffi                      1.15.1          py310h96bbf6e_0    conda-forge
charset-normalizer        2.0.12             pyhd8ed1ab_0    conda-forge
cryptography              35.0.0          py310ha82f1d4_2    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
executing                 0.9.1              pyhd8ed1ab_0    conda-forge
flake8                    4.0.1              pyhd8ed1ab_2    conda-forge
gflags                    2.2.2             hb1e8313_1004    conda-forge
glog                      0.6.0                h8ac2a54_0    conda-forge
gmp                       6.2.1                h2e338ed_0    conda-forge
greenlet                  1.1.2           py310h9d931ec_2    conda-forge
grpc-cpp                  1.45.2               hb472a99_3    conda-forge
idna                      3.3                pyhd8ed1ab_0    conda-forge
importlib-metadata        4.11.4          py310h2ec42d9_0    conda-forge
ipython                   8.4.0           py310h2ec42d9_0    conda-forge
jedi                      0.18.1          py310h2ec42d9_1    conda-forge
krb5                      1.19.3               hb49756b_0    conda-forge
libblas                   3.9.0           15_osx64_openblas    conda-forge
libbrotlicommon           1.0.9                h5eb16cf_7    conda-forge
libbrotlidec              1.0.9                h5eb16cf_7    conda-forge
libbrotlienc              1.0.9                h5eb16cf_7    conda-forge
libcblas                  3.9.0           15_osx64_openblas    conda-forge
libcrc32c                 1.1.2                he49afe7_0    conda-forge
libcurl                   7.83.1               h372c54d_0    conda-forge
libcxx                    14.0.6               hce7ea42_0    conda-forge
libedit                   3.1.20191231         h0678c8f_2    conda-forge
libev                     4.33                 haf1e3a3_1    conda-forge
libevent                  2.1.10               h815e4d9_4    conda-forge
libffi                    3.4.2                h0d85af4_5    conda-forge
libgfortran               5.0.0           9_3_0_h6c81a4c_23    conda-forge
libgfortran5              9.3.0               h6c81a4c_23    conda-forge
libgoogle-cloud           1.40.2               hc6327d4_0    conda-forge
libiconv                  1.16                 haf1e3a3_0    conda-forge
liblapack                 3.9.0           15_osx64_openblas    conda-forge
libnghttp2                1.47.0               h942079c_0    conda-forge
libopenblas               0.3.20          openmp_hb3cd9ec_0    conda-forge
libprotobuf               3.20.1               h2292cb8_0    conda-forge
libssh2                   1.10.0               h52ee1ee_2    conda-forge
libthrift                 0.16.0               h9702cd6_1    conda-forge
libutf8proc               2.7.0                h0d85af4_0    conda-forge
libzlib                   1.2.12               hfe4f2af_2    conda-forge
llvm-openmp               14.0.4               ha654fa7_0    conda-forge
lz4-c                     1.9.3                he49afe7_1    conda-forge
matplotlib-inline         0.1.3              pyhd8ed1ab_0    conda-forge
mccabe                    0.6.1                      py_1    conda-forge
ncurses                   6.3                  h96cf925_1    conda-forge
numpy                     1.23.1          py310ha3f357c_0    conda-forge
openssl                   1.1.1q               hfe4f2af_0    conda-forge
orc                       1.7.5                h4856350_0    conda-forge
oscrypto                  1.2.1              pyhd3deb0d_0    conda-forge
parquet-cpp               1.5.1                         2    conda-forge
parso                     0.8.3              pyhd8ed1ab_0    conda-forge
pexpect                   4.8.0              pyh9f0ad1d_2    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pip                       22.2               pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.30             pyha770c72_0    conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
pyarrow                   7.0.0           py310hb135652_8_cpu    conda-forge
pycodestyle               2.8.0              pyhd8ed1ab_0    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pycryptodomex             3.15.0          py310h6c45266_0    conda-forge
pyflakes                  2.4.0              pyhd8ed1ab_0    conda-forge
pygments                  2.12.0             pyhd8ed1ab_0    conda-forge
pyjwt                     2.4.0              pyhd8ed1ab_0    conda-forge
pyodbc                    4.0.34          py310hd4537e4_0    conda-forge
pyopenssl                 22.0.0             pyhd8ed1ab_0    conda-forge
pysocks                   1.7.1           py310h2ec42d9_5    conda-forge
python                    3.10.5          hdaaf3db_0_cpython    conda-forge
python_abi                3.10                    2_cp310    conda-forge
pytz                      2022.1             pyhd8ed1ab_0    conda-forge
re2                       2022.04.01           h96cf925_0    conda-forge
readline                  8.1.2                h3899abd_0    conda-forge
requests                  2.28.1             pyhd8ed1ab_0    conda-forge
setuptools                63.2.0          py310h2ec42d9_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
snappy                    1.1.9                h6e38e02_1    conda-forge
snowflake-connector-python 2.7.8           py310h692597a_0    conda-forge
snowflake-sqlalchemy      1.3.4              pyhd8ed1ab_0    conda-forge
sqlalchemy                1.4.39          py310h6c45266_0    conda-forge
sqlite                    3.39.2               hd9f0692_0    conda-forge
stack_data                0.3.0              pyhd8ed1ab_0    conda-forge
tk                        8.6.12               h5dbffcc_0    conda-forge
traitlets                 5.3.0              pyhd8ed1ab_0    conda-forge
tzdata                    2022a                h191b570_0    conda-forge
unixodbc                  2.3.10               h7b58acd_0    conda-forge
urllib3                   1.26.11            pyhd8ed1ab_0    conda-forge
wcwidth                   0.2.5              pyh9f0ad1d_2    conda-forge
wheel                     0.37.1             pyhd8ed1ab_0    conda-forge
xz                        5.2.5                haf1e3a3_1    conda-forge
zipp                      3.8.0              pyhd8ed1ab_0    conda-forge
zlib                      1.2.12               hfe4f2af_2    conda-forge
zstd                      1.5.2                ha9df2e0_2    conda-forge

I created this environment by creating a new conda environment and running mamba install sqlalchemy snowflake-sqlalchemy pydobc.

1. What did you do?

Running this, with appropriate environment variables and adapted connection_string

import os
import sqlalchemy as sa

user = os.environ.get("SNOWFLAKE_USER", "datajudge")
password = os.environ.get("SNOWFLAKE_PASSWORD")
account = os.environ.get("SNOWFLAKE_ACCOUNT", "")
connection_string = f"snowflake://{user}:{password}@{account}/datajudge/DBO?warehouse=datajudge&role=accountadmin"
engine = sa.create_engine(connection_string)
metadata = sa.MetaData()

schema = "dbo"
table_name = "bug_report_table"
column_name = "col_int"

data = [{"col_int": i} for i in range(1, 10)]

with engine.connect() as conn:
    conn.execute(f"DROP TABLE IF EXISTS {schema}.{table_name}")
    table = sa.Table(table_name, metadata, sa.Column(column_name, sa.Integer()), schema=schema)
    table.create(conn)
    conn.execute(table.insert().values(), data)

table = sa.Table(
    table_name,
    sa.MetaData(),
    autoload_with=engine,
    schema=schema,
)
selection = sa.select(table.c[column_name])

with engine.connect() as conn:
    rows = conn.execute(selection).fetchall()
print(rows)

The printed result corresponds to:

[(Decimal('1'),), (Decimal('2'),), (Decimal('3'),), (Decimal('4'),), (Decimal('5'),), (Decimal('6'),), (Decimal('7'),), (Decimal('8'),), (Decimal('9'),)]

1. What did you expect to see?

When manually downgrading snowflake-sqlalchemy to version 1.3.4, running the same script returns: [(1,), (2,), (3,), (4,), (5,), (6,), (7,), (8,), (9,)]

I observe that the records of in-database type sqlalchemy.Integer are now of type int, instead of type decimal.Decimal. In other words, from version 1.3.4 to 1.4.0 snowflake-sqlalchemy broke my expected behaviour.

1. Can you set logging to DEBUG and collect the logs?

The logs are very extensive and I’m not sure they’re of great use here. Happy to provided nevertheless if someone thinks otherwise.

[YYYasin19]

Sorry, I just realized that my comment may have been confusing. I am getting Decimal instead of float numbers directly.

The test table has the following columns

>> engine.execute("SELECT AVG(gpa) FROM students_modified_1").all()
[(3.7075,)]

>> engine.execute("SELECT AVG(age) FROM students_modified_1").all()
[(Decimal('35.250000'),)]

The reason is of course the Snowflake datatype, which includes a precision that can’t be captured by float, so the processor is indeed correct!

I am going to either convert everything manually to float or patch something around here.

Thanks!

[sfc-gh-aling]

hi @kklein , we have fixed in the issue in the latest release 1.4.1, please upgrade to the latest version and try it out, thanks! I’m closing the issue now, but feel free to reopen if the issue still exists.