IAM authentication leads to unclosed socket warning

Driver version

^2.0.907

Redshift version

’ version ‘PostgreSQL 8.0.2 on i686-pc-linux-gnu, compiled by GCC gcc (GCC) 3.4.2 20041017 (Red Hat 3.4.2-6.fc3), Redshift 1.0.38094��’

Client Operating System

Darwin Kernel Version 20.6.0

Python version

3.7, 3.9

Table schema

Problem description

Execution of a simple sample leads to unclosed socket warning:

1. Expected behaviour: All sockets closed
2. Actual behaviour: Seems like a memory leak
3. Error message/stack trace:

/usr/local/opt/python@3.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py:1264: ResourceWarning: unclosed <ssl.SSLSocket fd=8, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('...', 56127), raddr=('...', 443)>
  return list(_active.values()) + list(_limbo.values())
Object allocated at (most recent call last):
  File "/usr/local/opt/python@3.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py", lineno 833
    self = cls.__new__(cls, **kwargs)

.venv/lib/python3.9/site-packages/redshift_connector/iam_helper.py:131: ResourceWarning: unclosed <ssl.SSLSocket fd=8, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('....', 60344), raddr=('...', 443)>
  IamHelper.set_cluster_credentials(provider, info)
Object allocated at (most recent call last):
  File "/usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", lineno 1003
    self = cls.__new__(cls, **kwargs)

1. Any other details that can be helpful: There is no such error when setting iam=False.

Python Driver trace logs

Reproduction code

with redshift_connector.connect(
            iam=True,
            database=REDSHIFT_DATABASE,
            db_user=REDSHIFT_DB_USER,
            cluster_identifier=REDSHIFT_CLUSTER_IDENTIFIER,
            region=DATASOURCE_AWS_REGION,
            profile=DATASOURCE_AWS_PROFILE,
            timeout=self.timeout_sec
        ) as connection, connection.cursor() as cursor:
    cursor.execute("SELECT 1")
    return cursor.fetch_dataframe()

[Brooke-white]

Hey @Tradunsky , thanks for following up. We can monitor the referenced solution, and if merged, take steps to apply to redshift_connector as well.

[Brooke-white]

this response on the issue gives some context to the cause of the warning, and tradeoffs of the PRs that have been posted to resolve this.