'Engine' object has no attribute 'connection' when calling SQLAlchemy's `get_columns` with an `Engine`.

Hey @laughingman7743,

First of all, thanks a ton for putting together this nice Python wrapper for AWS Athena and also including a SQLAlchemy interface!

Sadly, when trying it out with Superset, I ran into the following issue:

  File "/root/venv/lib/python3.5/site-packages/pyathena/sqlalchemy_athena.py", line 187, in get_columns
    stop=stop_after_attempt(connection.connection.retry_attempt),
AttributeError: 'Engine' object has no attribute 'connection'

The trouble is that PyAthena expects a Connection in its get_columns interface (https://github.com/laughingman7743/PyAthena/blob/master/pyathena/sqlalchemy_athena.py#L157). In particular, it expects that connection.connection will exist, which sadly is not the case when connection is of the Engine type.

I am not familiar enough with SQLAlchemy to comment on what should be done differently, but this greatly hampers the usability of PyAthena with Superset. My current fix is to use PyAthenaJDBC (thank you for putting that together as well!), which is significantly slower.

Thus, I would like to see what can be done to fix this in PyAthena, as it otherwise works very well.

I am also happy to put together a PR with the fix – I’d just like to know your thoughts on what approach would work best with the overall architecture.

To make testing simpler, I’ve also put together a minimal example which demonstrates this issue.

from urllib.parse import quote_plus  # PY2: from urllib import quote_plus
from sqlalchemy.engine import create_engine
import sqlalchemy

conn_str = 'awsathena+rest://{aws_access_key_id}:{aws_secret_access_key}@athena.{region_name}.amazonaws.com:443/'\
           '{schema_name}?s3_staging_dir={s3_staging_dir}'
engine = create_engine(conn_str.format(
    aws_access_key_id=quote_plus('AWS_ACCESS_KEY_ID'),
    aws_secret_access_key=quote_plus('AWS_SECRET_ACCESS_KEY'),
    region_name='eu-west-1',
    schema_name='default',
    s3_staging_dir=quote_plus('s3://some-bucket')))
inspector = sqlalchemy.inspect(engine)
print(inspector.get_columns('table_name', 'schema_name'))

Thanks a ton again!

[mrshu]

@laughingman7743 I believe one way of doing that would be by calling .raw_connection() on the passed first argument (connection) when its type is Engine.

Do you think that may work? You may then be able to access the underlying DB-API connection.

[mrshu]

The first argument connection seems to be sqlalchemy.engine.Engine or sqlalchemy.engine.base.Connection. I try to check the instance type.

Makes a ton of sense – seems like that indeed fixed the issue for me.

Thanks a ton for your prompt responses – once you merge and release #65 I believe this issue can be closed.