Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Using structured fields with PyAthena

See original GitHub issue

I’m trying to query a table that has columns with struct fields in the following format:

{quantity=1, price=0.1, valid_data=2022-xx-xx 08:42:10.2431}

I can execute simple queries using the sub-fields inside these structures, but I would like to use ORM. However, I can’t seem to find a way to map struct fields and these seem to be simply interpreted as a string “{quantity… }”

Is there currently a way to use PyAthena along with SQLAlchemy ORM to map these fields?

Thanks!

Issue Analytics

State:
Created a year ago
Comments:5 (2 by maintainers)

Top GitHub Comments

1reaction

laughingman7743commented, Sep 30, 2022

You can register an issue in the python-bigquery-sqlalchemy repository. https://github.com/googleapis/python-bigquery-sqlalchemy/issues

1reaction

laughingman7743commented, Sep 28, 2022

Currently, struct type is not supported. Maybe python-bigquery-sqlalchemy’s handling of the STRUCT type would be helpful for implementation. https://github.com/googleapis/python-bigquery-sqlalchemy/blob/main/sqlalchemy_bigquery/_struct.py#L46

Top Results From Across the Web

Querying arrays with complex types and nested structures

Examples in this section show how to change element's data type, locate elements within arrays, and find keywords using Athena queries.

C-like structures in Python - Stack Overflow

You can access the fields of a class using a dictionary because the fields of a class, its methods and all its properties...

Working with complex types | The Athena Guide

Fields in nested structs are accessed the same way, just like how you would access properties in a JavaScript object.

Data Science with Python Pandas by Athena Kan - YouTube

Harvard Business Review named data scientist "the sexiest job of the 21st century." Python pandas is a commonly-used tool in the industry to ......

PyAthena - PyPI

You can use pandas.DataFrame.to_sql to write records stored in DataFrame to Amazon Athena. pandas.DataFrame.to_sql uses SQLAlchemy, so you need to install it.