Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Using structured fields with PyAthena

See original GitHub issue

I’m trying to query a table that has columns with struct fields in the following format:

{quantity=1, price=0.1, valid_data=2022-xx-xx 08:42:10.2431}

I can execute simple queries using the sub-fields inside these structures, but I would like to use ORM. However, I can’t seem to find a way to map struct fields and these seem to be simply interpreted as a string “{quantity… }”

Is there currently a way to use PyAthena along with SQLAlchemy ORM to map these fields?


Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:5 (2 by maintainers)

github_iconTop GitHub Comments

laughingman7743commented, Sep 30, 2022

You can register an issue in the python-bigquery-sqlalchemy repository.

laughingman7743commented, Sep 28, 2022

Currently, struct type is not supported. Maybe python-bigquery-sqlalchemy’s handling of the STRUCT type would be helpful for implementation.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Querying arrays with complex types and nested structures
Examples in this section show how to change element's data type, locate elements within arrays, and find keywords using Athena queries.
Read more >
C-like structures in Python - Stack Overflow
You can access the fields of a class using a dictionary because the fields of a class, its methods and all its properties...
Read more >
Working with complex types | The Athena Guide
Fields in nested structs are accessed the same way, just like how you would access properties in a JavaScript object.
Read more >
Data Science with Python Pandas by Athena Kan - YouTube
Harvard Business Review named data scientist "the sexiest job of the 21st century." Python pandas is a commonly-used tool in the industry to ......
Read more >
PyAthena - PyPI
You can use pandas.DataFrame.to_sql to write records stored in DataFrame to Amazon Athena. pandas.DataFrame.to_sql uses SQLAlchemy, so you need to install it.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found