[KED-2141] Impala support for kedro, custom SQLdataset
See original GitHub issueDescription
To use kedro with impala. Is is possible to extend the current SQLDataset to create a connection instead of accepting a string only?
Context
Integration with Impala is useful.
Possible Implementation
I am using the python package impyla. To query something with pandas, I need to first create a connection.
from impala.dbapi import connect
user=user
password=password
host=host
conn = connect(host=host, port=21050, user=user, password=password)
sql = 'select * from trable'
pd.read_sql(sql, conn)
In theory, as long as I can implement the connection creation logic manually, the rest should just match the standard pandasSQLdataset.
Issue Analytics
- State:
- Created 3 years ago
- Comments:9 (7 by maintainers)
Top Results From Across the Web
[KED-2141] Impala support for kedro, custom SQLdataset
I am using the python package impyla. To query something with pandas, I need to first create a connection. ... In theory, as...
Read more >kedro.extras.datasets.pandas.SQLQueryDataSet
It does not support save method so it is a read only data set. To save data to a SQL server use SQLTableDataSet...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found

Cool, see if I can come up with a PR this week or next week.
On Wed, 10 Mar 2021 at 5:12 PM, Merel Theisen notifications@github.com wrote:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.