question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[KED-2141] Impala support for kedro, custom SQLdataset

See original GitHub issue

Description

To use kedro with impala. Is is possible to extend the current SQLDataset to create a connection instead of accepting a string only?

Context

Integration with Impala is useful.

Possible Implementation

I am using the python package impyla. To query something with pandas, I need to first create a connection.

from impala.dbapi import connect

user=user
password=password
host=host

conn = connect(host=host, port=21050, user=user, password=password)
sql = 'select * from trable'
pd.read_sql(sql, conn)

In theory, as long as I can implement the connection creation logic manually, the rest should just match the standard pandasSQLdataset.

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:9 (7 by maintainers)

github_iconTop GitHub Comments

1reaction
noklamcommented, Mar 10, 2021

Cool, see if I can come up with a PR this week or next week.

On Wed, 10 Mar 2021 at 5:12 PM, Merel Theisen notifications@github.com wrote:

@noklam https://github.com/noklam This isn’t a priority for the team at the moment, so we’d be very happy to accept a PR.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/quantumblacklabs/kedro/issues/504#issuecomment-795127374, or unsubscribe https://github.com/notifications/unsubscribe-auth/AELAWL5S5DDXFB6IUHFBJQ3TC4ZYHANCNFSM4Q7NYALQ .

0reactions
stale[bot]commented, May 9, 2021

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Read more comments on GitHub >

github_iconTop Results From Across the Web

[KED-2141] Impala support for kedro, custom SQLdataset
I am using the python package impyla. To query something with pandas, I need to first create a connection. ... In theory, as...
Read more >
kedro.extras.datasets.pandas.SQLQueryDataSet
It does not support save method so it is a read only data set. To save data to a SQL server use SQLTableDataSet...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found