question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

cross db queries in redshift

See original GitHub issue

Describe the bug

When using source database that is different from the target (https://docs.getdbt.com/reference/resource-properties/database), I ran into an issue that adapter doesn’t fetch columns, since it seems like it looks only into target db for meta information about them: https://github.com/fishtown-analytics/dbt/blob/develop/plugins/redshift/dbt/include/redshift/macros/adapters.sql#L79

Steps To Reproduce

  1. Have target db different from source db
  2. Run model based on source table that doesn’t exist on target using adapter macroses

Expected behavior

Adapter expected to get meta information about source tables from source database

Screenshots and log output

Not applicable

System information

Which database are you using dbt with?

  • postgres
  • redshift
  • bigquery
  • snowflake
  • other (specify: ____________)

The output of dbt --version:

0.17.2

The operating system you’re using: OSX

The output of python --version: 3.9.4

Additional context

Not applicable

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Reactions:1
  • Comments:15 (9 by maintainers)

github_iconTop GitHub Comments

2reactions
sonaccommented, Apr 8, 2021

indeed, I’d be keen helping, since this functionality would ease our testing setup.

0reactions
jtcohen6commented, Jun 9, 2021

This was resolved by #3408 — as in, the bit we can do for now: querying sources that live in other databases when using RA3 nodes. When Redshift adds write DDL/DML support, we can consider adding support for other resources types in other databases.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Examples of using a cross-database query - Amazon Redshift
Use the following examples to help learn how to set up a cross-database query that references an Amazon Redshift database.
Read more >
Using cross-database queries with the query editor
You can use cross-database queries to access data from any of the databases on your Amazon Redshift cluster without having to connect to...
Read more >
Amazon Redshift Cross-Database Queries and Data ...
Users of Amazon Redshift can now run cross-database queries and share data across Redshift clusters as AWS released these enhancements to ...
Read more >
Query against two separate Redshift clusters in a single query?
No, it's no possible in Redshift directly. Additionally you cannot query across multiple databases on the same cluster.
Read more >
Cross-Database Queries - Amazon Redshift Research Project
Cross -database queries are not implemented by improvements in the use of the system tables such that Redshift can now access tables across...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found