question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

redshift RA3 nodes support cross-db references, but dbt prevents this in compilation

See original GitHub issue

https://github.com/fishtown-analytics/dbt/blob/841b2115a42cc20658ec69ba129d515412993d10/core/dbt/adapters/base/relation.py#L440-L442

this is tricky because there is a long term fork in redshift behavior based on node type. RA3 nodes support cross db references (this is now GA) but gen2 node types do not. the redshift/postgres connector may need to be subclassed for ra3 node types with the db-ref-count omitted. It might be possible to distinguish between node types by issuing a version command.

If simplicity of implementation is preferred, an override could maybe be added to the project.yml spec (like, force-enable-cross-db=true or something as an option which defaults to false for the next release and automatic detection could be added in a future release?

I don’t know the codebase well enough to assess what strategy would create the least tech debt.

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:5 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
salmonsdcommented, Jul 29, 2021

Following up here, this seems to be resolved in #3408 and should be included is v0.21.

cc: @jtcohen6

0reactions
jtcohen6commented, Jul 29, 2021

You’re right! This will be released in v0.21.0-b1 for starters, and it will require flipping on a ra3: true parameter in the Redshift profile. I’m going to close this issue 😃

Read more comments on GitHub >

github_iconTop Results From Across the Web

cross db queries in redshift · Issue #3236 - GitHub
With the advent of cross-db querying on RA3 nodes, ... Error [0A000]: ERROR: cross-database reference to database "sources" is not supported.
Read more >
Support for cross-database sources on Redshift RA3 instances
Cross-database queries for RA3 instances are now supported by dbt Cloud projects using a Redshift connection. With cross-database queries, ...
Read more >
Considerations - Amazon Redshift - AWS Documentation
4xlarge, ra3.16xlarge, and ra3.xlplus node types. Amazon Redshift supports joining data from tables or views across one or more databases in the same...
Read more >
Understanding Redshift RA3 Node: A Comprehensive Guide
In this article, you will gain information about Amazon Redshift, its key features, and the Node types supported by Redshift. You will also...
Read more >
DBT Cloud: Trying to conditional create tables in a DB based ...
When using Redshift with dbt, make sure to set ra3_node = true in your profiles.yml since the older node type of DC2 does...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found