Remove double underscore in epheremal model generated CTE (__dbt__CTE__...)
See original GitHub issueDescribe the bug
In Exasol identifiers starting with double underscore are illegal - hence emepheral models fail when using dbt-exasol adapter.
Steps To Reproduce
Configure you project to use dbt-exasol adapter (available on https://pypi.org/project/dbt-exasol/) and then create an ephemeral model. Once you use that model in a ref dbt generates a CTE named “dbt__CTE” which is not a valid identifier in exasol.
Expected behavior
Ephemeral models should work in Exasol. Therefore we suggest to generate those CTE without the leading “__” characters.
System information
Which database are you using dbt with?
- postgres
- redshift
- bigquery
- snowflake
- other (specify: exasol)
The output of dbt --version
:
All versions above 0.14.x
The operating system you’re using:
ubuntu 18.04.1
The output of python --version
:
Python 3.6.9
Additional context
N/A
Issue Analytics
- State:
- Created 3 years ago
- Comments:8 (5 by maintainers)
Top Results From Across the Web
Remove double underscore in epheremal model generated ...
The quick, Exasol-friendly answer here is to remove the leading double underscore. I can think of at least one thing that would break, ......
Read more >Should I move my CTEs into separate models? - dbt Discourse
Your CTE becomes a separate model, which you can then test (including ephemeral models!). This is often useful for testing uniqueness.
Read more >How do EPHEMERAL dbt models work? - YouTube
Let's talk about one of the more unique materializations within dbt - ephemeral ! In this video we will walk through how to...
Read more >Untitled
And so dbt fundamentally works by materializing these data models in your ... the idea of cascading the Delete, and recreate a views...
Read more >sqlfluff 0.5.4 - PyPI
Handle disabled dbt models. ... Fix false positive in L045 when CTE used in WHERE clause (#944) ... Fix bug in L036 with...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
I think the ideal solution to this is actually that dbt support a method or methods on the adapter for inserting CTEs that includes the question of “what prefix to use” as a sort of sub-problem. I think that involves some compiler changes so it accesses adapters, which is fine.
My reasoning here is that some databases don’t support CTEs and only support subqueries, and while I don’t imagine that behavior ever being a first-class citizen in dbt, it would be nice to give them that mechanism.
edit: Phenomenal timing @jtcohen6
I think opening a new PR would be reasonable, yeah. We might want to have a method that does the reverse: take a CTE name and return the name it came from as well, or None if it didn’t. Then systems like dbt-utils could use that to give useful error messages like this.
I don’t think we need to update dbt-utils immediately, but yes fixing it would be a good idea once this is in dbt-core. If a single plugin temporarily reports the wrong model name in a dbt-utils error message when a user incorrectly uses an ephemeral model, I think that is enough of an edge case to be ok!