“check” snapshot strategy fails if columns need to be quoted
See original GitHub issueDescribe the bug
If you use the “check” snapshot strategy (default) on data which contains columns with special characters in it and do one initial run and then a subsequent insert run. The second run will fail due to a syntax error (see “Screenshots and log output below”).
Steps To Reproduce
{% snapshot fake_snapshot2 %}
{{
config(
target_database='analytics',
target_schema='dbt_claire',
unique_key='id',
strategy='check',
check_cols='all'
)
}}
select
1 as id,
1 as "unnamed: 0",
current_timestamp() as created_at
{% endsnapshot %}
(stolen from slack discussion: https://getdbt.slack.com/archives/CBSQTAPLG/p1608759471264100?thread_ts=1608758278.258300&cid=CBSQTAPLG)
Expected behavior
I expect the snapshot to not fail on columns containing special characters. Special characters should work as long as you enclose it with "
.
Screenshots and log output
2020-12-23 21:00:59.335996 (MainThread): syntax error at or near ":"
2020-12-23 21:00:59.339215 (MainThread): LINE 81: ..._data._dbt_id is null) or snapshotted_data.Unnamed: 0 != sou...
System information
Which database are you using dbt with?
- postgres
- redshift
- bigquery
- snowflake
- other (specify: ____________)
The output of dbt --version
:
installed version: 0.18.1
latest version: 0.18.1
Up to date!
Plugins:
- postgres: 0.18.1
- snowflake: 0.18.1
- bigquery: 0.18.1
- redshift: 0.18.1
The operating system you’re using: MacOS Catalina Version 10.15.7 with Docker for Mac 3.0.3
The output of python --version
: Python 3.8.6
cc @clrcrl
Issue Analytics
- State:
- Created 3 years ago
- Comments:7 (4 by maintainers)
Top Results From Across the Web
Snapshots | dbt Developer Hub
Snapshot "strategies" define how dbt knows if a row has changed. There are two strategies built-in to dbt — timestamp and check ....
Read more >Check snapshot values - OSIsoft Documentation
Check snapshot values ... Use the piartool utility to retrieve a record dump of the snapshot for a point. Procedure. At a command...
Read more >VMware Snapshot: Best Practices & Examples
Once connected, go to the vSnapshot tab to check all of your snapshots information, including the size and age. Control the Number of...
Read more >Best practices for persistent disk snapshots - Google Cloud
When you request a snapshot, check the status of the operation by calling the globalOperations.get method. The following table shows the relationship ...
Read more >snapshot too old error - Ask TOM
you can check your v$ undo stats view to see if extents are expired "prematurely". you want your undo retention to be longer...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
I would like to pick this.
I’m pretty sure it’s this line here (or the related function — we don’t seem to be quoting the column names) https://github.com/fishtown-analytics/dbt/blob/dev/kiyoshi-kuromiya/core/dbt/include/global_project/macros/materializations/snapshot/strategies.sql#L167