question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

“check” snapshot strategy fails if columns need to be quoted

See original GitHub issue

Describe the bug

If you use the “check” snapshot strategy (default) on data which contains columns with special characters in it and do one initial run and then a subsequent insert run. The second run will fail due to a syntax error (see “Screenshots and log output below”).

Steps To Reproduce

{% snapshot fake_snapshot2 %}
{{
    config(
      target_database='analytics',
      target_schema='dbt_claire',
      unique_key='id',
      strategy='check',
      check_cols='all'
    )
}}
select
    1 as id,
    1 as "unnamed: 0",
    current_timestamp() as created_at
{% endsnapshot %}

(stolen from slack discussion: https://getdbt.slack.com/archives/CBSQTAPLG/p1608759471264100?thread_ts=1608758278.258300&cid=CBSQTAPLG)

Expected behavior

I expect the snapshot to not fail on columns containing special characters. Special characters should work as long as you enclose it with ".

Screenshots and log output

2020-12-23 21:00:59.335996 (MainThread):   syntax error at or near ":"
2020-12-23 21:00:59.339215 (MainThread):   LINE 81: ..._data._dbt_id is null) or snapshotted_data.Unnamed: 0 != sou...

System information

Which database are you using dbt with?

  • postgres
  • redshift
  • bigquery
  • snowflake
  • other (specify: ____________)

The output of dbt --version:

installed version: 0.18.1
   latest version: 0.18.1

Up to date!

Plugins:
  - postgres: 0.18.1
  - snowflake: 0.18.1
  - bigquery: 0.18.1
  - redshift: 0.18.1

The operating system you’re using: MacOS Catalina Version 10.15.7 with Docker for Mac 3.0.3

The output of python --version: Python 3.8.6

cc @clrcrl

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:7 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
sou-joshicommented, Apr 6, 2021

I would like to pick this.

1reaction
clrcrlcommented, Dec 23, 2020

I’m pretty sure it’s this line here (or the related function — we don’t seem to be quoting the column names) https://github.com/fishtown-analytics/dbt/blob/dev/kiyoshi-kuromiya/core/dbt/include/global_project/macros/materializations/snapshot/strategies.sql#L167

Read more comments on GitHub >

github_iconTop Results From Across the Web

Snapshots | dbt Developer Hub
Snapshot "strategies" define how dbt knows if a row has changed. There are two strategies built-in to dbt — timestamp and check ....
Read more >
Check snapshot values - OSIsoft Documentation
Check snapshot values ... Use the piartool utility to retrieve a record dump of the snapshot for a point. Procedure. At a command...
Read more >
VMware Snapshot: Best Practices & Examples
Once connected, go to the vSnapshot tab to check all of your snapshots information, including the size and age. Control the Number of...
Read more >
Best practices for persistent disk snapshots - Google Cloud
When you request a snapshot, check the status of the operation by calling the globalOperations.get method. The following table shows the relationship ...
Read more >
snapshot too old error - Ask TOM
you can check your v$ undo stats view to see if extents are expired "prematurely". you want your undo retention to be longer...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found