question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

state:modified picks up unchanged models if they have round brackets in their configs

See original GitHub issue

Describe the bug

When using defer / state to run changed models. A lot of models that are definitely unchanged are selected and run. dbt run -m state:modified+ --defer --state ./

I’ve already read these and don’t believe this is covered there:

After comparing production manifest.json to the one I’m generating locally I noticed that the only difference appears to be lists in config of those models sometimes have different order. This problems only seems to occur when round brackets are used instead of square brackets.

Steps To Reproduce

When defining config as below, state:modified selects unchanged models.

{{
  config(
    unique_key = 'my_key',
    materialized = 'table',
    cluster_by = ('my_cluster_id_1', 'my_cluster_id_2' )
    partition_by = {'field': 'partition_id', 'data_type': 'timestamp'}
  )
}}

This problem seems to disappear once square brackets are used.

{{
  config(
    unique_key = 'my_key',
    materialized = 'table',
    cluster_by = ['my_cluster_id_1', 'my_cluster_id_2' ]
    partition_by = {'field': 'partition_id', 'data_type': 'timestamp'}
  )
}}

Expected behavior

There should not be a difference in behaviour between square and round brackets. Ideally state:modified should work correctly with both.

Screenshots and log output

n/a

System information

Which database are you using dbt with?

  • postgres
  • redshift
  • bigquery
  • snowflake
  • other (specify: ____________)

The output of dbt --version:

installed version: 0.18.0
   latest version: 0.18.1

Your version of dbt is out of date! You can find instructions for upgrading here:
https://docs.getdbt.com/docs/installation

Plugins:
  - bigquery: 0.18.0

The operating system you’re using: MacOS / Linux

The output of python --version: Python 3.7.0

Additional context

Add any other context about the problem here.

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Reactions:1
  • Comments:5 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
jtcohen6commented, Oct 7, 2021

@dmateusp There is a note at the bottom of that docs page that links to all open issues with the tag state, so there is a way to find this one, if not by the quickest path. I’m not opposed to adding a note there to encourage arrays/dicts instead of tuples.

Frankly, I’d be even more interested in resolving this one, plain and simple. What do you think? Should we adapt the compare_key to match tuples and lists that contain otherwise identical content?

0reactions
github-actions[bot]commented, Apr 6, 2022

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please remove the stale label or comment on the issue, or it will be closed in 7 days.

Read more comments on GitHub >

github_iconTop Results From Across the Web

state:modified picks up unchanged models if they have round ...
When defining config as below, state:modified selects unchanged models. ... This problem seems to disappear once square brackets are used. {{ ...
Read more >
Working with entity states - EF6 - Microsoft Learn
Modified entities are updated in the database and then become Unchanged when SaveChanges returns. Deleted entities are deleted from the database ...
Read more >
Python regular expression to remove only specific square ...
Python regular expression to remove only specific square brackets and question mark in a string by leaving other square brackets unchanged. ...
Read more >
2022 North State Modifieds Rules
Each 0 – 100 lbs. of weights must be secured to the car with either two (2) ½” bolts or four ... Must...
Read more >
OFSAAI User Guide - Oracle Help Center
Applications are built using OFSAAI by assembling business definitions or business metadata starting from data-model to lower grain objects like Dimensions, ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found