state:modified picks up unchanged models if they have round brackets in their configs
See original GitHub issueDescribe the bug
When using defer / state to run changed models. A lot of models that are definitely unchanged are selected and run.
dbt run -m state:modified+ --defer --state ./
I’ve already read these and don’t believe this is covered there:
- https://github.com/fishtown-analytics/dbt/issues/2641
- https://next.docs.getdbt.com/reference/node-selection/state-comparison-caveats/#tests
After comparing production manifest.json
to the one I’m generating locally I noticed that the only difference appears to be lists in config
of those models sometimes have different order. This problems only seems to occur when round brackets are used instead of square brackets.
Steps To Reproduce
When defining config as below, state:modified selects unchanged models.
{{
config(
unique_key = 'my_key',
materialized = 'table',
cluster_by = ('my_cluster_id_1', 'my_cluster_id_2' )
partition_by = {'field': 'partition_id', 'data_type': 'timestamp'}
)
}}
This problem seems to disappear once square brackets are used.
{{
config(
unique_key = 'my_key',
materialized = 'table',
cluster_by = ['my_cluster_id_1', 'my_cluster_id_2' ]
partition_by = {'field': 'partition_id', 'data_type': 'timestamp'}
)
}}
Expected behavior
There should not be a difference in behaviour between square and round brackets. Ideally state:modified
should work correctly with both.
Screenshots and log output
n/a
System information
Which database are you using dbt with?
- postgres
- redshift
- bigquery
- snowflake
- other (specify: ____________)
The output of dbt --version
:
installed version: 0.18.0
latest version: 0.18.1
Your version of dbt is out of date! You can find instructions for upgrading here:
https://docs.getdbt.com/docs/installation
Plugins:
- bigquery: 0.18.0
The operating system you’re using: MacOS / Linux
The output of python --version
:
Python 3.7.0
Additional context
Add any other context about the problem here.
Issue Analytics
- State:
- Created 3 years ago
- Reactions:1
- Comments:5 (5 by maintainers)
Top GitHub Comments
@dmateusp There is a note at the bottom of that docs page that links to all open issues with the tag
state
, so there is a way to find this one, if not by the quickest path. I’m not opposed to adding a note there to encourage arrays/dicts instead of tuples.Frankly, I’d be even more interested in resolving this one, plain and simple. What do you think? Should we adapt the
compare_key
to match tuples and lists that contain otherwise identical content?This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please remove the stale label or comment on the issue, or it will be closed in 7 days.