question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Node selection for @ on dbt test is incomplete

See original GitHub issue

Describe the bug

Our CI runs dbt run followed by dbt test in a completely clean snowflake warehouse. When using the @ syntax for both, run passes fine, but tests will fail because there are relationship tests on models that aren’t built.

Steps To Reproduce

In as much detail as possible, please provide steps to reproduce the issue. Sample data that triggers the issue, example model code, etc is all very helpful here.

If you view the graph on https://gitlab-data.gitlab.io/analytics/dbt/snowflake/#!/model/model.gitlab_dw?g_v=1&g_i=@zuora you’ll see that the sfdc_executive_business_review model is not built, but on a dbt test this relationship test is run https://gitlab.com/gitlab-data/analytics/blob/master/transform%2Fsnowflake-dbt%2Fmodels%2Fsfdc%2Fbase%2Fschema.yml#L66

Expected behavior

I would either expect that test to be skipped or the relevant model to be built.

Screenshots and log output

These are the logs for an example failed job with @zuora

System information

Which database are you using dbt with?

  • snowflake

The output of dbt --version:

Running with dbt=0.14.2

The operating system you’re using: Mac OS X

The output of python --version: 3.7.4

Additional context

Add any other context about the problem here.

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Reactions:1
  • Comments:7 (6 by maintainers)

github_iconTop GitHub Comments

1reaction
jtcohen6commented, Nov 16, 2020

@ags2121 That behavior is not fixed. I saw your comment over on #2132, I agree that we should open a new issue to discuss.

1reaction
beckjakecommented, Oct 11, 2019

The repro for this case will be very complicated, but I think the issue is when tests get added in to the final graph.

  • We first collect all the matching models in the graph (and properly perform the children’s ancestors calculation there).
  • Then we go find all the tests that descend from those collected values.
  • But we never go back up the graph and find all the ancestors of the tests we just collected, we assume that the original collection pass got all of them.

I’m not sure if we should be excluding the tests whose ancestors aren’t in the graph, or doing another pass and adding all the test ancestors in, but one of those choices is the appropriate solution here.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Node selection for @ on dbt test is incomplete #1827 - GitHub
Our CI runs dbt run followed by dbt test in a completely clean snowflake warehouse. When using the @ syntax for both, run...
Read more >
Syntax overview | dbt Developer Hub
dbt's node selection syntax makes it possible to run only specific resources in a given invocation of dbt. This selection syntax is used...
Read more >
dbt Guide - GitLab
Documenting and testing new data models is a part of the process of creating them. A new dbt model is not complete without...
Read more >
AWS Glue | Programmatic Ponderings
Typical dbt tests check for non-null and unique values, values within an expected numeric range, and values from a known list of strings....
Read more >
Using DBT to Execute ELT Pipelines in Snowflake
sql files contain models and tests. Models are SELECT statements that result in a new (or incrementally updated) table or view in the...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found