Add a --continue-after-error option

Describe the feature

A new flag to attach to a specific step(s) of a run whose failure should not block the rest of the job from completing. Consider this job in our project, which separates fragile from non-fragile tests - if a normally robust test starts failing, I’m much more concerned compared to the fragile ones which tend to self-heal once the pipelines catch up.

dbt run
dbt test --exclude tag:fragile
dbt test -m tag:fragile

If a robust test fails, the fragile tests don’t get a chance to be tested at all.

Describe alternatives you’ve considered

The current workaround is to make multiple jobs. This doesn’t work with the exposure metadata tile beta, which needs all steps to be in a single dbt Cloud job:

dbt source snapshot-freshness
dbt test -m +exposure:*

If a single source is stale, all tests will be skipped. The workaround to that would be to make a separate job for each exposure, something like:

dbt source snapshot-freshness --select +exposure:my_first_exposure
dbt test -m +exposure:my_first_exposure

but that would get intractable after more than a few exposures.

Additional context

In my heart, I think of this as the --soldier-on flag, but something like --continue-after-error is a more practical name

Who will this benefit?

• People using the new metadata tiles
• People running separate jobs who could flatten them down to a single job again, which simplifies scheduling to avoid race conditions (currently I schedule the second test step like an hour after the first job, to make completely sure that everything I’m testing is actually finished building)

Are you interested in contributing this feature?

Interested, but unlikely to make a move on it anytime soon

[joellabes]

@mrogove you’re welcome to re-open issues! Closed doesn’t mean “we’ll never do this”, it just helps to prevent issues whose moment has passed from sticking around forever.

In this case though, it’s not actually a change that can be made in dbt core, so it makes more sense for you to submit a feature request to the dbt Cloud support team using the chat bubble inside the app

[Crimsonabyss]

hi, just following up this issue. has anyone raised the feature request yet? if not I am happy to do it.