question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Handling broken pipelines in dagit

See original GitHub issue

Right now when something is wrong w/ a pipeline definition, dagit will spew the error to the console and refuse to load.

Instead, it would be great to show the dagit UI and highlight which pipeline in the repository is broken

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:10 (7 by maintainers)

github_iconTop GitHub Comments

2reactions
tinkerwarecommented, Oct 3, 2019

I ran across an issue that seems related. Going over the Inputs tutorial at https://dagster.readthedocs.io/en/stable/sections/learn/tutorial/inputs.html I ran the test pipeline first with an untyped solid, and then changed the same solid to use type signatures. The runs view refused to load with a GraphQL error: GraphQL error] Pipeline "hello_inputs_pipeline" config errors: Error 1: Type failure at path "root:solids:add_hello_to_word:inputs:word:value" on type "String". Value at path root:solids:add_hello_to_word:inputs:word:value is not valid. Expected "String".

The backtrace I see in the console is:

An error occurred while resolving field LogMessageConnection.nodes
Traceback (most recent call last):
  File "/Users/cagatay/.pyenv/versions/3.7.4/envs/dagster-3.7/lib/python3.7/site-packages/graphql/execution/executor.py", line 450, in resolve_or_error
    return executor.execute(resolve_fn, source, info, **args)
  File "/Users/cagatay/.pyenv/versions/3.7.4/envs/dagster-3.7/lib/python3.7/site-packages/graphql/execution/executors/sync.py", line 16, in execute
    return fn(*args, **kwargs)
  File "/Users/cagatay/.pyenv/versions/3.7.4/envs/dagster-3.7/lib/python3.7/site-packages/dagster_graphql/schema/runs.py", line 184, in resolve_nodes
    RunConfig(mode=self._pipeline_run.mode),
  File "/Users/cagatay/.pyenv/versions/3.7.4/envs/dagster-3.7/lib/python3.7/site-packages/dagster/core/execution/api.py", line 70, in create_execution_plan
    environment_config = create_environment_config(pipeline, environment_dict, run_config)
  File "/Users/cagatay/.pyenv/versions/3.7.4/envs/dagster-3.7/lib/python3.7/site-packages/dagster/core/execution/context_creation_pipeline.py", line 50, in create_environment_config
    raise DagsterInvalidConfigError(pipeline, result.errors, environment_dict)
dagster.core.errors.DagsterInvalidConfigError: Pipeline "hello_inputs_pipeline" config errors:
    Error 1: Type failure at path "root:solids:add_hello_to_word:inputs:word:value" on type "String". Value at path root:solids:add_hello_to_word:inputs:word:value is not valid. Expected "String".

This is somewhat annoying, since making type changes to your solids that are incompatible with previous runs in your history will blow up Dagit, until you manually excise the problematic runs from history (haven’t tried this yet, so not sure if it’s possible). Is this the same problem this issue is addressing and should create a new one?

0reactions
hellendagcommented, Oct 16, 2020

Closing in favor of #2718.

Read more comments on GitHub >

github_iconTop Results From Across the Web

A Dagster Crash Course
They're usually tasked with “cleaning data”, fixing broken pipelines in the middle of the night, and are often on-call for questions and ...
Read more >
How to avoid running the rest of a dagster pipeline under ...
To achieve this result, I raise an error when the data meets the invalid condition so the pipeline stops and the rest of...
Read more >
D5034 [dagit] Gracefully handle repo location failures on start
Load workspaces that have no repos, broken repos, or only working repos. Verify that the error appears on Dagit startup for broken repos,...
Read more >
Debugging Broken Data Pipelines with Airflow - YouTube
Session presented by Francisco Alberini at Airflow Summit 2022“Why is my data missing?” “Why didn't my Airflow job run?
Read more >
Deploying Dagster to AWS - Medium
Dagit can also be used to launch pipeline runs and check their results, as well as all ... an invalid input can break...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found