Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Re-execution from a step with downstream dynamic steps produces an error

See original GitHub issue

https://dagster.slack.com/archives/C01U954MEER/p1640105707446800

If you try to re-execute the following subsets, you’ll get an error:

+hello_dynamic
generate_subtasks*
generate_subtasks+
generate_subtasks, hello_dynamic

The error:

dagster.core.errors.DagsterExecutionStepNotFoundError: Can not build subset plan from unknown steps: hello_dynamic[subtask_1], hello_dynamic[subtask_2], hello_dynamic[subtask_3], hello_dynamic[subtask_4], hello_dynamic[subtask_5]
  File "/Users/owen/src/dagster/python_modules/dagster/dagster/grpc/impl.py", line 354, in get_external_execution_plan_snapshot
    create_execution_plan(
  File "/Users/owen/src/dagster/python_modules/dagster/dagster/core/execution/api.py", line 745, in create_execution_plan
    return ExecutionPlan.build(
  File "/Users/owen/src/dagster/python_modules/dagster/dagster/core/execution/plan/plan.py", line 973, in build
    return plan_builder.build()
  File "/Users/owen/src/dagster/python_modules/dagster/dagster/core/execution/plan/plan.py", line 227, in build
    plan = plan.build_subset_plan(
  File "/Users/owen/src/dagster/python_modules/dagster/dagster/core/execution/plan/plan.py", line 764, in build_subset_plan
    raise DagsterExecutionStepNotFoundError(

These subsets re-execute without issue:

hello_dynamic
hello_dynamic*
hello_dynamic[subtask_1]

Issue Analytics

State:
Created 2 years ago
Reactions:10
Comments:8 (4 by maintainers)

Top GitHub Comments

3reactions

yuhancommented, Apr 7, 2022

Hey @johnsonpeng5 thanks for flagging. I’ll prioritize this issue in our upcoming bug bash (early next month)

1reaction

yuhancommented, May 24, 2022

Hi @kervel I merged a fix early today and it will go out in the 0.14.17 release on this Thursday. In your case, you’d be able to select buildEptOp[?] with its upstream step in Dagit and re-execute them all together. Please feel free to comment on this issue if you’re still facing any issue so I can reopen it.

Top Results From Across the Web

Execution - Dagster Docs

Executes a single-threaded, in-process run which materializes provided assets. ... Reexecution options for python-based execution in Dagster. Parameters:.

Using instruction result locality and re-execution to mitigate ...

In this paper, a method to mitigate silent data corruptions (SDCs) is proposed. This paper, first, shows and characterizes instruction result locality based ......

Troubleshooting Apply - Oracle Help Center

When an apply process moves a transaction to the error queue, you can examine the transaction to analyze the feasibility reexecuting the transaction ......

How do I dynamically trigger downstream builds in jenkins?

The Join Plugin then gives an error saying a CauseAction is required, so I used manager.hudson.queue.schedule(job, 0 causeAction) . Thanks for giving me...

dagster Changelog - pyup.io

Fixed a bug where if resource initialization failed for a dynamic op, ... them to be loaded from Snowflake into Pandas DataFrames in...