Configuration parsing error when reusing an input in a composite solid.
See original GitHub issueIf I have understood composite solids correctly, then I would expect to be able to run the following example where I reuse the arg
to calculate_result
in two of the inner solids
. However, I get an error:
dagster.core.errors.DagsterInvalidConfigError: Pipeline “my_pipeline” config errors: Error 1: Missing required field “calculate_result” at path root:solids Available Fields: “[‘calculate_result’, ‘noop’]”.
from dagster import execute_pipeline, pipeline, lambda_solid, composite_solid
@lambda_solid
def noop(arg):
return arg
@lambda_solid
def add_one(arg):
return arg + 1
@lambda_solid
def adder(first, second):
return first + second
@composite_solid
def calculate_result(arg):
left = add_one.alias('add_one_left')(arg)
right = add_one.alias('add_one_right')(arg)
return adder(left, right)
@pipeline
def my_pipeline():
return calculate_result(noop())
if __name__ == '__main__':
execute_pipeline(
my_pipeline,
environment_dict={
'solids': {
'noop': {'inputs': {'arg': {'value': 1}}},
# 'calculate_result': {'solids': {'add_one_left': {'inputs': {'arg': {'value': 10}}}}},
}
}
)
If I uncomment the line in the environment_dict
to fill the value for one of the solids inside, then the pipeline will run.
Alternatively, if I change calculate_sum
and my_pipeline
so that I pass the same argument twice to calculate_sum
then the pipeline will run:
@composite_solid
def calculate_result(arg1, arg2):
left = add_one.alias('add_one_left')(arg1)
right = add_one.alias('add_one_right')(arg2)
return adder(left, right)
@pipeline
def my_pipeline():
a = noop()
return calculate_result(a, a)
Issue Analytics
- State:
- Created 4 years ago
- Comments:5 (3 by maintainers)
Top Results From Across the Web
JSON Parser Error at Offset - TechDocs
This article provides information for Error: JSON Parser Error at Offset.
Read more >Cannot setup WireGuard VPN - Server Fault
Line unrecognized: `Address=10.0.0.1/24,fd86:ea04:1115::1/64' Configuration parsing error. What you can also check:.
Read more >Reply To: “Reusing” Fluent Simulation Settings
Just to be clear, I am not wondering about how to make sure only the inlet condition is changed but rather, how I...
Read more >KNIME Components Guide
In this guide, we explain how to create components and metanodes, how to create composite views and configuration dialogs for components, and how...
Read more >Bison 3.8.1 - GNU.org
The fact that the parser state is reusable even after an error simplifies reuse. For example, a calculator application which parses each input...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
https://dagster.phacility.com/D584
Thanks, and yes, this is another real-world use case 😄