Flows fail to take NamedTuple as a parameter
See original GitHub issueDescription
NamedTuples are currently handled the same way as tuples when Prefect builds its context (and probably at other times), which is incorrect since a NamedTuple can’t be initialized like a normal tuple. It needs one argument per defined property instead all of the arguments in a list.
Where the error happens: https://github.com/PrefectHQ/prefect/blob/cc2f4e16227d9f40c6aeace3bae887cc1fdc90f6/src/prefect/utilities/collections.py#L145-L177
Expected Behavior
It should initialize them properly using either the splat *[]
or using the ._make([])
method of the NamedTuple object, or perhaps just not modify them at all (just return them raw on L177).
I don’t know why things have to be transformed into dictionaries so don’t know what the right decision is.
Reproduction
from prefect import Flow, Parameter, task
from collections import namedtuple
@task
def printme(x):
print(x)
with Flow('Test') as flow:
myarg = Parameter('myarg')
printme(myarg)
flow.run(myarg=namedtuple('MyObj', ['x', 'y'])(1, 2))
Issue Analytics
- State:
- Created 4 years ago
- Comments:8 (2 by maintainers)
Top Results From Across the Web
Everything You Need to Know About Python's Namedtuples
Master how to use named tuples in Python by examples. Learn to convert namedtuple to dict, create namedtuple from dict, add optional fields...
Read more >Flow with a huge number of parameters #3852 - GitHub
Current behavior As #3837 mentioned, I have a flow which contains a huge number of parameters, since Parameter is a Task, it takes...
Read more >ICS 32A Fall 2022, Notes and Examples: Namedtuples
Having imported the namedtuple() function, we can call it by passing it two arguments: the name of our new type and a list...
Read more >Function that constructs NamedTuple with function argument ...
The problem I'm trying to solve is to have a function f that: takes as as one of its arugments a Tuple of...
Read more >Source code for dagster._core.storage.pipeline_run
SUCCESS = "SUCCESS" # Runs that have failed to complete. ... @whitelist_for_serdes class PipelineRunStatsSnapshot( NamedTuple( "_PipelineRunStatsSnapshot", ...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
That’s what I suspected. In that case, I think the solution I proposed might be the right one. Nonetheless, lmk if you want help with a PR on this once you’ve made up your mind on how you want to fix it (whether that happens now or later).
I could probably use
DotDict
as a work-around in the meantime, yes, but long-term I still believe this should be fixed.Since you still allow non-serialize-able objects to be passed in (e.g random classes) I’m confused as to why you’re even trying to break up/deep-serialize lists/tuples/sets to begin with? Are you trying to achieve a best-effort as-much-as-possible serialization? In that case, it seems like the solution is perhaps to do something like
if type(obj) in ('list', 'tuple', 'set')
instead, since that way you just won’t screw with custom types that inherit from those (I’m guessing NamedTuple isn’t the only one that would be problematic?). Or is there some Prefect Cloud criteria here that I’m missing?