question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Flows fail to take NamedTuple as a parameter

See original GitHub issue

Description

NamedTuples are currently handled the same way as tuples when Prefect builds its context (and probably at other times), which is incorrect since a NamedTuple can’t be initialized like a normal tuple. It needs one argument per defined property instead all of the arguments in a list.

Where the error happens: https://github.com/PrefectHQ/prefect/blob/cc2f4e16227d9f40c6aeace3bae887cc1fdc90f6/src/prefect/utilities/collections.py#L145-L177

Expected Behavior

It should initialize them properly using either the splat *[] or using the ._make([]) method of the NamedTuple object, or perhaps just not modify them at all (just return them raw on L177). I don’t know why things have to be transformed into dictionaries so don’t know what the right decision is.

Reproduction

from prefect import Flow, Parameter, task
from collections import namedtuple


@task
def printme(x):
    print(x)

with Flow('Test') as flow:
    myarg = Parameter('myarg')
    printme(myarg)

flow.run(myarg=namedtuple('MyObj', ['x', 'y'])(1, 2))

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:8 (2 by maintainers)

github_iconTop GitHub Comments

2reactions
fgblomqvistcommented, Mar 9, 2020

That’s what I suspected. In that case, I think the solution I proposed might be the right one. Nonetheless, lmk if you want help with a PR on this once you’ve made up your mind on how you want to fix it (whether that happens now or later).

1reaction
fgblomqvistcommented, Mar 9, 2020

I could probably use DotDict as a work-around in the meantime, yes, but long-term I still believe this should be fixed.

Since you still allow non-serialize-able objects to be passed in (e.g random classes) I’m confused as to why you’re even trying to break up/deep-serialize lists/tuples/sets to begin with? Are you trying to achieve a best-effort as-much-as-possible serialization? In that case, it seems like the solution is perhaps to do something like if type(obj) in ('list', 'tuple', 'set') instead, since that way you just won’t screw with custom types that inherit from those (I’m guessing NamedTuple isn’t the only one that would be problematic?). Or is there some Prefect Cloud criteria here that I’m missing?

Read more comments on GitHub >

github_iconTop Results From Across the Web

Everything You Need to Know About Python's Namedtuples
Master how to use named tuples in Python by examples. Learn to convert namedtuple to dict, create namedtuple from dict, add optional fields...
Read more >
Flow with a huge number of parameters #3852 - GitHub
Current behavior As #3837 mentioned, I have a flow which contains a huge number of parameters, since Parameter is a Task, it takes...
Read more >
ICS 32A Fall 2022, Notes and Examples: Namedtuples
Having imported the namedtuple() function, we can call it by passing it two arguments: the name of our new type and a list...
Read more >
Function that constructs NamedTuple with function argument ...
The problem I'm trying to solve is to have a function f that: takes as as one of its arugments a Tuple of...
Read more >
Source code for dagster._core.storage.pipeline_run
SUCCESS = "SUCCESS" # Runs that have failed to complete. ... @whitelist_for_serdes class PipelineRunStatsSnapshot( NamedTuple( "_PipelineRunStatsSnapshot", ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found