Upstream tasks issue with Task.map

Description

I’m encountering a bug that feels related to #2430 that was fixed in master. When an upstream_tasks parameter is provided to a Task.map function and the upstream task errors I get the following:

TypeError: Cannot map over unsubscriptable object of type <class 'prefect.engine.signals.TRIGGERFAIL'>: TRIGGERFAI...

There is no issue as long as the upstream task does not fail.

Expected Behavior

Expected the map function to behave similar to an unmapped function in this case.

Reproduction

from typing import List, Dict
from prefect import Flow
from prefect import Task
from prefect.triggers import any_failed


class TaskA(Task):
    def run(self) -> List[Dict]:
        records = [
            {"data": "dummy"},
            {"data": "dummier"},
            {"data": "dummiest"},
        ]
        return records


class TaskB(Task):
    def run(self) -> None:
        print("doing something unrelated")


class TaskC(Task):
    def run(self, data: List) -> None:
        raise RuntimeError("Uncomment this to fix")
        print(f"Data: {data}")


class TaskD(Task):
    def run(self, data: List) -> None:
        print("Cleaning up failure")


task_a = TaskA()
task_b = TaskB()
task_c = TaskC()
task_d = TaskD(trigger=any_failed)

# Create dummy flow
with Flow("dummy_flow") as flow:
    data = task_a()
    task_b_slug = task_b().slug
    task_c_slug = task_c.map(
        data=data, upstream_tasks=[flow.get_tasks(slug=task_b_slug)]
    ).slug
    task_d.map(data=data, upstream_tasks=[flow.get_tasks(slug=task_c_slug)])

flow.run()

Environment

I tested this in 0.11.5 and master and seems to be the same bahaviour.

[jacques-]

Ok, I think I’m following this now. Using imperative API has solved this for us.

I think we are going to be converting the rest of our flows to imperative API.

Thanks so much for taking the time to walk me through this, it is really appreciated!

P.S. I see now that https://docs.prefect.io/core/concepts/flows.html#apis talks about the more fine grained control you get from the imperative API - I didn’t catch why/when that would be useful when I read those docs originally but def do now!

[joshmeek]

@jacques- Glad it’s solved! Don’t hesitate to reach out with issues here or more questions on Slack if something comes up.