task_run_name's kwargs referring to task inputs shadowed by Prefect's context instead
See original GitHub issueDescription
Recently, a task_run_name
parameter was added to the task constructors, and documentation was written on how to dynamically set task Run Names dynamically based on inputs.
At runtime, Prefect automatically populates kwargs
with raw inputs and also prefect context:
https://github.com/PrefectHQ/prefect/blob/1bbefb71fa15cfc6bf6b6d766fe7591dafaa4717/src/prefect/engine/cloud/task_runner.py#L346
This leads to a hard-to-catch bug if you happen to use a task input variable under the same name as one of the keys in the Prefect Context. Common variable names like config
or parameters
or even map_index
, if used in task input, would be shadowed by the correspondent context
value and lead to crashes or unexpected task run names.
Expected Behavior
An exception could be raised if a duplicated name is detected… But most likely the existing behaviour just needs to be clearly explained in the documentation.
Reproduction
from prefect import task, Flow
@task
def generate_configs():
return [{"value": 1}, {"value": 2}, {"value": 3}]
@task(task_run_name=lambda **kwargs: f"{kwargs['config']['value']}")
def parse_configs(config):
print(config["value"])
with Flow("Task Run Names Bug") as flow:
parse_configs.map(generate_configs())
Environment
Reproducible in any environment with Prefect version 0.13.10+
, any Cloud deployment method.
Issue Analytics
- State:
- Created 3 years ago
- Comments:5 (1 by maintainers)
IMO a library shouldn’t issue a warning (note you want
warning.warn(...)
, notraise
here) that can’t be disabled by the user making some change to their code. The only way for a user to fix this is to change the argument names to their function, which seems drastic given that most users probably won’t need to accessconfig
when generating a task name (most users only want the input arguments to the task). Forcing the user to change their argument names just to silence a warning that doesn’t affect them seems like bad UX to me.I think we should stick with the simple solution, and maybe add a small note in the
Task
docstring for thetask_run_name
argument mentioning how conflicts are handled.PR #3622 closes the issue.