apply_defaults doesn't run for decorated task

Apache Airflow version: 2.1

Kubernetes version (if you are using kubernetes) (use kubectl version): N/A

Environment:

• Cloud provider or hardware configuration:
• OS (e.g. from /etc/os-release): Ubuntu 18.04.5 LTS
• Kernel (e.g. uname -a): Linux 4.4.0-18362-Microsoft #1049-Microsoft Thu Aug 14 12:01:00 PST 2020 x86_64 x86_64 x86_64 GNU/Linux
• Install tools:
• Others:

What happened: The apply_defaults doesn’t work for decorated tasks after upgrading to Airflow 2.1.

What you expected to happen: Missing task arguments to be filled by values from DAG default_args. Airflow 2.1 expected to apply by default (https://github.com/apache/airflow/pull/15667) but doesn’t work either with or without apply_defaults decorator being used.

How to reproduce it: Sample DAG attached as txt file. task_decorator_test.txt

If run in Airflow 2.0.2, this prints in the logs the conn_id value showing that it was picked up from the DAG default_args.

If run in Airflow 2.1, this causes TypeError for missing required argument in the scheduler. If a default value is given to the argument, (making it optional), the task can run, but the apply_defaults to use the DAG’s default_args value doesn’t run.

Anything else we need to know: issue happens with Airflow 2.1

[junnplus]

@jedcunningham yes, I think it fixed.

[junnplus]

apply_default is moved to metaclass of operator, it not process the signature of the python_callable for DecoratedOperator to fill in op_kwargs.

We can preprocess the padding of default_args before https://github.com/apache/airflow/blob/master/airflow/decorators/base.py#L206 like dag decorator.