@task decorator does not handle "templates_exts" argument correctly (files are not rendered)
See original GitHub issueApache Airflow version
2.2.2
What happened
When I use Taskflow, and especially the @task
decorator to create a task, the template files are not correctly rendered : I still have the filename inside templates_dict
instead of the templated file content, even though I supplied a valid templates_exts
argument (at least one which works with PythonOperator
).
I do not encounter the same issue using the old PythonOperator
syntax.
What you expected to happen
I expect that the behavior of the @task
decorator would be the same as the PythonOperator
class, or at least have a workaround explained in the documentation such as other arguments I would need to provide.
How to reproduce
Sample code below :
from airflow.decorators import task
@task(templates_dict={"query": "sql/sample.sql"}, templates_exts=[".sql"])
def aggregate_logs(**kwargs):
logging.info("query: %s", str(kwargs["templates_dict"]["query"]))
Which returns INFO - query: sql/sample.sql
.
However, if I use a PythonOperator
with the old syntax it works :
from airflow.operators.python import PythonOperator
def aggregate_logs(**kwargs):
logging.info("query: %s", str(kwargs["templates_dict"]["query"]))
aggregate_task = PythonOperator(
python_callable=aggregate_logs,
templates_dict={"query": "sql/sample.sql"},
templates_exts=[".sql"],
)
I get the entire templated file inside kwargs["templates_dict"]["query"]
, not just its name.
Operating System
AWS MWAA
Versions of Apache Airflow Providers
Not sure if relevant here since I only use Airflow packages in the example.
Deployment
MWAA
Deployment details
No response
Anything else
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project’s Code of Conduct
Issue Analytics
- State:
- Created 2 years ago
- Comments:7 (4 by maintainers)
I could try fixing this issue if @xingularity and @esthomw do not want to take over.
@potiuk I would love to address that issue, however, I am not sure if I currently have enough time. In that case please go ahead @Bowrna 😃 I will try to help with reviewing and testing