Using .sh for BashOperator
See original GitHub issueBackground: I have a simple script in the worker to be run, something like /usr/src/app/script.sh
In here It’s stated that BashOperator can be used for running script.
while as in tutorial / the actual logic, it is always treated as template file path, being lookup & TemplateNotFound
error from jinja will be resulted.
so - is it supposed to run bash script in the target container or it’s always required to use a template as wrapper?
Not sure if it is consistent to turn on the template feature for files only if a separated argument bash_template_file
is used?
e.g.
t1 = BashOperator(
task_id='script1',
bash_template_file='/usr/src/app/script.sh',
dag=dag)
Issue Analytics
- State:
- Created 8 years ago
- Reactions:1
- Comments:5 (5 by maintainers)
Top Results From Across the Web
BashOperator — Airflow Documentation
Use the BashOperator to execute commands in a Bash shell. airflow/example_dags/example_bash_operator.py[source]. run_this = BashOperator ...
Read more >How to run bash script file in Airflow - Stack Overflow
From the tutorial this is OK: t2 = BashOperator( task_id='sleep', bash_command='sleep 5', retries=3, dag=dag). But you're passing a multi-line command to it
Read more >Using the BashOperator | Astronomer Documentation
The BashOperator is part of core Airflow and can be used to execute a single bash command, a set of bash commands or...
Read more >Apache Airflow | How to use the BashOperator - Marc Lamberti
First, a folder named with the current execution date will be created in the folder dags of Airflow. Next, the bash script command.sh...
Read more >Airflow BashOperator Demystified | The Best Guide 2022 - Learn
The Airflow BashOperator allows you to specify any given Shell command or script and add it to an Airflow workflow. This can be...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
I agree, though I can’t break compatibility at this point. I agree that we should clarify the BashOperator parameters, other people have been confused using it.
One path of solution is to add that folder to your template_searchpath as you construct your DAG. https://github.com/airbnb/airflow/blob/master/airflow/models.py#L1530
Closing, feel free to reopen if you have follow up questions