question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Using .sh for BashOperator

See original GitHub issue

Background: I have a simple script in the worker to be run, something like /usr/src/app/script.sh

In here It’s stated that BashOperator can be used for running script.

while as in tutorial / the actual logic, it is always treated as template file path, being lookup & TemplateNotFound error from jinja will be resulted.

so - is it supposed to run bash script in the target container or it’s always required to use a template as wrapper?

Not sure if it is consistent to turn on the template feature for files only if a separated argument bash_template_file is used?

e.g.

t1 = BashOperator(
    task_id='script1',
    bash_template_file='/usr/src/app/script.sh',
    dag=dag)

Issue Analytics

  • State:closed
  • Created 8 years ago
  • Reactions:1
  • Comments:5 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
mistercrunchcommented, Aug 11, 2015

I agree, though I can’t break compatibility at this point. I agree that we should clarify the BashOperator parameters, other people have been confused using it.

One path of solution is to add that folder to your template_searchpath as you construct your DAG. https://github.com/airbnb/airflow/blob/master/airflow/models.py#L1530

dag = DAG(dag_id='mydag', template_searchpath="/usr/src/app/")

t1 = BashOperator(
    task_id='script1',
    bash_template_file='script.sh',
    dag=dag)
0reactions
mistercrunchcommented, Aug 12, 2015

Closing, feel free to reopen if you have follow up questions

Read more comments on GitHub >

github_iconTop Results From Across the Web

BashOperator — Airflow Documentation
Use the BashOperator to execute commands in a Bash shell. airflow/example_dags/example_bash_operator.py[source]. run_this = BashOperator ...
Read more >
How to run bash script file in Airflow - Stack Overflow
From the tutorial this is OK: t2 = BashOperator( task_id='sleep', bash_command='sleep 5', retries=3, dag=dag). But you're passing a multi-line command to it
Read more >
Using the BashOperator | Astronomer Documentation
The BashOperator is part of core Airflow and can be used to execute a single bash command, a set of bash commands or...
Read more >
Apache Airflow | How to use the BashOperator - Marc Lamberti
First, a folder named with the current execution date will be created in the folder dags of Airflow. Next, the bash script command.sh...
Read more >
Airflow BashOperator Demystified | The Best Guide 2022 - Learn
The Airflow BashOperator allows you to specify any given Shell command or script and add it to an Airflow workflow. This can be...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found