question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Unable to start airflow webserver in the background in docker container.

See original GitHub issue

Apache Airflow version: 2.1.2

Environment: continuumio/anaconda3 docker image

  • OS (e.g. from /etc/os-release): Debian 10 Buster
  • Kernel: Linux 242e786962c8 4.19.76-linuxkit #1 SMP Thu Oct 17 19:31:58 UTC 2019 x86_64 GNU/Linux
  • Install tools: Docker
  • Others: Host OS is windows 10

What happened:

I have installed airflow inside the docker container. And to run the webserver, I used the command

airflow webserver -D

to run in the background, however, the application is still running in the foreground and I have to open another terminal to run the airflow scheduler. The same issue is faced when trying to run the airflow scheduler in the background.

What you expected to happen: To start the airflow web server in the background.

How to reproduce it:

  1. Use the Dockerfile attached to run the image.
  2. Start a terminal in the container.
  3. Run airflow webserver -D

The Dockerfile:

FROM continuumio/anaconda3

WORKDIR /airflow

ENV AIRFLOW_HOME /airflow
ENV AIRFLOW_VERSION 2.1.2
ENV PYTHON_VERSION 3.8
ENV CONSTRAINT_URL "https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-${PYTHON_VERSION}.txt"

RUN apt-get update -y
RUN apt-get install -y --no-install-recommends \
        freetds-bin \
        ldap-utils \
        libffi-dev \
        libsasl2-2 \
        libsasl2-modules \
        libssl1.1 \
        locales  \
        lsb-release \
        sasl2-bin \
        sqlite3 \
        unixodbc
RUN pip install "apache-airflow==${AIRFLOW_VERSION}" --constraint "${CONSTRAINT_URL}"
RUN airflow db init
RUN airflow users create \
	    --username admin \
	    --firstname admin \
	    --lastname admin \
	    --password admin \
	    --role Admin \
	    --email admin@admin.org

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:8 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
potiukcommented, Jul 23, 2021

All works perfectly for me

1reaction
potiukcommented, Jul 23, 2021
docker run -it apache/airflow:latest bash

airflow@1abe97747ca7:/opt/airflow$ airflow webserver -D
[2021-07-23 10:11:16,569] {cli_action_loggers.py:105} WARNING - Failed to log action with (sqlite3.OperationalError) no such table: log
[SQL: INSERT INTO log (dttm, dag_id, task_id, event, execution_date, owner, extra) VALUES (?, ?, ?, ?, ?, ?, ?)]
[parameters: ('2021-07-23 10:11:16.565715', None, None, 'cli_webserver', None, 'airflow', '{"host_name": "1abe97747ca7", "full_command": "[\'/home/airflow/.local/bin/airflow\', \'webserver\', \'-D\']"}')]
(Background on this error at: http://sqlalche.me/e/13/e3q8)
  ____________       _____________
 ____    |__( )_________  __/__  /________      __
____  /| |_  /__  ___/_  /_ __  /_  __ \_ | /| / /
___  ___ |  / _  /   _  __/ _  / / /_/ /_ |/ |/ /
 _/_/  |_/_/  /_/    /_/    /_/  \____/____/|__/
[2021-07-23 10:11:16,590] {dagbag.py:496} INFO - Filling up the DagBag from /dev/null
[2021-07-23 10:11:16,690] {manager.py:784} WARNING - No user yet created, use flask fab command to do it.
Running the Gunicorn Server with:
Workers: 4 sync
Host: 0.0.0.0:8080
Timeout: 120
Logfiles: - -
Access Logformat: 
=================================================================            
airflow@1abe97747ca7:/opt/airflow$ 

Read more comments on GitHub >

github_iconTop Results From Across the Web

Not able to start airflow webserver in background in docker
I am trying to run airflow on an anaconda3 docker image. Everything is working fine, except I am not able to run airflow...
Read more >
Airflow Docker - Webserver not starting · Issue #15927 - GitHub
I'm running Airflow via Docker using the official docker-compose image, and whilst everything seems to work fine, the webserver gets stuck ...
Read more >
Setting up Apache Airflow 2 with Docker - Towards Data Science
Open your browser and go to 192.168.1.200:8080 . Enter your admin credential and Voilà! What's next? Now you are able to upload your...
Read more >
Running Airflow in Docker
airflow -webserver - The webserver is available at http://localhost:8080 . airflow-worker - The worker that executes the tasks given by the scheduler. airflow- ......
Read more >
How To Run Airflow on Windows (with Docker)
First of all, we create three services: a metadb, a scheduler and a webserver. Architecturally, Airflow stores its state in a database (the ......
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found