Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Run webserver without installing dag dependencies

See original GitHub issue

Description

In order to have the webserver easy to maintain and scale it is nice if we can ignore dag dependencies (ex: python libraries) on the webserver and only install them on the worker. currently webserver throwing an error {dagbag.py:259} ERROR - Failed to import

Use case / motivation

I want to have the webserver totally independent of what needs to be done! from my perspective only workers need to have all the dependencies installed, also the scheduler could ignore them!

Issue Analytics

State:
Created 3 years ago
Comments:5 (4 by maintainers)

Top GitHub Comments

2reactions

ashbcommented, Nov 24, 2020

2.0.0 does this already, as the Webserver now relies on Dag Serialization and reads things from the DB only.

Easiest issue fix ever – cos it’s already done 😁

0reactions

ashbcommented, Nov 24, 2020

From our testing, yes. It’s not yet released “GA”, but it’s getting very close! If you have time to test the beta releases (latest is beta3) that would be helpful!

Top Results From Across the Web

Run Apache Airflow DAG without Apache Airflow

You won't get the benefit of operators or dependencies, but you probably can script your way around it. And if you can't, just...

DAGs — Airflow Documentation

The Dag Dependencies view Menu -> Browse -> DAG Dependencies helps visualize dependencies between DAGs. The dependencies are calculated by the scheduler during ......

Running Airflow Locally: 3 Easy Steps - Learn - Hevo Data

Step 1: Creating a Local Project · Step 2: Installing the Dependencies · Step 3: Running Airflow Locally.

Run Apache Airflow on Windows 10 without Docker

Try to follow these steps. If you get stuck, try to resolve the error by installing missing dependencies, restart terminal or carefully check...

Need Help Installing Plugin · Issue #2 · ms32035/airflow-dag ...

When I run airflow via docker-compose, dag-dependency menu doesn't show up under Brose. Any help would be appreciated.