question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[Documentation] Section inconsistency between airflow.cfg and environment variable names

See original GitHub issue

Apache Airflow version: 1.10.11

Kubernetes version (if you are using kubernetes) (use kubectl version):

Environment:

  • Cloud provider or hardware configuration: All
  • OS (e.g. from /etc/os-release): All
  • Kernel (e.g. uname -a): All
  • Install tools: All
  • Others:

What happened:

I’m trying to set up Google OAuth authentication for the web UI.

I found the document google-authentication provides how to configure in airflow.cfg as follows, but I’m trying to configure using environment variables.

[webserver]
authenticate = True
auth_backend = airflow.contrib.auth.backends.google_auth

[google]
client_id = google_client_id
client_secret = google_client_secret
oauth_callback_route = /oauth2callback
domain = example1.com,example2.com

I referred to Configuration reference, but I found the following 2 issues.

  • Although “auth_backend” is listed under the section “webserver” in google-authentication, the environment variable name is “AIRFLOW__API__AUTH_BACKEND” in Configuration reference . Is this section inconsistency (“WEBSERVER” or “API”) between airflow.cfg and environment variable name expected?

  • 4 keys (“client_id”, “Client_secret”, “oauth_callback_route”, “domain”) are listed under the section “google” in google-authentication, but I could not find “google” section in Configuration reference.

What you expected to happen:

  • All the available environment variable keys are correctly listed in Configuration reference.
  • The document clarifies if a key can be configured in airflow.cfg but cannot be configured as an environment variable.

How to reproduce it:

Anything else we need to know:

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:13 (8 by maintainers)

github_iconTop GitHub Comments

1reaction
alexbeggcommented, Aug 15, 2020

@Minyus I just want to clarify some things. The same Security documentation page explains both the auth_backend under [api] (which as of 1.10.11 defaults to airflow.api.auth.backend.deny_all) and the auth_backend under [webserver] (which does not seem to be included in the default config, I assume because by default authenticate is False).

The fact that both have the same key name can be confusing, but I am sure that both settings do different things. Setting your API auth_backend won’t also set your webserver auth_backend, and vice-versa.

Also, not describing auth_backend under “webserver” on the Configuration reference page or even the default configuration is confusing things.

0reactions
mik-lajcommented, Apr 3, 2021

Documentation for Airflow 2.0 includes documentation for all environmental variables: http://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html http://airflow.apache.org/docs/apache-airflow/stable/cli-and-env-variables-ref.html#environment-variables I don’t think we can improve anything. The documentation for Airflow 1.10 does not clearly describe the differences between RBAC and non-RBAC interfaces, but we are unlikely to fix that as this release only gets bug fixes.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Setting Configuration Options - Apache Airflow
The first time you run Airflow, it will create a file called airflow.cfg in your $AIRFLOW_HOME ... or by creating a corresponding environment...
Read more >
Configuration Reference — Airflow Documentation
This page contains the list of all the available Airflow configurations that you can set in airflow.cfg file or using environment variables.
Read more >
Setting Configuration Options - Apache Airflow
This file contains Airflow's configuration and you can edit it to change any of the settings. You can also set options with environment...
Read more >
Configuration Reference - Apache Airflow
This config controls when your DAGs are updated in the Webserver ... Full class name of secrets backend to enable (will precede env...
Read more >
Develop your Astro project | Astronomer Documentation
Learn how to add Airflow dependencies and customize an Astro project to meet the unique requirements of your organization.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found