question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Allow Pools to be set via ENV

See original GitHub issue

Description

As a system/airflow admin - I prefer being able to see all the various configurations of my instance, and being able to version and audit them.

Pools are a very important subsystem in Airflow - however they are currently only configurable via the UI and CLI.

I would like to be able to configure Pools via ENV variables, as I am able to with configuration settings, connections, and variables.

Something like AIRFLOW_POOL_DEFAULT_SIZE = 128

Use case/motivation

No response

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Reactions:4
  • Comments:13 (11 by maintainers)

github_iconTop GitHub Comments

1reaction
fritz-astronomercommented, Oct 13, 2021

Yea @thcidale0808 - I imagine that would fit the needs of the individuals I’ve talked to with this. The ENV does it’s best to ADD or UPDATE pools. Users will be made aware that pools (like connections and variables) can also come from many sources, and if the env disappears, then it’s on the user to remove the pool (which also doesn’t feel like the worst “failure state” to me?)

1reaction
BasPHcommented, Oct 6, 2021

Yes; I always liked this hot reload functionality of Prometheus: https://prometheus.io/docs/prometheus/latest/configuration/configuration/. +1 for rewriting the configuration, it has other nasty problems e.g. https://github.com/apache/airflow/issues/8255

Read more comments on GitHub >

github_iconTop Results From Across the Web

How to add an Airflow Pool via environment variable?
Not sure if pools can be set via environment variables, but you can easily create them programmatically using create_pool(.
Read more >
Environment variables - AWS Amplify Hosting
Set environment variables ... Sign in to the AWS Management Console and open the Amplify console . ... In the Manage variables section,...
Read more >
Use multiple environments in ASP.NET Core | Microsoft Learn
To set the ASPNETCORE_ENVIRONMENT environment variable for an app running in an isolated Application Pool (supported on IIS 10.0 or later), see ...
Read more >
Allow Jetty's thread pool to be configured via the environment
Hello, I'm wondering, if it is possible to have some basic configuration for embedded Jetty in application.properties.
Read more >
Managing .env files and setting variables - Prisma
The following examples illustrate how to set the environment variable (for the current user) using both Command Prompt ( cmd.exe ) and PowerShell,...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found