Allow Pools to be set via ENV
See original GitHub issueDescription
As a system/airflow admin - I prefer being able to see all the various configurations of my instance, and being able to version and audit them.
Pools are a very important subsystem in Airflow - however they are currently only configurable via the UI and CLI.
I would like to be able to configure Pools via ENV variables, as I am able to with configuration settings, connections, and variables.
Something like AIRFLOW_POOL_DEFAULT_SIZE = 128
Use case/motivation
No response
Related issues
No response
Are you willing to submit a PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project’s Code of Conduct
Issue Analytics
- State:
- Created 2 years ago
- Reactions:4
- Comments:13 (11 by maintainers)
Top Results From Across the Web
How to add an Airflow Pool via environment variable?
Not sure if pools can be set via environment variables, but you can easily create them programmatically using create_pool(.
Read more >Environment variables - AWS Amplify Hosting
Set environment variables ... Sign in to the AWS Management Console and open the Amplify console . ... In the Manage variables section,...
Read more >Use multiple environments in ASP.NET Core | Microsoft Learn
To set the ASPNETCORE_ENVIRONMENT environment variable for an app running in an isolated Application Pool (supported on IIS 10.0 or later), see ...
Read more >Allow Jetty's thread pool to be configured via the environment
Hello, I'm wondering, if it is possible to have some basic configuration for embedded Jetty in application.properties.
Read more >Managing .env files and setting variables - Prisma
The following examples illustrate how to set the environment variable (for the current user) using both Command Prompt ( cmd.exe ) and PowerShell,...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Yea @thcidale0808 - I imagine that would fit the needs of the individuals I’ve talked to with this. The ENV does it’s best to ADD or UPDATE pools. Users will be made aware that pools (like connections and variables) can also come from many sources, and if the env disappears, then it’s on the user to remove the pool (which also doesn’t feel like the worst “failure state” to me?)
Yes; I always liked this hot reload functionality of Prometheus: https://prometheus.io/docs/prometheus/latest/configuration/configuration/. +1 for rewriting the configuration, it has other nasty problems e.g. https://github.com/apache/airflow/issues/8255