Environment variable ordering for refrencing secrets with KubernetesPodOperator

Apache Airflow version: 1.10.9

Kubernetes version (if you are using kubernetes) (use kubectl version): 1.16.6

Environment:

• Cloud provider or hardware configuration:
• OS (e.g. from /etc/os-release): Debian GNU/Linux
• Kernel (e.g. uname -a): Linux 4.19.0-0.bpo.6-amd64 #1 SMP Debian 4.19.67-2+deb10u2~bpo9+1 (2019-11-12) x86_64 Linux
• Install tools:
• Others:

What happened:

When env_vars and secrets is set on a KubernetesPodOperator the resulting pod will always have the secrets after the environment variables. This makes it impossible to use variable refrences from secrets based env vars to regular env vars.

What you expected to happen:

How to reproduce it:

Anything else we need to know:

The fix is fairly simple - change this:

            for k in pod.envs.keys():
                env.append({'name': k, 'value': pod.envs[k]})
            for secret in envs_from_key_secrets:
                KubernetesRequestFactory.add_secret_to_env(env, secret)

To this

            for secret in envs_from_key_secrets:
                KubernetesRequestFactory.add_secret_to_env(env, secret)
            for k in pod.envs.keys():
                env.append({'name': k, 'value': pod.envs[k]})

here https://github.com/apache/airflow/blob/96697180d79bfc90f6964a8e99f9dd441789177c/airflow/contrib/kubernetes/kubernetes_request_factory/kubernetes_request_factory.py#L159

However I am aware that this has been changed in airflow 2 which afaict avoids this issue by specifying allowing the user to specify a list of k8s.V1EnvVars which can be secrets or regular env vars with user specified ordering https://github.com/dimberman/airflow/blob/6018532d21796e04fdf47a77f36ae8308cc928ba/airflow/kubernetes/pod_generator.py#L203

[dimberman]

@Rested It’s out in 1.10.12, though some of the legacy stuff will be fixed in 1.10.13 😃

[eladkal]

Closing since this issue is 1.10 specific (1.10 is EOL)