Airflow Helm charts - better integration with Hashicorp Vault

Hello, I have an Airflow 2.1 deployment on kubernetes which is integrated with Hashicorp Vault as secure backend. I found out that the integration will be fully completed only when we get rid of two more keys which I am obligated to apply as secrets:

• gitSync ssh file
• Fernet key The easiest way would be if they could be loaded the same way the hashicorp vault provider client is loading its token from a file. For example i have the ssh key and the fernet key in vault as secrets which are injected by the Vault agent as pod annotations:

  vault.hashicorp.com/agent-inject-secret-airflow: "k8s-secrets/int/airflow"
  vault.hashicorp.com/role: "airflow"
  vault.hashicorp.com/agent-inject-template-airflow: |
    {{- with secret "k8s-secrets/int/airflow" -}}
    {{ .Data.vaulttoken -}}
    {{- end }}
  vault.hashicorp.com/agent-inject-secret-gitssh: "k8s-secrets/int/airflow"
  vault.hashicorp.com/agent-inject-template-gitssh: |
    {{- with secret "k8s-secrets/skube-int/airflow" -}}
    {{ .Data.gitssh -}}
    {{- end }}
  vault.hashicorp.com/agent-inject-secret-fernetkey: "k8s-secrets/int/airflow"
  vault.hashicorp.com/agent-inject-template-fernetkey: |
    {{- with secret "k8s-secrets/skube-int/airflow" -}}
    {{ .Data.fernetkey -}}
    {{- end }}

Then in /vault/secrets/ i have all the three files. The current configuration for GIT_SSH_KEY_FILE for Git-Sync SSH key is (in the _helpers.yaml file):

    {{- if .Values.dags.gitSync.sshKeySecret }}
    - name: GIT_SSH_KEY_FILE
      value: "/etc/git-secret/ssh"
    - name: GIT_SYNC_SSH
      value: "true" 

So there we might have another parameter Values.dags.gitSync.sshKeySecretFile which will point to the injected secret.

The Fernet Key however is a different story as its either written in the config, loaded from an ENV var or loaded from a secret:

  # Hard Coded Airflow Envs
  - name: AIRFLOW__CORE__FERNET_KEY
    valueFrom:
      secretKeyRef:
        name: {{ template "fernet_key_secret" . }}
        key: fernet-key

I am still not sure this could be implemented to read from a file. At first glance there are two solutions:

• Add an additional mechanism in the Airflow core to load the fernet from a file - this requires a lot of work i guess.
• Inject the key in the file as an export command: export AIRFLOW__CORE__FERNET_KEY={{ .Data.fernetkey -}} - this will probably require an additional script in the entrypoint of the container to execute the file

[potiuk]

How about we simply add an option “AIRFLOW__CORE__FERNET_KEY_FILE” in Airflow and read the key from there? this would be like a few line change. AS you might see from #16684, I do not feel particular warmth for the _CMD pattern. While flexible, it’s also dangerous and might lead to promote bad behaviour and maybe just being able to specify file name for the FERNET_KEY is a nice, generic solution tha can solve this particular problem?

[github-actions[bot]]

This issue has been closed because it has not received response from the issue author.