KubernetesPodOperator can't mount sercret as volume

Apache Airflow version: 1.10.12 and 2.0.0b3

Kubernetes version (if you are using kubernetes) (use kubectl version): 1.19.2

Environment:

• Cloud provider or hardware configuration: minikube
• OS (e.g. from /etc/os-release): Ubuntu 20.04
• Kernel (e.g. uname -a): 5.4.0-53

What happened: With the kubernetePodOperator and KubernetesExecutor when I try to mount secret as volume the pod (the one in kubernetesPodOperator) didn’t launch and the task return as failed. The error occur with either V1Volume object or airflow Secret object. Persistent volume claim work perfectly.

What you expected to happen:

airflow launch a worker pod on kubernetes and the worker pod launch a pod on kubernetes

How to reproduce it:

import airflow
from airflow import DAG
from kubernetes.client import models as k8s
from airflow.providers.cncf.kubernetes.operators.kubernetes_pod import KubernetesPodOperator
from airflow.operators.dummy_operator import DummyOperator
from airflow.kubernetes.secret import Secret

default_args = {
    'owner': 'debug',
    'depends_on_past': False,
    'start_date': airflow.utils.dates.days_ago(1),
}

pvc_volume = k8s.V1Volume(
    name='ml-data',
    persistent_volume_claim=k8s.V1PersistentVolumeClaimVolumeSource(claim_name='ml-data')

)

secret_volume = k8s.V1Volume(
    name='deploy-key',
    secret=k8s.V1SecretVolumeSource(default_mode=600, secret_name="dvc-deploy-key")
)

pvc_volume_mount = k8s.V1VolumeMount(
    name='ml-data', mount_path='/data/', sub_path=None, read_only=False
)

secret_volume_mount = k8s.V1VolumeMount(
    name='deploy-key', mount_path='/root/.ssh', sub_path=None, read_only=True
)

secret_file = Secret(deploy_type='volume',
                     deploy_target='/root/.ssh/',
                     secret='dvc-deploy-key')

dag = DAG(
    "testing",
    default_args=default_args,
    description='indexation train pipeline',
    schedule_interval=None
)

start = DummyOperator(
    task_id="start",
    dag=dag
)

t1 = KubernetesPodOperator(
    task_id=f't1',
    name=f't1',
    namespace='airflow',
    image="busybox",
    cmds=["echo", "main"],
    volume_mounts=[pvc_volume_mount],
    volumes=[pvc_volume],
    is_delete_operator_pod=False,
    get_logs=True,
    dag=dag,
)

t10 = KubernetesPodOperator(
    task_id=f't10',
    name=f't10',
    namespace='airflow',
    image="busybox",
    cmds=["echo", "main"],
    secrets=secret_file,
    volume_mounts=[pvc_volume_mount],
    volumes=[pvc_volume],
    is_delete_operator_pod=False,
    get_logs=True,
    dag=dag,
)

t2 = KubernetesPodOperator(
    task_id=f't2',
    name=f't2',
    namespace='airflow',
    image="busybox",
    cmds=["echo", "main"],
    volume_mounts=[secret_volume_mount],
    volumes=[secret_volume],
    is_delete_operator_pod=False,
    get_logs=True,
    dag=dag,
)

t3 = KubernetesPodOperator(
    task_id=f't3',
    name=f't3',
    namespace='airflow',
    image="busybox",
    cmds=["echo", "main"],
    volume_mounts=[pvc_volume_mount, secret_volume_mount],
    volumes=[pvc_volume, secret_volume],
    is_delete_operator_pod=False,
    get_logs=True,
    dag=dag,
)
start >> [t1, t10, t2, t3]

Anything else we need to know: I’m not sure if it’s an airflow issue or a kubernetes-client issue

How often does this problem occur? Once? Every time etc? This problem occur every time Any relevant logs to include? Put them here in side a detail tag:

worker.log

airflow@testingpulldata:/opt/airflow$ airflow tasks run testing pull_data "2020-12-02T15:12:29.757041+00:00" --local --pool default_pool --subdir /opt/airflow/dags/test.py

[2020-12-02 15:30:19,007] {dagbag.py:440} INFO - Filling up the DagBag from /opt/airflow/dags/test.py /home/airflow/.local/lib/python3.6/site-packages/airflow/providers/cncf/kubernetes/backcompat/backwards_compat_converters.py:26 DeprecationWarning: This module is deprecated. Please use kub ernetes.client.models.V1Volume. /home/airflow/.local/lib/python3.6/site-packages/airflow/providers/cncf/kubernetes/backcompat/backwards_compat_converters.py:27 DeprecationWarning: This module is deprecated. Please use kub ernetes.client.models.V1VolumeMount. Running <TaskInstance: testing.pull_data 2020-12-02T15:12:29.757041+00:00 [success]> on host testingpulldata

[VBhojawala]

Hello @scauglog ,

I Just wrote KubernetesPodOperator Guide Here.https://github.com/VBhojawala/airflow/blob/k8s-docs/docs/apache-airflow-providers-cncf-kubernetes/operators.rst#mounting-secrets-as-volume Example Dag in Guide runs fine and mounts Secret volume.

The worker.log given mentions Only DeprecationWarning which does not stop Pod from being launch IMHO.

[sstevens303]

This can be closed

---

From: Jed Cunningham @.> Sent: Wednesday, March 10, 2021 5:18 PM To: apache/airflow @.> Cc: Sharon Stevens @.>; Mention @.> Subject: [External Origin] Re: [apache/airflow] KubernetesPodOperator can’t mount sercret as volume (#12760)

@sstevens303https://github.com/sstevens303 are you still having issues mounting secrets or can this be closed?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/apache/airflow/issues/12760#issuecomment-796219133, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ARY5W5E27S3XZD7R4RTFTNDTC7O27ANCNFSM4UKYLRIA.