Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Run multiple containers in `KubernetesPodOperator`

See original GitHub issue

Description

The airflow.contrib.operators.kubernetes_pod_operator.KubernetesPodOperator does not have the possibility to start mutliple containers in the same pod.

Use case / motivation

Useful if one need to start sidecars or external services in the same pod. I am not sure about the API here, should it be possible to provide a list of images and cmds?

Related Issues

Issue Analytics

State:
Created 3 years ago
Reactions:1
Comments:5 (2 by maintainers)

Top GitHub Comments

5reactions

everglory99commented, Mar 12, 2021

Is this feature available in Airflow 2.0? I add a sidecar container (fluent-bit) in pod_template_file used in KubernetesPodOperator. However, the POD is always in running mode even the base container is finished successfully. I think this is because the sidecar container is still running. Is there a way to fix this issue? Thanks!

0reactions

WongyuChoicommented, Sep 20, 2022

@everglory99 Maybe Pod Mutation Hook can help?

Top Results From Across the Web

Use the KubernetesPodOperator | Cloud Composer

A common pattern when using the KubernetesPodOperator and the GKEStartPodOperator is to execute a shell script as the container entrypoint to group together ......

KubernetesPodOperator — apache-airflow-providers-cncf ...

This will create a sidecar container that runs alongside the Pod. The Pod must write the XCom value into this location at the...

Use the KubernetesPodOperator | Astronomer Documentation

The KubernetesPodOperator (KPO) runs a Docker image in a dedicated Kubernetes Pod. By abstracting calls to the Kubernetes API, the KubernetesPodOperator lets ...

Airflow kubernetes executor: Run 2 jobs on the same pod

I'm using Airflow with kubernetes executor and the KubernetesPodOperator . I have two jobs: A ...

Multi-Container Pods in Kubernetes

A Pod is is the smallest deployable unit that can be deployed and managed by Kubernetes. In other words, if you need to...