[Enhancement] Possibility to configure more containers in a Pod (sidecars, init containers)

Is your feature request related to a problem? Please describe.

We are deploying a Kafka cluster in our company, and we want to monitor it using the Elastic monitoring suite. To do this, we need to add a sidecar to all Kafka and Zookeeper pods. This sidecar will run Metricbeat, and will scrape metrics from Kafka/Zookeeper. Currently, it doesn’t seem possible to configure additional containers for Strimzi pods, when looking at PodTemplate.

Describe the solution you’d like

The solution would be to add a new configuration key in PodTemplate named containers, accepting a list of Container objects. This list would be merged with the containers created by Strimzi (Kafka/Zookeeper + TLS Sidecar).

Describe alternatives you’ve considered

Our workaround for now is to use k8s-sidecar-injector, and configure it to automatically inject a Metricbeats container. Then, we need to annotate the Zookeeper and Kafka pods (this is currently possible with Strimzi) to indicate to k8s-sidecar-injector to inject a container.

This solution is fine, but is cumbersome as it spreads the configuration in different Kubernetes objects.

I can try to contribute to this

If the suggested enhancement seems like a good idea, I’d be glad to try and implement it.

[3schwartz]

Hi @scholzj I’m also find this feature needed since we would like to migrate to Strimzi.

We are using Istio as a service mesh. Currently we need to implement a custom lifecycle hook such that envoy sidecars isn’t killed before Kafka have done a gracefully shutdown, see https://github.com/istio/istio/issues/7136. If it isn’t added, Kafka fails to shutdown gracefully and on restart it would need to startup from a forcefully shutdown. In cluster with a high data volume this can take quite some time. We use a start up probe since we have experienced endless restarts after forcefully shutdowns when only using ready- and liveness probes - and currently Strimzi only support these.

We are customizing the istio sidecar following the documentation from istio, and depends on istio which updates the sidecar configuration to be correct at scheduling. https://istio.io/latest/docs/setup/additional-setup/sidecar-injection/#customizing-injection

When using sidecar injectors, as proposed in this issue, to customize the istio sidecar I end up with error saying that two identical sidecars are added. I expect this is due to the order mutation admission webhook mutate the pod and I can’t control the order of mutation in current sidecar injectors.

Maybe I’m missing something related to the whole Strimzi Kafka and Istio comparability - if so I’m would be happy to learn 😃

[juergenhemelt]

I think it’s feasible. You may have to ensure certain things. Eg. you should check the exposed ports so that they do not conflict with the Kafka ports of the broker. You may also have to check for conflicting volume definitions. Or you prevent volume definitions initially. What else has to be considered?

Btw. regarding your comment on OPA in #5458 : Here is an example of how to use OPA as a sidecar: https://gitlab.com/cloud.yantra.oss/opa-gateway-sidecar It ensures the usage of up to date policies by having a central webserver hosting the policies which can be updated without having to restart the OPA pod. Seems to me like an approach which ensures up to date policies. Combined with caching it could be a perfect solution for balancing performance and timeliness of policies.