[Bug] Kafka pod's cannot be created with custom (non-default) cluster domain

Describe the bug I try to create a Kafka cluster through strimzi-kafka-operator in minikube. When operator tried to connect to zookeeper (on kafka’s pod-creation phase) it’s failed with error:

Failed to connect to Zookeeper
        minikube-kafka-zookeeper-0.minikube-kafka-zookeeper-nodes.infra-kafka.svc:2181.
        Connection was not ready in 300000 ms.

I think that the operator cannot resolve the zookeeper’s service name correctly, because it does not include the cluster domain suffix (in my environment it’s a “yc.minik”, not default “cluster.local”).

I checked these thoughts like this (inside zookeer’s container):

[kafka@minikube-kafka-zookeeper-0 kafka]$ nslookup minikube-kafka-zookeeper-0.minikube-kafka-zookeeper-nodes.infra-kafka.svc
Server:		10.96.0.10
Address:	10.96.0.10#53

** server can't find minikube-kafka-zookeeper-0.minikube-kafka-zookeeper-nodes.infra-kafka.svc: NXDOMAIN

[kafka@minikube-kafka-zookeeper-0 kafka]$ nslookup minikube-kafka-zookeeper-0.minikube-kafka-zookeeper-nodes.infra-kafka.svc.yc.minik
Server:		10.96.0.10
Address:	10.96.0.10#53

Name:	minikube-kafka-zookeeper-0.minikube-kafka-zookeeper-nodes.infra-kafka.svc.yc.minik
Address: 172.17.0.15

To Reproduce Steps to reproduce the behavior:

1. Launch minikube with custom dns-domain suffix, like: minikube start --dns-domain='yc.minik'
2. Install strimzi kafka operator through Helm or other way (but remember to set deployment’s env-variable KUBERNETES_SERVICE_DNS_DOMAIN=‘yc.minik’): helm install kafka-operator -n infra-kafka --set-string kubernetesServiceDnsDomain='yc.minik' strimzi/strimzi-kafka-operator
3. Create something like:

cat << EOF | kubectl create -n infra-kafka -f -
apiVersion: kafka.strimzi.io/v1beta1
kind: Kafka
metadata:
  name: minikube-kafka
  namespace: infra-kafka
spec:
  kafka:
    version: 2.4.0
    replicas: 1
    resources:
      limits:
        cpu: "200m"
        memory: "1Gi"
      requests:
        cpu: "50m"
        memory: "64Mi"
    listeners:
      plain: {}
      tls: {}
    config:
      offsets.topic.replication.factor: 1
      transaction.state.log.replication.factor: 1
      transaction.state.log.min.isr: 1
      log.message.format.version: "2.4"
    storage:
      type: ephemeral
  zookeeper:
    replicas: 1
    resources:
      limits:
        cpu: "200m"
        memory: "1Gi"
      requests:
        cpu: "50m"
        memory: "64Mi"
    storage:
      type: ephemeral
  entityOperator:
    topicOperator: {}
    userOperator: {}
EOF

1. See that no kafka’s pod will created by operator

Expected behavior Kafka cluster created and running successfully.

Environment (please complete the following information):

• Strimzi version: 0.17.0
• Installation method: Helm chart
• Kubernetes cluster: Kubernetes 1.16.8
• Infrastructure: Minikube

YAML files and logs Reports files attached. report-13-04-2020_22-07-46.zip

[Ranger-X]

I also investigated this problem more deeply and agree that this is issue in the minikube.

For anybody who googled this page 😃 I solved the issue by running minikube like this:

minikube start --vm-driver=virtualbox --kubernetes-version=1.16.8 --cpus 4 --disk-size 20g --memory 16384 --dns-domain='yc.minik' --extra-config='kubelet.cluster-domain=yc.minik'

So, an important option is --extra-config='kubelet.cluster-domain=yc.minik'. After this I successfully run Kafka cluster in the Minikube 😃

@scholzj Thank you very much for the right direction about this issue 😃

[scholzj]

Thanks for investigating anf updating the issue. Much appreciated!