[Question] kafka connect worker accessing problem with name

Hello; I need to your help again. I’ve just installed Kafka & KafkaConnect via Strimzi but KafkaConnect worker cannot be accessed via rest 8083 neither with name nor ip. If I set rest.advertised.host.name in kc config, operator says:

WARN AbstractConfiguration:129 - Configuration option "rest.advertised.host.name" is forbidden and will be ignored

The problem is described below:

devadmin@vdi-mk2-ubn:~$ kubectl get pods -n kafka
NAME                                                    READY   STATUS    RESTARTS   AGE
custom-kafka-connect-cluster-connect-6595dd78fd-55r6z   1/1     Running   0          7m26s
kafka-beytepe-entity-operator-697fd977fd-7bwfn          3/3     Running   0          2d
kafka-beytepe-kafka-0                                   2/2     Running   1          2d
kafka-beytepe-kafka-1                                   2/2     Running   2          2d
kafka-beytepe-kafka-2                                   2/2     Running   0          2d
kafka-beytepe-kafka-exporter-7cb9697f7c-rx64w           1/1     Running   1          2d
kafka-beytepe-zookeeper-0                               1/1     Running   1          2d
kafka-beytepe-zookeeper-1                               1/1     Running   0          2d
kafka-beytepe-zookeeper-2                               1/1     Running   0          2d
prometheus-prometheus-0                                 3/3     Running   1          2d
strimzi-cluster-operator-54565f8c56-bgmdw               1/1     Running   0          2d
tmp-shell                                               1/1     Running   0          39m
devadmin@vdi-mk2-ubn:~$ kubectl get svc -n kafka
NAME                                       TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                               AGE
custom-kafka-connect-cluster-connect-api   ClusterIP   10.233.20.11    <none>        8083/TCP,9404/TCP                     7m42s
kafka-beytepe-kafka-bootstrap              ClusterIP   10.233.11.125   <none>        9091/TCP,9092/TCP,9093/TCP,9404/TCP   2d
kafka-beytepe-kafka-brokers                ClusterIP   None            <none>        9091/TCP,9092/TCP,9093/TCP            2d
kafka-beytepe-kafka-exporter               ClusterIP   10.233.19.150   <none>        9404/TCP                              2d
kafka-beytepe-zookeeper-client             ClusterIP   10.233.0.16     <none>        9404/TCP,2181/TCP                     2d
kafka-beytepe-zookeeper-nodes              ClusterIP   None            <none>        2181/TCP,2888/TCP,3888/TCP            2d
prometheus-operated                        ClusterIP   None            <none>        9090/TCP                              2d

devadmin@vdi-mk2-ubn:~$ kubectl exec -i kafka-beytepe-kafka-0 -n kafka -- curl -X GET http://custom-kafka-connect-cluster-connect-api:8083/connectors | jq
Defaulting container name to kafka.
Use 'kubectl describe pod/kafka-beytepe-kafka-0 -n kafka' to see all of the containers in this pod.
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:02:10 --:--:--     0curl: (7) Failed connect to custom-kafka-connect-cluster-connect-api:8083; Connection timed out
command terminated with exit code 7

devadmin@vdi-mk2-ubn:~$ kubectl exec -it custom-kafka-connect-cluster-connect-6595dd78fd-55r6z -n kafka -- bash
[kafka@custom-kafka-connect-cluster-connect-6595dd78fd-55r6z kafka]$ curl -X GET http://custom-kafka-connect-cluster-connect-api:8083/connectors
["mssql-files-connector"]
[kafka@custom-kafka-connect-cluster-connect-6595dd78fd-55r6z kafka]$ exit
exit

devadmin@vdi-mk2-ubn:~$ kubectl exec -it kafka-beytepe-kafka-0 -n kafka -- bash
Defaulting container name to kafka.
Use 'kubectl describe pod/kafka-beytepe-kafka-0 -n kafka' to see all of the containers in this pod.
[kafka@kafka-beytepe-kafka-0 kafka]$ curl -X GET http://custom-kafka-connect-cluster-connect-api:8083/connectors
curl: (7) Failed connect to custom-kafka-connect-cluster-connect-api:8083; Connection timed out
[kafka@kafka-beytepe-kafka-0 kafka]$ curl -X GET http://10.233.20.11:8083/connectors
curl: (7) Failed connect to 10.233.20.11:8083; Connection timed out
[kafka@kafka-beytepe-kafka-0 kafka]$ exit
exit
command terminated with exit code 127

devadmin@vdi-mk2-ubn:~$ kubectl run tmp-shell -n kafka --rm -i --tty --image nicolaka/netshoot -- /bin/bash
If you don't see a command prompt, try pressing enter.
ba[C-5.0# C[Cg ^Cstom-kafka-connect-cluster-connect-apii
bash-5.0# ping custom-kafka-connect-cluster-connect-api
PING custom-kafka-connect-cluster-connect-api.kafka.svc.cluster.local (10.233.20.11) 56(84) bytes of data.
64 bytes from custom-kafka-connect-cluster-connect-api.kafka.svc.cluster.local (10.233.20.11): icmp_seq=1 ttl=64 time=0.048 ms
64 bytes from custom-kafka-connect-cluster-connect-api.kafka.svc.cluster.local (10.233.20.11): icmp_seq=2 ttl=64 time=0.093 ms
64 bytes from custom-kafka-connect-cluster-connect-api.kafka.svc.cluster.local (10.233.20.11): icmp_seq=3 ttl=64 time=0.087 ms
^C
--- custom-kafka-connect-cluster-connect-api.kafka.svc.cluster.local ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2018ms
rtt min/avg/max/mdev = 0.048/0.076/0.093/0.019 ms
bash-5.0# telnet custom-kafka-connect-cluster-connect-api 8083

telnet: can't connect to remote host (10.233.20.11): Operation timed out
bash-5.0#
bash-5.0#

I am not able to find what I’m doing wrongly, could you please advise?

[scholzj]

Thsi is the restriction:

    To Port: 8083/TCP
    From:
      PodSelector: strimzi.io/cluster=custom-kafka-connect-cluster,strimzi.io/kind=KafkaConnect,strimzi.io/name=custom-kafka-connect-cluster-connect
    From:
      NamespaceSelector: <none>
      PodSelector: strimzi.io/kind=cluster-operator

This say that only pods matching the selector strimzi.io/kind=cluster-operator or strimzi.io/cluster=custom-kafka-connect-cluster,strimzi.io/kind=KafkaConnect,strimzi.io/name=custom-kafka-connect-cluster-connect can connect to port 8083. And that is why you need to create your own network policy which would also allow your pods which need the access.

[tirelibirefe]

It worked! Thank you very much @scholzj you helped me a lot as usual! I’m very appreciated.