question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Unable to GET/POST from Kakfa Bridge

See original GitHub issue

I have a running instance of strimzi kafka cluster on Azure kubernetes Service . The cluster is enabled with SCRAM-SHA-512 authentication and tls is enabled .

I have deployed a kafka bridge using kafka-bridge template and the deployment for the bridge succeeded . When i try to access the bridge from curl/post http://20.80.69.53:8080/ it gives 200 connection ok { "bridge_version": "0.19.0" } However , when i try http://20.80.69.53:8080/topics , it times out and does not fetch the topic metadata

I’m quite sure its related to the tls configuration set to true and scram-sha-512 mode of authentcation used , the problem is even when i try to pass the user/password and certificate to the bridge , it simply does’nt connect . How can i connect to the bridge with credentials ? Is it even allowed ? I tried with plain mode and no tls and i was able to get topic metadata as well from kafka bridge , that means it works with tls disabled . How can i make it work with tls enabled ? its not working when i pass the credentials and certificate , below is my deployment listener section

kafka:
    config:
      inter.broker.protocol.version: 2.7
      log.message.format.version: 2.7
      offsets.topic.replication.factor: 3
      transaction.state.log.min.isr: 3
      transaction.state.log.replication.factor: 3
    listeners:
    - authentication:
        type: scram-sha-512
      name: tls
      port: 9093
      tls: false
      type: internal
    - authentication:
        type: scram-sha-512
      name: extnl
      port: 9094
      tls: true
      type: loadbalancer

I have an external kafka bootstrap server which exposes external load balancer to access the cluster . All pods are running , below are the services

tushar@Azure:~$ kubectl get svc -n strimzi-operator-aks-cluster
NAME                                        TYPE           CLUSTER-IP     EXTERNAL-IP      PORT(S)                      AGE
bridge-dep-tls-bridge-service               ClusterIP      10.0.159.125   <none>           8080/TCP                     3d18h
bridge-no-tls-bridge-service                ClusterIP      10.0.29.236    <none>           8080/TCP                     3d17h
bridge-rest-service                         LoadBalancer   10.0.56.23     20.80.69.53      8080:32724/TCP               11d
kafka-bridge-bridge-service                 ClusterIP      10.0.179.116   <none>           8080/TCP                     3d18h
kafka-connect-cluster-connect-api           ClusterIP      10.0.236.123   <none>           8083/TCP                     214d
kafka-gsmdp-cluster-kafka-bootstrap         ClusterIP      10.0.139.101   <none>           9091/TCP,9093/TCP            236d
kafka-gsmdp-cluster-kafka-brokers           ClusterIP      None           <none>           9090/TCP,9091/TCP,9093/TCP   236d
kafka-gsmdp-cluster-kafka-extnl-0           LoadBalancer   10.0.247.233   52.182.216.62    9094:32221/TCP               236d
kafka-gsmdp-cluster-kafka-extnl-1           LoadBalancer   10.0.107.4     52.182.216.76    9094:31083/TCP               236d
kafka-gsmdp-cluster-kafka-extnl-2           LoadBalancer   10.0.121.143   52.182.216.167   9094:30207/TCP               236d
kafka-gsmdp-cluster-kafka-extnl-bootstrap   LoadBalancer   10.0.197.248   52.182.216.185   9094:31607/TCP               236d
kafka-gsmdp-cluster-zookeeper-client        ClusterIP      10.0.47.81     <none>           2181/TCP                     236d
kafka-gsmdp-cluster-zookeeper-nodes         ClusterIP      None           <none>           2181/TCP,2888/TCP,3888/TCP   236d
my-connect-cluster-connect-api              ClusterIP      10.0.213.224   <none>           8083/TCP                     235d
new-bridge-service                          LoadBalancer   10.0.150.104   20.84.200.137    8080:32493/TCP               3d18h
notls-bridge-service                        LoadBalancer   10.0.145.253   20.98.184.27     8080:31112/TCP               3d17h
strimzi-aks-bridge-bridge-service           ClusterIP      10.0.253.89    <none>           8080/TCP                     11d

Thanks Best Regards

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:5 (2 by maintainers)

github_iconTop GitHub Comments

1reaction
scholzjcommented, Apr 11, 2022

Well, in the first case, you clearly didn’t configured the security. So I think that is quite clear. In the second, you did that. But you seem to be using a load balancer listener. If you really want to do it, use the Load Balancer IP address or DNs name to configure the connection. You should be able to find it in the status of the Kafka custom resource. But normally, if they run in the same Kubernetes cluster, you would use an internal listener such as the one oyu have on port 9093 and not connect through the load balancer listener since that can cause additional costs and have some performance impact as well.

0reactions
alpacino12commented, Apr 11, 2022

Closing

BEST REGARDS

Read more comments on GitHub >

github_iconTop Results From Across the Web

Creating consumer with empty HTTP POST request doesn't work
When executing an HTTP POST on the /consumers/{groupid} endpoint for creating a new consumer but with an empty body (no JSON at all), ......
Read more >
Strimzi Kafka Bridge Documentation (0.23.1)
Strimzi provides a way to run an Apache Kafka cluster on Kubernetes in various deployment configurations.
Read more >
How to access Kafka bridge - Stack Overflow
The bridge just provides an HTTP interface to Kafka on port 8080 so from within a pod in the Kubernetes cluster you can...
Read more >
Chapter 7. Kafka Bridge Red Hat AMQ 7.6
This chapter provides an overview of the AMQ Streams Kafka Bridge and helps you get started using its REST API to interact with...
Read more >
Connecting over HTTP with Kafka Bridge - IBM Event Streams
Use Kafka Bridge to connect to your Event Streams Kafka cluster over HTTP. ... POST /consumers/{groupid}/instances/{name}/positions/end; GET ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found