Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Connection to Kafka DevServices not possible

See original GitHub issue

Describe the bug

Quarkus cant connect to the auto-configured DevServices Kafka Broker.

Expected behavior

Quarkus should connect seamless

Actual behavior

Quarkus tries to connect, but cant establish connection.

2022-01-10 17:34:27,191 WARN  [org.apa.kaf.cli.NetworkClient] (kafka-producer-network-thread | kafka-producer-source-out) [Producer clientId=kafka-producer-source-out] Connection to node -1 (localhost/127.0.0.1:49156) could not be established. Broker may not be available.
2022-01-10 17:34:27,191 WARN  [org.apa.kaf.cli.NetworkClient] (kafka-producer-network-thread | kafka-producer-source-out) [Producer clientId=kafka-producer-source-out] Bootstrap broker localhost:49156 (id: -1 rack: null) disconnected
2022-01-10 17:34:27,362 WARN  [org.apa.kaf.cli.NetworkClient] (smallrye-kafka-consumer-thread-1) [Consumer clientId=kafka-consumer-source-in, groupId=code-with-quarkus] Connection to node -1 (localhost/127.0.0.1:49156) could not be established. Broker may not be available.
2022-01-10 17:34:27,362 WARN  [org.apa.kaf.cli.NetworkClient] (smallrye-kafka-consumer-thread-1) [Consumer clientId=kafka-consumer-source-in, groupId=code-with-quarkus] Bootstrap broker localhost:49156 (id: -1 rack: null) disconnected
2022-01-10 17:34:28,780 WARN  [org.apa.kaf.cli.NetworkClient] (kafka-producer-network-thread | kafka-producer-uppercase-out) [Producer clientId=kafka-producer-uppercase-out] Connection to node -1 (localhost/127.0.0.1:49156) could not be established. Broker may not be available.
2022-01-10 17:34:28,780 WARN  [org.apa.kaf.cli.NetworkClient] (kafka-producer-network-thread | kafka-producer-uppercase-out) [Producer clientId=kafka-producer-uppercase-out] Bootstrap broker localhost:49156 (id: -1 rack: null) disconnected
2022-01-10 17:34:28,826 WARN  [org.apa.kaf.cli.NetworkClient] (smallrye-kafka-consumer-thread-1) [Consumer clientId=kafka-consumer-source-in, groupId=code-with-quarkus] Connection to node -1 (localhost/127.0.0.1:49156) could not be established. Broker may not be available.
2022-01-10 17:34:28,826 WARN  [org.apa.kaf.cli.NetworkClient] (smallrye-kafka-consumer-thread-1) [Consumer clientId=kafka-consumer-source-in, groupId=code-with-quarkus] Bootstrap broker localhost:49156 (id: -1 rack: null) disconnected
2022-01-10 17:34:28,889 WARN  [org.apa.kaf.cli.NetworkClient] (smallrye-kafka-consumer-thread-0) [Consumer clientId=kafka-consumer-uppercase-in, groupId=code-with-quarkus] Connection to node -1 (localhost/127.0.0.1:49156) could not be established. Broker may not be available.
2022-01-10 17:34:28,889 WARN  [org.apa.kaf.cli.NetworkClient] (smallrye-kafka-consumer-thread-0) [Consumer clientId=kafka-consumer-uppercase-in, groupId=code-with-quarkus] Bootstrap broker localhost:49156 (id: -1 rack: null) disconnected
2022-01-10 17:34:28,919 WARN  [org.apa.kaf.cli.NetworkClient] (kafka-producer-network-thread | kafka-producer-source-out) [Producer clientId=kafka-producer-source-out] Connection to node -1 (localhost/127.0.0.1:49156) could not be established. Broker may not be available.
2022-01-10 17:34:28,920 WARN  [org.apa.kaf.cli.NetworkClient] (kafka-producer-network-thread | kafka-producer-source-out) [Producer clientId=kafka-producer-source-out] Bootstrap broker localhost:49156 (id: -1 rack: null) disconnected

How to Reproduce?

Steps to Reproduce:

  1. Go to https://code.quarkus.io/?e=smallrye-reactive-messaging-kafka&extension-search=origin:platform kafka
  2. Download generated project, extract it
  3. Run mvn quarkus:dev

Output of uname -a or ver

Microsoft Windows [Version 10.0.22000.376]

Output of java -version

openjdk version “17.0.1” 2021-10-19 OpenJDK Runtime Environment Temurin-17.0.1+12 (build 17.0.1+12) OpenJDK 64-Bit Server VM Temurin-17.0.1+12 (build 17.0.1+12, mixed mode, sharing)

GraalVM version (if different from Java)

No response

Quarkus version or git rev

2.6.1.Final

Build tool (ie. output of mvnw --version or gradlew --version)

Apache Maven 3.8.4 (9b656c72d54e5bacbed989b64718c159fe39b537) Maven home: C:\ProgramData\chocolatey\lib\maven\apache-maven-3.8.4 Java version: 17.0.1, vendor: Eclipse Adoptium, runtime: C:\Program Files\Eclipse Adoptium\jdk-17.0.1.12-hotspot Default locale: de_DE, platform encoding: Cp1252 OS name: “windows 10”, version: “10.0”, arch: “amd64”, family: “windows”

Additional information

$ docker --version
Docker version 20.10.11, build dea9396

$ docker ps -a
CONTAINER ID   IMAGE                         COMMAND                  CREATED         STATUS         PORTS                               NAMES
43bb689b4b88   vectorized/redpanda:v21.5.5   "sh -c 'while [ ! -f…"   4 seconds ago   Up 5 seconds   9644/tcp, 0.0.0.0:49157->9092/tcp   distracted_sinoussi
Docker Inspect 
$ docker inspect 43bb689b4b88
[
    {
        "Id": "43bb689b4b88f9a4b5ad56212683b6c9a3a291d73d7c71a14230b1b8bce924e7",
        "Created": "2022-01-10T16:41:03.347847731Z",
        "Path": "sh",
        "Args": [
            "-c",
            "while [ ! -f /var/lib/redpanda/redpanda.sh ]; do sleep 0.1; done; /var/lib/redpanda/redpanda.sh"
        ],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 7817,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2022-01-10T16:41:04.849808178Z",
            "FinishedAt": "0001-01-01T00:00:00Z"
        },
        "Image": "sha256:98ca2a6da0045c0fb3ffde6a98b354a303216e79cbd82c0341a994383aa73fcb",
        "ResolvConfPath": "/var/lib/docker/containers/43bb689b4b88f9a4b5ad56212683b6c9a3a291d73d7c71a14230b1b8bce924e7/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/43bb689b4b88f9a4b5ad56212683b6c9a3a291d73d7c71a14230b1b8bce924e7/hostname",
        "HostsPath": "/var/lib/docker/containers/43bb689b4b88f9a4b5ad56212683b6c9a3a291d73d7c71a14230b1b8bce924e7/hosts",
        "LogPath": "/var/lib/docker/containers/43bb689b4b88f9a4b5ad56212683b6c9a3a291d73d7c71a14230b1b8bce924e7/43bb689b4b88f9a4b5ad56212683b6c9a3a291d73d7c71a14230b1b8bce924e7-json.log",
        "Name": "/distracted_sinoussi",
        "RestartCount": 0,
        "Driver": "overlay2",
        "Platform": "linux",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": [],
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "571288e853ccf2cdab20ad617c290be140a587deb6a8fa2224316a6748cc09f7",
            "PortBindings": {
                "9092/tcp": [
                    {
                        "HostIp": "",
                        "HostPort": "0"
                    }
                ]
            },
            "RestartPolicy": {
                "Name": "",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": [],
            "CapAdd": null,
            "CapDrop": null,
            "CgroupnsMode": "host",
            "Dns": null,
            "DnsOptions": null,
            "DnsSearch": null,
            "ExtraHosts": [],
            "GroupAdd": null,
            "IpcMode": "shareable",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": null,
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": null,
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": null,
            "DeviceCgroupRules": null,
            "DeviceRequests": null,
            "KernelMemory": 0,
            "KernelMemoryTCP": 0,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": null,
            "OomKillDisable": false,
            "PidsLimit": null,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0,
            "MaskedPaths": [
                "/proc/asound",
                "/proc/acpi",
                "/proc/kcore",
                "/proc/keys",
                "/proc/latency_stats",
                "/proc/timer_list",
                "/proc/timer_stats",
                "/proc/sched_debug",
                "/proc/scsi",
                "/sys/firmware"
            ],
            "ReadonlyPaths": [
                "/proc/bus",
                "/proc/fs",
                "/proc/irq",
                "/proc/sys",
                "/proc/sysrq-trigger"
            ]
        },
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/7f42a7a2577b2c9cc2904a42af8506e745c0e5ce488938947413183d06a2fab5-init/diff:/var/lib/docker/overlay2/cb46326572a52dfdb21a8e8222d68c5f9b41b9242485e4d07ceb90d2e4553bfe/diff:/var/lib/docker/overlay2/2c551f41c6000a014134f217fdd60d6eb3119e2aa139f6be396108fad2246ce9/diff:/var/lib/docker/overlay2/f088d0ab6d0188c1e08bcb4f31d7928a707ffabfe73ce2326a67e8aad18157ef/diff:/var/lib/docker/overlay2/45d012e80025c8d2a215d56f16f62d8d6f2c4071f3e369308df98f4adcdc15a0/diff:/var/lib/docker/overlay2/7710b423cdb4c71f4c25753d8759cc141726058b1427979f292295890b59f7e5/diff:/var/lib/docker/overlay2/da3ca8da892bc123e231d2c3896a33a090f875d904b6ae74da24d38a8232bc4f/diff:/var/lib/docker/overlay2/81995f9f90ef2919e51ee0269b93e3ec875b3026c6989b86f3f96cc9a7e22709/diff",
                "MergedDir": "/var/lib/docker/overlay2/7f42a7a2577b2c9cc2904a42af8506e745c0e5ce488938947413183d06a2fab5/merged",
                "UpperDir": "/var/lib/docker/overlay2/7f42a7a2577b2c9cc2904a42af8506e745c0e5ce488938947413183d06a2fab5/diff",
                "WorkDir": "/var/lib/docker/overlay2/7f42a7a2577b2c9cc2904a42af8506e745c0e5ce488938947413183d06a2fab5/work"
            },
            "Name": "overlay2"
        },
        "Mounts": [
            {
                "Type": "volume",
                "Name": "6905004583ec1013b84061d51386a843da6a25278a1acb9176200521327439cc",
                "Source": "/var/lib/docker/volumes/6905004583ec1013b84061d51386a843da6a25278a1acb9176200521327439cc/_data",
                "Destination": "/var/lib/redpanda/data",
                "Driver": "local",
                "Mode": "",
                "RW": true,
                "Propagation": ""
            }
        ],
        "Config": {
            "Hostname": "43bb689b4b88",
            "Domainname": "",
            "User": "redpanda",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "ExposedPorts": {
                "9092/tcp": {},
                "9644/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "DEBIAN_FRONTEND=noninteractive"
            ],
            "Cmd": [
                "-c",
                "while [ ! -f /var/lib/redpanda/redpanda.sh ]; do sleep 0.1; done; /var/lib/redpanda/redpanda.sh"
            ],
            "Image": "docker.io/vectorized/redpanda:v21.5.5",
            "Volumes": {
                "/var/lib/redpanda/data": {}
            },
            "WorkingDir": "",
            "Entrypoint": [
                "sh"
            ],
            "OnBuild": null,
            "Labels": {
                "maintainer": "Vectorized.io",
                "org.testcontainers": "true",
                "org.testcontainers.sessionId": "8c989546-c621-4462-b5f7-89ffdc8c6cb6",
                "quarkus-dev-service-kafka": "kafka"
            }
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "1a90e1562dd85d895f0050baa8ef82bf6692151bdced5e3bcf7b247d8138e156",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {
                "9092/tcp": [
                    {
                        "HostIp": "0.0.0.0",
                        "HostPort": "49157"
                    }
                ],
                "9644/tcp": null
            },
            "SandboxKey": "/var/run/docker/netns/1a90e1562dd8",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "19faf7ba-b3ba-4ad9-bc89-077c3ab06722": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": [
                        "kafka-DfS94",
                        "43bb689b4b88"
                    ],
                    "NetworkID": "571288e853ccf2cdab20ad617c290be140a587deb6a8fa2224316a6748cc09f7",
                    "EndpointID": "2dac3c297eca6a824fff3af380968a0130458c6e29cf392342f6a0e0e60211f0",
                    "Gateway": "172.22.0.1",
                    "IPAddress": "172.22.0.2",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:ac:16:00:02",
                    "DriverOpts": null
                }
            }
        }
    }
]

Starting a Redpanda Container with the following config myself and setting kafka.bootstrap.servers=localhost:9092 works.

version: '3.7'
services:
  redpanda:
    command:
      - redpanda
      - start
      - --smp
      - '1'
      - --reserve-memory
      - 0M
      - --overprovisioned
      - --node-id
      - '0'
      - --kafka-addr
      - PLAINTEXT://0.0.0.0:29092,OUTSIDE://0.0.0.0:9092
      - --advertise-kafka-addr
      - PLAINTEXT://redpanda:29092,OUTSIDE://localhost:9092
    image: docker.vectorized.io/vectorized/redpanda:v21.11.2
    container_name: redpanda
    ports:
      - "9092:9092"
      - "29092:29092"

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:5 (4 by maintainers)

github_iconTop GitHub Comments

2reactions
twobierscommented, Jan 11, 2022

I think the problem was somehow on my end. Don’t know why and what caused it, but as I started my computer today it worked like a charm.

0reactions
cescoffiercommented, Jan 10, 2022

it might be a windows issue. I would need to setup a VM (would have to wait this weekend if @ozangunalp does not beat me on it).

Read more comments on GitHub >

github_iconTop Results From Across the Web

Dev Services for Kafka - Quarkus
If your environment does not support Docker, you will need to start the broker manually, or connect to an already running broker. You...
Read more >
Quarkus Dev Services for Kafka Broker Automatically - YouTube
https://twitter.com/danieloh30https://www.linkedin.com/in/daniel-oh-083818112/#Quarkus # Kafka # DevServices #CloudNative #Kubernetes ...
Read more >
Getting Started to Quarkus Reactive Messaging with Apache ...
Quarkus Dev Services provides you with an automatic way to boot up a Kafka cluster to test your application. Apache Kafka can replace...
Read more >
Why Can't I Connect to Kafka? | Troubleshoot Connectivity
Not unless you want your client to randomly stop working each time you deploy it on a machine that you forget to hack...
Read more >
Quarkus/SmallRye Reactive Messaging/Kafka connect with ...
Adding a server here does not help, it then tries to spread data over the brokers which is not my intention. Is it...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

Rest-Client-Reactive: Allow disabling contextual error messages

Next page

Hibernate ORM codestart doesn't compile out of the box, jdbc extension is needed