Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Kafka Docker - How to increase memory settings

See original GitHub issue

I’m using Confluent Kafka image to run Kafka via docker container on my machine. the kafka broker is often hang and unable to be connected when I did run some tests. It doesn’t back to work normally even I killed the container and re-inited it or restarted Docker, etc… I had to use Reset to Factory default , then init the docker container again to make it back to work normally.

It takes my time. I dig into investigating the problem and figured out it may be caused by the memory configuration in the Java process running Kafka

java -Xmx1G -Xms1G -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+ExplicitGCInvokesConcurrent -Djava.awt.headless=true -Xloggc:/var/log/kafka/kafkaServer-gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=100M -Dkafka.logs.dir=/var/log/kafka -Dlog4j.configuration=file:/etc/kafka/ -cp /usr/bin/../share/java/kafka/*:/usr/bin/../share/java/confluent-support-metrics/*:/usr/share/java/confluent-support-metrics/* /etc/kafka/

It seems 1Gb memory isn’t enough and made the kafka in the container hang. I’m looking for a way to increase that configuration to be higher. Anyone knows which ENV var or configuration helps to do that?

Issue Analytics

  • State:open
  • Created 4 years ago
  • Reactions:2
  • Comments:6

github_iconTop GitHub Comments

ethicalmohitcommented, Oct 17, 2019

Should work, there’s an example here that also logs JVM events out to a directory

@cricket007 This works. KAFKA_HEAP_OPTS=-Xmx512M -Xms512M without the double quote. Thanks.

Read more comments on GitHub >

github_iconTop Results From Across the Web

JVM Heap Space error · Issue #74 · wurstmeister/kafka-docker
Hi there, upon starting the system with docker-compose up I get the below output. Anything I can do to get sufficient memory for...
Read more >
Docker Configuration Parameters for Confluent Platform
The KAFKA_ADVERTISED_LISTENERS variable is set to localhost:29092 . It makes Kafka accessible from outside of the container by advertising its location on the ......
Read more >
Regarding memory consumption in Kafka - Stack Overflow
We are running kafka as docker container on kubernetes. Memory limit of 4GiB is configured for Kafka broker POD. With some large load...
Read more >
Kafka Connect S3: Out of memory: Java heap space
Try changing the flush size, increasing the JVM's memory, or adding more Kafka Connect workers so that each worker is running on a...
Read more >
zookeeper - Official Image | Docker Hub
One way to change it is mounting your config file as a volume: ... Another example use case for the JVMFLAGS is setting...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found