KafkaContainer and GenericContainer sometimes fail on Apple Silicon M1Pro

KafkaContainer and GenericContainer sometimes stuck at the random place at the container startup or even do not produce any logs.

Example of the log for kafka when stuck (nothing happens after that):

[2022-02-06 15:19:40,079] INFO ZooKeeper audit is disabled. (org.apache.zookeeper.audit.ZKAuditProvider)

===> Running preflight checks ... 

===> Launching ... 

===> Launching kafka ... 

Sometimes it produces an error. Example for GenericContainer used in tests example:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x000000401e193863, pid=8, tid=11
#
# JRE version: OpenJDK Runtime Environment Corretto-11.0.10.9.1 (11.0.10+9) (build 11.0.10+9-LTS)
# Java VM: OpenJDK 64-Bit Server VM Corretto-11.0.10.9.1 (11.0.10+9-LTS, mixed mode, tiered, compressed oops, g1 gc, linux-amd64)
# Problematic frame:
# J 207 c1 java.util.Objects.requireNonNull(Ljava/lang/Object;Ljava/lang/String;)Ljava/lang/Object; java.base@11.0.10 (15 bytes) @ 0x000000401e193863 [0x000000401e193860+0x0000000000000003]
#
# No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# //hs_err_pid8.log
Compiled method (c1)    1829  207       3       java.util.Objects::requireNonNull (15 bytes)
 total in heap  [0x000000401e193690,0x000000401e193ae0] = 1104
 relocation     [0x000000401e193808,0x000000401e193848] = 64
 main code      [0x000000401e193860,0x000000401e193a00] = 416
 stub code      [0x000000401e193a00,0x000000401e193a40] = 64
 metadata       [0x000000401e193a40,0x000000401e193a48] = 8
 scopes data    [0x000000401e193a48,0x000000401e193a78] = 48
 scopes pcs     [0x000000401e193a78,0x000000401e193ad8] = 96
 dependencies   [0x000000401e193ad8,0x000000401e193ae0] = 8
Could not load hsdis-amd64.so; library not loadable; PrintAssembly is disabled
#
# If you would like to submit a bug report, please visit:
#   https://github.com/corretto/corretto-11/issues/
#
qemu: uncaught target signal 6 (Aborted) - core dumped
./entrypoint.sh: line 1:     8 Aborted                 java -Dspring.profiles.active=$PROFILE $JVM_OPTS -jar /home/app/kafka-manager-1.0-SNAPSHOT.jar

To reproduce an issue run any test at com.testcontainers_issue.Test https://github.com/zhukovheorhii/testcontainers_issue.git

When I am running images via docker-compose without using testcontainers, no issues.

PC: MacBook Pro (16-inch, 2021) Apple M1 Pro

JDK Installed on PC Details: Azul Zulu 11.0.14+9 for ARM You can try JDK 11 from any vendor. It’s also reproducible for Liberica JDK

[thomasschoeftner]

just bumped into this too. When scanning docker-hub, I noticed that (as of today) there is a single release of the Confluent Platform container images (Kafka, Zookeeper, etc.) for arm64. - It’s version/tag 7.1.0-1-ubi8. I just tested and verified that those images for Kafka and Zookeeper are functional on current M1 macs.

One issue remains: The validator inside Kafka test-containers only allows version tags consisting of numbers and dots: !version.matches("[0-9]+(\\.[0-9]+)*")). Therefore, you need to pull the images (docker pull confluentinc/cp-kafka:7.1.0-1-ubi8) and re-tag them using a simpler version tag (docker tag confluentinc/cp-kafka:7.1.0-1-ubi8 confluentinc/cp-kafka:7.1.0).

After that confluentinc/cp-kafka:7.1.0 are running fine on arm64 macs.

[bsideup]

As seen in the error messages, Kafka Docker image itself fails on M1 due to it being x86 image emulated on ARM with QEMU.

There is unfortunately nothing Testcontainers can do, as it would fail without Testcontainers as well. Please report the crash to Confluent and let’s hope they provide ARM image soon.