question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Race condition when starting container causes IndexOutOfBoundsException when binding ports

See original GitHub issue

Description

An intermittent issue can be seen when starting containers. It seems to be a race condition that is exacerbated when running maven with debug logging -X. Without the debug logging containers fail to start with

[ERROR] DOCKER> Error occurred during container startup, shutting down...

The full stack trace of the exception is:

Caused by: java.lang.IndexOutOfBoundsException: Index 0 out of bounds for length 0
    at jdk.internal.util.Preconditions.outOfBounds (Preconditions.java:64)
    at jdk.internal.util.Preconditions.outOfBoundsCheckIndex (Preconditions.java:70)
    at jdk.internal.util.Preconditions.checkIndex (Preconditions.java:248)
    at java.util.Objects.checkIndex (Objects.java:372)
    at java.util.ArrayList.get (ArrayList.java:459)
    at com.google.gson.JsonArray.get (JsonArray.java:194)
    at io.fabric8.maven.docker.model.ContainerDetails.createPortBindings (ContainerDetails.java:200)
    at io.fabric8.maven.docker.model.ContainerDetails.getPortBindings (ContainerDetails.java:139)
    at io.fabric8.maven.docker.service.RunService.updateMappedPortsAndAddresses (RunService.java:461)
    at io.fabric8.maven.docker.service.RunService.createAndStartContainer (RunService.java:156)
    at io.fabric8.maven.docker.service.helper.StartContainerExecutor.startContainer (StartContainerExecutor.java:47)
    at io.fabric8.maven.docker.StartMojo.lambda$startImage$0 (StartMojo.java:299)

When adding some logging to the maven plugin I can see that during the the io.fabric8.maven.docker.model.ContainerDetails.createPortBindings method call the ports will usually have some mappings like this:

Ports:{"5432/tcp":[{"HostIp":"0.0.0.0","HostPort":"54290"}],}

However sometimes the array of mapped ports will be empty, e.g.

Ports:{"5432/tcp":[]}

This causes the exception as the array is expected to not be empty and the first element retrieved.

I believe the call to the api in io.fabric8.maven.docker.service.RunService.updateMappedPortsAndAddresses is sometimes returning this empty data for the port if called too early.

The solution I would propose would be to retry the API call and the createPortBindings activity if an empty array is encountered.

Info

  • docker-maven-plugin version : 0.34.1
  • Maven version (mvn -v) :
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: /usr/local/Cellar/maven/3.6.3_1/libexec
Java version: 11.0.10, vendor: AdoptOpenJDK, runtime: /Library/Java/JavaVirtualMachines/adoptopenjdk-11.jdk/Contents/Home
Default locale: en_GB, platform encoding: UTF-8
OS name: "mac os x", version: "10.14.6", arch: "x86_64", family: "mac"
  • Docker version : 3.2.2
  • If it’s a bug, how to reproduce : The issue is intermittent so impossible to consistently reproduce. It appeared to be more easily reproducible when running mvn verify with -X flag.

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:9 (4 by maintainers)

github_iconTop GitHub Comments

2reactions
git9999999commented, Mar 29, 2021

Good Morning 😃 now i did the test right, and now it works. 😃 thanks for fixing the plugin. cheers g

1reaction
ewencluleycommented, Mar 26, 2021

So im not seeing that log message and the line indicated by the stack trace io.fabric8.maven.docker.model.ContainerDetails.createPortBindings (ContainerDetails.java:200) is (in my code)

} else {

which cant be throwing that exception. So it really looks like the updated code is not being used.

Read more comments on GitHub >

github_iconTop Results From Across the Web

A reason for unexplained connection timeouts on Kubernetes ...
The Linux Kernel has a known race condition when doing source network address translation (SNAT) that can lead to SYN packets being dropped....
Read more >
docker-compose run will not cause race condition?
It means the Django App cannot connect with MySQL server because the MySQL server is not ready yet. However, when I was using...
Read more >
Vert.x Core Manual
If both verticles would bind to the same port, you would receive a socket exception. Fortunately, vert.x is handling this case for you....
Read more >
Release Notes - Diffusion Documentation - Push Technology
A race condition led to a possible NullPointerException in SessionPropertiesDispatcherImpl being reported on the server. This has now been resolved.
Read more >
Databricks runtime maintenance updates - Azure
[SPARK-39847] Fix race condition in RocksDBLoader. ... Fixed a regression that caused DBFS FUSE to fail to start when cluster environment variable ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found