Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Can't pull from docker.io explicitly, 500 error or

See original GitHub issue

Description

I have a maven configuration that was working until this weekend, where we’re setting a custom registry to pull from/push to by default in the parent, and then for a couple specific images that inherit from public images, we set a <docker.pull.registry>docker.io</docker.pull.registry> in their pom. Now all these builds fail.

For example, I have a Dockerfile that’s FROM debian:sid. As of the 20th, I get 500 errors when the plugin tries to pull it, both from my Mac and from a Ubuntu bamboo build server in AWS (completely different networks, etc.). The build server tells me “Unable to pull ‘debian:sid’ from registry ‘docker.io’ : Could not reach any registry endpoint” and the Mac tells me “Unable to pull ‘library/debian:sid’ from registry ‘docker.io’ : received unexpected HTTP status: 500 Server Error (Internal Server Error: 500)”.

Mac mvn -X output:

[DEBUG] http-outgoing-0 >> POST /v1.21/images/create?fromImage=docker.io%2Flibrary%2Fdebian&tag=sid HTTP/1.1
[DEBUG] http-outgoing-0 >> Accept: */*
[DEBUG] http-outgoing-0 >> Content-Type: application/json; charset=utf-8
[DEBUG] http-outgoing-0 >> X-Registry-Auth: eyJwYXNzd29yZCI6IiIsImF1dGgiOiIiLCJlbWFpbCI6IiIsInVzZXJuYW1lIjoiIn0=
[DEBUG] http-outgoing-0 >> Content-Length: 0
[DEBUG] http-outgoing-0 >> Host: 127.0.0.1:1
[DEBUG] http-outgoing-0 >> Connection: Keep-Alive
[DEBUG] http-outgoing-0 >> User-Agent: Apache-HttpClient/4.5.2 (Java/1.8.0_112)
[DEBUG] http-outgoing-0 >> Accept-Encoding: gzip,deflate
[DEBUG] http-outgoing-0 >> "POST /v1.21/images/create?fromImage=docker.io%2Flibrary%2Fdebian&tag=sid HTTP/1.1[\r][\n]"
[DEBUG] http-outgoing-0 >> "Accept: */*[\r][\n]"
[DEBUG] http-outgoing-0 >> "Content-Type: application/json; charset=utf-8[\r][\n]"
[DEBUG] http-outgoing-0 >> "X-Registry-Auth: eyJwYXNzd29yZCI6IiIsImF1dGgiOiIiLCJlbWFpbCI6IiIsInVzZXJuYW1lIjoiIn0=[\r][\n]"
[DEBUG] http-outgoing-0 >> "Content-Length: 0[\r][\n]"
[DEBUG] http-outgoing-0 >> "Host: 127.0.0.1:1[\r][\n]"
[DEBUG] http-outgoing-0 >> "Connection: Keep-Alive[\r][\n]"
[DEBUG] http-outgoing-0 >> "User-Agent: Apache-HttpClient/4.5.2 (Java/1.8.0_112)[\r][\n]"
[DEBUG] http-outgoing-0 >> "Accept-Encoding: gzip,deflate[\r][\n]"
[DEBUG] http-outgoing-0 >> "[\r][\n]"
[DEBUG] http-outgoing-0 << "HTTP/1.1 500 Internal Server Error[\r][\n]"
[DEBUG] http-outgoing-0 << "Api-Version: 1.31[\r][\n]"
[DEBUG] http-outgoing-0 << "Content-Length: 50[\r][\n]"
[DEBUG] http-outgoing-0 << "Content-Type: text/plain; charset=utf-8[\r][\n]"
[DEBUG] http-outgoing-0 << "Date: Mon, 21 Aug 2017 19:42:00 GMT[\r][\n]"
[DEBUG] http-outgoing-0 << "Docker-Experimental: true[\r][\n]"
[DEBUG] http-outgoing-0 << "Ostype: linux[\r][\n]"
[DEBUG] http-outgoing-0 << "Server: Docker/17.07.0-ce-rc3 (linux)[\r][\n]"
[DEBUG] http-outgoing-0 << "X-Content-Type-Options: nosniff[\r][\n]"
[DEBUG] http-outgoing-0 << "[\r][\n]"
[DEBUG] http-outgoing-0 << HTTP/1.1 500 Internal Server Error
[DEBUG] http-outgoing-0 << Api-Version: 1.31
[DEBUG] http-outgoing-0 << Content-Length: 50
[DEBUG] http-outgoing-0 << Content-Type: text/plain; charset=utf-8
[DEBUG] http-outgoing-0 << Date: Mon, 21 Aug 2017 19:42:00 GMT
[DEBUG] http-outgoing-0 << Docker-Experimental: true
[DEBUG] http-outgoing-0 << Ostype: linux
[DEBUG] http-outgoing-0 << Server: Docker/17.07.0-ce-rc3 (linux)
[DEBUG] http-outgoing-0 << X-Content-Type-Options: nosniff
[DEBUG] Connection can be kept alive indefinitely
[DEBUG] http-outgoing-0 << "received unexpected HTTP status: 500 Server Error[\n]"
[DEBUG] Connection [id: 0][route: {}->unix://127.0.0.1:1] can be kept alive indefinitely
[DEBUG] Connection released: [id: 0][route: {}->unix://127.0.0.1:1][total kept alive: 1; route allocated: 1 of 100; total allocated: 1 of 20]
[ERROR] DOCKER> Unable to pull 'library/debian:sid' from registry 'docker.io' : received unexpected HTTP status: 500 Server Error (Internal Server Error: 500) [received unexpected HTTP status: 500 Server Error (Internal Server Error: 500)]
[DEBUG] Connection manager is shutting down
[DEBUG] http-outgoing-0: Close connection
[DEBUG] Connection manager shut down
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 7.169 s
[INFO] Finished at: 2017-08-21T14:42:00-05:00
[INFO] Final Memory: 23M/358M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal io.fabric8:docker-maven-plugin:0.20.1:build (default-build) on project otxb-core-base-container: Unable to pull 'library/debian:sid' from registry 'docker.io' : received unexpected HTTP status: 500 Server Error (Internal Server Error: 500) -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal io.fabric8:docker-maven-plugin:0.20.1:build (default-build) on project otxb-core-base-container: Unable to pull 'library/debian:sid' from registry 'docker.io' : received unexpected HTTP status: 500 Server Error (Internal Server Error: 500)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:213)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:154)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:146)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
	at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
	at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:309)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:194)
	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:107)
	at org.apache.maven.cli.MavenCli.execute(MavenCli.java:993)
	at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:345)
	at org.apache.maven.cli.MavenCli.main(MavenCli.java:191)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
	at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
	at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: Unable to pull 'library/debian:sid' from registry 'docker.io' : received unexpected HTTP status: 500 Server Error (Internal Server Error: 500)
	at io.fabric8.maven.docker.AbstractDockerMojo.execute(AbstractDockerMojo.java:226)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
	... 20 more
Caused by: io.fabric8.maven.docker.access.DockerAccessException: Unable to pull 'library/debian:sid' from registry 'docker.io' : received unexpected HTTP status: 500 Server Error (Internal Server Error: 500)
	at io.fabric8.maven.docker.access.hc.DockerAccessWithHcClient.pullImage(DockerAccessWithHcClient.java:358)
	at io.fabric8.maven.docker.service.RegistryService.checkImageWithAutoPull(RegistryService.java:81)
	at io.fabric8.maven.docker.service.BuildService.autoPullBaseImage(BuildService.java:211)
	at io.fabric8.maven.docker.service.BuildService.buildImage(BuildService.java:58)
	at io.fabric8.maven.docker.BuildMojo.buildAndTag(BuildMojo.java:58)
	at io.fabric8.maven.docker.BuildMojo.executeInternal(BuildMojo.java:44)
	at io.fabric8.maven.docker.AbstractDockerMojo.execute(AbstractDockerMojo.java:223)
	... 22 more
Caused by: io.fabric8.maven.docker.access.hc.http.HttpRequestException: received unexpected HTTP status: 500 Server Error (Internal Server Error: 500)
	at io.fabric8.maven.docker.access.hc.ApacheHttpClientDelegate$StatusCodeCheckerResponseHandler.handleResponse(ApacheHttpClientDelegate.java:177)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:222)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:164)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:139)
	at io.fabric8.maven.docker.access.hc.ApacheHttpClientDelegate.post(ApacheHttpClientDelegate.java:97)
	at io.fabric8.maven.docker.access.hc.DockerAccessWithHcClient.pullImage(DockerAccessWithHcClient.java:355)
	... 28 more

Ubuntu bamboo mvn -X output:

build	21-Aug-2017 16:14:35	[DEBUG] Connection leased: [id: 2][route: {}->unix://127.0.0.1:1][total kept alive: 0; route allocated: 1 of 100; total allocated: 1 of 20]
build	21-Aug-2017 16:14:35	[DEBUG] Executing request POST /v1.21/images/create?fromImage=docker.io%2Fdebian&tag=sid HTTP/1.1
build	21-Aug-2017 16:14:35	[DEBUG] Target auth state: UNCHALLENGED
build	21-Aug-2017 16:14:35	[DEBUG] Proxy auth state: UNCHALLENGED
build	21-Aug-2017 16:14:35	[DEBUG] http-outgoing-2 >> POST /v1.21/images/create?fromImage=docker.io%2Fdebian&tag=sid HTTP/1.1
build	21-Aug-2017 16:14:35	[DEBUG] http-outgoing-2 >> Accept: */*
build	21-Aug-2017 16:14:35	[DEBUG] http-outgoing-2 >> Content-Type: application/json; charset=utf-8
build	21-Aug-2017 16:14:35	[DEBUG] http-outgoing-2 >> X-Registry-Auth: eyJwYXNzd29yZCI6IiIsImF1dGgiOiIiLCJlbWFpbCI6IiIsInVzZXJuYW1lIjoiIn0=
build	21-Aug-2017 16:14:35	[DEBUG] http-outgoing-2 >> Content-Length: 0
build	21-Aug-2017 16:14:35	[DEBUG] http-outgoing-2 >> Host: 127.0.0.1:1
build	21-Aug-2017 16:14:35	[DEBUG] http-outgoing-2 >> Connection: Keep-Alive
build	21-Aug-2017 16:14:35	[DEBUG] http-outgoing-2 >> User-Agent: Apache-HttpClient/4.5.2 (Java/1.8.0_112)
build	21-Aug-2017 16:14:35	[DEBUG] http-outgoing-2 >> Accept-Encoding: gzip,deflate
build	21-Aug-2017 16:14:35	[DEBUG] http-outgoing-2 >> "POST /v1.21/images/create?fromImage=docker.io%2Fdebian&tag=sid HTTP/1.1[\r][\n]"
build	21-Aug-2017 16:14:35	[DEBUG] http-outgoing-2 >> "Accept: */*[\r][\n]"
build	21-Aug-2017 16:14:35	[DEBUG] http-outgoing-2 >> "Content-Type: application/json; charset=utf-8[\r][\n]"
build	21-Aug-2017 16:14:35	[DEBUG] http-outgoing-2 >> "X-Registry-Auth: eyJwYXNzd29yZCI6IiIsImF1dGgiOiIiLCJlbWFpbCI6IiIsInVzZXJuYW1lIjoiIn0=[\r][\n]"
build	21-Aug-2017 16:14:35	[DEBUG] http-outgoing-2 >> "Content-Length: 0[\r][\n]"
build	21-Aug-2017 16:14:35	[DEBUG] http-outgoing-2 >> "Host: 127.0.0.1:1[\r][\n]"
build	21-Aug-2017 16:14:35	[DEBUG] http-outgoing-2 >> "Connection: Keep-Alive[\r][\n]"
build	21-Aug-2017 16:14:35	[DEBUG] http-outgoing-2 >> "User-Agent: Apache-HttpClient/4.5.2 (Java/1.8.0_112)[\r][\n]"
build	21-Aug-2017 16:14:35	[DEBUG] http-outgoing-2 >> "Accept-Encoding: gzip,deflate[\r][\n]"
build	21-Aug-2017 16:14:35	[DEBUG] http-outgoing-2 >> "[\r][\n]"
build	21-Aug-2017 16:14:40	[DEBUG] http-outgoing-2 << "HTTP/1.1 200 OK[\r][\n]"
build	21-Aug-2017 16:14:40	[DEBUG] http-outgoing-2 << "Content-Type: application/json[\r][\n]"
build	21-Aug-2017 16:14:40	[DEBUG] http-outgoing-2 << "Server: Docker/1.12.6 (linux)[\r][\n]"
build	21-Aug-2017 16:14:40	[DEBUG] http-outgoing-2 << "Date: Mon, 21 Aug 2017 16:14:40 GMT[\r][\n]"
build	21-Aug-2017 16:14:40	[DEBUG] http-outgoing-2 << "Transfer-Encoding: chunked[\r][\n]"
build	21-Aug-2017 16:14:40	[DEBUG] http-outgoing-2 << "[\r][\n]"
build	21-Aug-2017 16:14:40	[DEBUG] http-outgoing-2 << "3a[\r][\n]"
build	21-Aug-2017 16:14:40	[DEBUG] http-outgoing-2 << "{"status":"Pulling repository docker.io/library/debian"}[\r][\n]"
build	21-Aug-2017 16:14:40	[DEBUG] http-outgoing-2 << "[\r][\n]"
build	21-Aug-2017 16:14:40	[DEBUG] http-outgoing-2 << HTTP/1.1 200 OK
build	21-Aug-2017 16:14:40	[DEBUG] http-outgoing-2 << Content-Type: application/json
build	21-Aug-2017 16:14:40	[DEBUG] http-outgoing-2 << Server: Docker/1.12.6 (linux)
build	21-Aug-2017 16:14:40	[DEBUG] http-outgoing-2 << Date: Mon, 21 Aug 2017 16:14:40 GMT
build	21-Aug-2017 16:14:40	[DEBUG] http-outgoing-2 << Transfer-Encoding: chunked
build	21-Aug-2017 16:14:40	[DEBUG] Connection can be kept alive indefinitely
build	21-Aug-2017 16:14:40	[INFO] DOCKER> Pulling repository docker.io/library/debian
build	21-Aug-2017 16:14:42	[DEBUG] http-outgoing-2 << "75[\r][\n]"
build	21-Aug-2017 16:14:42	[DEBUG] http-outgoing-2 << "{"errorDetail":{"message":"Could not reach any registry endpoint"},"error":"Could not reach any registry endpoint"}[\r][\n]"
build	21-Aug-2017 16:14:42	[DEBUG] http-outgoing-2 << "[\r][\n]"
build	21-Aug-2017 16:14:42	[DEBUG] http-outgoing-2 << "0[\r][\n]"
build	21-Aug-2017 16:14:42	[DEBUG] http-outgoing-2 << "[\r][\n]"
build	21-Aug-2017 16:14:42	[DEBUG] Connection [id: 2][route: {}->unix://127.0.0.1:1] can be kept alive indefinitely
build	21-Aug-2017 16:14:42	[DEBUG] Connection released: [id: 2][route: {}->unix://127.0.0.1:1][total kept alive: 1; route allocated: 1 of 100; total allocated: 1 of 20]
build	21-Aug-2017 16:14:42	[ERROR] DOCKER> Unable to pull 'debian:sid' from registry 'docker.io' : Could not reach any registry endpoint  [Could not reach any registry endpoint ]
build	21-Aug-2017 16:14:42	[DEBUG] Connection manager is shutting down
build	21-Aug-2017 16:14:42	[DEBUG] http-outgoing-2: Close connection
build	21-Aug-2017 16:14:42	[DEBUG] Connection manager shut down
build	21-Aug-2017 16:14:42	[INFO] ------------------------------------------------------------------------
build	21-Aug-2017 16:14:42	[INFO] BUILD FAILURE
build	21-Aug-2017 16:14:42	[INFO] ------------------------------------------------------------------------
build	21-Aug-2017 16:14:42	[INFO] Total time: 9.548 s
build	21-Aug-2017 16:14:42	[INFO] Finished at: 2017-08-21T16:14:42+00:00
build	21-Aug-2017 16:14:42	[INFO] Final Memory: 26M/492M
build	21-Aug-2017 16:14:42	[INFO] ------------------------------------------------------------------------
build	21-Aug-2017 16:14:42	[ERROR] Failed to execute goal io.fabric8:docker-maven-plugin:0.20.1:build (default-build) on project otxb-core-base-container: Unable to pull 'debian:sid' from registry 'docker.io' : Could not reach any registry endpoint -> [Help 1]
build	21-Aug-2017 16:14:42	org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal io.fabric8:docker-maven-plugin:0.20.1:build (default-build) on project otxb-core-base-container: Unable to pull 'debian:sid' from registry 'docker.io' : Could not reach any registry endpoint 
build	21-Aug-2017 16:14:42		at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)
build	21-Aug-2017 16:14:42		at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
build	21-Aug-2017 16:14:42		at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
build	21-Aug-2017 16:14:42		at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
build	21-Aug-2017 16:14:42		at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
build	21-Aug-2017 16:14:42		at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
build	21-Aug-2017 16:14:42		at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:120)
build	21-Aug-2017 16:14:42		at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:355)
build	21-Aug-2017 16:14:42		at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:155)
build	21-Aug-2017 16:14:42		at org.apache.maven.cli.MavenCli.execute(MavenCli.java:584)
build	21-Aug-2017 16:14:42		at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:216)
build	21-Aug-2017 16:14:42		at org.apache.maven.cli.MavenCli.main(MavenCli.java:160)
build	21-Aug-2017 16:14:42		at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
build	21-Aug-2017 16:14:42		at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
build	21-Aug-2017 16:14:42		at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
build	21-Aug-2017 16:14:42		at java.lang.reflect.Method.invoke(Method.java:498)
build	21-Aug-2017 16:14:42		at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
build	21-Aug-2017 16:14:42		at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
build	21-Aug-2017 16:14:42		at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
build	21-Aug-2017 16:14:42		at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
build	21-Aug-2017 16:14:42	Caused by: org.apache.maven.plugin.MojoExecutionException: Unable to pull 'debian:sid' from registry 'docker.io' : Could not reach any registry endpoint 
build	21-Aug-2017 16:14:42		at io.fabric8.maven.docker.AbstractDockerMojo.execute(AbstractDockerMojo.java:226)
build	21-Aug-2017 16:14:42		at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:132)
build	21-Aug-2017 16:14:42		at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
build	21-Aug-2017 16:14:42		... 19 more
build	21-Aug-2017 16:14:42	Caused by: io.fabric8.maven.docker.access.DockerAccessException: Unable to pull 'debian:sid' from registry 'docker.io' : Could not reach any registry endpoint 
build	21-Aug-2017 16:14:42		at io.fabric8.maven.docker.access.hc.DockerAccessWithHcClient.pullImage(DockerAccessWithHcClient.java:358)
build	21-Aug-2017 16:14:42		at io.fabric8.maven.docker.service.RegistryService.checkImageWithAutoPull(RegistryService.java:81)
build	21-Aug-2017 16:14:42		at io.fabric8.maven.docker.service.BuildService.autoPullBaseImage(BuildService.java:211)
build	21-Aug-2017 16:14:42		at io.fabric8.maven.docker.service.BuildService.buildImage(BuildService.java:58)
build	21-Aug-2017 16:14:42		at io.fabric8.maven.docker.BuildMojo.buildAndTag(BuildMojo.java:58)
build	21-Aug-2017 16:14:42		at io.fabric8.maven.docker.BuildMojo.executeInternal(BuildMojo.java:44)
build	21-Aug-2017 16:14:42		at io.fabric8.maven.docker.AbstractDockerMojo.execute(AbstractDockerMojo.java:223)
build	21-Aug-2017 16:14:42		... 21 more
build	21-Aug-2017 16:14:42	Caused by: io.fabric8.maven.docker.access.DockerAccessException: Could not reach any registry endpoint 
build	21-Aug-2017 16:14:42		at io.fabric8.maven.docker.access.chunked.PullOrPushResponseJsonHandler.throwDockerAccessException(PullOrPushResponseJsonHandler.java:45)
build	21-Aug-2017 16:14:42		at io.fabric8.maven.docker.access.chunked.PullOrPushResponseJsonHandler.process(PullOrPushResponseJsonHandler.java:22)
build	21-Aug-2017 16:14:42		at io.fabric8.maven.docker.access.chunked.EntityStreamReaderUtil.processJsonStream(EntityStreamReaderUtil.java:26)
build	21-Aug-2017 16:14:42		at io.fabric8.maven.docker.access.hc.DockerAccessWithHcClient$HcChunkedResponseHandlerWrapper.handleResponse(DockerAccessWithHcClient.java:652)
build	21-Aug-2017 16:14:42		at io.fabric8.maven.docker.access.hc.ApacheHttpClientDelegate$StatusCodeCheckerResponseHandler.handleResponse(ApacheHttpClientDelegate.java:172)
build	21-Aug-2017 16:14:42		at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:222)
build	21-Aug-2017 16:14:42		at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:164)
build	21-Aug-2017 16:14:42		at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:139)
build	21-Aug-2017 16:14:42		at io.fabric8.maven.docker.access.hc.ApacheHttpClientDelegate.post(ApacheHttpClientDelegate.java:97)
build	21-Aug-2017 16:14:42		at io.fabric8.maven.docker.access.hc.DockerAccessWithHcClient.pullImage(DockerAccessWithHcClient.java:355)
build	21-Aug-2017 16:14:42		... 27 more

I can pull manually just fine on both the boxes:

ubuntu@ip-10-42-128-244:~$ docker pull docker.io/debian:sid
sid: Pulling from library/debian
b5499bef70c5: Pull complete 
Digest: sha256:c5e3138644b0b6116f95c05b211ac2a456f575c200b4edac5878d78144aebef4
Status: Downloaded newer image for debian:sid

As you can see it definitely forgets our custom registry and understands it’s trying to get from docker.io - it just fails. I have tried registry-1.docker.com, registry.docker.com, registry.hub.docker.com, and various other permutations of .io/.com and hostnames to no good effect.

This was working until the 19th; we didn’t change code, poms, or versions. Then it started failing on the 20th. Any thoughts are welcome.

  • Happening on my mac from home and from bamboo server in AWS - not network, maven version, docker version
  • POMs not changed
  • Builds not changed
  • Code not changed in any meaningful way
  • Different parent poms
  • Different base images (debian, ubuntu)
  • Multiple builds
  • Different default repos (otxb is in AWS ECR, otxp is in jfrog Artifactory) that are getting overridden for the pull

Info

  • d-m-p version : 0.20.1
  • Maven version (mvn -v) for my Mac:
Apache Maven 3.5.0 (ff8f5e7444045639af65f6095c62210b5713f426; 2017-04-03T14:39:06-05:00)
Maven home: /usr/local/Cellar/maven/3.5.0/libexec
Java version: 1.8.0_112, vendor: Oracle Corporation
Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0_112.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.12.6", arch: "x86_64", family: "mac"

for the ubuntu bamboo server, it’s Maven 3.3.

  • Docker version on Mac:
Client:
 Version:      17.07.0-ce-rc3
 API version:  1.31
 Go version:   go1.8.3
 Git commit:   665d244
 Built:        Thu Aug 17 00:56:49 2017
 OS/Arch:      darwin/amd64

Server:
 Version:      17.07.0-ce-rc3
 API version:  1.31 (minimum version 1.12)
 Go version:   go1.8.3
 Git commit:   665d244
 Built:        Thu Aug 17 01:32:35 2017
 OS/Arch:      linux/amd64
 Experimental: true
  • Docker version on build server:
ubuntu@ip-10-42-128-244:~$ docker version
Client:
 Version:      1.12.6
 API version:  1.24
 Go version:   go1.6.4
 Git commit:   78d1802
 Built:        Tue Jan 10 20:38:45 2017
 OS/Arch:      linux/amd64

Server:
 Version:      1.12.6
 API version:  1.24
 Go version:   go1.6.4
 Git commit:   78d1802
 Built:        Tue Jan 10 20:38:45 2017
 OS/Arch:      linux/amd64
  • If it’s a bug, how to reproduce : Just mvn install with our POMs/any pom setting a docker.pull.registry to docker.io.

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Reactions:2
  • Comments:10

github_iconTop GitHub Comments

4reactions
ernestmcommented, Aug 22, 2017

Well dip me in honey and call the bears, that fixes it! I was on 1.8.0_112-b16 but upgrading to 1.8.0_144 has made the issue go away. Thanks all! So for future searchers, if you’re using docker-maven-plugin, or I guess any other maven plugin or java scheme for downloading container images (?) from docker.io, upgrading your Java version to at least .144 will get you going again.

Thanks to everyone who pitched in to find this.

4reactions
kogitantcommented, Aug 22, 2017

My problem was resolved by upgrading the Jenkins pipeline to use JDK8u144 instead of JDK8u112 that it was using. When I ssh’d to the jenkins slave, the slave machine itself had JDK8u144 installed, and thus the build worked when executed directly on the machine.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Ask Question - Stack Overflow
Here are few suggestions: Try restarting your Docker service. Check your network connections. For example by the following shell commands:
Read more >
Docker pull results in "Request canceled while waiting for ...
Issue type – error When trying to pull docker containers from I am confronted with this error: Using default tag: latest Error response...
Read more >
Troubleshooting | Container Registry documentation
GKE returns an ImagePullBackoff error when it is unable to pull an image from a registry. The error might occur because the image...
Read more >
Failed to pull image docker-local/sampleimage:v1.0.0dev-1 ...
Failed to pull image docker-local/sampleimage:v1.0.0dev-1, error API error (500) ... This most likely means that you have a self-signed ...
Read more >
Troubleshoot common issues - Azure Container Instances
This error is most often encountered when deploying Windows images ... If the image can't be pulled, events like the following are shown...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

ARG not support in FROM

Next page

Cannot temporarily tag because target image already exists