Images created by Jib randomly causes 500 Internal server error in Red Hat Quay
See original GitHub issueEnvironment:
- Jib version: 1.5.1
- Build tool: Maven
- OS: Linux and Windows
Description of the issue: Images created by Jib randomly causes 500 Internal server error in Red Hat Quay. This has been reproduced with the HelloWorld example from Jib, but does not happen with images created in any other way than with Jib.
Expected behavior: Images created by Jib should consistently be accepted by Quay.
Steps to reproduce:
- Download https://github.com/GoogleContainerTools/jib/tree/master/examples/helloworld
- Change <image>gcr.io/REPLACE-WITH-YOUR-GCP-PROJECT/image-built-with-jib</image> to <image>QUAY-REGISTRY/image-built-with-jib</image>
- Run
mvn compile jib:build
jib-maven-plugin
Configuration:
<configuration>
<to>
<image>QUAY-REGISTRY/image-built-with-jib</image>
</to>
</configuration>
Log output:
> [2019-09-13T07:50:47.986Z] [ERROR] Failed to execute goal com.google.cloud.tools:jib-maven-plugin:1.5.1:build (build_docker_image) on project xx: 500 Internal Server Error
> [2019-09-13T07:50:47.986Z] [ERROR] <html>
> [2019-09-13T07:50:47.986Z] [ERROR] <head>
> [2019-09-13T07:50:47.986Z] [ERROR] <title>Internal Server Error</title>
> [2019-09-13T07:50:47.986Z] [ERROR] </head>
> [2019-09-13T07:50:47.986Z] [ERROR] <body>
> [2019-09-13T07:50:47.986Z] [ERROR] <h1>Internal Server Error</h1>
> [2019-09-13T07:50:47.986Z] [ERROR]
> [2019-09-13T07:50:47.986Z] [ERROR] </body>
> [2019-09-13T07:50:47.986Z] [ERROR] </html>
Additional Information: Red Hat’s response is that this only happens with Jib-created images and therefore points the finger at Jib.
Is it possible to get Google and Red Hat to cooperate in finding a solution to this? Red Hat Case #: 02456747
Issue Analytics
- State:
- Created 4 years ago
- Comments:42 (25 by maintainers)
Top Results From Across the Web
500 Internal Server Error while copying multi-arch image into ...
It works well on a local registry but not on the Openshift cluster internal registry. Environment. Red Hat OpenShift Container Platform (RHOCP).
Read more >Jib & Quay: 500s, 400s, and 401s - Google Groups
The essential issues: - On Quay v3.0.5 with the Jib "helloworld" example, Quay returns HTTP 500 responses. One theory is that this is...
Read more >OpenShift Maven Plugin - JKube - Eclipse
How to configure image name generated by Eclipse JKube? 17. Appendix ... to build and deploy your Java application with Red Hat OpenShift...
Read more >Home of Quarkus Cheat-Sheet - GitHub Pages
application , if container image is created (JiB, Docker extensions) is ... There is a Quarkus Test Resource that starts and stops H2...
Read more >Tag Latest Not Found In Repository Docker.Io/Library/Centos ...
Io/Library/Centos On Redhat Enterprise Linux Behind Proxy Server ... In RHEL 8.0 and 8.1, quay.io is listed before docker.io in the similar images...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Quay developer here; I just wanted to take a moment to chime in.
@jonjohnsonjr
Great catch! The https://quay.io service does not (yet) support Schema 2 images, yet. To the best of my knowledge, some on-premise installations may have this enabled depending on their configuration.
Our initial struggle was being able to reproduce this issue locally. There’s a ton of really great information in here that will get us one step closer. I, personally, have never heard of jib until this issue popped up so it’s all new ground for me.
The HTTP 500 response sounds like an issue on our end. I can’t promise when it will be fixed, but regardless of the payload received by Quay, it should respond with an appropriate status. I will do my best to provide updates if anyone is interested.
Please don’t hesitate to
@
me if anyone has questions I might be able to answer.Update:
I’ve been able to create the “Hello World” example using jib and push it to my local (development) Quay environment. Thanks to all for the reproduction steps!
Here’s some initial observations:
It’s still pretty premature but I believe that exception is not caught and handled correctly (hence the 5xx) and I’m not sure which chunk (if any?) is malformed. On my local system and example, the request which generates this error is:
Moving forward, I’m digging through the raw requests using Wireshark to see if I can find a malformed chunk. My goal is to produce a testable scenario to trigger this exception so that we can at least get it handled properly on Quay’s end. If I find a malformed request being generated by jib and am able to reproduce it, I’ll share that information as well.
For reference purposes, here’s essentially my current test-case:
master
)helloWorld
example to my Quay instancemvn -Djib.console=plain -Djava.util.logging.config.file=./logging.properties -DsendCredentialsOverHttp=true compile jib:build
HelloWorld.java
and the repository’s name inpom.xml
for each fresh run.Note: After encountering the HTTP 500 error initially for a given image, a subsequent retry succeeds. Hence the purpose of my changes to
pom.xml
andHelloWorld.java
in my test-case.I am far from a Java, maven, or jib expert so please don’t hesitate to share suggestions if you have them. I had hoped I could use
mitmproxy
but neither the command-line arguments (e.g.-Dhttp.proxyHost [...]
) or the maven settings file seemed to have worked. It is probably related to my local configuration and not using TLS/HTTPs.