Can't connect to docker using TLS and Docker credentials

Server configuration

• CentOS 7
• Docker version 1.8.2-el7.centos, build a01dc02/1.8.2
• Placed certificates in /etc/docker/. ca.pem (ca cert), cert.pem (server cert), key.pem (server private key).
• Added to /etc/sysconfig/docker with OPTIONS="--tlsverify -H tcp://127.0.0.1:443"

Client configuration

• Same system as server.
• I’ve created a client directory, let’s say /path/docker, and have added my ca.pem (ca cert), cert.pem (client cert), and key.pem (client private key) to said path.
• My jenkins is listening on localhost:8080.

I had to dig for it but found the docker cloud authentication configuration instructions https://github.com/jenkinsci/docker-plugin/issues/245#issuecomment-158012362.

Jenkins specs

• Jenkins ver. 1.646
• docker-plugin 0.16.0
• Added Docker Certificates Directory credentials and configured path to /path/docker (where client {ca,cert,key}.pem are located).
• Added a docker cloud with Docker URL pointed to https://127.0.0.1:443 and using aforementioned credentials.

Issue

I can’t get the docker plugin to properly connect. I click the test connection button and it throws an error. When I test with curl it succeeds with my client certificates.

cd /path/docker
#successful auth test
curl --cert ./cert.pem --key ./key.pem --cacert ./ca.pem https://127.0.0.1:443/info
#successfully denied access
curl -k https://127.0.0.1:443/info

Here’s a stack trace from the web UI when I use the Test Connection button in the docker cloud settings of the configure page.

Error in SSL Configuration

com.github.dockerjava.api.DockerClientException: Error in SSL Configuration
    at com.github.dockerjava.jaxrs.DockerCmdExecFactoryImpl.init(DockerCmdExecFactoryImpl.java:140)
    at com.github.dockerjava.core.DockerClientImpl.withDockerCmdExecFactory(DockerClientImpl.java:131)
    at com.github.dockerjava.core.DockerClientBuilder.build(DockerClientBuilder.java:73)
    at com.nirima.jenkins.plugins.docker.client.ClientBuilderForPlugin.build(ClientBuilderForPlugin.java:65)
    at com.nirima.jenkins.plugins.docker.DockerCloud$DescriptorImpl.doTestConnection(DockerCloud.java:615)
    at sun.reflect.GeneratedMethodAccessor183.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:298)
    at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:161)
    at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:96)
    at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:121)
    at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
    at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
    at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
    at org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:249)
    at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
    at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
    at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
    at org.kohsuke.stapler.Stapler.invoke(Stapler.java:649)
    at org.kohsuke.stapler.Stapler.service(Stapler.java:238)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:686)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1494)
    at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:132)
    at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:123)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
    at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:49)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
    at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
    at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
    at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
    at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
    at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1474)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:499)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:533)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
    at org.eclipse.jetty.server.Server.handle(Server.java:370)
    at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
    at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:960)
    at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1021)
    at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865)
    at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
    at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:668)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
    at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
Caused by: com.github.dockerjava.api.DockerClientException: org.bouncycastle.asn1.pkcs.PrivateKeyInfo cannot be cast to org.bouncycastle.openssl.PEMKeyPair
    at com.github.dockerjava.core.LocalDirectorySSLConfig.getSSLContext(LocalDirectorySSLConfig.java:57)
    at com.github.dockerjava.jaxrs.DockerCmdExecFactoryImpl.init(DockerCmdExecFactoryImpl.java:138)
    ... 61 more
Caused by: java.lang.ClassCastException: org.bouncycastle.asn1.pkcs.PrivateKeyInfo cannot be cast to org.bouncycastle.openssl.PEMKeyPair
    at com.github.dockerjava.core.CertificateUtils.loadPrivateKey(CertificateUtils.java:127)
    at com.github.dockerjava.core.CertificateUtils.createKeyStore(CertificateUtils.java:45)
    at com.github.dockerjava.core.LocalDirectorySSLConfig.getSSLContext(LocalDirectorySSLConfig.java:50)
    ... 62 more

Please note my SSL certs are x509 ASCII PEM format. Any idea what could be going wrong?

Workaround

This section is an edit of the original issue. I noted a workaround below.

[samrocketman]

It FINALLY worked because I had to change how the private key was presented.

mv key.pem key.bak
openssl rsa -in key.bak -text > key.pem

Apparently, it only takes the private key successfully if you give it an extended -text format. The wiki needs to be updated with how to use TLS auth. I’ll get around to it if nobody does.

[ndeloof]

Docker plugin 0.17 now uses docker-commons for credentials management, which should avoid such issues making clearer definition about the expected keys format.