Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

TCN is failing with error:14179044:SSL routines:tls_construct_server_key_exchange:internal error

See original GitHub issue

We’re seeing some ciphers such as TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (a.k.a. DHE-RSA-AES128-GCM-SHA256) failing with TCN 2.0.12.

Here is a very crude repro.

public static void main(String[] args) throws Exception {
    String cipher = "DHE-RSA-AES128-GCM-SHA256";
    
    SelfSignedCertificate cert = new SelfSignedCertificate();
    
    ChannelHandler handler = new ChannelInitializer<Channel>() {
      @Override
      protected void initChannel(Channel ch) throws Exception {
        ChannelPipeline pipeline = ch.pipeline();
        
        SslContext sslContext = SslContextBuilder.forServer(cert.certificate(), cert.privateKey())
          .sslProvider(SslProvider.OPENSSL)
          .ciphers(Arrays.asList(cipher))
          .build();
        
        pipeline.addLast(sslContext.newHandler(ch.alloc()));
        pipeline.addLast(new ChannelInboundHandlerAdapter() {
          @Override
          public void userEventTriggered(ChannelHandlerContext ctx, Object evt) throws Exception {
            if (evt instanceof SslHandshakeCompletionEvent) {
              SslHandshakeCompletionEvent event = (SslHandshakeCompletionEvent)evt;
              if (!event.isSuccess()) {
                Throwable cause = event.cause();
                if (cause instanceof SSLHandshakeException) {
                  cause.printStackTrace();
                }
              }
            }
            
            ctx.fireUserEventTriggered(evt);
          }
          
          @Override
          public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
            cause.printStackTrace();
          }
        });
      }
    };
    
    ServerBootstrap bootstrap = new ServerBootstrap()
        .channel(NioServerSocketChannel.class)
        .group(new NioEventLoopGroup())
        .childHandler(handler);
    
    Channel channel = bootstrap.bind(0).sync().channel();
    InetSocketAddress isa = (InetSocketAddress)channel.localAddress();
    
    System.out.println("Server: " + OpenSsl.versionString());
    System.out.println(cipher + " is supported: " + OpenSsl.isCipherSuiteAvailable(cipher));
    
    System.out.println("Try: openssl s_client -cipher " + cipher + " -connect localhost:" + isa.getPort());
    
    Thread.sleep(Long.MAX_VALUE);
  }

$ openssl s_client -cipher DHE-RSA-AES128-GCM-SHA256 -connect localhost:24619
CONNECTED(00000003)
depth=0 CN = example.com
verify error:num=18:self signed certificate
verify return:1
depth=0 CN = example.com
verify return:1
140045187453376:error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error:ssl/record/rec_layer_s3.c:1407:SSL alert number 80
---
Certificate chain
 0 s:/CN=example.com
   i:/CN=example.com
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIBqjCCAROgAwIBAgIJAL5hwsKKvJqWMA0GCSqGSIb3DQEBBQUAMBYxFDASBgNV
BAMTC2V4YW1wbGUuY29tMCAXDTE3MDgwMjAwMzk0NloYDzk5OTkxMjMxMjM1OTU5
WjAWMRQwEgYDVQQDEwtleGFtcGxlLmNvbTCBnzANBgkqhkiG9w0BAQEFAAOBjQAw
gYkCgYEAk5aJT4NDokMO+YSEuJUmGDxqMEiLV9bVu4SoPP1v/WXPsQfiiscuP3O+
T5lGk9M8PtBD/803jHra4YaFvoW8cNTxk2tBOzq9FadOGEpnuTIfg2RTSpDkH8X+
dMhQfqvUZGSAJ9GmjBZ1mhIF4ECTSoyeuRlagPMcpzgkVuOLN5ECAwEAATANBgkq
hkiG9w0BAQUFAAOBgQB2MmMeqI12VfIQk6bV5ppjYEFay0avSlHeWyExG8Uot9Uv
7j1hynx1dpWcUhH+33HtmHuJSqRy1AhskCLSHNg1LXIJfwZFrZXln2s/1RrA781p
e3mrOSQ47UIVrBRrnk8HV+UFVcZlCJTWY//IKiEQEyoCowzWOLTjxvNMnXxRfA==
-----END CERTIFICATE-----
subject=/CN=example.com
issuer=/CN=example.com
---
No client certificate CA names sent
---
SSL handshake has read 542 bytes and written 102 bytes
Verification error: self signed certificate
---
New, (NONE), Cipher is (NONE)
Server public key is 1024 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : 0000 
    Session-ID: E6E1CEE7455FA36EC45A81F0D1144392ED7A8CDB61F477F26372C60829949171
    Session-ID-ctx: 
    Master-Key: 
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1533170393
    Timeout   : 7200 (sec)
    Verify return code: 18 (self signed certificate)
    Extended master secret: yes
---

Notice this error in the above output, and Cipher : 0000 and the client exists on its own.

140045187453376:error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error:ssl/record/rec_layer_s3.c:1407:SSL alert number 80

javax.net.ssl.SSLHandshakeException: error:14179044:SSL routines:tls_construct_server_key_exchange:internal error
	at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.sslReadErrorResult(ReferenceCountedOpenSslEngine.java:1114)
	at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.unwrap(ReferenceCountedOpenSslEngine.java:1076)
	at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.unwrap(ReferenceCountedOpenSslEngine.java:1143)
	at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.unwrap(ReferenceCountedOpenSslEngine.java:1186)
	at io.netty.handler.ssl.SslHandler$SslEngineType$1.unwrap(SslHandler.java:216)
	at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1275)
	at io.netty.handler.ssl.SslHandler.decodeNonJdkCompatible(SslHandler.java:1189)
	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1223)
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489)
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965)
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:646)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:581)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:460)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.lang.Thread.run(Thread.java:748)
io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: error:14179044:SSL routines:tls_construct_server_key_exchange:internal error
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:459)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965)
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:646)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:581)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:460)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.lang.Thread.run(Thread.java:748)
Caused by: javax.net.ssl.SSLHandshakeException: error:14179044:SSL routines:tls_construct_server_key_exchange:internal error
	at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.sslReadErrorResult(ReferenceCountedOpenSslEngine.java:1114)
	at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.unwrap(ReferenceCountedOpenSslEngine.java:1076)
	at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.unwrap(ReferenceCountedOpenSslEngine.java:1143)
	at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.unwrap(ReferenceCountedOpenSslEngine.java:1186)
	at io.netty.handler.ssl.SslHandler$SslEngineType$1.unwrap(SslHandler.java:216)
	at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1275)
	at io.netty.handler.ssl.SslHandler.decodeNonJdkCompatible(SslHandler.java:1189)
	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1223)
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489)
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428)
	... 16 more

The same test when switching to previous TCN version w/ OpenSSL 1.0.2 works as expected.

$ openssl s_client -cipher DHE-RSA-AES128-GCM-SHA256 -connect localhost:27611
CONNECTED(00000003)
depth=0 CN = example.com
verify error:num=18:self signed certificate
verify return:1
depth=0 CN = example.com
verify return:1
---
Certificate chain
 0 s:/CN=example.com
   i:/CN=example.com
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIBqTCCARKgAwIBAgIIIygVaeXtHowwDQYJKoZIhvcNAQEFBQAwFjEUMBIGA1UE
AxMLZXhhbXBsZS5jb20wIBcNMTcwODAyMDA0MzE3WhgPOTk5OTEyMzEyMzU5NTla
MBYxFDASBgNVBAMTC2V4YW1wbGUuY29tMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCB
iQKBgQCYq2Dq6dNAj5HfDv1PYGXY0Of9/mEB1/SV0ap1BRLMkJ11FE4kc+RBYK1X
Fj7z3gT/B9bUcgcBLqd+QtuKiKQaqZ9LyEyMnXuCECp54ATF5cKCo7t17P7UjG5W
W+vG7BcpZgILpIzUOg9uHm9QmQF6N2PhAzyWeOHOygmVfTskHQIDAQABMA0GCSqG
SIb3DQEBBQUAA4GBABsQmq73/GdhijgSF6c/Us0RUKzCxdUNekjsezV8mZ9c0Rkr
cpgHVwA9Kxd2EOTWnWt1OGb0WpUNOmxxRoMvUXLMnIHOvGGfnOYoQUJGBJEpXof/
YK2YqfQD/DufOzVjpHp0/nZfhnNZQr9YiWBdFrP91Ch7vFmAS5MQ/ARNXKaT
-----END CERTIFICATE-----
subject=/CN=example.com
issuer=/CN=example.com
---
No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: DH, 1024 bits
---
SSL handshake has read 994 bytes and written 292 bytes
Verification error: self signed certificate
---
New, TLSv1.2, Cipher is DHE-RSA-AES128-GCM-SHA256
Server public key is 1024 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : DHE-RSA-AES128-GCM-SHA256
    Session-ID: A1EE6830C05434ECBE0435046BAE6CFD7F01971B039AF1003072F4EFCCEFE6FA
    Session-ID-ctx: 
    Master-Key: 9309D650B879B99E6BEB7DCCE35819AB17E350D1359F3670205F77E165443FF036F2A4D12107654C1C14C6226FAFDE3B
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1533170605
    Timeout   : 7200 (sec)
    Verify return code: 18 (self signed certificate)
    Extended master secret: no
---
^C

Notice that there is no error, Cipher : DHE-RSA-AES128-GCM-SHA256 and I need to exit with ^C as usual.

Expected behavior

Actual behavior

Steps to reproduce

Minimal yet complete reproducer code (or URL to code)

Netty version

4.1.28 w/ TCN 2.0.12 (OpenSSL static on Linux)

JVM version (e.g. `java -version`)

OS version (e.g. `uname -a`)

Issue Analytics

State:
Created 5 years ago
Comments:22 (22 by maintainers)

Top GitHub Comments

2reactions

rkapsicommented, Aug 23, 2018

@normanmaurer I think I found it…

The OpenSSL changelog says:

*) config has been changed so that by default OPENSSL_NO_DEPRECATED is used.
     Access to deprecated functions can be re-enabled by running config with
     "enable-deprecated". In addition applications wishing to use deprecated
     functions must define OPENSSL_USE_DEPRECATED. Note that this new behaviour
     will, by default, disable some transitive includes that previously existed
     in the header files (e.g. ec.h will no longer, by default, include bn.h)
     [Matt Caswell]

There are two cases in TCN that utilize it (both DHE related)…

https://github.com/netty/netty-tcnative/blob/master/openssl-dynamic/src/main/c/ssl.c#L449-L456

https://github.com/netty/netty-tcnative/blob/master/openssl-dynamic/src/main/c/sslutils.c#L233-L362

When I go ahead with running config with enable-deprecated and do an #ifndef for OPENSSL_USE_DEPRECATED it’s eventually failing with this compile error:

build	23-Aug-2018 17:32:44	[INFO] libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I./src -O3 -Werror -fno-omit-frame-pointer -Wunused-variable -DHAVE_OPENSSL -I/tmp/netty-tcnative/openssl-static/target/ssl/include -I/tmp/netty-tcnative/openssl-static/target/apr/include/apr-1 -D_LARGEFILE64_SOURCE -I/etc/alternatives/java_sdk/include -I/etc/alternatives/java_sdk/include/linux -c src/sslcontext.c  -fPIC -DPIC -o src/.libs/sslcontext.o
build	23-Aug-2018 17:32:44	[INFO] /bin/sh ./libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I./src     -O3 -Werror -fno-omit-frame-pointer -Wunused-variable -DHAVE_OPENSSL -I/tmp/netty-tcnative/openssl-static/target/ssl/include  -I/tmp/netty-tcnative/openssl-static/target/apr/include/apr-1  -D_LARGEFILE64_SOURCE -I/etc/alternatives/java_sdk/include -I/etc/alternatives/java_sdk/include/linux -c -o src/sslutils.lo src/sslutils.c
build	23-Aug-2018 17:32:44	[INFO] libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I./src -O3 -Werror -fno-omit-frame-pointer -Wunused-variable -DHAVE_OPENSSL -I/tmp/netty-tcnative/openssl-static/target/ssl/include -I/tmp/netty-tcnative/openssl-static/target/apr/include/apr-1 -D_LARGEFILE64_SOURCE -I/etc/alternatives/java_sdk/include -I/etc/alternatives/java_sdk/include/linux -c src/sslutils.c  -fPIC -DPIC -o src/.libs/sslutils.o
build	23-Aug-2018 17:32:44	[INFO] src/sslutils.c: In function 'get_dh':
build	23-Aug-2018 17:32:44	[INFO] src/sslutils.c:341:15: error: dereferencing pointer to incomplete type 'DH {aka struct dh_st}'
build	23-Aug-2018 17:32:44	[INFO]              dh->p = BN_bin2bn(dh0512_p, sizeof(dh0512_p), NULL);
build	23-Aug-2018 17:32:44	[INFO]                ^~
build	23-Aug-2018 17:32:44	[INFO] make: *** [src/sslutils.lo] Error 1
build	23-Aug-2018 17:32:44	[INFO] Makefile:507: recipe for target 'src/sslutils.lo' failed
build	23-Aug-2018 17:32:44	[INFO] rc: 2
build	23-Aug-2018 17:32:44	[INFO] ------------------------------------------------------------------------
build	23-Aug-2018 17:32:44	[INFO] Reactor Summary:
build	23-Aug-2018 17:32:44	[INFO] 
build	23-Aug-2018 17:32:44	[INFO] Netty/TomcatNative [Parent] ........................ SUCCESS [ 41.115 s]
build	23-Aug-2018 17:32:44	[INFO] Netty/TomcatNative [OpenSSL - Static] .............. FAILURE [04:00 min]
build	23-Aug-2018 17:32:44	[INFO] ------------------------------------------------------------------------
build	23-Aug-2018 17:32:44	[INFO] BUILD FAILURE
build	23-Aug-2018 17:32:44	[INFO] ------------------------------------------------------------------------
build	23-Aug-2018 17:32:44	[INFO] Total time: 04:46 min
build	23-Aug-2018 17:32:44	[INFO] Finished at: 2018-08-23T21:32:44+00:00
build	23-Aug-2018 17:32:44	[INFO] Final Memory: 21M/300M

1reaction

rkapsicommented, Aug 24, 2018

On it.

Top Results From Across the Web

No results found

Troubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.

Start Free

Top Related Reddit Thread

No results found

Top Related Tweet

No results found

Top Related Dev.to Post

No results found

TCN is failing with error:14179044:SSL routines:tls_construct_server_key_exchange:internal error

Expected behavior

Actual behavior

Steps to reproduce

Minimal yet complete reproducer code (or URL to code)

Netty version

JVM version (e.g. `java -version`)

OS version (e.g. `uname -a`)

Issue Analytics

Top GitHub Comments

Top Results From Across the Web

Top Related Medium Post

Top Related StackOverflow Question

Troubleshoot Live Code

Top Related Reddit Thread

Top Related Hackernoon Post

Top Related Tweet

Top Related Dev.to Post

Top Related Hashnode Post

AssertionError in Channel.config().getOptions() when nio is used

initChannel results in System.err (two way ssl authentication)

TCN is failing with error:14179044:SSL routines:tls_construct_server_key_exchange:internal error

Expected behavior

Actual behavior

Steps to reproduce

Minimal yet complete reproducer code (or URL to code)

Netty version

JVM version (e.g. java -version)

OS version (e.g. uname -a)

Issue Analytics

Top GitHub Comments

Top Results From Across the Web

Top Related Medium Post

Top Related StackOverflow Question

Troubleshoot Live Code

Top Related Reddit Thread

Top Related Hackernoon Post

Top Related Tweet

Top Related Dev.to Post

Top Related Hashnode Post

AssertionError in Channel.config().getOptions() when nio is used

initChannel results in System.err (two way ssl authentication)

JVM version (e.g. `java -version`)

OS version (e.g. `uname -a`)