Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Recent commit breaks Mutual TLS

See original GitHub issue

Commit f20063d26b640cb4121e092b5a3a0bb4c0a10eb5 appears to break Mutual TLS in the sense that the connection just hangs. There’re no Exceptions as far as I can tell and the only failure I’m getting is when the events in SslHandler#channelInactive(…) fire.

Interestingly, the problem occurs only in our production environment (which has more load, latencies and RTTs between servers). The problem doesn’t occur on my dev workstation (i.e. unable to reproduce in the form of a unit test).

Reverting the commit fixes the problem for me.

Expected behavior

Actual behavior

Steps to reproduce

Minimal yet complete reproducer code (or URL to code)

Netty version

4.1.12.Final-SNAPSHOT w/ TCN 2.0.2.Final-SNAPSHOT (openssl-static)

JVM version (e.g. java -version)

OS version (e.g. uname -a)

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Comments:12 (12 by maintainers)

github_iconTop GitHub Comments

1reaction
rkapsicommented, Jun 2, 2017

@Scottmitch here are the results for #6803

openssl_wrap_status w/ OPENSSL: works

2017-06-02 11:55:17,506 [ConnectorThread-6] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:17,506 [ConnectorThread-9] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:17,507 [ConnectorThread-4] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:17,666 [ConnectorThread-2] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:17,917 [ConnectorThread-7] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:18,012 [ConnectorThread-7] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:18,014 [ConnectorThread-6] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:18,027 [ConnectorThread-2] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:18,029 [ConnectorThread-9] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:18,036 [ConnectorThread-4] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:18,064 [ConnectorThread-2] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:18,104 [ConnectorThread-2] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:18,681 [ConnectorThread-1] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:18,684 [ConnectorThread-2] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:18,715 [ConnectorThread-1] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:18,721 [ConnectorThread-2] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:19,942 [ConnectorThread-5] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:19,985 [ConnectorThread-5] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:20,279 [ConnectorThread-5] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:20,319 [ConnectorThread-5] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:20,354 [ConnectorThread-14] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:20,395 [ConnectorThread-14] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:20,928 [ConnectorThread-5] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:20,967 [ConnectorThread-5] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:21,191 [ConnectorThread-6] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:21,225 [ConnectorThread-6] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:21,304 [ConnectorThread-5] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:21,332 [ConnectorThread-5] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:21,431 [ConnectorThread-1] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:21,460 [ConnectorThread-1] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:21,650 [ConnectorThread-10] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:21,684 [ConnectorThread-5] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:21,699 [ConnectorThread-10] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:21,722 [ConnectorThread-5] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:21,811 [ConnectorThread-15] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:21,851 [ConnectorThread-15] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:21,902 [ConnectorThread-2] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:21,938 [ConnectorThread-2] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:21,981 [ConnectorThread-15] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:55:22,019 [ConnectorThread-15] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0

openssl_wrap_status w/ JDK: works

2017-06-02 11:50:19,115 [ConnectorThread-7] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:19,148 [ConnectorThread-5] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:19,409 [ConnectorThread-5] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:19,413 [ConnectorThread-7] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:20,349 [ConnectorThread-3] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:20,412 [ConnectorThread-3] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:20,496 [ConnectorThread-1] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:20,543 [ConnectorThread-9] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:20,550 [ConnectorThread-1] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:20,589 [ConnectorThread-9] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:20,679 [ConnectorThread-5] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:20,728 [ConnectorThread-5] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:20,764 [ConnectorThread-3] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:20,775 [ConnectorThread-4] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:20,817 [ConnectorThread-3] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:20,841 [ConnectorThread-4] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:20,844 [ConnectorThread-3] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:20,905 [ConnectorThread-3] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:21,029 [ConnectorThread-0] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:21,082 [ConnectorThread-0] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:21,180 [ConnectorThread-3] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:21,253 [ConnectorThread-3] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:21,271 [ConnectorThread-9] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:21,331 [ConnectorThread-9] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:21,466 [ConnectorThread-15] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:21,515 [ConnectorThread-15] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:22,050 [ConnectorThread-13] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:22,111 [ConnectorThread-13] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:22,380 [ConnectorThread-10] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:22,454 [ConnectorThread-10] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:22,510 [ConnectorThread-15] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:22,569 [ConnectorThread-15] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:22,646 [ConnectorThread-5] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:22,708 [ConnectorThread-5] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:23,162 [ConnectorThread-7] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:23,220 [ConnectorThread-7] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:23,238 [ConnectorThread-10] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:23,293 [ConnectorThread-10] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:23,337 [ConnectorThread-14] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
2017-06-02 11:50:23,373 [ConnectorThread-14] SslHandler INFO : ==previous status was unwrap, new status is Status = OK HandshakeStatus = NEED_UNWRAP
bytesConsumed = 0 bytesProduced = 0
1reaction
rkapsicommented, May 31, 2017

@Scottmitch I’ll test JDK and the patch tomorrow.

Read more comments on GitHub >

github_iconTop Results From Across the Web

The NSA Warns of TLS Inspection - Schneier on Security
To minimize the risks described above, breaking and inspecting TLS traffic should only be conducted once within the enterprise network.
Read more >
Mixed authentication with client certificate breaks auth on ...
When you enable mutual tls or certificate check on domain, it will mean that server should ask client for client certificate. So if...
Read more >
FIGHTING BACK AGAINST SSL INSPECTION, OR HOW SSL ...
Mutual Authentication and SSL Interception. A solution for resisting SSL interception without breaking compatibility or requiring cooperation ...
Read more >
Mutual TLS between GitLab and Gitaly (#37090) · Issues
Problem to solve Allow mutual TLS between GitLab and Gitaly when using an external Gitaly server.
Read more >
In Defense of Mutual TLS | Hacker News
First: TLS (and mTLS) create secure channels. A channel bears many requests and responses. In many designs, a single channel will bear ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

impossible to restore the order of parameters after using QueryStringDecoder

Next page

Leaked Connection handling