question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Possible bug in PoolThreadCache

See original GitHub issue

Expected behavior

no jvm crash

Actual behavior

jvm crash

Steps to reproduce

no reproducer

Netty version

4.1.51

JVM version (e.g. java -version)

Zulu11.43+22-SA (11.0.9+11) (build 11.0.9+11-LTS)

OS version (e.g. uname -a)

OS:CentOS Linux release 7.8.2003 (Core) uname:Linux 3.10.0-1127.18.2.el7.x86_64 #1 SMP Sun Jul 26 15:27:06 UTC 2020 x86_64

I was working on a JVM crash and I suspect it’s a netty bug

jvm crashes when G1GC processes an object and this object has bad “next” value

inspect 0x7fc1800ebf60 instance of Oop for java/lang/ref/WeakReference @ 0x00007fc1800ebf60 @ 0x00007fc1800ebf60 (size = 48) <<Reverse pointers>>: _mark: 140464753441755 _metadata._klass: InstanceKlass for java/lang/ref/WeakReference referent: Oop for java/lang/Thread @ 0x00007fc18006f248 Oop for java/lang/Thread @ 0x00007fc18006f248 queue: Oop for java/lang/ref/ReferenceQueue$Null @ 0x00007fbca100f7c8 Oop for java/lang/ref/ReferenceQueue$Null @ 0x00007fbca100f7c8 next: ** BAD OOP 7fb85457a40000 ** discovered: null null

the value for “next” is left shifted ( by 8 bits) address of JavaThread 0x00007fb85457a400 JavaThread “data-server-tcp-socket-worker-1301” daemon [_thread_in_native, id=214144, stack(0x00007fb2b9ac0000,0x00007fb2b9bc1000)]

the object 0x7fc1800ebf60 (java.lang.ref.WeakReference<Thread>) was created by netty Here is path to roots for this object (NOTE: visualvm shows null instead of bad OOP values)

path_to_roots2

I suspect the netty might have some issues when working with j.l.r.WeakReference<Thread> in Recycler.java

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Reactions:1
  • Comments:16 (11 by maintainers)

github_iconTop GitHub Comments

1reaction
VladimirKempikcommented, Jul 21, 2021

I will try, I don’t possess the reproducer tho.

1reaction
franz1981commented, Jul 15, 2021

I can confirm what @normanmaurer and @hyperxpro said here: on JCTools Jdk 11 just work with sun misc Unsafe OOTB

Read more comments on GitHub >

github_iconTop Results From Across the Web

NettyClientWorkerThread cpu 100% · Issue #11956 · netty/netty
issue It 's a strange issue. Tomcat application run normally A few hours until 3 NettyClientWorkerThread use 100% cpu. jstack is bellow.
Read more >
Java SDK memory leak with bad cluster - Couchbase Forums
Found memory leak in Java SDK 2.1.6. It happens only with specific conditions - when we got problems with a cluster.
Read more >
Netty 4.0.28.Final released
This release contains 60+ changes, including bug-fixes, ... (#3218); Change PoolThreadCache to use LIFO for better cache performance (#3553) ...
Read more >
Huge number of io.netty.buffer.PoolThreadCache ...
0 or later (which brings in Netty 4.1 Beta6 or later) resolves the issue. It is unclear whether the problem was a bug...
Read more >
Incident report on memory leak caused by Cloudflare parser bug
At that point it was no longer possible for memory to be returned in an HTTP response. Because of the seriousness of such...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found