Is it legal to produce events from the background consumer thread using ProducerType.MULTI?

I’m trying to debug an issue where the queue appears to be full, but my background thread is in runnable state spinning on at com.lmax.disruptor.BatchEventProcessor.processEvents(BatchEventProcessor.java:190) without making any progress. I’m curious if there’s a memory safety issue when the consumer thread adds elements to the queue, but I don’t see this mentioned in the documentation.

The issue is odd because the background thread never appears to do a great deal of processing, and the queue grows slowly over time as through the state has become lost.

Any information or ideas are greatly appreciated, thanks!

[grumpyjames]

Been thinking on this over the weekend. Haven’t got any answers, but a few more questions.

It seems odd that we constantly pick up the read thread on the line from a closing brace. That seems like an unlikely outcome to me. What is that thread doing? Appending log messages to…somewhere? I’d expect to find the thread somewhere in client code rather than spinning in BatchEventProcessor - if repeated stack traces always find us at that closing brace, that is interesting.

Actions:

• We see the ‘read’ thread spinning, and the ring buffer appears to be constantly full. Can you tell us more about the metrics captured here? I’d be interested to know if, for instance, the read side’s sequence was still going up. If there were charts of read sequence/write sequence/depth over time you could share, that would be terrific 😃
• We should probably look into how a thread could get stuck at that brace. What’s actually executing there? I’d start just by doing some experiments with a debugger attached.
• The other possibility is that our jvm implementation is not telling us the whole truth. My knowledge here is out of date; I will attempt to remedy this…

If I trust that stack trace, this has the feeling of a resource bottleneck hurting the read thread in some way more than a memory visibility issue. If we’d found the read thread blocking on something near the sequence classes, I’d be more tempted towards the latter.

On that note - is it worth attempting to create a test program that simulates the usage of this ringbuffer, potentially with a smaller buffer/higher input rate to see if this is reproducible? That might give us a bit more hope 😃

[rrossiersf]

Hello, I am having the exact same issue. Very hard to reproduce and it is happening rarely (every 6 months) , but when it does it is a big problem for me. Publisher is reporting full buffer , but on the other side the reader is happily spinning not picking any new events. I am also into a multi producers , single consumer scenario. Not sure if it is going to help solve this problem but it is a problem that shows up on another system.

“pool-2-thread-17” #56 prio=5 os_prio=0 tid=0x000055d4a02d4800 nid=0x3e2a1 waiting on condition [0x00007f72dcad6000] : java.lang.Thread.State: TIMED_WAITING (parking) : at sun.misc.Unsafe.park(Native Method) : at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:338) : at com.lmax.disruptor.SleepingWaitStrategy.applyWaitMethod(SleepingWaitStrategy.java:92) : at com.lmax.disruptor.SleepingWaitStrategy.waitFor(SleepingWaitStrategy.java:65) : at com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(ProcessingSequenceBarrier.java:56) : at com.lmax.disruptor.BatchEventProcessor.processEvents(BatchEventProcessor.java:159) : at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:125) : at java.lang.Thread.run(Thread.java:748)