Siddhi CEP 3.1.0 hit deadlock condition on pattern query execution

I recently moved to Siddhi CEP 3.1.0 version. Executing this plan at the rate of 400 events/second. After 3 hours, found events are not processed as Siddhi threads are is in deadlock condition

"@Plan:name(‘cepexecutionplan’) " + "define stream deviceCheckInStream (serialNum string,currentCheckInTime long, sequenceNum long); " + "define stream swCompCheckInStream (serialNum string, timestamp long, sequenceNum long, info1 object, info2 object, streamType string); " +

     "@info(name = 'query1') " +
     " from deviceCheckInStream#window.time(6 minutes) " + 
     "select * "+                   
     " insert expired events into delayedDeviceCheckInStream; " + 

     "@info(name = 'query2') " + 
     " from (every e1=deviceCheckInStream -> nonOccurringEvent = deviceCheckInStream[sequenceNum > e1.sequenceNum and serialNum == e1.serialNum] or"
     + " delayedEvent=delayedDeviceCheckInStream[e1.sequenceNum ==  sequenceNum and serialNum == e1.serialNum]) within 370 sec "+ 
     " select e1.serialNum as serialNum, e1.currentCheckInTime as lastSuccessfulCheckInTime,  "
     + "nonOccurringEvent.currentCheckInTime as nonOccurringId, e1.sequenceNum as lastSuccessfulCheckInSeqNum  "+ 
     " having (nonOccurringId is null) "+ 
     " insert into devCheckInNonOccurrenceStream; " +

     "@info(name = 'query3') " +  
     " from (every e1=swCompCheckInStream -> e2 = swCompCheckInStream[sequenceNum != e1.sequenceNum and serialNum == e1.serialNum]) within 6 min "+ 
     " select e2.sequenceNum as currSeq, e1.serialNum, e2.timestamp, e1.streamType as prevStreamType, e2.streamType as currStreamType, "
     + "e1.info1 as previnfo1, e2.info1 as currinfo1, "
     + "e1.info2 as previnfo2, e2.info2 as currinfo2 "
     + " insert into swCompStatusOutputStream; ";`

Lock obtained in PatternMultiProcessStreamReceiver is stuck in getLastEvent, below is the thread dump traces. Any idea when this could happen? Any help on this?

at org.wso2.siddhi.core.event.ComplexEventChunk.getLastEvent(ComplexEventChunk.java:107) at org.wso2.siddhi.core.event.ComplexEventChunk.add(ComplexEventChunk.java:100) at org.wso2.siddhi.core.query.input.stream.state.LogicalPreStateProcessor.processAndReturn(LogicalPreStateProcessor.java:124) at org.wso2.siddhi.core.query.input.StateMultiProcessStreamReceiver.processAndClear(StateMultiProcessStreamReceiver.java:49) at org.wso2.siddhi.core.query.input.MultiProcessStreamReceiver.process(MultiProcessStreamReceiver.java:75) at org.wso2.siddhi.core.query.input.MultiProcessStreamReceiver.receive(MultiProcessStreamReceiver.java:112)

• locked <0x0000000084260b08> (a org.wso2.siddhi.core.query.input.stream.state.receiver.PatternMultiProcessStreamReceiver) at org.wso2.siddhi.core.query.input.stream.state.receiver.PatternMultiProcessStreamReceiver.receive(PatternMultiProcessStreamReceiver.java:58) at org.wso2.siddhi.core.stream.StreamJunction.sendEvent(StreamJunction.java:149) at org.wso2.siddhi.core.stream.StreamJunction$Publisher.send(StreamJunction.java:334) at org.wso2.siddhi.core.stream.input.InputDistributor.send(InputDistributor.java:34) at org.wso2.siddhi.core.stream.input.InputEntryValve.send(InputEntryValve.java:41) at org.wso2.siddhi.core.stream.input.InputHandler.send(InputHandler.java:57) at

[mohanvive]

@donkeysharp Thanks. Give us some time. We’ll have a look and get back…

[suhothayan]

I was able to reproduce the deadlock. This is happening because, the query output JoinStream is also used as an input to the same query. And the code is not written in a way to handle this case.

@info(name='Data together')
from every (Type1 = Type1Stream) -> every Type2 = Type2Stream[Type1.id != Type2.id] within 15 seconds -> not JoinStream[Type1.id == id1 and Type2.id == id2 and (Type1.ts == ts or Type2.ts == ts)] for 15 seconds
select Type1.id as id1, Type2.id as id2, Type1.ts
insert into JoinStream;

Also siddhi does not support adding within <time> in the middle of the pattern query yet. In Siddhi 4.2.20 it is also not properly validated.

This issue is being fixed in ff9f0bee66a36f0a3704284c035063721ac3a521 I ran the same test in 5.0.0 and it works without any issues.

Closing this, as the fix is already done to the master. If you need a fix for 3.x.x please reopen this issue.