Regression between 2.13.0.Final and main - Something in reactive messaging/Mutiny Context/manual ack'ing is broken

Describe the bug

The Quarkus Superheroes ecosystem CI failed this morning (https://github.com/quarkusio/quarkus/issues/23612#issuecomment-1261806651). The build run can be viewed at https://github.com/quarkusio/quarkus-super-heroes/actions/runs/3148883820.

Upon initial investigation is seems something has changed between Quarkus 2.13.0.Final and what is currently on main.

Some of the tests in the Superheroes event-statistics service are now failing:

4pS2DHvMeaN1_ojR7eO4haW8ao] - Writing message [{"name":"Chewbacca","score":1}]
08:23:56 INFO  [io.qu.sa.su.st.en.WebSocketsIT$EndpointTestClient] (globalEventExecutor-1-3) Got message: [{"name":"Chewbacca","score":1}]
java.lang.Exception: Missing onFailure/onError handler in the subscriber
        at io.smallrye.mutiny.subscription.Subscribers.lambda$static$0(Subscribers.java:19)
        at io.smallrye.mutiny.subscription.Subscribers$CallbackBasedSubscriber.onFailure(Subscribers.java:94)
        at io.smallrye.mutiny.operators.multi.MultiOperatorProcessor.onFailure(MultiOperatorProcessor.java:89)
        at io.smallrye.mutiny.operators.multi.replay.ReplayOperator$ReplaySubscription.drain(ReplayOperator.java:111)
        at io.smallrye.mutiny.operators.multi.replay.ReplayOperator$ReplaySubscription.access$400(ReplayOperator.java:47)
        at io.smallrye.mutiny.operators.multi.replay.ReplayOperator$UpstreamSubscriber.lambda$triggerDrainLoops$0(ReplayOperator.java:171)
        at java.base/java.util.concurrent.CopyOnWriteArrayList.forEach(CopyOnWriteArrayList.java:807)
        at io.smallrye.mutiny.operators.multi.replay.ReplayOperator$UpstreamSubscriber.triggerDrainLoops(ReplayOperator.java:171)
        at io.smallrye.mutiny.operators.multi.replay.ReplayOperator$UpstreamSubscriber.onFailure(ReplayOperator.java:145)
        at io.smallrye.mutiny.subscription.MultiSubscriber.onError(MultiSubscriber.java:73)
        at io.smallrye.mutiny.operators.multi.multicast.MultiPublishOp$PublishSubscriber.isEmptyOrCompleted(MultiPublishOp.java:342)
        at io.smallrye.mutiny.operators.multi.multicast.MultiPublishOp$PublishSubscriber.drain(MultiPublishOp.java:393)
        at io.smallrye.mutiny.operators.multi.multicast.MultiPublishOp$PublishSubscriber.onFailure(MultiPublishOp.java:200)
        at io.smallrye.mutiny.operators.multi.MultiOperatorProcessor.onFailure(MultiOperatorProcessor.java:89)
        at io.smallrye.mutiny.operators.multi.MultiFlatMapOp$FlatMapMainSubscriber.handleTerminationIfDone(MultiFlatMapOp.java:513)
        at io.smallrye.mutiny.operators.multi.MultiFlatMapOp$FlatMapMainSubscriber.ifDoneOrCancelled(MultiFlatMapOp.java:488)
        at io.smallrye.mutiny.operators.multi.MultiFlatMapOp$FlatMapMainSubscriber.drainLoop(MultiFlatMapOp.java:288)
        at io.smallrye.mutiny.operators.multi.MultiFlatMapOp$FlatMapMainSubscriber.drain(MultiFlatMapOp.java:268)
        at io.smallrye.mutiny.operators.multi.MultiFlatMapOp$FlatMapMainSubscriber.onFailure(MultiFlatMapOp.java:212)
        at io.smallrye.mutiny.operators.multi.MultiOperatorProcessor.onFailure(MultiOperatorProcessor.java:89)
        at io.smallrye.mutiny.subscription.MultiSubscriber.onError(MultiSubscriber.java:73)
        at io.smallrye.mutiny.subscription.SafeSubscriber.onError(SafeSubscriber.java:148)
        at io.smallrye.mutiny.helpers.HalfSerializer.onError(HalfSerializer.java:56)
        at io.smallrye.mutiny.helpers.StrictMultiSubscriber.onFailure(StrictMultiSubscriber.java:91)
        at io.smallrye.mutiny.subscription.MultiSubscriber.onError(MultiSubscriber.java:73)
        at io.smallrye.mutiny.operators.multi.multicast.MultiPublishOp$PublishSubscriber.isEmptyOrCompleted(MultiPublishOp.java:342)
        at io.smallrye.mutiny.operators.multi.multicast.MultiPublishOp$PublishSubscriber.drain(MultiPublishOp.java:393)
        at io.smallrye.mutiny.operators.multi.multicast.MultiPublishOp$PublishSubscriber.onFailure(MultiPublishOp.java:200)
        at io.smallrye.mutiny.operators.multi.MultiOperatorProcessor.onFailure(MultiOperatorProcessor.java:89)
        at io.smallrye.reactive.messaging.providers.locals.ContextDecorator$ContextMulti$ContextProcessor.onFailure(ContextDecorator.java:68)
        at io.smallrye.mutiny.subscription.MultiSubscriber.onError(MultiSubscriber.java:73)
        at io.smallrye.mutiny.subscription.SafeSubscriber.onError(SafeSubscriber.java:148)
        at io.smallrye.mutiny.helpers.HalfSerializer.onError(HalfSerializer.java:56)
        at io.smallrye.mutiny.helpers.StrictMultiSubscriber.onFailure(StrictMultiSubscriber.java:91)
        at io.smallrye.mutiny.operators.multi.MultiOperatorProcessor.onFailure(MultiOperatorProcessor.java:89)
        at io.smallrye.mutiny.operators.multi.MultiFlatMapOp$FlatMapMainSubscriber.handleTerminationIfDone(MultiFlatMapOp.java:513)
        at io.smallrye.mutiny.operators.multi.MultiFlatMapOp$FlatMapMainSubscriber.onItem(MultiFlatMapOp.java:187)
        at io.smallrye.mutiny.operators.multi.MultiOperatorProcessor.onItem(MultiOperatorProcessor.java:100)
        at io.smallrye.mutiny.operators.multi.MultiOnItemInvoke$MultiOnItemInvokeProcessor.onItem(MultiOnItemInvoke.java:41)
        at io.smallrye.mutiny.operators.multi.MultiOperatorProcessor.onItem(MultiOperatorProcessor.java:100)
        at io.smallrye.mutiny.operators.multi.MultiOnItemInvoke$MultiOnItemInvokeProcessor.onItem(MultiOnItemInvoke.java:41)
        at io.smallrye.mutiny.operators.multi.MultiOnItemInvoke$MultiOnItemInvokeProcessor.onItem(MultiOnItemInvoke.java:41)
        at io.smallrye.mutiny.operators.multi.MultiMapOp$MapProcessor.onItem(MultiMapOp.java:50)
        at io.smallrye.mutiny.subscription.MultiSubscriber.onNext(MultiSubscriber.java:61)
        at io.smallrye.mutiny.operators.multi.MultiFlatMapOp$FlatMapMainSubscriber.tryEmit(MultiFlatMapOp.java:230)
        at io.smallrye.mutiny.operators.multi.MultiFlatMapOp$FlatMapInner.onItem(MultiFlatMapOp.java:607)
        at io.smallrye.mutiny.operators.multi.MultiSelectWhereOp$MultiSelectWhereProcessor.onItem(MultiSelectWhereOp.java:58)
        at io.smallrye.mutiny.operators.multi.MultiScanWithSeedOp$ScanSubscriber.onItem(MultiScanWithSeedOp.java:108)
        at io.smallrye.mutiny.operators.multi.MultiScanWithSeedOp$ScanSeedProcessor.onItem(MultiScanWithSeedOp.java:154)
        at io.smallrye.mutiny.subscription.MultiSubscriber.onNext(MultiSubscriber.java:61)
        at io.smallrye.mutiny.operators.multi.MultiGroupByOp$State.drain(MultiGroupByOp.java:400)
        at io.smallrye.mutiny.operators.multi.MultiGroupByOp$State.request(MultiGroupByOp.java:329)
        at io.smallrye.mutiny.operators.multi.MultiOperatorProcessor.request(MultiOperatorProcessor.java:120)
        at io.smallrye.mutiny.subscription.SwitchableSubscriptionSubscriber.setOrSwitchUpstream(SwitchableSubscriptionSubscriber.java:206)
        at io.smallrye.mutiny.subscription.SwitchableSubscriptionSubscriber.onSubscribe(SwitchableSubscriptionSubscriber.java:108)
        at io.smallrye.mutiny.operators.multi.MultiOperatorProcessor.onSubscribe(MultiOperatorProcessor.java:79)
        at io.smallrye.mutiny.operators.multi.MultiGroupByOp$State.subscribe(MultiGroupByOp.java:344)
        at io.smallrye.mutiny.operators.multi.MultiGroupByOp$GroupedUnicast.subscribe(MultiGroupByOp.java:282)
        at io.smallrye.mutiny.operators.AbstractMulti.subscribe(AbstractMulti.java:40)
        at io.smallrye.mutiny.operators.multi.MultiScanWithSeedOp$ScanSubscriber.onCompletion(MultiScanWithSeedOp.java:94)
        at io.smallrye.mutiny.operators.multi.MultiScanWithSeedOp.subscribe(MultiScanWithSeedOp.java:33)
        at io.smallrye.mutiny.groups.MultiSubscribe.withSubscriber(MultiSubscribe.java:72)
        at io.smallrye.mutiny.operators.multi.MultiSelectWhereOp.subscribe(MultiSelectWhereOp.java:30)
        at io.smallrye.mutiny.operators.AbstractMulti.subscribe(AbstractMulti.java:40)
        at io.smallrye.mutiny.operators.multi.MultiFlatMapOp$FlatMapMainSubscriber.onItem(MultiFlatMapOp.java:193)
        at io.smallrye.mutiny.operators.multi.MultiGroupByOp$MultiGroupByProcessor.drain(MultiGroupByOp.java:220)
        at io.smallrye.mutiny.operators.multi.MultiGroupByOp$MultiGroupByProcessor.onItem(MultiGroupByOp.java:133)
        at io.smallrye.mutiny.operators.multi.MultiOnItemInvoke$MultiOnItemInvokeProcessor.onItem(MultiOnItemInvoke.java:41)
        at io.smallrye.mutiny.operators.multi.MultiOnItemInvoke$MultiOnItemInvokeProcessor.onItem(MultiOnItemInvoke.java:41)
        at io.smallrye.mutiny.subscription.MultiSubscriber.onNext(MultiSubscriber.java:61)
        at io.smallrye.mutiny.operators.multi.MultiFlatMapOp$FlatMapMainSubscriber.tryEmit(MultiFlatMapOp.java:230)
        at io.smallrye.mutiny.operators.multi.MultiFlatMapOp$FlatMapInner.onItem(MultiFlatMapOp.java:607)
        at io.smallrye.mutiny.subscription.MultiSubscriber.onNext(MultiSubscriber.java:61)
        at io.smallrye.mutiny.converters.uni.UniToMultiPublisher$UniToMultiSubscription.onItem(UniToMultiPublisher.java:93)
        at io.smallrye.mutiny.operators.uni.UniOnItemOrFailureFlatMap$UniOnItemOrFailureFlatMapProcessor.onItem(UniOnItemOrFailureFlatMap.java:56)
        at io.smallrye.mutiny.operators.uni.builders.UniCreateFromKnownItem$KnownItemSubscription.forward(UniCreateFromKnownItem.java:38)
        at io.smallrye.mutiny.operators.uni.builders.UniCreateFromKnownItem$KnownItemSubscription.access$100(UniCreateFromKnownItem.java:26)
        at io.smallrye.mutiny.operators.uni.builders.UniCreateFromKnownItem.subscribe(UniCreateFromKnownItem.java:23)
        at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:36)
        at io.smallrye.mutiny.operators.uni.UniOnItemOrFailureFlatMap$UniOnItemOrFailureFlatMapProcessor.performInnerSubscription(UniOnItemOrFailureFlatMap.java:99)
        at io.smallrye.mutiny.operators.uni.UniOnItemOrFailureFlatMap$UniOnItemOrFailureFlatMapProcessor.onItem(UniOnItemOrFailureFlatMap.java:54)
        at io.smallrye.mutiny.operators.uni.UniOnItemTransformToUni$UniOnItemTransformToUniProcessor.onItem(UniOnItemTransformToUni.java:60)
        at io.smallrye.mutiny.operators.uni.UniEmitOn$UniEmitOnProcessor.lambda$onItem$0(UniEmitOn.java:34)
        at io.smallrye.reactive.messaging.providers.extension.MutinyEmitterImpl.lambda$sendMessage$5(MutinyEmitterImpl.java:88)
        at io.vertx.core.impl.ContextInternal.dispatch(ContextInternal.java:264)
        at io.vertx.core.impl.ContextInternal.dispatch(ContextInternal.java:246)
        at io.vertx.core.impl.EventLoopContext.lambda$runOnContext$0(EventLoopContext.java:43)
        at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:569)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.util.NoSuchElementException: The context does not have a value for key CURRENT_MESSAGE
        at io.smallrye.mutiny.Context.get(Context.java:121)
        at io.quarkus.sample.superheroes.statistics.listener.SuperStats.ackMessage(SuperStats.java:143)
        at io.quarkus.sample.superheroes.statistics.listener.SuperStats.lambda$processFight$2(SuperStats.java:70)
        at io.smallrye.context.impl.wrappers.SlowContextualSupplier.get(SlowContextualSupplier.java:21)
        at io.smallrye.mutiny.groups.MultiOnItem.lambda$call$3(MultiOnItem.java:128)
        at io.smallrye.context.impl.wrappers.SlowContextualFunction.apply(SlowContextualFunction.java:21)
        at io.smallrye.mutiny.groups.MultiOnItem.lambda$call$2(MultiOnItem.java:95)
        at io.smallrye.context.impl.wrappers.SlowContextualFunction.apply(SlowContextualFunction.java:21)
        at io.smallrye.mutiny.groups.MultiOnItem.lambda$transformToUni$6(MultiOnItem.java:268)
        at io.smallrye.mutiny.operators.multi.MultiFlatMapOp$FlatMapMainSubscriber.onItem(MultiFlatMapOp.java:178)

Something in https://github.com/quarkusio/quarkus-super-heroes/blob/main/event-statistics/src/main/java/io/quarkus/sample/superheroes/statistics/listener/SuperStats.java isn’t working now. The class does manual ack’ing of messages and uses the Mutiny Context for storing off the message to be ack’ed so it can be retrieved later once processing of the message has completed. From my initial investigation it seems the Context is losing the message somehow, but i’m certainly no Kafka, reactive messaging, nor Mutiny expert.

Expected behavior

I would expect all the tests to pass.

Actual behavior

No response

How to Reproduce?

1. Clone quarkusio/quarkus main, cd quarkus, and build (mvn -Dquickly)
2. Clone quarkusio/quarkus-super-heroes main
3. cd into quarkus-super-heroes/event-statistics
4. Run ./mvnw clean verify -Dquarkus.platform.group-id=io.quarkus -Dquarkus.platform.version=999-SNAPSHOT -Dquarkus.http.host=0.0.0.0
5. The tests will fail with the error described above.

Output of uname -a or ver

No response

Output of java -version

No response

GraalVM version (if different from Java)

No response

Quarkus version or git rev

No response

Build tool (ie. output of mvnw --version or gradlew --version)

No response

Additional information

No response

[ozangunalp]

The fix is on https://github.com/quarkusio/quarkus-super-heroes/pull/151

[cescoffier]

Closing - we’re back to green!