Quarkus fails to start when using Kafka with error: "Unable to connect an emitter with the channel"
See original GitHub issueDescribe the bug
Hello,
Quarkus fails to start when using Kafka, with messaging emitters. The output is the following:
2022-02-04 20:17:24,558 ERROR [io.qua.dep.dev.IsolatedDevModeMain] (main) Failed to start quarkus: java.lang.RuntimeException: java.lang.RuntimeException: Failed to start quarkus
at io.quarkus.dev.appstate.ApplicationStateNotification.waitForApplicationStart(ApplicationStateNotification.java:51)
at io.quarkus.runner.bootstrap.StartupActionImpl.runMainClass(StartupActionImpl.java:122)
at io.quarkus.deployment.dev.IsolatedDevModeMain.firstStart(IsolatedDevModeMain.java:145)
at io.quarkus.deployment.dev.IsolatedDevModeMain.accept(IsolatedDevModeMain.java:456)
at io.quarkus.deployment.dev.IsolatedDevModeMain.accept(IsolatedDevModeMain.java:67)
at io.quarkus.bootstrap.app.CuratedApplication.runInCl(CuratedApplication.java:150)
at io.quarkus.bootstrap.app.CuratedApplication.runInAugmentClassLoader(CuratedApplication.java:106)
at io.quarkus.deployment.dev.DevModeMain.start(DevModeMain.java:132)
at io.quarkus.deployment.dev.DevModeMain.main(DevModeMain.java:62)
Caused by: java.lang.RuntimeException: Failed to start quarkus
at io.quarkus.runner.ApplicationImpl.doStart(Unknown Source)
at io.quarkus.runtime.Application.start(Application.java:101)
at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:104)
at io.quarkus.runtime.Quarkus.run(Quarkus.java:67)
at io.quarkus.runtime.Quarkus.run(Quarkus.java:41)
at io.quarkus.runtime.Quarkus.run(Quarkus.java:120)
at io.quarkus.runner.GeneratedMain.main(Unknown Source)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at io.quarkus.runner.bootstrap.StartupActionImpl$1.run(StartupActionImpl.java:103)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.RuntimeException: Error injecting org.eclipse.microprofile.reactive.messaging.Emitter<io.smallrye.reactive.messaging.kafka.Record<java.lang.String, java.lang.String>> com.example.app.service.OperationContextEventsService.operationContextEmitter
at com.example.app.service.OperationContextEventsService_Bean.create(Unknown Source)
at com.example.app.service.OperationContextEventsService_Bean.create(Unknown Source)
at io.quarkus.arc.impl.AbstractSharedContext.createInstanceHandle(AbstractSharedContext.java:101)
at io.quarkus.arc.impl.AbstractSharedContext$1.get(AbstractSharedContext.java:29)
at io.quarkus.arc.impl.AbstractSharedContext$1.get(AbstractSharedContext.java:26)
at io.quarkus.arc.impl.LazyValue.get(LazyValue.java:26)
at io.quarkus.arc.impl.ComputingCache.computeIfAbsent(ComputingCache.java:69)
at io.quarkus.arc.impl.AbstractSharedContext.get(AbstractSharedContext.java:26)
at io.quarkus.arc.impl.ClientProxies.getApplicationScopedDelegate(ClientProxies.java:18)
at com.example.app.service.OperationContextEventsService_ClientProxy.arc$delegate(Unknown Source)
at com.example.app.service.OperationContextEventsService_ClientProxy.arc_contextualInstance(Unknown Source)
at com.example.app.service.OperationContextEventsService_ProducerMethod_buildTopology_4f58f1097d120f9f4d1cca8a044213dc85ad958d_Bean.create(Unknown Source)
at com.example.app.service.OperationContextEventsService_ProducerMethod_buildTopology_4f58f1097d120f9f4d1cca8a044213dc85ad958d_Bean.get(Unknown Source)
at com.example.app.service.OperationContextEventsService_ProducerMethod_buildTopology_4f58f1097d120f9f4d1cca8a044213dc85ad958d_Bean.get(Unknown Source)
at io.quarkus.arc.impl.ArcContainerImpl.beanInstanceHandle(ArcContainerImpl.java:428)
at io.quarkus.arc.impl.ArcContainerImpl.beanInstanceHandle(ArcContainerImpl.java:441)
at io.quarkus.arc.impl.ArcContainerImpl.instanceHandle(ArcContainerImpl.java:412)
at io.quarkus.arc.impl.ArcContainerImpl.instance(ArcContainerImpl.java:229)
at io.quarkus.kafka.streams.runtime.TopologySupplier.get(TopologySupplier.java:13)
at io.quarkus.kafka.streams.runtime.TopologySupplier.get(TopologySupplier.java:9)
at io.quarkus.vertx.http.runtime.devmode.DevConsoleRecorder.addInfo(DevConsoleRecorder.java:33)
at io.quarkus.deployment.steps.DevConsoleProcessor$runtimeTemplates308161071.deploy_0(Unknown Source)
at io.quarkus.deployment.steps.DevConsoleProcessor$runtimeTemplates308161071.deploy(Unknown Source)
... 13 more
Caused by: javax.enterprise.inject.spi.DefinitionException: SRMSG00019: Unable to connect an emitter with the channel `operation-context`
at io.smallrye.reactive.messaging.providers.extension.ChannelProducer.getEmitter(ChannelProducer.java:191)
at io.smallrye.reactive.messaging.providers.extension.ChannelProducer.produceEmitter(ChannelProducer.java:136)
at io.smallrye.reactive.messaging.providers.extension.ChannelProducer_Subclass.produceEmitter$$superforward1(Unknown Source)
at io.smallrye.reactive.messaging.providers.extension.ChannelProducer_Subclass$$function$$12.apply(Unknown Source)
at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:54)
at io.quarkus.arc.runtime.devconsole.InvocationInterceptor.proceed(InvocationInterceptor.java:62)
at io.quarkus.arc.runtime.devconsole.InvocationInterceptor.monitor(InvocationInterceptor.java:51)
at io.quarkus.arc.runtime.devconsole.InvocationInterceptor_Bean.intercept(Unknown Source)
at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:41)
at io.quarkus.arc.impl.AroundInvokeInvocationContext.perform(AroundInvokeInvocationContext.java:41)
at io.quarkus.arc.impl.InvocationContexts.performAroundInvoke(InvocationContexts.java:32)
at io.smallrye.reactive.messaging.providers.extension.ChannelProducer_Subclass.produceEmitter(Unknown Source)
at io.smallrye.reactive.messaging.providers.extension.ChannelProducer_ProducerMethod_produceEmitter_86c8bb1e4975aaf33dff96bec0d2fc8fa44920ab_Bean.create(Unknown Source)
at io.smallrye.reactive.messaging.providers.extension.ChannelProducer_ProducerMethod_produceEmitter_86c8bb1e4975aaf33dff96bec0d2fc8fa44920ab_Bean.get(Unknown Source)
at io.smallrye.reactive.messaging.providers.extension.ChannelProducer_ProducerMethod_produceEmitter_86c8bb1e4975aaf33dff96bec0d2fc8fa44920ab_Bean.get(Unknown Source)
at io.quarkus.arc.impl.CurrentInjectionPointProvider.get(CurrentInjectionPointProvider.java:52)
... 36 more
--
It happens both with external and built-in redpanda test Kafka cluster, but not always, only ~7 times out of 10 startups.
An interesting thing I noticed on my machine is that the issue happens less frequently when laptop is not plugged in, or is in power saving mode. This leads me to believe it’s something related to resources not being available and loaded yet when some other service expects them to be. Since it’s timing related, it might be similar to https://github.com/quarkusio/quarkus/issues/17841 ?
Reproducer repo: https://github.com/peterhuba/quarkus-kafka-emitter-issue
Expected behavior
Quarkus starts up, and Kafka messaging works
Actual behavior
Quarkus fails to start, ~7 times out of 10. When it does start, it works without issues.
How to Reproduce?
- Clone https://github.com/peterhuba/quarkus-kafka-emitter-issue
- run
quarkus dev
, might need to re-run several times
If want to use an external Kafka cluster instead of the one used internally, set the quarkus.kafka.devservices.enabled
flag to false
Output of uname -a
or ver
Linux devbox 5.4.0-97-generic #110~18.04.1-Ubuntu SMP Mon Jan 17 20:50:31 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Output of java -version
openjdk version “11.0.13” 2021-10-19 OpenJDK Runtime Environment (build 11.0.13+8-Ubuntu-0ubuntu1.18.04) OpenJDK 64-Bit Server VM (build 11.0.13+8-Ubuntu-0ubuntu1.18.04, mixed mode, sharing)
GraalVM version (if different from Java)
it was run in Java mode only
Quarkus version or git rev
2.7.0.Final
Build tool (ie. output of mvnw --version
or gradlew --version
)
Apache Maven 3.8.4 (9b656c72d54e5bacbed989b64718c159fe39b537)
Additional information
No response
Issue Analytics
- State:
- Created 2 years ago
- Comments:7 (2 by maintainers)
@peterhuba Thanks for the reproducer.
I am not sure of the fix but I think I understand why it happens. Kafka Streams extension Dev UI tries to get the Topology object to visualize the stream topology. This in turn triggers the initialization of CDI injected beans such as Emitters, but these last are initialized lazily, just before the Application startup.
The workaround I tested is to move buildTopology, updatePaymentContext and injected properties to its own ApplicationScoped bean.
Could you confirm that this resolves the issue?
Having similar issue to the stated above.
This will cause a failure on startup that is not fatal.
Delaying seems to help but it’d be nice if there was a more definite fix.