question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Quarkus fails to start when using Kafka with error: "Unable to connect an emitter with the channel"

See original GitHub issue

Describe the bug

Hello,

Quarkus fails to start when using Kafka, with messaging emitters. The output is the following:

2022-02-04 20:17:24,558 ERROR [io.qua.dep.dev.IsolatedDevModeMain] (main) Failed to start quarkus: java.lang.RuntimeException: java.lang.RuntimeException: Failed to start quarkus
	at io.quarkus.dev.appstate.ApplicationStateNotification.waitForApplicationStart(ApplicationStateNotification.java:51)
	at io.quarkus.runner.bootstrap.StartupActionImpl.runMainClass(StartupActionImpl.java:122)
	at io.quarkus.deployment.dev.IsolatedDevModeMain.firstStart(IsolatedDevModeMain.java:145)
	at io.quarkus.deployment.dev.IsolatedDevModeMain.accept(IsolatedDevModeMain.java:456)
	at io.quarkus.deployment.dev.IsolatedDevModeMain.accept(IsolatedDevModeMain.java:67)
	at io.quarkus.bootstrap.app.CuratedApplication.runInCl(CuratedApplication.java:150)
	at io.quarkus.bootstrap.app.CuratedApplication.runInAugmentClassLoader(CuratedApplication.java:106)
	at io.quarkus.deployment.dev.DevModeMain.start(DevModeMain.java:132)
	at io.quarkus.deployment.dev.DevModeMain.main(DevModeMain.java:62)
Caused by: java.lang.RuntimeException: Failed to start quarkus
	at io.quarkus.runner.ApplicationImpl.doStart(Unknown Source)
	at io.quarkus.runtime.Application.start(Application.java:101)
	at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:104)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:67)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:41)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:120)
	at io.quarkus.runner.GeneratedMain.main(Unknown Source)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at io.quarkus.runner.bootstrap.StartupActionImpl$1.run(StartupActionImpl.java:103)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.RuntimeException: Error injecting org.eclipse.microprofile.reactive.messaging.Emitter<io.smallrye.reactive.messaging.kafka.Record<java.lang.String, java.lang.String>> com.example.app.service.OperationContextEventsService.operationContextEmitter
	at com.example.app.service.OperationContextEventsService_Bean.create(Unknown Source)
	at com.example.app.service.OperationContextEventsService_Bean.create(Unknown Source)
	at io.quarkus.arc.impl.AbstractSharedContext.createInstanceHandle(AbstractSharedContext.java:101)
	at io.quarkus.arc.impl.AbstractSharedContext$1.get(AbstractSharedContext.java:29)
	at io.quarkus.arc.impl.AbstractSharedContext$1.get(AbstractSharedContext.java:26)
	at io.quarkus.arc.impl.LazyValue.get(LazyValue.java:26)
	at io.quarkus.arc.impl.ComputingCache.computeIfAbsent(ComputingCache.java:69)
	at io.quarkus.arc.impl.AbstractSharedContext.get(AbstractSharedContext.java:26)
	at io.quarkus.arc.impl.ClientProxies.getApplicationScopedDelegate(ClientProxies.java:18)
	at com.example.app.service.OperationContextEventsService_ClientProxy.arc$delegate(Unknown Source)
	at com.example.app.service.OperationContextEventsService_ClientProxy.arc_contextualInstance(Unknown Source)
	at com.example.app.service.OperationContextEventsService_ProducerMethod_buildTopology_4f58f1097d120f9f4d1cca8a044213dc85ad958d_Bean.create(Unknown Source)
	at com.example.app.service.OperationContextEventsService_ProducerMethod_buildTopology_4f58f1097d120f9f4d1cca8a044213dc85ad958d_Bean.get(Unknown Source)
	at com.example.app.service.OperationContextEventsService_ProducerMethod_buildTopology_4f58f1097d120f9f4d1cca8a044213dc85ad958d_Bean.get(Unknown Source)
	at io.quarkus.arc.impl.ArcContainerImpl.beanInstanceHandle(ArcContainerImpl.java:428)
	at io.quarkus.arc.impl.ArcContainerImpl.beanInstanceHandle(ArcContainerImpl.java:441)
	at io.quarkus.arc.impl.ArcContainerImpl.instanceHandle(ArcContainerImpl.java:412)
	at io.quarkus.arc.impl.ArcContainerImpl.instance(ArcContainerImpl.java:229)
	at io.quarkus.kafka.streams.runtime.TopologySupplier.get(TopologySupplier.java:13)
	at io.quarkus.kafka.streams.runtime.TopologySupplier.get(TopologySupplier.java:9)
	at io.quarkus.vertx.http.runtime.devmode.DevConsoleRecorder.addInfo(DevConsoleRecorder.java:33)
	at io.quarkus.deployment.steps.DevConsoleProcessor$runtimeTemplates308161071.deploy_0(Unknown Source)
	at io.quarkus.deployment.steps.DevConsoleProcessor$runtimeTemplates308161071.deploy(Unknown Source)
	... 13 more
Caused by: javax.enterprise.inject.spi.DefinitionException: SRMSG00019: Unable to connect an emitter with the channel `operation-context`
	at io.smallrye.reactive.messaging.providers.extension.ChannelProducer.getEmitter(ChannelProducer.java:191)
	at io.smallrye.reactive.messaging.providers.extension.ChannelProducer.produceEmitter(ChannelProducer.java:136)
	at io.smallrye.reactive.messaging.providers.extension.ChannelProducer_Subclass.produceEmitter$$superforward1(Unknown Source)
	at io.smallrye.reactive.messaging.providers.extension.ChannelProducer_Subclass$$function$$12.apply(Unknown Source)
	at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:54)
	at io.quarkus.arc.runtime.devconsole.InvocationInterceptor.proceed(InvocationInterceptor.java:62)
	at io.quarkus.arc.runtime.devconsole.InvocationInterceptor.monitor(InvocationInterceptor.java:51)
	at io.quarkus.arc.runtime.devconsole.InvocationInterceptor_Bean.intercept(Unknown Source)
	at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:41)
	at io.quarkus.arc.impl.AroundInvokeInvocationContext.perform(AroundInvokeInvocationContext.java:41)
	at io.quarkus.arc.impl.InvocationContexts.performAroundInvoke(InvocationContexts.java:32)
	at io.smallrye.reactive.messaging.providers.extension.ChannelProducer_Subclass.produceEmitter(Unknown Source)
	at io.smallrye.reactive.messaging.providers.extension.ChannelProducer_ProducerMethod_produceEmitter_86c8bb1e4975aaf33dff96bec0d2fc8fa44920ab_Bean.create(Unknown Source)
	at io.smallrye.reactive.messaging.providers.extension.ChannelProducer_ProducerMethod_produceEmitter_86c8bb1e4975aaf33dff96bec0d2fc8fa44920ab_Bean.get(Unknown Source)
	at io.smallrye.reactive.messaging.providers.extension.ChannelProducer_ProducerMethod_produceEmitter_86c8bb1e4975aaf33dff96bec0d2fc8fa44920ab_Bean.get(Unknown Source)
	at io.quarkus.arc.impl.CurrentInjectionPointProvider.get(CurrentInjectionPointProvider.java:52)
	... 36 more



--

It happens both with external and built-in redpanda test Kafka cluster, but not always, only ~7 times out of 10 startups.

An interesting thing I noticed on my machine is that the issue happens less frequently when laptop is not plugged in, or is in power saving mode. This leads me to believe it’s something related to resources not being available and loaded yet when some other service expects them to be. Since it’s timing related, it might be similar to https://github.com/quarkusio/quarkus/issues/17841 ?

Reproducer repo: https://github.com/peterhuba/quarkus-kafka-emitter-issue

Expected behavior

Quarkus starts up, and Kafka messaging works

Actual behavior

Quarkus fails to start, ~7 times out of 10. When it does start, it works without issues.

How to Reproduce?

If want to use an external Kafka cluster instead of the one used internally, set the quarkus.kafka.devservices.enabled flag to false

Output of uname -a or ver

Linux devbox 5.4.0-97-generic #110~18.04.1-Ubuntu SMP Mon Jan 17 20:50:31 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Output of java -version

openjdk version “11.0.13” 2021-10-19 OpenJDK Runtime Environment (build 11.0.13+8-Ubuntu-0ubuntu1.18.04) OpenJDK 64-Bit Server VM (build 11.0.13+8-Ubuntu-0ubuntu1.18.04, mixed mode, sharing)

GraalVM version (if different from Java)

it was run in Java mode only

Quarkus version or git rev

2.7.0.Final

Build tool (ie. output of mvnw --version or gradlew --version)

Apache Maven 3.8.4 (9b656c72d54e5bacbed989b64718c159fe39b537)

Additional information

No response

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:7 (2 by maintainers)

github_iconTop GitHub Comments

2reactions
ozangunalpcommented, Feb 7, 2022

@peterhuba Thanks for the reproducer.

I am not sure of the fix but I think I understand why it happens. Kafka Streams extension Dev UI tries to get the Topology object to visualize the stream topology. This in turn triggers the initialization of CDI injected beans such as Emitters, but these last are initialized lazily, just before the Application startup.

The workaround I tested is to move buildTopology, updatePaymentContext and injected properties to its own ApplicationScoped bean.

Could you confirm that this resolves the issue?

1reaction
jkbbwrcommented, Feb 7, 2022

Having similar issue to the stated above.

data class Person(val name: String)

@ApplicationScoped
class TestConsumer(@Channel("reply") val reply: Emitter<Person>) {
    @Scheduled(every = "10s")
    fun produce() {
        reply.send(Person("me"))
    }
}

This will cause a failure on startup that is not fatal.

2022-02-07 15:10:07,003 ERROR [io.qua.sch.run.SimpleScheduler] (executor-thread-0) Error occured while executing task for trigger IntervalTrigger [id=1_dev.kibb.TestConsumer_ScheduledInvoker_produce_46db73690223bcc321054a6fc5b9736161d81e88, interval=10000]: javax.enterprise.inject.spi.DefinitionException: SRMSG00019: Unable to connect an emitter with the channel `reply`
	at io.smallrye.reactive.messaging.providers.extension.ChannelProducer.getEmitter(ChannelProducer.java:191)
	at io.smallrye.reactive.messaging.providers.extension.ChannelProducer.produceEmitter(ChannelProducer.java:136)
	at io.smallrye.reactive.messaging.providers.extension.ChannelProducer_Subclass.produceEmitter$$superforward1(Unknown Source)
	at io.smallrye.reactive.messaging.providers.extension.ChannelProducer_Subclass$$function$$12.apply(Unknown Source)
	at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:54)
	at io.quarkus.arc.runtime.devconsole.InvocationInterceptor.proceed(InvocationInterceptor.java:62)
	at io.quarkus.arc.runtime.devconsole.InvocationInterceptor.monitor(InvocationInterceptor.java:49)
	at io.quarkus.arc.runtime.devconsole.InvocationInterceptor_Bean.intercept(Unknown Source)
	at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:41)
	at io.quarkus.arc.impl.AroundInvokeInvocationContext.perform(AroundInvokeInvocationContext.java:41)
	at io.quarkus.arc.impl.InvocationContexts.performAroundInvoke(InvocationContexts.java:32)
	at io.smallrye.reactive.messaging.providers.extension.ChannelProducer_Subclass.produceEmitter(Unknown Source)
	at io.smallrye.reactive.messaging.providers.extension.ChannelProducer_ProducerMethod_produceEmitter_86c8bb1e4975aaf33dff96bec0d2fc8fa44920ab_Bean.create(Unknown Source)
	at io.smallrye.reactive.messaging.providers.extension.ChannelProducer_ProducerMethod_produceEmitter_86c8bb1e4975aaf33dff96bec0d2fc8fa44920ab_Bean.get(Unknown Source)
	at io.smallrye.reactive.messaging.providers.extension.ChannelProducer_ProducerMethod_produceEmitter_86c8bb1e4975aaf33dff96bec0d2fc8fa44920ab_Bean.get(Unknown Source)
	at io.quarkus.arc.impl.CurrentInjectionPointProvider.get(CurrentInjectionPointProvider.java:52)
	at dev.kibb.TestConsumer_Bean.create(Unknown Source)
	at dev.kibb.TestConsumer_Bean.create(Unknown Source)
	at io.quarkus.arc.impl.AbstractSharedContext.createInstanceHandle(AbstractSharedContext.java:101)
	at io.quarkus.arc.impl.AbstractSharedContext$1.get(AbstractSharedContext.java:29)
	at io.quarkus.arc.impl.AbstractSharedContext$1.get(AbstractSharedContext.java:26)
	at io.quarkus.arc.impl.LazyValue.get(LazyValue.java:26)
	at io.quarkus.arc.impl.ComputingCache.computeIfAbsent(ComputingCache.java:69)
	at io.quarkus.arc.impl.AbstractSharedContext.get(AbstractSharedContext.java:26)
	at io.quarkus.arc.impl.ClientProxies.getApplicationScopedDelegate(ClientProxies.java:18)
	at dev.kibb.TestConsumer_ClientProxy.arc$delegate(Unknown Source)
	at dev.kibb.TestConsumer_ClientProxy.produce(Unknown Source)
	at dev.kibb.TestConsumer_ScheduledInvoker_produce_46db73690223bcc321054a6fc5b9736161d81e88.invokeBean(Unknown Source)
	at io.quarkus.arc.runtime.BeanInvoker.invoke(BeanInvoker.java:20)
	at io.quarkus.scheduler.runtime.SimpleScheduler$ScheduledTask$1.run(SimpleScheduler.java:274)
	at io.quarkus.vertx.core.runtime.VertxCoreRecorder$13.runWith(VertxCoreRecorder.java:548)
	at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2449)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1478)
	at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:29)
	at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:29)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:833)

Delaying seems to help but it’d be nice if there was a more definite fix.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Apache Kafka Reference Guide - Quarkus
Channels are connected to message backends using connectors. ... If a message produced from a Kafka record is nacked, a failure strategy is...
Read more >
Failed to inject Kafka client into GrpcService Quarkus
Without going deeply into the topic, here's my solution: You can't inject Kafka Emmiter directly to grpc service, it'll throw an exception.
Read more >
Apache Kafka - SmallRye Reactive Messaging
To use the Kafka Connector, add the following dependency to your project: ... channel.key-deserialization-failure-handler : name of the bean handling ...
Read more >
Kafka Fail-Over Using Quarkus Reactive Messaging - DZone
You can feel free to use any means that's easier for you. In this example, I'm using Strimzi on top of an OpenShift...
Read more >
Home of Quarkus Cheat-Sheet - GitHub Pages
QuarkusApplication interface or use the Java main method to launch Quarkus. ... if a runtime configuration property does not match the provided value....
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found