Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Event Hubs processor fails to get events after a period of time

See original GitHub issue

There are times when fetching events from Event Hubs using EventProcessor, events are not fetched and instead, a timeout exception is thrown. Consequently, the processor stops fetching events. In the repro, the event hub has 32 partitions.

INFO	2020-07-20 22:00:40,690	1806056	c.a.c.a.i.handler.SendLinkHandler	[single-1]	onLinkRemoteClose connectionId[MF_4ee337_1595280634612], linkName[eh0-int-stream], errorCondition[amqp:link:detach-forced], errorDescription[Idle link tracker, link eh0-int-stream has been idle for 1800000ms TrackingId:4d82b29d-b922-4634-a2ba-29fd14c40764_G16, SystemTracker:client-link18456132, Timestamp:2020-07-20T22:00:39]
INFO	2020-07-20 22:00:40,690	1806056	c.a.c.a.i.handler.SendLinkHandler	[single-1]	processOnClose connectionId[MF_4ee337_1595280634612], linkName[eh0-int-stream], errorCondition[amqp:link:detach-forced], errorDescription[Idle link tracker, link eh0-int-stream has been idle for 1800000ms TrackingId:4d82b29d-b922-4634-a2ba-29fd14c40764_G16, SystemTracker:client-link18456132, Timestamp:2020-07-20T22:00:39]
ERROR	2020-07-20 22:00:40,698	1806064	c.a.c.a.i.ReactorSender	[single-1]	[eh0-int-stream] Error occurred in sender error handler.
Idle link tracker, link eh0-int-stream has been idle for 1800000ms TrackingId:4d82b29d-b922-4634-a2ba-29fd14c40764_G16, SystemTracker:client-link18456132, Timestamp:2020-07-20T22:00:39, errorContext[NAMESPACE: eastus2-int-testing-ehns-punuljxb.servicebus.windows.net, PATH: eh0-int-stream, REFERENCE_ID: ccec2d406a3a451a85e4ffafab3f8d9a_G16, LINK_CREDIT: 296]
com.azure.core.amqp.exception.AmqpException: Idle link tracker, link eh0-int-stream has been idle for 1800000ms TrackingId:4d82b29d-b922-4634-a2ba-29fd14c40764_G16, SystemTracker:client-link18456132, Timestamp:2020-07-20T22:00:39, errorContext[NAMESPACE: eastus2-int-testing-ehns-punuljxb.servicebus.windows.net, PATH: eh0-int-stream, REFERENCE_ID: ccec2d406a3a451a85e4ffafab3f8d9a_G16, LINK_CREDIT: 296]
	at c.a.c.a.i.ExceptionUtil.toException(ExceptionUtil.java:78)
	at c.a.c.a.i.handler.LinkHandler.processOnClose(LinkHandler.java:90)
	at c.a.c.a.i.handler.LinkHandler.handleRemoteLinkClosed(LinkHandler.java:108)
	at c.a.c.a.i.handler.LinkHandler.onLinkRemoteClose(LinkHandler.java:50)
	at c.a.c.a.i.handler.SendLinkHandler.onLinkRemoteClose(SendLinkHandler.java:20)
	at org.apache.qpid.proton.engine.BaseHandler.handle(BaseHandler.java:176)
	at org.apache.qpid.proton.engine.impl.EventImpl.dispatch(EventImpl.java:108)
	at org.apache.qpid.proton.reactor.impl.ReactorImpl.dispatch(ReactorImpl.java:324)
	at org.apache.qpid.proton.reactor.impl.ReactorImpl.process(ReactorImpl.java:291)
	at c.a.c.a.i.ReactorExecutor.run(ReactorExecutor.java:82)
	at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:68)
	at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:28)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
INFO	2020-07-20 22:00:40,703	1806069	c.a.c.a.i.ReactorSession	[single-1]	linkName[eh0-int-stream]: Error occurred. Removing and disposing send link.
com.azure.core.amqp.exception.AmqpException: Idle link tracker, link eh0-int-stream has been idle for 1800000ms TrackingId:4d82b29d-b922-4634-a2ba-29fd14c40764_G16, SystemTracker:client-link18456132, Timestamp:2020-07-20T22:00:39, errorContext[NAMESPACE: eastus2-int-testing-ehns-punuljxb.servicebus.windows.net, PATH: eh0-int-stream, REFERENCE_ID: ccec2d406a3a451a85e4ffafab3f8d9a_G16, LINK_CREDIT: 296]
	at c.a.c.a.i.ExceptionUtil.toException(ExceptionUtil.java:78)
	at c.a.c.a.i.handler.LinkHandler.processOnClose(LinkHandler.java:90)

Issue Analytics

State:
Created 3 years ago
Reactions:2
Comments:14 (7 by maintainers)

Top GitHub Comments

1reaction

123Jun321commented, Aug 28, 2020

Hi @conniey, I want to know when the new version will be released? could you give a specific date?

0reactions

connieycommented, Sep 16, 2020

Should be resolved by #14990

Top Results From Across the Web

Event Hubs messaging exceptions - .NET (legacy)

This article provides a list of Azure Event Hubs messaging exceptions ... Client can wait for a period of time, then retry the...

Ignore old events when processing events of an EventHub ...

The problem is as follows: Sometimes there is a long period of time where the EventProcessorClient is stopped and many events are retained...

Azure Event Hubs and its Complete Overview - Serverless360

Azure Event Hubs monitoring and management challenges are solved by using Serverless360. Read to know more about the complete overview of Azure Event...

Azure.Messaging.EventHubs.Processor 5.7.5 - NuGet

A partition is an ordered sequence of events that is held in an Event Hub. Partitions are a means of data organization associated...

Azure Event Hubs client library for .NET

This indicates that the Event Hubs service did not respond to an operation within the expected amount of time. This may have been...