Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Problem disposing consumer of partitioned topic

See original GitHub issue

I’m seeing an exception with the CancellationTokenSource when trying to dispose a partitioned consumer.

When this happens, it seems to deadlock my process. I assume this is because of the sub-consumers for each partition.

I’m just doing a await consumer.DisposeAsync(); with nothing special. My topic has 10 partitions and I can reproduce this problem against an empty queue. The cancellation needs to be triggered after the consumer gets to the mt/consumer(0, topic) created log message to reproduce.

System.InvalidOperationException: An attempt was made to transition a task to a final state when it had al
ready completed.
   at System.Threading.Tasks.TaskCompletionSource`1.SetException(Exception exception)
   at Pulsar.Client.Api.MultiTopicsConsumerImpl`1.stopConsumer()
   at <StartupCode$Pulsar-Client>.$MultiTopicsConsumerImpl.-ctor@606-98.MoveNext()
2023-06-29 12:57:07 "HOSTNAME" [INF] [PulsarLogger] [{}] consumer(1, topic, 0) mailbox has stopped normal
ly
2023-06-29 12:57:07 "HOSTNAME" [FTL] [PulsarLogger] [{}] mt/consumer(0, topic) mailbox failure
System.ObjectDisposedException: The CancellationTokenSource has been disposed.
   at System.Threading.CancellationTokenSource.ThrowObjectDisposedException()
   at Pulsar.Client.Api.MultiTopicsConsumerImpl`1.stopConsumer()
   at <StartupCode$Pulsar-Client>.$MultiTopicsConsumerImpl.-ctor@606-98.MoveNext()

Issue Analytics

State:
Created 3 months ago
Comments:8

Top GitHub Comments

1reaction

jvsteincommented, Jul 6, 2023

@Lanayx Just tested it out. It’s working as expected now. Thanks!

1reaction

jvsteincommented, Jul 5, 2023

I can reproduce against net6 and net7 on Linux. I haven’t tried reproducing on net48 on Windows.

Top Results From Across the Web

Consumer not reading from new partitions of a topic

I increased the number of partitions for a topic from 10 to 20 and have one kafka connect(consumer) for that topic.

Consumer stuck while reading from partitioned topic using ...

I am investigating a case in which the consumer is stuck while there are still many messages to read from a partitioned topic....

Solved: Kafka consumer group lag in one or two partition e...

Hi,. I have a strange problem at kafka channel topic like kafka consumer group lag( 15 lacs events) in one or two partition...

Work with consumer - Apache Pulsar

If the topic is partitioned, the first consumer subscribes to all partitioned topics, and other consumers are not assigned with partitions and receive...

Synchronize Different Consumers of the Same Kafka Topic

When Service-A polls a message from the topic, it should first extract the partition and offset of this message. Then, it calls Kafka...