consumer.Seek(...) throws KafkaException "Local: Errorneous state" when executed immediately
See original GitHub issueDescription
IConsumer.Seek(TopicPartitionOffset tpo) throws an exception, if immediately executed after IConsumer.Assign(TopicPartition). With a delay in between, it works. This should work immediately and suggest a non-completed task in Assign. I suggest creating async APIs where the state is sane when the API is awaited:
await c.AssignAsync(topic);
await c.SeekAsync(topic_partition_offset);
Nuget package version: 1.2.0 Kafka version: 2.1.1cp1 OS: Windows (kafka + zookeeper in docker confluentinc/cp-…)
How to reproduce
async Task Test()
{
var config = new ConsumerConfig { BootstrapServers="localhost:1234", GroupId = "test", };
using (var c = new ConsumerBuilder<Ignore, string>(config).Build())
{
var tp = new TopicPartition("test-topic", new Partition(0));
c.Assign(tp);
//await Task.Delay(TimeSpan.FromSeconds(1)); // activate this to "fix" it / perform exponential backoff
c.Seek(new TopicPartitionOffset(tp, 0)); // throws, if not delayed!
}
}
Checklist
Please provide the following information:
- A complete (i.e. we can run it), minimal program demonstrating the problem. No need to supply a project file.
- Confluent.Kafka nuget version.
- [ X] Apache Kafka version.
- [X ] Client configuration.
- [ X] Operating system.
- [N/A] Provide logs (with “debug” : “…” as necessary in configuration).
- [N/A] Provide broker log excerpts.
- [Well, you decide…] Critical issue.
Issue Analytics
- State:
- Created 4 years ago
- Comments:6 (3 by maintainers)
Top Results From Across the Web
confluent_kafka: how to reliably seek before reading data ...
In both cases: Consumer.seek usually or always fails with ""Local: Erroneous state". Consumer.positions always returns -1001 ...
Read more >Kafka .NET Client
If this happens, the Commit or StoreOffset call will throw a KafkaException with ErrorCode equal to Local_State (“Erroneous state”). In the context of...
Read more >KafkaConsumer (kafka 0.10.0.1 API)
A Kafka client that consumes records from a Kafka cluster. It will transparently handle the failure of servers in the Kafka cluster, and...
Read more >Manually committing offsets in Kafka using .Net
Sometimes it executes smoothly but sometimes it throws "Local:Waiting for Coordinator" exception. But in both the cases , when I try consuming ...
Read more >KafkaConsumer (clients 1.1.0 API) - javadoc.io
A client that consumes records from a Kafka cluster. This client transparently handles the failure of Kafka brokers, and transparently adapts as topic ......
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
I was trying to seek to the beginning of all partitions after restart (like told here): and was getting the same error as in this issue, however, I suspect seeking is not the right way to do it - the consumer example suggests a different way:
Seek() may only be used on partitions that have been Assign()ed. Assign() is called after SetPartitionsAssignedHandler() returns.