question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

consumer.Seek(...) throws KafkaException "Local: Errorneous state" when executed immediately

See original GitHub issue

Description

IConsumer.Seek(TopicPartitionOffset tpo) throws an exception, if immediately executed after IConsumer.Assign(TopicPartition). With a delay in between, it works. This should work immediately and suggest a non-completed task in Assign. I suggest creating async APIs where the state is sane when the API is awaited:

await c.AssignAsync(topic);
await c.SeekAsync(topic_partition_offset);

Nuget package version: 1.2.0 Kafka version: 2.1.1cp1 OS: Windows (kafka + zookeeper in docker confluentinc/cp-…)

How to reproduce

async Task Test()
{
   var config = new ConsumerConfig { BootstrapServers="localhost:1234", GroupId = "test",  };
   using (var c = new ConsumerBuilder<Ignore, string>(config).Build())
   {
      var tp = new TopicPartition("test-topic", new Partition(0));
      c.Assign(tp);
      //await Task.Delay(TimeSpan.FromSeconds(1)); // activate this to "fix" it / perform exponential backoff
      c.Seek(new TopicPartitionOffset(tp, 0)); // throws, if not delayed!
   }
}

Checklist

Please provide the following information:

  • A complete (i.e. we can run it), minimal program demonstrating the problem. No need to supply a project file.
  • Confluent.Kafka nuget version.
  • [ X] Apache Kafka version.
  • [X ] Client configuration.
  • [ X] Operating system.
  • [N/A] Provide logs (with “debug” : “…” as necessary in configuration).
  • [N/A] Provide broker log excerpts.
  • [Well, you decide…] Critical issue.

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:6 (3 by maintainers)

github_iconTop GitHub Comments

2reactions
nikoudelcommented, Oct 21, 2019

I was trying to seek to the beginning of all partitions after restart (like told here): image and was getting the same error as in this issue, however, I suspect seeking is not the right way to do it - the consumer example suggests a different way:

...
.SetPartitionsAssignedHandler((c, partitions) =>
{
	// This will fail:
	//foreach (var p in partitions)
	//{
	//    c.Seek(new TopicPartitionOffset(p, Offset.Beginning));
	//}

	// This works:
	return partitions.Select(tp => new TopicPartitionOffset(tp, Offset.Beginning));
})
...
0reactions
edenhillcommented, Jun 29, 2020

Seek() may only be used on partitions that have been Assign()ed. Assign() is called after SetPartitionsAssignedHandler() returns.

Read more comments on GitHub >

github_iconTop Results From Across the Web

confluent_kafka: how to reliably seek before reading data ...
In both cases: Consumer.seek usually or always fails with ""Local: Erroneous state". Consumer.positions always returns -1001 ...
Read more >
Kafka .NET Client
If this happens, the Commit or StoreOffset call will throw a KafkaException with ErrorCode equal to Local_State (“Erroneous state”). In the context of...
Read more >
KafkaConsumer (kafka 0.10.0.1 API)
A Kafka client that consumes records from a Kafka cluster. It will transparently handle the failure of servers in the Kafka cluster, and...
Read more >
Manually committing offsets in Kafka using .Net
Sometimes it executes smoothly but sometimes it throws "Local:Waiting for Coordinator" exception. But in both the cases , when I try consuming ...
Read more >
KafkaConsumer (clients 1.1.0 API) - javadoc.io
A client that consumes records from a Kafka cluster. This client transparently handles the failure of Kafka brokers, and transparently adapts as topic ......
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found