question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Consumer offsets not displaying (missing partitions)

See original GitHub issue

Kafka 0.10.0 on Ubuntu 14.04.04 LTS Trifecta 0.20.0

Using kafka-python client I consume a topic with 9 partitions and then view in Trifecta > Inspect > Consumers > {topic} > {group} and it only displays offsets for a portion of the partitions. I expect to see all partitions.

screen shot 2016-07-01 at 3 01 18 pm

Using the CLI tool I perform the following and it confirms they were all consumed and no lag:

ubuntu@db1:~$ kafka-consumer-offset-checker --group etl-diffs --topic rets.nnrmls.Agent --zookeeper localhost:2181

Group           Topic                          Pid Offset          logSize         Lag             Owner
etl-diffs       rets.nnrmls.Agent              0   3175            3175            0               none
etl-diffs       rets.nnrmls.Agent              1   3691            3691            0               none
etl-diffs       rets.nnrmls.Agent              2   3352            3352            0               none
etl-diffs       rets.nnrmls.Agent              3   3432            3432            0               none
etl-diffs       rets.nnrmls.Agent              4   3111            3111            0               none
etl-diffs       rets.nnrmls.Agent              5   2856            2856            0               none
etl-diffs       rets.nnrmls.Agent              6   3247            3247            0               none
etl-diffs       rets.nnrmls.Agent              7   3690            3690            0               none
etl-diffs       rets.nnrmls.Agent              8   3072            3072            0               none

I upgraded to latest 0.21 and same issue it appears.

screen shot 2016-07-01 at 3 17 39 pm

Issue Analytics

  • State:open
  • Created 7 years ago
  • Comments:6 (2 by maintainers)

github_iconTop GitHub Comments

1reaction
ldaniels528commented, Jul 3, 2016

I’ll take a look, and get it fixed. 😃

0reactions
likesh1980commented, Jan 31, 2018

I agree, the same happening for me as well. It seems like when I have a broker 0.10.1.x, it works fine, but when I have a broker 0.10.0.x, I can not view the messages. I will try with a different version and see if this helps.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Kafka topic partition has missing offsets - Stack Overflow
Even though, the application is continuously running and working without any obvious errors, I see a lag in the consumer group for the...
Read more >
missing consumer groups and topics · Issue #8 - GitHub
Any idea why the rest of them aren't showing? Seems like Consumer Group metrics and Internal metrics are missing it's just: "Topic/Partition ......
Read more >
KafkaConsumer (kafka 2.4.0 API)
This offset acts as a unique identifier of a record within that partition, and also denotes the position of the consumer in the...
Read more >
Operator KafkaConsumer - IBM
Primitive operator image not displayed. ... The consumer operator commits the offsets of those Kafka messages, which have been submitted as tuples.
Read more >
The Ultimate UI Tool for Kafka - Offset Explorer
The list will show both the start/end offsets of the partitions as well as the offset of consumers within each partition. You can...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found