question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Please create usage of KafkaConsumer and poll() I am not able to get data

See original GitHub issue
from kafka import KafkaConsumer

# To consume latest messages and auto-commit offsets
consumer = KafkaConsumer('testing_topic',
                         group_id='my-group',
                         bootstrap_servers=['192.168.10.5:9092'])

message = consumer.poll(500)
print(message.value)

Please suggest me how to use poll() like above. Of course above code is not working.

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Comments:7 (1 by maintainers)

github_iconTop GitHub Comments

28reactions
tvoinarovskyicommented, Jul 10, 2017

Ooops, missed 1 for cycle:

while True:
    # Response format is {TopicPartiton('topic1', 1): [msg1, msg2]}
    msg_pack = consumer.poll(timeout_ms=500)

    for tp, messages in msg_pack.items():
        for message in messages:
            # message value and key are raw bytes -- decode if necessary!
            # e.g., for unicode: `message.value.decode('utf-8')`
            print ("%s:%d:%d: key=%s value=%s" % (tp.topic, tp.partition,
                                                  message.offset, message.key,
                                                  message.value))
1reaction
wonyongHwangcommented, May 2, 2019

Thank a lot~ finally I find a solution

Read more comments on GitHub >

github_iconTop Results From Across the Web

Chapter 4. Kafka Consumers: Reading Data from Kafka
The parameter we pass, poll() , is a timeout interval and controls how long poll() will block if data is not available in...
Read more >
Kafka Consumer Poll runs indefinitely and doesn't return ...
I am getting the below error: org.apache.kafka.common.errors.SerializationException: Error deserializing key/value for partition at offset 2. If ...
Read more >
What happens when you call poll on Kafka Consumer?
There is a heartbeat thread that notifies cluster about consumer liveness. It is created within poll method if it does not exist.
Read more >
Kafka Tutorial: Creating a Kafka Consumer in Java
The Kafka consumer uses the poll method to get N number of records. Consumers in the same group divide up and share partitions...
Read more >
Creating Kafka Consumer in Java - Javatpoint
This can be done via a consumer group. In the consumer group, one or more consumers will be able to read the data...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found