Multiple consumers with same group id get same message

Kafka manual says that each message is delivered exactly to one consumer from a group (with a same group id). And this is what I see with Java high-level API and expected to see with Python’s SimpleConsumer. However, when I run 2 consumers simultaneously (see code below) and send new message, both instances of consumer receive it.

---

# consumer.py
from kafka.client import KafkaClient
from kafka.consumer import SimpleConsumer

kafka = KafkaClient("broker", 9092)
consumer = SimpleConsumer(kafka, "my-group", "my-topic",
                          auto_commit=False)
while True:
    msg = consumer.get_message()
    if msg:
        print(msg)
    else:
        print('no new messages')
kafka.close()

---

# producer.py
from kafka.client import KafkaClient
from kafka.producer import SimpleProducer

kafka = KafkaClient("broker", 9092)

producer = SimpleProducer(kafka, "my-topic")
producer.send_messages("my message")

kafka.close()

---

Note, that auto_commit in consumer is set to False to overcome problem with Kafka 0.8.1 and current version of kafka-python, but changing it to True only adds exceptions and doesn’t fix consumption problem.

[sadaf2605]

pip3 freeze | grep kafka
kafka==1.3.4.1

All my consumer with same group are getting all the same messages from all the producers.

[dpkp]

yes, #38 is the tracking issue. consumer rebalancing will be supported when using kafka brokers >= v0.9.0.0 . Development is on the 0.9 git branch.