Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
Once a topic is terminated it can no longer be read from by a new Consumer
See original GitHub issueDescribe the bug Topic termination documentation says
Terminate a topic (disallow further messages from being published on the topic)
But once you terminate a topic, even if the retention settings allow data retention, a new consumer cannot read from it.
To Reproduce Steps to reproduce the behavior:
- Produce data to a topic
./pulsar-client --url pulsar+ssl://pulsar.saas-dev.cogitocorp.us:7443 --proxy-url pulsar+ssl://pulsar.saas-dev.cogitocorp.us:7443 --proxy-protocol SNI produce persistent://cogito-dialog/wav/frankTest7 -f ~/platform2-signal/signal-stream-client-library/src/intTest/resources/python.wav
- Terminate The topic
./apache-pulsar-2.6.1/bin/pulsar-admin --admin-url http://platform-pulsar-broker:8080 topics terminate persistent://cogito-dialog/wav/frankTest7
- Consume from the Topic with SubcriptionInitialPosition = EARLIEST
./pulsar-client --url pulsar+ssl://pulsar.saas-dev.cogitocorp.us:7443 --proxy-url pulsar+ssl://pulsar.saas-dev.cogitocorp.us:7443 --proxy-protocol SNI consume -s tcpingress-test2 -p Earliest persistent://cogito-dialog/wav/frankTest7 -t Shared
- Get a message indicating we reached the end of the topic
09:12:07.353 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://cogito-dialog/wav/frankTest7][tcpingress-test2] Subscribed to topic on pulsar.saas-dev.cogitocorp.us/100.26.7.97:7443 -- consumer: 0
09:12:07.398 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ClientCnx - [pulsar.saas-dev.cogitocorp.us/100.26.7.97:7443] Broker notification reached the end of topic: 0
09:12:07.399 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ConsumerImpl - [tcpingress-test2] [persistent://cogito-dialog/wav/frankTest7] [8778c] Consumer has reached the end of topic
Retention period is 3 days
./apache-pulsar-2.6.1/bin/pulsar-admin --admin-url http://platform-pulsar-broker:8080 namespaces get-retention cogito-dialog/wav
{
"retentionTimeInMinutes" : 4320,
"retentionSizeInMB" : -1
}
Expected behavior
The Admin Docs say
Terminate a topic (disallow further messages from being published on the topic)
https://pulsar.apache.org/docs/en/pulsar-admin/#terminate
Also the Javadoc says
Terminate a topic. A topic that is terminated will not accept any more messages to be published and will let consumer to drain existing messages in backlog
https://github.com/apache/pulsar/blob/master/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v2/PersistentTopics.java#L2168-L2169
Neither says anything about future consumers
So either
- Topic termination should still allow new consumers to read data from the topic
- (If this works as designed) the Documentation should be updated to clearly indicate that i) Existing consumers once they catch up to the end will be notified there is no more data ii) New consumers will get NO data
Additional context
- Using Apache Pulsar 2.6.1
Issue Analytics
- State:
- Created 3 years ago
- Comments:8 (6 by maintainers)
Top Results From Across the Web
Top Related Medium Post
Top Related StackOverflow Question
Troubleshoot Live Code
Top Related Reddit Thread
Top Related Hackernoon Post
Top Related Tweet
Top Related Dev.to Post
Top Related Hashnode Post
I retried with the command line tools for Pulsar 2.6.1 ensuring that the subscriber name was new each time and was able to reproduce the behavior
TEST CASE 1: No Terminate Producer
Consumer
Response
TEST CASE 2: Include Terminate Producer
Termination
Consumer
Response
NOTE: If I do the reset-cursor
I do get data back But again it appears surprising that a new subscription, created after topic termination, needs to have its cursor reset
Closed as stale. Please create a new issue if it’s still relevant to the maintained versions.