question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Namespace retention policy not working as expected

See original GitHub issue

Describe the bug I`m trying to set a retention policy for namespace bin/pulsar-admin namespaces set-retention testtenant/testnamespace --size -1 --time 1m

The main purpose is to save messages on topics even if there is no subscription. But not to store them infinitely, and delete them when some time passed. When you set retention policy messages start writing to the topic without subscriptions but it seems for infinite time. I can receive messages even after 1 hour after message is published by the producer (when policy time is set for 1 minute).

To Reproduce Steps to reproduce the behavior:

  1. Create tenant (testtenant for example)
  2. Create namespace (testnamespace for example)
  3. Set retention for namespace bin/pulsar-admin namespaces set-retention testtenant/testnamespace --size -1 --time 1m
  4. Publish a message with python import pulsar client = pulsar.Client('pulsar://pulsar:6650') producer = client.create_producer('persistent://testtenant/testnamespace/test') producer.send(('Hello-pulsar').encode('utf-8')) client.close()
  5. Wait time more than 1 minute and try to receive message with python import pulsar client = pulsar.Client('pulsar://pulsar:6650') consumer = client.subscribe('persistent://testtenant/testnamespace/test', 'my-subscription') msg = consumer.receive() print("Received message '{}' id='{}'".format(msg.data(), msg.message_id())) consumer.acknowledge(msg) client.close()

Expected behavior Message in topic with message age more than in --time parameter deletes

Screenshots n/a

Desktop (please complete the following information):

  • Ubuntu 20.04
  • Pulsar deployed in Kubernetes with official pulsar helm
  • Pulsar 2.7.2

Additional context n/a

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:5 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
gaozhangmincommented, Aug 20, 2021

@Technoboy- so there is no guarantee that message will be deleted after a period in “time” parameter and we nothing can`t do to fix this behavior?

For opened ledger, broker will roll over it every 4 hours by default. So, opened ledger will be deleted after 4 hours.

0reactions
fsergiicommented, Aug 20, 2021

Thanks for your reply. So it seems not a bug then, but normal behavior. So I think I can close the issue.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Retention Policy Not working as expected or applied ...
1. Collect the policy and retention tags to determine which policy/tags caused the situation. · 2. Check the retention policy applied on the ......
Read more >
Retention policies and ECS 3.0 CE - Dell Community
I tested this in our lab on real ECS running 3.0. I created a retention policy on my namespace and the specified that...
Read more >
[GitHub] [pulsar] Technoboy- commented on issue #11719 ...
[GitHub] [pulsar] Technoboy- commented on issue #11719: Namespace retention policy not working as expected · GitBox Thu, 19 Aug 2021 18:24:50 -0700.
Read more >
Retention Policies not working - Microsoft Community
I have a retention policy which should delete emails older than 30 days from the 'Deleted items' folder.
Read more >
Compactor based retention rules not working #4572 - GitHub
Are coming in on daily basis, so I expect them to be cleaned up after 7d. This isn't happening. Examples, I still have...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found