"Topic policies cache have not init" randomly happen on different topics in the same namespace

Describe the bug I think my bug is related to this issue https://github.com/apache/pulsar/issues/10231 but actually it’s not because our Pulsar has disabled the brokerDeleteInactiveTopicsEnabled in broker level. And the issue “Topic policies cache have not init” happened on both namespaces with “inactiveTopicPolicies” enabled and disabled.

We are all using partition topics.

Our full adjusted configs:

allowAutoTopicCreationType=partitioned
brokerDeleteInactivePartitionedTopicMetadataEnabled=true
brokerDeleteInactiveTopicsEnabled=false
clusterName=mycluster
configurationMetadataStoreUrl=zookeeper:2181
forceDeleteNamespaceAllowed=true
forceDeleteTenantAllowed=true
functionsWorkerEnabled=false
managedLedgerDefaultAckQuorum=2
managedLedgerDefaultEnsembleSize=3
managedLedgerDefaultWriteQuorum=3
metadataStoreUrl=zookeeper:2181
numHttpServerThreads=32
numIOThreads=8
systemTopicEnabled=true
topicLevelPoliciesEnabled=true

To Reproduce Steps to reproduce the behavior: I have no idea of how to reproduce this issue but it happens on our production environment and not in staging. May be it’s related to production data

Expected behavior We can set topic policies properly without exception that:

Message: Topic policies cache have not init.

Stacktrace:

org.apache.pulsar.broker.service.BrokerServiceException$TopicPoliciesCacheNotInitException: Topic policies cache have not init.
        at org.apache.pulsar.broker.service.SystemTopicBasedTopicPoliciesService.getTopicPolicies(SystemTopicBasedTopicPoliciesService.java:207)
        at org.apache.pulsar.broker.service.TopicPoliciesService.lambda$getTopicPoliciesAsyncWithRetry$0(TopicPoliciesService.java:100)
        at org.apache.pulsar.client.util.RetryUtil.executeWithRetry(RetryUtil.java:48)
        at org.apache.pulsar.client.util.RetryUtil.lambda$null$1(RetryUtil.java:57)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:829)

Desktop (please complete the following information):

• OS: Ubuntu with Pulsar 2.10.0

Additional context Currently is there any Rest API for manually init the Topic Policies Cache or any way to check it via command line and manually init it?

[Raven888888]

Awesome to hear that @Technoboy- appreciate it Hope to use back topic level policy in the next release.

[Technoboy-]

@Raven888888 @diegosalvi @truong-hua We have found the root cause and pushed out the fix https://github.com/apache/pulsar/pull/17562 Thanks for reporting the issue.