[2.9.1] With partitioned topics, the Python client returns UnknownError in some cases when it should return a specific error class
See original GitHub issueDescribe the bug
The Python client raises _pulsar.UnknownError: Pulsar error: UnknownError
in cases when client logs indicate a specific exception code (for which there is already a specific error subclass) was recieved.
To Reproduce
0. Create a Client object with a Python logger that emits at INFO and above (e.g. logging.getLogger()
).
- Create a persistent topic with a backlog policy that is very “short”, e.g. 1mb.
- With a Python producer, send messages to the topic continually until the backlog quota is hit.
- Observe that the Python producer errors with an UnknownError rather than ProducerBlockedQuotaExceededException.
- Observe that the client logs show the arrival of a ProducerBlockedQuotaExceededException event.
- Attempt to re-create the producer on the quota-blocked topic.
- Observe that the same issue occurs: logs indicate ProducerBlockedQuotaExceededException, but an UnknownError is raised.
Expected behavior Specific error classes (the exception classes distributed in the pulsar-client package) should be raised instead of UnknownError when the client can match an error event (visible in the logs) to an exception class: ProducerBlockedQuotaExceededException
I suspect the issue is with the boost-python mapping code; I don’t think class-type checks are matching correctly on C++ exception objects.
Example logs:
chariot.chariot.pulsar_client::75192::Dummy-8::WARNING 2022-04-07 18:06:35,902 [127.0.0.1:58048 -> 127.0.0.1:6650] Received error response from server: ProducerBlockedQuotaExceededException (Cannot create producer on topic with backlog quota exceeded) -- req_id: 0
chariot.chariot.pulsar_client::75192::Dummy-8::ERROR 2022-04-07 18:06:35,902 [persistent://chariot1/chariot_namespace_heartbeat/chariot_topic_heartbeat-partition-0, SynchronousChariotProducer-heartbeat-1-ZacBentleyMBP-75192] Failed to create producer: ProducerBlockedQuotaExceededException
chariot.chariot.pulsar_client::75192::Dummy-8::ERROR 2022-04-07 18:06:35,902 Unable to create Producer for partition - 0 Error - ProducerBlockedQuotaExceededException
Example stacktrace which accompanies those logs:
Traceback (most recent call last):
File "/Users/zac.bentley/Desktop/Projects/Klaviyo/chariot/.venv/lib/python3.7/site-packages/pulsar/__init__.py", line 603, in create_producer
p._producer = self._client.create_producer(topic, conf)
_pulsar.UnknownError: Pulsar error: UnknownError
Issue Analytics
- State:
- Created a year ago
- Comments:9 (9 by maintainers)
Top GitHub Comments
I also reproduced this bug for a partitioned topic.
I will fix this issue soon.
When I create a truly non-partitioned topic, the behavior is correct (I get a
ResultProducerBlockedQuotaExceededException
on the client). So this bug only happens for partitioned topics. I’ll update the title accordingly.