Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[Broker & Bookie] Configure Bookkeeper client & server backpressure by default

See original GitHub issue

Is your enhancement request related to a problem? Please describe.

Pulsar doesn’t take advantage of Bookkeeper’s client-side and server-side backpressure features. https://github.com/apache/bookkeeper/issues/1086 and https://github.com/apache/bookkeeper/issues/1409 are the original Bookkeeper issues about the lack of backpressure handling. Similar problems exist with Pulsar since backpressure handling isn’t configured. One of the reported issues is #9562 .

Describe the solution you’d like

Configure waitTimeoutOnBackpressureMs in the Bookkeeper client configuration (Pulsar broker side) and waitTimeoutOnResponseBackpressureMs, maxAddsInProgressLimit, maxReadsInProgressLimit in the Bookkeeper server (“Bookie”) configuration with sane default values.

Additional context

#9562

Issue Analytics

State:
Created 2 years ago
Reactions:1
Comments:8 (8 by maintainers)

Top GitHub Comments

1reaction

eolivellicommented, Apr 29, 2021

@lhotari yes I have picked it to branch-7.2

1reaction

eolivellicommented, Apr 29, 2021

@lhotari since 2.8 you can configure the bookkeeper client inside the broker just by adding bookkeeper_xxx in broker.conf

check for instance the case of “bookkeeper_opportunisticStriping”

Top Results From Across the Web

Apache BookKeeper Internals — Part 4 — Back Pressure

In this post we'll cover all the back pressure mechanisms that BookKeeper server nodes employ to protect themselves from overload of both reads ......

BookKeeper configuration

The table below lists parameters that you can set to configure bookies. All configuration takes place in the bk_server.conf file in the bookkeeper-server/conf ......

Apache Pulsar: The Next Generation Messaging and Queuing ...

Using Apache BookKeeper as the underlying data storage, Pulsar guarantees ... And data durability, if you say, “Create user account,” and that message...

Client Optimization: How Tencent Maintains Apache Pulsar ...

Optimization 1: Server-side troubleshooting for client-side timeouts ... level log printing on brokers and bookies for your Pulsar cluster.

Kafka vs Pulsar - Performance, Features, and Architecture ...

Kafka in its default configuration is faster than Pulsar in all latency ... and understood: Pulsar servers (brokers) plus Apache BookKeeper (bookies) plus ......