ChannelPool: least outstanding request algorithm

Our ChannelPool uses a round-robbin approach to selecting a Channel. It’s effective in most cases, but may cause problems in other cases. A user found that one channel was slow and overloaded and caused various problems a “least outstanding request algorithm” will reduce problems like theirs in the future.

[igorbernstein2]

Hi eiiches,

Thanks for sharing! I’ve sketched out an alternative approach here: https://github.com/GoogleCloudPlatform/cloud-bigtable-client/pull/1774.

It should have a lower overhead to maintaining an ordered list of loaded connections while evenly spreading the load across the pool.

Also, GRPC actually has native load balancing , which we will switch to at some point in the future.

[eiiches]

Hi, I’ve been in contact with the Japanese support team about the issue.

Here’s my implementation of the algorithm: https://github.com/eiiches/cloud-bigtable-client/commits/feature/least-outstanding-requests-balancing-channel-pool

We’ve load-tested the implementation, and it worked well, but I’m still worried that there may be some corner case that the number of outstanding requests is not correctly incr/decremented.

I can open a PR if you like, but it will probably take some time to sign a Corporate CLA. So I’d also be glad if someone can implement it independently.

All that said, I believe these kind of load balancing algorithms should be ideally provided by gRPC modules to avoid re-implementing it everywhere.