Resolver/Balancer API

Grpc Implementations generally favor or at least support Client-Side-Loadbalancing. In Go you go

chan, err := grpc.Dial("my-custom-resolver://some-string-that-has-meaning/some-other-string-with-meaning")

which will basically call a previously registered resolver (my-custom-resolver) with the URL. This resolver then has the job to return 0 or more IP Addresses to finally connect to. It might query SRV records from a DNS resolver, or ask something like etcd, zookeeper, consul, etc…

There is also the old API where you pass grpc.WithBalancer(myBalancer) to grpc.Dial. I suspect this fits C# better.

How will this be achieved in grpc-dotnet?

[wicharypawel]

Hello, I would like to share the results of my work. Let me share draft version of the proposal.

grpc-dotnet load balancing support proposal (and related proposals).pdf (proposal starts at page 3)

After having initial review I would like to make pull request into official gRFC proposals repo.

Fork with the code is shared here: https://github.com/wicharypawel/grpc-dotnet/tree/load-balancing-v0.7.0 (commit hash: efd5eda4633acf97380146677ab8b985fa9abf9e) Example usage of this new feature is shared here: https://github.com/wicharypawel/net-core-grpc-load-balance/tree/load-balancing-proposal-examples (commit hash: f0628b80c469fb21502d527b8ccc85b005fe9ed2 )

[wicharypawel]

Hi, I think I have something interesting to share. The last few days I’ve been working on a better understanding of load balancing in grpc. I have expanded the capabilities of grpc dotnet with full support for load balancing.

I think it’s something of a worth looking at. I prepared two repositories:

The first repository contains fork of grpc-dotnet repository with extension of grpc.net.client capabilities (most of my work was squashed to erase some wrong ideas of mine 😄 )

link: https://github.com/wicharypawel/grpc-dotnet/commits/load-balancing-for-dotnet

Extensions include: (documentation links below)

• support for subchannels
• pick_first policy
• round_robin policy
• grpclb policy (core implementation)
• naive resolver plugin (for non-loadbalancing scenarios)
• dns resolver plugin (works with A, SRV and TXT records)
  • search TXT records for _grpc_config.my-service
  • search SRV records for _grpclb._tcp.my-service
  • search A records for my-service
• static resolver plugin (for local work)
• support for dns:// scheme
• support for xds:// scheme
• support for overriding dns server while working with dns resolver plugin
• base support for service config via TXT records
• create providers for resolver plugins and policies
• providers are stored in dynamic registry (based on java implementation)
• includes verbose logging
• all grpc dotnet tests are passing
• new code is covered with tests (see new Grpc.Net.Client.LoadBalancing.Tests project)
• after adding subchannels integration with Grpc.Net.ClientFactory was verified
• load reporting (clientstats) to lookaside server for grpclb policy
• fallback support for grpclb policy
• support for load balancing tokens for GrpcCall (currently used in grpclb)
• disable SRV records by default (currently used in grpclb)
• enable service config to load xds and xds_experimental
• implementation of xDS policy (includes bootstrap file support, LDS, RDS, CDS & EDS)
• api design is modeled based on java api
• refactor code to be reactive (apply observers)
• apply connectivity monitoring to subchannels

Work in progress

• official proposals for gRFC (read more below)

The second repository is an example of using this implementation and ton’s of other examples and snippets

link: https://github.com/wicharypawel/net-core-grpc-load-balance

Samples include:

• round robin for gRPC dotnet client
• lookaside load balancing for gRPC dotnet client
• lookaside load balancing for gRPC dotnet client with custom dns server
• lookaside load balancing for gRPC dotnet client with asp.net integration (Grpc.Net.ClientFactory)
• load balancing using envoy side car for gRPC dotnet client
• load balancing using istio service mesh for gRPC dotnet client
• load balancing using xDS policy for gRPC dotnet client
• include simple C# implementation of lookaside loadbalancer (grpclb protocol) for gRPC in K8s
• and 5 other scenarios including samples for lookaside loadbalancing in Go, Python and Java

How to start: I think best way is to pull second repository and initialize submodule (which includes first repository). Continue with Readme, in case of problems let me know here.

Kudos to @jtattermusch for his presentation about grpclb available here: https://github.com/jtattermusch/grpc-loadbalancing-kubernetes-examples

Related documentation:

• https://grpc.io/blog/grpc-load-balancing/
• https://github.com/grpc/grpc/blob/master/doc/load-balancing.md
• https://github.com/grpc/grpc/blob/master/doc/naming.md
• https://github.com/grpc/grpc/blob/master/doc/service_config.md
• https://github.com/grpc/proposal/blob/master/A5-grpclb-in-dns.md
• https://github.com/grpc/proposal/blob/master/A2-service-configs-in-dns.md
• https://github.com/grpc/proposal/blob/master/A24-lb-policy-config.md
• https://github.com/grpc/proposal/blob/master/A10-avoid-grpclb-and-service-config-for-localhost-and-ip-literals.md
• https://github.com/grpc/proposal/blob/master/A26-grpclb-selection.md
• https://github.com/grpc/proposal/blob/master/A27-xds-global-load-balancing.md
• https://github.com/grpc/grpc-proto/blob/master/grpc/service_config/service_config.proto
• https://tools.ietf.org/html/rfc2782 - describe SRV records in DNS
• https://kubernetes.io/blog/2018/11/07/grpc-load-balancing-on-kubernetes-without-tears/
• https://groups.google.com/forum/#!msg/grpc-io/0yGihF-EFQo/A4QKdXffBwAJ
• https://grpc.github.io/grpc-java/javadoc/io/grpc/LoadBalancer.html - JAVA LB API

Hope you find this helpful, I would be glad to help if you find that interesting to pull request.