Make backoff policy in AbstractManagedChannelImplBuilder and subclasses customizable

Currently AbstractManagedChannelImplBuilder.build() method creates backoff policy with default values:

  @Override
  public ManagedChannel build() {
    return new ManagedChannelOrphanWrapper(new ManagedChannelImpl(
        this,
        buildTransportFactory(),
        **new ExponentialBackoffPolicy.Provider(),**
        SharedResourcePool.forResource(GrpcUtil.SHARED_CHANNEL_EXECUTOR),
        GrpcUtil.STOPWATCH_SUPPLIER,
        getEffectiveInterceptors(),
        TimeProvider.SYSTEM_TIME_PROVIDER));
  }

This makes it impossible to configure backoff policy when using sub-classes such as NettyChannelBuilder.

Ideally we should be able to customize backoff policy on each builder implementation (NettyChannelBuilder is the one that I specifically care about at the moment) and use default policy if custom value hasn’t been provided.

Please let me know if there are any other ways of configuring this policy or if I’m missing something.

[mfateev]

Our use case is business-level transactions going through the system. The number of clients connecting the backend service is usually small (less than 100) and each minute of the outage has a high business impact. So any 5-minute network or DB outage is extended another 2 minutes by the backoff configuration that we cannot control. We also hide the gRPC client behind our client side library, so we don’t ever expect any of the issues that you outlined as we control the maximum timeout.

Do you have any empirical data on how many clients are necessary to kill service or network with the reconnect requests only with 10-second maximum reconnect timeout, for example? I have a suspicion that your concern is mostly theoretical for 99.9% of real-life deployments outside of the google infra. And looking at the number of developers that asked for this change the lack of the feature causes real pain.

[hmendesB2C2]

I have a side car app running in kubernetes that receives about 5k GRPC calls per second from a client, if there is a disconnect then the client can be waiting minutes for reconnecting. We need to be able to control the initial and max backoffs at least. It doesn’t make any sense to wait large seconds to reconnect. If my app was to kill the network then the 5k calls per second would have done that already.