Add ConnectivityState.TERMINATED or change SHUTDOWN to notify after gRPC threads end.

Is your feature request related to a problem?

We shutdown ManagedChannel asynchronously so need an asynchronous callback for determining when a channel has been completely cleaned up. This can be important for when libraries are dynamically unloaded / loaded at runtime as the order matters. Currently there is only ConnectivityState.SHUTDOWN, which means no new RPCs will be serviced, but the connection may still be operating. Notably, there is almost always a GOAWAY frame sent after this connectivity state is reached.

Describe the solution you’d like

Add ConnectivityState.TERMINATED to notify after connections have been completely closed. This would be sent after all gRPC threads have been ended.

Describe alternatives you’ve considered

ConnectivityState.SHUTDOWN could possibly be changed to report after gRPC threads have ended - it’s unclear to me what the use case is of being notified of shutdown before this has happened but there may be a use case, in which case a new enum is needed instead. That being said, because it’s only a timing issue, it would be rare to break applications by changing the behavior this way too and could be viable.

Additional context

We use gRPC in a library and have user reports of issues due to not being able to know when the gRPC library has finished shutting down.

https://github.com/open-telemetry/opentelemetry-java/issues/3521

[sanjaypujare]

ConnectivityState.SHUTDOWN could possibly be changed to report after gRPC threads have ended

That cannot be done since “shutdown” and “terminated” have clear meanings (in this context). See isShutdown and isTerminated.

So the only possibility is to add ConnectivityState.TERMINATED as you suggest. So the channel state becomes ConnectivityState.TERMINATED when isTerminated() is true for that channel. This will need changes in ConnectivityStateManager.gotoState(). Do you want to submit a PR?

Note that there is a ManagedClientTransport.Listener.transportTerminated() callback, but this is per subchannel and a channel has one or more subchannels.

[ejona86]

Okay. Sorry, but that does seem for the best in your specific case.

For posterity: it seems fairly easy to add a calback-based API for termination state. And it could be okay if you want to directExecutor() with it, but most applicable cases of directExecutor() just trigger a notification in another thread.