BlazeClient's idleTimeout works differently than BlazeServer's idleTimeout

Both BlazeClient and BlazeServer offer an idleTimeout configuration option. The option has the same name in both, and shares implementation (IdleTimeoutStage) but from user’s perspective it behaves differently.

In BlazeClient the idleTimeout is active only when a request is in-flight. It may kick in if the server takes too long to respond (unless other timeout does it first) or when the server starts sending a response and then pauses for too long. An idle (not borrowed from the pool) connection will remain open indefinitly as the idleTimeout is deactivated when a connection is returned to the pool. This may contradict users assumptions about the meaning of the word “idle” in the context of an HTTP client.

In BlazeServer the idleTimeout is always active, and it may terminate connections that have an in-flight request but nothing happened for too long, as well as connections which are kept open but unused.

I see this inconsistency as a bad developer experience. In addition to that I see the lack of mechanism for evicting unused connections from the client’s connection pool, as a missing feature and potential resource leak.

[RafalSumislawski]

I’m not convinced what we should do with the blaze-server idleTimeout (or if we should do anything with it).

What I am currently convinced to is that the changes in akka-http-client go in the right direction and we should do the same in blaze-client. It should have two timeouts:

• an idleTimeout that is active while the connection is being used. It’s main purpose being to prevent getting stuck in this state. The current implementation does exactly that.
• and a new keepAliveTimeout which would be active while the connection sits in the pool.

While this doesn’t solve the user experience problem I stated in the title of this issue “BlazeClient’s idleTimeout works differently than BlazeServer’s idleTimeout”. I think it solves the the more serious technical problems of:

• being unable to evict connections from the pool before server closes them, and therefore risking a race between client using the connection and server closing it due to timeout.
• being able to evict unused connection from the pool in order to free up resources

P.S. I feel a need to on more client-side timeout, sort of a maximumLifespan, that would be useful in forcing a rebalancing of connections though a load-balancer. But that’s a story for another ticket.

[RafalSumislawski]

http4s/http4s#3700 seems to be related. Maybe the can be solved together.