[Feature Request] Non-Blocking Async Interceptors for Async calls

Introduction

In current interceptor model, it’s impossible to run time consuming async operations without blocking the that interceptor’s thread.

We can still do some parallel operations by awaiting the thread, for example:

.addInterceptor(new Interceptor() {
    @Override
    public Response intercept(Chain chain) throws IOException {
        Response response = chain.proceed(chain.request());
        if(response.code() == STATUS_THAT_NEEDS_TIME_CONSUMING_OPERATION_TO_HANDLE) {
            executor.submit(new Runnable() {
                @Override
                public void run() {
                    // Show UI and take some credentials from user by using UI thread
                    // Access to remote authentication server
                    // do some time consuming cryptographic parallel operation using multiple CPU core
                    // store generated credentials via OS services
                    countDownLatch.countDown();
                }
            });
            countDownLatch.await(); // interceptor waits here until all other parallel operations get done.
            return chain.proceed(modifyRequestWithTheResultOfLongOperation(chain.request()));
        }
        return response;
    }
})

Dispatcher in OkHttpClient allows us the create new thread as much as the count of Integer.MAX_VALUE for AsyncCall’s. So we don’t get starve about threads. But I don’t sure about the solution above is good for performance or best practice.

Motivation

Having non-blocking interceptors looks like more suitable for async calls. I mean, interceptors may be implemented like Callbacks. In this way, we can proceed interceptor chain whenever we want, without the blocking interceptors thread.

Proposed Solution

In order to achieve this, we can create AsyncInterceptor interface like Interceptor interface. For instance, that interface and OkHttpClient implementation may be like this:

public class OkHttpClient {
  ...
  public List<Interceptor> interceptors();
  public List<Interceptor> networkInterceptors();
  public List<AsyncInterceptor> asyncInterceptors();
  ...
}

public interface AsyncInterceptor {
  void interceptRequest(Chain chain) throws IOException;
  void interceptResponse(Chain chain) throws IOException;

  interface Chain {
    Request request();
    Response response();
    void proceed(Request request) throws IOException;
    void proceed(Response response) throws IOException;
    void proceed(Exception error);
    ...
  }
}

In that case, interceptor chain should be look like this:

Same Example above with non-blocking interceptor

.addAsyncInterceptor(new AsyncInterceptor() {
  @Override
  public void interceptRequest(Chain chain) throws IOException {
    chain.proceed(chain.request());
  }
  @Override
  public void interceptResponse(Chain chain) throws IOException {
    if(chain.response().code == STATUS_THAT_NEEDS_TIME_CONSUMING_OPERATION_TO_HANDLE) {
      executor.submit(new Runnable() {
        @Override
        public void run() {
          // Show UI and take some credentials from user by using UI thread
          // Access to remote authentication server
          // do some time consuming cryptographic parallel operation using multiple CPU core
          // store generated credentials via OS services
          // Modifies request and sends again with Chain#proceed(Request)
          chain.proceed(modifyRequestWithTheResultOfLongOperation(chain.request()));
        }
      });
    }
    // Chain#proceed(Response)
    chain.proceed(chain.response());
  }
});

POC?

I implement basic POC version here: https://github.com/tolpp/okhttp/commit/3f3da9f284c06a599aa12861118c771038c82c05 This is not production ready, only for poc and show my purpose.

Is anyone else faced with same problem?

I can’t find too much actually. Some of that I find: This is the close one:

• https://stackoverflow.com/questions/32213382/okhttp-async-interceptor-or-authenticator

Maybe:

• https://stackoverflow.com/questions/32196424/how-to-add-headers-to-okhttp-request-interceptor/43422769
• https://stackoverflow.com/questions/28021485/retrofit-request-interceptor-blocks-the-main-thread

Similar issues:

• #3101

[ZakTaccardi]

I think this kind of problem is better solved at the application layer

It’s certainly solvable because we can attach headers asynchronously before delivering the initial request to OkHttpClient, but it’s certainly not better solved.

As coroutines users, we leverage actor coroutines to own and store state - this is how we achieve simple thread safety. This becomes a problem for us when APIs (like an OkHttpInterceptor) requests synchronous access to data. This also leads to developers being lazy and leveraging runBlocking to access the async actor data, simply because of the OkHttpInterceptor API. This leads to deadlocks in some rare edge cases.

In the builder of our SDK, we provide () -> SomeData (sync) and suspend () -> SomeData (async) overloads. If a client injects the sync overload, it gets internally wrapped by an async overload and our SDK only uses the async overload. That said, this is trivial to do because of coroutines (which OkHttp3 does not leverage). @swankjesse Maybe this is possible for something like OkHttp4 or OkHttp5?

[ZakTaccardi]

hopefully this will be on the wishlist for 4.0 #2903