question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

java.net.SocketTimeoutException from HTTP/2 connection leaves dead okhttp clients in pool

See original GitHub issue

Tried writing a unit test w/ TestButler on Android w/ no luck, so I’ll write up the steps to reproduce this and include some sample code. This happens if you connect to an HTTP/2 server and your network goes down while the okhttp client is connected to it:

  1. create an okhttp client
  2. tell it to read from the HTTP/2 server
  3. bring the network down
  4. tell it to read from the HTTP/2 server (it’ll get a SocketTimeoutException)
  5. bring the network back up
  6. tell it to read from the HTTP/2 server again (it’ll be stuck w/ SocketTimeoutExceptions)
  7. if you create new http clients at this point, it’ll work, but the dead http client will eventually come back in the pool and fail.

okhttp client should attempt to reopen the HTTP/2 connection instead of being stuck in this state

Code sample for Android (create a trivial view w/ a button and a textview):

public class MainActivity extends AppCompatActivity {
    OkHttpClient okhttpClient = new OkHttpClient();

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        Button loadButton = (Button) findViewById(R.id.loadButton);
        TextView outputView = (TextView) findViewById(R.id.outputView);

        loadButton.setOnClickListener(view -> Observable.fromCallable(() -> {
                    Request request = new Request.Builder()
                            .url(<INSERT URL TO YOUR HTTP/2 SERVER HERE>)
                            .build();

                    Response response = okhttpClient.newCall(request).execute();

                    return response.body().string();
                })
                .subscribeOn(Schedulers.io())
                .observeOn(AndroidSchedulers.mainThread())
                .subscribe(outputView::setText, t -> outputView.setText(t.toString()))
        );
    }
}

Issue Analytics

  • State:open
  • Created 7 years ago
  • Reactions:30
  • Comments:139 (21 by maintainers)

github_iconTop GitHub Comments

34reactions
jpearlcommented, Jun 26, 2017

I think i’m seeing another manifestation of this on 3.5.0, when the server forcibly closes the connection.

We try to establish both a h2 and http1.1 connection. The server responds with 200 to both:

06-26 15:07:55.286 22094 22380 I okhttp3.OkHttpClient: --> GET<url> http/1.1
06-26 15:07:55.524 22094 22380 I okhttp3.OkHttpClient: --> GET<url> h2

06-26 15:07:55.596 22094 22380 I okhttp3.OkHttpClient: <-- 200  <url> (71ms)
06-26 15:07:55.597 22094 22380 I okhttp3.OkHttpClient: <-- 200  <url> (303ms)

Then at some point we try to read from the http2 connection, which fails in checkNotClosed and throws a StreamResetException

06-26 15:06:01.560 22094 22126 I MyProject: Caused by: okhttp3.internal.http2.StreamResetException: stream was reset: PROTOCOL_ERROR
06-26 15:06:01.560 22094 22126 I MyProject: 	at okhttp3.internal.http2.Http2Stream$FramedDataSource.checkNotClosed(Http2Stream.java:428)
06-26 15:06:01.560 22094 22126 I MyProject: 	at okhttp3.internal.http2.Http2Stream$FramedDataSource.read(Http2Stream.java:330)
06-26 15:06:01.560 22094 22126 I MyProject: 	at okio.ForwardingSource.read(ForwardingSource.java:35)
06-26 15:06:01.560 22094 22126 I MyProject: 	at okio.RealBufferedSource$1.read(RealBufferedSource.java:409)
06-26 15:06:01.560 22094 22126 I MyProject: 	at com.google.android.exoplayer.upstream.HttpDataSource.read(HttpDataSourceImpl.java:699)
06-26 15:06:01.560 22094 22126 I MyProject: 	at com.google.android.exoplayer.upstream.HttpDataSource.read(HttpDataSourceImpl.java:424)

Then, since this is media, we do something that causes a seek to 0 in the media, which needs to reopen the request from the beginning. At this point, we see the same exception as is posted above:

06-26 15:08:39.387 22094 22126 I MyProject: Caused by: java.net.SocketTimeoutException: timeout
06-26 15:08:39.387 22094 22126 I MyProject: 	at okhttp3.internal.http2.Http2Stream$StreamTimeout.newTimeoutException(Http2Stream.java:587)
06-26 15:08:39.387 22094 22126 I MyProject: 	at okhttp3.internal.http2.Http2Stream$StreamTimeout.exitAndThrowIfTimedOut(Http2Stream.java:595)
06-26 15:08:39.387 22094 22126 I MyProject: 	at okhttp3.internal.http2.Http2Stream.getResponseHeaders(Http2Stream.java:140)
06-26 15:08:39.387 22094 22126 I MyProject: 	at okhttp3.internal.http2.Http2Codec.readResponseHeaders(Http2Codec.java:115)
06-26 15:08:39.387 22094 22126 I MyProject: 	at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:54)
06-26 15:08:39.387 22094 22126 I MyProject: 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
06-26 15:08:39.387 22094 22126 I MyProject: 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
06-26 15:08:39.387 22094 22126 I MyProject: 	at okhttp3.logging.HttpLoggingInterceptor.intercept(HttpLoggingInterceptor.java:212)
06-26 15:08:39.387 22094 22126 I MyProject: 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
06-26 15:08:39.387 22094 22126 I MyProject: 	at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:45)
06-26 15:08:39.387 22094 22126 I MyProject: 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
06-26 15:08:39.387 22094 22126 I MyProject: 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
06-26 15:08:39.387 22094 22126 I MyProject: 	at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
06-26 15:08:39.387 22094 22126 I MyProject: 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
06-26 15:08:39.387 22094 22126 I MyProject: 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
06-26 15:08:39.387 22094 22126 I MyProject: 	at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
06-26 15:08:39.387 22094 22126 I MyProject: 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
06-26 15:08:39.387 22094 22126 I MyProject: 	at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:120)
06-26 15:08:39.387 22094 22126 I MyProject: 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
06-26 15:08:39.387 22094 22126 I MyProject: 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
06-26 15:08:39.387 22094 22126 I MyProject: 	at okhttp3.logging.HttpLoggingInterceptor.intercept(HttpLoggingInterceptor.java:212)
06-26 15:08:39.387 22094 22126 I MyProject: 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
06-26 15:08:39.387 22094 22126 I MyProject: 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
06-26 15:08:39.387 22094 22126 I MyProject: 	at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:179)
06-26 15:08:39.387 22094 22126 I MyProject: 	at okhttp3.RealCall.execute(RealCall.java:63)

this seems to be very similar to the other cases here, which seem to all be related to an ungraceful shutdown of the connection, and it remaining pooled.

I’ve also confirmed that disabling the ConnectionPool “works around” this issue:

OkHttpClient.Builder clientBuilder = new OkHttpClient.Builder()
            .connectTimeout(connectTimeoutMillis, TimeUnit.MILLISECONDS)
            .retryOnConnectionFailure(true)
            .readTimeout(readTimeoutMillis, TimeUnit.MILLISECONDS).connectionPool(new ConnectionPool(0, 1, TimeUnit.NANOSECONDS));
13reactions
CarsonRedeyecommented, Mar 20, 2020

Also still getting this problem on emulator with api 22, and 3.14.4. Also I get a SocketTimeoutException after 2 minutes (what my readTimeout is set to), instead of 10 seconds (what my connectTimeout is set to). The workaround using .connectionPool(new ConnectionPool(0, 1, TimeUnit.NANOSECONDS)) still works. I’d say it’s time to re-open this 😦. Steps to reproduce are same as OP.

I can confirm the issue doesn’t exist when using a real device Note 9, API 29.

Read more comments on GitHub >

github_iconTop Results From Across the Web

java.net.SocketTimeoutException from HTTP/2 connection leaves ...
java.net.SocketTimeoutException from HTTP/2 connection leaves dead okhttp clients in pool · 1) create an okhttp client · 2) tell it to read from...
Read more >
SocketTimeoutException in Retrofit - android - Stack Overflow
I am trying to POST request to server for fetch data but sometime It's occure SocketTimeoutException !
Read more >
java.net.SocketTimeoutException with no lines referencing my ...
java.net.SocketTimeoutException: Read timed out is quite usual error then a peer is not responding for too long due to service outage or poor...
Read more >
Diff - platform/external/okhttp - Google Git
Previously OkHttp's connection pool + managed both idle and active connections for HTTP/2, but only idle + connections for HTTP/1.x.
Read more >
Socket Connection Timeout Issue In Built Jar File - ADocLib
I built my OkHttpClient using the code below as suggested in java.net.SocketTimeoutException from HTTP/2 connection leaves dead okhttp. Leave a Reply.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found