ChuckerInterceptor fetches the whole body even if the application isn't doing so

Simply plugging in ChuckerInterceptor causes the whole response body to be read, even if the application isn’t reading the whole of it. Not only this is a very unwanted side effect, this can also crash the app. This is an example crash while trying to (not) read a ~50MB file:

2020-03-04 22:49:34.915 8061-8119 E/AndroidRuntime: FATAL EXCEPTION: OkHttp Dispatcher
    Process: process, PID: 8061
    java.lang.OutOfMemoryError: Failed to allocate a 102720552 byte allocation with 25165824 free bytes and 31MB until OOM, target footprint 260095632, growth limit 268435456
        at java.lang.StringFactory.newStringFromBytes(StringFactory.java:225)
        at java.lang.StringFactory.newStringFromBytes(StringFactory.java:249)
        at okio.Buffer.readString(Buffer.kt:306)
        at okio.Buffer.readString(Buffer.kt:295)
        at com.chuckerteam.chucker.api.ChuckerInterceptor.processResponseBody(ChuckerInterceptor.kt:159)
        at com.chuckerteam.chucker.api.ChuckerInterceptor.processResponse(ChuckerInterceptor.kt:133)
        at com.chuckerteam.chucker.api.ChuckerInterceptor.intercept(ChuckerInterceptor.kt:65)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:100)
        at okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.kt:197)
        at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:502)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.lang.Thread.run(Thread.java:919)

you can reproduce this by starting a fake web server and looking at how much it sends (and observing the crash):

cat /dev/urandom > file
{ echo -ne "HTTP/1.0 200 OK\r\nContent-Length: $(wc -c <file)\r\n\r\n"; cat file } | pv | nc -l 8080

It is my impression that the interceptor should only observe the doings of the application and should report either what the application reads or what the device receives.

P.S. setting maxContentLength in ChuckerInterceptor() seems to have no effect

[MiSikora]

What is happening is that the trailer bytes of the compressed stream are missing because you read only part of the stream and we save it as such to a file. So when we try to close the file that is being used to read original bytes it fails because the source is exhausted without expected bytes. I didn’t predict such a use case. I need to consider what is the best way to solve it. Could you report it as a separate issue and link to our conversation?

[oakkitten]

i probably should put try ... catch around the closing call

https://github.com/ubergeek42/weechat-android/blob/37bf333d780a66684fa698766a8cc14ea320ec7a/app/src/main/java/com/ubergeek42/WeechatAndroid/utils/Utils.java#L177-L190