AndroidChannels: failing requests right after network recovery

What version of gRPC are you using?

1.20.0

What did you expect to see?

In our project we use android JobService, which starts right after device establish network connection. Inside that service we run Grpc request over AndroidChannel. We expect to see successful network calls at that point

What actually happens.

Most of the time, this request fails few times in a row (as JobService reschedules itself on failure). At first it seems that though JobService is started, no connection is available for some reason, but then we tried to add simple http request via OkHttpClient - and they work well. Additional digging in logs and debugging reveals, that this might be related to DnsNameResolver, as it might resolve address when request is already in progress. This issue seems highly related to #4028, but occurs in AndroidChannel.

Steps to reproduce

Given mentioned setup (JobService + AndroidChannel), it requires android device with API 24+. Issue can be reproduced by toggling off device internet connection (both WiFi and mobile, seems no difference), wait around 10-15 seconds, and toggling on connection. Example project, which can demonstrate this issue can be found at https://github.com/AntKos/android-grpc-jobservice.

[hijamoya]

I faced a similar issue, say we have a foreground service which always running on the phone, then we turn off screen and wait a period (~4 or 5 hours) until the device enter doze idle mode. After that, we turn on the screen and the foreground service can not send any request by the AndroidChannel (but the rest api by retrofit is still working), the only way to recover is to turn off then turn on the wifi to make the channel reconnect again.

The follow properties we logged for the channel when the situation happened: isShutdown: false isTerminated: false state: READY Where the AndroidChannel is a global singleton for the application.

Thanks.

[voidzcy]

@AntKos I tried to run your example on Android emulator and it works for localhost (gonna try with some remote server later). Does this issue only occurs on physical devices? If it does, can you tell us the specific device model you tested on? Also, for the OkHttpClient used as comparison, have you ever tried a different target instead of “www.google.com” (as google.com is highly likely to be cached by DNS)?

Update: We will try to consistently reproduce the issue and dig into it.