RuntimeExceptions that is not handled by library and can not be caught by consumers (Inet6AddressImpl.java, NativeCrypto.java)
See original GitHub issueStep 2: Describe your environment
- Android Studio version: Android Studio 3.5.3 (Build #AI-191.8026.42.35.6010548, built on November 15, 2019)
- Firebase Component: messaging, analytics, config-ktx
- Component version: messaging(20.1.0), analytics (17.2.1), config-ktx (19.0.4), gms:google-services (4.3.3)
- Target SDK: 28
Step 3: Describe the problem
We get crash reports on Crashlytics, which does not show any error line on our code, but only internal code lines of FirebaseRemoteConfig
. We have two such issues.
First issue (Inet6AddressImpl.java)
Inet6AddressImpl.java 126 java.net.Inet6AddressImpl.lookupHostByName
Fatal Exception: com.google.firebase.remoteconfig.FirebaseRemoteConfigClientException: The client had an error while calling the backend!
at com.google.firebase.remoteconfig.internal.ConfigFetchHttpClient.fetch(SourceFile:194)
at com.google.firebase.remoteconfig.internal.g.c(SourceFile:278)
at com.google.firebase.remoteconfig.internal.g.b(SourceFile:251)
at com.google.firebase.remoteconfig.internal.g.a(SourceFile:191)
at com.google.firebase.remoteconfig.internal.g.a(SourceFile:160)
at com.google.firebase.remoteconfig.internal.h.a(:4)
at com.google.android.gms.tasks.n.run(:2)
at java.util.concurrent.ThreadPoolExecutor.processTask(ThreadPoolExecutor.java:1187)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at java.lang.Thread.run(Thread.java:784)
Caused by java.net.UnknownHostException: Unable to resolve host "firebaseremoteconfig.googleapis.com": No address associated with hostname
at java.net.Inet6AddressImpl.lookupHostByName(Inet6AddressImpl.java:126)
at java.net.Inet6AddressImpl.lookupAllHostAddr(Inet6AddressImpl.java:105)
at java.net.InetAddress.getAllByName(InetAddress.java:1154)
at com.android.okhttp.Dns$1.lookup(Dns.java:39)
at com.android.okhttp.internal.http.RouteSelector.resetNextInetSocketAddress(RouteSelector.java:175)
at com.android.okhttp.internal.http.RouteSelector.nextProxy(RouteSelector.java:141)
at com.android.okhttp.internal.http.RouteSelector.next(RouteSelector.java:83)
at com.android.okhttp.internal.http.StreamAllocation.findConnection(StreamAllocation.java:174)
at com.android.okhttp.internal.http.StreamAllocation.findHealthyConnection(StreamAllocation.java:126)
at com.android.okhttp.internal.http.StreamAllocation.newStream(StreamAllocation.java:95)
at com.android.okhttp.internal.http.HttpEngine.connect(HttpEngine.java:281)
at com.android.okhttp.internal.http.HttpEngine.sendRequest(HttpEngine.java:224)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:461)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.connect(HttpURLConnectionImpl.java:127)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getOutputStream(HttpURLConnectionImpl.java:258)
at com.android.okhttp.internal.huc.DelegatingHttpsURLConnection.getOutputStream(DelegatingHttpsURLConnection.java:218)
at com.android.okhttp.internal.huc.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:26)
at com.google.firebase.remoteconfig.internal.ConfigFetchHttpClient.a(SourceFile:316)
at com.google.firebase.remoteconfig.internal.ConfigFetchHttpClient.fetch(SourceFile:182)
at com.google.firebase.remoteconfig.internal.g.c(SourceFile:278)
at com.google.firebase.remoteconfig.internal.g.b(SourceFile:251)
at com.google.firebase.remoteconfig.internal.g.a(SourceFile:191)
at com.google.firebase.remoteconfig.internal.g.a(SourceFile:160)
at com.google.firebase.remoteconfig.internal.h.a(:4)
at com.google.android.gms.tasks.n.run(:2)
at java.util.concurrent.ThreadPoolExecutor.processTask(ThreadPoolExecutor.java:1187)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at java.lang.Thread.run(Thread.java:784)
This issue is reported under different groups in Crashlytics:
- Inet6AddressImpl.java line 126 (%99 android 9)
- Inet6AddressImpl.java line 111 (%99 android 8)
- Inet6AddressImpl.java line 95, Inet6AddressImpl.java line 100 (%99 android 7)
- InetAddress.java line 440, InetAddress.java line 445, InetAddress.java line 441 (%100 android 6)
- InetAddress.java line 427, InetAddress.java line 434 (%100 android 5)
- InetAddress.java line 394 (%99 android 4)
Second issue (NativeCrypto.java)
NativeCrypto.java com.google.android.gms.org.conscrypt.NativeCrypto.SSL_do_handshake
%41 android 9 devices
Fatal Exception: com.google.firebase.remoteconfig.FirebaseRemoteConfigClientException: The client had an error while calling the backend!
at com.google.firebase.remoteconfig.internal.ConfigFetchHttpClient.fetch(ConfigFetchHttpClient.java:194)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler.fetchFromBackend(ConfigFetchHandler.java:278)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler.fetchFromBackendAndCacheResponse(ConfigFetchHandler.java:251)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler.fetchIfCacheExpiredAndNotThrottled(ConfigFetchHandler.java:191)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler.lambda$fetch$0(ConfigFetchHandler.java:160)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler$$Lambda$1.then(ConfigFetchHandler.java)
at com.google.android.gms.tasks.zzf.run(zzf.java:2)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588)
at java.lang.Thread.run(Thread.java:818)
Caused by javax.net.ssl.SSLHandshakeException: Handshake failed
at com.google.android.gms.org.conscrypt.ConscryptFileDescriptorSocket.startHandshake(ConscryptFileDescriptorSocket.java:38)
at com.android.okhttp.internal.http.SocketConnector.connectTls(SocketConnector.java:212)
at com.android.okhttp.Connection.connect(Connection.java:1322)
at com.android.okhttp.Connection.connectAndSetOwner(Connection.java:1410)
at com.android.okhttp.OkHttpClient$1.connectAndSetOwner(OkHttpClient.java:128)
at com.android.okhttp.internal.http.HttpEngine.nextConnection(HttpEngine.java:466)
at com.android.okhttp.internal.http.HttpEngine.connect(HttpEngine.java:447)
at com.android.okhttp.internal.http.HttpEngine.sendRequest(HttpEngine.java:353)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:476)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.connect(HttpURLConnectionImpl.java:118)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getOutputStream(HttpURLConnectionImpl.java:249)
at com.android.okhttp.internal.huc.DelegatingHttpsURLConnection.getOutputStream(DelegatingHttpsURLConnection.java:218)
at com.android.okhttp.internal.huc.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:25)
at com.google.firebase.remoteconfig.internal.ConfigFetchHttpClient.setFetchRequestBody(ConfigFetchHttpClient.java:316)
at com.google.firebase.remoteconfig.internal.ConfigFetchHttpClient.fetch(ConfigFetchHttpClient.java:182)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler.fetchFromBackend(ConfigFetchHandler.java:278)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler.fetchFromBackendAndCacheResponse(ConfigFetchHandler.java:251)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler.fetchIfCacheExpiredAndNotThrottled(ConfigFetchHandler.java:191)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler.lambda$fetch$0(ConfigFetchHandler.java:160)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler$$Lambda$1.then(ConfigFetchHandler.java)
at com.google.android.gms.tasks.zzf.run(zzf.java:2)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588)
at java.lang.Thread.run(Thread.java:818)
Caused by javax.net.ssl.SSLProtocolException: SSL handshake aborted: ssl=0x956e27a8: Failure in SSL library, usually a protocol error
error:1000043e:SSL routines:OPENSSL_internal:TLSV1_ALERT_INAPPROPRIATE_FALLBACK (third_party/openssl/boringssl/src/ssl/tls_record.cc:592 0xa86f9a08:0x00000001)
at com.google.android.gms.org.conscrypt.NativeCrypto.SSL_do_handshake(NativeCrypto.java)
at com.google.android.gms.org.conscrypt.NativeSsl.doHandshake(NativeSsl.java:6)
at com.google.android.gms.org.conscrypt.ConscryptFileDescriptorSocket.startHandshake(ConscryptFileDescriptorSocket.java:14)
at com.android.okhttp.internal.http.SocketConnector.connectTls(SocketConnector.java:212)
at com.android.okhttp.Connection.connect(Connection.java:1322)
at com.android.okhttp.Connection.connectAndSetOwner(Connection.java:1410)
at com.android.okhttp.OkHttpClient$1.connectAndSetOwner(OkHttpClient.java:128)
at com.android.okhttp.internal.http.HttpEngine.nextConnection(HttpEngine.java:466)
at com.android.okhttp.internal.http.HttpEngine.connect(HttpEngine.java:447)
at com.android.okhttp.internal.http.HttpEngine.sendRequest(HttpEngine.java:353)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:476)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.connect(HttpURLConnectionImpl.java:118)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getOutputStream(HttpURLConnectionImpl.java:249)
at com.android.okhttp.internal.huc.DelegatingHttpsURLConnection.getOutputStream(DelegatingHttpsURLConnection.java:218)
at com.android.okhttp.internal.huc.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:25)
at com.google.firebase.remoteconfig.internal.ConfigFetchHttpClient.setFetchRequestBody(ConfigFetchHttpClient.java:316)
at com.google.firebase.remoteconfig.internal.ConfigFetchHttpClient.fetch(ConfigFetchHttpClient.java:182)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler.fetchFromBackend(ConfigFetchHandler.java:278)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler.fetchFromBackendAndCacheResponse(ConfigFetchHandler.java:251)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler.fetchIfCacheExpiredAndNotThrottled(ConfigFetchHandler.java:191)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler.lambda$fetch$0(ConfigFetchHandler.java:160)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler$$Lambda$1.then(ConfigFetchHandler.java)
at com.google.android.gms.tasks.zzf.run(zzf.java:2)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588)
at java.lang.Thread.run(Thread.java:818)
Similar issue happens with above NativeSsl.java line 10 com.google.android.gms.org.conscrypt.NativeSsl.initialize
%74 android 5
Fatal Exception: com.google.firebase.remoteconfig.FirebaseRemoteConfigClientException: The client had an error while calling the backend!
at com.google.firebase.remoteconfig.internal.ConfigFetchHttpClient.fetch(ConfigFetchHttpClient.java:194)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler.fetchFromBackend(ConfigFetchHandler.java:278)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler.fetchFromBackendAndCacheResponse(ConfigFetchHandler.java:251)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler.fetchIfCacheExpiredAndNotThrottled(ConfigFetchHandler.java:191)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler.lambda$fetch$0(ConfigFetchHandler.java:160)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler$$Lambda$1.then(ConfigFetchHandler.java)
at com.google.android.gms.tasks.zzf.run(zzf.java:2)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
at java.lang.Thread.run(Thread.java:818)
Caused by javax.net.ssl.SSLHandshakeException: No enabled protocols; SSLv3 is no longer supported and was filtered from the list
at com.google.android.gms.org.conscrypt.NativeSsl.initialize(NativeSsl.java:10)
at com.google.android.gms.org.conscrypt.ConscryptFileDescriptorSocket.startHandshake(ConscryptFileDescriptorSocket.java:6)
at com.android.okhttp.Connection.upgradeToTls(Connection.java:1323)
at com.android.okhttp.Connection.connect(Connection.java:1225)
at com.android.okhttp.internal.http.HttpEngine.connect(HttpEngine.java:389)
at com.android.okhttp.internal.http.HttpEngine.sendRequest(HttpEngine.java:298)
at com.android.okhttp.internal.http.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:399)
at com.android.okhttp.internal.http.HttpURLConnectionImpl.connect(HttpURLConnectionImpl.java:110)
at com.android.okhttp.internal.http.HttpURLConnectionImpl.getOutputStream(HttpURLConnectionImpl.java:221)
at com.android.okhttp.internal.http.DelegatingHttpsURLConnection.getOutputStream(DelegatingHttpsURLConnection.java:218)
at com.android.okhttp.internal.http.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:25)
at com.google.firebase.remoteconfig.internal.ConfigFetchHttpClient.setFetchRequestBody(ConfigFetchHttpClient.java:316)
at com.google.firebase.remoteconfig.internal.ConfigFetchHttpClient.fetch(ConfigFetchHttpClient.java:182)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler.fetchFromBackend(ConfigFetchHandler.java:278)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler.fetchFromBackendAndCacheResponse(ConfigFetchHandler.java:251)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler.fetchIfCacheExpiredAndNotThrottled(ConfigFetchHandler.java:191)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler.lambda$fetch$0(ConfigFetchHandler.java:160)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler$$Lambda$1.then(ConfigFetchHandler.java)
at com.google.android.gms.tasks.zzf.run(zzf.java:2)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
at java.lang.Thread.run(Thread.java:818)
%71 android 9
Fatal Exception: com.google.firebase.remoteconfig.FirebaseRemoteConfigClientException: The client had an error while calling the backend!
at com.google.firebase.remoteconfig.internal.ConfigFetchHttpClient.fetch(ConfigFetchHttpClient.java:194)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler.fetchFromBackend(ConfigFetchHandler.java:278)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler.fetchFromBackendAndCacheResponse(ConfigFetchHandler.java:251)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler.fetchIfCacheExpiredAndNotThrottled(ConfigFetchHandler.java:191)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler.lambda$fetch$0(ConfigFetchHandler.java:160)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler$$Lambda$1.a(ConfigFetchHandler.java:4)
at com.google.android.gms.tasks.zzf.run(zzf.java:2)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1162)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:636)
at java.lang.Thread.run(Thread.java:764)
Caused by javax.net.ssl.SSLHandshakeException: Connection closed by peer
at com.android.org.conscrypt.NativeCrypto.SSL_do_handshake(NativeCrypto.java)
at com.android.org.conscrypt.SslWrapper.doHandshake(SslWrapper.java:374)
at com.android.org.conscrypt.ConscryptFileDescriptorSocket.startHandshake(ConscryptFileDescriptorSocket.java:217)
at com.android.okhttp.internal.io.RealConnection.connectTls(RealConnection.java:192)
at com.android.okhttp.internal.io.RealConnection.connectSocket(RealConnection.java:149)
at com.android.okhttp.internal.io.RealConnection.connect(RealConnection.java:112)
at com.android.okhttp.internal.http.StreamAllocation.findConnection(StreamAllocation.java:184)
at com.android.okhttp.internal.http.StreamAllocation.findHealthyConnection(StreamAllocation.java:126)
at com.android.okhttp.internal.http.StreamAllocation.newStream(StreamAllocation.java:95)
at com.android.okhttp.internal.http.HttpEngine.connect(HttpEngine.java:294)
at com.android.okhttp.internal.http.HttpEngine.sendRequest(HttpEngine.java:237)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:461)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.connect(HttpURLConnectionImpl.java:127)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getOutputStream(HttpURLConnectionImpl.java:258)
at com.android.okhttp.internal.huc.DelegatingHttpsURLConnection.getOutputStream(DelegatingHttpsURLConnection.java:218)
at com.android.okhttp.internal.huc.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java)
at com.google.firebase.remoteconfig.internal.ConfigFetchHttpClient.setFetchRequestBody(ConfigFetchHttpClient.java:316)
at com.google.firebase.remoteconfig.internal.ConfigFetchHttpClient.fetch(ConfigFetchHttpClient.java:182)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler.fetchFromBackend(ConfigFetchHandler.java:278)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler.fetchFromBackendAndCacheResponse(ConfigFetchHandler.java:251)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler.fetchIfCacheExpiredAndNotThrottled(ConfigFetchHandler.java:191)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler.lambda$fetch$0(ConfigFetchHandler.java:160)
at com.google.firebase.remoteconfig.internal.ConfigFetchHandler$$Lambda$1.a(ConfigFetchHandler.java:4)
at com.google.android.gms.tasks.zzf.run(zzf.java:2)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1162)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:636)
at java.lang.Thread.run(Thread.java:764)
Steps to reproduce:
-
First issue started to occur for some users, right after we integrated
FirebaseRemoteConfig
. Right now we can reproduce it on emulator device, by turning off network connections and opening app (emulator x86, api26, android 8.0, pixel2). IfFirebaseRemoteConfig
is not initialized, no crash occurs. -
Second issue is special, happened today on most of the users. Today we had internet related issue on the region (slow youtube, unaccessible sites, etc…), heard it is Google DNS related issue. During this regional internet problem,
FirebaseRemoteConfig
started to throw runtime exceptions.
Relevant Code:
These are all FirebaseRemoteConfig
code lines:
remoteConfig = FirebaseRemoteConfig.getInstance()
val configSettings = FirebaseRemoteConfigSettings.Builder()
.setFetchTimeoutInSeconds(..)
.setMinimumFetchIntervalInSeconds(..)
.build()
remoteConfig.setConfigSettingsAsync(configSettings)
remoteConfig.setDefaultsAsync(R.xml.remote_config_defaults)
...
remoteConfig.fetchAndActivate()
.addOnSuccessListener { .. }
.addOnCanceledListener { .. }
.addOnFailureListener { .. }
...
remoteConfig.getBoolean(key)
remoteConfig.getString(key)
remoteConfig.getLong(key)
All the lines above have try/catch except setConfigSettingsAsync
and setDefaultsAsync
. Replaced those with sync versions (setConfigSettings()
,setDefaults()
) and surrounded with try/catch, still can reproduce first issue mentioned above by turning off internet connection on emulator. If FirebaseRemoteConfig
is not initialized, crash does not occur.
Observations
- It is surprising that first issue is never reported here (we didn’t report either for long time, because it happens on some users, but in all android versions)
- Second issue is specific, which happened only today, when region had internet issue
None of the issues mentioned above has line corresponding to our code, and sorrounding them with try/catch can never catch crash exceptions. Hence, we conclude that FirebaseRemoteConfig
internally does some asynchronous tasks and does not catch some RuntimeExceptions (which we can’t catch either, because they are async) and those RuntimeExceptions make apps crash.
Issue Analytics
- State:
- Created 4 years ago
- Reactions:5
- Comments:5 (1 by maintainers)
Top GitHub Comments
This exception occurs when
remoteConfig.fetchAndActivate()
is called, which propagates error toaddOnFailureListener
callback. Not handling the exception correctly inaddOnFailureListener
allowed this exception escape the cage. It can be reproduced by putting phone into Airplane mode.We have the same issue