Spike in "Trust anchor for certification path not found " errors

I’ve been getting a lot of reports from Crashlytics in the past week about the following issue affecting ~10% of my user base. I’ve never seen this issue myself, and it never happens in our dev server environment.

I ran the ssllabs.com and digicert.com analysis and there were no issues.

We do use cert pinning through Android.

It’s happening on Android 7-10 and all the big devices and manufacturers.

network_security_config.xml

<?xml version="1.0" encoding="utf-8"?>
<network-security-config>
  <domain-config>
    <domain includeSubdomains="true">mydomain.com</domain>
    <pin-set>
      <!--
       NOTE recommendation is to have a second certificate to pin just in case the
       main one needs to be invalidated
       -->
      <pin digest="SHA-256">the_hash</pin>
    </pin-set>
  </domain-config>
</network-security-config>

Crash:

Caused by java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.
       at com.android.org.conscrypt.TrustManagerImpl.checkTrustedRecursive(TrustManagerImpl.java:654)
       at com.android.org.conscrypt.TrustManagerImpl.checkTrusted(TrustManagerImpl.java:499)
       at com.android.org.conscrypt.TrustManagerImpl.checkTrusted(TrustManagerImpl.java:422)
       at com.android.org.conscrypt.TrustManagerImpl.getTrustedChainForServer(TrustManagerImpl.java:343)
       at android.security.net.config.NetworkSecurityTrustManager.checkServerTrusted(NetworkSecurityTrustManager.java:94)
       at android.security.net.config.RootTrustManager.checkServerTrusted(RootTrustManager.java:88)
       at com.android.org.conscrypt.Platform.checkServerTrusted(Platform.java:208)
       at com.android.org.conscrypt.ConscryptFileDescriptorSocket.verifyCertificateChain(ConscryptFileDescriptorSocket.java:426)
       at com.android.org.conscrypt.NativeCrypto.SSL_do_handshake(NativeCrypto.java)
       at com.android.org.conscrypt.NativeSsl.doHandshake(NativeSsl.java:383)
       at com.android.org.conscrypt.ConscryptFileDescriptorSocket.startHandshake(ConscryptFileDescriptorSocket.java:231)
       at okhttp3.internal.connection.RealConnection.connectTls(RealConnection.java:367)
       at okhttp3.internal.connection.RealConnection.establishProtocol(RealConnection.java:325)
       at okhttp3.internal.connection.ExchangeFinder.findConnection(ExchangeFinder.java:197)
       at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:197)
       at okhttp3.internal.connection.ExchangeFinder.findConnection(ExchangeFinder.java:249)
       at okhttp3.internal.connection.ExchangeFinder.findHealthyConnection(ExchangeFinder.java:108)
       at okhttp3.internal.connection.ExchangeFinder.find(ExchangeFinder.java:76)
       at okhttp3.internal.connection.RealCall.initExchange$okhttp(RealCall.java:245)
       at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:32)
       at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:100)
       at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:96)
       at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:100)
       at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:83)
       at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:100)
       at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:76)
       at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:100)
       at okhttp3.logging.HttpLoggingInterceptor.intercept(HttpLoggingInterceptor.java:219)
       at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:100)
       at co.twenty.api.CustomPropertyInterceptor.intercept(CustomPropertyInterceptor.java:49)
       at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:100)
       at co.twenty.api.DefaultParamsInterceptor.intercept(DefaultParamsInterceptor.java:44)
       at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:100)
       at okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.java:197)
       at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.java:502)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
       at java.lang.Thread.run(Thread.java:764)

[eygraber]

I think that particular case is a wash for us (we’re not building for their intranet).

I’d say it’s very likely that this is the main cause of the errors that we’re seeing, since it spiked around the time we partnered with that campus.

[eygraber]

We recently released a version with this debugging in, and got our first hit!

Non-fatal Exception: javax.net.ssl.SSLPeerUnverifiedException: Hostname <my host> not verified:
    certificate: sha256/<redacted>
    DN: CN=securelogin.<some university>.edu,O=<name of university>,L=<city of university>,ST=<state of university>,C=US
    subjectAltNames: [securelogin.<some university>.edu, www.securelogin.<some university>.edu]

Looks like the university network is MITM us (I’m assuming it’s a configuration thing, and not intentional because we partner with them, unless there’s a bad actor somewhere).

I’ll keep the debugging in for a few versions and see if anything else interesting pops up.