Some tests fail on Android

I rigged up our test suite to run on an Android emulator. Here’s the logcat of the run: https://gist.github.com/swankjesse/a5b0014f315ede15fe0b916d1263fe0d

HTTPS client auth doesn’t work

2019-01-06 23:19:26.865 22793-22823/android.okhttp3.okhttpandroidtesting E/TestRunner: testFailure clientAuthForNeeds(okhttp3.internal.tls.ClientAuthTest)
    java.lang.AssertionError: expected:<CN=Jethro Willis> but was:<null>
        at org.junit.Assert.fail(Assert.java:88)
        at org.junit.Assert.failNotEquals(Assert.java:834)
        at org.junit.Assert.assertEquals(Assert.java:118)
        at org.junit.Assert.assertEquals(Assert.java:144)
at okhttp3.internal.tls.ClientAuthTest.clientAuthForNeeds(ClientAuthTest.java:136)

The X.509 DN parser treats whitespace differently

2019-01-06 23:19:27.431 22793-22823/android.okhttp3.okhttpandroidtesting E/TestRunner: testFailure trailingWhitespace(okhttp3.internal.tls.DistinguishedNameParserTest)
    org.junit.ComparisonFailure: cn="  a  b  " ,o=x expected:<[a  b]> but was:<[  a  b  ]>
        at org.junit.Assert.assertEquals(Assert.java:115)
        at okhttp3.internal.tls.DistinguishedNameParserTest.assertCn(DistinguishedNameParserTest.java:123)
at okhttp3.internal.tls.DistinguishedNameParserTest.trailingWhitespace(DistinguishedNameParserTest.java:111)

Anonymous server cipher suites don’t work

2019-01-06 23:19:52.892 22793-22823/android.okhttp3.okhttpandroidtesting E/TestRunner: testFailure tlsSuccessWithNoPeerCertificates(okhttp3.CallTest)
    java.lang.IllegalArgumentException: cipherSuite TLS_DH_anon_WITH_AES_128_GCM_SHA256 is not supported.
        at com.android.org.conscrypt.NativeCrypto.checkEnabledCipherSuites(NativeCrypto.java:942)
        at com.android.org.conscrypt.SSLParametersImpl.setEnabledCipherSuites(SSLParametersImpl.java:217)
        at com.android.org.conscrypt.ConscryptFileDescriptorSocket.setEnabledCipherSuites(ConscryptFileDescriptorSocket.java:700)
        at okhttp3.CallTest$26.configureSocket(CallTest.java:3662)
        at okhttp3.DelegatingSSLSocketFactory.createSocket(DelegatingSSLSocketFactory.java:74)
        at okhttp3.DelegatingSSLSocketFactory.createSocket(DelegatingSSLSocketFactory.java:28)
        at okhttp3.internal.connection.RealConnection.connectTls(RealConnection.java:309)
        at okhttp3.internal.connection.RealConnection.establishProtocol(RealConnection.java:284)
        at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:169)
        at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:257)
        at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:135)
        at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:114)
        at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:148)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:122)
        at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:94)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:148)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:122)
        at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:148)
        at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:125)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:148)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:122)
        at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:254)
        at okhttp3.RealCall.execute(RealCall.java:92)
at okhttp3.CallTest.tlsSuccessWithNoPeerCertificates(CallTest.java:1216)

Memory leak detection tests are broken

2019-01-06 23:20:14.359 22793-22823/android.okhttp3.okhttpandroidtesting E/TestRunner: testFailure exceptionThrownByOnResponseIsRedactedAndLogged(okhttp3.CallTest)
    java.lang.AssertionError: Timed out waiting for log message.
        at okhttp3.TestLogHandler.take(TestLogHandler.java:56)
at okhttp3.CallTest.exceptionThrownByOnResponseIsRedactedAndLogged(CallTest.java:755)

Tests assume too much about the TLS environment

2019-01-06 23:21:15.787 22793-22823/android.okhttp3.okhttpandroidtesting E/TestRunner: testFailure tls_defaultCiphers_noFallbackIndicator(okhttp3.ConnectionSpecTest)
    java.lang.IllegalArgumentException: cipherSuite SSL_RSA_WITH_RC4_128_MD5 is not supported.
        at com.android.org.conscrypt.NativeCrypto.checkEnabledCipherSuites(NativeCrypto.java:942)
        at com.android.org.conscrypt.SSLParametersImpl.setEnabledCipherSuites(SSLParametersImpl.java:217)
        at com.android.org.conscrypt.ConscryptFileDescriptorSocket.setEnabledCipherSuites(ConscryptFileDescriptorSocket.java:700)
at okhttp3.ConnectionSpecTest.tls_defaultCiphers_noFallbackIndicator(ConnectionSpecTest.java:88)

2019-01-06 23:21:15.795 22793-22823/android.okhttp3.okhttpandroidtesting E/TestRunner: testFailure allEnabledTlsVersions(okhttp3.ConnectionSpecTest)
    java.lang.AssertionError: expected:<[SSLv3, TLSv1.1]> but was:<[TLSv1.1]>
        at org.junit.Assert.fail(Assert.java:88)
        at org.junit.Assert.failNotEquals(Assert.java:834)
        at org.junit.Assert.assertEquals(Assert.java:118)
        at org.junit.Assert.assertEquals(Assert.java:144)
at okhttp3.ConnectionSpecTest.allEnabledTlsVersions(ConnectionSpecTest.java:235)

MediaType’s default charset isn’t null

2019-01-06 23:26:20.046 29557-29596/android.okhttp3.okhttpandroidtesting E/TestRunner: testFailure testCharsetNameIsDoubleQuotedAndSingleQuoted[Use get = true](okhttp3.MediaTypeTest)
    java.lang.AssertionError: expected null, but was:<UTF-8>
        at org.junit.Assert.fail(Assert.java:88)
        at org.junit.Assert.failNotNull(Assert.java:755)
        at org.junit.Assert.assertNull(Assert.java:737)
        at org.junit.Assert.assertNull(Assert.java:747)
at okhttp3.MediaTypeTest.testCharsetNameIsDoubleQuotedAndSingleQuoted(MediaTypeTest.java:180)

Overall nothing particularly alarming.

[swankjesse]

I don’t expect it to me too much work actually. We can use assumeTrue etc. to ignore platform tests that won’t run on Android.

And it’s way more robust to run all tests on Android than to maintain an Android subset.

[yschimke]

We can use assumeTrue etc. to ignore platform tests that won’t run on Android.

Expect to sprinkle this liberally 😃