Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

SSLPeerUnverifiedException on S3 actions

See original GitHub issue

I have recently re-written a service to use the newer AWS SDK (v2), but I am struggling with an error I just can’t seem to figure out.

Short snippet:

javax.net.ssl.SSLPeerUnverifiedException: Certificate for <s3s-nv.s3.amazonaws.com> doesn't match any of the subject alternative names: [*.s3.amazonaws.com, s3.amazonaws.com]

Description

This service communicates a lot with a few other AWS services, and everything there is fine, but when it is running in production, it seems to have issues writing to customer buckets with the error above.

I have gone ahead and changed this project to only use AWS SDK v1 for S3 with almost identical commands, and it works fine.

I am running in an EC2 instance, using JDK 8, and using the latest version of this library.

Full stack trace

javax.net.ssl.SSLPeerUnverifiedException: Certificate for <s3s-nv.s3.amazonaws.com> doesn't match any of the subject alternative names: [*.s3.amazonaws.com, s3.amazonaws.com]
	at org.apache.http.conn.ssl.SSLConnectionSocketFactory.verifyHostname(SSLConnectionSocketFactory.java:507)
	at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:437)
	at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:384)
	at software.amazon.awssdk.http.apache.internal.conn.SdkTlsSocketFactory.connectSocket(SdkTlsSocketFactory.java:113)
	at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
	at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376)
	at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at software.amazon.awssdk.http.apache.internal.conn.ClientConnectionManagerFactory$Handler.invoke(ClientConnectionManagerFactory.java:80)
	at com.sun.proxy.$Proxy58.connect(Unknown Source)
	at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393)
	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
	at software.amazon.awssdk.http.apache.internal.impl.ApacheSdkHttpClient.execute(ApacheSdkHttpClient.java:72)
	at software.amazon.awssdk.http.apache.ApacheHttpClient.execute(ApacheHttpClient.java:232)
	at software.amazon.awssdk.http.apache.ApacheHttpClient.access$500(ApacheHttpClient.java:98)
	at software.amazon.awssdk.http.apache.ApacheHttpClient$1.call(ApacheHttpClient.java:213)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeHttpRequestStage.executeHttpRequest(MakeHttpRequestStage.java:66)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeHttpRequestStage.execute(MakeHttpRequestStage.java:51)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeHttpRequestStage.execute(MakeHttpRequestStage.java:35)
	at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
	at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
	at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
	at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:73)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:42)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:77)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:39)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:64)
	... 31 common frames omitted
Wrapped by: software.amazon.awssdk.core.exception.SdkClientException: Unable to execute HTTP request: Certificate for <s3s-nv.s3.amazonaws.com> doesn't match any of the subject alternative names: [*.s3.amazonaws.com, s3.amazonaws.com]
	at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:98)
	at software.amazon.awssdk.core.exception.SdkClientException.create(SdkClientException.java:43)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.utils.RetryableStageHelper.setLastException(RetryableStageHelper.java:201)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:66)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:34)
	at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
	at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:56)
	at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:36)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42)
	at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
	at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26)
	at software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:189)
	at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:121)
	at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:147)
	at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:101)
	at software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45)
	at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:55)
	at software.amazon.awssdk.services.s3.DefaultS3Client.copyObject(DefaultS3Client.java:970)
	at software.amazon.awssdk.services.s3.S3Client.copyObject(S3Client.java:1503)
	at com.sednanetwork.ingest.receiver.Receiver.copyEmailToTenantBucket(Receiver.kt:131)
	at com.sednanetwork.ingest.receiver.Receiver.doReceive(Receiver.kt:86)
	at com.sednanetwork.ingest.receiver.Receiver.receive(Receiver.kt:52)
	at com.sednanetwork.ingest.IngestApplication$run$handler$1.invoke(IngestApplication.kt:53)
	at com.sednanetwork.ingest.IngestApplication$run$handler$1.invoke(IngestApplication.kt:19)
	at com.sednanetwork.sqs.SqsConsumer$launchWorker$1$1$1.invokeSuspend(SqsConsumer.kt:78)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)

Issue Analytics

State:
Created 3 years ago
Reactions:3
Comments:22 (5 by maintainers)

Top GitHub Comments

3reactions

raonitimocommented, Sep 23, 2020

Hi @debora-ito , I’ve observed endpoints .s3.amazonaws.com and .s3.us-east-1.amazonaws.com return different certificates.

Global endpoint:

$ true | openssl s_client -connect some-bucket.s3.amazonaws.com:443 2>/dev/null |  openssl x509 -noout -text

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            08:2d:f6:8e:e9:c6:93:15:be:bf:72:07:9b:38:10:fd
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: C=US, O=DigiCert Inc, OU=www.digicert.com, CN=DigiCert Baltimore CA-2 G2
        Validity
            Not Before: Nov  9 00:00:00 2019 GMT
            Not After : Mar 12 12:00:00 2021 GMT
        Subject: C=US, ST=Washington, L=Seattle, O=Amazon.com, Inc., CN=*.s3.amazonaws.com
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
...
        X509v3 extensions:
            X509v3 Subject Alternative Name:
                DNS:*.s3.amazonaws.com, DNS:s3.amazonaws.com
...

Regional endpoint:

$ true | openssl s_client -connect some-bucket.s3.us-east-1.amazonaws.com:443 2>/dev/null |  openssl x509 -noout -text

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            0d:64:50:6b:45:f3:0c:e3:5a:6c:2d:df:2c:18:b4:37
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: C=US, O=DigiCert Inc, OU=www.digicert.com, CN=DigiCert Baltimore CA-2 G2
        Validity
            Not Before: Aug  4 00:00:00 2020 GMT
            Not After : Aug  9 12:00:00 2021 GMT
        Subject: C=US, ST=Washington, L=Seattle, O=Amazon.com, Inc., CN=s3.amazonaws.com
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
...
        X509v3 extensions:
            X509v3 Subject Alternative Name:
                DNS:s3.amazonaws.com, DNS:*.s3.amazonaws.com, DNS:*.s3.dualstack.us-east-1.amazonaws.com, DNS:s3.dualstack.us-east-1.amazonaws.com, DNS:*.s3.us-east-1.amazonaws.com, DNS:s3.us-east-1.amazonaws.com, DNS:*.s3-control.us-east-1.amazonaws.com, DNS:s3-control.us-east-1.amazonaws.com, DNS:*.s3-control.dualstack.us-east-1.amazonaws.com, DNS:s3-control.dualstack.us-east-1.amazonaws.com, DNS:*.s3-accesspoint.us-east-1.amazonaws.com, DNS:*.s3-accesspoint.dualstack.us-east-1.amazonaws.com, DNS:*.s3.us-east-1.vpce.amazonaws.com
....

So, this code:

import software.amazon.awssdk.regions.Region
import software.amazon.awssdk.services.s3.S3Client
import software.amazon.awssdk.services.s3.model.DeleteObjectRequest
import software.amazon.awssdk.services.s3.model.DeleteObjectsRequest
import software.amazon.awssdk.services.s3.model.GetObjectRequest
import software.amazon.awssdk.services.s3.model.ListObjectsV2Request
import software.amazon.awssdk.services.s3.model.ObjectIdentifier
import software.amazon.awssdk.services.s3.model.S3Object
import software.amazon.awssdk.services.s3.presigner.S3Presigner

fun main() {
        val bucket = awsConfiguration.bucket
        val prefix = "foo/bar"

        val listObjectsV2PaginatorResult = s3Client.listObjectsV2Paginator(
            ListObjectsV2Request
                .builder()
                .bucket(bucket)
                .prefix(prefix)
                .build()
        )

        val keys: List<String> = listObjectsV2PaginatorResult
            .contents()
            .stream()
            .map { it.key() }
            .toList()

        logger.info { "==================>>>>> KEYS: $keys" }
}

Worked in EU-CENTRAL-1 , but returned SdkClientException: Unable to execute HTTP request: Certificate for <some-bucket.s3.amazonaws.com> doesn’t match any of the subject alternative names: [*.s3.amazonaws.com, s3.amazonaws.com] is US-EAST-1.

Full stack trace:

software.amazon.awssdk.core.exception.SdkClientException: Unable to execute HTTP request: Certificate for <some-bucket.s3.amazonaws.com> doesn't match any of the subject alternative names: [*.s3.amazonaws.com, s3.amazonaws.com]
    at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:97)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage$RetryExecutor.handleThrownException(RetryableStage.java:137)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage$RetryExecutor.execute(RetryableStage.java:95)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:63)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:43)
    at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
    at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:57)
    at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:37)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:81)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:61)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:43)
    at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
    at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26)
    at software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:198)
    at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:122)
    at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:148)
    at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:102)
    at software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45)
    at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:55)
    at software.amazon.awssdk.services.s3.DefaultS3Client.listObjectsV2(DefaultS3Client.java:4926)
    at software.amazon.awssdk.services.s3.paginators.ListObjectsV2Iterable$ListObjectsV2ResponseFetcher.nextPage(ListObjectsV2Iterable.java:147)
    at software.amazon.awssdk.services.s3.paginators.ListObjectsV2Iterable$ListObjectsV2ResponseFetcher.nextPage(ListObjectsV2Iterable.java:138)
    at software.amazon.awssdk.core.pagination.sync.PaginatedResponsesIterator.next(PaginatedResponsesIterator.java:58)
    at software.amazon.awssdk.core.pagination.sync.PaginatedItemsIterable$ItemsIterator.<init>(PaginatedItemsIterable.java:58)
    at software.amazon.awssdk.core.pagination.sync.PaginatedItemsIterable.iterator(PaginatedItemsIterable.java:48)
    at java.lang.Iterable.spliterator(Iterable.java:101)
    at software.amazon.awssdk.core.pagination.sync.SdkIterable.stream(SdkIterable.java:34)

Setting AWS_S3_US_EAST_1_REGIONAL_ENDPOINT to regional fixed it.

But I believe this is a bug because the exception is raised even if the region is being explicitly passed to the S3Client builder:

        S3Client  s3Client = S3Client.builder()
                .region(Region.of(region))
                .credentialsProvider(DefaultCredentialsProvider.create())
                .build();

1reaction

dlavelle7commented, Oct 10, 2020

I’m having the exact same issue as described here.

I have got it working using the UrlConnectionHttpClient instead, but would like to use the ApacheHttpClient for its performance.

@debora-ito, @raonitimo would this be the correct way to get the SDK to pick up the AWS_S3_US_EAST_1_REGIONAL_ENDPOINT setting:

AWS_S3_US_EAST_1_REGIONAL_ENDPOINT=regional java -jar my_app.jar

For a Spring app with a bean configuration like:

    @Bean
    public S3Client s3Client() {
        return S3Client.builder()
            .httpClient(ApacheHttpClient.builder().build())
            .build();
    }

This is my stack trace:

SdkClientException: Unable to execute HTTP request: 
Certificate for <my-app.s3.amazonaws.com> 
doesn't match any of the subject alternative names: [*.s3.amazonaws.com, s3.amazonaws.com]

The S3 bucket I’m getting the above error from is in US East (N. Virginia).

I also have made sure that the apache client i’ve installed is the version stated as having the fix:

[INFO] |  +- software.amazon.awssdk:apache-client:jar:2.15.0:compile
[INFO] |  |  +- org.apache.httpcomponents:httpclient:jar:4.5.12:compile

Top Results From Across the Web

javax.net.ssl.SSLPeerUnverifiedException while trying to ...

Caused by: javax.net.ssl.SSLPeerUnverifiedException: Host name 's3.eu-west-1.amazonaws.com' does not match the certificate subject provided ...

Custom web services data actions don't support an SSL ...

I threw a sample json document up on AWS S3 in order to test out custom 'web ... due to: SSLPeerUnverifiedException: Certificate for ......

ARTIFACTORY: AWS S3 connectivity and troubleshooting steps

The user should hold the below actions: "s3:PutObject", "s3:GetObject", ... SSLPeerUnverifiedException: Certificate for <com.

Actions, resources, and condition keys for Amazon S3

Actions Description Access level Resour... AbortMultipartUpload Grants permission to abort a multipart upload Write object* AbortMultipartUpload Grants permission to abort a multipart upload Write CreateAccessPoint Grants...

Issues while uploading document to AWS S3 Bucket from ...

We are trying to upload document from DA 16.4 to s3 file store but we are getting ... SSLPeerUnverifiedException: Peer not authenticated