Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

SSLError: SSL validation failed ... EOF occurred in violation of protocol

See original GitHub issue

Describe the bug

Python: 3.10.2 boto3/botocore: 1.24.17 OS: Alpine Linux 3.15

When attempting to upload a file to s3 I receive the following exception:

Traceback (most recent call last):
  File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 396, in _make_request
    conn.request_chunked(method, url, **httplib_request_kw)
  File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/connection.py", line 275, in request_chunked
    self.send(to_send)
  File "/opt/app/.venv/lib/python3.10/site-packages/botocore/awsrequest.py", line 218, in send
    return super().send(str)
  File "/usr/local/lib/python3.10/http/client.py", line 998, in send
    self.sock.sendall(data)
  File "/usr/local/lib/python3.10/ssl.py", line 1236, in sendall
    v = self.send(byte_view[count:])
  File "/usr/local/lib/python3.10/ssl.py", line 1205, in send
    return self._sslobj.write(data)
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:2384)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/app/.venv/lib/python3.10/site-packages/botocore/httpsession.py", line 448, in send
    urllib_response = conn.urlopen(
  File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 785, in urlopen
    retries = retries.increment(
  File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/util/retry.py", line 525, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/packages/six.py", line 769, in reraise
    raise value.with_traceback(tb)
  File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 396, in _make_request
    conn.request_chunked(method, url, **httplib_request_kw)
  File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/connection.py", line 275, in request_chunked
    self.send(to_send)
  File "/opt/app/.venv/lib/python3.10/site-packages/botocore/awsrequest.py", line 218, in send
    return super().send(str)
  File "/usr/local/lib/python3.10/http/client.py", line 998, in send
    self.sock.sendall(data)
  File "/usr/local/lib/python3.10/ssl.py", line 1236, in sendall
    v = self.send(byte_view[count:])
  File "/usr/local/lib/python3.10/ssl.py", line 1205, in send
    return self._sslobj.write(data)
urllib3.exceptions.SSLError: EOF occurred in violation of protocol (_ssl.c:2384)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/app/.venv/lib/python3.10/site-packages/dramatiq/worker.py", line 485, in process_message
    res = actor(*message.args, **message.kwargs)
  File "/opt/app/.venv/lib/python3.10/site-packages/dramatiq/actor.py", line 145, in __call__
    return self.fn(*args, **kwargs)
  File "/opt/app/src/bona/tasks/actors/uploads.py", line 54, in move_upload_to_s3
    s3.upload_file(
  File "/opt/app/.venv/lib/python3.10/site-packages/boto3/s3/inject.py", line 143, in upload_file
    return transfer.upload_file(
  File "/opt/app/.venv/lib/python3.10/site-packages/boto3/s3/transfer.py", line 288, in upload_file
    future.result()
  File "/opt/app/.venv/lib/python3.10/site-packages/s3transfer/futures.py", line 103, in result
    return self._coordinator.result()
  File "/opt/app/.venv/lib/python3.10/site-packages/s3transfer/futures.py", line 266, in result
    raise self._exception
  File "/opt/app/.venv/lib/python3.10/site-packages/s3transfer/tasks.py", line 139, in __call__
    return self._execute_main(kwargs)
  File "/opt/app/.venv/lib/python3.10/site-packages/s3transfer/tasks.py", line 162, in _execute_main
    return_value = self._main(**kwargs)
  File "/opt/app/.venv/lib/python3.10/site-packages/s3transfer/upload.py", line 758, in _main
    client.put_object(Bucket=bucket, Key=key, Body=body, **extra_args)
  File "/opt/app/.venv/lib/python3.10/site-packages/botocore/client.py", line 508, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/opt/app/.venv/lib/python3.10/site-packages/botocore/client.py", line 898, in _make_api_call
    http, parsed_response = self._make_request(
  File "/opt/app/.venv/lib/python3.10/site-packages/botocore/client.py", line 921, in _make_request
    return self._endpoint.make_request(operation_model, request_dict)
  File "/opt/app/.venv/lib/python3.10/site-packages/botocore/endpoint.py", line 119, in make_request
    return self._send_request(request_dict, operation_model)
  File "/opt/app/.venv/lib/python3.10/site-packages/botocore/endpoint.py", line 202, in _send_request
    while self._needs_retry(
  File "/opt/app/.venv/lib/python3.10/site-packages/botocore/endpoint.py", line 354, in _needs_retry
    responses = self._event_emitter.emit(
  File "/opt/app/.venv/lib/python3.10/site-packages/botocore/hooks.py", line 412, in emit
    return self._emitter.emit(aliased_event_name, **kwargs)
  File "/opt/app/.venv/lib/python3.10/site-packages/botocore/hooks.py", line 256, in emit
    return self._emit(event_name, kwargs)
  File "/opt/app/.venv/lib/python3.10/site-packages/botocore/hooks.py", line 239, in _emit
    response = handler(**kwargs)
  File "/opt/app/.venv/lib/python3.10/site-packages/botocore/retryhandler.py", line 207, in __call__
    if self._checker(**checker_kwargs):
  File "/opt/app/.venv/lib/python3.10/site-packages/botocore/retryhandler.py", line 284, in __call__
    should_retry = self._should_retry(
  File "/opt/app/.venv/lib/python3.10/site-packages/botocore/retryhandler.py", line 320, in _should_retry
    return self._checker(attempt_number, response, caught_exception)
  File "/opt/app/.venv/lib/python3.10/site-packages/botocore/retryhandler.py", line 363, in __call__
    checker_response = checker(
  File "/opt/app/.venv/lib/python3.10/site-packages/botocore/retryhandler.py", line 247, in __call__
    return self._check_caught_exception(
  File "/opt/app/.venv/lib/python3.10/site-packages/botocore/retryhandler.py", line 416, in _check_caught_exception
    raise caught_exception
  File "/opt/app/.venv/lib/python3.10/site-packages/botocore/endpoint.py", line 281, in _do_get_response
    http_response = self._send(request)
  File "/opt/app/.venv/lib/python3.10/site-packages/botocore/endpoint.py", line 377, in _send
    return self.http_session.send(request)
  File "/opt/app/.venv/lib/python3.10/site-packages/botocore/httpsession.py", line 475, in send
    raise SSLError(endpoint_url=request.url, error=e)
botocore.exceptions.SSLError: SSL validation failed for https://bucket.s3.us-east-2.amazonaws.com/key/parts.ext EOF occurred in violation of protocol (_ssl.c:2384)

Expected Behavior

File should be uploaded to s3

Current Behavior

getting the EOF exception

Reproduction Steps

This error is being generated by the following:

        s3: S3Client = boto3.client('s3')
        s3.upload_file(
            'local/path.ext',
            'bucket-name',
            'new/path.ext',
            ExtraArgs={'ChecksumAlgorithm': 'SHA256'},
        )

I’ve been able to reproduce a similar error against s3 and other ssl sites using the following.

from botocore.httpsession import create_urllib3_context
import urllib3
http = urllib3.PoolManager(ssl_context=create_urllib3_context(), retries=False)
http.urlopen('GET', 'https://www.howsmyssl.com/a/check')

Traceback (most recent call last):
  File "<input>", line 1, in <module>
    http.urlopen('GET', 'https://www.howsmyssl.com/a/check')
  File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/poolmanager.py", line 376, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 785, in urlopen
    retries = retries.increment(
  File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/util/retry.py", line 525, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/packages/six.py", line 769, in reraise
    raise value.with_traceback(tb)
  File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 386, in _make_request
    self._validate_conn(conn)
  File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1040, in _validate_conn
    conn.connect()
  File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/connection.py", line 414, in connect
    self.sock = ssl_wrap_socket(
  File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(
  File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/local/lib/python3.10/ssl.py", line 512, in wrap_socket
    return self.sslsocket_class._create(
  File "/usr/local/lib/python3.10/ssl.py", line 1070, in _create
    self.do_handshake()
  File "/usr/local/lib/python3.10/ssl.py", line 1341, in do_handshake
    self._sslobj.do_handshake()
urllib3.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)

Possible Solution

No response

Additional Information/Context

No response

SDK version used

1.24.17

Environment details (OS name and version, etc.)

Alpine Linux 3.15, Running in EKS

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:28 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
jschwar313commented, Dec 6, 2022

Actually, I figured out a way to search my log files for “SSLEOFError” and this hasn’t happened since 11/17/2022. I guess maybe it is fixed. Sorry about that.

1reaction
jschwar313commented, Dec 2, 2022

now it says this:

aws-cli/2.9.3 Python/3.9.11 Windows/10 exe/AMD64 prompt/off

I’ll see if updating the aws-cli helps at all. I’ll start an upload today.

Read more comments on GitHub >

github_iconTop Results From Across the Web

botocore.exceptions.SSLError: SSL validation failed on ...
On executing this code on my machine, I'm getting this error. botocore.exceptions.SSLError: SSL validation failed for ...
Read more >
EOF occurred in violation of protocol (_ssl.c:1131) - Help
Hi all! I'm running certbot on a recent CentOS 8 system (first installation feb '21). Initially, the certificate was correctly issued.
Read more >
AWS CLI: SSL Validation Failed - [SOLVED] - ShellHacks
Once you run some command in the AWS CLI, for example aws s3 ls , you may get the error as follows: SSL...
Read more >
How to fix error “[SSL: CERTIFICATE_ VERIFY_FAILED ... - Blog
Scenario: I was copying one file from s3 bucket to one of the newly launched servers x.x.x.x via AWS cli with below mentioned...
Read more >
SSL validation failed: EOF occurred in violation of protocol
When I use the following command: aws secretsmanager get-secret-value --secret-id Test I get this error: SSL validation failed for…
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

Add Client Side Encryption to S3 client

Next page

`FileNotFoundError` when using `download_file` method of s3 client object.