botocore.exceptions.SSLError: SSL validation failed on aws3 endpoint

Describe the bug i’m trying to download from s3 public bucket and got ssl error, while 2 weeks ago everything works and i didnt change anything on my workspace. i’m not sure why?

Steps to reproduce

    import boto3
    import botocore
    import urllib3

    from botocore.handlers import disable_signing

    s3_resource = boto3.resource(
                "s3",
                endpoint_url="https://s3.amazonaws.com",
                verify=False,
                region_name=None,
                aws_access_key_id=None,
                aws_secret_access_key=None,
            )
    s3_resource.meta.client.meta.events.register('choose-signer.s3.*', disable_signing)
    
    s3_resource.Bucket("mybucket").download_file(
                    "logo_detection_data/images/1.jpg", "/logo_detection_data/images/1.jpg", ExtraArgs=None)

Expected behavior download from s3 amazon public bucket.

Debug logs botocore.exceptions.SSLError: SSL validation failed for https://s3.amazonaws.com/libhub-readme/logo_detection_data/images/1.jpg [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)

[dmitrytretyakov]

In case of using invalid certificates and trying to skip validation:

import boto3

client = boto3.client('s3', 
    aws_access_key_id='ID',
    aws_secret_access_key='reallysecret',
    endpoint_url='https://myaws',
    verify=False
)
client.list_buckets()

1. Python 3.6.9 boto3 (1.20.1) botocore (1.22.12) IS OK

2. Python 3.6.9 boto3 (1.20.1) botocore (1.23.1) IS NOT OK:

Traceback. Click to expand

Traceback (most recent call last): File "/opt/k8s-module-venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 677, in urlopen chunked=chunked, File "/opt/k8s-module-venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 381, in _make_request self._validate_conn(conn) File "/opt/k8s-module-venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 978, in _validate_conn conn.connect() File "/opt/k8s-module-venv/lib/python3.6/site-packages/urllib3/connection.py", line 371, in connect ssl_context=context, File "/opt/k8s-module-venv/lib/python3.6/site-packages/urllib3/util/ssl_.py", line 386, in ssl_wrap_socket return context.wrap_socket(sock, server_hostname=server_hostname) File "/usr/lib/python3.6/ssl.py", line 407, in wrap_socket _context=self, _session=session) File "/usr/lib/python3.6/ssl.py", line 817, in __init__ self.do_handshake() File "/usr/lib/python3.6/ssl.py", line 1077, in do_handshake self._sslobj.do_handshake() File "/usr/lib/python3.6/ssl.py", line 689, in do_handshake self._sslobj.do_handshake() ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File “/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/httpsession.py”, line 394, in send chunked=self._chunked(request.headers), File “/opt/k8s-module-venv/lib/python3.6/site-packages/urllib3/connectionpool.py”, line 727, in urlopen method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] File “/opt/k8s-module-venv/lib/python3.6/site-packages/urllib3/util/retry.py”, line 386, in increment raise six.reraise(type(error), error, _stacktrace) File “/opt/k8s-module-venv/lib/python3.6/site-packages/urllib3/packages/six.py”, line 734, in reraise raise value.with_traceback(tb) File “/opt/k8s-module-venv/lib/python3.6/site-packages/urllib3/connectionpool.py”, line 677, in urlopen chunked=chunked, File “/opt/k8s-module-venv/lib/python3.6/site-packages/urllib3/connectionpool.py”, line 381, in _make_request self._validate_conn(conn) File “/opt/k8s-module-venv/lib/python3.6/site-packages/urllib3/connectionpool.py”, line 978, in validate_conn conn.connect() File “/opt/k8s-module-venv/lib/python3.6/site-packages/urllib3/connection.py”, line 371, in connect ssl_context=context, File "/opt/k8s-module-venv/lib/python3.6/site-packages/urllib3/util/ssl.py", line 386, in ssl_wrap_socket return context.wrap_socket(sock, server_hostname=server_hostname) File “/usr/lib/python3.6/ssl.py”, line 407, in wrap_socket _context=self, _session=session) File “/usr/lib/python3.6/ssl.py”, line 817, in init self.do_handshake() File “/usr/lib/python3.6/ssl.py”, line 1077, in do_handshake self._sslobj.do_handshake() File “/usr/lib/python3.6/ssl.py”, line 689, in do_handshake self._sslobj.do_handshake() urllib3.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File “boto3-bug.py”, line 8, in <module> print(client.list_buckets()) File “/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/client.py”, line 391, in _api_call return self._make_api_call(operation_name, kwargs) File “/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/client.py”, line 706, in _make_api_call operation_model, request_dict, request_context) File “/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/client.py”, line 725, in _make_request return self._endpoint.make_request(operation_model, request_dict) File “/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/endpoint.py”, line 102, in make_request return self._send_request(request_dict, operation_model) File “/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/endpoint.py”, line 137, in _send_request success_response, exception): File “/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/endpoint.py”, line 255, in _needs_retry caught_exception=caught_exception, request_dict=request_dict) File “/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/hooks.py”, line 357, in emit return self._emitter.emit(aliased_event_name, **kwargs) File “/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/hooks.py”, line 228, in emit return self._emit(event_name, kwargs) File “/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/hooks.py”, line 211, in _emit response = handler(**kwargs) File “/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/retryhandler.py”, line 183, in call if self._checker(attempts, response, caught_exception): File “/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/retryhandler.py”, line 251, in call caught_exception) File “/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/retryhandler.py”, line 277, in _should_retry return self._checker(attempt_number, response, caught_exception) File “/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/retryhandler.py”, line 317, in call caught_exception) File “/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/retryhandler.py”, line 223, in call attempt_number, caught_exception) File “/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/retryhandler.py”, line 359, in _check_caught_exception raise caught_exception File “/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/endpoint.py”, line 199, in _do_get_response http_response = self._send(request) File “/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/endpoint.py”, line 268, in _send return self.http_session.send(request) File “/opt/k8s-module-venv/lib/python3.6/site-packages/botocore/httpsession.py”, line 412, in send raise SSLError(endpoint_url=request.url, error=e) botocore.exceptions.SSLError: SSL validation failed for https://myaws/ [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)

[tim-finnigan]

Checking in - support for Python 3.6 has since been deprecated as of May 2022 per this announcement. If this is still an issue in Python 3.7+ please let us know.