SSL validation failed (_ssl.c:1051) Python 3.7

META:

Windows 10 python 3.7 pip 18.1 awscli==1.16.110 boto==2.49.0 boto3==1.9.100 botocore==1.12.100 certifi==2018.11.29

Issue Description

Since last week, I’ve been encountering an issue where if I try to use boto3 to upload to my S3 bucket, I get an error:

SSLError: SSL validation failed for https://s3.amazonaws.com/ [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1051)

I’ve tried most of the suggestions I could find but still am having this issue.

Test Case from Fresh Installation

On a windows 10 computer, I tried a fresh install of Anaconda for Python 3.7. Created new credentials in AWS IAM. pip install awscli aws configure Entered in AWS credentials and configuration information. pip install boto3

Attempted to run the following:

import boto3
s3 = boto3.resource('s3')
for bucket in s3.buckets.all():
    print(bucket.name)

Traceback

---

SSLCertVerificationError Traceback (most recent call last) ~\AppData\Local\Continuum\anaconda3\lib\site-packages\urllib3\connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw) 599 body=body, headers=headers, –> 600 chunked=chunked) 601

~\AppData\Local\Continuum\anaconda3\lib\site-packages\urllib3\connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw) 342 try: –> 343 self._validate_conn(conn) 344 except (SocketTimeout, BaseSSLError) as e:

~\AppData\Local\Continuum\anaconda3\lib\site-packages\urllib3\connectionpool.py in _validate_conn(self, conn) 838 if not getattr(conn, ‘sock’, None): # AppEngine might not have .sock –> 839 conn.connect() 840

~\AppData\Local\Continuum\anaconda3\lib\site-packages\urllib3\connection.py in connect(self) 343 server_hostname=server_hostname, –> 344 ssl_context=context) 345

~\AppData\Local\Continuum\anaconda3\lib\site-packages\urllib3\util\ssl_.py in ssl_wrap_socket(sock, keyfile, certfile, cert_reqs, ca_certs, server_hostname, ssl_version, ciphers, ssl_context, ca_cert_dir) 343 if HAS_SNI and server_hostname is not None: –> 344 return context.wrap_socket(sock, server_hostname=server_hostname) 345

~\AppData\Local\Continuum\anaconda3\lib\ssl.py in wrap_socket(self, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname, session) 411 context=self, –> 412 session=session 413 )

~\AppData\Local\Continuum\anaconda3\lib\ssl.py in _create(cls, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname, context, session) 852 raise ValueError(“do_handshake_on_connect should not be specified for non-blocking sockets”) –> 853 self.do_handshake() 854 except (OSError, ValueError):

~\AppData\Local\Continuum\anaconda3\lib\ssl.py in do_handshake(self, block) 1116 self.settimeout(None) -> 1117 self._sslobj.do_handshake() 1118 finally:

SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1051)

During handling of the above exception, another exception occurred:

SSLError Traceback (most recent call last) ~\AppData\Local\Continuum\anaconda3\lib\site-packages\botocore\httpsession.py in send(self, request) 257 preload_content=False, –> 258 decode_content=False, 259 )

~\AppData\Local\Continuum\anaconda3\lib\site-packages\urllib3\connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw) 637 retries = retries.increment(method, url, error=e, _pool=self, –> 638 _stacktrace=sys.exc_info()[2]) 639 retries.sleep()

~\AppData\Local\Continuum\anaconda3\lib\site-packages\urllib3\util\retry.py in increment(self, method, url, response, error, _pool, _stacktrace) 342 # Disabled, indicate to re-raise the error. –> 343 raise six.reraise(type(error), error, _stacktrace) 344

~\AppData\Local\Continuum\anaconda3\lib\site-packages\urllib3\packages\six.py in reraise(tp, value, tb) 684 if value.traceback is not tb: –> 685 raise value.with_traceback(tb) 686 raise value

~\AppData\Local\Continuum\anaconda3\lib\site-packages\urllib3\connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw) 599 body=body, headers=headers, –> 600 chunked=chunked) 601

~\AppData\Local\Continuum\anaconda3\lib\site-packages\urllib3\connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw) 342 try: –> 343 self._validate_conn(conn) 344 except (SocketTimeout, BaseSSLError) as e:

~\AppData\Local\Continuum\anaconda3\lib\site-packages\urllib3\connectionpool.py in _validate_conn(self, conn) 838 if not getattr(conn, ‘sock’, None): # AppEngine might not have .sock –> 839 conn.connect() 840

~\AppData\Local\Continuum\anaconda3\lib\site-packages\urllib3\connection.py in connect(self) 343 server_hostname=server_hostname, –> 344 ssl_context=context) 345

~\AppData\Local\Continuum\anaconda3\lib\site-packages\urllib3\util\ssl_.py in ssl_wrap_socket(sock, keyfile, certfile, cert_reqs, ca_certs, server_hostname, ssl_version, ciphers, ssl_context, ca_cert_dir) 343 if HAS_SNI and server_hostname is not None: –> 344 return context.wrap_socket(sock, server_hostname=server_hostname) 345

~\AppData\Local\Continuum\anaconda3\lib\ssl.py in wrap_socket(self, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname, session) 411 context=self, –> 412 session=session 413 )

~\AppData\Local\Continuum\anaconda3\lib\ssl.py in _create(cls, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname, context, session) 852 raise ValueError(“do_handshake_on_connect should not be specified for non-blocking sockets”) –> 853 self.do_handshake() 854 except (OSError, ValueError):

~\AppData\Local\Continuum\anaconda3\lib\ssl.py in do_handshake(self, block) 1116 self.settimeout(None) -> 1117 self._sslobj.do_handshake() 1118 finally:

SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1051)

During handling of the above exception, another exception occurred:

SSLError Traceback (most recent call last) <ipython-input-11-691d6da591c5> in <module> 4 s3 = boto3.resource(‘s3’) 5 # Print out bucket names ----> 6 for bucket in s3.buckets.all(): 7 print(bucket.name)

~\AppData\Local\Continuum\anaconda3\lib\site-packages\boto3\resources\collection.py in iter(self) 81 82 count = 0 —> 83 for page in self.pages(): 84 for item in page: 85 yield item

~\AppData\Local\Continuum\anaconda3\lib\site-packages\boto3\resources\collection.py in pages(self) 159 self._parent.meta.service_name, 160 self._py_operation_name, params) –> 161 pages = [getattr(client, self._py_operation_name)(**params)] 162 163 # Now that we have a page iterator or single page of results

~\AppData\Local\Continuum\anaconda3\lib\site-packages\botocore\client.py in _api_call(self, *args, **kwargs) 355 “%s() only accepts keyword arguments.” % py_operation_name) 356 # The “self” in this scope is referring to the BaseClient. –> 357 return self._make_api_call(operation_name, kwargs) 358 359 _api_call.name = str(py_operation_name)

~\AppData\Local\Continuum\anaconda3\lib\site-packages\botocore\client.py in _make_api_call(self, operation_name, api_params) 646 else: 647 http, parsed_response = self._make_request( –> 648 operation_model, request_dict, request_context) 649 650 self.meta.events.emit(

~\AppData\Local\Continuum\anaconda3\lib\site-packages\botocore\client.py in _make_request(self, operation_model, request_dict, request_context) 665 def _make_request(self, operation_model, request_dict, request_context): 666 try: –> 667 return self._endpoint.make_request(operation_model, request_dict) 668 except Exception as e: 669 self.meta.events.emit(

~\AppData\Local\Continuum\anaconda3\lib\site-packages\botocore\endpoint.py in make_request(self, operation_model, request_dict) 100 logger.debug(“Making request for %s with params: %s”, 101 operation_model, request_dict) –> 102 return self._send_request(request_dict, operation_model) 103 104 def create_request(self, params, operation_model=None):

~\AppData\Local\Continuum\anaconda3\lib\site-packages\botocore\endpoint.py in _send_request(self, request_dict, operation_model) 135 request, operation_model, context) 136 while self._needs_retry(attempts, operation_model, request_dict, –> 137 success_response, exception): 138 attempts += 1 139 # If there is a stream associated with the request, we need

~\AppData\Local\Continuum\anaconda3\lib\site-packages\botocore\endpoint.py in _needs_retry(self, attempts, operation_model, request_dict, response, caught_exception) 229 event_name, response=response, endpoint=self, 230 operation=operation_model, attempts=attempts, –> 231 caught_exception=caught_exception, request_dict=request_dict) 232 handler_response = first_non_none_response(responses) 233 if handler_response is None:

~\AppData\Local\Continuum\anaconda3\lib\site-packages\botocore\hooks.py in emit(self, event_name, **kwargs) 354 def emit(self, event_name, **kwargs): 355 aliased_event_name = self._alias_event_name(event_name) –> 356 return self._emitter.emit(aliased_event_name, **kwargs) 357 358 def emit_until_response(self, event_name, **kwargs):

~\AppData\Local\Continuum\anaconda3\lib\site-packages\botocore\hooks.py in emit(self, event_name, **kwargs) 226 handlers. 227 “”" –> 228 return self._emit(event_name, kwargs) 229 230 def emit_until_response(self, event_name, **kwargs):

~\AppData\Local\Continuum\anaconda3\lib\site-packages\botocore\hooks.py in _emit(self, event_name, kwargs, stop_on_response) 209 for handler in handlers_to_call: 210 logger.debug(‘Event %s: calling handler %s’, event_name, handler) –> 211 response = handler(**kwargs) 212 responses.append((handler, response)) 213 if stop_on_response and response is not None:

~\AppData\Local\Continuum\anaconda3\lib\site-packages\botocore\retryhandler.py in call(self, attempts, response, caught_exception, **kwargs) 181 182 “”" –> 183 if self._checker(attempts, response, caught_exception): 184 result = self._action(attempts=attempts) 185 logger.debug(“Retry needed, action of: %s”, result)

~\AppData\Local\Continuum\anaconda3\lib\site-packages\botocore\retryhandler.py in call(self, attempt_number, response, caught_exception) 249 def call(self, attempt_number, response, caught_exception): 250 should_retry = self._should_retry(attempt_number, response, –> 251 caught_exception) 252 if should_retry: 253 if attempt_number >= self._max_attempts:

~\AppData\Local\Continuum\anaconda3\lib\site-packages\botocore\retryhandler.py in _should_retry(self, attempt_number, response, caught_exception) 275 # If we’ve exceeded the max attempts we just let the exception 276 # propogate if one has occurred. –> 277 return self._checker(attempt_number, response, caught_exception) 278 279

~\AppData\Local\Continuum\anaconda3\lib\site-packages\botocore\retryhandler.py in call(self, attempt_number, response, caught_exception) 315 for checker in self._checkers: 316 checker_response = checker(attempt_number, response, –> 317 caught_exception) 318 if checker_response: 319 return checker_response

~\AppData\Local\Continuum\anaconda3\lib\site-packages\botocore\retryhandler.py in call(self, attempt_number, response, caught_exception) 221 elif caught_exception is not None: 222 return self._check_caught_exception( –> 223 attempt_number, caught_exception) 224 else: 225 raise ValueError(“Both response and caught_exception are None.”)

~\AppData\Local\Continuum\anaconda3\lib\site-packages\botocore\retryhandler.py in _check_caught_exception(self, attempt_number, caught_exception) 357 # the MaxAttemptsDecorator is not interested in retrying the exception 358 # then this exception just propogates out past the retry code. –> 359 raise caught_exception

~\AppData\Local\Continuum\anaconda3\lib\site-packages\botocore\endpoint.py in _do_get_response(self, request, operation_model) 198 http_response = first_non_none_response(responses) 199 if http_response is None: –> 200 http_response = self._send(request) 201 except HTTPClientError as e: 202 return (None, e)

~\AppData\Local\Continuum\anaconda3\lib\site-packages\botocore\endpoint.py in _send(self, request) 242 243 def _send(self, request): –> 244 return self.http_session.send(request) 245 246

~\AppData\Local\Continuum\anaconda3\lib\site-packages\botocore\httpsession.py in send(self, request) 274 return http_response 275 except URLLib3SSLError as e: –> 276 raise SSLError(endpoint_url=request.url, error=e) 277 except (NewConnectionError, socket.gaierror) as e: 278 raise EndpointConnectionError(endpoint_url=request.url, error=e)

SSLError: SSL validation failed for https://s3.amazonaws.com/ [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1051)

[shubhamsawantsjsu]

I have got the solution for this.

Don’t install boto3 using pip/pip3.

Use following steps::

$ git clone https://github.com/boto/boto3.git $ cd boto3 $ virtualenv venv … $ . venv/bin/activate $ pip install -r requirements.txt $ pip install -e .

It worked for me. Initially I was facing the same SSL Validation Exception problem. Then I uninstalled the boto3 (pip3 uninstall boto3) and re-installed it using virtualenv as stated above. Now, it is working fine.

[inclementweather]

I ran into this issue building a mlflow docker container.

Pip brings in the certifi module as a dependency for requests and that has its own cert bundle at /usr/local/lib/python3.7/site-packages/certifi/cacert.pem which does not match the system certificate bundle.

Setting the REQUESTS_CA_BUNDLE environment variable to the system bundle works for requests the but not boto3. Setting AWS_CA_BUNDLE doesn’t seem to have any affect at all.