[Question] Is it possible to disable https certificate validation?

Problem description

I use minio (https://min.io) to self-host an s3 compatible object storage. The service uses a self signed certificate. Accessing objects using smart open gives

SSLError: SSL validation failed for https://blahblah [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1076)

I am wondering whether I could disable the certificate validation.

• What are you trying to achieve? Accessing self-hosted s3-compatible object storage with a self-signed certificate.
• What is the expected result? Accessing objects without problems.
• What are you seeing instead? Got SSL problem

Steps/code to reproduce the problem

1. start a local minio instance with a self-signed certificate according to the documents. https://docs.min.io/docs/deploy-minio-on-docker-compose.html
2. try accessing objects stored on minio using smart-open

Versions

Please provide the output of:

import platform, sys, smart_open
print(platform.platform())
print("Python", sys.version)
print("smart_open", smart_open.__version__)

Darwin-19.0.0-x86_64-i386-64bit Python 3.7.5 (default, Oct 25 2019, 10:52:18) [Clang 4.0.1 (tags/RELEASE_401/final)] smart_open 1.9.0

Checklist

Before you create the issue, please make sure you have:

• Described the problem clearly
• Provided a minimal reproducible example, including any required data
• Provided the version numbers of the relevant software

[fakhavan]

I have the same issue using this smart_open to access S3 on minio. As you know, minio can simulate S3 storage which I use to test code before deploying into production. There is a guide on how boto3 could be used to access minio. You can take a look at here.

I have no issue accessing it using boto3 however when I try to use smart_open to do this, I receive botocore.exceptions.SSLError. I use s3://minioadmin:minioadmin@localhost:9000@data-test-bucket as URL and receive the following error:

botocore.exceptions.SSLError: SSL validation failed for https://localhost:9000/phq-dataeng/individual_event/1593661092355.tsv?uploads [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:852)

It would be great to have an option not to enforce SSL in the smart_open

[fakhavan]

@mpenkov Thank you for your quick reply. I was able to make it work. For future reference, this is how I was able to use smart_open with S3 on Minio:

transport_params = {
    'resource_kwargs': {
        'endpoint_url': 'http://localhost:9000',
        'aws_access_key_id':'minioadmin',
        'aws_secret_access_key':'minioadmin',
    }
}

with open('s3://data-test-bucket/test.txt', 'wb', transport_params=transport_params) as fout:
    fout.write(b'Yay I work on S3 on minio!')

I’d suggest adding it to the examples as well as it’s not clear regarding this kind of S3-like solutions.