Writing to S3 with server-side encryption doesn't work anymore
See original GitHub issueProblem description
Earlier versions of smart_open (I’ve tested v1.11.1 and v3.0.0) allowed writing to S3 with server-side encryption like this:
output_handle = smart_open.open(out_url, "w", transport_params={ 'multipart_upload_kwargs': { 'ServerSideEncryption': 'AES256' }})
In the latest versions this doesn’t work and I cannot find any documentation or figure out a different way of making it work. The result is I get an error like ValueError: the bucket 'myBucket' does not exist, or is forbidden for access (ClientError('An error occurred (AccessDenied) when calling the CreateMultipartUpload operation: Access Denied'))
Steps/code to reproduce the problem
- install the latest version of smart_open
- create an output handle as shown above
Versions
Please provide the output of:
import platform, sys, smart_open
print(platform.platform())
print("Python", sys.version)
print("smart_open", smart_open.__version__)
Linux-4.4.0-17763-Microsoft-x86_64-with-glibc2.10 Python 3.8.2 (default, Mar 26 2020, 15:53:00) [GCC 7.3.0] smart_open 5.0.0
Checklist
Before you create the issue, please make sure you have:
- Described the problem clearly
- Provided a minimal reproducible example, including any required data
- Provided the version numbers of the relevant software
Issue Analytics
- State:
- Created 2 years ago
- Comments:9
Top Results From Across the Web
Troubleshoot data delivery failure between Kinesis and S3
Kinesis Data Firehose supports Amazon S3 server-side encryption with AWS Key Management Service (AWS KMS) for encrypting data that's ...
Read more >Issues while reading and writing a KMS encrypted spark data ...
The bucket has server-side encryption setup. I'm having the following packages run from spark-default.conf: spark.jars.packages com.amazonaws: ...
Read more >S3 KMS encryption support · Issue #348 · shrinerb/shrine
We are using the KMS encryption support to upload/download files from S3. I managed to make it work by overwriting the S3 client...
Read more >Python, Boto3, and AWS S3: Demystified - Real Python
Get started working with Python, Boto3, and AWS S3. Learn how to create objects, ... Create a new file and upload it using...
Read more >All you need to know about encrypting AWS S3 buckets
Dance like nobody's watching, encrypt like everyone is. Learn how to enable S3 default encryption. But that's not enough.
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Try this:
Sadly no, my old WSL environment got deleted with the last Windows 10 upgrade (thanks Microsoft!)