URLencoding breakes S3 key
See original GitHub issueWhen uploading a file to S3 like this:
>>> from s3path import S3Path
>>> s=S3Path('/my-bucket/tmp/foo=bar/objname')
>>> s.write_text("foobar")
/path/to/my/venv/lib64/python3.8/site-packages/smart_open/s3.py:220: UserWarning: ignoring the following deprecated transport parameters: ['multipart_upload_kwargs', 'object_kwargs', 'resource_kwargs', 'session']. See <https://github.com/RaRe-Technologies/smart_open/blob/develop/MIGRATING_FROM_OLDER_VERSIONS.rst> for details
warnings.warn(message, UserWarning)
6
The location is transformed and the =
is urlencoded by a as_uri()
call. In S3 I now have a tmp/foo%3Dbar/
prefix. But I expected an tmp/foo=bar/
prefix.
The as_uri()
call in s3path.py leads to unexpected behaviour and breakes the behaviour of s3path <=v0.3
Issue Analytics
- State:
- Created 2 years ago
- Reactions:2
- Comments:7 (4 by maintainers)
Top Results From Across the Web
Amazon s3 URL + being encoded to %2? - Stack Overflow
If I understand your question correctly, then no, there is no way to really change this. The cause appears to be an unfortunate...
Read more >File not found if S3 key contains special characters · Issue #279
Solution: The S3 key need to be URL encoded before download. ... However, it breaks another test cases using wildcard.
Read more >Handling URL-Encoded S3 File Names - Medium
I was naming the file properly, so why was this happening? Turns out that our wrapper around the AWS SDK automatically URL-encodes the...
Read more >Url-Encoding Amazon S3 Resource Keys For Pre-Signed Urls ...
Ben Nadel looks at which characters need to be encoded when generating pre-signed URLs for Amazon S3 objects; and, how to use ColdFusion's ......
Read more >[#HADOOP-3733] "s3:" URLs break when Secret Key contains ...
"s3:" URLs break when Secret Key contains a slash, even if encoded. Status: Assignee: Priority: Resolution: Resolved. Steve Loughran.
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
I think this is OK actually, sorry for the false alarm. We had a workaround in place for the URL encoding issue, and that workaround is subtly broken by #78, but that’s fine because #78 lets us remove the workaround. 🎉
Deployed Version: 0.3.02