s3 copy_from fails if "key" has spaces in it.

I’m writing a lambda handler that takes event notifications from a remote S3 bucket and copies a newly uploaded object into “local” bucket.

Here is the important code:

    bucket = event['Records'][0]['s3']['bucket']['name']
    key = event['Records'][0]['s3']['object']['key']

    try:
        s3.Object(bucket + '-clone', key).copy_from(CopySource=bucket + '/' + key)

When I upload a file to execute the code I get the following error in Cloudwatch:

An error occurred (AccessDenied) when calling the CopyObject operation: 
Access Denied: ClientError Traceback (most recent call last): 
File "/var/task/lambda_function.py", line 15, in lambda_handler raise e ClientError: 
An error occurred (AccessDenied) when calling the CopyObject operation: Access Denied

It seems that a space gets translated to a + somewhere along the way. I added the following key = key.replace("+", " ") to my code, and now it copies properly without error.

[jamesls]

Closing due to inactivity.

[seanpaulkelley]

I am also having the same problem. A key with a space in it fails to get retrieved.

for record in event[‘Records’]: bucket = record[‘s3’][‘bucket’][‘name’] key = record[‘s3’][‘object’][‘key’] file_name_pattern = re.match( r’(.*?([^/]+)/?)$', key) file_name = file_name_pattern.group(2) download_path = ‘/tmp/{}{}’.format(uuid.uuid4(), file_name) s3_client.download_file(bucket, key, download_path)

My work around is to set key like key = urllib.unquote_plus(event[‘Records’][0][‘s3’][‘object’][‘key’].encode(“utf8”))

I added the stream loggger and here is my output

2016-12-21 17:48:48,423 botocore.vendored.requests.packages.urllib3.connectionpool [INFO] Starting new HTTPS connection (1): my-photos.s3.amazonaws.com [INFO] 2016-12-21T17:48:48.423Z b97124d2-c7a5-11e6-b626-712e694e52b4 Starting new HTTPS connection (1): my-photos.s3.amazonaws.com 2016-12-21 17:48:48,630 botocore.vendored.requests.packages.urllib3.connectionpool [DEBUG] “HEAD /9d781651-f5b9-445c-b238-721e0e581297Hannah%2BTrot.JPG HTTP/1.1” 404 0 [DEBUG] 2016-12-21T17:48:48.630Z b97124d2-c7a5-11e6-b626-712e694e52b4 “HEAD /9d781651-f5b9-445c-b238-721e0e581297Hannah%2BTrot.JPG HTTP/1.1” 404 0 2016-12-21 17:48:48,630 botocore.parsers [DEBUG] Response headers: {‘x-amz-id-2’: ‘rp4ws0BtuN07AbdxRcN4JOM3BNg/kHFOixWhgPQL0PmUnNN7em3AP1I2n+PUHvl9j3ncpEeyPg4=’, ‘server’: ‘AmazonS3’, ‘transfer-encoding’: ‘chunked’, ‘x-amz-request-id’: ‘24BB44067C63C2EE’, ‘date’: ‘Wed, 21 Dec 2016 17:48:47 GMT’, ‘content-type’: ‘application/xml’} [DEBUG] 2016-12-21T17:48:48.630Z b97124d2-c7a5-11e6-b626-712e694e52b4 Response headers: {‘x-amz-id-2’: ‘rp4ws0BtuN07AbdxRcN4JOM3BNg/kHFOixWhgPQL0PmUnNN7em3AP1I2n+PUHvl9j3ncpEeyPg4=’, ‘server’: ‘AmazonS3’, ‘transfer-encoding’: ‘chunked’, ‘x-amz-request-id’: ‘24BB44067C63C2EE’, ‘date’: ‘Wed, 21 Dec 2016 17:48:47 GMT’, ‘content-type’: ‘application/xml’} [DEBUG] 2016-12-21T17:48:48.630Z b97124d2-c7a5-11e6-b626-712e694e52b4 Response body:

[DEBUG] 2016-12-21T17:48:48.631Z b97124d2-c7a5-11e6-b626-712e694e52b4 Event needs-retry.s3.HeadObject: calling handler <botocore.retryhandler.RetryHandler object at 0x7f09428fa690> [DEBUG] 2016-12-21T17:48:48.631Z b97124d2-c7a5-11e6-b626-712e694e52b4 No retry needed. [DEBUG] 2016-12-21T17:48:48.631Z b97124d2-c7a5-11e6-b626-712e694e52b4 Event needs-retry.s3.HeadObject: calling handler <bound method S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector object at 0x7f09426b4ad0>> 2016-12-21 17:48:48,630 botocore.parsers [DEBUG] Response body:

2016-12-21 17:48:48,631 botocore.hooks [DEBUG] Event needs-retry.s3.HeadObject: calling handler <botocore.retryhandler.RetryHandler object at 0x7f09428fa690> 2016-12-21 17:48:48,631 botocore.retryhandler [DEBUG] No retry needed. 2016-12-21 17:48:48,631 botocore.hooks [DEBUG] Event needs-retry.s3.HeadObject: calling handler <bound method S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector object at 0x7f09426b4ad0>> An error occurred (404) when calling the HeadObject operation: Not Found: ClientError Traceback (most recent call last): File “/var/task/createThumbnail.py”, line 47, in handler s3_client.download_file(bucket, key, download_path) File “/var/runtime/boto3/s3/inject.py”, line 125, in download_file extra_args=ExtraArgs, callback=Callback) File “/var/runtime/boto3/s3/transfer.py”, line 269, in download_file future.result() File “/var/runtime/s3transfer/futures.py”, line 71, in result return self._coordinator.result() File “/var/runtime/s3transfer/futures.py”, line 231, in result raise self._exception ClientError: An error occurred (404) when calling the HeadObject operation: Not Found

END RequestId: b97124d2-c7a5-11e6-b626-712e694e52b4