question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

boto3.s3.transfer set Metadata incorrectly

See original GitHub issue

http://boto3.readthedocs.io/en/latest/_modules/boto3/s3/transfer.html says:

Setting metadata by documentation: transfer.upload_file(‘/tmp/myfile’, ‘bucket’, ‘key’, extra_args={‘Metadata’: {‘a’: ‘b’, ‘c’: ‘d’}})

If I upload a file, e.g. index.html with the following command:

from boto3.s3.transfer import S3Transfer
awsclient = boto3.client('s3', 'eu-west-1')
transfer = S3Transfer(awsclient)

transfer.upload_file("index.html",OUTBUCKET,"index.html",extra_args={'Metadata': {'Content-Type': 'text/html'}})

It sets up the S3 object metadata to: x-amz-meta-content-type: text/html

Instead of the expected: Content-Type: text/html

The main problem e.g.: uploading the index.html file with boto3 to an S3 bucket with static website hosting option, results the WEB browsers downloads it instead opens.

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Comments:5 (1 by maintainers)

github_iconTop GitHub Comments

2reactions
grahamlyonscommented, Jun 6, 2017

@stealthycoin is right - calling the API with an invalid key shows that Metadata is a different key from ContentType in the API:

ValueError: Invalid extra_args key 'metadata', must be one of: ACL, CacheControl, 
ContentDisposition, ContentEncoding, ContentLanguage, ContentType, Expires, 
GrantFullControl, GrantRead, GrantReadACP, GrantWriteACP, Metadata, RequestPayer, 
ServerSideEncryption, StorageClass, SSECustomerAlgorithm, SSECustomerKey, 
SSECustomerKeyMD5, SSEKMSKeyId, WebsiteRedirectLocation
1reaction
mikeatlascommented, Jan 13, 2020

Agreed, it is confusing. The AWS Web Console treats Content-Type as a “Metadata” property to set on the object, while the API gives Content-Type a primary parameter as well as a Metadata parameter, which, if you use the latter to try to set Content-Type, you get x-amz-meta-content-type key added.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Developers - boto3.s3.transfer set Metadata incorrectly -
The main problem e.g.: uploading the index.html file with boto3 to an S3 bucket with static website hosting option, results the WEB browsers...
Read more >
boto.s3: copy() on a key object loses 'Content-Type' metadata
The problem is when I do the example below, I actually lose my Content-Type, which defaults back to 'application/octet-stream' (not very useful if...
Read more >
S3 — Boto3 Docs 1.26.34 documentation - AWS
When copying an object, you can preserve all metadata (default) or specify new metadata. However, the ACL is not preserved and is set...
Read more >
Editing object metadata in the Amazon S3 console
You can also set some metadata when you upload the object and later edit it as your needs change. For example, you might...
Read more >
Working with Amazon S3 buckets with Boto3
When working with Python, one can easily interact with S3 with the Boto3 package. In this post, I will put together a cheat...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found