question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Support for object level Tagging in boto3 upload_file method

See original GitHub issue

I want to add tags to the files as I upload them to S3. Boto3 supports specifying tags with put_object method, however considering expected file size, I am using upload_file function which handles multipart uploads. But this function rejects ‘Tagging’ as keyword argument.

import boto3
client = boto3.client('s3', region_name='us-west-2')
client.upload_file('test.mp4', 'bucket_name', 'test.mp4',
                   ExtraArgs={'Tagging': 'type=test'})

ValueError: Invalid extra_args key 'Tagging', must be one of: ACL, CacheControl, ContentDisposition, ContentEncoding, ContentLanguage, ContentType, Expires, GrantFullControl, GrantRead, GrantReadACP, GrantWriteACP, Metadata, RequestPayer, ServerSideEncryption, StorageClass, SSECustomerAlgorithm, SSECustomerKey, SSECustomerKeyMD5, SSEKMSKeyId, WebsiteRedirectLocation

I found a way to make this work by using S3 transfer manager directly and modifying allowed keyword list.

from s3transfer import S3Transfer
import boto3

client = boto3.client('s3', region_name='us-west-2')
transfer = S3Transfer(client)
transfer.ALLOWED_UPLOAD_ARGS.append('Tagging')
transfer.upload_file('test.mp4', 'bucket_name', 'test.mp4',
                     extra_args={'Tagging': 'type=test'})

Even though this works, I don’t think this is the best way. It might create other side effects. Currently I am not able to find correct way to achieve this. Any advice would be great. Thanks.

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:11 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
drake-adlcommented, Aug 16, 2021

We utilize def convert_dict_to_string(tagging): return "&".join([k + "=" + v for k, v in tagging.items()]). The input param is a dictionary.

You can use: s3.put_object_tagging or s3.put_object with a Tagging arg.

1reaction
smithericsmithcommented, Dec 13, 2019

This would be very helpful for me as well. I avoid using the upload_file() call because it does not support Tagging, so I am forced to read the contents into memory and use put_object() simply because I want to have the files Tagged when created.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Support for object level Tagging in boto3 upload_file method
I want to add tags to the files as I upload them to S3. Boto3 supports specifying tags with put_object method, however considering...
Read more >
S3 — Boto3 Docs 1.26.34 documentation - AWS
A low-level client representing Amazon Simple Storage Service (S3) ... Objects with different object data will have different entity tags. The entity tag...
Read more >
Categorizing your storage using tags - AWS Documentation
Use object tagging to categorize storage. Each tag is a key-value pair. You can add tags to new objects when you upload them,...
Read more >
How to use Boto3 to upload files to an S3 Bucket? - Learn AWS
For allowed upload arguments see boto3.s3.transfer.S3Transfer.ALLOWED_UPLOAD_ARGS. Callback (function) -- A method which takes a number of bytes ...
Read more >
4 Easy Ways to Upload a File to S3 Using Python - Binary Guy
In such cases, boto3 uses the default AWS CLI profile set up on ... You can create different bucket objects and use them...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found