If the key of the object contains control characters, delete () of s3.Bucket.objectsCollection fails.
See original GitHub issueIs there a way to delete these objects while avoiding any errors while using batch delete?
reproduce
- upload new object which include
/\x10
in file name. - try batch delete with
s3.Bucket.objectsCollection
objs = bucket.objects.filter(Prefix="path/to/includes_escape")
for obj in objs.all():
print(repr((obj.key)))
-->
'path/to/includes_escape/\x10\x10\x10\x10\x10\x10\x10foo.jpg'
'path/to/includes_escape/\x10\x10\x10\x10\x10\x10\x10bar.jpg'
objs.delete()
--> ClientError: An error occurred (MalformedXML) when calling the DeleteObjects operation: The XML you provided was not well-formed or did not validate against our published schema
environment
- aws lambda python 3.7
- boto3==1.9.42
(edit)
debug trace
boto3.resources.collection [DEBUG] Calling paginated s3:list_objects with {'Bucket': 'sawadev-s3object-filter', 'Prefix': 'path/to/includes_escape'}
boto3.resources.factory [DEBUG] Loading s3:ObjectSummary
boto3.resources.model [DEBUG] Renaming ObjectSummary attribute key
boto3.resources.action [DEBUG] Calling s3:delete_objects with {
'Bucket': 'my-example-bucket',
'Delete': {'Objects': [
{'Key': 'path/to/includes_escape/\x10\x10\x10\x10\x10\x10\x10foo.jpg'},
{'Key': 'path/to/includes_escape/\x10\x10\x10\x10\x10\x10\x10bar.jpg'}
]}}
Issue Analytics
- State:
- Created 4 years ago
- Comments:9 (3 by maintainers)
Top Results From Across the Web
Cannot delete Amazon S3 key that contains bad character
This solved the issue, just delete the main folder: aws s3 rm "s3://BUCKET_NAME/folder/folder" --recursive.
Read more >Collections — Boto3 Docs 1.26.32 documentation - AWS
Overview¶. A collection provides an iterable interface to a group of resources. Collections behave similarly to Django QuerySets and expose a ...
Read more >Read file content from S3 bucket with boto3 - Edureka
I read the filenames in my S3 bucket by doing. objs = boto3.client.list_objects(Bucket='my_bucket') while 'Contents' in objs.keys(): ...
Read more >Listing object keys programmatically - AWS Documentation
Amazon S3 exposes a list operation that lets you enumerate the keys contained in a bucket. Keys are selected for listing by bucket...
Read more >Collect Logs from Your OCI Object Storage Bucket
When you configure a bucket for log collection, Oracle Logging ... only delete the associated configuration with the bucket but will have no ......
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
I am still seeing this issue. Has there been any resolution to this?
This issue has been automatically closed because there has been no response to our request for more information from the original author. With only the information that is currently in the issue, we don’t have enough information to take action. Please reach out if you have or find the answers we need so that we can investigate further.