Upload or put object in S3 failing silently
See original GitHub issueI’ve been trying to upload files from a local folder into folders on S3 using Boto3, and it’s failing kinda silently, with no indication of why the upload isn’t happening.
key_name = folder + '/'
s3_connect = boto3.client('s3', s3_bucket_region,)
# upload File to S3
for filename in os.listdir(folder):
s3_name = key_name + filename
print folder, filename, key_name, s3_name
upload = s3_connect.upload_file(
s3_name, s3_bucket, key_name,
)
Printing upload just says “None”, with no other information. No upload happens. I’ve also tried using put_object:
put_obj = s3_connect.put_object(
Bucket=s3_bucket,
Key=key_name,
Body=s3_name,
)
and I get an HTTPS response code of 200 - but no files upload.
First, I’d love to solve this problem, but second, it seems this isn’t the right behavior - if an upload doesn’t happen, there should be a bit more information about why (although I imagine this might be a limitation of the API?)
Issue Analytics
- State:
- Created 6 years ago
- Reactions:12
- Comments:32 (2 by maintainers)
Top Results From Across the Web
Silent Failure of S3 PutObject? - Stack Overflow
I have been trying to programmatically upload SNS messages to an S3 bucket using the S3.Object.put() method like so:
Read more >Resolve issues with uploading large files in Amazon S3
I'm trying to upload a large file using the Amazon S3 console. ... However, the upload persistently fails and I'm getting timeout errors....
Read more >S3 disk put() fails silently. Any ideas why? - Laracasts
I did try to use \Aws\S3\S3Client directly to upload a local file and that worked just fine: Copy Code $s3Client = new \Aws\S3\S3Client([...
Read more >S3 — Boto3 Docs 1.26.29 documentation - Amazon AWS
For more information about multipart uploads, see Uploading Objects Using ... and PUT requests for an object protected by Amazon Web Services KMS...
Read more >Direct to S3 File Uploads in Node.js - Heroku Dev Center
Upload files direct to S3 using Node.js on Heroku and avoid tying up ... To do so, edit package.json and add a "dependencies"...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Having a leading slash in your path to the object will create a folder with a blank name in your bucket root, which holds the uploads. e.g.:
will create / /uploads in your bucket root (note the blank folder) and put the object in there rather than in /uploads. Since boto was also spamming the bucket root with odd files, I didn’t notice the empty folder until much later and lost a lot of time. Could be your problem, too?
So just for your information (and perhaps this will help someone figure out what’s up,) using the Service Resource instead of the client works fine. The upload into S3 “folders” works perfectly. Here’s some example code:
So at least there is a workaround for the problem with the client.