question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

completing multipart upload

See original GitHub issue

I’m having trouble with completing a multipart upload

given the following test code

mp = s.create_multipart_upload(Bucket='datalake.primary', Key='test1')
uid = mp['UploadId']
p1 =s.upload_part(Bucket='datalake.primary', Key='test1', PartNumber=1, UploadId=uid, Body='part_0')
s.complete_multipart_upload(Bucket='datalake.primary', Key='test1', UploadId=uid, MultipartUpload=???)

I don’t know what I’m supposed to be setting MultipartUpload to and can’t work it out in the docs. I see it needs to be a dict but not sure what it should contain.

Without it, I get the error ClientError: An error occurred (InvalidRequest) when calling the CompleteMultipartUpload operation: You must specify at least one part

Issue Analytics

  • State:closed
  • Created 9 years ago
  • Reactions:4
  • Comments:7 (1 by maintainers)

github_iconTop GitHub Comments

6reactions
danielgtaylorcommented, Jan 30, 2015

@owenrumney this is really not obvious from the documentation, so it took me a few tries to get right. Multipart uploads require information about each part when you try to complete the upload. This is how you can accomplish it:

import boto3

bucket = 'my-bucket'
key = 'mp-test.txt'

s3 = boto3.client('s3')

# Initiate the multipart upload and send the part(s)
mpu = s3.create_multipart_upload(Bucket=bucket, Key=key)
part1 = s3.upload_part(Bucket=bucket, Key=key, PartNumber=1,
                       UploadId=mpu['UploadId'], Body='Hello, world!')

# Next, we need to gather information about each part to complete
# the upload. Needed are the part number and ETag.
part_info = {
    'Parts': [
        {
            'PartNumber': 1,
            'ETag': part['ETag']
        }
    ]
}

# Now the upload works!
s3.complete_multipart_upload(Bucket=bucket, Key=key, UploadId=mpu['UploadId'],
                             MultipartUpload=part_info)

I’ll see what can be done about updating the documentation upstream. Let me know if you have any other questions!

Also, you can enable low-level logging at any time with this:

boto3.set_stream_logger(name='botocore')
1reaction
a523commented, Jul 25, 2018

Is the “ MultipartUpload” REQUIRED?
I can’t find the “REQUIRED” behind the arg “MultipartUpload” from the docs of boto3 ,but i code

rep = s3.complete_multipart_upload(Bucket='bucket',
                                   Key='wentao.mp4',
                                   UploadId='2~in_WUwt5z4g7ri1yfT_MiaRqAs8MRXG')

the raised botocore.exceptions.ClientError: An error occurred (MalformedXML) when calling the CompleteMultipartUpload operation: Unknown

Read more comments on GitHub >

github_iconTop Results From Across the Web

Uploading and copying objects using multipart upload
Multipart upload is a three-step process: You initiate the upload, you upload the object parts, and after you have uploaded all the parts,...
Read more >
complete-multipart-upload — AWS CLI 2.0.34 Command ...
Completes a multipart upload by assembling previously uploaded parts. You first initiate the multipart upload and then upload all parts using the UploadPart ......
Read more >
Complete a multipart upload | Cloud Storage
Completes a multipart upload by concatenating the parts into a single object. Cloud Storage can take several minutes to process a request to...
Read more >
Working with multipart uploads - Hitachi Vantara Knowledge
Completing a multipart upload creates a multipart object from the uploaded parts. Aborting a multipart upload causes the uploaded parts to be ...
Read more >
complete-multipart-upload - Documentation – Commvault
Completes the multipart upload of an object. Usage. complete-multipart-upload --bucket <value> --key <value> --upload-id <value> ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found