how to restore Glacier to S3 given folder data

I create bucket policy for after 1 day data move s3 to Glacier. in my bucket has more than 100K objects. How can i restore Glacier object to S3? and what is most efficient way to download all object in given folder(directly glacier or move glacier data to s3 then download).

In the S3 Restoring Objects document only give Java and .NET example (http://docs.aws.amazon.com/AmazonS3/latest/dev/restoring-objects.html). Is there is method in boto3 please explain how to restore S3 object.

        buckt_ob = self.s3.Bucket('mybucket')
        for obj in buckt_ob.objects.filter(Prefix = 'folder'):
            storage_class = obj.storage_class
            restore = obj.restore

This code what i try but there is no such obj.restore

after restore complete how can i get notification like all folder is available for download

[kyleknap]

Yeah so this is a little tricky. So for the objects collection on the bucket resource, it will return ObjectSummary resources which do not have the restore property. There is a difference between ObjectSummary and Object resources because ObjectSummaries are loaded from different s3 operations. ObjectSummary is loaded from a ListObjects which has less information than the HeadObject operation which is used to load the Object resource. To get the restore attribute value you would need to do something like this as the HeadObject operation is the only api method that can get this information:

import boto3

s3 = boto3.resource('s3')
bucket = s3.Bucket('glacier_test')
for obj_sum in bucket.objects.all():
    obj = s3.Object(obj_sum.bucket_name, obj_sum.key)
    storage_class = obj.storage_class
    restore = obj.restore

Furthermore, the restore attribute just provides information on the status of a glacier restoration. It does not actually restore the object. To initiate the restoration process you need to access the client’s restore_object method. So something like this should work:

import boto3

s3 = boto3.resource('s3')
bucket = s3.Bucket('glacier_test')
for obj_sum in bucket.objects.all():
    resp = bucket.meta.client.restore_object(
        Bucket=obj_sum.bucket_name,
        Key=obj_sum.key,
        RestoreRequest={'Days': 1}
    )

Once you actual send the restore request, you will notice that the restore attribute will no longer be None:

import boto3

s3 = boto3.resource('s3')
bucket = s3.Bucket('glacier_test')
for obj_sum in bucket.objects.all():
    obj = s3.Object(obj_sum.bucket_name, obj_sum.key)
    storage_class = obj.storage_class
    restore = obj.restore
    print(obj.key, obj.storage_class, obj.restore)

Output:
test.txt GLACIER ongoing-request="true"
testfile GLACIER ongoing-request="true"

To know when the object is ready to download the value will be ongoing-request="true". So you will need to reload() the object to see if it is ready to download.

As a note to myself, based on my walkthrough, it seems that it would be good if we added the following to boto3:

1. Documentation on how to restore glacier objects
2. Waiters to pull for when a glacier object is ready to be downloaded
3. Possibly add the restore() method to the Object resource so we do not have to drop down to the client to restore the object.

@dduleep Let me know what you think of these suggestions.

[inv-senchuthomas]

Hai kyleknap,

I am creating a PHP based web application using Amazon’s S3 and glacier services.Now I want to give my site users a feature that they can choose any file and make it archive (means move file from S3 to Glacier) and unarchive (means move file from Glacier to S3). Can you provide an exple demonstrating this actions or any api’s for doing this Thanks in advance…