Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Files are deleted from the bucket regardless of the expiration rule defined on the lifecycle configuration

See original GitHub issue

Environment info

NooBaa Version: 5.7.0
Platform: OpenShift 4

Actual behavior

I’m trying to set a lifecycle configuration on my bucket so objects older than 4 days will be expired (deleted). Thing is that, for some reason, ~30 minutes after setting that cleanup rule all the files in the bucket are deleted, regardless of their creation date.

Expected behavior

Only files that are older than 4 days should be deleted

Steps to reproduce

Existing S3 bucket with existing files. some of the files were uploaded to the bucket more that 4 days ago, some of the files were uploaded to the bucket less than 4 days ago.
Call PutBucketLifecycleConfiguration REST API to set a cleanup rule that expires the files after 4 days.
Actual - ~30 minutes after that REST call, ALL the files are deleted from the bucket.

More information - Screenshots / Logs / Other output

The payload used to setup the lifecycle configuration:

<LifecycleConfiguration> <Rule> <ID>ExpirationRule</ID> <Prefix></Prefix> <Status>Enabled</Status> <Expiration> <Days>4</Days> </Expiration> </Rule> </LifecycleConfiguration>

Issue Analytics

State:
Created 2 years ago
Comments:6 (5 by maintainers)

Top GitHub Comments

1reaction

guymguymcommented, Dec 2, 2021

We can think of a test case to cover expiration - maybe a “gray box” approach will be most effective, such as using the S3 putLifecyclePolicy API and after that check the database applied the expected configuration.

1reaction

liranmaudacommented, Dec 2, 2021

Not sure why but it looks like it is not setting days, but it sets minutes https://github.com/noobaa/noobaa-core/blob/f0850fe66417ac7c113ae7222d8f1f2b820b35bc/src/server/bg_services/lifecycle.js#L55

I guess changing that to days will work properly. @eyalma01 if you like you can change and test, I will refactor the whole file on Sunday.

Top Results From Across the Web

Lifecycle configuration elements - AWS Documentation

Amazon S3 makes all expired objects unavailable. Whether the objects are permanently removed depends on the versioning state of the bucket. Important. Object ......

Lifecycle Policies | Linode

A Lifecycle rule can apply to all or a subset of objects in a bucket based on the element that you specify in...

Empty an Amazon S3 bucket with a lifecycle configuration rule

For buckets with versioning turned off, expire the current versions of the objects, and then permanently delete any incomplete multipart uploads.

Simplify your data lifecycle by using object tags with Amazon ...

Customers can define an entire bucket or a subset of objects to transition or expire with rules in the lifecycle configuration.

Deploying Lifecycle Rules to Manage Amazon S3 Storage

In non-versioned S3 buckets, the expiration action results in Amazon S3 permanently deleting the object automatically. For example, you may choose to delete...