Create TTL indexes for models

Problem

Create Time to Live (TTL) indexes on the model and let database do it automatically.

We are currently working on an app using Prisma with Postgres, and we wanted to set expiration time for entries in a row, something already available in Redis, Mongo and also Postgres (using TRIGGERS) or storing a timestamp and then manually code some sort of cron job to check what entries have expired. How to do this is currently not possible with Prisma (unless editing the generated SQL).

In Mongo:

👉 TTL indexes are special single-field indexes that MongoDB can use to automatically remove documents from a collection after a certain amount of time or at a specific clock time.

👉 A background thread in mongod reads the values in the index and removes expired documents from the collection.

Use Cases:

We want data entries/rows to be deleted automatically when the database table starts overflowing and our database memory reaches the max limit after a set time without manually writing cron jobs.

Few types of data that need to be cleaned periodically are:

• logs,
• user sessions,
• notifications,
• advertisements for limited time offers and discounts etc…

Consider a coupon generator app with Prisma, with the following schema:

model Coupon {
  id       String       @id @default(cuid()) @expire(after: “60s”, when: “useCount>100”)
  code     String       @unique
  region   RegionEnum?  @default(NONE)
  useCount Int
  users    CouponUser[]
  // ...
}

In this case, we want Postgres to set EXPIRE time for each coupon based on a directive @expire(after: “60s”, when: “useCount>100”) or just delete the affected row after a minute of insertion @expire(after: “1m”)

Also consider a Chat app using Prisma and Postgres, that allows messages to be deleted 60 seconds after the message is marked as ‘read’ with the following schema:

model Messages {
  id          String  @id @default(cuid()) @expire(after: “60s”, when: “read=true”)
  text        String
  sender_id   String
  receiver_id String
  read        Boolean @default(false)
  // ...
  expireAfter Int     @default(60)
  // ...
}

Suggested solution

Create Time to Live (TTL) indexes on the model and let database do it automatically after a certain amount of time or at a specific clock time. Prisma can then generate Postgres/SQL TRIGGERS based on the @expire directive for every model.

Alternatives

Our current alternative is using a cron to check for an expiring timestamp - to delete old records, which is heavy on the app. Postgres should be able to handle this seamlessly.

I really think Prisma should provide this sort of functionality, and should be in the roadmap as a feature for future releases.

[janpio]

We do not make statements about when features will be tackled until we tackle them. Right now this is not on our definitive roadmap, so we are not making any statements.

The original issue asks for support for other databases via constructs like triggers - that is definitely far off, as we are focussing on database features first (which might include the ability to define a trigger like that).

For MongoDB where this is a feature of the database itself, I see much higher chances: https://www.mongodb.com/docs/upcoming/core/index-ttl/

I also notice that the suggestion adds a new fiel level property. Would this not be more something to add to @@index as a additional attribute and maybe a new type? @@index([foo], expireAfterSeconds: 3600) or maybe @@index([foo], type: ttl, expireAfterSeconds: 3600)? Or is there a reason I am missing?

[KrishGarg]

Update 2: those who just want to get it done… here

Is this something that’s supported natively by the MongoDB driver?

If so, you could probably use the raw query support we recently released. Not ideal, but might be a decent enough workaround until we support this feature.

Let us know!

Yes. It is supported by the official driver and mongoose. Can you please give an example of how can we do the same with the raw query in Prisma as there isn’t much information in the docs about running raw commands? Also, will there be any issues in running the command multiple times? Like it may run every time we restart the server and create multiple indexes or something?

Update: I googled around and found some resources which might help others. Raw Commands: https://www.mongodb.com/docs/manual/reference/command/ On recreation of an index on app startup: https://www.mongodb.com/community/forums/t/behavior-of-createindex-for-an-existing-index/2248