Recently I needed to delete some documents that I save in MongoDB after some time. I can think of a few examples why we would want to delete data after some time:
- logs / events
- api keys / access tokens
- non-active users
- etc
We could do it by running a cronjob that deletes the data, delete the data every time we insert new data or any other solution.
Luckily for me, my wife told me that MongoDB already have that mechanism built-in.
TTL index
TTL (Time-To-Live) indexes are special single-field indexes that MongoDB can use to automatically remove documents from a collection after a certain amount of time.
A background thread in mongod reads the values in the index and removes expired documents from the collection (usually every minute).
For example, to create a TTL index on the lastModifiedDate
field of the eventlog
collection, use the following operation in the mongo shell:
db.eventlog.createIndex( { "lastModifiedDate": 1 }, { expireAfterSeconds: 3600 } )
- The indexed field must be BSON date type or an array of BSON dates
- If the indexed field in a document is not a date or an array that holds a date value(s), the document will not expire.
- If a document does not contain the indexed field, the document will not expire.
Conditional delete
As of MongoDB 3.2, a collection can be partially indexed using a specified filter expression, partialFilterExpression
. TTL index can also be used with partial indexes.
For example:
Delete documents that created 1 hour ago and their state
is TMP
db.eventlog.createIndex(
{ created_at: 1 },
{ expireAfterSeconds: 3600, partialFilterExpression: { state: 'TMP' } }
);
Delete documents that created 1 day ago and their count
is lower than 5
db.eventlog.createIndex(
{ created_at: 1 },
{ expireAfterSeconds: 86400, partialFilterExpression: { count: { $lt: 5 } } }
);
Additional info on partial indexes
Real world example
Recently I finished developing BundleMon, which is a free tool that helps you monitor your apps bundle size.
One of the components in BundleMon is a service that saves historic reports in order to compare bundle size between branches, So when you open a PR BundleMon saves a record with the current bundle size report.
No need to save the report more than 30 days, so I just added a TTL index:
db.reports.createIndex(
{ creationDate: 1 },
{ expireAfterSeconds: 2592000, partialFilterExpression: { prNumber: { $exists: true } } }
);
Top comments (4)
Hi, Liron I have a doubt.
Hey
You can read more about the remove method here
If you want to auto remove specific client data that is older than X time then you can use TTL index:
How can we achieve this with mongoose ??
stackoverflow.com/a/14597622/2602426