Sometimes, you want to delete something from your database without actually deleting it. You want to preserve it, in case you ever decide to restore it in the future, or to use it for some other purpose, such as analytics … but it should still appear as if it were actually deleted, not show up in unexpected places in your application. This is called the 'soft delete' pattern.
Instead of actually deleting a record from your database, which is irreversible, you simply 'mark' it as deleted, usually in another column in your table. Now the challenge remains, how to prevent soft-deleted records from 'leaking' into your application.
In this article, we will learn how to implement soft deletion in Django by taking the example of a simple note-taking app backend.
We will start by creating a base
SoftDeleteModel that can be inherited by the rest of our models.
class SoftDeleteModel(models.Model): is_deleted = models.BooleanField(default=False) def soft_delete(self): self.is_deleted = True self.save() def restore(self): self.is_deleted = False self.save() class Meta: abstract = True
Note that we have marked this model as
abstract = True. This means that Django will not create a database table for it.
Now, we can create our models as subclasses of
SoftDeleteModel to grant them the ability to be soft-deleted. Let's take the example of a
class Note(SoftDeleteModel): user = models.ForeignKey(User, on_delete=models.CASCADE, related_name="notes") title = models.CharField(max_length=32) content = models.CharField(max_length=255)
We can query our database filtering by
is_deleted in order to exclude records that have been soft-deleted.
Let's try playing around a bit with the code we've written so far. First, open up the Django shell by typing
python manage.py shell in your terminal.
Import the models required:
from django.contrib.auth.models import User from tutorialapp.models import Note
Since each note is foreign-keyed to a user, our first step is to create a
john = User.objects.create_user('john', 'email@example.com', 'johnpassword')
Now we can create a couple of notes:
my_note = Note.objects.create(user=john, title="Strawberry Fields", content="Strawberry Fields Forever") another_note = Note.objects.create(user=john, title="Here Comes The Sun", content="It's All Right")
You are now ready to soft delete and restore notes:
You can query for all notes, whether they have been soft deleted or not:
You can also filter only for notes that have not been soft deleted:
While our code is functionally correct, the disadvantage is that we will have to remember to filter by
is_deleted=False each time we write a query.
We can improve upon this behaviour by creating a custom model manager to apply the filter automatically, behind the scenes. If you've used Django in the past, you might be familiar with statements that look like this:
.objects part in the statement is the manager. Managers act as the 'bridge' between your Django code and the database. They control the database operations performed on the tables that they 'manage'.
Our new custom manager can be defined as:
class SoftDeleteManager(models.Manager): def get_queryset(self): return super().get_queryset().filter(is_deleted=False)
We will need to add the new manager to our
SoftDeleteModel base class:
class SoftDeleteModel(models.Model): is_deleted = models.BooleanField(default=False) objects = models.Manager() undeleted_objects = SoftDeleteManager() def soft_delete(self): self.is_deleted = True self.save() def restore(self): self.is_deleted = False self.save() class Meta: abstract = True
Note that, since we have added a custom manager to our class, we are required to explicitly add the default
objects manager as well.
Then, we can simply rewrite our query as,
to get a
QuerySet of undeleted notes.
We can still use
to get the full list of notes, including those that have been soft-deleted.
Now, what if you have multiple users, and you want to fetch all the notes belonging to a specific user? The naive approach is to simply write a query filtering against the user:
However, a more elegant and readable solution is to make use of the reverse relationships Django provides for this purpose.
Try soft deleting some of your notes and running this query. Do you notice something unusual about the results?
We find that the resultant
QuerySet contains records that we had soft deleted. This is because Django is using the default
objects manager to perform the reverse lookup, which, as you may recall, does not filter out soft deleted records.
How can we force Django to use our custom
SoftDeleteManager to perform reverse lookups? We can simply replace the default
objects manager in our
class SoftDeleteModel(models.Model): is_deleted = models.BooleanField(default=False) objects = SoftDeleteManager() all_objects = models.Manager() def soft_delete(self): self.is_deleted = True self.save() def restore(self): self.is_deleted = False self.save() class Meta: abstract = True
objects manager will automatically filter out soft-deleted objects when querying our database, ensuring they never leak into our application under any circumstances! If we want to, we can still include soft deleted objects in our queries by making use of the
We've already got a pretty solid soft deletion framework in our Django app, but we can make one final improvement. Knowing whether a record is soft deleted or not is useful, but another piece of information that would be nice to know is when the record was soft deleted. For this, we can add a new a attribute
deleted_at to our
deleted_at = models.DateTimeField(null=True, default=None)
We can also update our
restore methods as follows:
def soft_delete(self): self.deleted_at = timezone.now() self.save() def restore(self): self.deleted_at = None self.save()
For undeleted records, the value of
deleted_at will be null, while for soft deleted records, it will contain the date and time at which it was deleted.
The addition of the new
deleted_at attribute makes our previously created
is_deleted attribute redundant, because we can simply perform a null-check on
deleted_at to find out whether the record is soft deleted or not.
SoftDeleteModel now looks like this:
class SoftDeleteModel(models.Model): deleted_at = models.DateTimeField(null=True, default=None) objects = SoftDeleteManager() all_objects = models.Manager() def soft_delete(self): self.deleted_at = timezone.now() self.save() def restore(self): self.deleted_at = None self.save() class Meta: abstract = True
And our rewritten
SoftDeleteManager looks like this:
class SoftDeleteManager(models.Manager): def get_queryset(self): return super().get_queryset().filter(deleted_at__isnull=True)
In addition to our previous capabilities, we can now also see the exact date and time at which our record was soft deleted:
Soft deletion is a powerful software pattern that comes with several advantages, including better data preservation and restoration, history tracking, and faster recovery from failures.
At the same time, it should be used with care. Sensitive and personal data including payment-related information should always be hard-deleted. Users should always have the option to have their data permanently deleted, if they wish. Several jurisdictions around the world have information privacy and data protection laws that include the 'right to be forgotten', such as the European Union's GDPR. It might also make sense to periodically delete or archive data that is very old, to avoid eating up excess database storage space.
If you would like to view the complete source code for the example used in this tutorial, it is available on GitHub.