While optimizing a django database migration which iterated over batches of a large table (~40 Mio rows), I noticed that the memory usage would consistently grow. Memory consumption kept growing even after I
del'ed all inner-loop variables and asked the garbage collector to do their work.
I finally figured out that Django's
DEBUG mode, which is well known to use up memory for traceability, has the same effect in migrations, too. Adding the following two lines to your migration ensures smooth, memory-efficient processing, no matter the environment:
from django.conf import settings
settings.DEBUG = False