In the last Understand Django article, we dug into file management. We saw how Django handles user uploaded files and how to deal with them safely.
With this article, you'll learn about commands. Commands are the way to execute scripts that interact with your Django app. We'll see built-in commands and how to build your own.
Why Commands?
Django makes it possible to run code from a terminal with ./manage.py
, but why is this helpful or needed? Consider this script:
# product_stat.py
from application.models import Product
print(Product.objects.count())
which you could try running with:
$ python product_stat.py
The problem with this script is that Django is not ready to run yet. If you tried to run this kind of code, you would get an ImproperlyConfigured
exception. There are a couple of modifications that you could make to get the script to run.
- Call
django.setup()
. - Specify the
DJANGO_SETTINGS_MODULE
.
# product_stat.py
import django
django.setup()
from application.models import Product
print(Product.objects.count())
Note that django.setup()
must be before your Django related imports (like the Product
model in this example). Now the script can run if you supply where the settings are located too.
$ DJANGO_SETTING_MODULE=project.settings python product_stat.py
This arrangement is less than ideal, but why else might we want a way to run commands through Django?
Try running ./manage.py -h
.
What you'll likely see is more commands than what Django provides alone. This is where we begin to see more value from the command system. Because Django provides a standard way to run scripts, other Django applications can bundle useful commands and make them easily accessible to you.
Now that you've had a chance to see why commands exist to run scripts for Django apps, let's back up and see what commands are.
We Hereby Command
Django gives us a tool to run commands before we've even started our project. That tool is the django-admin
script. We saw it all the way back in the first article where I provided a short set of setup instructions to get you started if you've never used Django before.
After you've started a project, your code will have a manage.py
file, and the commands you've seen in most articles are in the form of:
$ ./manage.py some_command
What's the difference between django-admin
and manage.py
? In truth, not much!
django-admin
comes from Django's Python packaging. In Python packages, package developers can create scripts by defining an entry point in the packaging configuration. In Django, this configuration looks like:
[options.entry_points]
console_scripts =
django-admin = django.core.management:execute_from_command_line
Meanwhile, the entire manage.py
of one of my projects is:
#!/usr/bin/env python
"""Django's command-line utility for administrative tasks."""
import os
import sys
def main():
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'project.settings')
try:
from django.core.management import execute_from_command_line
except ImportError as exc:
raise ImportError(
"Couldn't import Django. Are you sure it's installed and "
"available on your PYTHONPATH environment variable? Did you "
"forget to activate a virtual environment?"
) from exc
execute_from_command_line(sys.argv)
if __name__ == '__main__':
main()
If you look closely, you can see that the different scripts are both ways to invoke the execute_from_command_line
function to run a command. The primary difference is that the latter script will attempt to set the DJANGO_SETTINGS_MODULE
environment variable automatically. Since Django needs to have DJANGO_SETTINGS_MODULE
defined for most commands (note: startproject
does not require that variable), manage.py
is a more convenient way to run commands.
execute_from_command_line
is able to present what commands are available to a project, whether a command comes from Django itself, an installed app, or is a custom command that you created yourself. How are the commands discovered? The command system does discovery by following some packaging conventions.
Let's say your project has an app named application
. Django can find the command if you have the following packaging structure.
application
├── __init__.py
├── management
│ ├── __init__.py
│ └── commands
│ ├── __init__.py
│ └── custom_command.py
├── models.py
└── views.py
... Other typical Django app files
With this structure, you could run:
$ ./manage.py custom_command
Notes:
- Django will create a command for a module found in
<app>/management/commands/<command name>.py
. - Don't forget the
__init__.py
files! Django can only discover the commands ifmanagement
andcommands
are proper Python package directories. - The example uses
custom_command
, but you can name your command with whatever valid Python module name that you want.
Unfortunately, we can't slap some Python code into custom_command.py
and assume that Django will know how to run it. Within the custom_command.py
module, Django needs to find a Command
class that subclasses a BaseCommand
class that is provided by the framework. Django requires this structure to give command authors a consistent way to access features of the command system.
With the Command
class, you can add a help
class attribute. Adding help can gives users a description of what your command does when running ./manage.py custom_command -h
.
The Command
class will also help you with handling arguments. If your command needs to work with user input, you'll need to parse that data. Thankfully, the class integrates with Python's built-in argparse
module. By including an add_arguments
method, a command can parse the data and pass the results to the command's handler method in a structured way. If you've had to write Python scripts before, then you may understand how much time this kind of parsing can save you (and for those who haven't, the answer is "a lot of time!").
Other smaller features exist within the Command
class too. Perhaps you only want your command to run if your project has satisified certain pre-conditions. Commands can use the requires_migration_checks
or requires_system_checks
to ensure that the system is in the correct state before running.
I hope it's clear that the goal of the Command
class is to help you with common actions that many commands will need to use. There is a small API to learn, but the system is a boon to making scripts quickly.
Command By Example
Let's consider a powerful use case to see a command in action. When you initially start a Django app, all of your app's interaction will probably be through web pages. After all, you were trying to use Django to make a web app, right? What do you when you need to do something that doesn't involve a browser?
This kind of work for your app is often considered background work. Background work is a pretty deep topic and will often involve special background task software like Celery. When your app is an early stage, Celery or similar software can be overkill and far more than you need.
A simpler alternative for some background tasks could be a command paired with a scheduling tool like cron.
On one of my projects, I offer free trials for accounts. After 60 days, the free trial ends and users either need to pay for the service or discontinue using it. By using a command and pairing it with the Heroku Scheduler, I can move accounts from their trial status to expired with a daily check.
The following code is very close to what my expire_trials
command looks like in my app. I've simplified things a bit, so that you can ignore the details that are specific to my service.
# application/management/commands/expire_trials.py
import datetime
from django.core.management.base import BaseCommand
from django.utils import timezone
from application.models import Account
class Command(BaseCommand):
help = "Expire any accounts that are TRIALING beyond the trial days limit"
def handle(self, *args, **options):
self.stdout.write("Search for old trial accounts...")
# Give an extra day to be gracious and avoid customer complaints.
cutoff_days = 61
trial_cutoff = timezone.now() - datetime.timedelta(days=cutoff_days)
expired_trials = Account.objects.filter(
status=Account.TRIALING, created__lt=trial_cutoff
)
count = expired_trials.update(status=Account.TRIAL_EXPIRED)
self.stdout.write(f"Expired {count} trial(s)")
I configured the scheduler to run python manage.py expire_trials
every day in the early morning. The command checks the current time and looks for Account
records in the trialing state that were created before the cutoff time. From that query set, the affected accounts are set to the expired account state.
How can you test this command? There are a couple of approaches you can take when testing a command.
If you need to simulate calling the command with command line arguments, then you can use call_command
from django.core.management
. Since the example command doesn't require arguments, I didn't take that approach.
Generally, my preference is to create a command object and invoke the handle
method directly. In my example above, you can see that the command uses self.stdout
instead of calling print
. Django does this so that you could check your output if desired.
Here is a test for this command:
# application/tests/test_commands.py
from io import StringIO
from application.models import Account
from application.tests.factories import AccountFactory
def test_expires_trials():
"""Old trials are marked as expired."""
stdout = StringIO()
account = AccountFactory(
status=Account.TRIALING,
created=timezone.now() - datetime.timedelta(days=65),
)
command = Command(stdout=stdout)
command.handle()
account.refresh_from_db()
assert account.status == Account.TRIAL_EXPIRED
assert "Expired 1 trial(s)" in stdout.getvalue()
In this test, I constructed a command instance and checked the account state after the command invocation. Also, observe that the StringIO
instance is injected into the Command
constructor. By building the command this way, checking the output becomes a very achievable task via the getvalue
method.
Overall, this scheme of making a command and running it on a schedule avoids all the work of setting up a background worker process. I've been extremely satisfied with how this technique has worked for me, and I think it's a great pattern when your app doesn't have to do a lot of complex background processing.
Useful Commands
Django is full of useful commands that you can use for all kinds of purposes. Thus far in this series, we've discussed a bunch of them, including:
-
check
- Checks that your project is in good shape. -
collectstatic
- Collects static files into a single directory. -
createsuperuser
- Creates a super user record. -
makemigrations
- Makes new migration files based on model changes. -
migrate
- Runs any unapplied migrations to your database. -
runserver
- Runs a development web server to check your app. -
shell
- Starts a Django shell that allows you to use Django code on the command line. -
startapp
- Makes a new Django app from a template. -
startproject
- Makes a new Django project from a template. -
test
- Executes tests that checks the validity of your app.
Here is a sampling of other commands that I find useful when working with Django projects.
dbshell
The dbshell
command starts a different kind of shell. The shell is a database program that will connect to the same database that your Django app uses. This shell will vary based on your choice of database.
For instance, when using PostgreSQL, ./manage.py dbshell
will start psql
. From this shell, you can execute SQL statements directly to inspect the state of your database. I don't reach for this command often, but I find it very useful to connect to my database without having to remember database credentials.
showmigrations
The showmigrations
command has a simple job. The command shows all the migrations for each Django app in your project. Next to each migration is an indicator of whether the migration is applied to your database.
Here is an example of the users
app from one of my Django projects:
$ ./manage.py showmigrations users
users
[X] 0001_initial
[X] 0002_first_name_to_150_max
[ ] 0003_profile
In my real project, I've applied all the migrations, but for this example, I'm showing the third migration as it would appear if the migration wasn't applied yet.
showmigrations
is a good way to show the state of your database from Django's point of view.
sqlmigrate
The sqlmigrate
command is very handy. The command will show you what SQL statements Django would run for an individual migration file.
Let's see an example. In Django 3.1, the team changed the AbstractUser
model so that the first_name
field could have a maximum length of 150 characters. Anyone using the AbstractUser
model (which includes me) had to generate a migration to apply that change.
From my showmigrations
output above, you can see that the second migration of my users
app applied this particular framework change.
To see the Postgres SQL statements that made the change, I can run:
$ ./manage.py sqlmigrate users 0002
BEGIN;
--
-- Alter field first_name on user
--
ALTER TABLE "users_user" ALTER COLUMN "first_name" TYPE varchar(150);
COMMIT;
From this, we can tell that Postgres executed an ALTER COLUMN
DDL statement to modify the length of the first_name
field.
squashmigrations
Django migrations are a stack of separate database changes that produce a final desired schema state in your database. Over time, your Django apps will accumulate migration files, but those files have a shelf life. The squashmigrations
command is designed to let you tidy up an app's set of migration files.
By running squashmigrations
, you can condense an app's migrations into a significantly smaller number. The reduced migrations can accurately represent your database schema, and make it easier to reason about what changes happened in the app's history. As a side benefit, migration squashing can make Django's migration handling faster because Django gets to process fewer files.
Even More Useful Commands
The commands above come with the standard Django install. There's even more cool stuff out there to help with your project development!
A package that I often reach for with my Django projects is the django-extensions package. This package is fully of goodies, including some great optional commands that you can use!
A couple of my favorites include:
shell_plus
How often do you fire up a Django shell, import a model, then do some ORM queries to see the current state of the database? This is something I do quite often.
The shell_plus
command is like the regular shell, but the command will import all your models automatically. For the five extra characters of _plus
, you can save your fingers a lot of typing to import your models and get directly to whatever you needed the shell for.
The command will also import some commonly used Django functions and features like reverse
, settings,
timezone
, and more.
graph_models
When I'm live streaming my side projects on my Twitch channel, I will often want to show the model relationships of my Django project. With the graph_models
command, I can create an image of all my models and how those models relate to each other (using UML syntax). This is a great way to:
- Remind myself of the data modeling choices in my apps.
- Orient others to what I'm doing with my project.
This particular command requires some extra setup to install the right tools to create images, but the setup is manageable and the results are worth it.
Aside from shell_plus
and graph_models
, there are 20 other commands that you can use that may be very useful to you. You should definitely check out django-extensions.
Summary
In this article, you saw Django commands. We covered:
- Why commands exist in the Django framework
- How commands work
- How to create your own custom command and how to test it
- Useful commands from the core framework and the django-extensions package
In the next article, I'm going to describe internationalization. Internationalization is the process of making your app work for multiple spoken languages. You'll learn about:
- Enabling internationalization for your app
- Tools to help manage an internationalized app
- Extra considerations like handling time formatting
If you'd like to follow along with the series, please feel free to sign up for my newsletter where I announce all of my new content. If you have other questions, you can reach me online on Twitter where I am @mblayman.
Top comments (0)