Django Continuous Deployment to PythonAnywhere

This is a streamlined version of my original Python Django tutorial. Here, I want to focus on the continuous deployment stage, so you may get your website online quicker. If, after reading this, you are curious about multi-branch deployments and continuous integration, be sure to check out the original article. I hope you enjoy it. 🍭

In this hands-on guide, we're going to learn how to deploy a Django website to PythonAnywhere using Semaphore Continuous Integration and Delivery (CI/CD). No matter how reliable the code is, we still need to implement CI/CD to detect and remedy errors quickly. When we have confidence in the accuracy of our code, you can ship updates faster and with fewer mistakes.

Things You’ll Need

You’ll need GitHub and Semaphore accounts. I’d recommend installing Semaphore’s commad line tool for an easy setup.

To get started, fork, clone the Django demo project and add it to Semaphore:

$ cd semaphore-demo-python-django
$ sem init

The application you just forked is a simple task manager; we can create and edit tasks, we also have a separate admin site to manage users and permissions. The website is built with Python and Django. The data will be stored in MySQL.

A Note About Continuous Integration

Continuous Integration is a programming discipline in which the application is built and tested each time there is a push. By making multiple small changes instead of a big one, problems are detected earlier and corrected faster. Such a paradigm, clearly, calls for an automated system to carry out all the steps. In such systems, code travels over a path—a pipeline—and it must pass an ever-growing number of tests before it can reach the users.

The project includes a complete CI pipeline with a complete suite of tests. No setup required:

• Unit tests with nose and coverage.
• Static code analysis with pylint.
• In-browser tests with selenium.

The in-depth explanation about the CI stage can be found on the original full-length tutorial

Deploy to PythonAnywhere

Websites are meant to run on the internet; let see how we can publish our site the world to enjoy. PythonAnywhere is a hosting provider that, as the name suggests, specializes in Python. In this section, we'll learn how to use it.

Create the Website

Head to PythonAnywhere (PA) and create an account. In order to do an automated deployment, we need SSH access, and that requires a paid account, so sign up for the entry-level tier (hacker 🎩).

Once you have your account, create the services for the application:

1. Create a MySQL database called pydjango_production with a good password.
2. Generate an API Token.
3. Add your public SSH key to PA servers:

$ ssh-copy-id YOUR_PA_USERNAME@ssh.pythonanywhere.com

Create an environment file for your application: .env

# This value is found on PythonAnywhere Accounts->API Token.
export API_TOKEN=YOUR_API_TOKEN

# Django Secret Key - Use a long random string for security.
export SECRET_KEY=A_RANDOM_STRING_FOR_DJANGO

# These values can be located on PythonAnywhere Databases tab.
export DB_HOST=YOUR_PA_USERNAME.mysql.pythonanywhere-services.com
export DB_USER=YOUR_PA_USERNAME
export DB_PASSWORD=YOU_DB_PASSWORD
# The name of the DB is prefixed with USERNAME$
export DB_NAME='YOUR_PA_USERNAME$pydjango_production'
export DB_PORT=3306

Scp the file to the server:

$ scp .env YOUR_PA_USERNAME@ssh.pythonanywhere.com:~

Now we're ready to create the website. Luckily for us, there is an official helper script. If you wish to use a custom domain instead of the default one (USERNAME.pythonanywhere.com), add a --domain= option.

$ ssh YOUR_PA_USERNAME@ssh.pythonanywhere.com
$ source ~/.env
$ pa_autoconfigure_django.py --python=3.7 YOUR_GITHUB_REPO_URL

The script should take a few minutes to complete. Take a cup of coffee, and don't forget to stretch 🧘🏽.

Once done, there’s only one thing left to do. We need the app to have access to our environment file. Edit the WSGI file for your new website, by default it’s located at /var/www. Add three lines to that file, as shown below:

# This file contains the WSGI configuration required to serve up your
# Django app
import os
import sys

# Add your project directory to the sys.path
settings_path = '/home/YOUR_PA_USERNAME/YOUR_PA_USERNAME.pythonanywhere.com'
sys.path.insert(0, settings_path)

# Set environment variable to tell django where your settings.py is
os.environ['DJANGO_SETTINGS_MODULE'] = 'pydjango_ci_integration.settings'

# -–-> ADD THESE NEXT THREE LINES TO YOUR WSGI.PY <-------
from dotenv import load_dotenv
env_file = os.path.expanduser('~/.env')
load_dotenv(env_file)
# --------------------------------------------------------

# Set the 'application' variable to the Django wsgi app
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()

That’s it, after reloading the website, it should be online 🥇.

Benefits of Continuous Deployment

Deployment is a complex process with a lot of moving parts. It would be a shame if, after painstakingly testing everything, the application crashes due to a faulty deployment.

Continuous Deployment (CD) is an extension of the CI concept; in fact, most integration tools don't make a great distinction between CI and CD. A CD pipeline performs all the deployment steps as a repeatable, battle-hardened process.

Deployment with Semaphore

We’re going to create a new pipeline to deploy the updates automatically on each update.

The deployment process needs some secret data, for example, the SSH key to connect to PythonAnywhere. The environment file also has sensitive information, so we need to protect it.

Storing secrets in Semaphore is easy as pie:

$ sem create secret ssh-key \
    -f $HOME/.ssh/id_rsa:/home/semaphore/.ssh/id_rsa_pa

Now do the same for the environment file:

$ sem create secret env \
    -f ./.env:/home/semaphore/.env

Create a new pipeline file at .semaphore/deploy.yml with the next three code boxes:

version: v1.0
name: Deploy Django to PythonAnywhere
agent:
  machine:
    type: e1-standard-2
    os_image: ubuntu1804

The pipeline has only one “Deploy” block. It begins by invoking the secrets we just created: env and ssh key. We can also define environment variables at this point, adjust the values for your user:

blocks:
  - name: Deploy
    task:
      secrets:
        - name: env
        - name: ssh-key
      env_vars:
        - name: SSH_USER
          value: YOUR_PA_USERNAME

The job uses checkout to clone the repository, then envsubst expands the values of deployment.sh with the corresponding environment values. Finally, the files are copied to PA and the deployment script executed remotely:

      jobs:
        - name: Deploy to PythonAnywhere
          commands:
            - checkout
            - cat deployment.sh | envsubst > ~/deploy.sh
            - chmod 0600 ~/.ssh/id_rsa_pa
            - ssh-keyscan -H ssh.pythonanywhere.com >> ~/.ssh/known_hosts
            - ssh-add ~/.ssh/id_rsa_pa
            - scp ~/.env ~/deploy.sh ${SSH_USER}@ssh.pythonanywhere.com:~
            - ssh ${SSH_USER}@ssh.pythonanywhere.com bash deploy.sh

I mentioned deployment.sh, but haven't shown it yet. Here it is:

# pull updated version of branch from repo
cd ${SSH_USER}.pythonanywhere.com
git fetch --all
git reset --hard origin/$SEMAPHORE_GIT_BRANCH

# perform django migration task
source ~/.env
source ~/.virtualenvs/${SSH_USER}.pythonanywhere.com/bin/activate
python manage.py migrate

# restart web application
touch /var/www/${SSH_USER}_pythonanywhere_com_wsgi.py

In short, the script does 3 things:

1. Updates the app code from the repository.
2. Executes manage.py migrate, in case there new code has additional tables.
3. Restarts the web application.

Now all that remains is to link the pipelines. This is achieved adding a promotion to the end of .semaphore/semaphore.yml:

promotions:
  - name: Deploy
    pipeline_file: deploy.yml

Push all the updated files to your repository:

$ git add .semaphore/*
$ git add deployment.sh
$ git commit -m "add deployment"
$ git push origin master

Semaphore will start working immediately. Wait until the CI pipeline is done:

And hit the Promote button to start the deployment:

Enjoy your new website.

Conclusion

We've discovered the incredible potential of a CI/CD platform. I hope that the tools and practices discussed here can add value to your projects, improve your team effectiveness, and make your life easier.

For the next steps, I suggest browsing Semaphore Blog for more examples and tutorials, and, of course, writing a pipeline for your own application. Good luck!

Did you find the post useful? Let me know by ❤️-ing or 🦄-ing below! Do you have any questions or suggestions for other tutorials? Let me know in the comments. Thank you for reading!