How to set up Django with Postgres from Neon

In this blog post, you’ll learn how to connect Django to Neon Serverless Postgres. We’ll start with a simple Django CRUD app as our example and demonstrate how to integrate Neon as the backing database.

If you’re unfamiliar with Neon, consider it an open-source, standard Postgres version of AWS Aurora. It is PostgreSQL delivered as a cloud service, but it goes a step further than other managed Postgres services. Under the hood, Neon has separated the compute and storage layers of the database so that, as users, we simply get a URL to our database, and Neon handles the rest. When we’re not using the DB, Neon scales compute to zero, and we pay nothing; when usage goes up, Neon scales up to handle the load. You can sign up for their free tier here to follow along.

Prerequisites 🤔

Before getting started with this tutorial, you’ll need:
Git installed on your local machine - to install it, follow this guide.

• Python 3.* Installed - follow this guide if you don't have it already
• Pip - Python package manager
• Django 5.0 - follow this guide to install Django.
• Basic knowledge of Python and Django concepts and conventions

Getting Started with Django Setup 💡

Let’s start by setting up a CRUD application using Django before we dive into the Neon part. If you already know Django, you can proceed to the next section. Here, we will create a basic Django project, set up an app, define our model, and perform our first migration.

Our CRUD app is a note-taking application with creation, reading, updating, and deleting capabilities. Please find the source code for the Django project available here.

To follow along with the tutorial, clone the project by running:

git clone https://github.com/theinfosecguy/neon-django-api.git

From your terminal, open up the project in your IDE of choice. We’ll first set up a virtual environment with pipenv. Install it with the pip install pipenv command.

Create a new virtual environment using the pipenv shell command. Now that our virtual environment is created let’s install Django using the pipenv install django command.

Lastly, install Django REST Framework using pipenv install django_rest_framework command.

We’re done with initial dependencies we’ll require. To test your progress so far, run:

python3 manage.py runserver

You’ll see an error saying Error loading psycopg2 or psycopg module which means you’re all set to follow along. We’ll resolve this further in the article.

Another problem is that Django uses sqlite database by default. We’ll switch to Postgres after integrating Neon in the upcoming sections.

Neon Setup and Integration ⚡

Let’s walk you through the process of configuration of Neon with Django. This is how we’ll hook Neon up to our Django project, set up the initial database and tables, and map our Django models into Neon’s schema.

Firstly, you need to have a Neon account. Sign up at neon.tech and create a Neon Project. Each Project is its own Postgres Cluster.

Once done, it will open up a prompt with the connection details for your newly created project. Select Django from the dropdown and copy the database configuration to your clipboard.

Then, head to the project directory, open the file neon_project\settings.py, and find the variable DATABASES. It’ll look somewhat like this as per typical Django boilerplate code where, by default, it uses a SQLite database under the hood.

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': BASE_DIR / 'db.sqlite3',
    }
}

Replace the value of this with the new configuration.

DATABASES = {
  'default': {
    'ENGINE': 'django.db.backends.postgresql',
    'NAME': 'neon_example_db',
    'USER': 'your-username-here',
    'PASSWORD': 'your-password-here',
    'HOST': 'ep-steep-recipe-44764255.us-east-2.aws.neon.tech',
    'PORT': '5432',
    'OPTIONS': {'sslmode': 'require'},
  }
}

Follow through the rest of the article however we strongly advise that you revisit this after local testing. Switch to either environment variables or adopt a secret management service such as Hashicorp Vault or AWS KMS to store these secrets. Here’s an example of an environment variable-based approach with Python.

DATABASES = {
  'default': {
    'ENGINE': os.environ.get('db_engine', 'django.db.backends.postgresql'),
    'NAME': os.environ.get('db_name'),
    'USER': os.environ.get('db_user'),
    'PASSWORD': os.environ.get('db_password'),
    'HOST': os.environ.get('db_host'),
    'PORT': os.environ.get('db_port', '5432'),
    'OPTIONS': {'sslmode': 'require'},
  }
}

For instance, the .env file could look like the below example.

db_engine=django.db.backends.postgresql
db_name=neon_example_db
db_user= # your-username-here
db_password= # your-password-here
db_host=ep-steep-recipe-44764255.us-east-2.aws.neon.tech
db_port=5432

Lastly, we need a Postgres database driver for Django. We’ll install [psycopg](https://pypi.org/project/psycopg2/), which should serve the purpose of this tutorial. This will resolve the “Error loading psycopg2 or psycopg module” error we faced while setting up the project initially.

pipenv install psycopg2-binary

Once installed, run the following command so that Django can read the new settings. Django will automatically make the necessary migrations to initialize our Neon database.

python3 manage.py migrate

Now that our database is ready, run the following command to run the development server.

python3 manage.py runserver

Django provides a built-in admin panel that we can use for read and write API operations. Head to http://127.0.0.1:8000/notes/api/ and create a new note entry.

Once the new note entry is created, refresh the page. You’ll see that the new record is created and is being fetched with the GET All Notes API call on the top.

Next, we can verify the same from the Neon interface. Head to https://console.neon.tech/app/projects/{{your_project_id_here}}/tables and scroll down in the tables section to our notes table. You can see that the records are visible in the interface, too.

We have now successfully integrated NeonDB with our Django app 🔥. NeonDB is incredibly easy to integrate into a Django project, providing smooth access to perform database tasks.

For instance, in this tutorial, we've strategically switched from our ordinary SQLite database to a Postgres one to perfectly match the features NeonDB offers. This transition also highlights the adaptability and integration that NeonDB provides for Django development. Stay tuned for part two of this tutorial where we demonstrate some of the unique branching features of Neon with our CRUD app.