Playing with Django Models

Hey folks,
Recently I was building a webapp for a friend of mine, helping him out for some freelancing stuff, where I had to dig into quite a lot of database and ER modelling stuff, and it was pretty cool relating old DBMS classes with everything django provides. So I thought this would be an excellent opportunity to share what I learnt with the community. Of course this assumes you are familiar with python and django. If not, this is an excellent place to get started.

Before we get into anything, lets sync up on our essentials. I have used django 2.2 and python 3.7 throughout this post. I highly recommend using a virtual environment to setup all dependencies. Now while django does come with a default database engine of SQLite, I needed something more secure and so I used MySQL. Plus I thought it would be a good chance to learn to use django with some different kinds of databases. Everything in this post can also be done using SQLite, by the way. Either way, I will come up with a post on setting up django with MySQL shortly.

So to get started, let's say we are building an e-commerce website, and lets have two apps for now: products and orders. In django, an app is an independent module, and in that app we will keep everything related to that. It can be thought of as a independent service, rather than an 'app' as we commonly use the term. So after setting up our project, lets create an app like this:

python manage.py startapp products
python manage.py startapp orders 

For now we will focus only on the products app. Lets open up that app in our favourite code editor (I like to use SublimeText). Inside of our products app, there should be python file models.py. Before we proceed any further, lets ask what is a model, and what is this models.py all about. The answer to that is not that difficult. The core of any application is data. Now what a relational database allows us to do is to organize, manage that data in a secure, fault tolerant manner. Unfortunately, that data alone is not enough to build an app. We need to perform some logic, something that the app eventually does, lets say in our case with all the items we have we can place an order and notify a seller, send mails to customers, we can apply promotions, discounts and the like. So we need a way to represent the data in the database. That is where the model comes in. A model is like an ORM. In the simplest terms, it means that database entities can be represented as python classes, and each row as an object. And on that objects we can do all sorts of manipulations. We'll see all that in a minute. And all of that we do in our models.py file.

So lets jump into models.py. The way we make our own models or python representation of entities in a RDBMS is by python classes. So here I have defined a class that will represent a product object:

from django.db import models

# Create your models here.
class Products(models.Model):
    title = models.CharField(max_length=120)
    description = models.TextField()

So what have we done here? For our products app, we have defined model. So all product objects will have a title, and a description. We also need to register our model, so that is visible in our app. For that we need to include it in admin.py.

from django.contrib import admin

# Register your models here.
from products.models import Products

We also need to add this app in our main settings.py in the INSTALLED_APPS portion to tell django that we want to use this app.

So now what we need to do is bring our database upto sync with this new entity that we have defined. For that we need to run database migrations. We can just go and run python manage.py makemigrations, and we should see something like this:

Migrations for 'products':
  products/migrations/0001_initial.py
    - Create model Products

Then lets apply these migrations by doing python manage.py migrate. We get something like this:

Operations to perform:
  Apply all migrations: admin, auth, contenttypes, products, sessions
Running migrations:
  Applying products.0001_initial... OK

So what happened here? You may have noticed when we ran makemigrations, a new file was created inside your products app, in a folder called migrations. This file basically tells us the differences between the current configuration in the database and what we have in our models. In our case, we added the new entity in our models, and it was absent from the database. We then applied those changes and synced the database with our model. Note that these changes weren’t actually applied until we hit migrate. Lets open up that file which was created while doing makemigrations.

from django.db import migrations, models


class Migration(migrations.Migration):

    initial = True

    dependencies = [
    ]

    operations = [
        migrations.CreateModel(
            name='Products',
            fields=[
                ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
                ('title', models.CharField(max_length=120)),
                ('description', models.TextField()),
            ],
        ),
    ]

See that operations list up there? That tells django what are the migration operations to be done. In ths case, it need to create a model in the database. This new model is having the fields I gave in models.py, and something additional called an AutoField. This is a special kind of field that was added automatically by django, and is used as a unique identifier. If we want to change it, we just make the change in the migrations file and re-run migrations. But we’ll leave that for another day :)

Ok so all that talk is fine, but what difference did it make to our app. Lets find out:
I have my appserver running, so lets quickly check out the admin app once. We can access it at localhost:8000/admin. Once you log in, here’s what it should look like:

Lets try to create to an object from the admin app:

Note how the fields in the form are based off of the field types we defined in the model. We can play around with loads of other field types. I’d like to mention here that the django documentation is fantastic and just about anything you will need to build a production ready app can be found in the docs. I will refer the docs very frequently, and add a few more fields to our model. Meanwhile here’s an object I created from the console. (There are other ways to do that too, more on that later)

So after adding some more fields, my model looks like this:

from django.db import models

# Create your models here.
class Products(models.Model):
    PRODUCT_CATEGORIES=[
        ('CLOTH', 'Clothing'),
        ('HOME', 'Home'),
        ('STAT', 'Stationery'),
    ]
    title = models.CharField(max_length=120, verbose_name='Product Title')
    description = models.TextField(blank=False, verbose_name='Product Description')
    price = models.DecimalField(max_digits=9, decimal_places=2, verbose_name='Price in INR')
    size = models.PositiveIntegerField(verbose_name='Size')
    product_category = models.CharField(max_length=40, verbose_name='Product Category', choices=PRODUCT_CATEGORIES)
    create_ts = models.DateTimeField(auto_now_add=False)

Lets look into these new fields:
Well the first one is our plain old title, except now we have added a few arguments or options to the field. We already had a max_length, no points for guessing what that does. The verbose_name is a name of the field we would like to give. It is optional, if we don’t specify that, django will automatically take the field name itself (see the migration file). Take a look at the two previous screenshots: in the first one I did not use a verbose_name, the second one I did. Also, we specified that the title field will be a CharField with limited capacity, as opposed to a TextField, which we used for the description. In the description, notice we have used blank=False. This means that this field will be required. Conversely, blank=True means we will allow that field to be blank.
So the next one is the price, and an obvious choice was the DecimalField. I have placed some more options, max_digits and decimal_places. The first one specifies the number of digits in the double decimal itself, and the second the number of digits after the decimal. In my example, we can have 7 digits before the decimal and 2 after, so 9999999.99 is the maximum value we can store in the price field. (What are we selling, diamonds?!). I have also used a PostitiveIntegerField for storing the size of an item.
Now suppose for a field, we want the values to be limited from a set of values, and we can choose only from those. Of course, we can do that in the front-end, or in the form itself, but we can also do it in the model. For that we just have to pass a list of allowed values into the field as the option choices. Notice here for the list of tuples I have passed, the first item in the tuple will be stored in the database, and the second one will be displayed in the admin app (or any other model form).
The DateTimeField is used to store a date-time timestamp. Here I have used it as a timestamp for when my object was first created. The auto_now_add is automatically stamped with the current time when the object first get created. Even if it is edited, this field remains the same. This makes it different from its cousin auto_now, which gets updated.
Ok, so now lets sync-up our database by running makemigrations and then migrate

You are trying to add a non-nullable field 'create_ts' to products without a default; we can't do that (the database needs something to populate existing rows).
Please select a fix:
 1) Provide a one-off default now (will be set on all existing rows with a null value for this column)
 2) Quit, and let me add a default in models.py
Select an option: 

Ok, so now what's that about? Well you see we want to be absolutely sure we enforce all the constraints on all rows, and django won't allow us to apply these migrations if even a single row has data which violates them. I had one object in my database from my previous migration, just having two fields. So now django is giving me the option to either provide a defalult for the coloumn, or to fix things in my model. We'll fix the model as best as we can. So I changed the following fields in models.py:

create_ts = models.DateTimeField(default=timezone.now, editable=False)

The editable_false will ensure that this field wont show up in the admin or any other form, and cant be edited. But even after this we will get the same error, for the other fields now. This time I decided to give a default value like this:

Select an option: 1
Please enter the default value now, as valid Python
The datetime and django.utils.timezone modules are available, so you can do e.g. timezone.now
Type 'exit' to exit this prompt
>>>

I gave True for all, as I had just the one item in my database, and migrations file was created.

Migrations for 'products':
  products/migrations/0003_auto_20190810_2048.py
    - Add field create_ts to products
    - Add field price to products
    - Add field product_category to products
    - Add field size to products

I applied the migrations. Meanwhile if we have a look at the migrations file we can see some AddField, and AlterFields on the new fields and the title respectively. Now our form should look like this (notice create_ts is absent):

Lets take a quick peek at our database (notice how the first item has default values):

So that's pretty cool, right? But there's plenty more we can do, and the docs are extremely helpful so don't stop here.
Now ORMs also include some API's to query the database so you dont have to run actual queries to retrieve or update data. What's the point of representing database data as python objects if you cant use them? This is where QuerySet APIs come in. QuerySets allow us to run all operations on our database though instance of our model class. Let me show you what I mean. Lets run: python manage.py shell, and we are presented with a usual python shell. The special thing about this is we are in our virtual environment, and all our django apps are available here. Lets try out this:

Python 3.7.1 (default, Dec 14 2018, 19:28:38) 
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from products.models import Products
>>> print(Products.objects.all())
<QuerySet [<Products: Products object (2)>, <Products: Products object (3)>, <Products: Products object (4)>, <Products: Products object (5)>]>
>>> 

Whoa! What did we do here? Well, we just imported our products app (products folder had an __init__.py, so its a module) and we imported our model class Product. Products.objects.all() is a QuerySet API that we called, and it returned a list of all objects from the database currently. Kinda like running

SELECT * FROM TABLE_NAME

You can loosely translate it as "get all objects(rows) of the model(entity)". So now this is what we will do to perform CRUD operations on our database. Now that we have our data as python objects, we can do all sorts of logic on them, render it out as a template to the front-end, perform manipulations on data, etc. Each item in the database is nothing but a python object, and to access any attribute of that row, all we do is treat it as we would a regular python object i.e.: object.attribute. Let me show what I mean:

Python 3.7.1 (default, Dec 14 2018, 19:28:38) 
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from products.models import Products
>>> all_products = Products.objects.all()
>>> for product in all_products:
...     print(product.id, "--", product.title, "--", product.price, "--", product.product_category)
... 
2 -- Sample Item -- 1.00 -- True
3 -- ABCD Item -- 300.00 -- CLOTH
4 -- XYZ Items -- 100.00 -- HOME
5 -- PQRST -- 33.00 -- STAT
>>> 

See, a database row is no different from a instance of the model class, and the field names are its attributes. We can do all the stuff you'd do on a normal SQL database using QuerySets. For example:

>>> all_products = Products.objects.filter(product_category='HOME')
>>> for product in all_products:
...     print(product.id, "--", product.title, "--", product.price, "--", product.product_category)
... 
4 -- XYZ Items -- 100.00 -- HOME

is equivalent to:

Hell, you can even create, delete, update objects (or is it rows, hard to tell :P) with these APIs and this is what we do in apps. For instance (pun intended), we could use the last loop example to render out a product listing page in a real e-commerce website. Like I said, the docs are fantastic.

Go crazy, build something to improve your life, or others life!