Introduction and Motivation

In recent times, I needed to incorporate some private (one-on-one) chatting functionalities into an existing application whose backend was written in Python (Django) so that authenticated users can privately chat themselves. The application's frontend was in React but for this tutorial, we will be using SvelteKit — whose version 1.0 was recently released — to interact with the websocket-powered backend using django channels. The chats will be persisted in the database so that users' chatting histories will not be lost.

Tech Stack

As briefly pointed out in the introduction, we'll be using:

• Python(v3.10), Django(v4.1.2) and Channels(v4.0.0) at the Backend;
• SvelteKit(v1.0.1), Bootstrap(v5) and Fontawesome(v6.2.0) at the frontend.

Assumption and Objectives

It is assumed that you are familiar with Django — not necessarily django channels — any JavaScript-based modern frontend web framework or library and some TypeScript.

In the course of this series of tutorials, you will learn about:

• performing CRUD operations in a websocket-powered django application;
• working with websocket in SvelteKit — a performant frontend framework.

Source code

This tutorial's source code can be accessed here:

╭─[Johns-MacBook-Pro] as sirneij in ~/Documents/Devs/chatting/frontend using node v18.11.0                                21:37:36
╰──➤ npm run dev

Implementation

Step 1: Install and setup django, other dependencies and app

As a first step, create a folder, mine was chatting, that will house the entire project, both frontend and backend. Then, change directory into the newly created folder and create a sub-folder to house the app's backend. I called mine backend. Following that, make and activate a virtual environment and then install django and channels. Thereafter, create a django project. I used chattingbackend as my project's name. Include channels or daphne in your project's setting's INSTALLED_APPS. Ensure you create a new app, I used chat as app's name, and include it there as well. Don't forget to install and link corsheaders so that frontend requests will be accepted by our backend server. See updated settings.py file. Channels uses asgi and as a result, we must set it up. Make your chattingbackend/asgi.py look like:

# chattingbackend / asgi.py
...
import os

from channels.auth import AuthMiddlewareStack
from channels.routing import ProtocolTypeRouter, URLRouter
from channels.security.websocket import AllowedHostsOriginValidator
from django.core.asgi import get_asgi_application

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'chattingbackend.settings')
# Initialize Django ASGI application early to ensure the AppRegistry
# is populated before importing code that may import ORM models.
django_asgi_app = get_asgi_application()

import chat.routing

application = ProtocolTypeRouter(
    {
        'http': django_asgi_app,
        'websocket': AllowedHostsOriginValidator(AuthMiddlewareStack(URLRouter(chat.routing.websocket_urlpatterns))),
    }
)

Refer to channels' docs for explanations. This code requires that we have a routing.py file in the created chat application. Ensure it's created as well. It's content should be:

# chat / routing.py
from django.urls import path

from chat import consumers

websocket_urlpatterns = [
    path('ws/chat/<int:id>/', consumers.ChatConsumer.as_asgi()),
]

It's just like the normal urls.py file of Django apps where we link views.py logic to a route. In channels, views.py is equivalent to consumers.py. You can now create the file, consumers.py, and for now, make it look like:

# chat / consumers.py
import json

from channels.generic.websocket import WebsocketConsumer


class ChatConsumer(WebsocketConsumer):
    def connect(self):
        self.accept()

    def disconnect(self, close_code):
        pass

    def receive(self, text_data):
        text_data_json = json.loads(text_data)
        message = text_data_json["message"]

        self.send(text_data=json.dumps({"message": message}))

Don't worry about the details yet. We can now link this asgi.py file to our project's settings.py:

# chattingbackend / settings.py
...
WSGI_APPLICATION = 'chattingbackend.wsgi.application'
ASGI_APPLICATION = 'chattingbackend.asgi.application'
...

You can now launch the project in your terminal using python manage.py runserver. If everything goes well, you should see something like:

Watching for file changes with StatReloader
Performing system checks...

System check identified no issues (0 silenced).

You have 18 unapplied migration(s). Your project may not work properly until you apply the migrations for app(s): admin, auth, contenttypes, sessions.
Run 'python manage.py migrate' to apply them.
December 31, 2022 - 10:20:28
Django version 4.1, using settings 'chattingbackend.settings'
Starting ASGI/Daphne version 3.0.2 development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

The project is now served by daphne.

Step 2: Create Message model and write consumers.py logic

Since we need to persist users' chats in the database, let's create a model for it:

# chat / models.py

from django.contrib.auth import get_user_model
from django.db import models


class Message(models.Model):
    sender = models.ForeignKey(get_user_model(), on_delete=models.CASCADE, null=True, blank=True)
    message = models.TextField(null=True, blank=True)
    thread_name = models.CharField(null=True, blank=True, max_length=200)
    timestamp = models.DateTimeField(auto_now_add=True)

    def __str__(self) -> str:
        return f'{self.sender.username}-{self.thread_name}' if self.sender else f'{self.message}-{self.thread_name}'

Apart from the automatic id django adds to the model's field, we only have four more fields: sender which stores the user who sends the message, message is the chat content, thread_name stores the name of the private room so that it'll be easy to filter messages by its room name, and timestamp stores when the message gets created. Pretty simple. No hassles!

Let's write our consumers.py logic now:

# chat / consumers.py

import json

from channels.db import database_sync_to_async
from channels.generic.websocket import AsyncWebsocketConsumer
from django.contrib.auth import get_user_model

from api.utils import CustomSerializer
from chat.models import Message


class ChatConsumer(AsyncWebsocketConsumer):
    async def connect(self):
        current_user_id = self.scope['user'].id if self.scope['user'].id else int(self.scope['query_string'])
        other_user_id = self.scope['url_route']['kwargs']['id']
        self.room_name = (
            f'{current_user_id}_{other_user_id}'
            if int(current_user_id) > int(other_user_id)
            else f'{other_user_id}_{current_user_id}'
        )
        self.room_group_name = f'chat_{self.room_name}'
        await self.channel_layer.group_add(self.room_group_name, self.channel_name)
        await self.accept()
        # await self.send(text_data=self.room_group_name)

    async def disconnect(self, close_code):
        await self.channel_layer.group_discard(self.room_group_name, self.channel_layer)
        await self.disconnect(close_code)

    async def receive(self, text_data=None, bytes_data=None):
        data = json.loads(text_data)
        message = data['message']
        sender_username = data['senderUsername'].replace('"', '')
        sender = await self.get_user(sender_username.replace('"', ''))

        await self.save_message(sender=sender, message=message, thread_name=self.room_group_name)

        messages = await self.get_messages()

        await self.channel_layer.group_send(
            self.room_group_name,
            {
                'type': 'chat_message',
                'message': message,
                'senderUsername': sender_username,
                'messages': messages,
            },
        )

    async def chat_message(self, event):
        message = event['message']
        username = event['senderUsername']
        messages = event['messages']

        await self.send(
            text_data=json.dumps(
                {
                    'message': message,
                    'senderUsername': username,
                    'messages': messages,
                }
            )
        )

    @database_sync_to_async
    def get_user(self, username):
        return get_user_model().objects.filter(username=username).first()

    @database_sync_to_async
    def get_messages(self):
        custom_serializers = CustomSerializer()
        messages = custom_serializers.serialize(
            Message.objects.select_related().filter(thread_name=self.room_group_name),
            fields=(
                'sender__pk',
                'sender__username',
                'sender__last_name',
                'sender__first_name',
                'sender__email',
                'sender__last_login',
                'sender__is_staff',
                'sender__is_active',
                'sender__date_joined',
                'sender__is_superuser',
                'message',
                'thread_name',
                'timestamp',
            ),
        )
        return messages

    @database_sync_to_async
    def save_message(self, sender, message, thread_name):
        Message.objects.create(sender=sender, message=message, thread_name=thread_name)

That's a lot! But let's go through it. First, instead of using the initial WebsocketConsumer, we preferred its AsyncWebsocketConsumer counterpart which allows us to use python's async/await syntax. This can be more performant. Next, we have three main methods: connect, disconnect and receive. They are the basic requirements for any Generic Consumers. If you are using JsonWebsocketConsumer or its async counterpart, your receive method will be receive_json and some other internal methods may change such as self.send_json instead of self.send. This is because you have opted to auto-encode and decode all incoming and outgoing contents as JSON. Using WebsocketConsumer or AsyncWebsocketConsumer doesn't pose that specificity.

These methods do what their names connote. connect gets fired as soon as a connection from the frontend is established. Hence, in most cases, all preliminary set ups such as getting request user, creating room name etc, should be done in the method. While disconnect does the exact opposite — it gets fired as soon as a connection from the frontend is disconnected. As for receive, it gets called whenever subsequent requests are made after connection. Here, we might handle saving data to the database and performing other logical operations.

Focusing on each method, in connect we tried to get current user's id by checking self.scope for user key. Because we used AuthMiddleware which depends on SessionMiddleware, the consumer's scope will be populated by each session's user object. If you are using session-based authentication, you just need to use self.scope['user'].id to get the user id. However, since you might be using token-based authentication system, scope might not have the user object. To get around this, you might roll out your own AuthMiddleware or, for simplicity sake, I chose to opt for including current user's id via query_string so that my websocket URL will look like /path?query_string. Then, we get the id of the second user in the chatting room. Using these data, we formulated the room_name and then room_group_name. The technique for getting room_name was purely optional. After that, we used our channel_layer whose database is redis, to add our room. Then the connection was accepted. These lines:

# chat / consumers.py
...
await self.channel_layer.group_add(self.room_group_name, self.channel_name)
await self.accept()
...

are almost always important in the connect method. These lines are what we reversed in the disconnect method. You can do much more than those in the disconnect method depending on your app's complexity.

Now the receive method! It handles the logic of sending and receiving messages as well as saving them in the database. First, we turned text_data — text_data houses all the text data accompanying a request — into JSON. We then extract the message content and sender's username from it. Using the username, we fetched the sender from the database via the get_user method. This method has database_sync_to_async decorator. It is mandatory to use this decorator whenever you use async consumer and need to interact with the database. Instead of using database_sync_to_async as a decorator, you are allowed to use it inline:

...
sender = await database_sync_to_async(self.get_user)(sender_username.replace('"', ''))
...

After that, we saved the message into the database using the retrieved data, retrieved all messages from the database, and then send them back to the frontend using the magic method chat_message. It's a common practice to have such magic methods and they are specified using the "type" object of group_send. Check the explanation under the consumer created here for more clarity. All objects included in the dictionary can be accessed in the event argument of the method "type" points to. In the method, we finally send these data to the frontend.

In get_messages method, we decided to write a custom serializer which depends on Django's core serializer so that a model's related fields' fields can also be serialized. The custom serializer looks like:

# api / utils.py
from django.core.serializers.json import Serializer

# FYI: It can be any of the following as well:
# from django.core.serializers.pyyaml import Serializer
# from django.core.serializers.python import Serializer
# from django.core.serializers.json import Serializer

JSON_ALLOWED_OBJECTS = (dict, list, tuple, str, int, bool)


class CustomSerializer(Serializer):
    def end_object(self, obj):
        for field in self.selected_fields:
            if field == 'pk':
                continue
            elif field in self._current.keys():
                continue
            else:
                try:
                    if '__' in field:
                        fields = field.split('__')
                        value = obj
                        for f in fields:
                            value = getattr(value, f)
                        if value != obj and isinstance(value, JSON_ALLOWED_OBJECTS) or value == None:
                            self._current[field] = value

                except AttributeError:
                    pass
        super(CustomSerializer, self).end_object(obj)

It allows you to specify the related field's fields you need to serialize using __ (double underscore).

With these, we are done with the chatting logic.

In the next part, we will focus on developing the application's frontend using SvelteKit and TypeScript.

NOTE: There are other backend stuff excluded from this article that are part of the complete code. They were intentionally exluded as those are basic django stuff which I assumed you are already aware of and can implement if need be.

Outro

Enjoyed this article? Consider contacting me for a job, something worthwhile or buying a coffee ☕. You can also connect with/follow me on LinkedIn. It isn't bad if you help share it for wider coverage. I will appreciate...