Introduction and Motivation
In recent times, I needed to incorporate some private (one-on-one) chatting functionalities into an existing application whose backend was written in Python (Django) so that authenticated users can privately chat themselves. The application's frontend was in React but for this tutorial, we will be using SvelteKit — whose version 1.0 was recently released — to interact with the websocket-powered backend using django channels. The chats will be persisted in the database so that users' chatting histories will not be lost.
Tech Stack
As briefly pointed out in the introduction, we'll be using:
- Python(v3.10), Django(v4.1.2) and Channels(v4.0.0) at the Backend;
- SvelteKit(v1.0.1), Bootstrap(v5) and Fontawesome(v6.2.0) at the frontend.
Assumption and Objectives
It is assumed that you are familiar with Django — not necessarily django channels — any JavaScript-based modern frontend web framework or library and some TypeScript.
In the course of this series of tutorials, you will learn about:
- performing CRUD operations in a websocket-powered django application;
- working with websocket in SvelteKit — a performant frontend framework.
Source code
This tutorial's source code can be accessed here:
Sirneij
/
chatting
Full-stack private chatting application built using Django, Django Channels, and SvelteKit
chatting
chatting
is a full-stack private chatting application which uses modern technologies such as Python
— Django
and Django channels
— and TypeScript/JavaScript
— SvelteKit
. Its real-time feature utilizes WebSocket
.
recording.mp4
chatting
has backend
and frontend
directories. Contrary to its name, backend
is a classic full-fledged application, not only backend code. Though not refined yet, you can chat and enjoy real-time conversations there as well. frontend
does what it implies. It houses all user-facing codes, written using SvelteKit
and TypeScript
.
Run locally
To locally run the app, clone this repository and then open two terminals. In one terminal, change directory to backend
and in the other, to frontend
. For the frontend
terminal, you can run the development server using npm run dev
:
╭─[Johns-MacBook-Pro] as sirneij in ~/Documents/Devs/chatting/frontend using node v18.11.0 21:37:36
╰──➤ npm run dev
In the backend
terminal, create and activate a virtual…
Implementation
Step 1: Install and setup django, other dependencies and app
As a first step, create a folder, mine was chatting
, that will house the entire project, both frontend and backend. Then, change directory into the newly created folder and create a sub-folder to house the app's backend. I called mine backend
. Following that, make and activate a virtual environment and then install django and channels. Thereafter, create a django project. I used chattingbackend
as my project's name. Include channels
or daphne
in your project's setting's INSTALLED_APPS
. Ensure you create a new app, I used chat
as app's name, and include it there as well. Don't forget to install and link corsheaders so that frontend requests will be accepted by our backend server. See updated settings.py
file. Channels uses asgi
and as a result, we must set it up. Make your chattingbackend/asgi.py
look like:
# chattingbackend / asgi.py
...
import os
from channels.auth import AuthMiddlewareStack
from channels.routing import ProtocolTypeRouter, URLRouter
from channels.security.websocket import AllowedHostsOriginValidator
from django.core.asgi import get_asgi_application
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'chattingbackend.settings')
# Initialize Django ASGI application early to ensure the AppRegistry
# is populated before importing code that may import ORM models.
django_asgi_app = get_asgi_application()
import chat.routing
application = ProtocolTypeRouter(
{
'http': django_asgi_app,
'websocket': AllowedHostsOriginValidator(AuthMiddlewareStack(URLRouter(chat.routing.websocket_urlpatterns))),
}
)
Refer to channels' docs for explanations. This code requires that we have a routing.py
file in the created chat
application. Ensure it's created as well. It's content should be:
# chat / routing.py
from django.urls import path
from chat import consumers
websocket_urlpatterns = [
path('ws/chat/<int:id>/', consumers.ChatConsumer.as_asgi()),
]
It's just like the normal urls.py
file of Django apps where we link views.py
logic to a route. In channels, views.py
is equivalent to consumers.py
. You can now create the file, consumers.py
, and for now, make it look like:
# chat / consumers.py
import json
from channels.generic.websocket import WebsocketConsumer
class ChatConsumer(WebsocketConsumer):
def connect(self):
self.accept()
def disconnect(self, close_code):
pass
def receive(self, text_data):
text_data_json = json.loads(text_data)
message = text_data_json["message"]
self.send(text_data=json.dumps({"message": message}))
Don't worry about the details yet. We can now link this asgi.py
file to our project's settings.py
:
# chattingbackend / settings.py
...
WSGI_APPLICATION = 'chattingbackend.wsgi.application'
ASGI_APPLICATION = 'chattingbackend.asgi.application'
...
You can now launch the project in your terminal using python manage.py runserver
. If everything goes well, you should see something like:
Watching for file changes with StatReloader
Performing system checks...
System check identified no issues (0 silenced).
You have 18 unapplied migration(s). Your project may not work properly until you apply the migrations for app(s): admin, auth, contenttypes, sessions.
Run 'python manage.py migrate' to apply them.
December 31, 2022 - 10:20:28
Django version 4.1, using settings 'chattingbackend.settings'
Starting ASGI/Daphne version 3.0.2 development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
The project is now served by daphne
.
Step 2: Create Message
model and write consumers.py
logic
Since we need to persist users' chats in the database, let's create a model for it:
# chat / models.py
from django.contrib.auth import get_user_model
from django.db import models
class Message(models.Model):
sender = models.ForeignKey(get_user_model(), on_delete=models.CASCADE, null=True, blank=True)
message = models.TextField(null=True, blank=True)
thread_name = models.CharField(null=True, blank=True, max_length=200)
timestamp = models.DateTimeField(auto_now_add=True)
def __str__(self) -> str:
return f'{self.sender.username}-{self.thread_name}' if self.sender else f'{self.message}-{self.thread_name}'
Apart from the automatic id
django adds to the model's field, we only have four more fields: sender
which stores the user who sends the message, message
is the chat content, thread_name
stores the name of the private room so that it'll be easy to filter messages by its room name, and timestamp
stores when the message gets created. Pretty simple. No hassles!
Let's write our consumers.py
logic now:
# chat / consumers.py
import json
from channels.db import database_sync_to_async
from channels.generic.websocket import AsyncWebsocketConsumer
from django.contrib.auth import get_user_model
from api.utils import CustomSerializer
from chat.models import Message
class ChatConsumer(AsyncWebsocketConsumer):
async def connect(self):
current_user_id = self.scope['user'].id if self.scope['user'].id else int(self.scope['query_string'])
other_user_id = self.scope['url_route']['kwargs']['id']
self.room_name = (
f'{current_user_id}_{other_user_id}'
if int(current_user_id) > int(other_user_id)
else f'{other_user_id}_{current_user_id}'
)
self.room_group_name = f'chat_{self.room_name}'
await self.channel_layer.group_add(self.room_group_name, self.channel_name)
await self.accept()
# await self.send(text_data=self.room_group_name)
async def disconnect(self, close_code):
await self.channel_layer.group_discard(self.room_group_name, self.channel_layer)
await self.disconnect(close_code)
async def receive(self, text_data=None, bytes_data=None):
data = json.loads(text_data)
message = data['message']
sender_username = data['senderUsername'].replace('"', '')
sender = await self.get_user(sender_username.replace('"', ''))
await self.save_message(sender=sender, message=message, thread_name=self.room_group_name)
messages = await self.get_messages()
await self.channel_layer.group_send(
self.room_group_name,
{
'type': 'chat_message',
'message': message,
'senderUsername': sender_username,
'messages': messages,
},
)
async def chat_message(self, event):
message = event['message']
username = event['senderUsername']
messages = event['messages']
await self.send(
text_data=json.dumps(
{
'message': message,
'senderUsername': username,
'messages': messages,
}
)
)
@database_sync_to_async
def get_user(self, username):
return get_user_model().objects.filter(username=username).first()
@database_sync_to_async
def get_messages(self):
custom_serializers = CustomSerializer()
messages = custom_serializers.serialize(
Message.objects.select_related().filter(thread_name=self.room_group_name),
fields=(
'sender__pk',
'sender__username',
'sender__last_name',
'sender__first_name',
'sender__email',
'sender__last_login',
'sender__is_staff',
'sender__is_active',
'sender__date_joined',
'sender__is_superuser',
'message',
'thread_name',
'timestamp',
),
)
return messages
@database_sync_to_async
def save_message(self, sender, message, thread_name):
Message.objects.create(sender=sender, message=message, thread_name=thread_name)
That's a lot! But let's go through it. First, instead of using the initial WebsocketConsumer
, we preferred its AsyncWebsocketConsumer
counterpart which allows us to use python's async/await
syntax. This can be more performant. Next, we have three main methods: connect
, disconnect
and receive
. They are the basic requirements for any Generic Consumers
. If you are using JsonWebsocketConsumer or its async
counterpart, your receive
method will be receive_json
and some other internal methods may change such as self.send_json
instead of self.send
. This is because you have opted to auto-encode and decode all incoming and outgoing contents as JSON. Using WebsocketConsumer
or AsyncWebsocketConsumer
doesn't pose that specificity.
These methods do what their names connote. connect
gets fired as soon as a connection from the frontend is established. Hence, in most cases, all preliminary set ups such as getting request user, creating room name etc, should be done in the method. While disconnect
does the exact opposite — it gets fired as soon as a connection from the frontend is disconnected. As for receive
, it gets called whenever subsequent requests are made after connection. Here, we might handle saving data to the database and performing other logical operations.
Focusing on each method, in connect
we tried to get current user's id
by checking self.scope
for user
key. Because we used AuthMiddleware
which depends on SessionMiddleware
, the consumer's scope
will be populated by each session's user object. If you are using session-based authentication, you just need to use self.scope['user'].id
to get the user id. However, since you might be using token-based authentication system, scope
might not have the user object. To get around this, you might roll out your own AuthMiddleware
or, for simplicity sake, I chose to opt for including current user's id via query_string
so that my websocket URL will look like /path?query_string
. Then, we get the id of the second user in the chatting room. Using these data, we formulated the room_name
and then room_group_name
. The technique for getting room_name
was purely optional. After that, we used our channel_layer
whose database is redis, to add our room. Then the connection was accepted. These lines:
# chat / consumers.py
...
await self.channel_layer.group_add(self.room_group_name, self.channel_name)
await self.accept()
...
are almost always important in the connect
method. These lines are what we reversed in the disconnect
method. You can do much more than those in the disconnect
method depending on your app's complexity.
Now the receive
method! It handles the logic of sending and receiving messages as well as saving them in the database. First, we turned text_data
— text_data
houses all the text
data accompanying a request — into JSON. We then extract the message content and sender's username from it. Using the username, we fetched the sender from the database via the get_user
method. This method has database_sync_to_async
decorator. It is mandatory to use this decorator whenever you use async
consumer and need to interact with the database. Instead of using database_sync_to_async
as a decorator, you are allowed to use it inline:
...
sender = await database_sync_to_async(self.get_user)(sender_username.replace('"', ''))
...
After that, we saved the message into the database using the retrieved data, retrieved all messages from the database, and then send them back to the frontend using the magic method chat_message
. It's a common practice to have such magic methods and they are specified using the "type"
object of group_send
. Check the explanation under the consumer created here for more clarity. All objects included in the dictionary can be accessed in the event
argument of the method "type"
points to. In the method, we finally send these data to the frontend.
In get_messages
method, we decided to write a custom serializer which depends on Django's core serializer so that a model's related fields' fields can also be serialized. The custom serializer looks like:
# api / utils.py
from django.core.serializers.json import Serializer
# FYI: It can be any of the following as well:
# from django.core.serializers.pyyaml import Serializer
# from django.core.serializers.python import Serializer
# from django.core.serializers.json import Serializer
JSON_ALLOWED_OBJECTS = (dict, list, tuple, str, int, bool)
class CustomSerializer(Serializer):
def end_object(self, obj):
for field in self.selected_fields:
if field == 'pk':
continue
elif field in self._current.keys():
continue
else:
try:
if '__' in field:
fields = field.split('__')
value = obj
for f in fields:
value = getattr(value, f)
if value != obj and isinstance(value, JSON_ALLOWED_OBJECTS) or value == None:
self._current[field] = value
except AttributeError:
pass
super(CustomSerializer, self).end_object(obj)
It allows you to specify the related field's fields you need to serialize using __
(double underscore).
With these, we are done with the chatting logic.
In the next part, we will focus on developing the application's frontend using SvelteKit and TypeScript.
NOTE: There are other backend stuff excluded from this article that are part of the complete code. They were intentionally exluded as those are basic django stuff which I assumed you are already aware of and can implement if need be.
Outro
Enjoyed this article? Consider contacting me for a job, something worthwhile or buying a coffee ☕. You can also connect with/follow me on LinkedIn. It isn't bad if you help share it for wider coverage. I will appreciate...
Top comments (0)