Build a ChatBot Using Python, Django

A ChatBot has become one of the must-have features of modern-day web applications. It plays a huge role in customer service which was once traditionally held by human beings. Through ChatBot, you can automatically generate a response to a user’s input by making use of different machine-learning techniques.

Note:- If you encounter any issues throughout the tutorial, you can check out the code in the GitHub repository.

Table of Contents

1. Prerequisite
2. Project Configuration
3. WebSockets Using Django Channels
4. Tailwind CSS Configuration
5. Frontend - Create a WebSocket Connection
6. Enable a Channel Layer
7. Backend - WebSocket Consumers
8. ChatterBot - Generate Automated Response
9. ChatterBot Training
10. Offload Automated Response Generation to Celery
11. Conclusion

Prerequisite

This guide assumes that you have intermediate-level Django knowledge.

Project Configuration

To implement the ChatBot, the following technologies will be used:

• ChatterBot Library - To generate automated responses to a user’s input.
• Celery - Getting the automated response from a machine-learning dialog engine is going to take a while. So Celery will be used to perform this task in the background.
• WebSockets using Django Channels - To send to the client the automated response generated by the machine learning model immediately when it’s available.
• Redis - Will be used as a message broker and result backend for Celery. In addition, it will be used as a channel layer for WebSocket communication.
• TailwindCSS - To create the user interface.

We will dive into the details of each of them, but first, let’s set up the project. I have created a starter Docker template with all the configurations so that we can move forward with the development without wasting time.

Create a folder on your local machine > cd into it > clone the template:

mkdir chatbot
cd chatbot
git clone https://github.com/earthcomfy/blog-template .

This will generate the following folders and files inside your project:

There isn’t much going on except a basic Django setup in Docker with web, db, redis, and celery services. Refer to this article for more information on how to set up a Django project inside Docker. If you don’t want to use Docker, you can also follow along by setting up a virtual environment and running the necessary services on your machine.

In the root directory of your project, there is a file named .env.template, rename it to .env and update the environment variables accordingly.

Great, now build the image and spin up the containers by running:

docker-compose up --build

Go to http://0.0.0.0:8000/ to check if everything worked correctly.

Great, everything is working. Now close Docker by pressing ctrl + c and create a Django app named chat

docker-compose run --rm web python manage.py startapp chat

Add it to the list of installed apps inside settings/base.py

# settings/base.py

INSTALLED_APPS = [
    "django.contrib.admin",
    "django.contrib.auth",
    "django.contrib.contenttypes",
    "django.contrib.sessions",
    "django.contrib.messages",
    "django.contrib.staticfiles",
    # Local apps
    "chat"
]

Hurray! Let’s move on to the next section where the fun begins.

WebSockets Using Django Channels

It goes without saying that in a typical Django project the client makes an HTTP request > Django calls the view that is responsible for managing this request and returns a response back to the client.

This communication is pretty standard. However, ever since Django introduced ASGI and started supporting it natively, writing Django applications with asynchronous code has become possible. ASGI not only allows you to run asynchronous apps, but it also makes it possible to work with more advanced protocols like WebSockets.

Django Channels, which is built on ASGI, goes beyond HTTP and support other protocols such as WebSockets. The underlying implementation of Django Channels is very similar to regular HTTP views.

That being said, the following packages are needed to set up channels in a project:

• channels
• daphne

Both of them were added to the requirements.txt file so they must have already been installed when you built your Docker image.

Daphne is an HTTP, HTTP2, and WebSocket protocol server for ASGI, and was developed to power Django Channels. This server needs to be added to the list of installed apps so that it can take over the runserver management command.

# settings/base.py

INSTALLED_APPS = [
    "daphne", # here
    "django.contrib.admin",
    "django.contrib.auth",
    "django.contrib.contenttypes",
    "django.contrib.sessions",
    "django.contrib.messages",
    "django.contrib.staticfiles",
    # Local apps
    "chat"
]

Note:- Please be wary of any other third-party apps that require an overloaded or replacement runserver command. Daphne provides a separate runserver command and may conflict with it. In order to solve such issues, make sure daphne is at the top of your INSTALLED_APPS

Next, go to your project's asgi.py file and adjust it as follows to wrap the Django ASGI application:

# asgi.py

import os

from channels.auth import AuthMiddlewareStack
from channels.routing import ProtocolTypeRouter, URLRouter
from channels.security.websocket import AllowedHostsOriginValidator
from decouple import config
from django.core.asgi import get_asgi_application

os.environ.setdefault("DJANGO_SETTINGS_MODULE", config("DJANGO_SETTINGS_MODULE"))

# Initialize Django ASGI application early to ensure the AppRegistry
# is populated before importing code that may import ORM models.
django_asgi_app = get_asgi_application()

application = ProtocolTypeRouter(
    {
        "http": django_asgi_app,
        # We will add WebSocket protocol later. For now, it's just HTTP.
    }
)

Finally, set your ASGI_APPLICATION setting to point to that routing object as your root application:

# settings/base.py

ASGI_APPLICATION = "config.asgi.application"

Perfect! From this point onward, the daphne development server will be run. To ensure this, fire up the containers by running:

docker-compose up

Notice the ASGI/Daphne version 4.0.0 development server

Alright, this application is going to require a single view where the user and the ChatBot can interact. For this, let’s first set up Tailwind CSS in the project.

Tailwind CSS Configuration

There are many ways to set up Tailwind CSS in Django. One way to do so is by using a package called django-tailwind. This package provides an easy way to use Tailwind CSS in a Django project.

This package was included in requirements.txt so it must have already been installed when you created the Docker image.

After installation, add tailwind to the list of installed apps in the settings:

# settings/base.py

INSTALLED_APPS = [
    "daphne",
    "django.contrib.admin",
    "django.contrib.auth",
    "django.contrib.contenttypes",
    "django.contrib.sessions",
    "django.contrib.messages",
    "django.contrib.staticfiles",
    # Third-party apps
    "tailwind",
    # Local apps
    "chat",
]

Now, Create a TailwindCSS-compatible Django app. When prompted for the app name you can give it any name or proceed with the default theme

docker-compose run --rm web python manage.py tailwind init

Add the app - theme to the list of installed apps in the settings:

# settings/base.py

INSTALLED_APPS = [
    # ...
    # Local apps
    "chat",
    "theme",
]

Register the generated app theme by adding it to the settings as follows:

# settings/base.py

# Tailwind
TAILWIND_APP_NAME = "theme"

Install Tailwind CSS dependencies by running the following command:

docker-compose run --rm web python manage.py tailwind install

Perfect! Now, head over to theme/templates/base.html This file will be used as a base template to include Tailwind’s style sheet in all the other templates. Let’s modify it as follows:

{% comment %} theme/templates/base.html {% endcomment %}

{% load static tailwind_tags %}
<!DOCTYPE html>
<html lang="en">
  <head>
    <title>{% block title %}Django Chatbot{% endblock %}</title>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <meta http-equiv="X-UA-Compatible" content="ie=edge" />
    {% tailwind_css %}
  </head>

  <body>
    {% block body %} {% endblock %}
  </body>
  {% block scripts%}{% endblock %}
</html>

Then, head over to the chat app created earlier and create a templates directory. Within the templates directory, create another directory named chat, and within that create a file named chat.html

Put the following snippet inside the file:

{% comment %} chat/templates/chat/chat.html {% endcomment %}

{% extends 'base.html' %} {% block body %}
<div class="p-6 w-[800px]">
  <h1 class="text-3xl tracking-tight font-light" id="chat-header"></h1>
  <div
    id="chat-log"
    class="mt-4 w-full relative p-6 overflow-y-auto h-[30rem] bg-gray-50 border border-gray-200"
  ></div>
  <div class="mt-4">
    <input
      id="chat-message-input"
      class="py-2 outline-none bg-gray-50 border border-gray-300 text-gray-900 text-sm focus:border-blue-500"
      type="text"
      placeholder="Write your message here."
    />
    <button
      id="chat-message-submit"
      class="py-2 px-4 ml-2 border border-transparent text-sm font-medium rounded-md text-white bg-blue-800 hover:bg-blue-900"
      type="submit"
    >
      Send
    </button>
  </div>
</div>
{% endblock %}

The above is a basic Django HTML file with a div where chat history will be displayed and an input box for the user’s input with the submit button.

Next, create a view that will render the above template:

# chat/views.py

from django.views.generic import TemplateView

class ChatView(TemplateView):
    template_name: str = "chat/chat.html"

Then, Create urls.py module inside the chat app and map the ChatView to the URL patterns.

# chat/urls.py

from django.urls import path
from .views import ChatView

app_name = "chat"

urlpatterns = [path("", ChatView.as_view(), name="chat_view")]

Finally, go to the project's urls.py file and include the chat app's URL:

# config/urls.py

from django.contrib import admin
from django.urls import include, path

urlpatterns = [
    path("admin/", admin.site.urls),
    path("", include("chat.urls", namespace="chat")),
]

Done! Now, make sure the styles are applied properly and that what we have so far is working.

Start tailwind:

docker-compose run web python manage.py tailwind start

In another terminal spin up the containers:

docker-compose run web python manage.py tailwind start

Go to http://localhost:8000/ and you will see the following 🎉

Note:- If the styles aren’t applied, make sure you have started tailwind and restart the containers.

Frontend - Create a WebSocket Connection

Inside chat.html, you need to write a script that will open a WebSocket connection to a Django Server and listen to the established connection to send and receive data.

Open the chat.html file and add the following script just under the body block i.e. {% endblock %}

{% block scripts %}
<script>
  var wss_protocol = window.location.protocol == "https:" ? "wss://" : "ws://";
  var chatSocket = new WebSocket(
    wss_protocol + window.location.host + "/ws/chat/"
  );
  var messages = [];

  chatSocket.onopen = function (e) {
    document.querySelector("#chat-header").innerHTML =
      "Welcome to Django Chatbot";
  };

  chatSocket.onmessage = function (e) {
    var data = JSON.parse(e.data);
    var message = data["text"];
    messages.push(message);

    var str = '<ul class="space-y-2">';
    messages.forEach(function (msg) {
      str += `<li class="flex ${
        msg.source == "bot" ? "justify-start" : "justify-end"
      }">
      <div class="relative max-w-xl px-4 py-2 rounded-lg shadow-md
        ${
          msg.source == "bot"
            ? "text-gray-700 bg-white border border-gray-200"
            : "bg-blue-600 text-white"
        }">
        <span className="block font-normal">${msg.msg}</span></div></li>`;
    });
    str += "</ul>";
    document.querySelector("#chat-log").innerHTML = str;
  };

  chatSocket.onclose = function (e) {
    alert("Socket closed unexpectedly, please reload the page.");
  };

  document.querySelector("#chat-message-input").focus();
  document.querySelector("#chat-message-input").onkeyup = function (e) {
    if (e.keyCode === 13) {
      // enter, return
      document.querySelector("#chat-message-submit").click();
    }
  };

  document.querySelector("#chat-message-submit").onclick = function (e) {
    var messageInputDom = document.querySelector("#chat-message-input");
    var message = messageInputDom.value;
    chatSocket.send(
      JSON.stringify({
        text: message,
      })
    );

    messageInputDom.value = "";
  };
</script>
{% endblock %}

• The above script creates a WebSocket connection to the /ws/chat/ path > listens to different events and manipulates the DOM.
• Depending on the source of the message (user or bot), different alignments and styles are applied (see under the chatsocket.onmessage)

If you run the server at this point, type in any message, and hit submit, the WebSocket connection will be opened but you will see an error in the JavaScript console because we don’t have a consumer that accepts WebSocket connections yet.

Enable a Channel Layer

A Channel layer provides an abstraction for multiple consumers to talk with each other and with other parts of Django. It is the middleman that passes messages from senders to receivers. The Channel layer that will be used in this guide is Redis. Aside from serving as a WebSocket communication, Redis can also be used as a:

• Message Broker and Result backend for Celery.
• Caching.

It is an all-in-one tool!

We have already configured a Redis service inside Docker. Django Channels also needs to know how to interface with Redis. This is provided by a package called channels_redis

channels_redis is an officially maintained channel layer that uses Redis as its backing store. It is already included in requirements.txt so it must have already been installed when you built your Docker image.

Next, add the Channel layer settings:

# settings/base.py

# Django Channels
CHANNEL_LAYERS = {
    "default": {
        "BACKEND": "channels_redis.core.RedisChannelLayer",
        "CONFIG": {
            "hosts": [config("REDIS_BACKEND")],
        },
    },
}

A channel layer has the following:

1) Channel - a mailbox where messages can be sent to. Each channel has a name. Anyone who has the name of a channel can send a message to the channel.

2) Group - a group of related channels. A group has a name. Anyone who has the name of a group can add/remove a channel to the group by name and send a message to all channels in the group.

Backend - WebSocket Consumers

When a WebSocket API opens a connection, Channels need to be informed where to look to accept and handle the connection. This is where Consumers come in. Consumers correspond to Views in normal Django HTTP requests.

When a message is sent from the client, it is received by the consumers listening to the group or channel on the other end.

Create a file named consumers.py inside the chat app and add the following:

# chat/consumers.py

import json

from asgiref.sync import async_to_sync
from channels.generic.websocket import WebsocketConsumer

class ChatConsumer(WebsocketConsumer):
    def receive(self, text_data):
        text_data_json = json.loads(text_data)

        # The consumer ChatConsumer is synchronous while the channel layer
        # methods are asynchronous. Therefore wrap the methods in async-to-sync
        async_to_sync(self.channel_layer.send)(
            self.channel_name,
            {
                "type": "chat_message",
                "text": {"msg": text_data_json["text"], "source": "user"},
            },
        )

        # We will later replace this call with a celery task that will
        # use a Python library called ChatterBot to generate an automated
        # response to a user's input.
        async_to_sync(self.channel_layer.send)(
            self.channel_name,
            {
                "type": "chat.message",
                "text": {"msg": "Bot says hello", "source": "bot"},
            },
        )

    # Handles the chat.mesage event i.e. receives messages from the channel layer
    # and sends it back to the client.
    def chat_message(self, event):
        text = event["text"]
        self.send(text_data=json.dumps({"text": text}))

• The above consumer accepts a WebSocket connection on the path /ws/chat/, takes a message, and sends a response to the client (for now the response is just a simple message “Bot says hello”).
• Any consumer has a self.channel_layer and self.channel_name attribute, which contains a pointer to the channel layer instance (Redis in our case) and the channel name respectively.

Similar to urls.py Channels uses a routing configuration to route to the consumer. Inside the chat app, create a file named routing.py and add the following:

# chat/routing.py

from django.urls import re_path

from . import consumers

websocket_urlpatterns = [
    re_path(r"ws/chat/$", consumers.ChatConsumer.as_asgi()),
]

Next, head over to asgi.py and modify it as follows:

# config/asgi.py

"""
ASGI config for config project.

It exposes the ASGI callable as a module-level variable named ``application``.

For more information on this file, see
https://docs.djangoproject.com/en/4.1/howto/deployment/asgi/
"""

import os

from channels.auth import AuthMiddlewareStack
from channels.routing import ProtocolTypeRouter, URLRouter
from channels.security.websocket import AllowedHostsOriginValidator
from django.core.asgi import get_asgi_application
from decouple import config

os.environ.setdefault("DJANGO_SETTINGS_MODULE", config("DJANGO_SETTINGS_MODULE"))

# Initialize Django ASGI application early to ensure the AppRegistry
# is populated before importing code that may import ORM models.
django_asgi_app = get_asgi_application()

import chat.routing

application = ProtocolTypeRouter(
    {
        "http": django_asgi_app,
        "websocket": AllowedHostsOriginValidator(
            AuthMiddlewareStack(URLRouter(chat.routing.websocket_urlpatterns))
        ),
    }
)

• The above code points the root routing configuration to the chat.routing module. This means when a connection is made to the development server (channel's development server), ProtocolTypeRouter checks whether it's a normal HTTP request or a WebSocket.
• If it's a WebSocket, AuthMiddlewareStack will take it from there and will populate the connection’s scope with a reference to the currently authenticated user, similar to how Django’s AuthenticationMiddleware populates the request object of a view function with the currently authenticated user.
• Next, URLRouter will route the connection to a particular consumer based on the provided URL patterns.

Now, if you head over to http://localhost:8000/ and type in any message, you will get a response that says “Bot says hello”

Great! Now, the response to a user’s input needs to be automated. Let’s do that in the next section using ChatterBot.

ChatterBot - Generate Automated Response

ChatterBot is a Python library that makes it easy to generate automated responses to a user’s input. To use this in your project, add it to the list of installed apps in the settings:

# settings/base.py

INSTALLED_APPS = [
    # ...
    # Third-party apps
    "tailwind",
    "chatterbot.ext.django_chatterbot",
    # Local apps
    "chat",
    "theme",
]

Under the hood, ChatterBot uses logic adapters to determine how to respond to a given input statement. You can have multiple logic adapters for your bot. Most of the logic adapters use the Naive Bayesian classification algorithm to determine if an input statement meets a particular set of criteria and to generate a response accordingly. The Naive Bayesian classification algorithm is based on conditional probability. Learn more about it here.

For a list of logic adapters, you can check this out. For the purpose of this tutorial, let’s use a single adapter called Best Match Adapter. As the name implies, the BestMatch logic adapter selects a response based on the best-known match to a given statement. Let’s add it to the settings:

# settings/base.py

# Chatterbot
CHATTERBOT = {
    "name": "User Support Bot",
    "logic_adapters": [
        "chatterbot.logic.BestMatch",
    ],
}

If you use multiple adapters, the response with the highest calculated confidence score will be considered a valid response to the input. The confidence score returned by each logic adapter determines how valid or close the response is to the output.

The following diagram from ChatterBot documentation provides a very nice flow of what goes behind the scenes.

ChatterBot Training

In the beginning, a ChatterBot instance is like a blank slate. This means that it starts off with an empty database. You fill it with statements (a single string of text representing something that can be said) and gradually it learns to reply to your input with higher accuracy. This training process is going to take some time. In order to simplify this process, ChatterBot provides multiple ways to initially train your bot. In this tutorial, you will learn how to train your bot using sample list data.

Let’s create a simple custom management command where we will pass in a list of statements. Head over to the chat app and create a management directory. Within the management directory, create another directory named commands, and within that create 2 files __init__.py and train.py

Put the following script inside train.py

# chat/management/commands/train.py

from django.core.management.base import BaseCommand
from chatterbot import ChatBot
from chatterbot.ext.django_chatterbot import settings
from chatterbot.trainers import ListTrainer

class Command(BaseCommand):
    help = "Training the chatbot"

    def handle(self, *args, **options):
        chatterbot = ChatBot(**settings.CHATTERBOT)
        trainer = ListTrainer(chatterbot)
        trainer.train(
            [
                "Hello",
                "Hi there!",
                "How are you doing?",
                "I'm doing great.",
                "That is good to hear",
                "Thank you.",
                "You're welcome.",
            ]
        )
        self.stdout.write(self.style.SUCCESS("Successfull!"))

Next, run the following command to load the example dialog into your project’s database.

docker-compose run --rm web python manage.py train

Now, create a superuser:

docker-compose exec web python manage.py createsuperuser

and go to the admin panel http://localhost:8000/admin/ Go to the statements tab and voila, you will see a list of texts with their responses.

Offload Automated Response Generation to Celery

Needless to say, generating a response from ChatterBot is time-intensive. This is where Celery comes to the rescue. Celery will pick up queued tasks and manages a separate server to run them in the background. As mentioned before, we will use Redis as a message broker. It is already set up in Docker so let’s proceed to defining the task.

Create a file named tasks.py inside the chat app and put the following Celery task:

# chat/tasks.py

from asgiref.sync import async_to_sync
from celery import shared_task
from channels.layers import get_channel_layer
from chatterbot import ChatBot
from chatterbot.ext.django_chatterbot import settings

channel_layer = get_channel_layer()

@shared_task
def get_response(channel_name, input_data):
    chatterbot = ChatBot(**settings.CHATTERBOT)
    response = chatterbot.get_response(input_data)
    response_data = response.serialize()

    async_to_sync(channel_layer.send)(
        channel_name,
        {
            "type": "chat.message",
            "text": {"msg": response_data["text"], "source": "bot"},
        },
    )

Finally, update consumers.py with the following content:

# chat/consumers.py

import json

from asgiref.sync import async_to_sync
from channels.generic.websocket import WebsocketConsumer

from .tasks import get_response

class ChatConsumer(WebsocketConsumer):
    def receive(self, text_data):
        text_data_json = json.loads(text_data)
        get_response.delay(self.channel_name, text_data_json)

        async_to_sync(self.channel_layer.send)(
            self.channel_name,
            {
                "type": "chat_message",
                "text": {"msg": text_data_json["text"], "source": "user"},
            },
        )

    def chat_message(self, event):
        text = event["text"]
        self.send(text_data=json.dumps({"text": text}))

You have to restart the container so that Celery can pick up the task and add it to its queue. Press ctrl + c and run:

docker-compose up

Go to http://localhost:8000/ and type in hello and you will get the response back 🚀

Conclusion

This tutorial is intended to be a starting place. The ChatterBot documentation is very concise so make sure to check it out to implement more features. For instance, you can train your data with corpus data. You can use multiple logic adapters or even create your own. In addition, check out Django Channel’s documentation to understand more about consumers, channel layers, and so on.

If you got lost somewhere throughout the guide, check out the project on GitHub

Happy coding! 🖤