A Starter Guide to Building Real-time Applications with Node.js

In a world where the value of time is steadily increasing, building applications that users can interact with in real-time has become a norm for most of the developers. Most of the applications we see today, whether they are mobile, desktop, or web applications, have at least a single real-time feature included. As an example, real-time messaging and notifications are two of the most commonly used real-time features used in applications.

In this article, we are introducing you to the development of real-time applications using Node.js. In fact, Node is one of the best programming languages out there to build real-time applications due to its event-driven and asynchronous nature. Before diving into building a real-time application head-first, we’ll see what kind of real-time applications we can build using Node.js.

If you want to know more other areas where Node.js excels or simply falls short, read my article When You Should and Shouldn’t Use Node.js for Your Project.

Where are Real-time Applications Used?

As I mentioned above, messaging and notification delivery are two of the most common use cases for real-time applications. But we can use real-time applications for a myriad of other purposes. Let’s see what they are.

Real-time Messaging

Most of us are familiar with the use of real-time messaging applications, especially in mobile devices, in the form of Whatsapp, Facebook Messenger, and numerous other messaging applications. However, real-time messaging is used not limited to purely messaging applications. We see real-time messaging features in on-demand taxi apps, delivery apps, and collaborative platforms.

Real-time Notification Delivery

Enabling real-time notifications has proved to be a game-changer when it comes to increasing user engagement with applications. For this reason, you would hardly see a modern application that does not deliver notifications in real-time to its users.

Live Streaming

Live streams that users can interact with in real-time are becoming more and more popular after social media platforms integrated live video streams to their applications. Instagram and Facebook live video streaming features are the best examples of this.

Real-time Tracking

With the introduction of popular taxi and delivery applications, like Uber and Amazon, tracking the progress of users’ taxi rides or deliveries in real-time has become an essential requirement. Their real-time progress updates increase the usability and reliability of these applications.

IoT Devices

Real-time features are essential for IoT devices. Data captured by the sensors placed in IoT devices are transmitted, processed, and displayed to the end-users with a minimum delay. Since most of the inputs captured by these devices, like temperature and lighting, constantly change with the time, applications working with IoT devices should be able to receive and send data in real-time.

How can We Build Real-time Applications?

Is building a real-time application different from building a normal web application? The answer is, yes.

Think of a messaging application where users can send messages in real-time. These messages should appear on the other users’ application as soon as the messages are sent. If we implement this application like a normal web application, where only the client can initiate requests to the server to receive data, the user has to either refresh the web page regularly to see the newest messages or the client-side should send AJAX requests to the server in short time intervals to retrieve the newest messages. The former of the two is not very user friendly and the latter is a waste of application resources. Then, clearly, we must have a different method to build real-time applications that makes better sense.

WebSocket provides the solution we need. WebSocket is a communication protocol that allows both the client and server to initiate communication. In other words, with WebSocket, the server can send data to the client any time without the client having to request data first. In the case of the previous messaging application, we can use WebSockets to instantly send messages to all the users through the server. We can use the WebSocket API to communicate using WebSockets when building applications.

Socket.io

However, when implementing a real-time application using Node, we don’t have to directly use the WebSocket API. Instead, Javascript and Node.js library Socket.io, which is an API to the WebSocket API, provides a much simpler implementation of WebSockets for us to use. In this tutorial, we will be using Socket.io to create and manage WebSocket connections between the client and the server.

Building a Real-time Chatroom with Node.js

Now that we have covered the background on real-time application development, we can start creating our own real-time application. In this tutorial, we are going to build a simple chatroom that users can use to communicate with other connected users. Any number of users can connect to the chatroom and the messages one user sends become instantly visible to all the users connected to the chatroom.

Our simple chatroom is going to have the following set of features.

• Change the username of the user
• Send messages
• Show if another user is currently typing a message

Cool, now that we have our requirements, let’s start building up the environment and setting up the structure

Setting up Application Environment

First, create a new directory for the application. Then, run the npm init to set up the package.json file. Make sure that, at this step you assign app.js as your main script, if you didn’t , don’t worry, you can always change it in your package.json at a later point.

Install dependencies

In this tutorial, we are using the express, ejs, socket.io, and nodemon packages to build the application.

• Ejs is a popular JS template engine
• We discussed the use of socket.io earlier
• Nodemon is a package that restarts the server every time we make a change to the application code. It eliminates the need to manually stop and start the server every time we make a change. Unlike the other packages, we install nodemon as a development dependency since we use it only for development purposes.

Install express, ejs, and socket.io using the following command.

npm install express ejs socket.io --save

Install nodemon as a development dependency using this command.

npm install nodemon --save-dev

To start the application with nodemon, we should add a start script to our package.json file.

"scripts": {
    "start": "nodemon app.js",
 },

Then, we can start the application by running the following command on the command-line.

npm run start

If it fails, don’t worry, it’s basically because we don’t have any code file yet.

Set up the application structure

With all the dependencies which will need for this project installed, let’s build app the project structure. For that you’ll need to create a few directories and for now, one file called app.js. Let’s get that done so that your app structure looks as follows:

|--app.js
|--views
|--node_modules
|--package.json
|--public
   |--css
   |--js

I think the structure is pretty clear, but let’s quickly go over it:

• app.js: file we will use to host our server-side code
• views: folder containing the views (ejs)
• node_modules: where we installed our dependencies
• package.json npm configuration file
• public: directory we will use to store our assets, like css files, javascript files (for the client side), and images.

First steps building the server

The first we need to do before we even consider doing the real-time connections is to get express up and running, for that, let’s open our app.js file and paste the following code:

const express = require('express')
const socketio = require('socket.io')
const app = express()

app.set('view engine', 'ejs')
app.use(express.static('public'))

app.get('/', (req, res)=> {
    res.render('index')
})

const server = app.listen(process.env.PORT || 3000, () => {
    console.log("server is running")
})

Once we have express configured and using ejs as template system, we can start working on the sockets.io initialization. For that add the following code at the end of your app.js file.

//initialize socket for the server
const io = socketio(server)

io.on('connection', socket => {
    console.log("New user connected")
})

The code is pretty straight forward, we are initializing socket.io from our server connection (express) and then we set up an even using io.on() which will be triggered every time a new connection to the socket gets established.

If you now run your server with npm start you will be able to receive new socket connections. So let’s start building our front-end.

Building our front-end

We won’t spend much time making our front-end look amazing, but we will explain how the connection to the server works, how to emit and capture socket events and we will apply all of that into our chat example.

Let’s start by creating a template into our views folder, for that create a index.ejs file and paste the following code:

<!DOCTYPE html>
<head>
    <title>Simple realtime chatroom</title>
    <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.5.0/css/bootstrap.min.css">
</head>
<body>
    <div class="container">
        <div class="title">
            <h3>Realtime Chat Room</h3>
        </div>

        <div class="card">
            <div class="card-header">Anonymous</div>
            <div class="card-body">
                <div class="input-group">
                    <input type="text" class="form-control" id="username" placeholder="Change your username" >
                    <div class="input-group-append">
                        <button class="btn btn-warning" type="button" id="usernameBtn">Change</button>
                    </div>
                </div>
            </div>
            <div class="message-box">
                <ul class="list-group list-group-flush" id="message-list"></ul>
                <div class="info"></div>
            </div>

            <div class="card-footer">
                <div class="input-group">
                    <input type="text" class="form-control" id="message" placeholder="Send new message" >
                    <div class="input-group-append">
                        <button class="btn btn-success" type="button" id="messageBtn">Send</button>
                    </div>
                </div>
            </div>
        </div>

    </div>
    <script src="https://cdnjs.cloudflare.com/ajax/libs/socket.io/2.0.4/socket.io.js"></script>
    <script src="/js/chatroom.js"></script>
</body>
</html>

Note how we have included the script of the client-side socket.io library and the custom javascript file we are going to use in this code.

<script src="https://cdnjs.cloudflare.com/ajax/libs/socket.io/2.0.4/socket.io.js"></script>
<script src="/js/chatroom.js"></script>

We also have a button with ID messageBtn to send a new message and another button with ID usernameBtn to submit a new username. Username and message inputs have IDs username and message respectively. All user messages are expected to appear inside the unordered list with the ID message-list. If a user is tying a message, that information will appear inside the div with class info.

If you open our browser and you head to http://localhost:3000/ your app will look something like this:

But it’s not doing anything, the buttons won’t work and will be pretty much a static application. So next let’s start connecting the front-end to the server.

For that create a new Javascript file named chatroom.js inside the js folder (note in the HTML above, that I’m already referencing to this file) of the public directory. Inside the Javascript file, we need to connect to socket from the front-end. We can do it like this.

(function connect(){
    let socket = io.connect('http://localhost:3000')
})()

Visit your website again and in your terminal (on the server-side) you will see something like:

Awesome! your app is already working, though it doesn’t do much. Let’s build functionality next

Changing the user name

The default username we use for every connection is “Anonymous”. We give the users the option to change this username. We’ll set up the back-end to change the username when the front-end emits a change_username event. Go back to your server-side code (app.js) and edit your connection event to add new code.

io.on('connection', socket => {
    console.log("New user connected")

    socket.username = "Anonymous"

    socket.on('change_username', data => {
        socket.username = data.username
    })
})

Next, we need to adjust our front-end, so that when we press the change username button, it emits an event to the server with the name change_username. See how we built the name by emitting and capturing the same event name?

Inside chatroom.js, we are going to add an event listener to usernameBtn to emit a change_username event when the button is clicked.

(function connect(){
    let socket = io.connect('http://localhost:3000')

    let username = document.querySelector('#username')
    let usernameBtn = document.querySelector('#usernameBtn')
    let curUsername = document.querySelector('.card-header')

    usernameBtn.addEventListener('click', e => {
        console.log(username.value)
        socket.emit('change_username', {username: username.value})
        curUsername.textContent = username.value
        username.value = ''
    })
})()

Now if you reload the web page and submit a new username, you will see your current username changed to the new one. Next, let’s start sending messages.

Sending Messages

The next feature we are going to implement is sending messages. Here things start to get a little bit different, so far we said that every time the front-end emits a message the server will receive it, however in our new case, the front-end needs to emit a new_message event, which then will need to be sent to all the connected clients, so that they can print the new message.

First, we will set up the front-end to emit a new_message event when a new message is submitted. Since the client-side should also be configured to receive new messages other users send from the server, the application should also listen to receive_message events on the front-end and show the new message on the web page appropriately. We can achieve both these tasks using the following code which goes inside the previous connect function in chatroom.js.

let message = document.querySelector('#message')
let messageBtn = document.querySelector('#messageBtn')
let messageList = document.querySelector('#message-list')

messageBtn.addEventListener('click', e => {
    console.log(message.value)
    socket.emit('new_message', {message: message.value})
    message.value = ''
})

socket.on('receive_message', data => {
    console.log(data)
    let listItem = document.createElement('li')
    listItem.textContent = data.username + ': ' + data.message
    listItem.classList.add('list-group-item')
    messageList.appendChild(listItem)
})

Every time the receive_message event happens on the client side, we change our DOM to render the message into the screen.

On the back-end side, when we receive a new_message event we need to emit a new event to all the clients, for that we use io.sockets.emit() function. Change your connection event in your app.js file as follows:

io.on('connection', socket => {
    console.log("New user connected")

    socket.username = "Anonymous"

    socket.on('change_username', data => {
        socket.username = data.username
    })

    //handle the new message event
    socket.on('new_message', data => {
        console.log("new message")
        io.sockets.emit('receive_message', {message: data.message, username: socket.username})
    })

})

When handling the new_message event, the server itself emits a receive_message event to the connected clients with data about the new message. This event is received by all the users connected to the server, including the one who sent the message, so that the new message is displayed on their chatroom interfaces.

If you now open your web app in your browser (you can have multiple instances) you can start chatting (with yourself? :p)

You can connect to the chatroom using two separate browsers and play around with the feature of sending messages, and see how the messages one user sends instantly appear on both users’ application interfaces.

I’m Typing….

In most real-time messaging apps we use today, we see a simple text that says “user X is typing…” whenever another user is typing a message. This gives the application a more real-time feeling and improves user experience. We are going to add this feature to our application.

First, let’s consider the front-end implementation. We add a new event listener to the message input box to emit a typing event whenever a keypress occurs. SInce keypresses on the message input box indicate that the user is typing a message, the typing event tells the server that the user is typing a message. The client-side also listens to typing events emitted by the server to know whether another user is currently typing a message and show it on the user interface.

Again, inside the connect function in chatroom.js, we add the following code.

let info = document.querySelector('.info')

message.addEventListener('keypress', e => {
    socket.emit('typing')
})

socket.on('typing', data => {
    info.textContent = data.username + " is typing..."
    setTimeout(() => {info.textContent=''}, 5000)
})

If one user is typing a message, other users are shown the text “<!-- raw HTML omitted --> is typing…” for 5 seconds.

Now we need to set up the back-end to handle typing events. The code we use here is this.

socket.on('typing', data => {
    socket.broadcast.emit('typing', {username: socket.username})
})

Here, socket.io uses the broadcast function to notify the connected clients. When we use broadcast, every user except the one who is typing the message receives the typing event from the server. So, every user except the one typing the message is shown the text “<!-- raw HTML omitted --> is typing…”.

Again, you can connect to the chatroom from two browsers and see how this works in real-time.

Awesome!

Summary

Today, using real-time features with desktop, mobile, and web applications has almost become a necessity. In this article, we covered a number of applications of real-time apps and learned how to create a real-time chatroom with the help of Node.js and Socket.io. To continue from here, you can either try to improve this chatroom by adding more features and using a database to persist older messages or implement another real-time application that has a different use case.

Thanks for reading!

If you like the story, please don't forget to subscribe to our free newsletter so we can stay connected: https://livecodestream.dev/subscribe