We all love design patterns, and we all wonder when it's best to use them, I'm going to use one of them to apply one business case that you might stumble upon in work. The pattern I'm talking about is "Publisher Subscriber".
Today I'm going to make a realtime API that updates all the connected clients to it whenever and actions takes place on the db, so a super admin user using a dashboard can instantly know if other admins have signed in or out without refreshing the page every couple of seconds, other case is instantly knowing that an order is received on the platform you are working on.
This tutorial, I'm going to use:
- NodeJS with Express for server side logic
- ReactJS to build a simple client app
- Socket.io for realtime connection between both sides
To follow along, you can write the code step by step as I'll cover most of it, or you can clone the two repos:
First lets setup our server we start by initializing the folder structure
npm init -y
then we add the packages we use, in this tutorial I'm going to use ES6 syntax in the backend so we need babel to bundle our code, beside some other libraries we will use later on.
npm add nodemon dotenv babel-loader
@babel/preset-env @babel/node @babel/core -D
these are devDependencies, that's why we use -D flag because we dont need them for more than development.
1.nodemon for hot running
2.dotenv for .env configuration
3.babel stuff for bundling
now for the heavy lifters
npm add express mongoose socket.io
1.express to setup our server
2.mongoose to connect to our mongodb
3.socket.io the one responsible for the realtime connection
now that was a bit boring, let's write some Javascript
index.js
import express from 'express'
import dotenv from 'dotenv'
dotenv.config()
const app = express()
app.get('/', (req,res)=>{
res.send('Hello')
})
const PORT = process.env.PORT || 5000;
app.listen(PORT, () => {
console.log(`Server up and running on port ${PORT}`);
})
before running this code you have to setup some configuration
.env
PORT=5000
MONGO_DB_URL=mongodb://localhost:27017
MONGO_DB_DBNAME=store
.babelrc
{
"presets": [
"@babel/preset-env"
]
}
package.json
....
"scripts": {
"start": "babel-node index.js",
"dev": "nodemon --exec babel-node index.js"
},
....
now when you type npm run dev, you will find the server up and running and if you type in your browser http://localhost:5000 you will get the following:
now let's make three folders and adjust our code as follows:
then for better environment variables handling
config/variables.js
import dotenv from 'dotenv'
dotenv.config()
const DB_URL = `${process.env.MONGO_DB_URL}/${process.env.MONGO_DB_DBNAME}`;
const PORT = process.env.PORT;
export {
DB_URL,
PORT
}
initialize and connect to database
config/db.js
import {DB_URL} from '../config/variables'
mongoose.connect(DB_URL, {
useNewUrlParser:true,
useUnifiedTopology:true
}, () => {
console.log(DB_URL);
console.log(`DB up and running`);
})
order model
models/order.js
import mongoose, {Schema} from 'mongoose'
const schema = new Schema({
customer:{
type:String,
required:true
},
price:{
type:Number,
required:true
},
address:{
type:String,
required:true
}
}, {
timestamps:true
})
const Order = mongoose.model('order', schema)
export default Order;
order controller
controllers/order.js
import express from 'express'
import Order from '../models/order'
import {io} from '../index'
const router = express.Router()
router.get('/', async (req, res) => {
try {
const orders = await Order.find()
res.send(orders)
} catch (error) {
res.send(error)
}
})
router.post('/', async (req, res) => {
try {
const order = new Order(req.body)
await order.save()
res.status(201).send(order)
} catch (error) {
res.send(error)
}
})
export default router
now the important part
index.js
import express from 'express'
import {PORT} from './config/variables'
import cors from 'cors'
import http from 'http'
// import { Server } from 'socket.io';
import socketIO from 'socket.io';
// import './config/sockets'
import './config/db'
import orderRouter from './controllers/order'
const app = express()
const server = http.createServer(app)
const io = socketIO(server, {
transports:['polling'],
cors:{
cors: {
origin: "http://localhost:3000"
}
}
})
io.on('connection', (socket) => {
console.log('A user is connected');
socket.on('message', (message) => {
console.log(`message from ${socket.id} : ${message}`);
})
socket.on('disconnect', () => {
console.log(`socket ${socket.id} disconnected`);
})
})
export {io};
app.use(express.json())
app.use(cors())
app.use('/orders', orderRouter)
app.get('/', (req,res) => {
res.send('Hello')
})
server.listen(PORT, () => {
console.log(`Server up and running on port ${PORT}`);
})
let me explain what happened here
the way we configure the server will differ when using socket.io because it deals with the server instance itself so
const server = http.createServer(app)
then we wrap it with io, allow some cors which will be the client side after a short while on port 3000
const io = socketIO(server, {
transports:['polling'],
cors:{
cors: {
origin: "http://localhost:3000"
}
}
})
configuring io and exporting it to be used in the order controller
io.on('connection', (socket) => {
console.log('A user is connected');
socket.on('message', (message) => {
console.log(`message from ${socket.id} : ${message}`);
})
socket.on('disconnect', () => {
console.log(`socket ${socket.id} disconnected`);
})
})
export {io};
then we go the order controller and change the code to
controllers/order.js
router.post('/', async (req, res) => {
try {
const order = new Order(req.body)
await order.save()
const orders = await Order.find()
io.emit('order-added', orders)
res.status(201).send(order)
} catch (error) {
res.send(error)
}
})
which means that whenever someone will add an order, it will be posted to all clients connected the socket, so will be updated instantly with the orders array in the db
Now we can go to the client side and consume this API, we use create-react-app because we don't need a complex app we just need to demonstrate the behavior
here, I made a simple ui components called Orders, for the code you can easily find it in the repo, but I'm interested in this part
const [orders, setOrders] = useState([])
useEffect(() => {
const getOrders = async () => {
const response = await axios.get('http://localhost:5000/orders')
const ordersData = response.data;
setOrders(ordersData)
}
getOrders()
}, [])
useEffect(() => {
const socket = io('ws://localhost:5000')
socket.on('connnection', () => {
console.log('connected to server');
})
socket.on('order-added', (newOrders) => {
setOrders(newOrders)
})
socket.on('message', (message) => {
console.log(message);
})
socket.on('disconnect', () => {
console.log('Socket disconnecting');
})
}, [])
first we have the state which is an empty array initially
the first useEffect call is a call to the get orders endpoint we have just made to get all orders and then we populate the view with it
the second useEffect call, we connect using socket.io-client which we will install on the client side using npm i socket.io-client, then we specify that on order-added event from the socket we will have the orders being sent with the event and set it to be the new array, so whenever a new order is added we will be notified with the new array of orders in the db.
to test it, I opened the browser on port 3000 to open my react app then used postman to make a post to my server on port 5000 to add an order and viola my react-app updated instantly
That was my first post, I hope you liked it.
Oldest comments (27)
Awesome, thanks for sharing!
Thanks!!
A really nice straighton example without bells and whistles.
Just the way i like it. Great work!
Thanks, It really means alot
Please share GitHub link
you can clone from here:
github.com/omar-diaa-48/real-time-...
github.com/omar-diaa-48/real-time-...
Great
Thanks
clear and easy
Thanks for saying that :)
Where can we use this api?
If you clone the 2 repos, you can run both apps and you should be able to use it, you can set what client can be connected to the server in the
originfield when declaring theioobject.It's a good breakdown, but I find some issues with it.
It has some bells and whistles, eg. you didn't really need to add the database though it's a nice touch, but it is missing the important bells and whistles, like HTTPS instead of HTTP [setting up an ssl with LetsEncrypt is pretty easy these days], also validating the data sent to your server, and logs for catching bots/anyone trying to abuse it, and also how do you validate communications are not being intercepted.
All in all you did a pretty good explanation and showed how server and client side are connected. This part always confuses newbies.
Granted some of these items require longer discussions and your tutorial is already quite long, they at least deserve to be mentioned, AT THE LEAST, HTTPS since SSL/TLS is a requirement not a luxury nowadays.
Thanks for notifying me, I will look further in these topics and might post another one focusing on those items
I would argue that SSL/TLS is not required for the scope of this tutorial. There are plenty other ones that show you how to add it to your nodejs application or using a proxy to do so.
Newbies don't know this. It should at least mention it and use an SSL secured url by the end. HTTPS is vital [mainly because you don't get that fancy lock icon and clients go apeshit if you don't have that lol], but in all seriousness it's not something that should be overlooked. At least link to another tutorial that shows how to do it for this specific stack since that can vary too.
Nice! Thanks for putting this out. You can also set the language for the code for better markdown code formatting
Sure, thanks for telling me
a good idea, but if my backend use another lanuage, how to do
it depends, some other frameworks like .NET uses SignalR, but I prefer Socket.io, any ways how to set it up will be very similar, just syntax differences but not concepts
Beautiful. I have created whole end to end application for truuth.id for their Liveness product.
I have used socket.io in both front end JS and back end with Python Flask. AWS Lambda, ECS, Farget, ALB and some more tools.
Perfectly working. Only glitches is AWS Socket API gateway does not support socket.io yet. As socket.io use both socket and http connection to make full duplex channel for stream and messages.
Any way will try your nodejs based this tutorial as well.
Wish you all the best. Thanks
I think it is great. Simple. For people that are relatively senior (like me) that know node and express and react well and simply want to know/see socket-io in the mix. Great.
Thanks for the article 👌
I thought in new version of socket.io we do not need to define the transport layer to be polling I guess, please correct me If I m wrong, because I saw the query params have the transport polling by default.
you can set it from the client to match the server, or change what the server is waiting for which is the default, I need to investigate the major differences between them, but I couldn't find much difference.
thanks man it helps a lot
Hi, I need help for the last section of using postman to make a post to the server on port 5000 to add an order. What is the HTTP link I need to use in postman and how is the JSON object structured to POST?