Welcome back, In the previous article, we discussed about part one of our unit testing guide by setting up the environment for a NodeJS api with Express and TypeScript. We set up jest as a testing framework and supertest for providing high level of abstraction to test http requests.
In this second part of the guide, we are going to build the api and connect it to an online database, MongoDB Atlas and in the third and last part of the guide, we are going to write unit tests for each line of code after which we will conclude with a glimpse on life after unit testing, just like you have been thinking, intergration testing.
Here is the github repository for the first part of the guide.
If you don't have it on your local machine, run
git clone https://github.com/Abeinevincent/nodejs_unit_testing_guide.git .
inside your desired directory to get on the same track with us.
Connecting MongoDB to our NodeJS Backend
Developed by MongoDB Inc, MongoDB is a source-available, cross-platform, document-oriented database program, classified as a NoSQL database product. It utilizes JSON-like documents with optional schemas.
In this guide we will connect to a beginner friendly MongoDB atlas for simplicity. If you already have some experience with MongoDB, feel free to use any other MongoDB service such as MongoDB Campass.
In addition we will use mongoose to connect MongoDB to our NodeJS backend and query the database. It is JavaScript object-oriented programming library that creates a connection between MongoDB and the Node.js JavaScript runtime environment and provides a uniform API for accessing numerous different databases including MongoDB.
Steps
On the terminal run the following command:
npm install mongoose dotenv
Mongoose is an ODM for connecting our backend to MongoDB and querying the database, whereas dotenv is for helping us access our environment variables such as DB connection url in our NodeJS API.
I am using npm and if youre using yarn you may replace npm install with yarn add.
mongoose provides its own type definitions, so you don't need @types/mongoose installed.
Head over to MongoDB atlas and login or create an account to access the database. I recommend proceeding with a free cluster for now in development. In the databases section click on connect and grab the database url.
Inside the home directory, create a new file named .env and add the following code:
MONGODB_URL=YourConnectionUrl
To complete the connection, lets add the following code to our src/index.ts
import dotenv from "dotenv";
import mongoose from "mongoose";
dotenv.config()
in the imports section and
// DB CONNECTION
if (!process.env.MONGODB_URL) {
throw new Error("MONGODB_URL environment variable is not defined");
}
mongoose
.connect(process.env.MONGODB_URL)
.then(() => {
console.log("MongoDB connected to the backend successfully");
})
.catch((err) => console.log(err));
before starting the backend server
On the terminal run
npm run dev
to start the development server in development mode.
Output should be as shown below:
Overview
Before we start writing code, lets remind ourselves of our beloved folder structure which is the mostly for helping us stay on track by separating concerns.
First we have models for accomodating our database schema, then services for handling services logic forexample the userService.ts may handle user related operations like creating a new user among others, then controllers for handling specific routes or endpoints and the routes for defining routes of our application. As a plus we also have middlewares for accomodating middleware functions that can be used across multiple contollers or services. We also have helpers for accomodating utility functions or helper modules that assist in various tasks related to the services or contollers and __tests__ for holding our unit tests that we will write in the next article. Lastly we have utils for accomodating generic and reusable functions that can be used anywhere in the application. Most of the times, you will place here functions that do not belong to specific category but are instead generic.
If you are used to a different folder structure from this, dont worry, you will still understand the concepts very well, just follow along.
Writing DB Models
A model is blueprint for organizing and storing information in the database.
In this guide, our api will have only 2 models one for users other for products. We will have an ecommerce-like api in order to understand every concept in some use case and also know how to write a unit test for it.
We will write our models inside src/models.
Create a new file: src/models/User.ts and place the following code:
// Import necessary modules
import mongoose, { Schema, Document, Types } from "mongoose";
// Define the interface for User document
export interface IUser extends Document {
email: string;
username: string;
password: string;
isAdmin: boolean;
savedProducts: Types.ObjectId[];
}
// Create a schema for the User model
const userSchema: Schema<IUser> = new Schema({
email: { type: String, required: true },
username: { type: String, required: true },
password: { type: String, required: true },
isAdmin: { type: Boolean, default: false },
savedProducts: [{ type: mongoose.Schema.Types.ObjectId, ref: "Product" }],
}, {timestamps: true});
// Create and export the User model
export default mongoose.model<IUser>("User", userSchema);
The model has 5 fields as described.
Interface (IUser) defines the structure of a user document, specifying the types for each field.
Schema (userSchema) describes how the data should be organized for a user. Each field is defined with its type, and in the case of savedProducts, it's an array of MongoDB ObjectIds referencing the 'Product' model.
mongoose.model('User', userSchema) creates the Mongoose model named 'User' using the schema we defined.
Create another file: src/models/Product.ts and place the following code:
// Import necessary modules
import mongoose, { Schema, Document } from "mongoose";
// Define the interface for Product document
export interface IProduct extends Document {
title: string;
description: string;
image: string;
category: string;
quantity: string;
inStock: boolean;
}
// Create a schema for the Product model
const productSchema: Schema<IProduct> = new Schema(
{
title: { type: String, required: true },
description: { type: String, required: true },
image: { type: String, required: true },
category: { type: String, required: true },
quantity: { type: String, required: true },
inStock: { type: Boolean, default: false },
},
{ timestamps: true }
);
// Create and export the Product model
export default mongoose.model<IProduct>("Product", productSchema);
Writing Services
Create a new file: src/services/userService.ts and place the following code:
// Import necessary modules
import { generateToken } from "../utils/jwtUtils";
import User, { IUser } from "../models/User";
import { hashPassword } from "../utils/passwordUtils";
// Create a new user
export const createUser = async (userInput: IUser): Promise<IUser> => {
try {
// Hash the user's password before storing it
const hashedPassword = await hashPassword(userInput.password);
// Create the user with the hashed password
const newUser = await User.create({
...userInput,
password: hashedPassword,
});
return newUser;
} catch (error) {
throw new Error(`Error creating user: ${error.message}`);
}
};
// Login user
export const loginUser = async (
email: string,
password: string
): Promise<{ user: Omit<IUser, "password">; token: string }> => {
try {
// Find user by email
const user = await User.findOne({ email });
if (!user) {
throw new Error("User not found");
}
// Compare the provided password with the stored hashed password
const isPasswordValid = await comparePassword(password, user.password);
if (!isPasswordValid) {
throw new Error("Invalid password");
}
// Generate JWT token
const token = generateToken({
id: user._id,
username: user.username,
email: user.email,
isAdmin: user.isAdmin,
});
// Destructure password from the data returned
const { password: _password, ...userData } = user.toObject();
return { user: userData as Omit<IUser, "password">, token };
} catch (error) {
throw new Error(`Error logging in: ${error.message}`);
}
};
// Get all users
export const getAllUsers = async (): Promise<IUser[]> => {
try {
const users = await User.find();
return users;
} catch (error) {
throw new Error(`Error getting users: ${error.message}`);
}
};
// Get user by ID with his saved products
export const getUserById = async (userId: string): Promise<IUser | null> => {
try {
const user = await User.findById(userId).populate("savedProducts");
return user;
} catch (error) {
throw new Error(`Error getting user: ${error.message}`);
}
};
// Update user by ID
export const updateUser = async (
userId: string,
updatedUser: Partial<IUser>
): Promise<IUser | null> => {
try {
const user = await User.findByIdAndUpdate(userId, updatedUser, {
new: true,
});
return user;
} catch (error) {
throw new Error(`Error updating user: ${error.message}`);
}
};
// Delete user by ID
export const deleteUser = async (userId: string): Promise<void> => {
try {
await User.findByIdAndDelete(userId);
} catch (error) {
throw new Error(`Error deleting user: ${error.message}`);
}
};
We are handling auth in the same file containing other user services since our app is abit minimal but feel free to separate login and register/create user to a separate file say auth.ts.
In the section for creating a user, we have a function for hashing our password using bcrypt from src/utils/passwordUtils.ts whose code is as shown below:
import bcrypt from "bcrypt";
// Hash a password
export const hashPassword = async (password: string): Promise<string> => {
const saltRounds = 10;
const hashedPassword = await bcrypt.hash(password, saltRounds);
return hashedPassword;
};
// Compare a password with its hash
export const comparePassword = async (
password: string,
hashedPassword: string
): Promise<boolean> => {
return bcrypt.compare(password, hashedPassword);
};
Bcrypt with its type definitions for typescript can be installed using the following commands:
npm install bcrypt
and
npm install --save-dev @types/bcrypt
Lets move to the productsService,
create another file; src/services/productService.ts and put the following code:
// Import necessary modules
import Product, { IProduct } from "../models/Product";
// Create a new product
export const createProduct = async (
productInput: IProduct
): Promise<IProduct> => {
try {
const newProduct = await Product.create(productInput);
return newProduct;
} catch (error) {
throw new Error(`Error creating product: ${error.message}`);
}
};
// Get all products
export const getAllProducts = async (): Promise<IProduct[]> => {
try {
const products = await Product.find();
return products;
} catch (error) {
throw new Error(`Error getting products: ${error.message}`);
}
};
// Get product by ID
export const getProductById = async (
productId: string
): Promise<IProduct | null> => {
try {
const product = await Product.findById(productId);
return product;
} catch (error) {
throw new Error(`Error getting product: ${error.message}`);
}
};
// Update product by ID
export const updateProduct = async (
productId: string,
updatedProduct: Partial<IProduct>
): Promise<IProduct | null> => {
try {
const product = await Product.findByIdAndUpdate(productId, updatedProduct, {
new: true,
});
return product;
} catch (error) {
throw new Error(`Error updating product: ${error.message}`);
}
};
// Delete product by ID
export const deleteProduct = async (productId: string): Promise<void> => {
try {
await Product.findByIdAndDelete(productId);
} catch (error) {
throw new Error(`Error deleting product: ${error.message}`);
}
};
Writing Controllers
All set for our 2 services,lets proceed to controllers, starting with userController once again.
Create a new file: src/controllers/userController.ts and place the following code:
import { Request, Response } from 'express';
import * as UserService from '../services/userService';
// Create a new user
export const createUser = async (req: Request, res: Response): Promise<void> => {
try {
const newUser = await UserService.createUser(req.body);
res.status(201).json(newUser);
} catch (error) {
res.status(500).json({ error: error.message });
}
};
// Login user
export const loginUser = async (req: Request, res: Response): Promise<void> => {
const { email, password } = req.body;
try {
const { user, token } = await UserService.loginUser(email, password);
res.status(200).json({ user, token });
} catch (error) {
res.status(401).json({ error: error.message });
}
};
// Get all users
export const getAllUsers = async (_req: Request, res: Response): Promise<void> => {
try {
const users = await UserService.getAllUsers();
res.status(200).json(users);
} catch (error) {
res.status(500).json({ error: error.message });
}
};
// Get user by ID
export const getUserById = async (req: Request, res: Response): Promise<void> => {
try {
const user = await UserService.getUserById(req.params.userId);
if (user) {
res.status(200).json(user);
} else {
res.status(404).json({ error: 'User not found' });
}
} catch (error) {
res.status(500).json({ error: error.message });
}
};
// Update user by ID
export const updateUser = async (req: Request, res: Response): Promise<void> => {
try {
const updatedUser = await UserService.updateUser(req.params.userId, req.body);
if (updatedUser) {
res.status(200).json(updatedUser);
} else {
res.status(404).json({ error: 'User not found' });
}
} catch (error) {
res.status(500).json({ error: error.message });
}
};
// Delete user by ID
export const deleteUser = async (req: Request, res: Response): Promise<void> => {
try {
await UserService.deleteUser(req.params.userId);
res.status(204).send();
} catch (error) {
res.status(500).json({ error: error.message });
}
};
and then to productController,
lets create another file: src/controllers/productController.ts and place the following code:
import { Request, Response } from 'express';
import * as ProductService from '../services/productService';
// Create a new product
export const createProduct = async (req: Request, res: Response): Promise<void> => {
try {
const newProduct = await ProductService.createProduct(req.body);
res.status(201).json(newProduct);
} catch (error) {
res.status(500).json({ error: error.message });
}
};
// Get all products
export const getAllProducts = async (_req: Request, res: Response): Promise<void> => {
try {
const products = await ProductService.getAllProducts();
res.status(200).json(products);
} catch (error) {
res.status(500).json({ error: error.message });
}
};
// Get product by ID
export const getProductById = async (req: Request, res: Response): Promise<void> => {
try {
const product = await ProductService.getProductById(req.params.productId);
if (product) {
res.status(200).json(product);
} else {
res.status(404).json({ error: 'Product not found' });
}
} catch (error) {
res.status(500).json({ error: error.message });
}
};
// Update product by ID
export const updateProduct = async (req: Request, res: Response): Promise<void> => {
try {
const updatedProduct = await ProductService.updateProduct(req.params.productId, req.body);
if (updatedProduct) {
res.status(200).json(updatedProduct);
} else {
res.status(404).json({ error: 'Product not found' });
}
} catch (error) {
res.status(500).json({ error: error.message });
}
};
// Delete product by ID
export const deleteProduct = async (req: Request, res: Response): Promise<void> => {
try {
await ProductService.deleteProduct(req.params.productId);
res.status(204).send();
} catch (error) {
res.status(500).json({ error: error.message });
}
};
All set for our controllers too. We should be proceeding to the routes but we need to take a moment and think about how this api will be used permission wise, a case in point a user should not be able to delete a product, update a product, or delete his fellow users. We need a way of distributing permissions and protecting some sensitive endpoints from being hit by unauthorized or unauthenticated users. Realistically, some endpoints are not as vulnerable as others forexample, an endpoint that will be consuming the getAllProducts controller may be hit by any user, authenticated or not, because it only returns a list products, read only,and on the user interface,you may need to display these products even for guests before you prompt him to login just in case he gets interested in purchasing it.The endpoints he may need to hit after desiring to purchase may ofcourse be protected and require him to be authenticated and also have some specific previlages or permissions.
To achieve this we will use a library called jsonwebtoken.
It is a proposed Internet standard for creating data with optional signature and/or optional encryption whose payload holds JSON that asserts some number of claims. The tokens are signed either using a private secret or a public/private key. It is also a good way of securely transmitting information between parties because they can be signed, which means you can be certain that the senders are who they say they are. You can read more about json web token here
Open your terminal and run the following command
npm install jsonwebtoken
After installing the libray, we also need to install @types/jsonwebtoken for providing its typescript definitions:
On the same terminal, run
npm i --save-dev @types/jsonwebtoken
Lets get started writing some jwt logic,
Create another file: utils/jwtUtils.ts and place the following code:
// utils/jwtUtils.ts
import { Request, Response, NextFunction } from "express";
import jwt, { VerifyErrors } from "jsonwebtoken";
import { Types } from "mongoose";
// Generate jwt token
type JWTPayload = {
id: Types.ObjectId;
username: string;
email: string;
isAdmin: boolean;
};
// Custom Request type with 'user' property
interface CustomRequest extends Request {
user: {
id: Types.ObjectId;
username: string;
email: string;
isAdmin: boolean;
};
}
export const generateToken = (payload: JWTPayload): string => {
if (!process.env.JWT_SEC) {
throw new Error("JWT_SEC environment variable is not defined");
}
const token = jwt.sign(payload, process.env.JWT_SEC, {
expiresIn: process.env.JWT_EXPIRY_PERIOD,
});
return token;
};
// Verify token
export const verifyToken = (
req: CustomRequest,
res: Response,
next: NextFunction
): void => {
const authHeader = req.headers.token;
if (authHeader) {
const token = Array.isArray(authHeader)
? authHeader[0].split(" ")[1]
: authHeader.split(" ")[1];
if (!process.env.JWT_SEC) {
throw new Error("JWT_SEC environment variable is not defined");
}
jwt.verify(
token,
process.env.JWT_SEC,
(err: VerifyErrors | null, user: any) => {
if (err) return res.status(403).json("Token is not valid!");
req.user = user;
next();
}
);
} else {
res.status(401).json("You are not authenticated!");
}
};
// Authorize account owner
export const verifyTokenAndAuthorization = (
req: CustomRequest,
res: Response,
next: NextFunction
): void => {
verifyToken(req, res, () => {
if (req.user.id.toString() === req.params.id || req.user.isAdmin) {
next();
} else {
return res.status(403).json("You are not allowed to do that!");
}
});
};
// Authorize admin
export const verifyTokenAndAdmin = (
req: CustomRequest,
res: Response,
next: NextFunction
): void => {
verifyToken(req, res, () => {
if (req.user.isAdmin) {
next();
} else {
return res.status(403).json("You are not allowed to do that!");
}
});
};
We have all this code in src/utils/jwtUtils.ts because our app is abit minimal, feel free to separate each jwt middleware function in a separate file in the middlewares folder: src/middlewares, and pass appropriate props with their typescript types.
we are exporting four functions; one for creating our jwt token, generateToken, other for verifying our jwt, verifyToken, other for authorizing a user, account owner or admin, to do hit some endpoints such as updating his account detailslike username, password, among others, verifyTokenAndAuthorization and lastly for authorising only admins to be able to hit some endpoints such as deleting or updaing products, verifyTokenAndAdmin.
This way, youre able to protect some endpoints that may have sensitive data such that you either need them to be hit by an authenticated user or by a user with specific previlages in addition to being authorised.
The endpoints you may need one to hit when logged in regardless of his previlages will only require generateToken function.
Donot forget to add the environment variable we were using in .env file like this:
JWT_SEC=YourJWTSecret
JWT_EXPIRY_PERIOD=YourJWTExpiryTime e.g 15m (15 minutes)
On the expiry time of JWT, I recommend a period of time not more than 30 minutes for security purposes. However in development, you can set whatever you want to avoid being locked out frequently while coding. On the frontend, this token is decoded and the user activity tracked such that if say the user spends YourJWTExpiryTime with out actively using your app, he is logged out automatically.
The period should be in format: 15m for 15 minutes, 15s for 15 seconds, 15h for 15 hours, 15d for 15 days, 15w for 15 weeks, 15y for 15 years and so on.
Lets now proceed to the routes and get our jwt functions consumed too accordingly:
Writing Routes
Create a new file: src/routes/userRoute.ts and place the following code:
import express from "express";
import {
verifyTokenAndAuthorization,
verifyTokenAndAdmin,
} from "../utils/jwtUtils";
import {
getAllUsers,
updateUser,
deleteUser,
createUser,
loginUser,
} from "../controllers/userController";
const userRoutes = (router: express.Router) => {
// Route for creating a new user
router.post("/users/create", createUser);
// Route for logging in a user
router.post("/users/login", loginUser);
// Route for getting all users (only accessible by admin)
router.get("/users/all", verifyTokenAndAdmin, getAllUsers);
// Route for updating a user (protected, only account owner or admin)
router.put("/users/update/:id", verifyTokenAndAuthorization, updateUser);
// Route for deleting a user (protected, only accessible by admin)
router.delete("/users/delete/:id", verifyTokenAndAuthorization, deleteUser);
};
export default userRoutes;
and another one: src/routes/productRoute.ts and place the following code:
import express from "express";
import {
verifyToken,
verifyTokenAndAuthorization,
verifyTokenAndAdmin,
} from "../utils/jwtUtils";
import {
getAllProducts,
createProduct,
getProductById,
updateProduct,
deleteProduct,
} from "../controllers/productController";
const productRoutes = (router: express.Router) => {
// Route for getting all products
router.get("/products/all", getAllProducts);
// Route for creating a new product (protected, only accessible by admin)
router.post("/products/create", verifyTokenAndAdmin, createProduct);
// Route for getting a product by ID
router.get("/products/:productId", verifyToken, getProductById);
// Route for updating a product by ID (protected, only accessible by admin)
router.put("/products/update/:productId", verifyTokenAndAdmin, updateProduct);
// Route for deleting a product by ID (protected, only accessible by admin)
router.delete(
"/products/delete/:productId",
verifyTokenAndAdmin,
deleteProduct
);
};
export default productRoutes;
Some routes are protected as we discussed while writing jwt logic.
Lets finalise our api by connecting our routes to the index file inside routes and then the main index file.
Create a new file: src/routes/index.ts and place the following code:
import express from "express";
import userRoutes from "./userRoute";
import productRoutes from "./productRoute";
const router = express.Router();
export default (): express.Router => {
// USER
userRoutes(router);
// PRODUCT
productRoutes(router);
return router;
};
All set, lets headover again to the main index file: src/index.ts and add the following line of code after db connection and before starting the backend server:
// Serve other routes
app.use("/api/v1/", routes());
Don't forget to import the routes in the imports section like this:
import routes from "./routes";
Our api is now steady and running at PORT:8800, of course on your local machine 😄.
Conclusion
In this second part of the unit testing guide, we embarked on a journey to build a robust and scalable API using Node.js, Express, MongoDB, and TypeScript. We covered various aspects of the development process, from setting up the project structure to implementing user and product functionalities.
Recap of Achievements:
Project Structure:
We started by establishing a well-organized project structure, with dedicated folders for tests, models, controllers, services, routes, middlewares, helpers and utility functions. We may not have used some of the folders because we only have 2 schema we were using for illustration but I explained what each folder contains and you might use them all in a real world production grade API. This modular approach enhances maintainability and readability.
User Authentication:
Security is paramount in any application, and we implemented user authentication using JSON Web Tokens (JWT). We created middleware functions to verify tokens, authorize account owners, and validate administrator privileges.
Database Interaction:
Leveraging MongoDB Atlas as our database, we seamlessly integrated Mongoose as the ODM (Object Data Modeling) library. This allowed us to define schemas, models, and perform CRUD operations effortlessly.
User and Product Functionalities:
We implemented essential user functionalities, such as user registration, login, and profile updates. Additionally, we extended our API to handle product-related operations, including creation, retrieval, updating, and deletion.
Express Routes:
Our routes were thoughtfully designed to encapsulate functionality and maintain a clear separation of concerns. JWT authentication was selectively applied to secure some routes, ensuring that sensitive operations are only accessible by authorized and authenticated users.
What's Next?
As we conclude this phase, we eagerly anticipate the next chapter. In Part 3 of this guide, we will delve into the realm of testing. Comprehensive unit tests are crucial for ensuring the reliability and correctness of our API. Stay tuned as we explore the testing landscape and employ best practices to validate the functionality of our endpoints.
Thank you for reading to this far, your commitment to understanding the intricacies of development and security is commendable. Until next time, happy coding!
Checkout these helpful Links:
Github Repo Twitter
Top comments (0)