DEV Community

Jim Frenette
Jim Frenette

Posted on • Updated on • Originally published at jimfrenette.com

Hugo + Node.js Koa App Connected to MongoDB

#hugo #node #koa #mongodb

This project demonstrates how to create a development environment utilizing a Docker stack of Nginx to serve the static website, Nodejs for the api applications, MongoDB for the api data and Traefik for reverse proxy. This stack is suitable for deployment to staging and production environments.

Prerequisites

These products will need to be installed in order to complete this tutorial.

Project Setup

Create a directory for the entire project, e.g., hugo-koa-mongo. All of the project files will go in this folder. This folder will be referred to as the project root.

Hugo Static Website Generator

To get started, open a terminal in the project root and create a new Hugo site in a folder named www as follows.

hugo new site www
Enter fullscreen mode Exit fullscreen mode

Add a Theme

There are numerous themes available at themes.gohugo.io to choose from. You can install one of them if you prefer or use this example to install my hugo-starter theme. Download and extract the theme into the www/themes/starter folder, or use Git and clone the theme from it's git repository. For example,

git init
cd www
git submodule add https://github.com/jimfrenette/hugo-starter.git themes/starter
Enter fullscreen mode Exit fullscreen mode

After the theme has been installed, update the config.toml site configuration file to use the theme. For example,

config.toml 
theme = "starter"
Enter fullscreen mode Exit fullscreen mode

Preview the site on the hugo dev server

cd www

hugo server
Enter fullscreen mode Exit fullscreen mode

If the site loads, we're ready to move onto the next step.

MongoDB

We will spin up a MongoDB Docker container for the api database. To demonstrate, we need to populate it with some data. For this, I have exported tables from the Chinook database into csv files which can then be imported using mongoimport.

You can download the csv files within the source code for this project or complete the process on your own as follows.

  1. Download the Chinook_Sqlite.sqlite database.

  2. Open it with DB Browser for SQLite

  3. Export these tables to csv files:

    • Album.csv
    • Artist.csv
    • Genre.csv
    • MediaType.csv
    • Track.csv

We're going to copy an entrypoint folder with a shell script and all the csv files we exported into the MongoDB Docker image in order to populate the database. In the project root, create a new folder named docker with an entrypoint-initdb.d folder as follows.

mkdir -p docker/entrypoint-initdb.d
Enter fullscreen mode Exit fullscreen mode

Copy or move all of the exported csv files into the docker/entrypoint-initdb.d folder.

In the docker folder, create a mongo.dockerfile that will create an image from mongo and copy the files in entrypoint-initdb.d into the docker-entrypoint-initdb.d folder of the new image.

mongo.dockerfile 
FROM mongo

COPY ./entrypoint-initdb.d/* /docker-entrypoint-initdb.d/
Enter fullscreen mode Exit fullscreen mode

In the docker/entrypoint-initdb.d folder, create this importChinook.sh script. This script will run when the image is created to populate MongoDB using the csv files.

importChinook.sh 
mongoimport --db chinook --collection Album --type csv -f AlbumId,Title,ArtistId --file /docker-entrypoint-initdb.d/Album.csv
mongoimport --db chinook --collection Artist --type csv -f ArtistId,Name --file /docker-entrypoint-initdb.d/Artist.csv
mongoimport --db chinook --collection Genre --type csv -f GenreId,Name --file /docker-entrypoint-initdb.d/Genre.csv
mongoimport --db chinook --collection MediaType --type csv -f MediaTypeId,Name --file /docker-entrypoint-initdb.d/MediaType.csv
mongoimport --db chinook --collection Track --type csv -f TrackId,Name,AlbumId,MediaTypeId,GenreId,Composer,Milliseconds,Bytes,UnitPrice --file /docker-entrypoint-initdb.d/Track.csvnpm i nodemon -D
Enter fullscreen mode Exit fullscreen mode

Node.js Koa API

The API is built using Koa.js Next generation web framework for Node.js. This app will accept requests to /api and return json data from the MongoDB Docker container.

In the project root, create a folder named api with src/server/chinook and src/server/routes folders within. For example,

mkdir -p api/src/server/{chinook,routes}
Enter fullscreen mode Exit fullscreen mode

In the api/src/server/routes folder, create a chinook folder for the respective routes.

Project structure

Project structure image with folders and file tree

Initialize the Node.js app with npm init to create the package.json manifest file that will include all of the application dependency definitions and npm script commands for starting and building the app. For example,

cd api

npm init -y
Enter fullscreen mode Exit fullscreen mode

The following npm i or npm install commands are run from the api directory. When the install commands are run, the package.json file is updated with the respective package version info.

Install the MongoDB Node.js driver, mongodb.

npm i mongodb
Enter fullscreen mode Exit fullscreen mode

Install mongoose for a schema-based solution to model the application data. It also includes built-in type casting, validation, query building, business logic hooks and more.

npm i mongoose
Enter fullscreen mode Exit fullscreen mode

Models

In the src/server/chinook folder, create the data models. For example,

album.js 
const mongoose = require('mongoose');
const Schema = mongoose.Schema;

const AlbumSchema = new Schema({
    AlbumId: Number,
    Name: String,
    ArtistId: Number
},{ 
    collection: 'Album'
});

const chinook = mongoose.connection.useDb('chinook');

module.exports = chinook.model('Album', AlbumSchema);
Enter fullscreen mode Exit fullscreen mode
artist.js 
const mongoose = require('mongoose');
const Schema = mongoose.Schema;

/*
 notice there is no ID. That's because Mongoose will assign
 an ID by default to all schemas

 by default, Mongoose produces a collection name by passing the model name to
 the utils.toCollectionName method.
 This method pluralizes the name Artist to Artists.
 Set this option if you need a different name for your collection.
*/

const ArtistSchema = new Schema({
    ArtistId: Number,
    Name: String
},{ 
    collection: 'Artist'
});

const chinook = mongoose.connection.useDb('chinook');

module.exports = chinook.model('Artist', ArtistSchema);
Enter fullscreen mode Exit fullscreen mode
track.js 
const mongoose = require('mongoose');
const Schema = mongoose.Schema;

const TrackSchema = new Schema({
    TrackId: Number,
    Name: String,
    AlbumId: Number,
    MediaTypeId: Number,
    GenreId: Number,
    Composer: String,
    Milliseconds: Number,
    Bytes: Number,
    UnitPrice: String
},{ 
    collection: 'Track'
});

const chinook = mongoose.connection.useDb('chinook');

module.exports = chinook.model('Track', TrackSchema);
Enter fullscreen mode Exit fullscreen mode

Koa

Install koa and koa-router.

npm i koa koa-router
Enter fullscreen mode Exit fullscreen mode

Routes

In the src/server/routes folder, create the default api route. For example,

index.js 
const Router = require('koa-router');
const router = new Router();

router.get('/api/', async (ctx) => {
  ctx.body = {
    status: 'success',
    message: 'hello, world!'
  };
})

module.exports = router;
Enter fullscreen mode Exit fullscreen mode

In the src/server/routes/chinook folder, create the api/chinook routes. For example,

album.js 
const Router = require('koa-router');

const connect = require('../../chinook/connect');

connect();

const router = new Router();
const BASE_URL = `/api/chinook`;

const Album = require('../../chinook/album');

function getAlbums(artist) {
    return new Promise((resolve, reject) => {
        var query = Album.find({ 'ArtistId': artist });
        query.exec((err, results) => {
            if (err) return handleError(err);
            resolve(results);
        });
    });
}

router.get(BASE_URL + '/albums/:artist', async (ctx) => {
    try {
        ctx.body = await getAlbums(ctx.params.artist);
    } catch (err) {
        console.log(err)
    }
})

module.exports = router;
Enter fullscreen mode Exit fullscreen mode
artist.js 
const Router = require('koa-router');

const connect = require('../../chinook/connect');

connect();

const router = new Router();
const BASE_URL = `/api/chinook`;

const Artist = require('../../chinook/artist');

function getArtists() {
    return new Promise((resolve, reject) => {
        var query = Artist.find();
        query.exec((err, results) => {
            if (err) return handleError(err);
            resolve(results);
        });
    });
}

router.get(BASE_URL + '/artists', async (ctx) => {
    try {
        ctx.body = await getArtists();
    } catch (err) {
        console.log(err)
    }
})

module.exports = router;
Enter fullscreen mode Exit fullscreen mode
track.js 
const Router = require('koa-router');

const connect = require('../../chinook/connect');

connect();

const router = new Router();
const BASE_URL = `/api/chinook`;

const Track = require('../../chinook/track');

function getTracks(album) {
    return new Promise((resolve, reject) => {
        var query = Track.find({ 'AlbumId': album });
        query.exec((err, results) => {
            if (err) return handleError(err);
            resolve(results);
        });
    });
}

router.get(BASE_URL + '/tracks/:album', async (ctx) => {
    try {
        ctx.body = await getTracks(ctx.params.album);
    } catch (err) {
        console.log(err)
    }
})

module.exports = router;
Enter fullscreen mode Exit fullscreen mode

App Entrypoint

Create a src/server/index.js application entrypoint file as follows to initiate the app, routes and configure the MongoDB connection.

index.js 
const Koa = require('koa');
const mongoose = require('mongoose');
const indexRoutes = require('./routes/index');
const artistRoutes = require('./routes/chinook/artist');
const albumRoutes = require('./routes/chinook/album');
const trackRoutes = require('./routes/chinook/track');

/**
 * Koa app */
const app = new Koa();
const PORT = process.env.PORT || 1337;
const server = app.listen(PORT, () => {
    console.log(`Server listening on port: ${PORT}`);
});

/**
 * MongoDB connection */
const connStr =  'mongodb://mongo:27017/default';
mongoose.connect(connStr);
const db = mongoose.connection;
db.on('error', console.error.bind(console, 'connection error:'));
db.once('open', () => {
    console.log('connected');
});

app.use(indexRoutes.routes());
app.use(artistRoutes.routes());
app.use(albumRoutes.routes());
app.use(trackRoutes.routes());

module.exports = server;
Enter fullscreen mode Exit fullscreen mode

npm-run-script

To build the respective dev or prod versions of the api server, in the package.json file under scripts, define the dev and start commands. These commands are executed when the Docker container is started based on the settings in the docker-compose.yml.

package.json 
...

"scripts": {
    "dev": "nodemon ./src/server/index.js",
    "start": "node ./src/server/index.js",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
}
Enter fullscreen mode Exit fullscreen mode

Since nodemon is needed to watch and rebuild our api app in dev mode, let's install it and save it as a dev dependency.

npm i nodemon -D
Enter fullscreen mode Exit fullscreen mode

Docker Compose

To install the docker images, create our containers and start up our environment, add this docker-compose.yml file to the project root. Note that the volume paths map the project files to their paths within the Docker containers. For example, the Hugo publish directory www/public maps to the nginx server path for html, /usr/share/nginx/html.

version: "3"

services:

  app:
    image: node:alpine
    container_name: "${DEV_PROJECT_NAME}_node"
    user: "node"
    working_dir: /home/node/app
    labels:
      - 'traefik.backend=${DEV_PROJECT_NAME}_node'
      - 'traefik.frontend.rule=Host: ${DEV_PROJECT_HOST}; PathPrefix: /api'
    environment:
      - NODE_ENV=production
    volumes:
      - ./api:/home/node/app
      - ./api/node_modules:/home/node/node_modules
    expose:
      - "1337"
    # command: "node ./src/server/index.js"
    command: "npm run dev"
    depends_on:
      - mongo

  mongo:
    build:
      context: ./docker
      dockerfile: mongo.dockerfile
    container_name: "${DEV_PROJECT_NAME}_mongo"
    labels:
      - 'traefik.backend=${DEV_PROJECT_NAME}_mongo'
    ports:
      - "27017:27017"
    volumes:
      - mongodata:/data/db

  nginx:
    image: nginx
    container_name: "${DEV_PROJECT_NAME}_nginx"
    labels:
      - 'traefik.backend=${DEV_PROJECT_NAME}_nginx'
      - 'traefik.frontend.rule=Host: ${DEV_PROJECT_HOST}'
    volumes:
      - ./www/public:/usr/share/nginx/html

  traefik:
    image: traefik
    container_name: "${DEV_PROJECT_NAME}_traefik"
    command: -c /dev/null --docker --logLevel=INFO
    ports:
      - "80:80"
      - "8080:8080"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

volumes:
  mongodata:
Enter fullscreen mode Exit fullscreen mode

I like to use an .env file to configure docker-compose variables. In the project root, create this .env file.

### PROJECT SETTINGS

DEV_PROJECT_NAME=hkm
DEV_PROJECT_HOST=localhost
Enter fullscreen mode Exit fullscreen mode

In the project root, run docker-compose up -d which starts the containers in the background and leaves them running. The -d is for detached mode.

nginx 403 webpage screen capture

If you get a 403 Forbidden nginx server message, it's because we didn't publish the Hugo site.

cd www

hugo
Enter fullscreen mode Exit fullscreen mode

To see the published Hugo site, restart the services in the project root using docker-compose. The -d switch is for disconnected mode, for example,

docker-compose down

docker-compose up -d
Enter fullscreen mode Exit fullscreen mode

API Test

Load localhost/api/chinook/artists in a browser to see the json response.

For troubleshooting, view the docker conatainer logs or spin up in connected mode by omitting the -d switch, e.g., docker-compose up.

All of the source code for this tutorial is available on GitHub.

Source Code

Originally published at jimfrenette.com/2019/05/hugo-plus-nodejs-koa-app-mongodb-connection

Top comments (0)

Read next

nedsoft profile image

Automate Application Deployment to AWS Elastic Beanstalk with Terraform and CodePipeline

Chinedu Orie -

nesterow profile image

A take on Go Style Error Handling in JavaScript

Anton Nesterov -

florianrappl profile image

🎃 50 Articles to Avoid Web Dev Nightmares

Florian Rappl -

usman_awan profile image

Exploring REST and GraphQL APIs: Advantages, Challenges, and Use Cases

USMAN AWAN -