Node.js Open-Source Monitoring Tools

What is the most important feature your Node.js application can have? Do you think it’s having fancy fuzzy logic for your full-text search, or maybe using sockets for real-time chats? You tell me. What’s the fanciest, most amazing and sexy feature you can add to your Node.js application?

Want to know mine? High performance with no downtime. Performant applications need to do three things well.

1. Ensure minimal downtime
2. Have predictable resource usage
3. Scale effectively based on load

In Part 1, Node.js Key Metrics to Monitor, we talked about key Node.js metrics you should monitor in order to understand the health of your application. I also explained bad practices in Node.js you should avoid, such as blocking the thread and creating memory leaks, but also a few neat tricks you can use to boost the performance of your application, like using the cluster module to create worker processes and forking long-running processes to run separately from the main thread.

In this article, I’ll explain how to add monitoring to your Node.js application with 5 different open-source tools. They may not have full-blown features like Sematext or Datadog, but keep in mind they’re open-source products and can hold their own just fine.

Node.js monitoring with Appmetrics

Node Application Metrics Dashboard shows the performance metrics of your running Node.js application. It’s a simple module you install and require at the top of your main Node.js source file. You install the module from npm by running the following command in your terminal.

$ npm install appmetrics-dash

Appmetrics provides a very easy-to-use web-based dashboard. Everything you need to do in order to get a dashboard for all HTTP servers created by your application then you add this snippet in your app.js file, or whatever you call your main source file.

// Before all other 'require' statements
require('appmetrics-dash').attach()

You’ll now have a new server route /appmetrics-dash where you can see a plethora of useful metrics.

• CPU Profiling
• HTTP Incoming Requests
• HTTP Throughput
• Average Response Times (top 5)
• CPU
• Memory
• Heap
• Event Loop Times
• Environment
• Other Requests
• HTTP Outbound Requests

This tool doesn’t only display metrics. It lets you generate Node Reports and Heap Snapshots directly from the dashboard. Apart from that, you have access to Flame Graphs. Pretty cool for an open source tool.

Monitoring Node.js with Express Status Monitor

Express.js is the de-facto framework of choice for Node.js developers. Express Status Monitor is an incredibly simple, self-hosted module, you add to your Express server. It exposes a /status route that reports real-time server metrics with the help of Socket.io and Chart.js.

Installing the tool from npm is as simple as this.

$ npm install express-status-monitor

After you have the module installed, you need to add it before any other middleware or router.

app.use(require('express-status-monitor')())

Once you run your server go to the /status route and check out your Node.js metrics.

Monitoring Node.js with Prometheus

Unless you’re living under a rock, you must have heard of Prometheus. It’s the most notable and famous open-source monitoring tool you can use today. Prometheus is 100% open source and community-driven. All components are available under the Apache 2 License on GitHub. It’s also a graduated member project of the Cloud Native Computing Foundation, alongside projects like Kubernetes and Fluentd.

To start monitoring with Prometheus, you need to download the latest release and install it.

$ tar xvfz prometheus-\*.tar.gz
$ cd prometheus-\*

Then you start it by running the executable file but before running this command you need to create a prometheus.yml file. It’s a configuration file for collecting metrics from monitored targets by scraping metrics HTTP endpoints on these targets.

# prometheus.yml
scrape_configs:
  - job_name: 'prometheus'
    scrape_interval: 1s
    static_configs:
      - targets: ['127.0.0.1:3000']
        labels:
          service: 'test-prom'
          group: 'production'

Now you can run Prometheus.

$ ./prometheus --config.file=prometheus.yml

However, I’m rather lazy, and I like Docker a lot. So the way I do it is to run the official Prometheus Docker image and avoid all the hassle of downloading it.

Monitoring Node.js with Prometheus and Docker

First of all, navigate to the root directory of your Node.js application. Here you’ll create a prometheus-data directory and place the prometheus.yml file in it. Once you’ve done this, go ahead and move on to running the Prometheus Docker container.

You’ll grab the official Prometheus Docker image and run a container with the docker run command.

$ docker run -d \
    --name prometheus \
    --network="host" \
    -v "$(pwd)/prometheus-data":/prometheus-data \
    prom/prometheus \
    -config.file=/prometheus-data/prometheus.yml

I’ve chosen to run the container with –network="host" to let the Prometheus container access the localhost of my host, and in doing so, the Node.js application’s exposed HTTP endpoint. Otherwise, if you were running both Prometheus and Node.js in containers, you’d set up a network between the two to only be visible between each other.

The -v flag will map the prometheus-data directory from the host to an identically named directory inside of the container.

With the Prometheus container running, you can move on to adding the needed configuration in your Node.js application to expose a metrics endpoint. You’ll need to install the Prometheus Client for Node.js from npm first.

$ npm install prom-client

Then you’ll add the base configuration for Prometheus.

// after all 'require' statements
const client = require('prom-client')
const collectDefaultMetrics = client.collectDefaultMetrics
collectDefaultMetrics({ timeout: 1000 })
app.get('/metrics', (req, res) => {
  res.set('Content-Type', client.register.contentType)
  res.end(client.register.metrics())
})

What’s left is to run the Node.js server and open up http://localhost:9090/graph to access the Prometheus graphs.

Monitoring Node.js performance with Clinic.js

Clinic.js includes three tools to help diagnose and pinpoint Node.js performance issues. It’s surprisingly easy to use. All you need to do is install the module from npm and run it. This will generate reports for you that make troubleshooting much easier.

To install Clinic.js run this command in your terminal.

$ npm install clinic

Once you have it installed it’s all up to choosing what kind of report to generate. You can pick between three.

• Doctor
  • Collects metrics by injecting probes
  • Assess health and heuristics
  • Creates recommendations
• Bubbleprof – a new, completely unique, approach to profiling your Node.js code
  • Collects metrics using async_hooks
  • Tracks latency between operations
  • Creates bubble graphs
• Flame – uncovers the bottlenecks and hot paths in your code with flamegraphs
  • Collects metrics by CPU sampling
  • Tracks top-of-stack frequency
  • Creates flame graphs

Let’s start by running the Doctor and load testing a Node.js application.

$ clinic doctor -- node app.js

While it’s running, run a load test with your desired tool.

$ loadtest -n 1000 -c 100 [http://localhost:3000/api](http://localhost:3000/api)

Once it’s done running, stop the server and Clinic.js Doctor will open up a report you can check out.

Using this same approach you can run either Bubbleprof or Flame and get graphs for the respective tools.

Monitoring Node.js with PM2

Running Node.js applications in production gets a lot easier with PM2. It’s a process manager that easily lets you run apps in cluster mode. Or, in English, it’ll spawn a process for every CPU core your host has.

Start by installing PM2.

$ npm install pm2 -g

Once it’s installed you spawn the PM2 daemon by running this command in your terminal, if your main source file is app.js.

$ pm2 start app.js -i 0

The -i 0 flag stands for instances. This will run your Node.js application in cluster mode, where the 0 stands for number of CPU cores. You can manually put whatever number you want, but letting PM2 count the cores and spawn that amount of workers is much easier.

Monitoring Node.js with PM2 is just as easy.

$ pm2 monit

This command will open a dashboard in the terminal. Here you can monitor processes, logs, loop delay, process memory, and CPU.

Wrapping up Monitoring Node.js with Open-Source Tools

Performance metrics are crucial for keeping your users happy. In this article, I’ve shown you how to add monitoring to your Node.js application with 5 different open-source tools. After learning about Node.js key metrics to monitor in Part 1 of this series, adding tooling to monitor your apps in real life is the natural learning progression. The last part of the series will cover production ready Node.js monitoring with Sematext.

If you want to check out sample code, here’s a repo with all samples. You can also clone the repo and pick any of the tools right away.

If you need full-stack observability for your software stack, check out Sematext. We’re pushing to open source our products and make an impact.

Hope you guys and girls enjoyed reading this as much as I enjoyed writing it. If you liked it, slap that tiny share button so more people will see this tutorial. Until next time, be curious and have fun.