Jan Küster for Meteor Software

Posted on Mar 8

Build Your Own Uptime Monitor with MeteorJS + Fetch + Plotly.js ☄️🔭

#javascript #tutorial #meteor #beginners

In this tutorial I'll going to show you how to create your own uptime monitor in six easy steps. The app will use of the following stack, database to frontend:

MongoDB to store our data as documents, close to JS objects
MeteorJS to build and run our app and handle the data layer out-of-the-box
fetch at the heart of the monitoring requests to the external server
React as our frontend library
Plotly as our chart renderer

Concept and Architecture
Create a New React Project with MeteorJS
Implementation
Where to go from here
About me

Concept and Architecture

There is an HTTP endpoint on your external server that is constantly requested by your Meteor app's server using fetch.
Each response and it's status, including timeouts and latency measures, is stored as a "document" in the Database, which is MongoDB by default.
Meteor "watches" the collection of documents and "publishes" updates of the collection to clients, that "subscribe" to it. This allows live-monitoring to be set up for custom periods of time.
The React client application subscribes to the live changes and uses Plotly to display them as a line graph showing uptime and response time for a given period.

Please note that you should only use this for websites that you own or have the permission to monitor. Make sure that you always act within the law.

Create a New React Project with MeteorJS

If you do not have MeteorJS installed on your system, you may want to perform this quick install of the latest stable version:

$ npm install -g meteor
# on Linux||MacOS, you can also use the shell-installer script via
$ curl https://install.meteor.com/ | sh

Then, it's time to create a new project, once the meteor tool is installed and available:

$ meteor create uptime

This creates a new React-based Meteor project in the folder uptime.

Note: If you don't want to use React, then you can also see what other frontends are available by running meteor create --help. Note, however, that this tutorial focuses on Meteor+React

Go to the uptime folder and install some dependencies, then run the project for the first time:

$ cd uptime
$ meteor npm install react-plotly.js plotly.js node-abort-controller
$ echo "{}" >> settings.json
$ meteor --settings=settings.json

Your project is now running on http://localhost:3000

Implementation

Most of the following steps are on the server. However, shared (isomorphic) code between client and server is placed in a separate folder called /imports. A new React project should have this folder by default, though.

Step 1 - Configuration and Configurability

For our uptime measurements we use a simple HTTP HEAD request. We use HEAD, because there is no response body sent over the wire and being parsed.

However, the request can be configured using Meteor's settings.json file, similar to .env, allowing for easy changes in the future without the need to modify any code.

Let's create a new monitor in settings.json:

{
  "monitors": {
    "myDomain": {
      "name": "MyDomainXYZ",
      "method": "HEAD",
      "url": "https://mycooldomain.xyz/reachability",
      "timeout": 10000,
      "interval": 5000,
      "response": 204
    }
  }
}

This configuration involves sending a HEAD request to the specified URL every 5 seconds and measuring the response time. The expected response is a 204 status code, but if there is no response within 10 seconds, the server will be considered 'down' and the request will be aborted.

It is important to note that using other request methods besides HEAD may take longer to process and could potentially consume more server resources.

Step 2 - Create Collections

To send data to the client and persist it between server restarts, two MongoDB collections are required: one to preserve the overall monitor status and another to store the summary of each request as a document.

Create both collections in /imports/collections/MonitorCollection.js and /imports/collections/StatusCollection.js:

import { Mongo } from 'meteor/mongo'
export const MonitorCollection = new Mongo.Collection('monitor')

import { Mongo } from 'meteor/mongo'
export const StatusCollection = new Mongo.Collection('status')

They will be used later to automatically sync the data from the server to the client using "publish/subscribe".

Step 3 - Create a Handler for the Collections

Instead of directly accessing the collections we actually want to control inflow and outflow of data. For this we create a Monitor object in server/Monitor.js, that will manage all this for us.

import { Meteor } from 'meteor/meteor'
import { MonitorCollection } from '../imports/collections/MonitorCollection'
import { StatusCollection } from '../imports/collections/StatusCollection'

export const Monitor = {}

First, we want to be able to initially create a status for a configured domain, so that it's status/info is available, even if we haven't fired a request yet:

// ... continue in server/Monitor.js

Monitor.init = async ({ name, url, method, interval }) => {
  return StatusCollection.upsertAsync({ name }, {
    $set: { name, url, method, interval } 
  })
}

This will also update the status, in case we changed our configuration in settings.js.

Okay, now let's think about what we actually want to measure. We need a name as key, a timestamp of the request, a duration, a hasTImedOut flag and an error, in case we want to read error details.

Furthermore, we want to update status only once it has changed and not every time a new document arrives. A simple first implementation could look like this:

// ... continue in server/Monitor.js

Monitor.add = async document => {
  const { name, hasTimedOut, duration, error } = document
  const status = await StatusCollection.findOneAsync({ name })
  let statusChanged = false

  // we just went down :-/
  if (status.up && hasTimedOut) {
    status.up = false
    statusChanged = true
  }

  // we're back in business :-)
  if (!status.up && !hasTimedOut) {
    status.up = true
    statusChanged = true
  }

  if (statusChanged) {
    console.debug('[Monitor]: status changed:', status)
    await StatusCollection.updateAsync(status._id, { $set: status })
    // here is a good spot to send an Email to
    // inform about the status change!
  }

  await MonitorCollection.insertAsync(document)
}

Finally, we want to define the amount of data, send to the client. We do this by defining Meteor Publications. On the one hand, we want to publish the status of the monitor as a single document:

Meteor.publish('status', function ({ name }) {
  return StatusCollection.find({ name },  { limit: 1 })
})

On the other hand we want a good bunch of request summaries, that will help us to build a nice graph of the uptime and response duration:

// ... continue in server/Monitor.js

Meteor.publish('monitor', function ({ name }) {
  return MonitorCollection.find({ name }, { limit: 250, hint: { $natural: -1 }})
})

Note the hint: { $natural: -1 } which tells the collection to so a natural backwards search (start with the newest entry), omitting the need for sort.

Step 4 - Create an Uptime Request

Now, that we have set up a collection management, we still need a handler that fires and evaluates our requests by our given configuration.

For this we create a class this time. Create a new file server/UptimeRequest.js and start with the following:

import { fetch } from 'meteor/fetch'
import { AbortController } from "node-abort-controller"
import { PerformanceObserver, performance } from 'node:perf_hooks'

export class UptimeRequest {
  constructor ({ name, method, url, timeout, response }) {
    // config
    this.name = name
    this.method = method
    this.url = url
    this.timeout = timeout
    this.expectedResponse = response
    this.actualResponse = null

    // results / measurements
    this.response = null
    this.timestamp = null
    this.performance = null
    this.error = null
    this.hasTimedOut = null
    this.response = null
  }

  async fire () {
    // here we implement the actual request
    // and measure the outcome
  }

  toDocument () {
    // here we will return results that matter
  }
}

As you can see there is no interval as this class will be instantiated each time for a new request, thus one instance represents one request.

Let's implement the fire method:

//... in the UptimeRequest class

async fire () {
    this.timestamp = new Date()

    // prepare performance measures
    // see https://nodejs.org/api/perf_hooks.html#performanceentryduration
    const observer = new PerformanceObserver((items) => {
      items.getEntries().forEach((entry) => {
        this.performance = entry
      })
    })
    observer.observe({ entryTypes: ["measure"], buffer: true })

    const start = `${this.name} - start`
    const ended = `${this.name} - ended`
    const controller = new AbortController()
    const options = {
      method: this.method,
      priority: 'high',
      redirect: 'error',
      signal: controller.signal
    }

    const timeout = setTimeout(() => {
      this.hasTimedOut = true
      controller.abort()
    }, this.timeout)

    try {
      performance.mark(start)
      this.response = await fetch(this.url, options)
    } catch (error) {
      this.error = error
    } finally {
      performance.mark(ended)
      clearTimeout(timeout)
      performance.measure(this.url, start, ended)
      observer.disconnect()
    }

    this.actualResponse = this.response?.status

    // post checkup
    if (!this.error && this.response.status !== this.expectedResponse) {
      this.error = new Error('Response code mismatch')
    }

    // normalize error
    if (this.error) {
      const error = {}
      Object.getOwnPropertyNames(this.error).forEach(prop => {
        error[prop] = this.error[prop]
      })
      delete error.stack
      this.error = error
    }
  }

// ...

Without going into too much details, this method creates a fetch request with a so called AbortSignal that aborts the request, once our given timeout has been exceeded. On top it's surrounded by Node's native performance measurement to measure the overall request-response duration.

You can read up everything that happens in this method using the following resources:

Finally we want to provide a summary as document to the outside consumers by implementing a simple toDocument method:

//... in the UptimeRequest class

  toDocument () {
    return {
      name: this.name,
      response: {
        expected: this.expectedResponse,
        actual: this.actualResponse,
      },
      fired: this.timestamp,
      duration: this.performance?.duration ?? 0,
      hasTimedOut: !!this.hasTimedOut,
      error: this.error
    }
  }

// ...

Step 5 - Server Startup

We can finally run our server-side code that will request the external server and saves the results and status in the database. For this, remove all code in server/main.js and replace it with this one:

import { Meteor } from 'meteor/meteor'
import { UptimeRequest } from './UptimeRequest'
import { Monitor } from './Monitor'

Meteor.startup(async () => {
  const { myDomain } = Meteor.settings.monitors

  await Monitor.init(myDomain)

  const monitor = async () => {
    const request = new UptimeRequest(myDomain)
    await request.fire()
    return Monitor.add(request.toDocument())
  }

  Meteor.setInterval(monitor, myDomain.interval)
})

On startup we first register or update our domain (see the Monitor.init method in step 1.3) and then run a request for each interval, adding the result to the collections and updating the status.

Step 6 - The Monitoring Frontend

The client part is much shorter and will actually fit in one step. This is, because here it combines the magic of Meteor's reactivity together with the magic of Plotly's auto-generation of charts, just to save you hours and hours of coding! 🧙‍♂️

For this you only need to change the file imports/ui/App.jsx to the following:

import React from 'react'
import { MonitorCollection } from '../collections/MonitorCollection'
import { StatusCollection } from '../collections/StatusCollection'
import { useTracker, useSubscribe, useFind } from 'meteor/react-meteor-data'
import Plot from 'react-plotly.js'

const name = 'MyDomainXYZ'

export const App = () => {
  // subscribe to status, call
  // statusLoading() to know if it's loading or not
  const statusLoading = useSubscribe('status', { name })

  // subscribe to monitor docs,
  // use monitorLoading() to know if it's loading or not
  const monitorLoading = useSubscribe('monitor', { name })

  // the actual status doc, which will be undefined,
  // as long as the subscription is still loading
  const [status] = useFind(() => StatusCollection.find())
  const [uptime, duration] = useTracker(() => {
    const docs = MonitorCollection.find({}, {
      // make sure the client sorts the docs,
      // in case they arrive in a different order
      // than the server queried them
      sort: { fired: 1 }
    })
    const updatime = {
      x: [], // doc index as Date
      y: [], // up or down
      type: 'scatter'
    }

    const duration = {
      x: [],
      y: [],
      type: 'scatter'
    }
    docs.forEach((doc, index) => {
      updatime.x.push(doc.fired)
      duration.x.push(doc.fired)

      updatime.y.push(doc.hasTimedOut ? 'down' : 'up')
      duration.y.push(doc.hasTimedOut ? 0 : doc.duration)
    })
    return [updatime, duration]
  }, [])

  return (
    <div>
      <h1>Uptime Monitor for "{name}"</h1>
      <StatusInfo status={status} loading={statusLoading() || monitorLoading()}/>
      <Plot data={[uptime]} layout={{ width: 800, height: 400, title: 'Uptime' }} config={{ responsive: true }}/>
      <Plot data={[duration]} layout={{ width: 800, height: 400, title: 'Duration' }} config={{ responsive: true }}/>
    </div>
  )
}

// pure component, could be used with memo
const StatusInfo = ({ status, loading }) => {
  if (loading || !status) return (<div>Status loading...</div>)

  return (
    <div>
      <span>Status: {status.up ? 'up' : 'down'}</span><span> | </span>
      <span>URL: {status.url}</span><span> | </span>
      <span>Method: {status.method}</span><span> | </span>
      <span>Interval: {status.interval / 1000}s</span>
    </div>
  )
}

The result are two line-graphs that update every 5 seconds, as defined in our config:

The magic behind this very short code is actually the hooks, provided by the react-meteor-data package, which is already included in every new Meteor-React project. Here are some excerpts:

`useTracker`

You can use the useTracker hook to get the value of a Tracker reactive function in your React "function components." The reactive function will get re-run whenever its reactive inputs change, and the component will re-render with the new value.

To provide some more context: This hook uses Meteor's Tracker to listen on changes in a Data-set, such as our two collections. Under the hood there is an observer that runs, every time a document is added, removed or updated on this collection, invoking a Tracker computation, which in turn invokes a new React computation, similar to what you already know from other React hooks.

`useSubscribe`

At its core, it is a very simple wrapper around useTracker (with no deps) to create the subscription in a safe way, and allows you to avoid some of the ceremony around defining a factory and defining deps. Just pass the name of your subscription, and your arguments.

`useFind`

The useFind hook can substantially speed up the rendering (and rerendering) of lists coming from mongo queries (subscriptions). It does this by controlling document object references. By providing a highly tailored cursor management within the hook, using the Cursor.observe API, useFind carefully updates only the object references changed during a DDP update.

Where to go from here

The app in this tutorial is far from complete. However, I compiled a few ideas for your next steps with this one:

update to Meteor 3.0 and deploy to an ARM device like a Raspberry PI (we may cover this in the future if demand is high)
deploy your app to Meteor Cloud (managed hsoting) or your own infrastructure using Meteor-Up
extend the app to monitor multiple clients
use a job / cron-job library, instead of setInterval
create a dashboard for your monitors
add Meteor Accounts to manage users, authentication etc.
improve the Plot layouts and make them visually more appealing
use caching and capped collections or redis-oplog for scaling

About me 👋

I regularly publish articles about Meteor.js and JavaScript here on dev.to. I also recently co-hosted the weekly Meteor.js Community Podcast, which covers the latest in Meteor.js and the community.

You can also find me (and contact me) on GitHub, Twitter/X and LinkedIn.

If you like what you read and want to support me, you can sponsor me on GitHub, send me a tip via PayPal or sponsor a book from my Amazon wishlist.

DEV Community

Build Your Own Uptime Monitor with MeteorJS + Fetch + Plotly.js ☄️🔭

Table of Contents