Observability & Monitoring in Nodejs using Signoz

Hey there! If you've ever been deep into coding your Node.js app, tweaking and tuning every bit, only to find your server still can't handle the load, you're not alone. It's like everything looks perfect, but something's still off, and it's super annoying, right?

Spotting the Issue
So, how do you figure out what's wrong? Well, there's this saying that goes, "You can't improve what you don't measure." That's spot on because the first step to solving our mystery is to shine a light on those hidden problems.

When we hit this wall, we tried a bunch of stuff:

• flame graphs: These are super cool for seeing where your CPU time is going. It's like a detective tool for your functions.
  

• Memory Snapshots:
  When your app is guzzling memory like there's no tomorrow, memory snapshots are your best bet. Chrome's dev tools are pretty handy for this.

But, what if you want to keep a constant eye on your app?
Like having a health check that tells you everything's cool or if it's time to dive back in and fix stuff.

Discovering a Lifesaver: OpenTelemetry and SigNoz
That's when we stumbled upon OpenTelemetry. It's this awesome open-source project that helps you track all sorts of data from your app. Think of it as a Swiss Army knife for telemetry data (stuff like logs, metrics, traces).

Now, while OpenTelemetry is like having the right tools, we still needed a place to see all this info easily. That's where SigNoz comes in. Imagine finding a treasure chest in your backyard. That's how we felt when we found SigNoz. It's this platform where you can see everything going on with your app - logs, metrics, traces, you name it.

The best part? We set it all up for free using Oracle Cloud's VM.

in this blog, i am mostly going to cover how to host signoz for free and better monitor and observe your backend.

Create free oracle cloud account and create free VM and get access to the VM using ssh:

this tutorial will help you through each and every setup, i choose ubunutu as os, which is easy to work with and have lot of online guides when you are stuck!

now you have free VM with, now we are going to use docker to take care of everything for us, if you are not that much familiar with docker, it provides secure and reliable environment for applications to run,

1. just run following command on the VM in the dir in which you want to keep all the signoz related data

git clone -b main https://github.com/SigNoz/signoz.git && cd signoz/deploy/

1. run install script, it will install docker and required things as own it's own

./install.sh

this will look something like this when it is fetching the images from docker hub:

it will take some time, so relax, and wait till it is complete

When it's complete, you can now visit the dashboard by going to your

<IP>:3301

it will look something like this, where you first need to create account

you can see there are lot of features which will be helpful when monitoring and for observability, you can also invite other team members to the same dashboard,

now let's configure opentelemtry in our nodejs

// tracing.js
'use strict'
const process = require('process');
const opentelemetry = require('@opentelemetry/sdk-node');
const { getNodeAutoInstrumentations } = require('@opentelemetry/auto-instrumentations-node');
const { OTLPTraceExporter } = require('@opentelemetry/exporter-trace-otlp-http');
const { Resource } = require('@opentelemetry/resources');
const { SemanticResourceAttributes } = require('@opentelemetry/semantic-conventions');

const signoz_url="https://localhost"

const exporterOptions = {
   url: `https:localhost:4318/v1/traces`
   }

const traceExporter = new OTLPTraceExporter(exporterOptions);
const sdk = new opentelemetry.NodeSDK({
   traceExporter,
   instrumentations: [getNodeAutoInstrumentations()],
   resource: new Resource({
      [SemanticResourceAttributes.SERVICE_NAME]: 'node_app'
      })
});

// initialize the SDK and register with the OpenTelemetry API
// this enables the API to record telemetry

sdk.start()

// gracefully shut down the SDK on process exit
process.on('SIGTERM', () => {
   sdk.shutdown()
 .then(() => console.log('Tracing terminated'))
 .catch((error) => console.log('Error terminating tracing', error))
 .finally(() => process.exit(0));
 });

run your node backend with this script using below command

node -r ./tracing.js index.js

after usage, you will be able to see your service in the dashboard, you can see traces also, logs, and filter them and create your own custom dashboard if you want, and create custom alerts also.

Wrapping Up

Fixing performance issues in Node.js can feel like finding a needle in a haystack. But with the right tools - OpenTelemetry for gathering data and SigNoz for making sense of it all. It's like having a friend who's got your back, showing you exactly where the problem lies so you can fix it and move on.

if you have a question or just want to discuss things

reach me out on twitter