Monitoring Custom Metrics in Node.js with Prometheus on Kubernetes
Monitoring is an essential aspect of application maintenance, especially in a microservices environment. It ensures that developers and system administrators can track the health and performance of services. In this tutorial, we'll walk you through monitoring a simple Node.js microservice with custom metrics using Prometheus in a Kubernetes environment.
Overview
We will:
- Create a Node.js application with custom metrics using Prometheus.
- Deploy the application on a Kubernetes cluster.
- Install Prometheus and configure it to scrape custom metrics from our application.
- Visualize the data on Prometheus and set up an Alertmanager for notifications.
Prerequisites
- Basic knowledge of Kubernetes and Node.js.
- A working Kubernetes cluster (Minikube or any managed Kubernetes service).
- Helm installed for managing Kubernetes packages.
- kubectl configured to interact with your Kubernetes cluster.
- A Node.js environment for creating and testing the application locally.
Step 1: Setting Up the Node.js Application with Custom Metrics
1.1 Create a Node.js Application
Start by setting up a simple Node.js application. Initialize the project with:
mkdir nodejs-metrics-app
cd nodejs-metrics-app
npm init -y
Install the necessary dependencies:
npm install express prom-client
1.2 Instrument Metrics with Prometheus Client
Create an index.js file:
const express = require('express');
const Prometheus = require('prom-client');
const app = express();
const PORT = process.env.PORT || 3000;
// Initialize the Prometheus registry
const register = new Prometheus.Registry();
Prometheus.collectDefaultMetrics({ register });
// Define custom metrics
const httpRequestCount = new Prometheus.Counter({
name: 'http_request_total',
help: 'Total number of HTTP requests',
labelNames: ['method', 'route', 'status'],
});
const httpRequestDuration = new Prometheus.Histogram({
name: 'http_request_duration_seconds',
help: 'HTTP request duration in seconds',
labelNames: ['method', 'route', 'status'],
buckets: [0.1, 0.5, 1, 2, 5],
});
// Middleware to measure request duration
app.use((req, res, next) => {
const end = httpRequestDuration.startTimer();
res.on('finish', () => {
httpRequestCount.inc({ method: req.method, route: req.path, status: res.statusCode });
end({ method: req.method, route: req.path, status: res.statusCode });
});
next();
});
// Sample API endpoints
app.get('/health', (req, res) => {
res.send('Service is running');
});
app.get('/crash', (req, res) => {
process.exit(1); // Force the application to crash
});
// Expose metrics endpoint
app.get('/metrics', async (req, res) => {
res.set('Content-Type', register.contentType);
res.end(await register.metrics());
});
app.listen(PORT, () => {
console.log(`App listening on port ${PORT}`);
});
1.3 Test Locally
Start the application locally to ensure everything works:
node index.js
Visit http://localhost:3000/metrics to see the Prometheus metrics output.
Step 2: Containerize the Application
Create a Dockerfile for the Node.js application:
# Use a Node.js base image
FROM node:14
# Create app directory
WORKDIR /app
# Install app dependencies
COPY package*.json ./
RUN npm install
# Bundle app source code
COPY . .
# Expose port
EXPOSE 3000
# Start the application
CMD ["node", "index.js"]
Build and push the Docker image:
docker build -t your-dockerhub-username/nodejs-metrics-app .
docker push your-dockerhub-username/nodejs-metrics-app
Step 3: Deploy the Application on Kubernetes
3.1 Create Kubernetes Manifests
Create a Kubernetes manifest file deployment.yaml to deploy the application:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nodejs-metrics-app
namespace: dev
spec:
replicas: 2
selector:
matchLabels:
app: nodejs-metrics-app
template:
metadata:
labels:
app: nodejs-metrics-app
spec:
containers:
- name: nodejs-metrics-container
image: your-dockerhub-username/nodejs-metrics-app:latest
ports:
- containerPort: 3000
---
apiVersion: v1
kind: Service
metadata:
name: nodejs-metrics-service
namespace: dev
spec:
type: LoadBalancer
selector:
app: nodejs-metrics-app
ports:
- protocol: TCP
port: 80
targetPort: 3000
Apply the manifest:
kubectl create namespace dev
kubectl apply -f deployment.yaml
3.2 Verify Deployment
Ensure the pods are running:
kubectl get pods -n dev
Check the services:
kubectl get svc -n dev
Step 4: Install Prometheus Using Helm
Install Prometheus using the Helm chart:
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update
helm install prometheus-stack prometheus-community/kube-prometheus-stack --namespace monitoring --create-namespace
4.1 Expose Prometheus
Expose Prometheus for local access:
kubectl port-forward svc/prometheus-stack-kube-prometheus-prometheus -n monitoring 9090:9090
Visit http://localhost:9090 to access the Prometheus UI.
Step 5: Configure Service Discovery for Custom Metrics
5.1 Create a ServiceMonitor for Prometheus
Create a service-monitor.yaml file:
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: nodejs-metrics-monitor
namespace: dev
spec:
selector:
matchLabels:
app: nodejs-metrics-app
endpoints:
- port: 3000
path: /metrics
interval: 15s
Apply the manifest:
kubectl apply -f service-monitor.yaml
5.2 Verify Custom Metrics in Prometheus
Open Prometheus and query the custom metric http_request_total. You should now see data collected from your Node.js application.
Step 6: Set Up Alerting with Alertmanager
6.1 Configure Alertmanager
Create an Alertmanager configuration file:
apiVersion: v1
kind: Secret
metadata:
name: alertmanager-config
namespace: monitoring
stringData:
alertmanager.yaml: |
route:
receiver: 'email-alert'
receivers:
- name: 'email-alert'
email_configs:
- to: 'your-email@example.com'
from: 'alertmanager@example.com'
smarthost: 'smtp.example.com:587'
auth_username: 'your-email@example.com'
auth_identity: 'your-email@example.com'
auth_password: 'your-app-password'
6.2 Deploy the Alertmanager Configuration
kubectl apply -f alertmanager-config.yaml
6.3 Trigger an Alert
Force a crash in the application:
kubectl exec -it <nodejs-pod-name> -n dev -- curl http://localhost:3000/crash
Check if you receive an alert notification.
Conclusion
In this tutorial, we covered the basics of instrumenting custom metrics in a Node.js application, deploying the application on Kubernetes, and setting up Prometheus to monitor those metrics. Additionally, we configured Alertmanager to receive notifications when something goes wrong. This setup can be further enhanced by integrating a visualization tool like Grafana and exploring advanced metrics.
Top comments (0)