Building Context-Aware Applications in Node.js: A Deep Dive

So you're building your awesome application and would like to have some context-aware code. For example, you want all the logs generated in your app (or across your microservices) to contain the requestId that started the operation. Or perhaps you would like to always have access to the userId of whoever initiated the operation, such as to generate audit logs. But how can you keep track of this context and make it available to all downstream function calls, or even across microservices communicated via message queues, gRPC, REST, and more?

In this article, we will explore the answers to these questions, focusing on Node.JS applications. We will cover various aspects of building context-aware applications and discuss how to implement and leverage AsyncLocalStorage to store and propagate the context throughout your codebase. Additionally, we'll see how to automatically add this context to logs and share the context between microservices.

Table of Contents

1. Building the Context
2. Leveraging AsyncLocalStorage for Context Storage
3. Accessing Context Downstream
4. Automatically Adding Context to Logs
5. Passing Context between Microservices

1. Building the Context

Before diving into the technical details, let's clarify what we mean by "context" in this context (pun intended). Context refers to the information that you want to propagate across various parts of your application or microservices. This can include request-related data, user-related data, or any other relevant information.

To build the context, you need to identify the specific data points that are crucial to your application's functionality. Make a list of the key data points you want to track and ensure they are accessible throughout your codebase.

Example: Express Middleware for Retrieving Request and User ID

Let's consider an example where you want to extract the requestId and userId from the headers of an incoming request in an Express application. To achieve this, you can create a custom middleware that runs before your route handlers and populates the context with the necessary information.

Here's an example implementation of the middleware:

import { RequestContext } from "../requestContext";

// Express middleware for retrieving request and user ID
function contextMiddleware(req: Request, res: Response, next: NextFunction): void {
  // Retrieve the request ID from the headers
  const requestId = req.headers['x-request-id'];

  // Retrieve the user ID from the headers
  const { userId } = decodeToken(req.headers['authorization']);

  // Create the context object
  const context = {
    requestId,
    userId,
  };

  // Store the context; More on this below
  RequestContext.start(context);

  next();
}

In this example, the middleware function contextMiddleware is responsible for extracting the requestId and userId from the headers of the incoming request. It then creates a context object containing this information.

By adding the contextMiddleware before your route handlers, you ensure that the context is populated with the requestId and userId from the headers of each incoming request. Now to make it possible for the context to be accessed downstream in your application.

2. Leverage AsyncLocalStorage for Context Storage

In newer versions of node.js, we can leverage the power of AsyncLocalStorage to store and manage the context throughout the execution flow. AsyncLocalStorage provides a mechanism to associate contextual data with the execution of asynchronous functions and enables the data to be automatically propagated as the functions are called.

In this section, we will explore how to integrate AsyncLocalStorage into your application and use it to store and retrieve the desired context. We will create a ContextStore class to hold the context data and a RequestContext class to create and access the current execution context.

// requestContext.ts
import { AsyncLocalStorage } from 'async_hooks';

export interface IInitContext {
  userId?: string;
  requestId?: string;
}

export class ContextStore {
  private _userId?: string;
  private _requestId: string;

  constructor({ userId, requestId }: IInitContext) {
    this._userId = userId;
    this._requestId = requestId;
  }

  toJSON() {
    return {
      userId: this._userId,
      requestId: this._requestId,
    };
  }

  get userId() {
    return this._userId;
  }
  get requestId() {
    return this._requestId;
  }
}

export class RequestContext {
  static als = new AsyncLocalStorage();

  static start = (ctx: IInitContext): void => {
    this.als.enterWith(new ContextStore(ctx));
  };

  static getStore(): ContextStore {
    const store = this.als.getStore();
    return store as ContextStore;
  }
}

3. Accessing Context Downstream

Now that we have successfully set the context using RequestContext, we can simply call the getStore() function within your downstream functions. This will retrieve the context associated with the current execution flow.

import { RequestContext } from "../requestContext";

// Downstream function
function randomDownstreamFunction() {
  const requestContext = RequestContext.getStore();

  // Access and use the context as needed
  console.log(requestContext.userId);
}

By retrieving the context, you can access all the relevant data points and use them for logging, auditing, or any other requirements specific to your application.

4. Automatically Adding Context to Logs

Logging is an essential aspect of any application, and having the context included in your logs can greatly assist in troubleshooting and understanding the flow of operations. To automatically add the context to your logs, you can utilize popular logging libraries such as winston, pino, or bunyan.

Here's an example using the winston library:

import { format, createLogger, transports } from "winston";
import { RequestContext } from "../requestContext";

const addContext = format((info) => {
  const requestContext = RequestContext.getStore();
  if (requestContext) {
    const oldCtx = info.ctx || {};
    info.ctx = { ...oldCtx, ...requestContext.toJSON() };
  }

  return info;
});

const logger = createLogger({
  format: format.combine(
    format.json(),
    addContext(),
  ),
  transports: [
    new transports.Console(),
  ],
});

// Usage example
function processRequest(request) {
  logger.info('Processing request...');
}

This will create logs like this

By customizing the log format and retrieving the context using getStore(), you can add the relevant context information to your logs effortlessly.

5. Passing Context between Microservices

In a microservices architecture, passing the context from one service to another becomes vital. Context propagation ensures that downstream services have access to the same context data as the originating service. This allows for end-to-end tracking and maintains the integrity of the context throughout the system.

To pass the context between microservices, you can include it in the payload of the message or the headers of the HTTP request. The specific implementation details will depend on the communication mechanism you are using (message queues, gRPC, REST, etc.).

For example, when using message queues, you can serialize the context data and include it as a part of the message payload. On the receiving end, the consumer can deserialize the data and set it as the context using RequestContext class we created.

import { RequestContext } from "../requestContext";

// publisher
function publish(data: Record<string, any>): Promise<void> {
    // add ctx to message
    let _ctx = {};
    const requestContext = RequestContext.getStore();
    if (requestContext) {
      _ctx = requestContext.toJSON();
    }
    const publishData = { data, _ctx };

    // publish via rabbitMq
}

// listener
async function onMessage(message) {
  const messageBody = msg.content.toString();

  const { data, _ctx = {} } = JSON.parse(messageBody);

  // build context from message
  RequestContext.start(_ctx);

  // process the message here
}

Similarly, when using RPC / HTTP-based communication, you can include the stringified context as Metadata or headers respectively in the request. The receiving service can extract the context and set it using AsyncLocalStorage to make it accessible throughout its execution flow.

By ensuring context propagation, you enable seamless communication and consistent context availability across your microservices.

Conclusion

Building context-aware applications in node.js requires careful consideration of the data points you want to track and their availability throughout your codebase. By leveraging AsyncLocalStorage, you can easily store and retrieve the context, making it accessible to downstream functions or microservices. Additionally, integrating the context with your logging mechanism and ensuring context propagation between microservices further enhances the effectiveness of your application.

In this article, we have covered the fundamental aspects of building context-aware applications in node.js. We hope this guide serves as a valuable resource to help you create engaging and contextually rich applications.

Happy coding!