Trace & Observe Modern Apps using AWS X-Ray

Hi fellas, In this blog I am going to share my experience using one of the coolest AWS services named AWS X-Ray. AWS X-Ray is a fully managed monitoring and observability service that helps you collect data about requests that the application serves. It provides tools that enable you to filter, view, and gain insights into the collected data, helping you identify optimization opportunities and issues.

AWS X-Ray is a service that can help you figure out what's going on across all your systems. It lets you see how requests are routed through different service touchpoints and gives you a good idea of how your applications are performing. You can use it to monitor performance, identify bottlenecks, and troubleshoot errors. With AWS X-Ray, you can keep an eye on your systems and make sure everything's running smoothly. It allows you to visualize complex and detailed service relationships within highly distributed applications. You can trace message pathways and call stacks at any scale.

Below is the AWS X-Ray workflow image from AWS's official blog.

AWS X-Ray has been designed to work seamlessly with distributed systems. Over the last decade or two, as complex distributed systems have emerged, debugging has changed and has taken on a new meaning. Engineers can now analyze and debug applications, audit their applications securely, and compile data from AWS resources to determine bottlenecks in cloud architecture and improve application performance.

Now let's see the implementation of AWS X-Ray with a nodejs application. AWS X-Ray provides an SDK that can be imported and utilized within your application.

const express = require('express');
const app = express();
const serviceName = "HELLO-MICROSERVICE"
const port = 8000;

// Require AWS X-Ray SDK
const AWSXRay = require('aws-xray-sdk');
AWSXRay.captureHTTPsGlobal(require('http'));

// Use AWS X-Ray middleware
app.use(AWSXRay.express.openSegment('MyApp'));

app.get('/hello', (req, res) => {
  const seg = xray.getSegment();
  seg.addAnnotation('hello-microservice', serviceName);
  seg.addMetadata("Request Meta", req);

  res.send('Hello AWS X-Ray!');
});

// Close the X-Ray segment for the current request
app.use(AWSXRay.express.closeSegment());

app.listen(port, () => {
  console.log(`App listening at http://localhost:${port}`);
});

Make sure to install aws-xray-sdk before, when setting up instrumentation in nodejs app.

X-Ray Daemon

The AWS X-Ray daemon is a core software application that listens for traffic on UDP port 2000. It gathers raw segment data and relays it to the AWS X-Ray API. The daemon needs to work in conjunction with the AWS X-Ray SDKs and must be running so that the data sent by the SDKs can reach the X-Ray service. The X-Ray daemon is an open-source project that you can follow on GitHub. See more details of the AWS X-Ray daemon here.

Segments

AWS X-Ray receives service data in segments. XRay then groups segments that have a common request into traces. The resources running your application logic send data about their work as segments. Technically, it's an object that contains some metadata about the request including,

• The host - a hostname, alias, or IP address
• The request – method, client address, path, user agent
• The response – status, content
• The work done – start and end times, subsegments
• Issues that occur – errors, faults, and exceptions, including automatic capture of exception stacks.

Subsegments

To better monitor the work done by your application, you can use subsegments to break down the data into smaller pieces. Subsegments provide detailed timing information about downstream calls made by your application to complete the original request. They also contain additional details about calls to external services, such as AWS, HTTP APIs, or SQL databases. Furthermore, you can define custom subsegments to instrument-specific functions or lines of code within your application.

Traces

AWS X-Ray can help you trace requests as they move across various services. It captures important information about the path, duration, and performance of each request. A unique ID, called a Trace ID, is used to track the path of a request through your application. A trace is essentially a collection of all the segments generated by a single request. This request is usually an HTTP GET or POST request that passes through a load balancer, hits your application code, and generates downstream calls to other AWS services or external web APIs.

A trace ID and a sampling decision are added to HTTP requests in tracing headers named X-Amzn-Trace-Id. The first X-Ray-integrated service that receives the request adds the tracing header, which is then read by the X-Ray SDK and included in the response.

X-Amzn-Trace-Id: Root=1-5759e988-bd862e3fe1be46a994272793;Sampled=1

Service Graph

AWS X-Ray sends data about your app's services to create a graph that shows all the resources and services your app consists of. This graph is a JSON document that contains important information about your app's components. By using this graph, X-Ray can create a visual map of your app's pieces. This map helps you see how everything works together.

X-Ray Cost

AWS X-Ray doesn't charge upfront fees or commitment. you only pay for what you use based on the number of traces, and segments recorded and retrieved. For the Free tier,

• The first 100,000 traces recorded each month are free.
• The first 1,000,000 traces retrieved or scanned each month are free.

After the free tier or specific usage in the free tier, this is how this service charges you, which I think is not that expensive.

• Traces recorded cost $5.00 per 1 million traces recorded ($0.000005 per trace)
• Traces retrieved cost $0.50 per 1 million traces retrieved ($0.0000005 per trace).
• Traces scanned cost $0.50 per 1 million traces scanned ($0.0000005 per trace).
• X-Ray Insights traces stored costs $1.00 per million traces recorded ($0.000001 per trace).

AWS X-Ray enables customers to choose their sampling rate. Customers considering AWS X-Ray may want to estimate their costs for recorded traces by multiplying their request or API call rate by the chosen sampling rate. Read more here about the specific regions and their costs.

Earlier I mentioned that this service works smoothly with distributed systems or microservices in general. This is a microservices-based project on nodejs that is available on GitHub that is provided by CloudAcademy. You can clone and do hands-on with X-Ray in a real microservices-based application.

Summary

Let's recap what we have learned in this article. We learned what AWS X-Ray is and how we can use it with our applications irrespective of its architecture. AWS X-Ray is a service that helps you monitor and optimize your applications. It tracks requests, identifies issues, and provides insights into complex service relationships. You can trace message pathways and call stacks at scale, ensuring everything is running smoothly.

Resources

Here are the resources to get you started with AWS X-Ray.

AWS X-Ray hands-on workshop
AWS X-Ray with Nodejs
AWS X-Ray with Go
AWS X-Ray troubleshooting
AWS X-Ray Costing

if you like this article, please like and share it with your cloud friends and follow me on Linkedin, Github, Twitter.

Peace ✌🏻