In this article, you’ll learn how to run Jaeger locally, why and when you should do it, as well as what are Jaeger’s limitations when running locally.
Let’s start with the basics: a distributed tracing system is generally composed of client and backend components.
I will touch briefly on client components, though most of this post is about backend components.
Client Components
The client part is usually a set of libraries installed inside an application which “instrument” it — generating a “span” object for each interesting event happening in runtime inside the service.
A modern and recommended open-source client SDK that does that is OpenTelemetry.
Spans on the client alone are meaningless — they need to be accessible to a person who consumes them. Consumers are usually dev-ops teams monitoring a system or developers maintaining the system and adding new features.
Trace Usages
There are many ways in which collected trace data can be used and provide value. These are the common ones:
- Aggregate spans to trace — group all spans (events) which are part of the same trace (logical operation) arriving from different distributed services, into a single entity
- Query the collected data (show me all traces in the last hour starting at endpoint
GET /usersin service X) - Visualize the data — usually in a graph, or timeline
- Find errors (exceptions, 500s, etc) and investigate their root-cause
- Investigate performance bottlenecks
Backend Components
To fulfill the requirements above, we need to set up backend components.
They are used to collect spans from client components, process them, store them in a database, expose an API for the data and UI to view traces and perform queries.
Jaeger (a CNCF graduated project) is a popular open-source project with backend components that does that and is easy to set up.
To use Jaeger in production, it is recommended to install it in a cloud environment with load-balancing, auto-scaling, replications, and all that jazz.
However, it is sometimes enough to just run it locally in a lightweight and simple setup.
Running Locally
The recommended approach for running Jaeger backend locally is to use docker:
$ docker run -d --name jaeger \
-e COLLECTOR_ZIPKIN_HOST_PORT=:9411 \
-p 5775:5775/udp \
-p 6831:6831/udp \
-p 6832:6832/udp \
-p 5778:5778 \
-p 16686:16686 \
-p 14268:14268 \
-p 14250:14250 \
-p 9411:9411 \
jaegertracing/all-in-one:1.23
And access the UI in http://localhost:16686.
You can then configure an Opentelemetry Client SDK installation or OpenTelemetry Collector to use Jaeger exporter and send trace data to this local Jaeger.
import { NodeTracerProvider } from "@opentelemetry/node";
import { SimpleSpanProcessor } from "@opentelemetry/tracing";
import { JaegerExporter } from "@opentelemetry/exporter-jaeger";
const provider = new NodeTracerProvider();
provider.addSpanProcessor(new SimpleSpanProcessor(new JaegerExporter()));
provider.register();
(This is how exporting data to local jaeger looks like in nodejs).
Running Local Jaeger: Benefits
1. Faster Debugging
If you work on a service codebase (e.g., fixing a bug, developing a new feature, or implementing integration to other services/databases/messaging systems, etc.), most likely that this is what you do:
- You start an instance of the service on your local dev station
- Send traffic to it to test your changes
- Validate the behavior you were expecting
By instrumenting it locally, you can debug development issues faster.
For example: find the point in your app where an error occurred with less logging to console, breakpoints, etc.
An example of a trace in Jaeger showing error while accessing Redis key as a list (the red underlines are not part of the Jaeger UI)
2. Running Tests
When running your integration test suite locally — if a test fails, it can sometimes be easier to understand what went wrong by examining it in Jaeger UI, where you can view highlighted errors and events organized into a hierarchical structure.
3. Instrumentation Development
If you are writing a new instrumentation library, observing the tracing output in a UI can be much easier than browsing through textual logs. You can browse the JS instrumentation package for more info and examples.
Running Local Jager: Limitations
Jaeger is free, relatively easy to set up, and will do a good job for most basic setups and tracing needs.
The UI and features set are quite basic and you may quickly find yourself in the need of more advanced features.
Other alternatives can give value and increase development productivity in the following cases:
1. Lack of End to End Visibility in Async Messaging
When using async messaging systems, generally, there are two cases for traces.
The first one is when the message broker generates one trace. Jaeger does a great job of displaying us with that one trace.
The second case, common in batch processing scenarios, is when the senders and receivers in message brokers like Kafka and AWS SQS generate multiple traces (for example, each receive starts a new trace). In this case, Jaeger will display these traces separately. That makes it more complicated to track and debug complex transactions.
More advanced backends might have an out-of-the-box solution for that and will detect it and merge those traces into a single logical flow.
2. Cross Environment Traces
If your organization works with Jaeger in production, and let’s say you want to use Jaeger to do your tests, sending your local traces to the production Jaeger is highly suboptimal.
Not only can it pollute the production environment, but it makes it difficult to find your traces within this trace jungle. By running Jaeger locally, you get an isolated playground for your tests and development.
However, one major pitfall in this scenario is that your local Jaeger will show only the part of the trace generated from your local dev station. It means you lose the context of how your traces communicate and affect downstream and upstream services (i.e., production and staging).
In this case, you won’t have an isolated and local development session while seeing the full effect of your changes across the different environments.
3. Advanced Search
Free text search on all data or based on trace attributes. For example, if you want to search a token in the payload within specific traces.
4. Trace Data Processing and Insights
Jaeger presents raw traces and highlights errors, however, generating insights based only on that data isn’t trivial and quite complex.
Examples for such insights can be API breaking change detection, aggregation of traces based on structure, parameter journey in a trace, dependency analysis, comparison to baseline from production or staging, etc.
5. Enhanced UI
Jaeger UI enumerates all attributes for a span in a long list. It does not group or organizes related data, show JSON content in a tree, highlight common data like HTTP status code, and other good stuff that makes our life easier.
The Bottom Line
Running local Jaeger offers great benefits. It is easy to set up, and when it comes to faster debug and running tests, you get this extra confidence you might want when working locally.
Jaeger is a great tool and does an awesome job answering your basic tracing needs.
However, when your work with services gets a bit more complex, or when you want to take your productivity to the next level when working locally, you might want to consider other alternatives.
If the limitations I mentioned above are a deal-breaker for you (if not now, they might be in the future), there are several vendors in the market, supplying various solutions which enhance your tracing-based workflow. One that can help you overcome all these issues is Aspecto.
Aspecto gives you everything you get with Jaeger but with enhanced UI, search, and troubleshooting capabilities for local development and debugging. It takes 2 minutes to get started with, it’s free and OpenTelemetry-based. Think of it as Jaeger and Chrome DevTools fusion for your distributed applications.
Top comments (0)