Monitoring Gin and GORM with OpenTelemetry

Still using Jaeger/Sentry? Uptrace is an OpenTelemetry tracing tool that stores data in extremely efficient ClickHouse database and provides powerful filtering and grouping.

• What is tracing?
• OpenTelemetry and Uptrace
• Creating spans
• Example application
• Configuring OpenTelemetry
• Instrumenting Gin
• Instrumenting GORM
• Recording logs
• Running the example
• What's next

OpenTelemetry and Uptrace

OpenTelemetry is a vendor-neutral API for distributed traces and metrics. You can use OpenTelemetry to collect traces, errors, and logs. Uptrace stores that data, aggregates and processes it to help you pinpoint failures and find performance bottlenecks.

Uptrace is an open source APM and blazingly fast distributed tracing tool powered by OpenTelemetry and ClickHouse. It is a popular alternative to Jaeger and can be installed by downloading a DEB/RPM package or a pre-compiled binary.

What is tracing?

OpenTelemetry Tracing allows you to see how a request progresses through different services and components, timings of every operation, any logs and errors as they occur. In a distributed environment, tracing also helps you understand relationships and interactions between distributed micro-services and systems.

Using tracing, you can break down requests into spans. Span is an operation (unit of work) your app performs handling a request, for example, a database query or a
network call.

Trace is a tree of spans that shows the path that a request makes through an app. Root span is the first span in a trace.

To learn more about tracing, see Distributed tracing using OpenTelemetry.

Creating spans

You can create spans using OpenTelemetry Go Tracing API like this:

import "go.opentelemetry.io/otel"

var tracer = otel.Tracer("app_or_package_name")

func someFunc(ctx context.Context) error {
    ctx, span := tracer.Start(ctx, "some-func")
    defer span.End()

    // the code you are measuring

    return nil
}

Example application

In this tutorial, you will be instrumenting a toy app that uses Gin router and GORM database client. You can retrieve the source code with the following command:

git clone git@github.com:uptrace/uptrace.git
cd example/gin-gorm

Configuring OpenTelemetry

Uptrace provides OpenTelemetry Go distro that configures
OpenTelemetry SDK for you. To install the distro:

go get github.com/uptrace/uptrace-go

Then you need to initialize the distro whenever you app is started:

import "github.com/uptrace/uptrace-go/uptrace"

uptrace.ConfigureOpentelemetry(
    // copy your project DSN here or use UPTRACE_DSN env var
    //uptrace.WithDSN("https://<key>@uptrace.dev/<project_id>"),

    uptrace.WithServiceName("myservice"),
    uptrace.WithServiceVersion("v1.0.0"),
)

See documentation for details.

Instrumenting Gin

You can instrument Gin router using OpenTelemetry Gin instrumentation provided by OpenTelemetry:

import (
    "github.com/gin-gonic/gin"
    "go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin"
)

router := gin.Default()
router.Use(otelgin.Middleware("service-name"))

otelgin instrumentation will save the active span in the Go context.Context. You can retrieve the context from the http.Request, extract the span from it,
and use the span to record attributes:

func (h *Handler) Index(c *gin.Context) {
    ctx := c.Request.Context()

    // Extract span from the request context.
    span := trace.SpanFromContext(ctx)

    // Check if the span was sampled and is recording the data.
    if span.IsRecording() {
        span.SetAttributes(
            attribute.String("string_key", "string_value"),
            attribute.Int("int_key", 42),
            attribute.StringSlice("string_slice_key", []string{"foo", "bar"}),
        )
    }

    otelgin.HTML(c, http.StatusOK, indexTmpl, gin.H{
        "traceURL": otelplay.TraceURL(trace.SpanFromContext(ctx)),
    })
}

Instrumenting GORM

You can instrument GORM database client using OpenTelemetry GORM instrumentation:

import (
    "github.com/uptrace/opentelemetry-go-extra/otelgorm"
    "gorm.io/gorm"
)

if err := db.Use(otelgorm.NewPlugin()); err != nil {
    panic(err)
}

After the database is instrumented, you should use WithContext method to propagate the active trace context:

user := new(User)
if err := h.db.WithContext(ctx).Where("username = ?", username).First(user).Error; err != nil {
    _ = c.Error(err)
    return
}

Recording logs

You can also record log messages using OpenTelemetry Zap instrumentation for Zap logging library:

// Create Zap logger.
log := otelzap.New(zap.NewExample())

// Extract the active context from the request.
ctx := c.Request.Context()

// Use the logger and the context to record log messages on the active span.
log.Ctx(ctx).Error("hello from zap",
    zap.Error(errors.New("hello world")),
    zap.String("foo", "bar"))

// otelzap also supports an alternative syntax.
log.ErrorContext(ctx, "hello from zap",
    zap.Error(errors.New("hello world")),
    zap.String("foo", "bar"))
}

Running the example

You can start Uptrace backend with a single command using Docker example:

docker-compose up -d

And then start the app passing Uptrace DSN as an env variable:

UPTRACE_DSN=http://project2_secret_token@localhost:14317/2 go run .

The app should be serving requests on http://localhost:9999 and should render a link to Uptrace UI. After opening the link, you should see this:

What's next?

Next, you can learn about OpenTelemetry Go Tracing to create your own instrumentations or browse existing OpenTelemetry instrumentations provided by the community.

Popular Go instrumentations:

• OpenTelemetry net/http
• OpenTelemetry gRPC
• OpenTelemetry database/sql