NeMo Guardrails vs Guardrails AI: traces, metrics and exported data

The choice of NeMo Guardrails vs Guardrails AI usually turns on the API, but an SRE or platform team looks at it differently: what is visible about each guardrail once it runs in production? Both libraries are open source under Apache 2.0 and sit between the application and the model. Yet they do not expose the same OpenTelemetry signals, nor at the same level of maturity.

This article does not compare detection quality, which no official source here can settle. It reads the documentation and code of both repositories to answer three operational questions: which spans and metrics come out, how a block can be tied to a request, and which sensitive data may travel to the backend. For the wider picture of controls, see the overview of AI agent guardrails in production.

Versions read: the NeMo Guardrails README gives 0.24.1 as the latest release, while the development branch shows 0.25.0-rc1 in pyproject.toml; Guardrails AI is at 0.11.0 in its pyproject.toml (repositories consulted on 31 October 2026). The content applies to any OpenTelemetry backend, including Dynatrace SaaS and Dynatrace Managed, with no specific integration documented by either project.

NeMo Guardrails vs Guardrails AI: two integration models

NeMo Guardrails describes itself as a toolkit of programmable guardrails between application code and the LLM. It is set up in a configuration folder (a config.yml file, input, output and dialog rails), then called through LLMRails or, for the newer engine, Guardrails. It can also run as a server (nemoguardrails server).

Guardrails AI presents itself as a Python framework that runs input and output Guards and can also produce structured data. A Guard object is built in code and validators are added with .use(); each validator picks an action on failure through OnFailAction (reask, fix, filter, refrain, noop, exception, fix_reask, custom, according to the source code).

The operational consequence is simple. With NeMo Guardrails, policy lives in version-controlled files, reviewed like any configuration. With Guardrails AI, it lives in the application’s Python code: changing a threshold means a deployment.

Traces: what each library emits

NeMo Guardrails

Tracing needs the tracing extra (pip install nemoguardrails[tracing]), the OpenTelemetry SDK configured by the application, and a tracing block in the configuration. The library depends on the OpenTelemetry API only: without a TracerProvider, spans are silently dropped, as the documentation states.

The default engine, LLMRails, produces one tree per request: guardrails.request (SERVER), then one guardrails.rail span per activated rail, one guardrails.action per action and a CLIENT span per LLM call. The attributes an SRE needs are rail.type, rail.name, rail.stop and rail.decisions, plus the GenAI attributes of the call (gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, gen_ai.request.model). A blocked request can therefore be found through rail.stop.

One detail matters: LLMRails rebuilds its spans after the request ends, from the interaction log. Durations reflect what happened, but the spans do not exist while the request runs. The opt-in IORails engine creates spans during the request; the documentation labels it experimental, with names that may change.

Guardrails AI

The guardrails/telemetry module creates one guard span per call, one step span per iteration and one <validator name>.validate span per validation, using a tracer named guardrails-ai. Attributes include validation_passed, validator.name and validator.on_fail. The code also exposes a disable_tracing setting.

For export, a default_otlp_tracer function sends spans over OTLP/HTTP when OTEL_EXPORTER_OTLP_PROTOCOL and an endpoint (OTEL_EXPORTER_OTLP_ENDPOINT or OTEL_EXPORTER_OTLP_TRACES_ENDPOINT) are set; otherwise it writes to the console (stderr). An application that already configures its own TracerProvider can skip it: the code obtains its tracer through trace.get_tracer.

Metrics: the sharpest difference

NeMo Guardrails documents a catalogue of OpenTelemetry metrics, with a significant caveat: they exist only on the experimental IORails engine, and LLMRails emits none. To enable them, set metrics.enabled: true and configure a MeterProvider before creating the engine. The reference page lists, among others:

• guardrails.requests.blocked (counter, rail.type label set to Input or Output): the basis of a block rate;
• guardrails.request.duration (histogram in seconds, bucket bounds from 5 ms to 10 s);
• guardrails.requests.errors with the error.type label;
• saturation metrics: admission queue, active requests, rejections (guardrails.nonstream.rejections).

The nemoguardrails server command can expose a Prometheus scrape endpoint, but the documentation states that it publishes only three admission metrics (queued, active, rejections) and drops the others. To export everything, the export has to be configured in the application.

Guardrails AI, in the files read, defines no counters or histograms of its own: only spans are emitted. A per-validator failure rate is therefore derived from traces (the validation_passed attribute and the validator name), through a pipeline that turns spans into metrics, for example in the contrib Collector. This is a reading of the code, not a claim that nothing exists anywhere in the project’s documentation.

Sensitive data in telemetry

This criterion should weigh most in a security review. NeMo Guardrails does not record prompt and response content by default: capture is opt-in, through tracing.enable_content_capture or the OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT variable, which overrides the configuration in both directions. The documentation warns that this content may include personal data and describes specific behaviour with masking rails.

In the Guardrails AI code, span attributes carry the validated value: validator.validate.input.value for the validator, input.value and output.value for the step. No opt-in capture option appears in the files read. A Collector configured to drop or hash these attributes before export then becomes the right place to enforce a retention policy.

There is also the project’s own telemetry: Guardrails AI contains « Hub » telemetry whose activation depends on allow_metrics_collection or the guardrails configure setting; the code describes it as anonymous metrics. The README also announces, on 6 July 2026, that validators are moving to PyPI packages installable with pip and that hosted remote inferencing is being discontinued, with a planned cutoff of 25 August 2026. Validators that relied on that service need checking before production.

How to check that it works

1. Run the NeMo Guardrails documentation quick start with ConsoleSpanExporter and look for guardrails.request and guardrails.rail.
2. For Guardrails AI, trigger a failure on purpose (the README example with OnFailAction.EXCEPTION) and look for the …validate span whose validation_passed is false.
3. Then switch to OTLP export to the Collector and check that no prompt or response reaches the backend unless content capture is intended.

Best practices

• Set a latency budget (SLO) for the guardrail layer, then measure it with span durations, not a single test.
• Compute the block rate per rail type or validator, never as a global aggregate.
• Treat spans as sensitive data: filtering in the Collector, short retention.
• Pin versions and read the changelog: IORails is experimental on the NeMo side, and metric names may change.
• Tie guardrail spans to token costs, see the cost of an AI agent in tokens with OpenTelemetry.

Frequently asked questions

Which library should a platform team choose?

The sources read do not name a winner. NeMo Guardrails offers file-based configuration and a (experimental) metrics catalogue; Guardrails AI offers validators in code and detailed spans. The right choice depends on the need for central configuration and the level of control wanted over exported data.

Can both be used together?

Nothing in the documentation read forbids it, but no joint integration is described: this point is not verified. Each layer adds latency and its own spans, to be measured separately.

How can the block rate be tracked?

With NeMo Guardrails IORails, use guardrails.requests.blocked divided by guardrails.requests. With Guardrails AI, count validator spans whose validation_passed is false in the Collector. The GenAI semantic conventions are still evolving: names may change.

Conclusion

Both projects speak OpenTelemetry, but one ships metrics on an experimental engine and the other content-rich spans. For an SRE team, the decision rests less on detection than on three points: where policy lives, which metrics come out without a pipeline, and which data leaves the application. For a coding agent in the enterprise, see also monitoring a coding agent. Sources: NVIDIA-NeMo/Guardrails and guardrails-ai/guardrails.

This blog is independent and is not affiliated with Dynatrace LLC, NVIDIA or Guardrails AI. Product names belong to their owners; technical content comes from the cited official repositories.

---

Originally published on span.observer. French version: https://span.observer/fr/nemo-guardrails-vs-guardrails-ai/.