AI Guardrails Compared: NeMo Guardrails, Llama Guard, Guardrails AI, LLM Guard and Presidio

Keep an AI application measurable in operation? Managed LLMOps observes tracing, errors, cost and latency of an AI application (for example with Langfuse), uses evaluations and documents changes to prompt, model, provider and retrieval with a fallback path. From €1,990 excl. VAT per month (as of October 2026). See Managed LLMOps

Guardrails are check and control rules around a language model: they check inputs, outputs, retrieved content and tool calls against a policy. They are a layer next to permissions and architecture, not a replacement for them. The comparison of NeMo Guardrails, Llama Guard, Guardrails AI, LLM Guard and Presidio by licence, maintenance status, German, latency and integration follows below. As of October 2026. Four findings matter for the selection: LLM Guard is archived, Guardrails AI was acquired by Harvey in September, Llama Guard 4 is tied to a licence with an EU clause, and NeMo Guardrails sends telemetry by default.

Table of contents

• What guardrails are and what they are not
• The tools at a glance
• Four findings on maintenance, licence and outbound connections
• German, latency and integration
• Decision aid
• Policy as code
• Testing guardrails
• Operation and monitoring with Langfuse
• Our approach at WZ-IT
• Status and methodology
• Further guides

What guardrails are and what they are not

NVIDIA draws the line itself in the security guidelines of NeMo Guardrails: the model is treated as a browser under the complete control of the user, its outputs as untrusted, and every invoked service is called in the context of the user. According to the runtime security FAQ, no single rail guarantees protection against every attack. Filters alone do not carry the load (Protection against prompt injection); the measures that do are described in AI agents: permissions and approvals and RAG with permissions.

The tools at a glance

Generative classifiers such as Llama Guard 3-8B and 4-12B are language models with 8 to 12 billion parameters; smaller variants such as Llama Guard 3-1B or Qwen3Guard-Gen-0.6B exist. Every call is an additional model run.

NeMo Guardrails in brief

NeMo Guardrails places rails between the application and the model. Input rails can reject or modify a request, for example to mask personal data. Dialog rails control in Colang which action or answer follows. Retrieval rails check retrieved passages, execution rails the inputs and outputs of tools, output rails the model's answer. A configuration without rails passes requests to the model unchanged. Colang 1.0 is the default, Colang 2.0 optional (README).

Four findings on maintenance, licence and outbound connections

LLM Guard is archived

The README of the repository states that the project and the associated Hugging Face models are no longer developed or maintained. The Open WebUI example pipeline depends on this library.

Guardrails AI was acquired by Harvey

Harvey announced the acquisition on 9 September 2026. Harvey's announcement contains no statement about the further development of the open-source framework. The repository remains under Apache-2.0 and had a push on 27.10.2026. According to the project's announcement of 6 July 2026, the hub installation procedure (guardrails hub install) with the private validator registry and the hosted remote inference were scheduled for shutdown on 25 August 2026 (the README speaks of a "planned cutoff"); the same text also names 6 August 2026 for the hosted inference. Whether the shutdown has taken place is not shown by the primary sources; the issue is still open on 28.10.2026. According to the migration guide, validators are standalone PyPI packages (pip install guardrails-ai-<name>, 50 of 64 at the time of the announcement), and models run locally (use_local=True) or at your own endpoint. guardrails configure enables anonymous metrics by default, and --disable-metrics switches them off (source code).

Llama Guard 4 and the EU clause

The Llama Guard 4 folder contains the Llama 4 Community License and the acceptable use policy. For multimodal Llama 4 models, the policy does not grant the rights under section 1(a) to individuals domiciled in, or companies with their principal place of business in, the EU; end users of a product or service that incorporates such a model are excepted. According to its model card, Llama Guard 4 is natively multimodal. Whether the clause covers operation as a guard model is for the legal department to assess before use (LLM licences). The same policy also ships with the text classifier Prompt Guard 2 (mDeBERTa base), which is not a multimodal model.

NeMo Guardrails sends telemetry

The README describes an anonymous ping to NVIDIA at start and periodically afterwards. The payload contains no prompts, completions, model names, endpoints or IP addresses, but the ping itself is an outbound connection. It can be switched off before start with NEMO_GUARDRAILS_NO_USAGE_STATS=1 or DO_NOT_TRACK=1. The documentation also notes: NIM containers need an NGC key to download, and examples use the hosted NVIDIA endpoint (NVIDIA Build), which according to the README is for evaluation and testing only, without confidential data or personal data. For the egress list see GDPR-compliant AI, question 5.

German, latency and integration

German

Meta evaluates Prompt Guard 2 on German, the 22M model with a larger gap. Llama Guard 3 supports German. In its default configuration, Presidio contains recognisers and models for English. For German you add a German NLP model for names and the German recognisers, for example for tax ID, identity card and insurance numbers. The example pipeline in the Open WebUI Pipelines project sets language to en and lists US entities, so it has to be adapted.

Latency

NVIDIA writes in its FAQ that there is no deployment-independent p50, p95 or throughput figure. Meta reports 92.4 ms (86M) and 19.3 ms (22M) per classification for Prompt Guard 2 on an A100 at 512 tokens. Levers according to the FAQ: parallel rails (rails.input.parallel, rails.output.parallel), speculative generation in IORails, which runs the input check in parallel with the response and discards the response if the check blocks, and stream_first: False for streaming, which checks outputs before delivery but costs latency.

Integration with LiteLLM and Open WebUI

LiteLLM runs guardrails from config.yaml in the modes pre_call, during_call, post_call and logging_only, with Presidio, Guardrails AI and other providers. The Generic Guardrail API (beta) connects your own service that answers BLOCKED, NONE or GUARDRAIL_INTERVENED. Assignment per API key, tag-based modes, model-level guardrails, dynamic parameters per call and blocking teams from changing guardrails require an Enterprise licence according to the documentation. In Open WebUI, Filter Functions implement the rules with inlet, request, stream and outlet. According to the documentation, enforcement belongs in request because it runs before every model call, including tool loops. An exception in inlet or in the first request call of a turn aborts the turn, whereas in outlet it is usually only logged. A filter runs as Python code on the server and can only be created by administrators. The Pipelines project itself advises against Pipelines for basic filters.

Decision aid

Rule: one building block per task, each with test cases and a metric. Six points are checked before adoption:

1. Licence: code, model weights and usage policy, not just the repository label.
2. Maintenance: latest version, archival, owner of the project.
3. Self-hosting and outbound connections: does everything run on your own infrastructure? Telemetry, model downloads, hosted endpoints, entered in the egress list.
4. German: own test cases, not vendor statements.
5. Latency: p50 and p95 of your own configuration on real traffic, including streaming.
6. Failure behaviour: a decision whether a failed check opens or closes, tested with timeouts and errors.

Policy as code

The policy exists as text, the rules as configuration in the repository, the test cases next to them. Maintenance effort arises with thresholds and rules (false alarms), with test cases (new attack patterns, new data sources) and with dependencies (model and library versions, project status). NeMo Guardrails separates config.yml (models, active rails), Colang files (flows) and actions.py (custom checks); LiteLLM uses config.yaml. One possible layout, with the structure of config following the NeMo Guardrails README:

guardrails/
  POLICY.md           policy in text form, with version
  config/config.yml   models, active rails, thresholds
  config/*.co         Colang flows for topics and sequences
  config/actions.py   custom checks
  tests/cases.jsonl   attack, legitimate and German cases with expected result

Excerpt from the documentation for masking personal data:

rails:
  config:
    sensitive_data_detection:
      input:
        entities:
          - PERSON
          - EMAIL_ADDRESS
  input:
    flows:
      - mask sensitive data on input

The NVIDIA FAQ recommends versioning library, configuration, prompts, custom actions and external model versions together and releasing them only after a regression test, with staged rollout and rollback.

Testing guardrails

You test the configuration, not the tool: attack cases direct and indirect (instructions in documents and tool outputs), legitimate cases with technical terms and code, personal data cases and German cases. You measure the block rate for attacks and the false block rate for legitimate cases per category, with thresholds agreed in advance, as a gate on every change to model, library, configuration and prompt. NeMo Guardrails offers check() and nemoguardrails evaluate for this, garak (Apache-2.0) scans specifically for vulnerabilities, and Langfuse datasets keep the test set versioned.

The trade-off is shown by a study of the built-in guardrails of three cloud platforms (Unit 42, 2 June 2025) with 1,000 benign and 123 malicious prompts, with the filters at their strictest setting (percentages as stated in the study):

Similar detection rates (91 and 92 percent) come with very different false block rates (0.6 and 13.1 percent). The detection rate alone says little; both values belong in the same measurement. Rounding: 114 of 123 is arithmetically 92.7%, while the study states "approximately 92%". For platform 3 the running text of the study elsewhere gives 14.0% of benign prompts blocked in error, whereas table 1 and the calculation (131 of 1,000) give 13.1%. A regression test of your own configuration is not a penetration test.

Operation and monitoring with Langfuse

Every rail decision should be logged with rule name, version, result (in NeMo PASSED, MODIFIED or BLOCKED) and duration; according to the NVIDIA FAQ, the application adds the configuration version and the reason for the decision. Langfuse breaks down the latency of the checks within a trace; the documentation recommends using it to assess whether a check is worth the wait. Scores and annotation queues support sampling for false alarms. Personal data in traces can be redacted with masking. Failures of the check itself (timeout, error) are logged as their own event.

Our approach at WZ-IT

Managed LLMOps observes tracing, errors, latency and cost of an application, uses evaluations to make changes in quality visible, and documents changes to prompt, model, provider and retrieval with a fallback path. Essential covers one application with agreed trace and evaluation volumes and best-effort handling within the agreed service hours, from €1,990 excl. VAT per month. The measures that carry the load, permissions before the search and a narrow tool catalogue with approvals, are described in Protection against prompt injection. Prices excl. VAT, as of October 2026.

Status and methodology

• Status: 28.10.2026. Licences, versions and maintenance status come from the LICENSE files, READMEs, releases and model cards of the repositories; the sources were retrieved on that day.
• Not measured: There are no benchmarks of our own. Latency figures come from vendors (Meta); the Unit 42 detection rates concern cloud platforms (evaluation before 10 March 2025), not the tools compared here, and come from a vendor of security products.
• Licences: The Llama licences have to be read per model. The model page of Llama Guard 3-8B names the Llama 3.1 Community License, while the folder in the repository contains licence texts of other Llama versions. Whether the EU clause applies to Llama Guard 4 is a legal question and is not decided here. This article is not legal advice.
• Guardrails AI: The statements on the hub and inference follow the project's announcement (the dates there are inconsistent: 6 and 25 August 2026); whether the shutdown has taken place is not documented. Harvey's announcement contains no statement on its plans for the framework.

Further guides

• Protection against prompt injection - why filters alone do not carry the load
• AI agents: permissions and approvals - identity, tool catalogue, approvals
• Measuring RAG quality - test set, regression test, Langfuse
• What is LiteLLM? - the gateway in which guardrails can run
• LLM licences for commercial use - Llama, Qwen and other licences

Keep an AI application measurable in operation? Managed LLMOps makes quality, cost and latency of an AI application visible and documents changes with a fallback path. Managed LLMOps · LLM hosting