What is Langfuse? Observability for LLM applications

Have AI operations set up traceable and auditable? WZ-IT builds and operates the open-source LLM stack including observability - every request traceable, GDPR-compliant and from one team. See LLM hosting

A language model in production is a black box without observability: you see that it answers, but not why it answers that way - and you cannot prove it. Langfuse closes this gap. This article explains what Langfuse does, why traceability matters especially under the EU AI Act and where Langfuse sits in the open-source LLM stack. As of July 2026.

Table of contents

• The problem: AI without traceability
• What Langfuse does
• Tracing, evaluation and prompt management
• Langfuse and the EU AI Act
• Langfuse in the stack

The problem: AI without traceability

An AI application consists of more than one model call: a request runs through retrieval from documents, the assembly of the context, the actual LLM call and often several steps in sequence. When an answer is wrong, expensive or unexpected, you have to be able to trace this chain - otherwise every improvement stays guesswork.

In the enterprise a second requirement is added: provability. Whoever decides to deploy an AI system must be able to demonstrate what it did in response to which input. Both - debugging and evidence - require a recording that does not exist with pure model inference.

What Langfuse does

Langfuse is an open-source platform for the engineering and observability of LLM applications - open, self-hostable and extensible. It records what runs through an AI application and makes it visible and analyzable.

Two properties are decisive for sovereign operation: Langfuse is self-hostable (for production a Kubernetes/Helm deployment is recommended), so the recorded requests stay on your own infrastructure. And it is based on OpenTelemetry, which increases compatibility and avoids vendor lock-in - with over 100 integrations, including the OpenAI SDK, LangChain, LlamaIndex and LiteLLM.

Tracing, evaluation and prompt management

Langfuse covers three core tasks:

• Tracing/observability - every request is recorded as a trace, including LLM calls, retrieval and embeddings, across multi-step flows and conversations. This is the data foundation for everything else.
• Evaluation - the quality of the answers can be assessed systematically, for example via LLM-as-a-judge, with datasets, experiments and custom scoring.
• Prompt management - prompts are versioned, tested (through a playground) and optimized, instead of being buried in the code.

This is complemented by metrics dashboards and analyses. "The model answers somehow" thus becomes a controllable, measurable service.

Langfuse and the EU AI Act

The EU AI Act requires logging and traceability for certain AI systems. Langfuse delivers exactly this technical basis: every interaction is captured as a trace and can be analyzed, so that audit logs and evidence can be built on top of it.

Langfuse does not replace a legal assessment of whether and how the EU AI Act applies to a concrete system. But it delivers the dependable data trail without which such requirements are hardly achievable in practice - an important building block when AI should not only run but also be accountable.

Langfuse in the stack

In the open-source LLM stack, Langfuse is the observability layer. It sits crosswise to the other components: the gateway LiteLLM can be connected directly to Langfuse, so that every request running through the gateway is automatically captured as a trace - whether a vLLM inference or another model works behind it. This turns a running model into a traceable, auditable service - the prerequisite for accountable AI operation in the enterprise.