Have AI operations set up traceable and auditable? WZ-IT builds and operates the open-source LLM stack including observability - every request traceable, GDPR-compliant and from one team. See LLM hosting
A language model in production is a black box without observability: you see that it answers, but not why it answers that way - and you cannot prove it. Langfuse closes this gap. This article explains what Langfuse does, why traceability matters especially under the EU AI Act and where Langfuse sits in the open-source LLM stack. As of October 2026.
Table of contents
- The problem: AI without traceability
- What Langfuse does
- Tracing, evaluation and prompt management
- Langfuse and the EU AI Act
- Langfuse in the stack
The problem: AI without traceability
An AI application consists of more than one model call: a request runs through retrieval from documents, the assembly of the context, the actual LLM call and often several steps in sequence. When an answer is wrong, expensive or unexpected, you have to be able to trace this chain - otherwise every improvement stays guesswork.
In the enterprise a second requirement is added: provability. Whoever decides to deploy an AI system must be able to demonstrate what it did in response to which input. Both - debugging and evidence - require a recording that does not exist with pure model inference.
What Langfuse does
Langfuse is an open-source platform for the engineering and observability of LLM applications - open, self-hostable and extensible. It records what runs through an AI application and makes it visible and analyzable.
Two properties are decisive for sovereign operation: Langfuse is self-hostable (for production a Kubernetes/Helm deployment is recommended), so the recorded requests stay on your own infrastructure. And it is based on OpenTelemetry, which increases compatibility and avoids vendor lock-in - with over 100 integrations, including the OpenAI SDK, LangChain, LlamaIndex and LiteLLM.
Current state (October 2026): version 4 has been self-hostable since 29 July 2026 (release v4.0.0). It requires at least ClickHouse 25.12, PostgreSQL 15 and Redis 7.0; version 3 receives security patches until the end of January 2027 (upgrade guide). Since January 2026 Langfuse has belonged to ClickHouse, the maker of the database its trace analysis is built on; according to the announcement, licensing and self-hosting remain unchanged and the core stays MIT-licensed. The article self-hosted Langfuse and the EU AI Act covers the upgrade path and operations.
Tracing, evaluation and prompt management
Langfuse covers three core tasks:
- Tracing/observability - every request is recorded as a trace, including LLM calls, retrieval and embeddings, across multi-step flows and conversations. This is the data foundation for everything else.
- Evaluation - the quality of the answers can be assessed systematically, for example via LLM-as-a-judge, with datasets, experiments and custom scoring.
- Prompt management - prompts are versioned, tested (through a playground) and optimized, instead of being buried in the code.
This is complemented by metrics dashboards and analyses. "The model answers somehow" thus becomes a controllable, measurable service.
Langfuse and the EU AI Act
The EU AI Act requires logging and traceability for certain AI systems; for high-risk systems under Annex III these obligations apply from 2 December 2027 under the Digital Omnibus (Regulation (EU) 2026/1744). Langfuse delivers exactly this technical basis: every interaction is captured as a trace and can be analyzed, so that audit logs and evidence can be built on top of it.
Langfuse does not replace a legal assessment of whether and how the EU AI Act applies to a concrete system. But it delivers the dependable data trail without which such requirements are hardly achievable in practice - an important building block when AI should not only run but also be accountable.
Langfuse in the stack
In the open-source LLM stack, Langfuse is the observability layer. It sits crosswise to the other components: the gateway LiteLLM can be connected directly to Langfuse, so that every request running through the gateway is automatically captured as a trace - whether a vLLM inference or another model works behind it. This turns a running model into a traceable, auditable service - the prerequisite for accountable AI operation in the enterprise.
