One application with agreed trace and evaluation volumes
Monthly review and evaluation runs
Findings report and prioritised recommendations
Best-effort handling within agreed service hours
Managed LLMOps: monitor, evaluate and change AI applications in a controlled way
Your AI application is in production. We make visible how well it answers, what it costs and what shifts after model or prompt changes.
Starting point
Suitable for production AI applications that change over time
- An AI, agent or RAG application is already in production or used by real users
- Models, prompts, knowledge sources or providers change regularly
- Quality, cost and latency should be visible before users complain
- Technical findings and changes need a documented ownership path
Ongoing control
Measure the behaviour of the AI application, not only availability
Monthly quality and operations report with traceable changes and recommended actions.
Traces, errors and usage
Track agreed calls, error patterns, latency, token or model cost and technical failures.Quality and regression
Use representative evaluations to make relevant changes in answer and retrieval quality visible.Controlled changes
Document and assess prompt, model, provider and retrieval changes and introduce them with a fallback path.
Understand Langfuse, tracing and evaluation as the technical basis for traceable LLM operations.
Read the guideService scope
Baseline scope: LLMOps Essential
Base scope for one application: agreed trace and eval volumes, monthly review, findings report and prioritised recommendations.
Service boundary
Not included in Essential
- Code changes and engineering allowance
- Guaranteed response time, 24/7 on-call and infrastructure consumption
Working model
Establish a baseline, then control change
Before the retainer starts, the application, telemetry, test cases and current state are documented. Later deviations can then be assessed against a confirmed baseline.
- 01
Scope application and risk
Capture the use case, data paths, models, retrieval, users and critical failure modes.
- 02
Build the baseline
Document telemetry, evaluations, metrics, versions and current behaviour.
- 03
Review continuously
Assess operating values, quality, changes and security advisories on the agreed schedule.
- 04
Control follow-up work
Prioritise findings and prepare implementation by your team or a separately commissioned WZ-IT service.
What we need from you
- A description of the AI application and its most important user paths
- An overview of models, providers, prompts, retrieval and data flows
- Access to existing telemetry or the ability to add it under control
- Business contacts and representative test or acceptance cases
Scope and investment
LLMOps and AI application operations
A one-off baseline entry and three monthly tiers. They differ in the number of applications, the review rhythm and the engineering hours.
LLMOps Readiness & Baseline
Prepare operational takeover of one production AI application and document its initial quality.
from €4,900excluding VAT and separately quoted third-party costs
- One production AI application and telemetry review
- Agreed KPI/evaluation set and baseline measurement
- Alerting and operating plan
- Baseline report and takeover prerequisites
1–2 weeks
Outcome and verification: Traceable baseline and agreed service scope before recurring operations begin.
Quoted separately: Fundamental application remediation and platform build · Ongoing operations and additional applications
Managed LLMOps Essential
Regularly assess answer quality, errors, latency and cost for one AI application.
from €1,990 / monthexcluding VAT and separately quoted third-party costs
Monthly service · One application
Outcome and verification: Monthly quality and operations report with traceable changes and recommended actions.
Managed LLMOps Professional
Support production AI applications with regular regression evaluation and planned optimisation.
from €3,990 / monthexcluding VAT and separately quoted third-party costs
- Up to three applications with agreed volume limits
- Fortnightly reviews and regression evaluations
- Four engineering hours per month for agreed optimisation
- Contractually defined response target and documented handling
Monthly service · Up to three applications
Outcome and verification: Quality reports, evidenced optimisations and verified model, prompt or data-path changes.
Quoted separately: Larger features, new integrations and major upgrades · Model/cloud consumption and on-call coverage unless expressly agreed
Managed AI Application Operations
Operate the application, connected systems and LLMOps within a documented service scope.
from €7,900 / monthexcluding VAT and separately quoted third-party costs
- Agreed application including named integrations and LLMOps
- Monitoring, incident and change processes
- Eight engineering hours per month
- Quarterly service reviews and contractually defined SLA hours
Monthly service · Agreed application and integrations
Outcome and verification: Documented incidents, changes and service reports with clear responsibilities.
Quoted separately: New applications, larger development work and major upgrades · Hardware, infrastructure/model consumption and blanket 24/7 coverage
Service hours, response targets, supported components and volume limits are agreed before takeover. Response means starting work, not guaranteed resolution. Monthly engineering allowances do not roll over and cover agreed work only. Minimum term: three months. Hardware, hosting, model usage and licences are not included in the service fee. Initial onboarding is priced separately. Essential and Professional cover AI quality; Application Operations adds the agreed application and integration operations. 24/7 on-call coverage is not automatically included.
All prices are net and exclude statutory VAT. The offers are addressed to businesses.
Clear responsibilities
Combine LLMOps with infrastructure, code maintenance and the business application
Each module remains separately available. This makes it clear whether a finding belongs to the AI application, the code or the underlying infrastructure.
Managed AI Server
Operate GPU, operating system, drivers, containers, inference runtime, monitoring and infrastructure.
View operationsAI Code Review Retainer
Have code, authentication, data, dependency and release risks reviewed by a human on an ongoing basis.
View review serviceSoftware Maintenance
Have confirmed technical changes, dependency care and controlled releases implemented on an ongoing basis.
View maintenanceFrequently asked questions about Managed LLMOps
Boundaries, tracing, evaluation, changes and ongoing operations
- LLMOps covers the controlled delivery, observation and change of applications using large language models. Depending on scope, this includes tracing, evaluations, prompt and model versions, retrieval, cost, latency, error patterns and documented response processes.
- MLOps covers training, deployment and monitoring of classic machine learning models. LLMOps addresses applications built on large language models that are usually not trained in-house: the focus is on prompts, retrieval, model versions, evaluation of generated answers, and cost and latency per request.
- LLM observability is the monitoring part of LLMOps: every request is captured as a trace with prompt, retrieved context, model answer, token usage, latency and errors. LLMOps uses this data to assess quality with evaluations and to introduce changes in a controlled way.
- Depending on the application: answer quality from evaluations, quality of retrieved documents in RAG, latency, errors and aborts, token and model cost per request, and usage. Which metrics apply and at what deviation action is taken is defined in the baseline.
- Self-hosted, yes. The open-source edition of Langfuse is MIT-licensed and runs on your own infrastructure, for example on a server in Germany; traces then do not go to a cloud service. Traces contain prompts and answers and therefore often personal data. The open-source edition masks such data client-side before it reaches Langfuse; data retention and server-side masking within Langfuse itself require the Enterprise edition. We clarify which variant is needed in the baseline.
- Managed LLMOps looks at the behaviour of the AI application: traces, quality, retrieval, models, prompts, cost and latency. AI server maintenance and operations cover the technical infrastructure underneath, such as GPU, operating system, drivers, containers and inference runtime.
- No. Langfuse is one possible open-source foundation for tracing and evaluation. The telemetry used or retained depends on the existing stack and requirements for data storage and integration.
- Managed LLMOps prioritises and documents findings. Code changes, infrastructure work or extensive RAG optimisation are handled by your team or through a separately agreed WZ-IT service.
- No. Representative evaluations and monitoring can reveal quality changes and defined error classes, but they cannot provide a general guarantee that every answer to an unknown case will be correct.
- from €1,990 excl. VAT / month. The published price covers the stated standard scope. We check data quality, interfaces, access permissions and the target environment before commissioning. Extensions are priced separately.
- No. A defined scope and a suitable environment can allow direct implementation. Existing architecture, data pipelines and pilot results are reused after review.
Reviews & projects
What clients say about working with us
From local AI integration to architecture, data sovereignty and ongoing operations.
WZ-IT moved our studio infrastructure from decentralised individual devices to a central platform: every site is securely connected via VPN, new devices are onboarded automatically and an entire site is provisioned from a template, without manual steps on location. What impressed me most is the breadth and depth of their knowledge: Timo and Robin are not a typical IT provider who sets up a server and leaves. The two of them think their way into highly complex infrastructure and software topics, work through every requirement we put in front of them, and build networking, provisioning and operations so that everything fits together in the end. WZ-IT is an excellent partner for complex software, network and architecture projects.
Steve KirchnerManaging Director, nextGYM GmbH
International
Built in Germany's Ruhr Valley. Running worldwide.
WZ-IT designs, develops and operates infrastructure and software for clients in Germany and internationally. We deliver projects remotely and continue supporting them in ongoing operations after go-live.
Selected projects
- LiveRemote access platform for distributed water treatment plantsABCO Water Systems · Australia · Secure Access & OT
- 81%AWS migration with substantially lower operating costsEVA Real Estate · UAE · Cloud exit & infrastructure
- 98%Provision new sites in minutes instead of hoursnextGYM GmbH · Germany · Automation & IoT
Read client reviews
- Integrate a local AI solution
- Modernize your infrastructure - sovereign
- Design an open-source AI architecture
- Plan a sovereign open-source stack
- Secure your Proxmox & backup setup
- Modernize your legacy software
- Cut cloud cost - up to −81%
- Build a high-availability Proxmox cluster
- Virtualize with Managed Proxmox
- Get collaboration fully managed
- Ship your prototype to production
- Connect sites and clusters securely
Discuss LLMOps
Which AI application should be monitored?
Tell us the use case, models, retrieval, operating location and existing telemetry. We will assess the baseline and retainer scope.
