[email protected]

Managed LLMOps: monitor, evaluate and change AI applications in a controlled way

Your AI application is in production. We make visible how well it answers, what it costs and what shifts after model or prompt changes.

Reviews
Cloud Wolke HerausforderungServer NachhaltigkeitIT Beratung Service Consulting SoforthilfeTimo Wevelsiep Robin ZinsExperten für Innovation Migration AWSHetzner Hosting zuverlässig

Companies worldwide trust WZ-IT

  • Stadtwerke Brühl
  • DGHO e.V.
  • solarwerkstatt berlin
  • Krüger Dirndl
  • ABCO Water Systems
  • Golem.de
  • EVADXB
  • nextGYM
  • AInergy
  • ml&s
  • Odiseo Solutions
  • Annota
  • ARGE
  • SweetConnect GmbH
  • Aphy AG
  • CORGOS
  • Rekorder
  • SolidProof
  • Yonju
  • Keymate
  • Paritel
  • Mr. Clipart
  • Millenium
  • Negosh
  • Führerscheinmacher
  • BTI
  • Deutsche Gesellschaft für Datenschutz
Read client reviews

Starting point

Suitable for production AI applications that change over time

  • An AI, agent or RAG application is already in production or used by real users
  • Models, prompts, knowledge sources or providers change regularly
  • Quality, cost and latency should be visible before users complain
  • Technical findings and changes need a documented ownership path

Ongoing control

Measure the behaviour of the AI application, not only availability

Monthly quality and operations report with traceable changes and recommended actions.

  • Traces, errors and usage

    Track agreed calls, error patterns, latency, token or model cost and technical failures.
  • Quality and regression

    Use representative evaluations to make relevant changes in answer and retrieval quality visible.
  • Controlled changes

    Document and assess prompt, model, provider and retrieval changes and introduce them with a fallback path.

Understand Langfuse, tracing and evaluation as the technical basis for traceable LLM operations.

Read the guide

Service scope

Baseline scope: LLMOps Essential

Base scope for one application: agreed trace and eval volumes, monthly review, findings report and prioritised recommendations.

Included
  • One application with agreed trace and evaluation volumes

  • Monthly review and evaluation runs

  • Findings report and prioritised recommendations

  • Best-effort handling within agreed service hours

Service boundary

Not included in Essential

  • Code changes and engineering allowance
  • Guaranteed response time, 24/7 on-call and infrastructure consumption

Working model

Establish a baseline, then control change

Before the retainer starts, the application, telemetry, test cases and current state are documented. Later deviations can then be assessed against a confirmed baseline.

  1. 01

    Scope application and risk

    Capture the use case, data paths, models, retrieval, users and critical failure modes.

  2. 02

    Build the baseline

    Document telemetry, evaluations, metrics, versions and current behaviour.

  3. 03

    Review continuously

    Assess operating values, quality, changes and security advisories on the agreed schedule.

  4. 04

    Control follow-up work

    Prioritise findings and prepare implementation by your team or a separately commissioned WZ-IT service.

What we need from you

  • A description of the AI application and its most important user paths
  • An overview of models, providers, prompts, retrieval and data flows
  • Access to existing telemetry or the ability to add it under control
  • Business contacts and representative test or acceptance cases

Scope and investment

LLMOps and AI application operations

A one-off baseline entry and three monthly tiers. They differ in the number of applications, the review rhythm and the engineering hours.

LLMOps Readiness & Baseline

Prepare operational takeover of one production AI application and document its initial quality.

from €4,900excluding VAT and separately quoted third-party costs
  • One production AI application and telemetry review
  • Agreed KPI/evaluation set and baseline measurement
  • Alerting and operating plan
  • Baseline report and takeover prerequisites

1–2 weeks

Outcome and verification: Traceable baseline and agreed service scope before recurring operations begin.

Quoted separately: Fundamental application remediation and platform build · Ongoing operations and additional applications

Managed LLMOps Essential

Regularly assess answer quality, errors, latency and cost for one AI application.

from €1,990 / monthexcluding VAT and separately quoted third-party costs

Monthly service · One application

Scope as described above

Outcome and verification: Monthly quality and operations report with traceable changes and recommended actions.

Managed LLMOps Professional

Support production AI applications with regular regression evaluation and planned optimisation.

from €3,990 / monthexcluding VAT and separately quoted third-party costs
  • Up to three applications with agreed volume limits
  • Fortnightly reviews and regression evaluations
  • Four engineering hours per month for agreed optimisation
  • Contractually defined response target and documented handling

Monthly service · Up to three applications

Outcome and verification: Quality reports, evidenced optimisations and verified model, prompt or data-path changes.

Quoted separately: Larger features, new integrations and major upgrades · Model/cloud consumption and on-call coverage unless expressly agreed

Managed AI Application Operations

Operate the application, connected systems and LLMOps within a documented service scope.

from €7,900 / monthexcluding VAT and separately quoted third-party costs
  • Agreed application including named integrations and LLMOps
  • Monitoring, incident and change processes
  • Eight engineering hours per month
  • Quarterly service reviews and contractually defined SLA hours

Monthly service · Agreed application and integrations

Outcome and verification: Documented incidents, changes and service reports with clear responsibilities.

Quoted separately: New applications, larger development work and major upgrades · Hardware, infrastructure/model consumption and blanket 24/7 coverage

Service hours, response targets, supported components and volume limits are agreed before takeover. Response means starting work, not guaranteed resolution. Monthly engineering allowances do not roll over and cover agreed work only. Minimum term: three months. Hardware, hosting, model usage and licences are not included in the service fee. Initial onboarding is priced separately. Essential and Professional cover AI quality; Application Operations adds the agreed application and integration operations. 24/7 on-call coverage is not automatically included.

All prices are net and exclude statutory VAT. The offers are addressed to businesses.

Frequently asked questions about Managed LLMOps

Boundaries, tracing, evaluation, changes and ongoing operations

Reviews & projects

What clients say about working with us

From a knowledge base with an AI assistant and local AI integration to data sovereignty and ongoing operations.

AI Platform

We were looking for a secure yet easy-to-use alternative to the large American providers and came across WZ-IT. WZ-IT advised us on choosing the right models and implemented the integration for us. Beforehand, we were able to try everything out calmly on a test instance. Today WZ-IT runs Open WebUI for us on our own domain, with a convenient Microsoft login and open-source models hosted by a German provider. Our team can now work productively with AI without resorting to external tools or shadow AI. Working together is straightforward, and questions are answered quickly and clearly. With WZ-IT we also have a partner at our side for further AI projects. (translated)
Carl-Frederic KornHead of E-Commerce & Online Marketing, Krüger Dirndl GmbH
See the AI platform

Knowledge Management & AI

The knowledge from many years in the solar industry lived in the head of our former managing director, and it still lives in the heads of our employees today. We wanted to secure it, not as a folder full of notes, but in a way that lets us ask questions in daily work and get reliable answers. WZ-IT advised us from the start and weighed different approaches and knowledge management systems against each other with us before we decided. What was built is a custom knowledge base with an AI assistant that brings together conversations, notes, documentation and manuals: from the platform through onboarding our team to support in operation. Conversations are transcribed and facts are extracted; we review and correct everything ourselves before it is stored, because we do not want to trust an AI blindly. The system runs in our own environment, and the data stays with us. What surprised us was how efficient it was: WZ-IT used existing open-source software and extended it specifically for our requirements, instead of selling an expensive custom build from scratch, as is often usual with IT service providers. Timo and Robin understood our actual problem, preserving knowledge during the generational handover, and implemented it step by step: from the proof of concept through setup and validation with our own data to productive operation, explaining everything clearly along the way. Next, we want to extend this to the knowledge of our employees, so that this does not depend on individual people either.
Joshua von AmelnManaging Director, solarwerkstatt berlin GmbH
See knowledge transfer

Read client reviews

International

Selected projects

WZ-IT designs, develops and operates infrastructure and software for clients in Germany and internationally. We deliver projects remotely and continue supporting them in ongoing operations after go-live.

Discuss LLMOps

Which AI application should be monitored?

Tell us the use case, models, retrieval, operating location and existing telemetry. We will assess the baseline and retainer scope.

Timo and Robin, founders of WZ-IT

Choose your starting point

The enquiry is non-binding. We only need technical access after the scope has been confirmed.

We usually respond within one business day.