Can I bring my own model?

Yes, if vLLM supports the model and its licence allows us to operate it. Common families such as Qwen, Gemma, Mistral, DeepSeek and gpt-oss qualify. We check licence, size and the right GPU before the proposal and support you in converting or quantising your own models.

Absolutely. You can upgrade to a more powerful GPU at any time (e.g. from RTX PRO 4000 to RTX PRO 6000) or add additional servers for load balancing. We offer both vertical and horizontal scaling.

Which LLMs are supported?

Models that vLLM supports and whose licence allows us to operate them, for example gpt-oss, Qwen3.8 and Qwen3.5, Gemma 4, Mistral Small 4, DeepSeek-V4-Flash, EuroLLM and Llama 3.3. Meta's use policy excludes companies based in the EU from the multimodal Llama 4 models, so we do not operate them. We check which model fits which GPU before the proposal. As of October 2026.

How fast is inference?

The managed GPU server package uses vLLM on an RTX PRO 4000 Blackwell (24 GB) or RTX PRO 6000 Blackwell (96 GB). With vLLM, significantly higher throughput is possible under high load; in its own benchmarks the vLLM project measures up to 24x more throughput compared to HuggingFace Transformers. Exact performance depends on the model, context length and the number of simultaneous requests.

Yes, both Ollama and vLLM provide OpenAI-compatible APIs. You can easily switch your applications by changing the API URL - the code remains almost identical. Additionally, there are native client libraries for Python, JavaScript, Go and more languages.

What happens in case of technical problems?

Our monitoring usually detects problems before they have an impact. In case of disruptions, we respond immediately. As a managed service customer, you have priority support via email, with optional phone/video support available.

Can I host multiple models simultaneously?

Yes, depending on the available VRAM you can load multiple models in parallel or switch between them. Ollama manages this automatically. On the Pro configuration with 96 GB VRAM you can, for example, keep a 27B and a 7B model loaded at the same time.

We keep the system (OS, CUDA, Ollama/vLLM) up-to-date and install security updates promptly. Model updates are done on request - you decide when you want to use a new model release.

What does LLM hosting cost at WZ-IT?

A managed GPU server with one GPU costs €699 excl. VAT / month (24 GB) or €1,799 excl. VAT / month (96 GB), each plus a one-time setup fee and cancellable monthly. Servers with two to four GPUs are available on request. There is no billing per token; model, vLLM, Open WebUI and operations are included.

How can I host an LLM myself?

You need a GPU with enough memory for the model weights and context, a model server such as vLLM, an interface or API for users, and someone who takes care of drivers, updates and monitoring. Our guide to the open-source LLM stack describes the setup; you can hand operations over to us.

Is LLM hosting in Germany GDPR-compliant?

Operation in a German data centre under a data processing agreement removes the hard parts: no third-country transfer and no data sent to the model vendor. Legal basis, transparency and, where needed, a data protection impact assessment remain your responsibility; we provide the technical and contractual basis.

LLM hosting in Germany: run open-source LLMs on a dedicated GPU

You want to host a language model yourself, without sending data to cloud APIs and without a team of your own for GPU, vLLM and updates. We run open-source LLMs such as Qwen, gpt-oss or Mistral on dedicated GPU servers in Germany, with an OpenAI-compatible API.

Qwen · gpt-oss · Mistral
OpenAI-compatible API
Ollama & vLLM

Supported models

Reviews

IT Beratung Service Consulting Soforthilfe

Companies worldwide trust WZ-IT

Read client reviews

Why local

Why host an LLM yourself? Reasons for operating in Germany

Full control over your AI infrastructure

Data Protection & GDPR
German locations can be selected for models, storage and processing. External integrations, remote access and subprocessors are documented so that the specific privacy assessment remains traceable.
Data Sovereignty
You determine the models, interfaces and data flows. Unneeded external model APIs can be omitted; enabled integrations and outbound connections are documented transparently.
Compliance & Audits
Meet strict compliance requirements in regulated industries such as healthcare, finance or public administration.
Cost Efficiency
No hidden API costs, no surprises when it comes to billing. Predictable monthly fixed costs instead of pay-per-token with cloud providers.
Performance & Latency
Dedicated GPU resources without sharing. Optimal latency for your applications without dependence on global cloud services.
Adaptability
Fine-tuning and customization of your models to your specific requirements. No restrictions from API limits or vendor lock-ins.

Comparison

LLM Hosting vs. Cloud APIs

Why self-hosting is often the better choice

Feature	Hosted by WZ-IT	Cloud APIs (OpenAI, etc.)
Data Privacy	Hosted in Germany, privacy-focused	Data goes to US providers
Costs	Fixed monthly costs	Variable token prices, often more expensive
Control	Full control over models	Dependency on provider
Customization	Fine-tuning possible anytime	Limited or expensive
Latency	Optimal (Germany)	Variable, depends on region
Availability	Dedicated resources, no rate limits	Rate limits, outages possible

Service scope

LLM hosting with operations: what is included

We take care of everything - you simply use your LLMs. From initial setup to daily operations: Our Managed LLM Hosting handles all technical aspects.

Set up

Setup & Configuration (optional)
On request: installation of the models you want (Llama, Qwen, Gemma, Mistral, DeepSeek, GPT-OSS), Ollama or vLLM as the model server, OpenWebUI as the web interface and optional API endpoints for your applications.

Operate

Operations & Maintenance
We monitor your LLM infrastructure around the clock, perform system updates and optimize GPU performance. How quickly we respond to an alert depends on the agreed service level.

Advise

Support & Consulting
Our team supports you with model selection, integration into your applications and optimization for your specific use cases. Priority support via email and optionally phone/video.

Billing

Transparent Pricing
Fixed monthly costs without hidden fees, no pay-per-token billing. Single-GPU servers are cancellable monthly; the term of multi-GPU servers is stated in the proposal. We agree the concrete price based on the configuration in the initial call.

Models

Open-source LLMs for hosting: supported models

A selection of current open-weight models, as of October 2026. Sizes refer to the weights; context and concurrent requests need additional memory.

gpt-oss-120b and gpt-oss-20b
OpenAI's open models under Apache 2.0: 120b is a mixture-of-experts with 117B parameters, 5.1B active, and 128K context. 120b fits on one 96 GB card, 20b on a 24 GB card.
- Weights (MXFP4): about 65 GB (120b), about 14 GB (20b)
Gemma 4
Google's current generation under Apache 2.0, as a 31B dense model or a 26B mixture-of-experts with 3.8B active parameters, each with 256K context.
- Weights: about 62 GB (31B, BF16), about 23 GB (31B, W4A16)
DeepSeek-V4-Flash
Large mixture-of-experts model under the MIT licence with 284B parameters, 13B active, and up to 1M tokens of context. Realistic on the server with four RTX PRO 6000 (384 GB).
- Weights: about 167 GB (mixed FP4/FP8)
Mistral Small 4
European model by Mistral AI under Apache 2.0: 119B parameters as a mixture-of-experts, 6.5B active, with 256K context. In NVFP4 it fits on one 96 GB card.
- Weights: about 71 GB (NVFP4), about 121 GB (FP8)
EuroLLM 22B
European open-weight model under Apache 2.0, designed for European languages: 22B dense parameters, 32K context. Fits on one 96 GB card.
- Weights: about 45 GB (BF16)
Qwen3.8-27B
Alibaba's current dense model under Apache 2.0 with 27.8B parameters and 262K context, a strong all-rounder for text and code. For larger tasks there is Qwen3.5-122B on two GPUs.
- Weights: about 31 GB (FP8), about 22 GB (NVFP4)

Model pages with licence, sizes and recommended server tier:

Qwen Hosting gpt-oss Hosting Gemma Hosting Mistral Hosting Devstral Hosting DeepSeek Hosting Llama Hosting

Runtime

We Support Ollama & vLLM

Ollama

Simple installation and operation
OpenAI-compatible API
Automatic model management

The easiest way to run LLMs locally. Perfect for development, prototyping, and small to medium production workloads.

vLLM

Considerably higher throughput under high load
PagedAttention for efficient memory management
Continuous batching for maximum throughput

High-performance inference engine for production workloads. Optimized for maximum throughput and minimal latency under high load.

Which framework is right for you? The managed GPU server package is built on vLLM because it handles many concurrent requests efficiently. Ollama suits simple use cases, development and moderate load. We clarify which framework fits your use case in the initial consultation.

Infrastructure

Infrastructure Requirements for LLM Hosting

What is needed for professional LLM hosting? Hosting Large Language Models places special demands on the infrastructure. We ensure that everything is optimally configured.

Hardware & GPU

LLMs need GPUs with enough VRAM. Rule of thumb for the weights: quantized (Q4) roughly half a gigabyte per billion parameters, in FP16 about four times that. Context and KV cache add on top depending on context length and concurrency - so we plan around 48 GB for Llama 3.1 70B and around 20 GB for Gemma 3 27B.

Memory & Storage

In addition to GPU memory, you need sufficient RAM (at least 64 GB) and fast NVMe storage for model files and caching. The model files themselves take roughly 15 GB (7B) to 150 GB (70B) on disk in FP16; quantized operation needs correspondingly less.

Network & Bandwidth

For production applications with multiple users, a stable, fast network connection is essential. Our servers offer gigabit connectivity with low latency within Germany.

Software & Updates

The software stack including CUDA drivers, vLLM as the model server, Open WebUI and the container runtime is installed, configured and kept up to date by us. On request we set up Ollama for development and testing.

Monitoring & Security

Professional monitoring of GPU utilization, temperature monitoring and security updates are part of the service. Automated backups are available as an option.

Scaling & Load Balancing

As requirements grow, we scale your infrastructure horizontally (multiple servers) or vertically (more powerful GPUs). Load balancing between multiple instances is possible.

Minimum recommended server tier by model size

The starting point is the size of the model weights in the chosen format. Memory for context, concurrent requests and the runtime comes on top.

Weights up to about	Minimum recommended	Examples (as of October 2026)
19 GB	Managed GPU Server 24	gpt-oss-20b (MXFP4, about 14 GB)
75 GB	Managed GPU Server 96	Qwen3.8-27B (FP8, about 31 GB), gpt-oss-120b (MXFP4, about 65 GB), Mistral Small 4 (NVFP4, about 71 GB)
150 GB	Managed GPU Server 192	Qwen3.5-122B (FP8, about 127 GB), Mistral Small 4 (FP8, about 121 GB)
300 GB	Managed GPU Server 384	DeepSeek-V4-Flash (about 167 GB)

Which tier fits your use depends on context length and concurrent requests. We check this before the proposal. The three-GPU server (288 GB) is meant for several models side by side.

View server tiers and prices

Pricing

LLM hosting on a managed GPU server

Fixed monthly costs without hidden fees, no pay-per-token billing. Single-GPU servers are cancellable monthly; the term of multi-GPU servers is stated in the proposal. We agree the concrete price based on the configuration in the initial call.

Managed GPU Server 24€699 excl. VAT / month

What's included?

Dedicated GPU server in German data center
Data center, power & network
24/7 monitoring
Security updates & system maintenance
Privacy-focused infrastructure; the data center operator is certified to ISO 27001 & BSI C5
Ollama/vLLM installation (optional)
Free initial consultation on model selection
Provisioning as agreed in the initial call

Assess your LLM hosting requirements

All prices are net and exclude statutory VAT. The offers are addressed to businesses.

Related Services

Turnkey

AI platform

Open WebUI with gpt-oss, Qwen and Ministral, operated in Germany by German providers, SSO and Section 203 supplementary agreement, without your own GPU.

from €149.90 excl. VAT / monthView →AI Server with GPU

GPU Server for AI

Our hardware base for LLM hosting. Powerful GPUs for inference & fine-tuning

View →On-premises

AI Cube

Local AI without vendor lock-in - AI Cube from €7,990 excl. VAT one-time plus AI Cube Care from €349.90 excl. VAT / month.

View →

Who it fits

Who Benefits from LLM Hosting?

Typical use cases and target groups

Who it fits

Agencies & Service Providers
Offer your clients AI-powered services like content creation, SEO analysis or chatbots - with your own LLM infrastructure instead of expensive OpenAI API costs.
Research & Education
Universities, research institutions and educational organizations use their own LLMs for scientific work, studies and teaching without data privacy concerns.
Mid-sized Companies
SMEs in Germany use LLMs for internal knowledge bases, customer service automation, code analysis or document classification.

Example Applications

Internal Chatbot with RAG
Connect your LLM with your own documents, wikis or databases. Employees ask questions in natural language and receive precise answers from your knowledge base.
Code Assistance
Development teams use LLMs locally for code completion, review and documentation - without sending source code to external APIs.

More examples

Content Generation
Create product descriptions, marketing texts or social media content with your own LLM in your corporate language.
Document Analysis
Analyze and classify large volumes of documents, contracts or emails automatically and privacy-focused.

RAG

RAG (Retrieval-Augmented Generation)
With RAG, you connect your LLM with external knowledge sources. The model searches your documents and generates answers based on actual facts from your database. Ideal for company wikis, support databases or research archives.
Internal AI assistant with RAG

Overview

What are Large Language Models (LLMs)?

Large Language Models (LLM) are AI models that can understand and generate natural language. For companies, they offer enormous opportunities: from automating customer communication and intelligent document analysis to coding assistants and knowledge management.

With our LLM Hosting Germany, you can operate these powerful models in your own privacy-focused infrastructure - without having to pass on your sensitive data to global cloud providers.

Whether gpt-oss, Qwen, Mistral or other open models - we handle installation, operations and optimisation of your LLM infrastructure.

Frequently Asked Questions (FAQ)

Everything you need to know about LLM hosting

Knowledge base

In-depth guides on this topic

Guides

What clients say about working with us

From local AI integration to architecture, data sovereignty and ongoing operations.

WZ-IT moved our studio infrastructure from decentralised individual devices to a central platform: every site is securely connected via VPN, new devices are onboarded automatically and an entire site is provisioned from a template, without manual steps on location. What impressed me most is the breadth and depth of their knowledge: Timo and Robin are not a typical IT provider who sets up a server and leaves. The two of them think their way into highly complex infrastructure and software topics, work through every requirement we put in front of them, and build networking, provisioning and operations so that everything fits together in the end. WZ-IT is an excellent partner for complex software, network and architecture projects.

Steve KirchnerManaging Director, nextGYM GmbH

View project

International

Built in Germany's Ruhr Valley. Running worldwide.

WZ-IT designs, develops and operates infrastructure and software for clients in Germany and internationally. We deliver projects remotely and continue supporting them in ongoing operations after go-live.