What does a local AI server cost?

Run local AI with predictable costs? The AI Cube sits as an appliance in your own network, the Managed GPU servers run in a German data centre. Neither is billed per token. Support, consulting and implementation by WZ-IT. Book a call

The cost of a local AI server can only be answered if hardware, platform, integration and operations are considered separately. An online price for a computer is not comparable with a ready-to-use multi-user platform. And a comparison with a model API needs a common unit: the cost per million tokens.

This article lists the prices of WZ-IT's offers, calculates the cost per token at different utilisation levels and sets it against the list prices of a European model API. All assumptions are shown in tables. Prices as of October 2026.

Table of contents

• Cost overview: AI Cube and Managed GPU servers
• Four cost blocks
• Break-even per token: the assumptions
• Cost per million tokens by utilisation
• What the break-even volumes mean
• When the token price does not decide
• Compare three years instead of purchase price
• When standard hardware is not enough

Cost overview: AI Cube and Managed GPU servers

With the AI Cube the device belongs to the customer; operations run via AI Cube Care. The GPU servers are an ongoing service; host system, term and provisioning are confirmed in the quote. Neither option bills per token or per user. Whether ownership or a service model fits better is covered in Buy or rent an AI server?.

Four cost blocks

1. Hardware and base system. For local language models, available memory, memory bandwidth and desired throughput determine the hardware class. How much memory model and context occupy is calculated in Sizing GPU and VRAM for LLMs.

2. Setting up the AI platform. Operating system, drivers, model runtime, user interface, model, user management, network, hardening and function test. Anyone doing this internally has to account for the time and the knowledge needed in the team later.

3. Integration into workflows. Using your own documents manually in a knowledge interface is different from a permanently synchronised RAG pipeline. Interfaces, source permissions and automations depend on the project.

4. Ongoing operations. Updates, security review, monitoring, backup, recovery, model changes and support arise over the entire service life, either internally or as a managed service.

Break-even per token: the assumptions

A model API bills per million tokens, own hardware costs a fixed amount per month. The break-even is the monthly token volume at which both cost the same. The comparison uses the list prices of the IONOS AI Model Hub, a German provider processing in Germany, for two open-weight models that also run on WZ-IT hardware.

gpt-oss-120b is a mixture-of-experts model with 117 billion parameters that, according to its model card, fits on a single 80 GB GPU; it therefore runs on GPU Server 96 and on the 128 GB of the AI Cube. Mistral Small 24B fits on 24 GB with 4-bit quantization. All prices exclude VAT.

Cost per million tokens by utilisation

Dividing the monthly fixed cost by the tokens actually processed gives the cost per million tokens. It falls with every additional request processed.

Above the break-even, own hardware is cheaper per token; below it, the API is. Whether the hardware can actually process that volume is a second question that depends on model, context and concurrency.

What the break-even volumes mean

A request with 4,000 tokens of context and a 1,000-token answer uses 5,000 tokens. This gives:

For context: a publicly documented measurement on a single GB10 with the smaller model Qwen3.6-35B-A3B (NVFP4, vLLM, 1,024 input tokens, 256 output tokens) reached around 320 output tokens per second with eight and around 432 with 16 concurrent chats. This is not a measurement for gpt-oss-120b, but it shows the order of magnitude.

Three factual conclusions follow:

• With use during office hours only, the pure token price of European APIs for open-weight models is usually below the cost of own hardware. The break-even requires a utilisation that an interactive assistant for a small team does not reach.
• Under continuous load the picture shifts. Round-the-clock batch processing, such as classification, document extraction or embedding large collections, uses hardware evenly and can reach the break-even.
• API prices change. List prices, model catalogues and retirement dates are set by the provider. The fixed cost of own hardware is known over the term.

When the token price does not decide

In many projects the token price is not the decisive criterion:

Which European APIs exist and how they differ in location, contract and models is compared in European LLM APIs compared. The general trade-off is covered in Cloud AI vs. self-hosted. A mixed setup is also possible: embedding and reranker locally, the language model locally or via an API depending on protection needs, bundled behind a gateway such as LiteLLM.

Compare three years instead of purchase price

A robust calculation uses the same assumptions for both options:

Local platform: purchase price or monthly rate + setup + integration + power and operations + risk buffer for replacement.

Cloud platform: 36 months of user licences or API consumption + integrations + administration + potential add-on features and data regions.

Anyone who already knows their API consumption can enter it directly into the table above. Anyone who does not measures it in a pilot phase, for example via the usage logs of a gateway. Non-financial factors also belong in the decision: permitted data types, control over model and updates, offline capability and dependence on the provider.

When standard hardware is not enough

With the standard configuration an AI Cube serves up to ten simultaneously generated answers, depending on model and context length; for teams of five to twenty people one device is generally enough. A larger setup makes sense when several large models run in parallel, very long contexts are needed, or availability and redundancy have to be contractually assured. We then plan multi-GPU systems or clusters based on measured load; these architectures are quoted separately.

How account count, concurrency and answer length become a load profile is shown in Size a local AI server by users. Which models fit on 128 GB is covered in LLM models on 128 GB unified memory.

Sources and further reading

• IONOS Cloud, price list including AI Model Hub
• OpenAI, gpt-oss-120b model card
• Public GB10 measurement with Qwen3.6-35B-A3B NVFP4 (dgx-spark-bench)
• NVIDIA: DGX Spark hardware with 128 GB unified memory
• WZ-IT: Sizing GPU and VRAM for LLMs
• WZ-IT: Cloud AI vs. self-hosted