European LLM APIs Compared: Mistral, STACKIT, IONOS, Mittwald, T-Systems

Choosing a language model for RAG or an assistant? WZ-IT connects European model APIs or your own inference through a gateway and documents the data flows, see LLM hosting in the AI hub. Book a meeting

Anyone who wants to add a language model to a RAG application or an internal assistant without calling OpenAI or Anthropic directly ends up with a small group of European providers: Mistral AI, STACKIT, IONOS, mittwald and T-Systems. All five offer an OpenAI-compatible interface. That is where the similarities end.

The providers differ in data centre location, in what they state about storage and training, in their model catalogue and above all in whether they offer the embedding and reranker models a RAG application needs alongside the language model. This article compares the five services based on their own documentation, puts Azure OpenAI and Amazon Bedrock in context with regard to the CLOUD Act, and shows when running your own inference is the better choice. All model lists and prices reflect September 2026.

Table of Contents

1. The five providers at a glance
2. Location, storage and contract
3. Model catalogues in September 2026
4. Interface and pricing model
5. What to watch for with each provider
6. Azure OpenAI, AWS Bedrock and the CLOUD Act
7. Models come and go
8. Mixed operation through a gateway
9. When your own inference makes sense
10. Our approach at WZ-IT
11. Further guides

The five providers at a glance

Mistral is the only one of the five that develops its own models. The other four run open-weight models from third parties (Qwen, gpt-oss, Llama, Gemma, Mistral) on their own infrastructure. T-Systems additionally brokers proprietary models from OpenAI, Anthropic and Google.

Location, storage and contract

Three questions matter for the data protection assessment: where processing takes place, what is stored, and whether a data processing agreement (DPA) under Art. 28 GDPR is in place.

Evidence: Mistral hosts data in the EU by default and in the US when the US endpoint is used explicitly (Mistral Help Center). On training, Mistral distinguishes between free mode, in which inputs and outputs may be used for training, and pay-as-you-go, where customers have a right to opt out (Mistral Help Center, training). STACKIT does not store customer data from requests and does not train on it (STACKIT FAQ). IONOS processes exclusively in German data centres, does not log content and keeps only timestamp, model and token count for billing (IONOS Data Handling). mittwald stores no content but keeps metadata such as IP address and API key (mittwald AI Hosting data protection). T-Systems provides a DPA on request and states ISO 27001 for its operations (T-Systems Enterprise Trust).

STACKIT's product documentation does not describe the DPA; it has to be clarified as part of the cloud contract. For all providers: certifications apply to the scope the provider states, not automatically to every individual service.

Model catalogues in September 2026

A RAG application needs three kinds of models: an embedding model for the index, optionally a reranker to refine the ranking of hits, and a language model for the answer. The table shows a selection of language models and the complete embedding and reranker offerings.

Sources: Mistral model overview, STACKIT shared models, IONOS LLMs, embeddings and rerankers, mittwald models, T-Systems models.

Three observations from the table:

• gpt-oss-120b is offered by four of the five providers. It is an obvious common denominator if switching providers without switching models should remain possible.
• For German text, T-Systems offers Jina Embeddings v2 Base DE, and IONOS and T-Systems both offer the multilingual BGE-M3. Which model suits German documents is covered in the article on embedding models for German.
• Rerankers are the gap. Only IONOS and mittwald list one. Why reranking has a strong effect on retrieval quality is explained in the article on hybrid search and reranking.

Interface and pricing model

Prices per million tokens (input / output), as of September 2026, according to Mistral, STACKIT, IONOS, mittwald and T-Systems. The figures are not directly comparable because models, tiers and minimum commitments differ.

OpenAI-compatible means that the chat completions endpoint, and usually the embeddings endpoint, follow the OpenAI format; an existing OpenAI client works after changing the base URL, key and model name (Mistral migration guide). Not every feature is implemented identically. Tool calling, structured outputs, context limits and rate limits vary by model and provider and belong in a test before switching. The OpenAI API has no reranking endpoint; providers with a reranker expose their own route for it.

What to watch for with each provider

Mistral. The only provider with its own model family and the widest range of specialised models (OCR, speech, code, moderation). The training setting is not switched off automatically and should be checked and documented at organisation level. Mistral names sub-processors outside the EU under standard contractual clauses; the list is in its Trust Center.

STACKIT. Price tiers by model size, a small, well-maintained catalogue, region eu01 in Germany. Two models are marked as deprecated (Qwen3.6 27B, Gemma 3 27B). There is no reranker. Additional models can be requested through a service request.

IONOS. The broadest RAG offering of the five: several embedding models, a reranker and its own collections API for semantic search. IONOS publishes retirement dates with successors: Llama 3.1 8B ends on 15 October 2026, Mistral Nemo and BGE Large 1.5 on 22 October 2026 (IONOS models).

mittwald. Package pricing instead of pure consumption billing, which makes costs predictable but requires an estimate of usage. An mStudio account with an organisation and an active plan is required. Besides chat, embedding and reranker models, the catalogue includes OCR (GLM-OCR), speech recognition (whisper-large-v3-turbo) and text-to-speech.

T-Systems. The largest model selection, because proprietary models are brokered as well. That is exactly where the limitation lies: only the open-source models run in the T Cloud in Germany. GPT models run on Azure in Sweden, Claude and Mistral Medium 3 on Google Cloud in Europe, Gemini on Google Cloud worldwide. For this category, the terms of the upstream providers apply, and they may use sub-processors worldwide (T-Systems Enterprise Trust). If you choose the service for its German location, restrict the permitted models in your gateway.

Azure OpenAI, AWS Bedrock and the CLOUD Act

US hyperscalers offer EU processing but remain US companies. The CLOUD Act obliges providers of communication and cloud services to disclose data in their possession, custody or control, regardless of whether it is stored inside or outside the United States (18 U.S.C. § 2713). Choosing an EU region does not change that.

Microsoft notes that regions can be added to a data zone without prior notice. How far the CLOUD Act reaches into a legally separate EU subsidiary such as the one behind the AWS European Sovereign Cloud is disputed among lawyers. This classification is not legal advice; for data with high protection needs it belongs in the data protection impact assessment. The obligations the EU AI Act brings for RAG applications are covered in the article RAG and the AI Act.

For European providers without a US parent, the CLOUD Act does not apply directly. That holds for the models they run themselves, not for brokered models as with T-Systems.

Models come and go

A model catalogue is not a stable product. IONOS has retired seven models since May 2025, including Llama 3.1 405B on 15 September 2026 and the German model Teuken 7B on 16 April 2026. STACKIT marks predecessors as deprecated before they are removed, and Mistral maintains its own deprecation list.

For a RAG application this has two consequences:

• Language model: a successor answers differently. Before switching, run a regression test with a fixed set of questions, as described in the article on RAG evaluation.
• Embedding model: vectors from different models are not interchangeable. For the switch from BGE Large 1.5 to BGE M3, IONOS explicitly requires re-embedding existing content (IONOS embedding models). With large document collections, that is a project in its own right.

If model versions must remain stable for years, for example because answers are part of an audited process, an API gives you less control than running the model yourself.

Mixed operation through a gateway

Embedding, reranker and language model do not have to run with the same provider. A common split:

With an API, only the question and the retrieved passages leave the network, not the whole collection. That is less than with a fully outsourced service, but not nothing: permission checks must apply before retrieval so that no passages reach the model that the user is not allowed to see. How to implement that is described in the article on RAG permissions.

A gateway such as LiteLLM bundles providers behind one endpoint, logs usage and costs and allows fallbacks. The model list in the gateway can also enforce that only models operated in Germany or Europe are reachable. WZ-IT runs LiteLLM as managed LiteLLM.

When your own inference makes sense

A European model API is sufficient for many applications. Your own inference is the better choice when one of the following applies:

WZ-IT offers two routes to your own inference. The AI Cube is an AI appliance in your own network with 1, 2 or 4 TB of NVMe storage. Managed GPU servers from WZ-IT run in a German data centre with an NVIDIA RTX PRO 4000 Blackwell (24 GB) or RTX PRO 6000 Blackwell (96 GB) and are operated with vLLM and Open WebUI. Which model size fits which amount of GPU memory is shown in the article on VRAM sizing. The choice of inference software is compared in vLLM, Ollama or llama.cpp.

Our approach at WZ-IT

1. Assessment. Use case, types of data, protection needs, number of users and load profile. This determines whether a model API, your own inference or a mixed setup fits.
2. Provider selection. Matching location, DPA, training policy, model catalogue and embedding and reranker offering against the requirements, based on the current provider documentation.
3. Test with your own data. A fixed set of questions from your collection against the candidate models, assessing answer quality, source references and response times.
4. Connection through a gateway. LiteLLM with approved models, logging and cost control, local embeddings and rerankers as needed.
5. Operation. Tracking model retirements, regression tests before model changes, support, consulting and implementation by WZ-IT.

For a structured start with your own document collection, the RAG sprint is available.

Further guides

• Best embedding models for German, BGE-M3, Qwen3-Embedding and other models for German documents compared.
• RAG and the AI Act, when a RAG system is high-risk AI and which obligations apply.
• RAG knowledge base in the public sector, document collection, permissions and operation on your own hardware.
• Hybrid search and reranking, why a reranker improves retrieval quality.
• On-premise AI solutions compared, six operating models from cloud workspace to GPU server.
• AI solutions from WZ-IT, the hub with all offerings for local AI and LLM operations.

European API or your own inference? We compare the providers for your use case, test with your data and connect the right model through a gateway. Book a meeting

Sources

• Mistral, model overview
• Mistral, API pricing
• Mistral Help Center, where data is stored
• Mistral Help Center, use of data for training
• Mistral Help Center, SOC 2 and ISO 27001
• Mistral, Data Processing Addendum
• Mistral, migration guides
• STACKIT, available shared models
• STACKIT, AI Model Serving FAQ
• STACKIT, AI Model Serving product page
• IONOS, AI Model Hub models
• IONOS, large language models
• IONOS, embedding models
• IONOS, reranking models
• IONOS, data handling
• IONOS, AI Model Hub product page
• mittwald, AI Hosting
• mittwald, models in AI Hosting
• mittwald, data protection in AI Hosting
• T-Systems AI Foundation Services, models
• T-Systems AI Foundation Services, plans and pricing
• T-Systems AI Foundation Services, Enterprise Trust
• Microsoft Learn, deployment types in Microsoft Foundry Models
• AWS, geographic cross-Region inference in Amazon Bedrock
• AWS Security Blog, open-weight models on Bedrock in the AWS European Sovereign Cloud
• 18 U.S.C. § 2713, Cornell Law School