Mistral Large on-premise: licence per model, hardware requirements and open-weight alternatives

Run Mistral models on your own hardware or a dedicated GPU server in Germany?

The LLM hosting page compares the operating options, Mistral hosting classifies the licence per model. On the AI Cube, vLLM and Open WebUI run preinstalled in your own network.

View LLM hosting · View AI Cube

"Mistral Large" refers to several models with different licences: Large 2 is under a research licence, Large 3 under Apache 2.0, and Large 4 has no licence text yet. This guide reads the licence of each model from the model card and LICENSE file, calculates hardware requirements with disclosed assumptions and classifies alternatives. Status 28 October 2026, checked against Hugging Face and the Mistral model overview.

Table of contents

• In brief
• The Mistral lineup as open weights
• Licences: what they allow companies
• Mistral Large 4: release status
• Hardware requirements: calculation and table
• Running with vLLM and llama.cpp
• EU and data protection when running locally
• Alternatives in the open-weight market
• Selection criteria
• Status and methodology
• Our approach at WZ-IT
• Further guides

In brief

• Mistral Large 3 (675 billion parameters, mixture of experts) is under Apache 2.0 and needs an 8-GPU node: 681.5 GB of weights in FP8, 403.1 GB in NVFP4.
• The licence belongs to the model, not the family. Mistral Medium 3.5 and Devstral 2 exclude companies with more than USD 20 million monthly revenue, Large 2 and Pixtral Large are restricted to research purposes.
• Mistral Large 4 has been available as a preview via the API since 6 October 2026. Weights and licence text were not published on 28 October 2026.
• For a single 96 GB GPU, Mistral Small 4 (Apache 2.0) fits by calculation in NVFP4 with 70.8 GB of weights.

The Mistral lineup as open weights

The model overview lists open-weight models alongside commercial ones. The table shows the text models relevant to companies. Weight sizes are the file sizes of one format on Hugging Face (several repositories contain the weights twice, in Mistral and Transformers format).

The model cards of Large 3, Medium 3.5, Small 4 and Ministral 3 list German among the supported languages. With MoE models all experts have to sit in memory, the active parameters only determine the compute load per token.

Licences: what they allow companies

What was read are the LICENSE files and the frontmatter of the model cards, not the family name.

Three consequences in practice:

1. "Mistral Large on-premise" can mean two licences. Anyone who loads a Large 2 model because it appears in older guides uses it in a company outside the licence. Large 3 is the Apache 2.0 model.
2. The revenue threshold of Medium 3.5 is a group question. The wording: "the global consolidated monthly revenue of your company (or that of your employer) exceeds $20 million (or its equivalent in another currency) for the preceding month". What counts is the monthly revenue of the preceding month, not annual revenue. How group revenue is counted is not regulated in detail by the text. Your legal department or Mistral AI ([email protected]) clarifies this, this page is not legal advice.
3. Derivatives stay bound. The licence expressly covers "derivatives, modifications, or combined works". Quantizations, GGUF files and fine-tunes of Medium 3.5 fall under the same revenue threshold on that reading (assessment, not legal advice). For Large 3, the original is under Apache 2.0.

Steps before the download: record the exact repository ID, read the LICENSE file and model card, compare thresholds with your own figures, store the licence status with a date in the model register. The comparison beyond Mistral is in Open LLM licences for commercial use.

Mistral Large 4: release status

Mistral Large 4 has run as a public preview via Mistral Studio and the API since 6 October 2026. Mistral announces the weights for the end of the month and lists only "Open" as the licence on the model page, without a text; the page's weights table shows weights and licence as "Coming soon". On 28 October 2026 no Large 4 repository with files was available in the mistralai organisation on Hugging Face. An announcement page names 29 October 2026 as the expected release for mistralai/Mistral-Large-4-1T-A52B.

Based on the parameter count (1.05 trillion), the weights come to about 1.05 TB at 8 bit and about 0.53 TB at 4 bit. This is a calculation without file size: for Large 3, NVFP4 at 403.1 GB is 19 percent above the rule of thumb because scaling values per 16 weights are added to the 4-bit weights and attention, embedding and gate layers stay unquantized (params.json). Hardware and licence of Large 4 will be assessed only after publication.

Hardware requirements: calculation and table

The calculation follows Sizing GPU and VRAM for LLMs:

1. Weights = file size of the checkpoint.
2. KV cache per token: classic attention 2 × layers × KV heads × head dimension × bytes (Medium 3.5: 88 layers, 8 KV heads, dimension 128, so 352 KiB). Large 3 and Small 4 use multi-head latent attention, vLLM stores kv_lora_rank + qk_rope_head_dim values per layer (source code): Large 3 61 × (512 + 64) × 2 bytes = 68.6 KiB, Small 4 36 × (256 + 64) × 2 bytes = 22.5 KiB (values from params.json and config.json).
3. Cache budget = GPUs × GPU memory × 0.92 (vLLM default for --gpu-memory-utilization) - weights - 4 GB reserve per GPU.
4. Tokens in cache = cache budget ÷ KV cache per token. With classic attention the KV heads are spread across the GPUs, as long as their number does not exceed the number of KV heads. With multi-head latent attention there is only one latent KV head (num_kv_heads=1, max_tp_shards=1 in the source code): the cache for each token sits in every tensor-parallel rank, so the tokens are the budget ÷ KV cache per token ÷ number of GPUs. The vLLM docs name an 8x KV cache duplication for DeepSeek-R1 with -tp 8 and -dcp (decode context parallelism) as a remedy.

Assumptions: KV cache in BF16, 1 GB = 10^9 bytes, reserve as a planning value (likely higher for large models with tensor parallelism), sessions = tokens in cache ÷ 32,768. In the MLA rows the token count is already divided by the number of GPUs (step 4). An FP8 KV cache doubles the number of sessions by calculation, quality has to be checked. For Large 3 this matches the model card: FP8 on a node with 8 × H200, NVFP4 on a node with H100 or A100. GPU memory: H100 80 GB (SXM, the NVL variant has 94 GB), H200 141 GB, RTX PRO 6000 96 GB. On 128 GB of unified memory, Small 4 in NVFP4 (70.8 GB) fits by calculation from this lineup, see LLM models on 128 GB unified memory. Quantization overview: LLM quantization.

Running with vLLM and llama.cpp

vLLM is the route Mistral recommends (Large 3, Medium 3.5). For Large 3 the model cards name --tensor-parallel-size 8, the Mistral tokenizer mode (--tokenizer_mode mistral --config_format mistral --load_format mistral) and, for tool calling, --enable-auto-tool-choice --tool-call-parser mistral. --max-model-len defaults to 262144, "not necessary for most scenarios". Medium 3.5 adds --reasoning-parser mistral and, per its model card, requires a nightly version of vLLM, and Small 4 uses two GPUs with --attention-backend FLASH_ATTN_MLA. On H100 and A100, vLLM falls back to Marlin FP4 for NVFP4: a memory gain, but according to the model card no speed gain over FP8. For contexts above 64k, Mistral observed a quality drop with NVFP4 and recommends the FP8 weights there. Docker, parameters, security and metrics are covered in vLLM production deployment, the engine itself in What is vLLM? and Multi-GPU inference without NVLink.

llama.cpp is under the MIT licence. The llama-server offers an OpenAI-compatible API, continuous batching and parallel slots (--parallel), --api-key and, with --metrics, Prometheus metrics; /health is reachable without a key. GGUF files come from third parties, for example Unsloth: Large 3 in Q4_K_M 407.0 GB, Small 4 in UD-Q4_K_M 73.8 GB (file sizes, retrieved 28 October 2026). The Medium 3.5 model card warns about GGUF files generated with a faulty Transformers configuration. llama.cpp suits few concurrent users and memory shared by GPU and CPU; for many users vLLM is the choice, see vLLM vs. Ollama and vLLM, Ollama or llama.cpp.

EU and data protection when running locally

• Vendor: according to its licence text, Mistral AI is a French société par actions simplifiée based in Paris (MRL). When run locally, prompts and documents stay on your infrastructure, the vendor is not involved in processing, only in licence and further development.
• Data paths: weights are downloaded once, after which HF_HUB_OFFLINE=1 stops calls to the Hub. vLLM collects anonymous usage data by default (VLLM_NO_USAGE_STATS=1 switches that off, docs), Hugging Face libraries send telemetry (HF_HUB_DISABLE_TELEMETRY=1, docs).
• What stays open: legal basis, access rights, logging, deletion concept and obligations under the EU AI Act have to be settled per application. Assessment criteria are in GDPR-compliant AI: assessment criteria, the obligations in The EU AI Act for companies. This page is not legal advice.

Alternatives in the open-weight market

Other vendors also differ in licence per model. What was read are model cards and LICENSE files (retrieved 28 October 2026).

Model choice is classified in Which LLM to self-host?, the overall setup in The open-source LLM stack.

Selection criteria

The table classifies models and is not a hosting offer, the checkpoints on the Mistral hosting page are the ones that are hosted. Quality on German technical texts is shown only by a test with your own questions, as described in Measuring RAG quality.

Status and methodology

• Status: 28 October 2026. Models, licence texts and file sizes were read on Hugging Face and docs.mistral.ai on 28 October 2026. Mistral Large 4 will be reassessed once the weights are published.
• Calculation: file sizes from the repositories (one format), cache values from config.json and params.json. Reserve and BF16 cache are assumptions, the MLA cache and its duplication per GPU follow the vLLM source code and the vLLM docs. Not measured: cache sizes on real hardware, throughput, latency, answer quality.
• Not covered: Mistral's audio, time-series and moderation models, API usage.

Our approach at WZ-IT

Before the proposal we check which checkpoint is permitted for your company. We run Mistral Small 4 and Devstral Small 2 on the Managed GPU Server in Germany with vLLM and Open WebUI, which checkpoint suits which GPU configuration is on the Mistral hosting page. For Mistral Medium 3.5 and Devstral 2, the Managed GPU Server 192 (two GPUs) is the minimum recommended there, and both are subject to the revenue threshold of the Modified MIT License. For operation in your own network, the AI Cube offers 128 GB of unified memory with vLLM and Open WebUI preinstalled. LLM hosting compares the operating options.

Further guides

• Open LLM licences for commercial use - Apache 2.0, MIT, Llama, Qwen and Mistral compared
• Which LLM to self-host? - model families and selection
• Sizing GPU and VRAM for LLMs - derivation of the memory calculation
• LLM models on 128 GB unified memory - models for the AI Cube
• The open-source LLM stack - engine, gateway, observability
• vLLM production deployment - Docker, parameters, security
• LLM quantization - FP8, NVFP4 and AWQ
• Multi-GPU inference without NVLink - splitting models across several GPUs

Use Mistral models in your own operation? WZ-IT checks licence and hardware before the proposal and runs the model on a dedicated GPU server in Germany. For operation in your own network there is the AI Cube. LLM hosting · Managed GPU Server · AI Cube