Llama 4 vs. Qwen 3.5 vs. DeepSeek V4: Which Open-Source Model for Local Enterprise AI? (Q4 2026 Update)

Want to use an open language model in your organisation without handing data to cloud providers?

We test candidates against your tasks and run the chosen model where it fits: on the AI Cube inside your own network (from €7,990 excl. VAT one-time plus AI Cube Care from €349.90 excl. VAT per month) or on a Managed GPU Server from €699 excl. VAT per month. The LLM hosting page compares the operating options.

Explore AI Cube · Explore Managed GPU Server · Schedule a consultation

In May 2026 this comparison came down to three model families: Llama 4 from Meta, Qwen 3.5 from Alibaba and DeepSeek V4. Since then the field has shifted considerably. Qwen3.8-27B, Gemma 4, Mistral Small 4, IBM Granite 4.2 and Meta's Muse Glimmer 30B are models that run on a single GPU and are licensed under Apache 2.0. At the same time, GLM-5.3, MiMo-V2.6 and DeepSeek-V4.1-Flash are new large models that can only be run sensibly on multi-GPU systems.

This article assesses the models from the perspective of a company that wants to run AI in its own data centre or on dedicated hardware: size, licence, context window and above all what actually fits into 24 GB, 96 GB or 128 GB of memory. All figures come from the model cards on Hugging Face and vendor pages, as of October 2026.

Table of Contents

1. What has changed since May 2026
2. Models at a glance
3. How to read benchmarks
4. What fits on 24 GB, 96 GB and 128 GB
5. Large models for multi-GPU systems
6. Context windows and RAG
7. Licensing
8. Recommendation by use case
9. Our approach at WZ-IT
10. Further guides

What has changed since May 2026

Dates without a day refer to the publication of the weights on Hugging Face. Llama 4 has not received new weights since April 2025; the Llama 4 models on Hugging Face date from the release on 5 April 2025. Meta's new open model is Muse Glimmer 30B, a dense model distilled from Muse Spark and licensed under Apache 2.0 without the conditions of the Llama licence. DeepSeek has also published updated checkpoints of the V4 series, DeepSeek-V4-Flash-0731 and DeepSeek-V4-Pro-0813, both under MIT.

Models at a glance

As of October 2026, figures according to the model cards. "Total / active" distinguishes, for Mixture-of-Experts (MoE) models, between all stored parameters and those used per token.

Two corrections compared with the first version of this article: DeepSeek V4 was released in April 2026 as Flash (284B / 13B) and Pro (1.6T / 49B); the previously stated 685B / 37B describe the previous generation. According to its model card, Qwen 3.5 supports 201 languages and dialects. Llama 4 officially supports 12 languages, including German.

Relevant for German-speaking organisations: Granite 4.2 explicitly lists German as a tested language, Mistral Small 4 lists German among its supported languages, and Gemma 4 states support for more than 140 languages. According to its model card, Kolibri-1 from Aleph Alpha is built for German and English, with around 24% German training data and a tokenizer adapted to German word structure.

How to read benchmarks

The first version of this article contained a benchmark table with partly estimated values. That table has been removed because several values could not be traced to a primary source.

The model cards contain extensive benchmark tables, but the values are only comparable within a single model card:

• Different harnesses: Z.AI measures coding benchmarks for GLM-5.3 with Claude Code as the agent environment, while DeepSeek uses its own environment for V4.1-Flash.
• Different reasoning levels: Many models support reasoning_effort. Published scores are usually produced at the highest level with very long outputs, which increases latency and cost in day-to-day operation.
• Vendor benchmarks within a family: Qwen reports 61.7 on SWE-bench Pro for Qwen3.8-27B versus 53.5 for Qwen3.6-27B (model card). Comparisons within one family are more meaningful than comparisons across vendors.

Public benchmarks say little about German domain texts, your own document formats or structured output. A test with 50 to 200 representative tasks from your own data is more reliable. The article Which LLM to self-host? describes such a selection process.

What fits on 24 GB, 96 GB and 128 GB

Important with Mixture-of-Experts: Memory requirements depend on the total parameters, not the active ones. Active parameters only determine the compute per token; all experts must be held in memory. KV cache, activations and runtime come on top of the weights. The KV cache grows with context length and the number of parallel requests, see Sizing GPU and VRAM.

The sizes below are the actual file sizes of the weights on Hugging Face, as of October 2026. Some GGUF files come from community quantizers (unsloth), FP8 and NVFP4 variants from the vendors or NVIDIA. Which format suits which hardware is explained in LLM quantization.

24 GB VRAM: Managed GPU Server 24

WZ-IT's Managed GPU Server 24 uses an NVIDIA RTX PRO 4000 Blackwell with 24 GB GDDR7 ECC.

On 24 GB, dense models up to roughly 30B parameters run at 4 bits. The earlier recommendation in this article to run Llama 4 Scout or Qwen 3.5 on a 24 GB card was wrong: Llama 4 Scout needs around 55 GB at 4 bits.

96 GB VRAM: Managed GPU Server 96

The Managed GPU Server 96 uses an NVIDIA RTX PRO 6000 Blackwell Max-Q with 96 GB GDDR7 ECC.

NVFP4 is natively supported by Blackwell GPUs. For Kolibri-1, Aleph Alpha lists two A100 80 GB or one H200 as the minimum; a single 96 GB card is not on the list. Running it with vLLM requires a dedicated Kolibri plugin (model card). On 96 GB the model is therefore a candidate for your own tests with short context, not a default choice. For Qwen3.5-122B-A10B with more context and concurrency, a system with two 96 GB cards is the more stable choice, see vLLM with Qwen 122B on two RTX PRO 6000 and Multi-GPU inference without NVLink.

128 GB unified memory: AI Cube

The AI Cube is based on the NVIDIA GB10 with 128 GB of unified memory, shared by the operating system, inference server, weights and KV cache.

More on models in this class is in the article LLM models on 128 GB unified memory.

The model is decided, but the GPU is missing?

The model pages for Qwen, Gemma, gpt-oss and DeepSeek show which Managed GPU Server is the minimum recommended tier. Operated in a German data centre, with vLLM and an OpenAI-compatible API.

Explore LLM hosting · Explore Managed GPU Server

Large models for multi-GPU systems

The models that lead vendor benchmarks are too large for single cards. Weight sizes according to Hugging Face, as of October 2026:

The rule of thumb for 4 bits is around 0.5 GB per billion parameters. Most enterprise applications such as chat, RAG, summarisation and classification do not need this class. It pays off when models in the 30B and 120B class demonstrably fall short in your own tests. For such cases WZ-IT plans an individual system with multiple GPUs. Alternatively, a large model can be used via a European API, see European LLM APIs compared.

Context windows and RAG

Large context windows are a differentiator on paper, but in local operation they are limited by memory:

• Llama 4 Scout: 10M tokens. The KV cache for such lengths far exceeds the memory of any single GPU.
• DeepSeek V4, V4.1-Flash and MiMo-V2.6: 1M tokens. DeepSeek states a global KV cache of 890 bytes per token for V4.1-Flash, around a quarter of V4 Flash (model card). This lowers the memory needed for long contexts but does not change the size of the weights.
• Qwen3.8-27B, Gemma 4, Mistral Small 4: 256K to 262K tokens. Sufficient for RAG pipelines, because the retrieval stage only passes on the relevant sections anyway.
• Granite 4.2: 128K native. Sufficient for typical RAG queries.

For enterprise RAG, retrieval quality matters more than the maximum context length. A model with 256K context and good retrieval usually gives better answers than a model that is handed entire document collections. How search and reranking work together is described in Hybrid search and reranking.

Licensing

As of October 2026, according to the licence files on Hugging Face and vendor pages. This overview is a technical assessment, not legal advice.

Llama 4 and the EU: The Acceptable Use Policy does not grant the rights under Section 1(a) of the licence for multimodal Llama 4 models to individuals domiciled in, or companies with their principal place of business in, the EU. End users of a product that incorporates such a model are exempt. Since Llama 4 Scout and Maverick natively process text and images, this affects the models themselves, not just the image capability. The licence also requires a separate licence above 700 million monthly active users, the notice "Built with Llama" and the prefix "Llama" in the name of derived models.

Not every Qwen is the same: The small and mid-sized Qwen models are Apache 2.0. Qwen3.8-Flash-Next, by contrast, uses the Qwen Community License 1.0: anyone offering model inference as a service to third parties, or an AI work assistant for coding or office work, needs a separate licence from Qwen. Internal use is exempt as long as the model and its outputs are not made available to any third party.

GLM-5.3 is not MIT: GLM-5.3-Flash is MIT, while the larger GLM-5.3 uses its own licence with a security review for very large model-as-a-service providers.

For most companies, Apache 2.0 and MIT models are the clearest choice from a legal point of view. A more detailed assessment of fine-tuning, hosting for third parties and derived models is in the article Open LLM licences for commercial use.

Recommendation by use case

As of October 2026. The table lists candidates for your own tests, not a fixed ranking.

For the majority of enterprise applications, the 30B class is sufficient. Qwen3.8-27B, Gemma 4 and Muse Glimmer 30B run on a 24 GB card, are licensed under Apache 2.0 and process images. If you need more quality headroom or concurrency, move to 96 GB or the AI Cube with Mistral Small 4, gpt-oss-120b or Qwen3.5-122B-A10B. The large models with several hundred billion parameters only make sense once your own tests show that the smaller models cannot solve the task.

Still unsure which tool fits? The AI opportunity check tests which approach holds up on a concrete process and ends with a pilot recommendation. From €3,900 excl. VAT.

Our approach at WZ-IT

WZ-IT selects models based on the specific use case and runs them on dedicated hardware:

1. Gather requirements: languages, document types, share of images, number of users, data protection needs and licence requirements.
2. Test candidates: compare two to four models with representative tasks from your own data, using identical quantization and inference engine.
3. Define hardware: AI Cube inside your own network (from €7,990 excl. VAT one-time, plus AI Cube Care from €349.90 excl. VAT / month) or a Managed GPU Server with 24 GB (€699 excl. VAT / month) or 96 GB (€2,199 excl. VAT / month), each plus setup.
4. Set up operations: vLLM or Ollama, Open WebUI or your preferred interface, gateway, monitoring and updates.
5. Plan model changes: new releases are checked with the same test tasks before they replace the running version.

Without your own hardware, gpt-oss-120b, Qwen and Ministral are available on the AI platform, in a dedicated Open WebUI instance with models operated in Germany by German providers.

Support, consulting and implementation by WZ-IT.

Further guides

• vLLM vs. Ollama vs. llama.cpp - inference engines compared
• vLLM with Qwen 122B on two RTX PRO 6000 - multi-GPU operation in practice
• Run GPT-OSS 120B on the AI Cube - large model locally
• DGX Spark vs. AI Cube - hardware for local models
• European LLM APIs compared - when the model does not have to run locally
• Which LLM to self-host? - selection process
• Sizing GPU and VRAM - calculating memory requirements
• Open LLM licences for commercial use - licensing questions in detail
• GDPR-compliant AI: assessment criteria - compliance
• LLM hosting - language model operation by WZ-IT

Which model fits your use case? We test candidates with your data and run the selected model on the AI Cube or a Managed GPU Server, with Open WebUI or your preferred interface. Schedule a consultation | AI Cube

Sources

• Qwen3.8-27B, model card
• Qwen3.8-Flash-Next, model card
• Qwen3.8-Flash-Next, Qwen Community License 1.0
• Qwen3.5-397B-A17B, model card
• Qwen3.5-122B-A10B, model card
• Gemma 4 31B, model card
• Gemma 4 26B A4B, model card
• Google, Gemma releases
• Mistral AI, Mistral Small 4
• Mistral Small 4, model card
• IBM Granite 4.2 30B, model card
• GLM-5.3, model card
• GLM-5.3, licence
• GLM-5.3-Flash, model card
• DeepSeek V4 Flash, model card
• DeepSeek V4 Pro, model card
• DeepSeek-V4.1-Flash, model card
• MiMo-V2.6-Flash, model card
• MiMo-V2.6-Pro, model card
• gpt-oss-120b, model card
• Meta, Muse Glimmer 30B model card
• NVIDIA, Nemotron 3.5 Lightning 30B-A3B model card
• Aleph Alpha, Kolibri-1 model card
• Mistral AI, Mistral Large 4
• DeepSeek-V4-Flash-0731, model card
• DeepSeek-V4-Pro-0813, model card
• Meta, Llama 4 model card
• Meta, Llama 4 Community License
• Meta, Llama 4 Acceptable Use Policy