Which LLM to self-host? Models and licenses

Have the right model and matching hardware chosen for you? WZ-IT plans and operates local AI - from model and license choice through GPU servers to operation, GDPR-compliant from one team. See GPU servers

"Which LLM should I take?" is the first question when building local AI - and the wrongly framed one. There is no universally best model, only one that fits the use case, hardware and license requirement. This article shows how to make that choice: by size, by license and by capability. As of July 2026.

Table of contents

• Not "the best" model but the right one
• Model size by use case
• The license matters too
• Open model families at a glance
• How to choose the model

Not "the best" model but the right one

Leaderboards and benchmarks tempt you to always want the largest, newest model. For production that is rarely the right decision. A model that reliably solves a well-defined task and runs on the available GPU is more valuable than a top model that blows the hardware or whose license restricts use.

The choice is made along three axes: size (what the task needs and the hardware carries), license (what enterprise use permits) and capability (how well the model does the concrete task).

Model size by use case

The parameter count is the most important lever for hardware demand and capability. The rule of thumb: as large as necessary, not as large as possible.

• Smaller models (single-digit billion range) suffice for well-defined tasks - classification, simple summaries, structured extraction - and run on modest hardware.
• Medium models cover a broad everyday range: knowledge retrieval via RAG, drafts, assistance functions.
• Large models bring the broadest capabilities but demand considerably more GPU memory and compute more slowly.

Modern models increasingly use a mixture-of-experts architecture, in which only part of the parameters is active per request - this somewhat decouples the nominal size from the compute effort. How much VRAM a given size concretely needs is shown in Sizing GPU and VRAM.

The license matters too

An often overlooked point: not every "open" model is equally free to use. The license determines what is permitted in the enterprise.

• Apache 2.0 and MIT are permissive: commercial use, fine-tuning and redistribution without restriction. Model families like Qwen and Mistral (Apache 2.0) or DeepSeek (MIT) fall under this.
• Meta's Llama license allows commercial use but contains restrictions - among them a user cap and use restrictions relevant to the EU for certain functions.

For sovereign, uncomplicated use in the European enterprise context, permissively licensed models are the safe choice. The license therefore belongs in the selection just as much as the capability.

Open model families at a glance

Instead of individual, quickly outdated version numbers, it is worth looking at the families that are continuously developed:

• Qwen - broad model spectrum under Apache 2.0, strong at multilingual and general tasks.
• Mistral - European origin, current models under Apache 2.0.
• DeepSeek - MIT-licensed, strong at reasoning and code.
• Llama - very widespread and well supported, but with the license restrictions mentioned.

Within each family there are sizes for different use cases. The selection therefore starts at the family and license and refines via the size.

How to choose the model

A pragmatic path in four steps:

1. Sharpen the use case - what exactly should the model do, and how well must it do it?
2. Fix the license - for European enterprise use, prefer permissive licenses.
3. Align the size to the hardware - choose the largest model size that reliably fits the available GPU (see VRAM sizing).
4. Test and compare - measure two or three candidates on real tasks instead of trusting benchmarks.

And it need not stay with one model: via a gateway like LiteLLM several models can be combined - a small one for simple, a larger one for demanding tasks. Where the model sits in the overall picture is shown in What is local AI?.