Best Embedding Models for German in 2026: BGE-M3, Qwen3-Embedding and E5 Compared

Building a RAG system on German documents? WZ-IT measures embedding models with reference questions on your collection and runs retrieval, reranker and language model on your own hardware, see RAG Proof of Value and internal AI assistant in the AI hub. Schedule a meeting

In a RAG system, the embedding model decides which passages reach the language model at all. If search does not find the right paragraph, even the largest language model cannot give a sourced answer. Even so, the embedding model is often picked from the first tutorial and never questioned again.

For German documents the choice in 2026 is wider than two years ago: BGE-M3, the Qwen3-Embedding family, multilingual-e5, EmbeddingGemma, Granite Embedding R2 and several Jina generations are available as open weights, Mistral Embed as a European API. The differences lie less in a single benchmark number than in license, context length, vector dimension, memory footprint and which inference servers fully support the model. This post compares the models along these criteria. As of September 2026.

Table of contents

1. What the embedding model decides in a RAG system
2. The models at a glance
3. Licenses: not every open model is cleared for commercial use
4. What benchmarks say about German text
5. Context length, dimensions and storage
6. VRAM and hardware
7. Serving with TEI, vLLM and Ollama
8. Which model for which use case
9. Changing the model means re-indexing
10. Our approach at WZ-IT
11. Further guides

What the embedding model decides in a RAG system

An embedding model turns text into a vector, a list of numbers. Texts with similar meaning end up close to each other in vector space. During indexing, every section (chunk) of a document is embedded and stored in a vector database such as Qdrant or PostgreSQL with pgvector. When a question comes in, it is embedded with the same model, and the database returns the nearest chunks. The basics are covered in the knowledge article What is RAG?.

This flow defines the requirements for the model:

The embedding model is only one component. How documents are split (chunking) and whether keyword search and a reranker are combined with it (hybrid search and reranking) affect result quality just as much.

The models at a glance

The table lists the models that regularly come into question for German-language RAG systems in 2026. Figures from the respective model cards on Hugging Face, parameter counts from the model files.

Values in brackets: these models support Matryoshka Representation Learning (MRL). Vectors can be truncated to a smaller dimension with little loss in quality and a much smaller index.

Some specifics that matter in production:

• BGE-M3 produces three representations from one model: a dense vector, sparse per-token weights (similar to BM25) and ColBERT-style multi-vectors (model card). This enables hybrid search without a second model.
• Qwen3-Embedding expects an instruction on the query side (Instruct: ... Query: ...), documents are embedded without one. Without the instruction, retrieval performance drops by about 1 to 5 percent according to Qwen. The instruction should be written in English even for German text, because the training instructions were mostly English (model card).
• multilingual-e5 requires the prefixes query: and passage:, the instruct variant an instruction on the query side. Input is truncated to 512 tokens (model card).
• jina-embeddings-v2-base-de is trained bilingually for German and English, explicitly including mixed input (model card).
• Jina AI has been part of Elastic since October 2025 (Elastic announcement). The models remain available on Hugging Face and are also offered through the Elastic Inference Service.

Licenses: not every open model is cleared for commercial use

Open weights do not automatically mean free use in a business. This mainly affects the Jina models, which rank high in many comparisons.

The jina-embeddings-v3 model card states it explicitly: using the model beyond AWS and Azure or on-premises within a company falls under CC BY-NC 4.0, and commercial use requires contacting sales (model card, License section). jina-embeddings-v4 was initially listed as CC BY-NC 4.0; according to the model card the correct license is the Qwen Research License, because the model is derived from Qwen2.5-VL-3B (model card).

For a production RAG system in a business, without an additional contract, this leaves BGE-M3, Qwen3-Embedding, multilingual-e5, Granite Embedding R2, jina-embeddings-v2-base-de and, under the Gemma terms, EmbeddingGemma.

What benchmarks say about German text

The Massive Text Embedding Benchmark (MTEB) is the usual reference. For choosing a German model it is of limited use, because the well-known leaderboards average across many languages and tasks.

Multilingual overall score. The Qwen3-Embedding model card publishes the multilingual MTEB (MMTEB) averages across all tasks, with comparison values as of 24 May 2025 (model card). For jina-embeddings-v5-text-small, Jina reports 67.7 (model card).

This average includes classification, clustering, bitext mining and other tasks that play no role in RAG.

German retrieval. Retrieval on German text is more meaningful for RAG. MTEB includes the benchmark MTEB(deu, v1) with the retrieval tasks GermanQuAD, GermanDPR, XMarket and GerDaLIR (MTEB benchmark definition). Results for the current models are not consistently available there. One task with values for almost all candidates is the German part of MIRACL (retrieval on German Wikipedia, hard-negatives variant). The following values come from the public MTEB results repository, as of September 2026.

Three observations from both tables:

1. Bigger is not automatically better. In German retrieval the 4B model beats the 8B model, even though the 8B model leads the overall score.
2. Overall scores can mislead. multilingual-e5-large-instruct ranks ahead of BGE-M3 in the MMTEB average, but clearly behind it in German retrieval.
3. Wikipedia is not your collection. Regulations, contracts, manuals, maintenance logs and scanned PDFs behave differently from encyclopedia articles. MTEB now includes RTEB(deu, beta), a German retrieval benchmark with legal, healthcare and business data that partly relies on non-public datasets (benchmark definition).

Benchmarks are useful for narrowing the list to three or four candidates. The decision is made by a test set of real questions with their matching passages from your own collection. How to build and evaluate such a test set is described in the knowledge article RAG evaluation.

Context length, dimensions and storage

Context length. The maximum input length caps the chunk size. multilingual-e5 cuts off after 512 tokens, and the rest of a longer section does not flow into the vector. BGE-M3 and jina-embeddings-v2-base-de handle 8,192 tokens, Qwen3-Embedding, Granite Embedding R2 and jina-embeddings-v5 up to 32,768 tokens. A long input length does not mean that long chunks search better: one vector for ten pages represents their content only roughly. It does, however, allow sections to be cut along the document structure instead of a fixed token count.

Dimension and index size. In pgvector a vector takes 4 × dimensions + 8 bytes, the vector type can be indexed up to 2,000 dimensions, halfvec up to 4,000 (pgvector README). For one million chunks, excluding index structures, this gives:

The Qwen3 models with 2,560 and 4,096 dimensions can be truncated to 1,024 dimensions via MRL. Whether the quality loss is acceptable belongs in the test set. Qdrant does not have this index limit; there, large dimensions affect memory and search latency. The differences between the two databases are covered in Qdrant vs. pgvector.

VRAM and hardware

Embedding models are small compared to language models. In FP16 or BF16 the weights take about two bytes per parameter:

On top of that comes memory for processing, which grows with batch size and input length. With 8,192 or 32,768 tokens per input, this share is not negligible.

For hardware planning this means:

• Models up to about 600M parameters also run on a CPU. For search queries in production that is often enough. Initial indexing of a large collection, however, takes many times longer on a CPU than on a GPU.
• Embedding model, reranker and language model share GPU memory when they run on the same machine. On a 24 GB GPU such as the NVIDIA RTX PRO 4000 Blackwell, a Qwen3-Embedding-8B (approx. 15 GB) leaves little room for a language model. With BGE-M3 or Qwen3-Embedding-0.6B most of the memory remains free.
• Initial indexing is the peak load. After that, only new and changed documents are embedded, plus user queries of a few dozen tokens each.

How VRAM for the whole system of language model, KV cache, embedding and reranker adds up is explained in Sizing GPU and VRAM.

Serving with TEI, vLLM and Ollama

Three inference servers are typically considered for embedding models. All three provide an HTTP API, vLLM and TEI additionally in OpenAI-compatible format.

Sources: TEI, supported models, vLLM, embedding models, vLLM, BGE-M3, Ollama library, Ollama embed API. As of September 2026.

What matters in production:

• Sparse vectors from BGE-M3. vLLM provides them when the model is started with an overridden architecture, because the config.json otherwise loads it as plain XLM-RoBERTa:

vllm serve BAAI/bge-m3 \
  --runner pooling \
  --hf-overrides '{"architectures": ["BgeM3EmbeddingModel"]}' \
  --pooler-config.task token_classify

With --pooler-config.task embed the same instance returns dense vectors. In TEI, sparse support for BGE-M3 has been submitted as a pull request but not merged yet.

• Blackwell GPUs in TEI. For Blackwell-generation GPUs with compute capability 12.0, which includes the RTX PRO 4000 and 6000 Blackwell, TEI ships a separate image marked as experimental (TEI documentation).
• Prefixes and instructions have to be set in the application, not in the server. A pipeline step that forgets query: for E5 or omits the instruction for Qwen3 degrades search without any error message.
• Custom code. jina-embeddings-v2 and v3 load their own model code (trust_remote_code). In production this is an additional code dependency that has to be reviewed with every update.

Ollama suits prototypes and single workstations. For production systems with many concurrent queries and initial indexing of large collections, TEI or vLLM are the more robust options. The comparison of inference servers for language models is in vLLM vs. Ollama vs. llama.cpp; we set them up as vLLM operations or with Ollama.

Which model for which use case

The table is a starting point for the candidate list, not a ranking. Only the test set confirms or refutes the choice.

For the API option: during indexing the entire document collection is transferred to the provider, and in operation every search query. Which European providers offer which embedding models is compared in European LLM APIs compared. Embedding and reranker need little compute and can run locally even when the language model is accessed through an API.

Changing the model means re-indexing

Vectors from different models cannot be compared. Changing the embedding model means re-embedding the whole collection. The same applies to changing the dimension via MRL, the prefixes or the instruction.

Practical consequences:

1. Document model, revision, dimension and instruction. Existing models on Hugging Face change too; for jina-embeddings-v3, for example, a bug in Matryoshka truncation was fixed after release (model card). A pinned revision in the deployment prevents vectors from changing unnoticed.
2. Build the new index in parallel. The old index stays active until the new one is complete and performs at least as well on the test set.
3. Run the test set again. Compare recall and ranking metrics before and after the switch, not just samples.
4. Carry permissions over. The new index must hold the same access information per chunk as the old one, see RAG with permissions.

Our approach at WZ-IT

We do not choose the embedding model from a leaderboard, but by measuring on your collection.

1. Capture collection and questions. Document types, languages, length, scanned share and typical questions from the business units.
2. Narrow down candidates. After license, context length, dimension and hardware, usually three to four models remain.
3. Measure with reference questions. In the RAG Proof of Value we work with 40 to 60 reference questions including source and no-answer tests, with a baseline measurement and two optimization cycles.
4. Set up operations. Embedding model and reranker run with TEI or vLLM on the AI Cube in your network or on a managed GPU server from WZ-IT with NVIDIA RTX PRO 4000 Blackwell (24 GB) or RTX PRO 6000 Blackwell (96 GB).
5. Document and hand over. Model revision, dimension, instructions and test set are recorded so that a later model change remains measurable.

Support, consulting and implementation by WZ-IT. Building it into a production assistant with permissions and source references is described on the internal AI assistant page.

Further guides

• Hybrid search and reranking, combining keyword search, dense vectors and cross-encoders.
• Chunking strategies, chunk sizes, structure and metadata.
• RAG evaluation, test sets, retrieval and answer metrics.
• European LLM APIs compared, providers with embedding and reranker models in the EU.
• RAG knowledge base in the public sector, building one for public authorities with in-house operation.
• AI solutions from WZ-IT, the hub with all AI services.

Which embedding model fits your documents? We measure the candidates with reference questions from your collection and run the chosen model on your hardware or on a managed GPU server from WZ-IT. Schedule a meeting

Sources

• BAAI, BGE-M3 model card
• Qwen, Qwen3-Embedding-8B model card
• Qwen, Qwen3-Embedding-0.6B model card
• Qwen, Qwen3-Embedding-4B model card
• intfloat, multilingual-e5-large model card
• intfloat, multilingual-e5-large-instruct model card
• Google, EmbeddingGemma model card
• Google, Gemma Terms of Use
• IBM, Granite Embedding 311M Multilingual R2 model card
• Jina AI, jina-embeddings-v2-base-de model card
• Jina AI, jina-embeddings-v3 model card
• Jina AI, jina-embeddings-v4 model card
• Jina AI, jina-embeddings-v5-text-small model card
• Elastic, Jina AI joins Elastic
• Mistral AI, Mistral Embed model card
• Mistral AI, text embeddings
• MTEB, benchmark definitions
• MTEB, RTEB benchmark definitions
• MTEB, results repository
• MTEB leaderboard
• pgvector README
• Hugging Face, Text Embeddings Inference: supported models and hardware
• Text Embeddings Inference, pull request for BGE-M3 sparse
• vLLM, supported embedding models
• vLLM, pooling models: BGE-M3
• Ollama, Qwen3-Embedding in the model library
• Ollama, embed API