Deciding which engine to run language models on in your company? In LLM hosting, vLLM runs with an OpenAI-compatible API and operations by WZ-IT. Anyone who needs a dedicated GPU can use the managed GPU server from €699 net per month. View LLM hosting · Clarify the sizing together
The question "SGLang or vLLM" is usually answered with a factor: x times faster, y percent lower GPU costs. Such numbers hold for one model, one version, one GPU and one load profile, and published tests reach opposite results. This article therefore compares what both projects document and provides a decision table and a measurement plan for your own load. The statements come from the project documentation, the repositories and the GitHub API, the benchmark statements from the source pages named, as of 28 October 2026. The article contains no benchmarks of its own.
Table of contents
- Short answer
- Architecture: two data structures for the same purpose
- Features and hardware according to the documentation
- Maturity and release cadence
- Benchmarks: sources and their conditions
- Measuring your own comparison
- Decision table
- Operations and switching
- As of and methodology
- Our approach at WZ-IT
- Further guides
Short answer
Both engines are Apache-2.0 licensed, run on NVIDIA and AMD hardware, speak the OpenAI API and cover the usual features. The difference lies less in the feature list than in behaviour under load with recurring prefixes and in maturity per model and platform.
- Without a noticeable prefix share (single documents, classification, batch), two published tests show a tie, a third sees vLLM ahead on throughput and first token. The choice then follows model support and operations.
- With a high prefix share (agents with long tool descriptions, long system prompts, multi-turn conversations), two vendor tests report advantages for SGLang, a third with a different load sees vLLM ahead. A comparison of your own is worthwhile here.
- Blanket factors from blog articles do not transfer, see the table of test conditions below.
Architecture: two data structures for the same purpose
vLLM manages the KV cache in blocks (PagedAttention, paper) and groups requests with continuous batching. Prefix caching (APC) forms a hash for every full block from the hash of its predecessor, the tokens of the block and extra values such as the LoRA ID or a cache salt for tenant isolation; only full blocks are cached (vLLM, design). In the source code the method is on by default as long as the model supports it (arg_utils.py).
SGLang stores the KV cache in a radix tree (RadixAttention, paper). The tree is on by default (--disable-radix-cache is False); for individual models and operating modes SGLang switches it off itself, for example on decode servers of prefill/decode separation. Under memory pressure a policy picks the prefixes to evict: lru as default, plus lfu, slru, priority and tlru, the last one for agents and multi-turn conversations with a target for time to first token (SGLang, eviction). HiCache adds two further tiers for the KV cache: CPU memory and external storage.
For an identical system prompt, both behave the same: the shared part is not recomputed. How a hash over blocks and a radix tree differ for partially overlapping prompts depends on block size and load profile; no documentation quantifies it. Both methods only shorten prefill. vLLM itself states that prefix caching brings no gain for long answers, because generation dominates there (vLLM, APC).
Features and hardware according to the documentation
| Feature | vLLM | SGLang |
|---|---|---|
| Licence | Apache-2.0 | Apache-2.0 |
| Steward | listed on pytorch.org as a PyTorch Foundation project | hosted by LMSYS (non-profit) |
| Orientation per README | "Easy, fast, and cheap LLM serving for everyone" | optimised for agentic workloads, large-scale serving and RL rollouts |
| Hardware | NVIDIA, AMD, Intel, CPUs (x86, ARM, PowerPC, IBM Z experimental); via plugins e.g. TPU, Gaudi, Ascend, Apple Silicon | NVIDIA (e.g. RTX PRO 6000, DGX Spark), AMD Instinct, TPU v6e/v7, Intel, Apple Silicon, Ascend |
| Speculative decoding | EAGLE, MTP, draft model, PARD, n-gram, suffix, DFlash | EAGLE-2/3, MTP, draft model, NGRAM, DFlash, UNO |
| Structured output | xgrammar, guidance | xgrammar (default), Outlines, llguidance |
| Tool calling | parser per model (--tool-call-parser) |
parsers e.g. for DeepSeek, Llama, Qwen, Mistral, GLM, Kimi K2 |
| Multi-LoRA | dense and MoE | several adapters per batch (S-LoRA, Punica) |
| Quantisation | FP8, MXFP8/MXFP4, NVFP4, INT8, INT4, GPTQ, AWQ, GGUF and more | FP8, MXFP4, AWQ, GPTQ, ModelOpt (FP8/FP4) and more |
| Prefill/decode separated | documented, marked experimental | PD disaggregation with Mooncake and NIXL |
| Expert parallelism | --enable-expert-parallel, DeepEP backends, load balancing EPLB (--enable-eplb) |
--moe-a2a-backend (DeepEP, Mooncake, NIXL), load balancing EPLB |
| API | OpenAI-compatible, Anthropic Messages, gRPC | OpenAI-compatible, Anthropic- and Ollama-compatible |
| Metrics | Prometheus /metrics, prefix vllm:, prefix cache hits |
Prometheus with --enable-metrics, prefix sglang:, cache_hit_rate |
| Container | vllm/vllm-openai |
lmsysorg/sglang (CUDA images only with CUDA 13 now) |
Sources vLLM: README, features, installation, metrics, PyTorch Foundation. Sources SGLang: README, documentation, speculative decoding, PD separation, expert parallelism, metrics.
The table shows that something is documented, not that it runs smoothly for a particular model on a particular GPU. Only a test with the target model shows that.
Maturity and release cadence
| vLLM | SGLang | |
|---|---|---|
| Current version | v0.32.0, 18 October 2026 | v0.5.22, 19 October 2026 |
| Previous versions | v0.31.0 (5 Oct), v0.30.0 (22 Sep), v0.29.0 (9 Sep), v0.28.0 (26 Aug) | v0.5.21 (2 Oct), v0.5.20 (18 Sep), v0.5.19 (5 Sep), v0.5.18 (22 Aug) |
| Interval | 13 to 14 days | 13 to 17 days |
Both projects release roughly every two weeks and carry a version number below 1.0. In practice that means: pin versions, read release notes, test updates. Examples of breaking changes from the documentation: since v0.5.20 SGLang no longer publishes CUDA 12 images (last tag v0.5.19-cu129, installation); vLLM removed the request fields guided_* in v0.12.0 in favour of structured_outputs (vLLM, structured outputs). GitHub stars are deliberately not a criterion here: they measure awareness, not suitability.
Benchmarks: sources and their conditions
| Source | Who measures | Conditions per the source | Result (qualitative) | Limit |
|---|---|---|---|---|
| SGLang paper, December 2023 | SGLang authors | comparison with systems of the time (including vLLM v0.2.5), mostly A10G GPUs, tasks such as agent control, JSON decoding, RAG, multi-turn chat | up to 6.4x higher throughput | status of 2023; vLLM has had prefix caching since |
| Spheron, updated 19 October 2026 | GPU cloud provider | 1x H100, Llama 3.3 70B in FP8, vLLM v0.18.0 and SGLang v0.5.9; runs with unique prompts and with 80% shared prefix | tie without prefix, SGLang with shorter time to first token for a shared prefix | versions far below today's |
| RunPod, updated 13 September 2026 | GPU cloud provider | 2x H100, DeepSeek-R1-Distill-Llama-70B, multi-turn prompt with about 7,000 tokens of context | SGLang ahead on cache hits; vLLM faster on a single request without context | single requests, no load |
| Jarvislabs, 18 May 2026 | GPU cloud provider | H100, Qwen2.5-7B, Qwen3-30B-A3B, Qwen3-32B; ShareGPT and RULER 16K | vLLM as the strongest default; SGLang with lower decode latency in some runs but a later first token | versions not stated |
| Red Hat Developers, 14 October 2026 | trade article, reports the Modal measurement | Modal: more than a dozen models, ten combinations of input and output length, NVIDIA GPUs, default settings, prompts from random novel excerpts | both comparable | hardly any shared prefixes; the article names this limit itself |
| InferenceX (SemiAnalysis), continuous | analyst firm, Apache-2.0 repository | runs through public GitHub Actions workflows, recipes in the repository, many models and GPUs | no single figure in this article | hardware provided by AMD, NVIDIA and cloud providers according to its acknowledgements |
The deviations follow from the conditions: versions differ by more than ten releases, models and quantisation change, and the metric is sometimes throughput in tokens per second, sometimes time to first token, sometimes latency per output token. A test with a high prefix share and one with random prompts measure different things. Numbers from these sources therefore do not belong in a hardware decision, but the direction does: the prefix share is the quantity that tips the balance.
Measuring your own comparison
- Derive the load profile from your own logs: input and output length, share of common prefixes (system prompt, tools, conversation history), concurrent requests at peak. The number of concurrent users is usually smaller than the number of people with access.
- Create equal conditions: same GPU, same model, same quantisation, pin container tags, record both versions, start parameters from the projects' recipes.
- One measurement client for both endpoints: for production servers vLLM recommends GuideLLM (vLLM, benchmark CLI);
python -m sglang.bench_servingsupports avllmbackend (SGLang, bench serving). - Concurrency levels instead of a single value: one request, the target value and double it, several runs per level after warm-up; report cold and warm cache separately.
- Read metrics together: time to first token (p50 and p95), latency per output token, throughput under a fixed latency limit and the cache hit rate from
/metrics. - Spot-check quality: compare outputs and tool calls for the same prompts.
- Include operations in the assessment: start-up time, memory footprint, behaviour under overload, update path.
Decision table
| Load | First candidate | Rationale | To measure |
|---|---|---|---|
| Chat assistant with a shared system prompt on one GPU server, 5 to 50 concurrent users | vLLM; test SGLang against it with a high prefix share | Both cache the shared prompt; the difference shows with branching histories | time to first token (p95), hit rate |
| Agents with long tool descriptions, many steps | test SGLang, vLLM as reference | SGLang names agentic workloads as its target and offers tlru for multi-turn conversations |
latency per step, time to finished task |
| Batch with varying prompts (classification, extraction) | vLLM | the prefix advantage disappears; two tests show a tie, one has vLLM ahead | requests per second under a latency limit |
| Large MoE models across several GPUs or nodes | assess both on equal terms | both document expert parallelism and prefill/decode separation; the recipe for the model decides | the project's recipe, network (RDMA) |
| JSON schema output with high repetition | both | both support xgrammar | duration with and without schema |
| AMD, TPU or Apple Silicon | check the hardware pages of both projects | support differs per model and version | model runs stably |
Operations and switching
Both engines provide an OpenAI-compatible API; a switch usually changes only the base URL and model name for clients. Check tool call parser, chat template, reasoning parser and structured output per model. A gateway such as LiteLLM can run both behind one address, for example for an A/B test with real traffic. For operations: metrics (vllm: and sglang: prefix), pinned image tags and an update window per release. GPU sizing is separate from the engine choice: GPU and VRAM for LLMs and multi-GPU inference cover it.
As of and methodology
Retrieved on 28 October 2026: versions, release dates and licences via the GitHub API and the LICENSE files (both Apache License 2.0); features and hardware from the README and documentation of both projects (source files of the documentation repositories); default behaviour of prefix caching from vllm/engine/arg_utils.py and the SGLang server argument reference. Benchmark sources are listed with date and the conditions they state; numbers from them are not adopted. It was not checked whether every listed feature works with every model. The tables are updated with new releases.
Our approach at WZ-IT
vLLM runs in LLM hosting, on the managed GPU server and in the AI Cube, together with Open WebUI for teams with several concurrent users. Sizing starts with the load profile, that is concurrent users, context lengths and prefix share; model and hardware follow. The difference from Ollama is covered in vLLM vs. Ollama.
Further guides
- What is vLLM? - basics of the engine
- vLLM vs. Ollama - entry level versus throughput
- vLLM, Ollama or llama.cpp - comparison with a section on SGLang
- Multi-GPU inference without NVLink - tensor, pipeline and data parallelism
- GPU and VRAM sizing for LLMs - KV cache and context length
- LLM hosting, managed GPU server and vLLM expertise
Which engine fits your load is shown by a measurement with your model. We size hardware and model to your load profile; vLLM runs as the model server. View LLM hosting · Book a call
