LLM gateway comparison: LiteLLM, OpenRouter and four alternatives

Operate an LLM gateway yourself instead of distributing keys across every application? WZ-IT operates LiteLLM as a self-hosted LLM gateway in Germany, with virtual keys, budgets, fallbacks and connections to vLLM, Ollama and Langfuse. LLMOps covers tracing and evals in running operation. View LiteLLM hosting · View LLMOps

As soon as several applications or teams use language models, the question arises of a central place for keys, budgets and failover. This comparison sets six gateways side by side on operations, licence, routing, budgets, keys, Langfuse connection, guardrails, streaming and data flow. The information comes from documentation, licence files and price pages retrieved on 28 October 2026; nothing was tested. Two acquisitions this year (Portkey completed, OpenRouter agreed) are part of the comparison. What a gateway does in principle is explained in What is LiteLLM?. As of October 2026.

Table of contents

• Summary
• The six gateways at a glance
• Operations, licence and owner
• Routing, budgets and keys
• Observability, protection and data flow
• Assessment of the other gateways
• OpenRouter in the enterprise
• LiteLLM: licence, operations and security
• Decision table
• Questions before you choose
• Status and methodology
• How WZ-IT does it
• Further guides

Summary

• Operations decide first: Services (OpenRouter, Cloudflare) save operating effort and insert one more party between application and model provider. Self-operation (LiteLLM, Portkey, Kong, Agent Router) needs updates, owners and, depending on the product, a database or Redis.
• Licence: LiteLLM and Portkey are under MIT, LiteLLM with a licensed enterprise/ directory, Portkey with a commercial Enterprise version. Agent Router is under Apache 2.0. At Kong, load balancing, spend limits and PII masking are Enterprise only.
• Owner: Portkey has belonged to Palo Alto Networks since 29 May 2026; Stripe's acquisition of OpenRouter has been agreed since 19 August 2026.
• Cost: OpenRouter charges 5.5% (Standard) or 8% (Business) when credits are bought. LiteLLM without an Enterprise licence and Agent Router cause operating costs instead of fees.

The six gateways at a glance

The selection covers gateways whose documentation was read: LiteLLM (open-source proxy), OpenRouter and Cloudflare AI Gateway (services), Portkey (open-source gateway with a hosted platform), Kong AI Gateway (AI plugins for the Kong API gateway) and Agent Router, called Envoy AI Gateway until 10 September 2026, for Kubernetes. Connecting directly to the model provider remains the alternative when only one application uses one model.

Operations, licence and owner

The ownership changes are in the press releases from Palo Alto Networks and Stripe. Neither announcement says whether terms, subprocessors or the open-source branch change as a result; that is a check before signing. On 19 August 2026 Stripe announced that it had agreed to acquire OpenRouter; in its own announcement OpenRouter expected to close "in the coming weeks" and writes that product and current commitments remain unchanged. No closing announcement was to be found in the sources named here on 28 October 2026. portkey.ai carries the notice "Portkey is now PRISMA AIRS AI Gateway", and on 16 July 2026 Palo Alto Networks announced the general availability of the Prisma AIRS AI Gateway, which takes in innovations from the Portkey acquisition. In its repository, Portkey announces that the core of its enterprise gateway will move into open source with version 2.0 (pre-release).

Routing, budgets and keys

Observability, protection and data flow

Langfuse describes guides for LiteLLM, OpenRouter, Portkey (through the OpenAI-compatible Langfuse client) and Kong (through the OpenTelemetry plugin). For Cloudflare and Agent Router the path runs via OpenTelemetry export; Langfuse accepts OTLP over HTTP but not over gRPC (Langfuse). In self-operation the gateway itself adds no further party; prompts still reach the connected model providers, and with self-hosted models they stay in your own network (vLLM vs. Ollama).

Switching between gateways. All six document an OpenAI-compatible interface (at Portkey as part of the Universal API). Base URL and model names can be swapped; virtual keys, budgets, guardrail configurations and dashboards cannot. Anyone who wants to be able to switch keeps this configuration in files instead of only in the interface (assessment).

Assessment of the other gateways

• Portkey: The MIT gateway covers routing, fallbacks, load balancing and guardrails. Logs and traces come from the hosted app; budget limits exist for Enterprise and selected Pro customers, SSO only in Enterprise. The acquisition press release says nothing about the open-source gateway or existing customers.
• Kong: Fits when Kong already carries the API traffic. AI Proxy translates requests to several providers, including OpenAI, Anthropic, Mistral, Ollama and vLLM, and applies from Kong Gateway 3.6. For multiple targets, spend limits and PII masking, the documentation names Enterprise plugins; "Enterprise" there means "AI License Required" or "AI Gateway Enterprise". Kong also documents "AI Gateway 2.x" with a control plane managed in Konnect and self-operated data plane nodes (architecture); this comparison describes the plugin variant on Kong Gateway.
• Cloudflare: Analytics, cache and rate limiting without your own operations. Spend limits, dynamic routing, DLP, guardrails and BYOK are marked beta in the documentation.
• Agent Router: The prerequisites are in the documentation. Release 1.2 of 9 October 2026 was the first under the new name; CRDs, Helm charts and CLI stayed unchanged (release).

OpenRouter in the enterprise

OpenRouter bills inference at the provider's list price and charges the platform fee when credits are bought (pricing). Assuming the fee is the percentage of the purchase amount:

With your own provider keys (BYOK), the fee is waived on Standard and Business up to 25,000 US dollars of list-price inference per month; after that 5 percent of the list price of the request applies. The contracts with the model providers then stay with you.

On data flow: OpenRouter stores prompts and responses only with opt-in (exception: content flagged as suspected CSAM), and metadata such as token count and latency is stored (privacy). The data_collection parameter defaults to allow, which permits providers that store data and may train on it; deny and zdr restrict that (provider selection). Requests to eu.openrouter.ai are processed at providers in Europe (EU/EEA, United Kingdom, Switzerland), but only on Business and Enterprise. Account, billing, authentication and security data may be stored or processed in the US (in-region routing). OpenRouter names a data processing agreement with standard contractual clauses, whose EU addendum for API payloads must be elected in writing.

LiteLLM: licence, operations and security

• Licence: MIT, except enterprise/. Enterprise is a licence key on the same image; SSO is free for up to five users, audit logs, multi-tenancy and further features need the licence, which is priced by usage (Enterprise).
• Operations: PostgreSQL is required for keys, teams and spend, a second instance adds Redis 7.0 or newer, and per pod the documentation recommends 1 vCPU and 4 Gi of memory, with 4 Gi as the floor. Versions are pinned, not latest (production).
• Security: The gateway holds the keys of all models. The advisories of 30 September and 1 October 2026 and the compromised PyPI versions of March 2026 are described in What is LiteLLM? and LiteLLM vulnerability: hardening the gateway. On 27 October 2026 another critical advisory followed (GHSA-w4c7-34wj-hqm8): it affects proxies with an active Enterprise licence; according to the advisory, open-source deployments without a licence are not affected.

An alternative follows from the goal, rarely from the licence: no operations (service), a Kubernetes platform with Envoy (Agent Router), existing Kong (Kong plugins).

Decision table

Our assessment from the information above:

Questions before you choose

1. Which data classes pass through the gateway, and may an intermediary see them?
2. Where do gateway, database, logs and backups run?
3. Which required features are in the open-source version, which behind a licence?
4. Who is the owner, and what applies to contract and subprocessors after a change?
5. How are keys, teams and budgets managed, and who rotates keys?
6. How do traces reach Langfuse: callback, Broadcast or OTLP?
7. Who applies updates, and how quickly after an advisory?

Status and methodology

• Status: 28 October 2026. All information comes from the vendors' documentation, licence files and price pages. There were no functional tests; features marked beta are named as such.
• Assumptions: The fee calculation assumes that the fee equals the percentage of the purchase amount.
• Not covered: Helicone, MLflow AI Gateway, TensorZero and hyperscaler gateways; latency and throughput.

How WZ-IT does it

We operate LiteLLM as a self-hosted LLM gateway in Germany: with virtual keys, budgets, provider fallbacks, logging, network segmentation and connections to vLLM, Ollama and Langfuse (LiteLLM hosting). Beforehand we check whether Enterprise features such as SSO for more than five users, audit logs or a secret manager are needed. LLMOps monitors tracing, evals, cost and latency of the application behind the gateway with Langfuse. For model operations, LLM hosting and the managed GPU server are available.

Further guides

• What is LiteLLM? - gateway basics and operating rules
• What is Langfuse? - tracing and evaluation
• LiteLLM vulnerability: hardening the gateway - advisory and hardening
• The open-source LLM stack - where the gateway sits
• vLLM vs. Ollama - model operations behind the gateway
• MCP in the enterprise - gateways for MCP servers

Make the gateway decision or have LiteLLM operated? WZ-IT runs LiteLLM with budgets, keys and a Langfuse connection and monitors the application behind it. LiteLLM hosting · LLMOps · Book a call