11 — Technology category · TR-60.60

Model & MCP Gateways

A proxy in front of model provider APIs and MCP servers that gives applications one endpoint and centrally holds the keys, routes and falls back between models, applies quotas and policy, caches responses and records cost per caller.

LLM Gateway
LLM Proxy
MCP Gateway
Model Router

In scope

One endpoint across model providers with fallback routing
Custody of provider keys and per-team quotas
Response caching and cost tracking per caller
Allow lists of which applications may reach which models, MCP servers and tools

Out of scope

General-purpose gateways for application APIs (see TR-60.20)
Inspecting prompts, responses and tool calls for attacks (see TR-110.80)
Tracing and scoring of model calls (see TR-110.60)

Example technologies · 6

Well-known technologies in this category, named as facts. Not a shortlist or an endorsement.

LiteLLM
Kong AI Gateway
Cloudflare AI Gateway
Prisma AIRS AI Gateway (formerly Portkey)
Amazon Bedrock AgentCore Gateway
TrueFoundry AI Gateway

In the catalog · 1

LiteLLM platform