Tetrate

Tetrate: Enterprise AI gateway built on Envoy for routing models and attributing agent spend

Tetrate is the enterprise service-mesh company founded by early engineers behind the Istio and Envoy open-source projects. Its Agent Router product line applies that traffic-engineering pedigree to AI workloads: an Envoy-based gateway that sits between your agents and every LLM provider, exposing a single OpenAI-compatible endpoint so teams can route, fail over, and meter model traffic without rewriting agent code.

The product comes in two forms. Tetrate Agent Router Service is the hosted, self-serve edition: sign up with GitHub or Google, receive $5 in free credit, and pay as you go at model cost plus a 5% routing fee. Agent Router Enterprise adds cross-team spend attribution, admin controls, SSO-based developer onboarding, AI guardrails, and deployment across cloud, on-prem, edge, or regional configurations.

Key Features

• Multi-model routing with automatic failover across providers, so agents keep working when a model or vendor degrades
• 100% OpenAI-compatible endpoint requiring zero agent rewrites; typical time to first request is under five minutes
• Per-team, per-agent, and per-project token and cost attribution for chargeback and budget control
• AI Guardrails enforcing PII redaction, prompt filtering, policy compliance, and transaction blocking
• MCP Gateway for securely connecting agents to tools at scale
• Unified logging across providers and frameworks for debugging multi-agent chains in one place

Ideal Use Case

Platform and infrastructure teams at enterprises where multiple departments ship AI agents against different model providers. Instead of every team managing its own API keys, retries, and spend spreadsheets, Tetrate provides a central gateway with an approved LLM catalog, scoped access profiles via SSO, and finance-grade attribution of exactly which team, agent, and project consumed which tokens.

How Tetrate differentiates

Most AI gateways are young startups; Tetrate has operated traffic-critical infrastructure for years. The company was founded by early engineers behind Istio and Envoy, raised a $40M Series B led by Sapphire Ventures ($52.5M total funding), and builds Agent Router on Envoy — the proxy with roughly 28K GitHub stars that is production-tested at Netflix, Lyft, and Airbnb. That lineage matters when the gateway sits in the hot path of every agent request.

FAQ

Does Tetrate Agent Router require changing my agent code? No. It exposes a 100% OpenAI-compatible endpoint, so pointing existing agents at the gateway is a base-URL and API-key change rather than a rewrite.

How is Tetrate Agent Router Service priced? Pay-as-you-go: you pay the underlying model cost plus a 5% fee. New sign-ups receive $5 in free credit to start routing immediately.

Can Tetrate run on-premises? Yes. Agent Router Enterprise supports cloud, on-prem, edge, and regional deployments for organizations with data-residency requirements.

What guardrails are included? PII redaction, prompt filtering, policy compliance checks, and transaction blocking, all enforced centrally at the gateway rather than per agent.

tl;dr

Tetrate turns its Envoy service-mesh pedigree into an enterprise AI gateway: one OpenAI-compatible endpoint for all models, automatic failover, centrally enforced guardrails, and per-team attribution of agent token spend.