Request ID, attempt count, and served model on every response
Gateway responses now state how the request was handled:Ngrok-AIG-Request-Id: the request’s ID, on every response including rejections and errors.Ngrok-AIG-Attempts: how many upstream attempts the gateway made.Ngrok-AIG-Model: the upstream model that produced the response.
generateContent and streamGenerateContent surface returns these headers too, where it previously returned only Content-Type.
Bodies on that surface are still forwarded byte-for-byte.The ai-gateway Traffic Policy action is retired
The AI Gateway no longer runs as a Traffic Policy action on your own endpoint. A policy that still contains theai-gateway action fails validation with ERR_NGROK_3820.- Send LLM traffic to
https://gateway.ngrok.aiwith an access key, and manage the gateway at app.ngrok.ai. - Existing credits were migrated to app.ngrok.ai automatically. No action is needed to keep your balance.
- Remove the
ai-gatewayaction from any endpoint that still carries it. - The action’s result variables, such as
actions.ngrok.ai_gateway, are gone. Read the response headers and the Usage API instead.
app.ngrok.ai and gateway.ngrok.ai
Manage your AI Gateway from app.ngrok.ai and send traffic to a shared gateway URL:- Manage access keys, access key configurations, provider keys, credits, and settings at app.ngrok.ai.
- Send LLM traffic from your applications to
https://gateway.ngrok.aiwith an access key only—provider keys are stored server-side. - Updated the Quickstart, Overview, How It Works, and Access Keys.
- Provider keys are now account-scoped in app.ngrok.ai BYOK.
- Endpoint-based Traffic Policy setup is deprecated.
AI Gateway API keys & credits
You can now authenticate requests to your AI Gateway with AI Gateway API Keys—no provider accounts needed. The AI Gateway manages the provider keys for OpenAI and Anthropic on your behalf. Purchase prepaid credits to fund usage.- AI Gateway API Keys: Create keys to authenticate requests
- Credits: Understand pricing and balance management
- Bring Your Own Keys: Use your own provider keys for additional providers
- Updated Quickstart, Overview, and all SDK guides to use AI Gateway API Keys as the primary path
Native Anthropic SDK support
Use Anthropic’s official Python and TypeScript SDKs directly with the AI Gateway. The gateway now supports the native/v1/messages API format alongside the OpenAI-compatible format. See the Anthropic SDK guide for setup instructions.Groq provider support
Groq is now a built-in provider. Access Llama and Mixtral models running on Groq’s LPU hardware with automatic failover and observability.Expanded model catalog
Added new providers to the model catalog: Hyperbolic, InceptionLabs, Inference.net, and OpenRouter. Updated model aliases and provider-specific model IDs for improved compatibility.Early access release
Initial Early Access release of the AI Gateway.Features
- OpenAI-compatible API for routing requests to multiple AI providers
- Support for OpenAI, Anthropic, Google, and other providers
- Multi-provider failover across models, providers, and API keys
- Multi-key failover per provider
- CEL-based model selection strategies
- Custom provider support via internal endpoints
- Token counting and usage metrics