Skip to main content
The ngrok AI Gateway routes requests to AI providers like OpenAI and Anthropic. Get started with an API key from ngrok—no provider accounts needed. Bring your own keys to access additional providers like Google, DeepSeek, and more.

​
Why use the AI Gateway?

No Provider Accounts Needed

Use OpenAI, Anthropic, and more without signing up for each provider. ngrok manages the provider keys for you.

Automatic Failover

If one provider fails, the gateway automatically tries the next model, provider, or key.

Compatible With Popular SDKs

Works with official and third-party SDKs. Change the base URL and use your AI Gateway API Key.

Self-Hosted Models

Route to local models like Ollama or vLLM alongside cloud providers using Bring Your Own Keys.

​
Quick example

Point your SDK at your ngrok endpoint and use your AI Gateway API Key: 
from openai import OpenAI

client = OpenAI(
    base_url="https://your-ai-gateway.ngrok.app/v1",
    api_key="ng-xxxxx-g1-xxxxx"  # Your AI Gateway API Key
)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}]
)
On each request, the AI Gateway:
  1. Receives your request with your AI Gateway API Key
  2. Validates your key
  3. Selects which model and provider to use
  4. Forwards the request with ngrok’s managed provider API keys
  5. If it fails, retries with the next option in your failover chain
  6. Returns the response

​
What can you do?

Use CaseDescription
Use without provider accountsGet started with OpenAI and Anthropic using just an AI Gateway API Key
Automatic model selectionUse ngrok/auto to let the gateway pick the best model
Multi-provider failoverIf one provider fails, automatically try another
Custom selection strategiesDefine exactly how models are selected using CEL expressions
Cost-based routingRoute to the cheapest available model automatically
Access controlRestrict which providers and models clients can use
Self-hosted modelsRoute to Ollama, vLLM, or other local inference servers (requires BYOK)
Content modificationRedact PII, sanitize responses, or inject prompts
AI Gateway requires the Pay-as-you-go plan. See Credits for pricing details.

​
Next steps

Quickstart

Get your AI Gateway running in 5 minutes

How It Works

Understand the request flow and failover behavior

SDK Integration

Connect your application to the AI Gateway

Bring Your Own Keys

Already have provider API keys? Use them with the AI Gateway