Skip to main content
The ngrok AI Gateway (ngrok.ai) enables your applications to call both hosted AI providers and local models through a single endpoint. Control who has access, decide what they can call, and choose whether requests use credits, your provider keys, or a model you run yourself.

​
Why use the AI Gateway?

Start without provider accounts

Use built-in OpenAI and Anthropic with an access key and credits—no OpenAI or Anthropic signup required.

Route to Self-Hosted Models

Route to local models like Ollama or vLLM alongside cloud providers with a model you run yourself.

Automatic Failover

If one provider or key fails, ngrok.ai automatically tries the next model or key.

Compatible With Popular SDKs

Works with official and third-party SDKs. Point the base URL at gateway.ngrok.ai and use your access key.

​
Quick example

Sign up at app.ngrok.ai and point your SDK at the gateway with your access key: 
from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.ngrok.ai/v1",
    api_key="ng-xxxxx-g1-xxxxx"  # Your access key
)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}]
)
You don’t need to sign up for OpenAI or bring your own OpenAI key to make this request. Your ngrok.ai credits will cover any provider costs.
Every request consumes credits. No subscription plan required to get started. See Credits for more details and when you might need a ngrok subscription for features like team members or internal endpoints.
On each request, the AI Gateway:
  1. Receives your request with your access key at gateway.ngrok.ai
  2. Validates the key and loads your account configuration
  3. Selects which model and provider to use
  4. Forwards the request using ngrok.ai inference or your provider keys
  5. Retries with the next key or model on failure
  6. Returns the response

​
What can you do?

Use CaseDescription
Use without provider accountsGet started with OpenAI and Anthropic using an access key and credits
Per-key routingAccess key configurations scope providers, models, and credentials
Multi-model failoverList multiple models in a request; the gateway tries each in order
Multi-key failoverAttach multiple provider keys in a configuration routing rule
Self-hosted modelsRoute to Ollama, vLLM, or other servers with a model you run yourself

​
Next steps

Quickstart

Get ngrok.ai integrated in 5 minutes

Access Keys

Learn about managing access to ngrok.ai

How It Works

Request flow and failover behavior

SDK Integration

Connect your application to ngrok.ai

Bring Your Own Keys

Use your own provider API keys for providers