How It Works

​

Request flow

When you send a request to https://gateway.ngrok.ai:

1. Your app sends a request with your access key to https://gateway.ngrok.ai
2. The gateway validates the key and loads its access key configuration
3. The gateway resolves the model from your request body
4. Upstream credentials are selected from the configuration’s routing rules—ngrok.ai inference or your stored provider keys
5. Unsupported parameters are stripped for the chosen model before forwarding
6. The request is forwarded to the upstream provider
7. On failure, the gateway retries with the next provider key or model candidate
8. The response is returned to your app and usage is recorded

Manage keys, configurations, providers, and credits at app.ngrok.ai. Your apps send traffic to gateway.ngrok.ai.

​

Authentication

Every request must include a valid access key. Send it as Authorization: Bearer ng-xxxxx-g1-xxxxx (OpenAI-compatible APIs) or x-api-key (Anthropic native API). Provider keys stay in app.ngrok.ai. The AI Gateway injects those from your configuration after validating the access key.

​

Response headers

Every response from gateway.ngrok.ai carries ngrok’s own headers describing how the gateway handled the request. All three are exposed through CORS, so browser clients can read them. These are ngrok’s headers rather than the provider’s, so a managed-key caller and a Bring Your Own Keys caller see the same values. Provider rate-limit headers behave differently: the gateway forwards those only when the attempt used your own provider key.

​

The model header is authoritative

The gateway passes the response body through byte-for-byte. The model or modelVersion field inside the body is the provider’s own word and is never rewritten to name the model that served the response. On a multi-model failover the body can therefore name a different model than the header. Read Ngrok-AIG-Model when you need to know what answered.

​

Streamed responses

On a streamed response the headers commit with the first flushed byte. The attempt count is final at that point, because the gateway cannot retry once bytes have reached your client.

​

Trace IDs are not a substitute

Ngrok-Trace-Id is emitted only when the trace is sampled, it is not exposed through CORS, and it identifies the whole client request rather than a single attempt. Use Ngrok-AIG-Request-Id to identify a request.

​

Model selection

The gateway determines which model and provider to use from your request. To choose model IDs and provider-qualified model names, see Choose a model.

​

Access key configurations

Configurations define:

• Access scope: allowed providers and models
• Routing rules: ngrok.ai inference vs provider keys per provider

An access key without a configuration uses allow-all scope. Built-in OpenAI and Anthropic default to ngrok.ai inference when you have credits.

​

Failover

The gateway retries automatically when upstream requests fail.

​

What triggers failover?

• Rate limits (HTTP 429)
• Auth errors from exhausted or invalid provider keys
• Timeouts and server errors (HTTP 5xx)

​

Failover order

1. Next provider key in the routing rule (ordered list)
2. Next step in the routing rule (if configured)
3. Next model in the request’s models array

To attach several provider keys and control the order they’re tried, see Multi-key failover.

​

Timeouts and token limits

Configure account-level limits in Account Settings.

​

Next steps