> ## Documentation Index > Fetch the complete documentation index at: https://ngrok.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # How It Works > Understand the ngrok.ai request flow and failover behavior. ## Request flow When you send a request to `https://gateway.ngrok.ai`: ```mermaid theme={null} sequenceDiagram participant Client participant Gateway as gateway.ngrok.ai participant Provider as AI Provider Client->>Gateway: POST /v1/chat/completions (access key) Gateway->>Gateway: Validate access key Gateway->>Gateway: Apply configuration and select model Gateway->>Provider: Forward request (injected credentials) Provider-->>Gateway: Response (or error) Gateway-->>Client: Return response ``` 1. **Your app sends a request** with your [access key](/docs/ai-gateway/concepts/access-keys) to `https://gateway.ngrok.ai` 2. **The gateway validates the key** and loads its [access key configuration](/docs/ai-gateway/guides/access-key-configurations) 3. **The gateway resolves the model** from your request body 4. **Upstream credentials are selected** from the configuration's routing rules—ngrok.ai inference or your stored [provider keys](/docs/ai-gateway/guides/attaching-provider-keys) 5. **Unsupported parameters are stripped** for the chosen model before forwarding 6. **The request is forwarded** to the upstream provider 7. **On failure, the gateway retries** with the next provider key or model candidate 8. **The response is returned** to your app and usage is recorded Manage keys, configurations, providers, and credits at [app.ngrok.ai](https://app.ngrok.ai). Your apps send traffic to `gateway.ngrok.ai`. ## Authentication Every request must include a valid **access key**. Send it as `Authorization: Bearer ng-xxxxx-g1-xxxxx` (OpenAI-compatible APIs) or `x-api-key` (Anthropic native API). Provider keys stay in [app.ngrok.ai](https://app.ngrok.ai). The AI Gateway injects those from your configuration after validating the access key. ## Response headers Every response from `gateway.ngrok.ai` carries ngrok's own headers describing how the gateway handled the request. | Header | Value | When present | | - | - | - | | `Ngrok-AIG-Request-Id` | The request's ID. Use it to look the request up afterwards. | Every response, including rejections and errors. | | `Ngrok-AIG-Attempts` | How many upstream attempts the gateway made. | Once the attempt loop runs, on both success and failure. Absent when the request failed before any attempt, such as a malformed body or an unknown model. | | `Ngrok-AIG-Model` | The upstream model that produced the response. | Success only. | All three are exposed through CORS, so browser clients can read them. These are ngrok's headers rather than the provider's, so a managed-key caller and a [Bring Your Own Keys](/docs/ai-gateway/concepts/bring-your-own-keys) caller see the same values. Provider rate-limit headers behave differently: the gateway forwards those only when the attempt used your own provider key. ### The model header is authoritative The gateway passes the response body through byte-for-byte. The `model` or `modelVersion` field inside the body is the provider's own word and is never rewritten to name the model that served the response. On a multi-model failover the body can therefore name a different model than the header. Read `Ngrok-AIG-Model` when you need to know what answered. ### Streamed responses On a streamed response the headers commit with the first flushed byte. The attempt count is final at that point, because the gateway cannot retry once bytes have reached your client. ### Trace IDs are not a substitute `Ngrok-Trace-Id` is emitted only when the trace is sampled, it is not exposed through CORS, and it identifies the whole client request rather than a single attempt. Use `Ngrok-AIG-Request-Id` to identify a request. ## Model selection The gateway determines which model and provider to use from your request. | Model in request | What happens | | - | - | | `gpt-4o` | Catalog lookup → OpenAI | | `openai:gpt-4o` | Explicit OpenAI routing | | `your-provider:your-model` | A [model you run yourself](/docs/ai-gateway/guides/use-a-model-you-run-yourself) | | `models` array | Fail over through listed models in order | To choose model IDs and provider-qualified model names, see [Choose a model](/docs/ai-gateway/guides/model-selection-strategies). ## Access key configurations Configurations define: * **Access scope**: allowed providers and models * **Routing rules**: ngrok.ai inference vs provider keys per provider An access key without a configuration uses allow-all scope. Built-in OpenAI and Anthropic default to ngrok.ai inference when you have [credits](/docs/ai-gateway/concepts/credits). ## Failover The gateway retries automatically when upstream requests fail. ### What triggers failover? * Rate limits (HTTP 429) * Auth errors from exhausted or invalid provider keys * Timeouts and server errors (HTTP 5xx) ### Failover order 1. Next provider key in the routing rule (ordered list) 2. Next step in the routing rule (if configured) 3. Next model in the request's `models` array To attach several provider keys and control the order they're tried, see [Multi-key failover](/docs/ai-gateway/guides/key-selection-failover). ## Timeouts and token limits Configure account-level limits in [Account Settings](/docs/ai-gateway/guides/account-settings). | Setting | Default | Description | | - | - | - | | Per-request timeout | 60s | Max time for a single upstream attempt | | Total timeout | 120s | Max time including all failover attempts | | Max input tokens | none | Reject requests exceeding estimated input tokens | | Max output tokens | none | Cap completion length | ## Next steps Create keys to authenticate requests Scope and route traffic per key Model IDs, provider prefixes, and custom provider models How the gateway handles failures This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.