Skip to main content

Documentation Index

Fetch the complete documentation index at: https://ngrok.com/docs/llms.txt

Use this file to discover all available pages before exploring further.

CEL-based api_key_selection strategies are deprecated. Attached provider keys use simple ordered failover: the most recently attached key is tried first. For advanced routing logic, use model selection strategies instead.
When multiple keys are configured for a provider, the gateway automatically fails over between them:
  1. Keys are tried in reverse order of attachment—the most recently attached key first.
  2. If a key fails, the next key is tried automatically.
  3. Failover triggers include:
    • Rate limit errors (HTTP 429)
    • Quota exceeded responses from the provider
    • Timeout or server errors (HTTP 5xx)

​
With attached provider keys

Attach multiple keys for the same provider to your AI Gateway API Key. The most recently attached key is tried first, then the gateway falls back to older keys in reverse order of attachment: 
# Attach backup key first (tried last)
ngrok api ai-gateway-provider-keys create \
  --ai-gateway-api-key-id aigk_xxxxx \
  --provider-id openai \
  --description "OpenAI Backup" \
  --value sk-proj-backup...

# Attach primary key last (tried first)
ngrok api ai-gateway-provider-keys create \
  --ai-gateway-api-key-id aigk_xxxxx \
  --provider-id openai \
  --description "OpenAI Primary" \
  --value sk-proj-primary...
Attach your highest-capacity or most reliable key last—the most recently attached key is tried first.

​
With Traffic Policy keys (custom providers only)

For custom or self-hosted providers, configure multiple keys in Traffic Policy: 
on_http_request:
  - type: ai-gateway
    config:
      providers:
        - id: openai
          api_keys:
            - value: ${secrets.get('openai', 'key-one')}    # tried first
            - value: ${secrets.get('openai', 'key-two')}    # failover
            - value: ${secrets.get('openai', 'key-three')}  # last resort
providers[].api_keys is deprecated for standard providers. Use attached provider keys instead.

​
Next steps