Skip to main content
This guide covers managing provider API keys when using your own keys (BYOK). If you’re using AI Gateway API Keys, key management is handled automatically—see AI Gateway API Keys.
When you configure provider keys in the gateway, anyone with your endpoint URL can make requests using your provider keys. See Securing Your Gateway to add authorization.

​
Key storage options

​
Inline keys (development only)

Store keys directly in your Traffic Policy: 
providers:
  - id: "openai"
    api_keys:
      - value: "sk-proj-abc123…"
Inline keys are visible in Traffic Policy configurations. Use only for local development and testing. See ngrok Secrets for the recommended approach.
Store keys in ngrok Vaults & Secrets—they’re encrypted at rest, never visible in your Traffic Policy, and can be rotated without redeploying: 
providers:
  - id: "openai"
    api_keys:
      - value: ${secrets.get('openai', 'key-one')}
      - value: ${secrets.get('openai', 'key-two')}

​
Creating secrets

​
Using the CLI

ngrok api secrets create \
  --name openai \
  --secret-data '{"key-one": "sk-proj-abc123…", "key-two": "sk-proj-def456…"}'

​
Using the dashboard

  1. Navigate to Vaults & Secrets in the ngrok Dashboard
  2. Create a vault if you don’t have one (for example, ai-keys)
  3. Create a secret in the vault (for example, openai)
  4. Add your provider API keys as key-value pairs

​
Multiple keys for failover

Configure multiple provider API keys for automatic failover when keys hit rate limits or fail: 
providers:
  - id: "openai"
    api_keys:
      - value: ${secrets.get('openai', 'primary')}
      - value: ${secrets.get('openai', 'backup')}
      - value: ${secrets.get('openai', 'emergency')}
Failover behavior:
  1. Gateway tries the first key
  2. If it fails (rate limit, quota exceeded, error), tries the next key
  3. Continues until a key succeeds or all keys are exhausted
Key ordering matters: Keys are tried in the order listed. Place your highest-capacity keys first.

​
Key rotation

To rotate provider API keys without downtime:
1

Add the new key

api_keys:
  - value: ${secrets.get('openai', 'old-key')}
  - value: ${secrets.get('openai', 'new-key')}  # Added
2

Deploy and monitor

Deploy the updated Traffic Policy. Monitor traffic to ensure the new key works correctly.
3

Remove the old key

Once confirmed, remove the old key:
api_keys:
  - value: ${secrets.get('openai', 'new-key')}
4

Revoke the old key

Revoke the old key with your provider (OpenAI, Anthropic, etc.) to complete the rotation.

​
Multi-provider example

on_http_request:
  - type: ai-gateway
    config:
      providers:
        - id: "openai"
          api_keys:
            - value: ${secrets.get('openai', 'team-a-key')}
            - value: ${secrets.get('openai', 'team-b-key')}
            - value: ${secrets.get('openai', 'shared-backup')}
        
        - id: "anthropic"
          api_keys:
            - value: ${secrets.get('anthropic', 'primary')}
            - value: ${secrets.get('anthropic', 'secondary')}
        
        - id: "google"
          api_keys:
            - value: ${secrets.get('google', 'key')}

​
Security best practices

  • Add authorization to your gateway when using server-side keys—see Securing Your Gateway
  • Never commit provider API keys to version control
  • Use ngrok secrets for all production keys
  • Rotate keys regularly to minimize exposure risk
  • Monitor key usage to detect anomalies
  • Use different keys for different environments (dev, staging, prod)
  • Set up alerts for rate limit errors to proactively add capacity

​
Passthrough mode

If you don’t configure any keys for a provider, the gateway forwards whatever key your SDK sends: 
providers:
  - id: "openai"
    # No api_keys - SDK key is forwarded
This is useful for:
  • Development environments where each developer uses their own key
  • Applications that already manage their own keys
  • Quick testing without configuration changes

​
Next steps