AI Gateway: attach provider keys to AI Gateway keys

You can now attach your own OpenAI, Anthropic, or other provider keys directly to an AI Gateway key.

Your apps and agents authenticate to AI Gateway with that key. AI Gateway then uses the attached provider credential when sending the request upstream.

This gives Bring Your Own Key (BYOK) traffic the same authorization model as AI Gateway-managed inference: provider credentials stay out of application code, developer environments, and Traffic Policy.

Why we changed BYOK

AI Gateway has supported BYOK from the beginning, but the first setup required provider credentials to be pasted directly in your Traffic Policy.

That made BYOK work, but it left you with two separate problems to solve:

1. How to securely store provider keys?
2. How to protect the endpoint that could use it?

Vaults & Secrets improved storage by letting you encrypt provider keys, reference them by name, and rotate them without editing the raw value in policy.

Authorization was still separate. Because AI Gateway endpoints are reachable over the internet, you still had to add a JWT check, mTLS, an IP allowlist, or another Traffic Policy rule to control who could call the endpoint.

So BYOK still depended on Traffic Policy twice: once to reference the provider key, and again to protect access to the endpoint.

This change solves both of those problems.

How it works

Attach a provider key to an existing AI Gateway key:

1ngrok api ai-gateway-provider-keys create \2  --ai-gateway-api-key-id aigk_xxxxx \3  --provider-id openai \4  --description "OpenAI Primary" \5  --value "sk-..."

The full provider key is shown once at creation. After that, the API only returns a redacted value like sk-abc***xyz. To rotate it, delete the old key and create a new one.

Now your app can call AI Gateway with the AI Gateway key instead of the raw provider key:

1import OpenAI from "openai";2 3const client = new OpenAI({4	baseURL: "https://your-gateway.ngrok.app/v1",5	apiKey: "ng-xxxxx-g1-xxxxx", // Your AI Gateway key6});7 8const response = await client.chat.completions.create({9	model: "gpt-4o",10	messages: [{ role: "user", content: "Hello!" }],11});

Same SDK, same AI Gateway endpoint. Your app sends the AI Gateway key, and AI Gateway manages which provider key gets used upstream.

How your keys are stored

We encrypt your provider keys with AES-256. Plaintext is only held in memory for the upstream request and is never written to disk or logs.

You can attach up to fifteen keys per provider to a single AI Gateway key for failover and staged rollouts.

How does this change impact you?

If you're on Traffic Policy BYOK today, plan the move. Your existing providers[].api_keys and api_key_selection config will still work, but you can't add new keys or rotate existing ones through Traffic Policy anymore. Migrate to the new system above. The migration guide walks through it step by step.

Self-hosted and custom providers still configure keys in Traffic Policy. If you use base_url to point at a model server you run yourself (Ollama, vLLM) or a provider we don't manage first-class, the provider key stays where it is. Attached provider keys only apply to the standard managed providers—OpenAI, Anthropic, and the rest.

If you're using AI Gateway–managed provider keys, nothing changes. You can keep using AI Gateway the same way. If you'd rather send requests through your own OpenAI, Anthropic, or other provider account, you can now attach your own provider key to an AI Gateway key.

No changes to billing. BYOK is still free.

Get started

1. Create an AI Gateway if you don't have one
2. Create an AI Gateway key
3. Attach a key from your provider (OpenAI, Anthropic)
4. Point your SDK at the gateway and go

See the managing provider keys guide for the full walkthrough.

Questions or running into something? Find us on Discord.