Let your coding agent set up AI Gateway for your next feature
Shipped by Brenden Ehlers, Alice Wasko, and Prakhar Sinha
Say your help desk app needs a Summarize button on long ticket threads. The code is the easy part. The setup is the detour: open the AI Gateway dashboard, build a virtual model, write an access config so the app’s key can’t wander off to pricier models, create the key, copy its token, then go back to your editor and explain all of it to your coding agent.
The agent can take that detour for you now. The ngrok MCP server has tools that create and update AI Gateway virtual models, access configs, access keys, custom providers, provider keys, firewall policies, and gateway settings. Give your agent the whole feature in one prompt:
Add a Summarize button to ticket threads. Call AI Gateway through a virtual model named
ticket-summarizerthat tries Claude Haiku 5.5 first and falls back to Gemini 3.8 Flash. Give the app its own key that can only use those two models.
Your agent picks its own tools, so the steps vary. Here’s how one run went. Steps 2 through 4, including the trip to the browser, took two minutes:
aigw_list_modelschecks the model IDs and prices.anthropic/claude-haiku-5-5costs $0.10 per million input tokens and $0.50 per million output tokens.aigw_create_virtual_modelsavesticket-summarizerwith Claude Haiku 5.5 first andgoogle/gemini-3.8-flashsecond.aigw_create_access_configcreateshelp-desk-summaries, which allows those two models and nothing else.aigw_create_access_keysends you a link to the AI dashboard with the key’s name, description, and config already filled in.- The agent writes the endpoint with
model: "ticket-summarizer", and you run it.
Step 4 is where you come in. Access key tokens never go through the chat, so the agent can’t paste one into a transcript or a commit. You click Create, copy the token into .env.local, and tell the agent you’re done. It checks that the key exists and carries on. The tool reports “The token was shown only to the user,” then the key, trimmed here:
Why the agent should reach for a virtual model
Your app code names ticket-summarizer, not a vendor. If Claude Haiku 5.5 has a bad afternoon, the gateway moves on to Gemini 3.8 Flash and the button keeps working. When you’d rather try GPT-6.1 Terra first next month, ask the agent to reorder the list. Nobody redeploys the help desk. The virtual models guide covers failover and limits.
Running a model of your own, like a fine-tune on your own GPU? Ask the agent to add it as a custom provider with aigw_create_provider and put it at the top of ticket-summarizer. The agent never sees provider API keys either. aigw_create_provider_key sends you a dashboard link, the same way access keys do.
What the agent can and can’t do
- Each change hits live traffic right away. Tools that can break traffic, such as updating a virtual model, access config, custom provider, firewall policy, or gateway settings, are marked destructive, so most clients ask you first. Keep those prompts on.
- Every tool needs the same write scope the dashboard uses, such as
ai-gateway:virtual-models:write, and your ngrok role still applies. The tools and scopes reference lists each one. - The agent can’t spend your money.
aigw_add_creditshands you the billing page and buys nothing. - Deletes go through the existing
delete_resourcetool and fail while something still uses the resource.
Get started
You need an ngrok account with AI Gateway credits. Claude Haiku 5.5 and Gemini 3.8 Flash bill to your ngrok credits, so you don’t need an Anthropic or Google account. The example uses the OpenAI SDK’s chat completions format, which both models support.
Sign in
Create an ngrok account or sign in, and check your balance under Credits.
Connect the ngrok MCP server
Add
https://mcp.ngrok.com/mcpto your coding agent and sign in. With Claude Code, run:Approve the AI Gateway write scopes on the consent screen. The quickstart has commands for Codex, Cursor, Amp, and other clients.
Ask for the feature
Open your app’s repo in your agent and send the prompt from above. Approve each change it proposes.
Create the key
Open the link the agent sends, check the name and config, and click Create. Copy the token into
.env.local, keep that file out of version control, and tell the agent you’re done:Review the code
The agent writes the endpoint in your app’s style and adds the OpenAI SDK if your app doesn’t have it yet. Check two things in its diff: it reads the key from
AI_GATEWAY_API_KEY, and it sendsticket-summarizeras the model. At its core, the call looks like this:Try it
Ask the agent to run the new endpoint against a sample thread, such as this one:
You get a one-sentence summary. One run returned this, and the wording varies between runs:
CSV exports in the acme-prod workspace have been timing out after 30 seconds since Tuesday’s update, so the next step is to investigate whether that update caused the issue.
Then ask the agent to try the same call with
openai/gpt-6.1-terra. The key does exactly what the access config says, so the gateway refuses it:Ask what it cost
Ask your agent “Which model served my last AI Gateway requests, and what did they cost?” It answers with
aigw_list_usage, one of the AI Gateway read tools. In the run above, Claude Haiku 5.5 from Anthropic answered each request on the first attempt, with 79 input tokens and about 150 output tokens, for about $0.0001 a summary.If Claude Haiku 5.5 fails, the agent can also show each model the gateway tried. That needs request capture turned on. The same details are in Logs if you’d rather look for yourself.