Skip to main content
Agents run tools with real access to your machines, repos, and secrets, and one bad tool call can do lasting damage. An agent can delete an operating system by accident with a single shell command, push to a public repo whose name is close to your private one, or send your company’s secrets and API keys somewhere they don’t belong. Firewall stops those calls in the AI Gateway, before your agent runs them:
  • No code changes: you attach a policy to access keys, so every agent that uses those keys gets the same checks.
  • Blocked before it runs: Firewall checks each tool call the model returns before your agent receives it, even in a streamed response.
  • Agents recover: a block returns a normal response that tells the model what was blocked and why, so the agent can change course instead of failing.
You describe what to check in a Firewall policy: a named set of rules, a mode, and what to do when a check can’t run. To set one up, see Guard agent tool calls with Firewall.
Firewall is an early release. Some behavior on this page may change as the feature develops.

​
Where checks run

Each rule runs at one of two points:
  • Request rules run before the gateway sends the request to the model. They read the conversation, the results of tools the agent ran, and the tools the agent offers the model. A blocked request never reaches the model.
  • Response rules run on each tool call the model returns. A blocked tool call is removed from the response, so your agent never receives it and can’t run it.
Firewall checks OpenAI Chat Completions (/v1/chat/completions), OpenAI Responses (/v1/responses), Anthropic Messages (/v1/messages), and Gemini generate content (generateContent and streamGenerateContent) requests, both streamed and not streamed. That covers agents that call those endpoints through the gateway, such as Claude Code and Codex. For other endpoints, see Things to know.

​
Rules

Each rule has a type, which says how it matches, and a tier, which says what happens when it matches.

​
Rule types

Tool names and regex rules are fast and predictable. Use a decision rule when the risk depends on intent or context that a pattern can’t capture. Decision rules use Jev, a decision model. Jev is a separate model that ngrok calls to judge the rule, not the model your request asks for.

​
What a rule reads

A rule reads either request parts or tool calls, not both. Tool names rules read tool definitions or tool calls. Decision rules on request parts read the system and developer messages plus the newest input: the latest user message and any tool results it carries. Regex rules on request parts read the whole conversation.

​
Tiers

Use soft block and allow together for exceptions. For example, a soft block regex rule that matches git push and an allow regex rule that matches git push origin feature/ block every push except pushes to feature branches.

​
Mode and errors

A policy’s mode decides whether matches block anything:
  • Enforce blocks requests and tool calls that match a blocking rule.
  • Dry run checks everything and reports what matched, but lets it all through. Use it to test a policy on real traffic before you enforce it.
A check can fail to run, for example when the decision model times out. On error decides what happens then:
  • Block treats what it couldn’t check as blocked. This only applies in Enforce mode.
  • Continue lets it through and reports that it wasn’t checked.
If Firewall can’t read the tool calls in a response at all, and the policy uses Enforce and Block, the request fails with a 500 error instead.

​
Where a policy applies

An account has at most one baseline, and a configuration picks at most one policy.

​
How policies combine

A request can be checked by up to three policies: the baseline, the policy of its key’s configuration, and a policy in the request. Firewall runs every rule from all of them, so a policy can add checks but never remove one:
  • Each policy keeps its own Mode and On error setting. A Dry run policy only reports, even when your baseline enforces, so you can try a new policy on keys that already have one.
  • An allow rule only overrides soft block rules in its own policy. A request can’t allow what your baseline blocks.
When you make a policy your baseline, configurations that picked the same policy stop picking it, so its rules don’t run twice.

​
What your agent sees

A block is not an HTTP error. The gateway returns a normal 200 response, so your agent keeps working and the model can react to the block:
  • Blocked request: the response holds one assistant message that says which rule blocked the request.
  • Blocked tool call: the gateway replaces the call with assistant text that names the tool and the rule, and tells the model not to retry it with the same arguments.
The response also carries a safeguard_results field, and the Ngrok-AIG-Firewall-Request header reports request rule results. See What a blocked caller gets for the exact formats.

​
Things to know

  • Other endpoints: Firewall can’t check requests to other endpoints, such as the Gemini Interactions API. On error decides what happens to them. If a policy that applies to the key uses Enforce and Block, the gateway rejects them with ERR_NGROK_27228. Otherwise, they go through unchecked with the header Ngrok-AIG-Firewall: unscreened; reason=unsupported_endpoint.
  • Legacy functions: Firewall checks tool calls, not the function calls a model returns for the legacy Chat Completions functions field. When a policy has response rules, On error decides those requests the same way, and the header reason is legacy_functions. Send tools instead of functions.
  • Streamed Gemini function call arguments: with streamFunctionCallArguments, Gemini sends each function call in fragments, which Firewall doesn’t join. When a policy has response rules, On error decides those requests the same way, and the header reason is stream_function_call_arguments. When they go through, request rules still run, but tool calls aren’t checked.
  • OpenAI-defined tools: in a Responses request, Firewall can’t read the arguments of tools that OpenAI defines and your app runs, such as local shell, computer use, and apply_patch. Tool names rules still match these calls by name, such as local_shell or apply_patch. For other rules, On error decides.
  • Provider-run tools: tools that the provider runs before the response returns, such as OpenAI and Anthropic web search or Gemini code execution, never reach your agent, so Firewall doesn’t check them.
  • previous_response_id: when Firewall blocks a tool call in a Responses request, OpenAI still keeps the original call. If your next request uses previous_response_id, OpenAI expects a tool output for a call your app never received. Send the conversation in input instead.
  • Latency: tool names and regex rules add almost no latency. Each decision rule calls the decision model, which can add up to 1.5 seconds before the check counts as failed.
  • Streaming: in Enforce mode, the gateway holds each streamed tool call until its checks finish, so your agent never receives a call that wasn’t checked. The rest of the stream keeps flowing.
  • Usage logs: for now, Firewall results don’t appear in usage logs. Read the response headers and safeguard_results to see what Firewall did.

​
Next steps