Manage traffic on a region-aware API gateway—without the networking
By Joel Hans
December 4, 2024
By Joel Hans
December 4, 2024
When you use ngrok as your API gateway, operating as the front door to one or many APIs you need to ship and secure, there inevitably comes a time where you need to conditionally act on traffic based on its qualities.
Notably, where a request comes from.
We recently shipped a new Traffic Policy variable named conn.server_region, which allows you to make decisions in your Traffic Policy rules based on which region of the ngrok network received the traffic.
For some use cases, that’s an improvement over our other implementation for manipulating traffic based on geography, which consisted of matching the conn.geo.country_code variable to lists of country codes: conn.geo.country_code in ['<COUNTRY-01>', '<COUNTRY-02>'].
We’ll take a moment to walk through how we actioned this request from multiple customers, but you can also skip ahead to the examples if you’d like to get started right away.
We often start diagnosing issues or papercuts in our platform by asking ourselves, “Why are folks struggling with this? What’s confusing them? What gap—whether in the features we ship or knowledge we share—isn’t getting delivered intact or fast enough?”
From there, we’d start making it easier… or at least harder to get things wrong.
But in this case, we had multiple customers directly ask us for this variable that allowed them to conditionally manage traffic based on the region, not specific country codes. They didn’t like the experience of, for example, blocking traffic using those lists of country codes, as the rules quickly got longwinded, inefficient to maintain, and hard to debug.
So, how exactly did we make this easier?
Every time a request hits a public ngrok endpoint, our global server load balancer (GSLB) automatically routes it through one of our 8 globally distributed Points of Presence (PoPs) based on the lowest latency.
If you’re sending a request from Buenos Aires, our GSLB performs DNS lookups at all PoPs and picks the fastest. Based on physical proximity alone, that’s almost guaranteed to be our South America PoP in São Paulo, which we code internally as sa.
Unlike most of the variables in Traffic Policy, which the engine only knows at runtime, server region data is already known when an ngrok service starts up.
We added a function to return the regionID from the service’s startup configuration, then plumbed that into the existing map of connection variables (conn.*).
The conn.server_region variable then carries the regionID returned by serverRegion() into the scope of Traffic Policy, allowing you to write expressions with the sa string or to interpolate it into any action you take on traffic as it passes through the ngrok network toward your upstream service.
When we got those initial requests from our customers, we felt confident in one use case: geo-blocking.
That said, our customers have surprised us with how they’re using it, and we’re also uncovering new use cases all the time with our flexible and expressive Traffic Policy engine… even more when paired with internal endpoints.
ngrok’s global server load balancer (GSLB) helps get requests into our network ASAP and to the policies you’ve built into your API gateway, but you can minimize latency even more by running multiple instances in different regions—how about in Frankfurt and Singapore?
With internal endpoints and the forward internal action, this is no longer a complex networking and operational challenge, but a handful of lines in YAML.
on_http_request:
- expressions:
- "conn.server_region == 'ap'"
actions:
- type: forward-internal
config:
url: https://ap.internal
- actions:
- type: forward-internal
config:
url: https://eu.internalngrok routes traffic originating from the ap region to your Singapore instance, with all other traffic headed to the EU.
With the above example, you have a nice split in your traffic, but one big caveat: If your Singapore service goes offline, what happens to the requests with ap as their conn.server_region?
By default, the forward internal action uses the on_error: halt behavior when encountering an error while forwarding traffic, like an outage.
You can switch that behavior to on_error: continue, which executes the following rule or action.
on_http_request:
- expressions:
- "conn.server_region == 'ap'"
actions:
- type: forward-internal
config:
url: https://ap.internal
on_error: continue
- actions:
- type: forward-internal
config:
url: https://eu.internalThe next time things go dark in the ap region, you’ll at least know new requests are safely re-routed to your EU instance, and aside from a bit of extra latency, your users won’t know the difference.
With a third regional deployment in Ohio, USA, you’re nearing the resiliency end-game.
Sure.
Right after you provision your new services, get the new internal endpoint connected to the ngrok network, and reconfigure your API gateway, you make a terrible realization: You can’t just chuck EU traffic over to the US with reckless abandon!
Traffic Policy can save you from a massive compliance mess, too.
With an expression using a NOT logical operator—!(conn.server_region == 'eu')—you can prevent failover traffic from the EU reaching your US-based service, and instead provide them with a relevant error message that interpolates said region into the response.
on_http_request:
- expressions:
- "conn.server_region == 'ap'"
actions:
- type: forward-internal
config:
url: https://ap.internal
on_error: continue
- actions:
- type: forward-internal
config:
url: https://eu.internal
on_error: continue
- expressions:
- "!(conn.server_region == 'eu')"
actions:
- type: forward-internal
config:
url: https://us.internal
on_error: continue
- actions:
- type: custom-response
config:
status_code: 503
body: '{"error":"Service unavailable for traffic in the ${conn.server_region} region."}'
headers:
content-type: "application/json"Your API service is growing fast… and in parts of the world you weren’t quite prepared for. As you start down the road of deciding which region you want to invest in next, why not start tallying which regions, of all you haven't yet covered, are most popular?
The below rule only triggers the log action if two conditions are met:
200s)By interpolating the conn.server_region variable into the log action, you can create a new event for each matching response, then aggregate your data later to see which region tallied up fastest.
on_http_response:
- expressions:
- "res.status_code > '200' && res.status_code <= '300'"
- "!(conn.server_region in ['ap', 'eu', 'us'])"
actions:
- type: "log"
config:
metadata:
success: true
message: "Successful response sent."
server_region: "${conn.server_region}"We’re really proud to ship this feature, which is otherwise not trivial to implement at your load balancer or within your API service, in a form factor our API gateway users can build in minutes and a few lines of YAML. That’s all derived from our commitment to turning the hard parts of networking into easy ones.
Better yet, we’re proud of shipping a feature clearly driven from customers feedback and experience. It’s even more evidence we’re ready to implement new API gateway functionality based on what our users actually need—which is often different from what we think they need at the get-go.
When you’re ready to join those already using region-based logic in their API gateways, sign up for an ngrok account and follow our get started guide for delivering your first APIs to production using ngrok.
Curious about the flexible and expressive nature of our Traffic Policy engine? Dive into our docs for all the details about connecting expressions to actions.
Questions about how to use ngrok as an API gateway? Find us in our community repo (ngrok/ngrok) or sign up for the next Office Hours, our monthly Q&A livestream.