For software engineers, the "then" mental model for an API gateway is that it's the inflexible monolith that sits between the 😈 big scary public internet and 👼 your production services.
That picture doesn't hold up anymore. The way you build and run services has inevitably changed, and you might:
All these changes make gateways more, not less, important than ever before, but maybe not in the ways you expect. Does the traditional API gateway work for those, too?
By the end of this post, you'll learn about:
An API gateway is the "front door" between the public internet and your network, which opens and shuts, and routes traffic to the appropriate upstream service, depending on who's knocking.
The need for API gateways evolved from a lineage of load balancers and reverse proxies. If you poked back into the networks of 20+ years ago, you had tools like HAProxy and Apache for load balancing and basic reverse proxying, but as apps modernized, they needed more Layer 7/HTTP logic.
Apache did have mod_rewrite for path-based routing to upsteams, and nginx followed up later with slightly nicer location blocks for that plus niceties like header manipulation.
That got you part of the way, but didn't save you from implementing the same bundle of auth and rate limiting code in every service.
That's why what could've been a single piece of infrastructure gets blown up into three or more.
The big breakthrough came in API gateways, which standardized and collapsed what was once many jobs into one. What if your API gateway, as the front door, could also perform the same auth and rate limit checks to every request, no matter its intended destination? What if you could cut out 80% of your boilerplate code and offload it to the front door? What if it could help you implement, say, usage-based pricing without building a massive piece of custom middleware?
To get us on the same page, here's the "shape" of an API gateway:
In this shape, API gateways help you complete a bunch of important jobs, like how you:
They help you move faster by offloading all those dull jobs away from your services so you're not shoe-horning auth into everything or coding up an army of Lua plugins to handle every bit of custom functionality, like signature verification. As you offload, your gateway also standardizes traffic behavior across APIs and apps, which also lets you ship faster and more securely.
Everyone should worry about the security of your network, and a well-defined API gateway makes it hard to completely mess that up with unexpected gotchas or footguns.
API gateways are still important 20 years later. But whether or not you were developing two decades ago, your day-to-day doesn't look anything like it did back then.
Today, a gateway isn't a monolithic deployment pattern like API gateways of the past. Instead, it's more flexible and modular. I works no matter where you're developing new things right now or how your stack evolves in the future.
Across networks, clouds, etc...
I hope these eight shapes will give you inspiration to craft gateways tailor-made for whatever you're building today.
localhost and the LLMs building on your behalf—for example, that might be building an API yourself and scaffolding a frontend with Lovable.The agent-assisted gateway matters because you no longer have to wait for production to surface bugs in how you handle traffic. Local environments stop being toys and look more like reliable testbeds for how the thing you're building will behave when it's ready to go live.
Build on frontier models with ngrok →
The localhost gateway matters because you get to control exactly who exposes local services, under which domains, and route only valid, authenticated requests.
Everyone can share their WIPs without undermining networking security with random, unsecured URLs.
Build a golden path for exposing localhost →
The ephemeral workload gateway matters because it gives you production-like environments for every branch without the pain of long-lived staging. Everyone on your team can safely poke at builds while also avoiding zombie previews that leak your roadmap to the public internet and the shared staging environment that becomes everyone's WIP junk drawer.
The self-hosted gateway matters because you get the feel of a managed platform without surrendering control—perfect if you're ready to migrate off Heroku or Vercel but still want that polish. Doubly so when it comes to really gnarly problems like geo-aware load balancing, DDoS protection, or wiring up a WAF.
ngrok can help you escape the PaaS(t) →
The microservices gateway matters because you don't actually need to go "full service mesh" to coordinate your microservices across clusters. Spin up and ship new services without rewriting all your routing, and also get a central place to debug once requests start boomeranging across clusters.
Get multicloud and multicluster with ngrok for microservices →
localhost, you accept them on a single hostname like webhooks.example.com.
The gateway validates the webhook is coming from a legitimate source and hasn't been tampered with, maybe applies some kind of transformation, and routes the data to the appropriate service.The webhook gateway matters because it standardizes a class of requests that otherwise get re-implemented poorly in every new service. Instead of reinvinting webhook validation for the thirteenth time, you centralize it and feel confident that your integrations are still secure as they pass through this "side door".
Verify webhooks once with ngrok →
The database gateway matters because it makes a task, which you might normally throw an SSH tunnel at and forget about, far more robust. It's one that's recently surprised even us, and we've seen gateways in all kinds of odd and interesting forms over the years. Support customer-facing queries, or even replicate your database across clouds on a private connection, without duct-taping credentials into middleware or learning the hard way that "just IP restrict it" isn't a foolproof strategy.
Get your DB talking internet with ngrok →
The AI gateway matters because it turns a chaotic multi-model stack into something you can lean on. You get safe and controlled access while keeping a lid on costs and keeping an eye on usage across the board. Models and AI agents are already handling a massive portion of today's traffic, and we'll all soon be clamoring to find just the right way to route, secure, and manage it all.
To make this easier to reference (and easier for you to decide which gateway shape fits your stack), here’s a quick summary of each type and the features that define it.
However you slice it, gateways aren't just a box you add at the very sharpest edge of your network any more. The best way to understand is to put one in front of something small today and see how it changes the way you build.
Gateways aren't a box you add at the very last edge of your network anymore, but a capability you should want to start adding wherever your services live and in what stage. The faster you adopt that mindset, especially in an era of LLM-induced chaos, the less friction you’ll hit as your stack evolves.
I recommend trying one in small places:
Each of these gives you a chance to see how gateways work as this flexible capability you can compose anywhere you need it. ngrok works in all these places and more—to get a sense of how you build them up with endpoints and our language for traffic management (Traffic Policy), check out our gallery of gateway examples.
If you skimmed, here's the link to the cheatsheet to bookmark it for later. And, if you have questions about any of this, there's my email or ngrok's Discord.