Four ways multicloud breaks you (and how API gateways help)
By Joel Hans
March 11, 2025
By Joel Hans
March 11, 2025
At ngrok, we’re building a model of networking where doing things the right way is also the easy way.
That’s a big divergence from the typical multicloud effort, which is done in ways both complex and extremely hard. Which is why I’d start a typical conversation about multicloud with a strong warning: You must start with a strong forcing function.
You need to know you’ll find value in exchange for all the extraordinarily hard work that goes into actively using multiple cloud providers for different workloads.
You need to justify the cost of becoming an expert in not just one cloud provider’s catalog of tools, security groups, and configuration nuances, but n+1.
IAMs and EKS already broke my brain once—please, no more!
Despite the inevitable headwinds, the multicloud chatter seems to only get louder from ngrok users.
Multicloud makes sense if you need to:
That’s a lot of wonderful-sounding benefits. What were those pain points, again?
The good and bad news is that regardless of why you’re headed toward multicloud, your pains will generally follow a few well-trodden paths.
First, you need to reckon with cross-cloud networking. How does service A in cloud Y communicate with service B in cloud Z? How does service C communicate with load-balanced replicas across both these clouds?
These clouds aren’t designed to communicate with one another, which means you need to set up the entire translation layer. Every provider has different networking models—think virtual private clouds, peering, and private links that keep your data within that singular cloud—and these don’t naturally extend across clouds. Connecting them together requires you to kludge together both Layer 4 (transport) and Layer 7 (application) tools to create a cross-cloud translation layer.
At Layer 4, that means standing up VPNs like AWS Site-to-Site VPN or Azure VPN Gateway on your first cloud, then a software gateway device on the second (or third) to establish network-level connectivity. You’ll also encounter an entire ecosystem of software-defined wide area network (SD-WAN) networking vendors that can help with two of the pain points I’ll cover in a moment: IaC and observability.
For Layer 7, you’ll be drawn toward cloud-agnostic service meshes, which handle service-to-service communication (east-west) across clouds. You’ll still need another solution to handle external ingress from user-generated traffic into your internal networks (north-south), but we’ll get there eventually.
Relevant tools:
No two clouds handle identity and access management (IAM) the same. AWS IAM ≠ Google IAM permissions ≠ Azure AD roles ≠ …
You get the idea.
Normalizing IAM policies across n+1 clouds once might be an approachable task, but nothing in infrastructure is truly set it and forget it.
When you inevitably need to change part of your security implementation, you must keep these two or more ecosystems in sync or risk policy drift, where small changes to roles, permissions, and enforcement mechanisms gradually diverge without a structured way to audit and re-sync them.
The solution? Say hello to the lovely world federated IAM, which handles identity across clouds. In a multicloud setting, these services also help you enforce Zero Trust by ensuring all traffic passes through an identity-aware proxy (more on those in a moment) and enforce AuthN/AuthZ with JWT validation or mTLS.
Once you nail those down, you can turn your eye toward protecting your architecture itself, which involves firewalls and DDoS protection measures that work across multiple clouds.
Relevant tools:
Every cloud also has a unique way of defining and deploying services and networks, which makes your task of standardization nearly impossible.
The cloud-native services you might already use—let’s take AWS Lambda for example—don’t simply port over to Google Cloud Functions. The same idea applies to network and security settings, like security groups in AWS versus Google Firewall rules. Every facet of your infrastructure is subject to the same fragmentation, even down to the networking primitives that make your communication layer possible—coming back to load balancers, firewalls, and delivery networks (aka DDoS protection).
If you get this to work once, congratulations! If you can wrangle the configuration drift after a quarter’s worth of regular maintenance… go take a peek at our careers page.
Many organizations solve this problem with IaC tools that standardize configuration across multiple clouds or help you spin up a control plane, which abstracts and manages all your services for handling data. Configuration drift can still happen—you might need a little per-cloud tweaking based on differences in defaults or primitives—but they generally save you from reinventing the wheel over and over again.
Relevant tools:
If you manage to hurdle these problems well enough to get a multicloud setup working—that’s a big if—you find yourself in a widely distributed and materially siloed environment. The worst of both worlds when it comes to observability.
A field rich in choices, features, hype, venture capital, and your money.
This is a familiar story by now, but each cloud’s monitoring stack doesn't work together natively. Unless you want to constantly context-switch between different monitoring tools to debug issues, like unexpectedly high latency between an AWS function and a GCP K8s cluster, you’re going to need to centralize this disparate data somewhere.
You can start dumping your disparate multicloud data into a ready-made provider or you can build yourself a solution starting with an “awesome” list of open source tools like OpenTelemetry. Both are perfectly valid, and both will cost a pretty penny.
Just hire yourself a team of data engineers (or a single Christian). It’s easier said than done, and no matter what, it’s expensive. And troublesome to maintain. And extraordinarily complex to parse, understand, and take action on.
Relevant tools:
At this point, I wouldn’t blame you if you felt as though no forcing function could possibly outweigh the cost of going multicloud.
And you can also see that while there are plenty of tools that help you along the way, none of what I’ve mentioned so far helps solve more that a single problem. They have their lane in the multicloud and they stick to it.
I’m not going to claim an API gateway will that entirely (it certainly can’t help syncing your database across clouds!) but they are uniquely capable of mitigating all four of the pains we’ve been tracking so far. API gateways help you:
Sold you on the why for API gateways—what about the why not for so many of them?
These are the gateways that come packaged with your big-time cloud providers—think AWS API Gateway, GCP API Gateway (and Cloud Endpoints), and Azure API Management.
These are powerful tools if you’re in a single cloud, but you can’t just run AWS API Gateway in GCP—each tool is locked to its respective providers. To get cross-cloud communication, you need to deploy each provider’s offering and attempt, using two disparate security and configuration models, and synchronize their behavior.
Not simple, to say the least.
Kong Gateway, NGINX, and Envoy are all good examples. They’re powerful and environment-agnostic, but come with significant setup and maintenance burdens.
You can put replicas of a self-deployable API gateway in each cloud, but then you still need to integrate the security layer for each, then synchronize their configuration using a separate control plane or IaC tool like Terraform. You avoid vendor lock-in, but lose the plug-and-play option that makes the multicloud motion pay off in light of your forcing function.
When you build your multicloud architecture around an API gateway with these caveats, you’re not just missing out on features—you’re accepting a future for your infrastructure that, at scale, explodes with complexity.
What if you could build your multicloud architecture around an API gateway that comes bundled with all the cross-cloud networking needs, like load balancing, DDoS protection, and automatic failover if one of your clouds goes down?
That manages all your certificates for you and lets you centrally implement the fundamentals of Zero Trust security on your services, from integrating with your IdP or enforcing JWT validation or mTLS?
That also lets you compose traffic management policies onto specific services for unified governance at the root level plus less work supporting developers who want to ship their APIs ASAP?
That integrates observability for all your API services, from real-time logs and metrics, into a single dashboard that you can automatically stream into the rest of your observability stack?
And that works exactly the same on every cloud, cluster, region, and environment?
Now you see how we’re trying to make the right way the easy way—so easy that you can try out our multicloud API gateway in about 20 minutes with our end-to-end tutorial.
When you’re done, you’ll have wired up active/active multicloud routing and composable traffic management policies that don’t get more complex as you scale out. The next time something goes wrong in one cloud, you'll be set up to route and shift traffic between environments—even during incidents—with minimal firefighting.
If you have questions along the way: