Ingress to Kubernetes apps and services with vcluster

This guide shows you how to add public ingress to an app running on a vcluster virtual cluster. You’ll use the ngrok Kubernetes Operator to route public traffic to the app, then secure it with OAuth. vcluster is an open source project for creating virtual clusters that run inside the namespaces of a host cluster. The pods you deploy on a vcluster are scheduled on the underlying cluster, while other resources, such as deployments and CRDs, exist only inside the virtual cluster. This isolation makes vcluster useful for development environments, internal developer platforms, and experiments that still need to route external traffic.

What you’ll need

The vcluster CLI installed locally.
A Kubernetes cluster to host the virtual cluster. If you don’t have one, minikube creates a local cluster for testing.
An ngrok account.
kubectl and Helm 3.0.0+ installed on your local workstation.
The ngrok Kubernetes Operator installed on your virtual cluster.
A reserved domain from the ngrok dashboard or API; this guide refers to it as NGROK_DOMAIN.

Set up a local virtual cluster with vcluster

To follow this guide, you need a host Kubernetes cluster with a virtual cluster running on it. If you already have one, skip to Install a sample application. If you don’t have a host cluster, create one locally with minikube:

minikube start --profile dc1 --memory 4096

Create a virtual cluster named my-vcluster. This creates a namespace on the host cluster and switches your active kube context to the new virtual cluster:

vcluster create my-vcluster --expose-local

Confirm the virtual cluster is running:

kubectl get namespaces

If your context is no longer pointed at the virtual cluster, run vcluster connect my-vcluster to reconnect. See vcluster’s deployment docs for other ways to create and connect to a virtual cluster. Install the ngrok Kubernetes Operator on the virtual cluster before continuing.

Install a sample application

With the ngrok Kubernetes Operator running on your virtual cluster, deploy a sample application to see how the Operator routes external traffic to it. Create a Kubernetes manifest named tinyllama.yaml with the contents below. It defines the tinyllama demo app from ngrok-samples/tinyllama (a Service and Deployment), then adds an Ingress that tells the ngrok Kubernetes Operator to route NGROK_DOMAIN to the tinyllama service. Replace NGROK_DOMAIN with the domain you reserved:

apiVersion: v1
kind: Service
metadata:
  name: tinyllama
  namespace: default
spec:
  ports:
    - name: http
      port: 80
      targetPort: 8080
  selector:
    app: tinyllama
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: tinyllama
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: tinyllama
  template:
    metadata:
      labels:
        app: tinyllama
    spec:
      containers:
        - name: tinyllama
          image: ghcr.io/ngrok-samples/tinyllama:main
          ports:
            - name: http
              containerPort: 8080
---
# ngrok Kubernetes Operator configuration
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: tinyllama-ingress
  namespace: default
spec:
  ingressClassName: ngrok
  rules:
    - host: NGROK_DOMAIN
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: tinyllama
                port:
                  number: 80

Apply the manifest to your virtual cluster:

kubectl apply -f tinyllama.yaml

Troubleshooting: If you get an error when applying the manifest, confirm that you’ve replaced NGROK_DOMAIN in tinyllama.yaml with your reserved domain and try again.

Open your reserved domain (for example, https://NGROK_DOMAIN) in a browser to see the demo app. ngrok routes requests to the ngrok Kubernetes Operator, which forwards them to the tinyllama service as long as your virtual cluster is running.

Add OAuth protection to your demo app

Now that your demo app is publicly accessible through ngrok, you can add capabilities like authentication without deploying extra infrastructure. This section restricts access to Google accounts under a specific domain. With the Traffic Policy system and the oauth action, ngrok handles OAuth entirely on its network. You don’t need to add services to your cluster or change any routes, because ngrok authenticates and authorizes requests before they reach your endpoint. To enable the oauth action, create an NgrokTrafficPolicy custom resource and apply it to your Ingress with an annotation. You can also apply the policy to a specific backend or as the default backend for an Ingress. See the doc on using the Operator with Ingresses for details. Edit your tinyllama.yaml manifest with the following, leaving the Service and Deployment as they were. Note the new annotations field and the NgrokTrafficPolicy CR.

...
---
# Configuration for ngrok's Kubernetes Operator
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: tinyllama-ingress
  namespace: default
  annotations:
    k8s.ngrok.com/traffic-policy: oauth
spec:
  ingressClassName: ngrok
  rules:
    - host: NGROK_DOMAIN
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: tinyllama
                port:
                  number: 80
---
# Traffic Policy configuration for OAuth
apiVersion: ngrok.k8s.ngrok.com/v1alpha1
kind: NgrokTrafficPolicy
metadata:
  name: oauth
  namespace: default
spec:
  policy:
    on_http_request:
      - type: oauth
        config:
          provider: google

Re-apply your manifest:

kubectl apply -f tinyllama.yaml

When you open your demo app again, ngrok asks you to log in with Google. To limit access to just yourself or your colleagues, tighten the policy. Use expressions and CEL interpolation to reject OAuth logins that aren’t under example.com. Update the NgrokTrafficPolicy portion of your manifest after changing example.com to your domain:

# Traffic Policy configuration for OAuth
apiVersion: ngrok.k8s.ngrok.com/v1alpha1
kind: NgrokTrafficPolicy
metadata:
  name: oauth
  namespace: default
spec:
  policy:
    on_http_request:
      - type: oauth
        config:
          provider: google
      - expressions:
          - "!actions.ngrok.oauth.identity.email.endsWith('@example.com')"
        actions:
          - type: custom-response
            config:
              body: You are not authorized, ${actions.ngrok.oauth.identity.name}.
              status_code: 400

Check your deployed app again. If you log in with an email that isn’t under your domain, ngrok rejects the request.

What’s next?

You’ve used the open source ngrok Kubernetes Operator to add secure ingress to an app on a virtual cluster without managing IPs, network interfaces, or VPC routing. Because ngrok handles ingress and middleware execution, you can follow the same process for any Kubernetes environment, such as EKS, GKE, and OpenShift. To clean up, disconnect from your virtual cluster and delete it. This removes the namespace and all of its resources, returning the host cluster to its initial state:

vcluster disconnect
vcluster delete my-vcluster

To go further, explore the Kubernetes docs for how the Operator works and how to integrate ngrok with an existing production cluster, or try bindings and endpoint pooling.

General Guides

CLI Credentials

Dashboard SSO

Databases

Endpoints SSO

Event Destinations

Frameworks

JWT Validation

Kubernetes Ingress

OAuth

Protocols

Webhooks

Web Servers

Ingress to Kubernetes apps and services with vcluster

What you’ll need

Set up a local virtual cluster with vcluster

Install a sample application

Add OAuth protection to your demo app

What’s next?

​What you’ll need

​Set up a local virtual cluster with vcluster

​Install a sample application

​Add OAuth protection to your demo app

​What’s next?

What you’ll need

Set up a local virtual cluster with vcluster

Install a sample application

Add OAuth protection to your demo app

What’s next?