What you’ll need
- A MicroK8s cluster reachable with
kubectl. If you don’t have one, Canonical’s MicroK8s get-started guide covers installing it. - An ngrok account.
kubectland Helm 3.0.0+ installed on your local workstation.- The ngrok Kubernetes Operator installed on your cluster.
- A reserved domain from the ngrok dashboard or API; this guide refers to it as
NGROK_DOMAIN.
Set up your MicroK8s cluster
This guide assumes a single-node MicroK8s cluster reachable withkubectl.
MicroK8s runs on your local workstation, an IoT device, a virtual machine in the cloud, or any system running one of the supported Linux flavors.
If you don’t have a cluster yet, follow Canonical’s MicroK8s get-started guide to install the binary for your target system.
MicroK8s deploys a single-node Kubernetes cluster automatically.
Confirm your cluster is ready before continuing:
microk8s is running, your cluster is ready.
Install an example app
Deploy the demo app you’ll expose through ngrok. Create a new Kubernetes manifest (tinyllama.yaml) with the YAML below.
This manifest defines the tinyllama demo LLM application from ngrok-samples/tinyllama as a service and deployment.
showLineNumbers
Configure the ngrok Kubernetes Operator
With your example app running alongside the ngrok Kubernetes Operator, configure the Kubernetes Gateway API to route traffic arriving onNGROK_DOMAIN to the tinyllama service.
First you’ll create a GatewayClass, which represents a class of cluster-level Gateways.
Then you’ll configure the Gateway with a listener and an HTTPRoute that specifies how the Gateway routes requests.
-
Create a new file called
gatewayclass.yamlon your workstation with the following YAML: -
Install the GatewayClass:
-
Create a new file named
tinyllama-gateway.yamlon your workstation with the following YAML, replacingNGROK_DOMAINwith the domain you reserved:TheshowLineNumbersGatewayallows north/south traffic to enter the cluster from external requests, and theHTTPRouteterminates the connection at the pod running thetinyllamaservice. -
Apply the Gateway and HTTPRoute:
-
Access your tinyllama demo app by navigating to your ngrok domain, for example,
https://NGROK_DOMAIN. ngrok’s network and the ngrok Kubernetes Operator route traffic to your app from any device or external network.