Automate Kubernetes testing with GitHub Actions and the ngrok Operator

April 29, 2025

min read

On the customer success (CS) team, part of our work is anticipating our customers' needs from ngrok and making sure we can address those needs effectively. One way we're doing that with the ngrok Kubernetes Operator is by working with Jon Stacks on our infrastructure team to build out a customer testing framework for the Operator using GitHub Actions.

Why GitHub Actions? Our day-to-day work in CS is not administering Kubernetes clusters, so we didn't want to get too deep in the weeds of setting up a cluster on one of the major cloud providers. But we also didn't want to fiddle with one-off manual tests with Minikube since we need to test dozens or hundreds of ngrok Kubernetes Operator resources at once.

Deploying a Kubernetes cluster in a GitHub Action let us abstract away much of the cluster setup and management so we could iterate with the ngrok Operator quickly and in a version-controlled way. We've learned a ton along the way, and are sharing an overview of this project in case you need for something similar for the CI pipelines you use to ship apps and APIs to Kubernetes.

The GitHub Action workflow file

Every GitHub Action needs a YAML workflow file to define what the action will do. Here's a condensed version of ours:

...
    steps:
      - name: Checkout repository
        uses: actions/checkout@v4
      - name: Setup go
        uses: actions/setup-go@v5
        with:
          go-version: "1.23.5"
      - name: Start k8s cluster
        uses: engineerd/setup-kind@v0.6.2
      - name: Install Helm
        uses: azure/setup-helm@v4
      - name: Install ngrok Helm chart
        run: |
          helm repo add ngrok https://charts.ngrok.com
      - name: Deploy the ngrok-operator Helm chart
        run: |
          helm install ngrok-operator ngrok/ngrok-operator \
          --create-namespace \
          --namespace ngrok-operator \
          --version 0.18.1 \
          --set credentials.apiKey=${{ secrets.NGROK_API_KEY }} \
          --set credentials.authtoken=${{ secrets.NGROK_AUTHTOKEN }}
      - name: Set ngrok URL
        run: |
          NGROK_URL=${{ vars.NGROK_URL }} envsubst < ${{ github.event.inputs.plan }} > tmp && mv tmp ${{ github.event.inputs.plan }}
        env:
          NGROK_URL: ${{ vars.NGROK_URL }}
      - name: Run Go script (app Helm install, etc.)
        run: go run ./... ${{ github.event.inputs.plan }}
        env:
          JOB_NUMBER: ${{ matrix.job-number }}
      - name: Operator cleanup
        run: sleep 30

In human words as opposed to computer words, the steps we've defined in this workflow are:

Check out our custom testing-based version of this repository.
Install and set up Go.
Start a Kubernetes cluster.
Install Helm.
Install the Helm chart for the ngrok Kubernetes Operator.
Deploy the ngrok-operator Helm chart.
Run a Go script (more on that later).
Sleep for 30 seconds to give the Operator time to clean up ngrok resources when we're done.

Helm, test plans, and the Go program

Now that you have the high level overview, let's dig in a bit.

The main branch of our repository that we deploy into a Github Action looks like this:

.github/
|-- workflow.yaml
charts/app
|-- templates/
|-- values.yaml
|-- Chart.yaml
|-- .helmignore
plans/
|-- agent-endpoints-small-w-traffic-policy.yaml
main.go

So at the top level, we have the GitHub directory where our workflow file is defined, as well as a Helm chart, a directory of test plans that we've written, and a Go script.

Much of the heavy lifting happens in the Go script. The script takes in a plan file, parses it, and uses the parsed values to deploy a Helm chart based on those values. Here's a snippet of our main() function:

...
    // Read and parse the plan file
    contents, err := os.ReadFile(planFile)
	if err != nil {
		logger.Error("Failed to read file", "err", err)
		os.Exit(1)
	}

	cfg := &Config{}
	if err := yaml.Unmarshal(contents, cfg); err != nil {
		logger.Error("Failed to unmarshal YAML", "err", err)
		os.Exit(1)
	}

	ctx, cancel := context.WithTimeout(context.Background(), 15*time.Minute)
	defer cancel()

	g, gctx := errgroup.WithContext(ctx)

    // Run the plan defined in the file, including deploying the Helm chart and running the defined tests
	for idx, plan := range cfg.Plans {
		planLogger := logger.With("plan", plan.Name)
		g.Go(func() error { return runPlan(gctx, planLogger, plan, idx+1) })
	}

	g.Wait()
...

For each Helm chart defined in a plan file, there is also a scenario we want to test, like: Is the app we deployed via Helm chart available at the ngrok URL we expect? Is the HTTP status code what we expect? And how long does it take for our ngrok endpoint become available?

Here's what part of a test plan file in the plans/ directory looks like:

plans:
  - name: "Test Plan 1"
    install:
      helm:
        chart: ./charts/app
        release-name: game2048
        namespace: default
        set:
          ngrokEndpoints.url: https://$NGROK_URL:443
          ingress.host: $NGROK_URL
        values-files:
          - "./charts/app/values/disable-ingress.yaml"
          - "./charts/app/values/enable-aep-crds.yaml"
          - "./charts/app/values/enable-aep-crd-traffic-policy.yaml"
    tests:
      - wait-url-ready:
          url: https://$NGROK_URL
          retries: 5
          expected-status-code: 302

Each test plan file like the above can include one test plan or one thousand. Running these test plans with GitHub Actions means we can run the action with a specific test plan or all of them at once. This setup also gives us the flexibility to run the same tests against different Kubernetes APIs—Gateway API, Ingress, or regular CRDs—to make sure the ngrok Operator behaves the way we expect with all of them.

We simplified this repo to deploy one endpoint to one ngrok URL so any ngrok account type from free to pay-as-you-go can try this walkthrough—but if you want to scale up this framework to test multiple endpoints and domains, you'll want to dynamically generate your endpoint hostnames instead of hard-coding a single hostname.

Deploy a K8s cluster and ngrok Operator yourself

Now that you have a sense of how the pieces of this project fit together, you can hop over to our GitHub repository to clone it and run it yourself. All you have to do to get started is set a few environment variables. ‍

Check out the README for detailed instructions.

Deciphering the Github Action logs

You ran the Action. It has a green checkmark next to it. What does that mean? What actually happened?

Within your Action run, the main logs you're interested in are under the Run Go script (app Helm install, etc.) step. This is where you can see what test plan is being installed:

time=2025-04-29T15:09:08.904Z level=DEBUG msg="Installing chart" plan="Test Plan 1" chart=./charts/app release-name=game2048-1 namespace=default args="[install game2048-1 ./charts/app --namespace default --wait --timeout 15m --atomic --set fullnameOverride=app-2048-1 --set ingress.host=exciting-needlessly-eel.ngrok-free.app --set ngrokEndpoints.url=https://exciting-needlessly-eel.ngrok-free.app:443 --values ./charts/app/values/disable-ingress.yaml --values ./charts/app/values/enable-aep-crds.yaml --values ./charts/app/values/enable-aep-crd-traffic-policy.yaml]"

The next important log line is where you see that you successfully fetched from ;our endpoint URL:

time=2025-04-29T15:09:25.379Z level=INFO msg="Successfully fetched URL" plan="Test Plan 1" url=https://exciting-needlessly-eel.ngrok-free.app attempt=2 successes=1

Since we got a successful fetch, the Go script will start spinning down resources so your Kubernetes cluster can be fully cleaned up:

time=2025-04-29T15:10:25.438Z level=DEBUG msg="Uninstalling chart" plan="Test Plan 1" chart=./charts/app release-name=game2048-1 namespace=default

Make it your own

This repository is meant to be a small self-contained example of how we're using the ngrok Operator with GitHub Actions internally. We're using ngrok CRDs directly to create an agent endpoint with a simple Traffic Policy rule.

If you wanted to tweak this, you could use a Kubernetes Ingress resource instead of CRDs, or you could try the Kubernetes Gateway API.

You could edit the Traffic Policy rules to do something else like send a custom-response action or use the redirect action to point traffic to a different URL—see our Traffic Policy actions for more ideas.

If your ngrok plan allows for internal bindings and additional domains, you could edit the existing test plan file, plans/agent-endpoints-with-traffic-policy.yaml, to have more than one test. Or you could write new plans to make your CI pipelines even more robust, like:

Running suite of unit/end-to-end/integration tests.
Testing different versions of the ngrok Operator with a matrix strategy.
Similarly, testing different versions of your apps/APIs with the same version of the ngrok Operator.

Just keep in mind that containers used in these jobs are ephemeral and short-lived, so this works best with automated testing—not so much deploy previews or anything you want to keep around for a while.

Reach out if you need a hand

This was a dense one! If you have questions or run into any issues setting up this project, don't hesitate to reach out to us at support@ngrok.com. Here's the repo link again: https://github.com/ngrok/ngrok-operator-gh-action-blog

Have fun!

Share this post

Jen Kagan