User Guide

The guide contains information on how to use the ngrok Kubernetes Operator. This is the place to start if you have the operator installed and want to use it to add ingress, gateways, routes, and other ngrok features to your clusters' apps and services.

Start by reading the ingress-to-edge-relationship documentation to understand how ingress, gateway, and route objects are converted into ngrok edges.

For see or contribute a specific example, file a PR in the examples section of our repo see.

TLS and HTTPS

For http based traffic, the ngrok Kubernetes Operator will and can only provide HTTPS secured traffic. This is because the controller is responsible for creating the ngrok tunnel and edge, and ngrok only supports HTTPS for http traffic. By default if you use a standard ngrok subdomain, all traffic will be over https. If you are using a custom domain, please see the custom domain documentation for more details.

Additionally, TLS Edges may be supported soon in the future!

IP Restrictions

note

IPPolicy modules are currently only supported with the Operator.

ngrok offers the ability to restrict access to your edges by IP address via IP Restrictions. These are configurable via the IPPolicy CRD and can be attached to Ingress objects via NgrokModuleSet.

Modules

ngrok's Cloud Edge Modules allow you to configure features like compression, IP Restrictions, OAuth, adding/removing headers, and more.

note

These modules are currently only supported with the ingress controller. The Gateway API supports some of these capabilities using route rules

Design

Reusable

NgrokModuleSets are designed to be reusable. This allows you to define a set of modules and their configuration once and apply it to multiple Ingresses. Ex:

---
kind: NgrokModuleSet
apiVersion: ingress.k8s.ngrok.com/v1alpha1
metadata:
  name: module-set-1
modules:
  compression:
    enabled: true
  tlsTermination:
    minVersion: "1.2"
  headers:
    request:
      add:
        a-request-header: "my-custom-value"
        another-request-header: "my-other-custom-value"
      remove:
        - "x-remove-at-edge"
    response:
      add:
        a-response-header: "a-response-value"
---
kind: Ingress
apiVersion: networking.k8s.io/v1
metadata:
  name: example-ingress
  annotations:
    k8s.ngrok.com/modules: module-set-1
---
kind: Ingress
apiVersion: networking.k8s.io/v1
metadata:
  name: example-ingress-2
  annotations:
    k8s.ngrok.com/modules: module-set-1

In this example, the compression, tlsTermination, and headers modules are applied to both Ingresses and the same configuration is used for both. If you change the configuration of the NgrokModuleSet, the change will be applied to all Ingresses that use it.

Composable

NgrokModuleSets are designed to be composable. If multiple NgrokModuleSets are applied to an Ingress and a module is configured in more than one, the last one wins. Ex:

---
kind: NgrokModuleSet
apiVersion: ingress.k8s.ngrok.com/v1alpha1
metadata:
  name: module-set-1
modules:
  compression:
    enabled: false
---
kind: NgrokModuleSet
apiVersion: ingress.k8s.ngrok.com/v1alpha1
metadata:
  name: module-set-2
modules:
  compression:
    enabled: true
  tlsTermination:
    minVersion: "1.2"
---
kind: Ingress
apiVersion: networking.k8s.io/v1
metadata:
  name: example-ingress
  annotations:
    k8s.ngrok.com/modules: module-set-1,module-set-2

In this example, the result is the compression module is enabled since module-set-2 was supplied last. If however, the annotation is k8s.ngrok.com/modules: module-set-2,module-set-1 the order will result in the compression module being disabled since module-set-1 is supplied last and overrides the value of enabled from module-set-2.

RBAC

Since NgrokModuleSets are Kubernetes Resources(Custom Resources), you can use RBAC to control who can create, update, get, list, delete them. This allows you to control who can create and manage NgrokModuleSets, while being more permissive with Ingresses and allowing teams to self-service using pre-made configurations.

Supported Modules

Circuit Breaker

Circuit breakers are used to protect upstream servers by rejecting traffic to them when they become overwhelmed.

kind: NgrokModuleSet
apiVersion: ingress.k8s.ngrok.com/v1alpha1
metadata:
  name: circuit-breaker
modules:
  circuitBreaker:
    trippedDuration: 10s
    rollingWindow: 10s
    numBuckets: 10
    volumeThreshold: 10
    errorThresholdPercentage: "0.50"

Compression

If an HTTP request includes an Accept-Encoding header, HTTP responses will be automatically compressed and a Content-Encoding response header will be added.

Enabled

kind: NgrokModuleSet
apiVersion: ingress.k8s.ngrok.com/v1alpha1
metadata:
  name: compression-enabled
modules:
  compression:
    enabled: true

Disabled

kind: NgrokModuleSet
apiVersion: ingress.k8s.ngrok.com/v1alpha1
metadata:
  name: compression-disabled
modules:
  compression:
    enabled: false

Headers

Request

The Request Headers module allows you to add and remove headers from HTTP requests before they are sent to your upstream server.

kind: NgrokModuleSet
apiVersion: ingress.k8s.ngrok.com/v1alpha1
metadata:
  name: request-headers
modules:
  headers:
    request:
      add:
        a-request-header: "my-custom-value"
        another-request-header: "my-other-custom-value"
      remove:
        - "x-remove-before-upstream"

Response

The Response Headers module allows you to add and remove headers from HTTP responses before they are returned to the client.

kind: NgrokModuleSet
apiVersion: ingress.k8s.ngrok.com/v1alpha1
metadata:
  name: response-headers
modules:
  headers:
    response:
      add:
        a-response-header: "a-response-value"
        another-response-header: "another-response-value"
      remove:
        - "x-remove-from-resp-to-client"

IP Restrictions

IP Restrictions allow you to attach one or more IP policies to the route.

Policies may be specified by either their ID in the ngrok API or by the name of an ippolicy.ingress.k8s.ngrok.com Custom Resource if managed by the ingress controller.

kind: IPPolicy
apiVersion: ingress.k8s.ngrok.com/v1alpha1
metadata:
  name: policy-1
spec:
  description: "My Trusted IPs"
  rules:
    - action: "allow"
      cidr: 1.2.3.4/32
      description: "My Home IP"
    - action: "allow"
      cidr: 1.2.3.5/32
      description: "My Work IP"
---
kind: NgrokModuleSet
apiVersion: ingress.k8s.ngrok.com/v1alpha1
metadata:
  name: ip-restrictions
modules:
  ipRestriction:
    policies:
      - "policy-1" # Reference to the `ippolicy.ingress.k8s.ngrok.com` Custom Resource above
      - "ipp_1234567890" # Reference to an IP Policy by its ngrok API ID

OAuth

The OAuth module enforces an OAuth authentication flow in front of any route it is enabled on.

Ngrok Managed OAuth Application

Google

kind: NgrokModuleSet
apiVersion: ingress.k8s.ngrok.com/v1alpha1
metadata:
  name: ngrok-managed-google-oauth
modules:
  oauth:
    google:
      optionsPassthrough: true
      inactivityTimeout: 10m
      maximumDuration: 24h
      authCheckInterval: 20m
      emailAddresses:
        - my-email@my-domain.com
      # Or specify a list of domains instead of individual email addresses
      # emailDomains:
      # - my-domain.com

User Managed OAuth Application

Google

---
kind: Secret
apiVersion: v1
metadata:
  name: google-oauth-secret
type: Opaque
data:
  CLIENT_SECRET: "<base64-encoded-client-secret>"
---
kind: NgrokModuleSet
apiVersion: ingress.k8s.ngrok.com/v1alpha1
metadata:
  name: user-managed-google-oauth
modules:
  oauth:
    google:
      optionsPassthrough: true
      inactivityTimeout: 10m
      maximumDuration: 24h
      authCheckInterval: 20m
      clientId: "<client-id>.apps.googleusercontent.com"
      clientSecret:
        name: google-oauth-secret # The name of the k8s secret
        key: CLIENT_SECRET # The key in the k8s secret containing the client secret
      scopes:
        - openid
        - email

OpenID Connect (OIDC)

The OIDC module restricts endpoint access to only users authorized by a OpenID Identity Provider.

---
kind: Secret
apiVersion: v1
metadata:
  name: oidc-secret
type: Opaque
data:
  CLIENT_SECRET: "<base64-encoded-client-secret>"
---
kind: NgrokModuleSet
apiVersion: ingress.k8s.ngrok.com/v1alpha1
metadata:
  name: oidc
modules:
  oidc:
    clientId: "<client-id>.apps.googleusercontent.com"
    clientSecret:
      name: oidc-secret
      key: CLIENT_SECRET
    maximumDuration: 24h
    inactivityTimeout: 3h
    issuer: https://accounts.google.com
    optionsPassthrough: true
    scopes:
      - openid
      - email

SAML

The SAML module restricts endpoint access to only users authorized by a SAML IdP.

TLS Termination

Allows you to configure whether ngrok terminates TLS traffic at its edge or forwards the TLS traffic through unterminated.

kind: NgrokModuleSet
apiVersion: ingress.k8s.ngrok.com/v1alpha1
metadata:
  name: tls
modules:
  tlsTermination:
    minVersion: "1.3"

Mutual TLS

This mutual TLS (mTLS) module performs authentication when the ngrok edge terminates TLS on incoming connections to your HTTP endpoint. The client must present a valid TLS certificate that is signed by one of the specified CAs or the connection will be rejected.

kind: NgrokModuleSet
apiVersion: ingress.k8s.ngrok.com/v1alpha1
metadata:
  name: mtls
modules:
  mutualTLS:
    # These IDs can be found on https://dashboard.ngrok.com/security/tls/cert-authorities
    certificateAuthorities:
      - ca_2gDZI7eD3fpxmKGOVqzVuETKedf

Webhook Verification

The webhook verification module allows ngrok to assert requests to your endpoint originate from a supported webhook provider like Slack or Github.

---
apiVersion: v1
kind: Secret
metadata:
  name: github-webhook-token
type: Opaque
data:
  SECRET_TOKEN: "<base64-encoded-webhook-secret>"

---
kind: NgrokModuleSet
apiVersion: ingress.k8s.ngrok.com/v1alpha1
metadata:
  name: webhook-verification
modules:
  webhookVerification:
    provider: github
    secret:
      name: github-webhook-token
      key: SECRET_TOKEN

Configuring Multiple Modules

The following NgrokModuleSet named example:

Enables a circuit breaker
Enables compression
Adds and removes headers from both the request and response
Restricts access to the route to a list of trusted IPs defined in policy-1
Uses a ngrok managed OAuth application to authenticate users
Configures TLS termination

---
kind: IPPolicy
apiVersion: ingress.k8s.ngrok.com/v1alpha1
metadata:
  name: policy-1
spec:
  description: "My Trusted IPs"
  rules:
    - action: "allow"
      cidr: 1.2.3.4/32
      description: "My Home IP"
    - action: "allow"
      cidr: 1.2.3.5/32
      description: "My Work IP"
---
kind: NgrokModuleSet
apiVersion: ingress.k8s.ngrok.com/v1alpha1
metadata:
  name: example
modules:
  circuitBreaker:
    trippedDuration: 10s
    rollingWindow: 10s
    numBuckets: 10
    volumeThreshold: 20
    errorThresholdPercentage: "0.50"
  compression:
    enabled: true
  headers:
    request:
      add:
        a-request-header: "my-custom-value"
        another-request-header: "my-other-custom-value"
      remove:
        - "x-remove-before-upstream"
    response:
      add:
        a-response-header: "a-response-value"
        another-response-header: "another-response-value"
      remove:
        - "x-remove-from-resp-to-client"
  ipRestriction:
    policies:
      - policy-1
  oauth:
    google:
      optionsPassthrough: true
      inactivityTimeout: 10m
      maximumDuration: 24h
      authCheckInterval: 20m
  tlsTermination:
    minVersion: "1.3"
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: example
  annotations:
    k8s.ngrok.com/modules: "example"
spec:
  ingressClassName: ngrok
  rules:
    - host: <my-host>.ngrok.app
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: <service-name>
                port:
                  number: <service-port>

Traffic Policy

Traffic policies for inbound and outbound traffic can simplifify edge management.

warning

Traffic Policy is currently in preview. Breaking changes may occur at any time with no notice, including changes to the structure of policy documents, the behaviors of policies, and the pricing of this feature.

Design

NgrokTrafficPolicy objects can be defined with rules composed of expressions and actions that validate and filter traffic via the policy engine.

kind: NgrokTrafficPolicy
apiVersion: ngrok.k8s.ngrok.com/v1alpha1
metadata:
  name: traffic-policy-example
spec:
  policy:
    inbound:
      - name: "greetings"
        expressions:
          - "req.method == 'GET'"
          - "req.path.startsWith('/greetings')"
        actions:
          - type: "custom_response"
            config:
              status_code: 200
              content: Hello
              headers:
                content-type: text/plain

Learn more about the policy expressions and actions here:

Ingress Composition

Traffic policies can be added to Ingress objects using annotations.

---
kind: NgrokTrafficPolicy
apiVersion: ngrok.k8s.ngrok.com/v1alpha1
metadata:
  name: traffic-policy-test
spec:
  policy:
    inbound:
      - name: "deny_patch"
        expressions:
          - "req.method == 'PATCH'"
        actions:
          - type: "deny"
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: example
  annotations:
    k8s.ngrok.com/traffic-policy: traffic-policy-test
spec:
  ingressClassName: ngrok
  rules:
    - host: <my-host>.ngrok.app
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: <service-name>
                port:
                  number: <service-port>

note

Using both an NgrokModuleSet and an NgrokTrafficPolicy will result in an error.

note

Ingress is currently limited to a max of one traffic policy in your traffic policy annotation list.

Gateway Composition

Traffic policies can be added to Gateway HTTPRoute objects using an ExtensionRef filter.

---
kind: NgrokTrafficPolicy
apiVersion: ngrok.k8s.ngrok.com/v1alpha1
metadata:
  name: traffic-policy-test
spec:
  policy:
    inbound:
      - name: "deny_patch"
        expressions:
          - "req.method == 'PATCH'"
        actions:
          - type: "deny"
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: example-route
spec:
  parentRefs:
    - kind: Gateway
      name: ngrok-gateway
  hostnames:
    - "<my-host>.ngrok.app"
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /
      filters:
        - type: ExtensionRef
          extensionRef:
            group: ngrok.k8s.ngrok.com
            kind: NgrokTrafficPolicy
            name: traffic-policy-test
      backendRefs:
        - name: <service-name>
          kind: Service
          port: <service-port>

TCP and TLS Edges

ngrok offers TCP and TLS Edges which can be used to provide ingress to TCP or TLS based services. Both are implemented as CRDs and function similarly in broad strokes, albeit with slightly different configuration options offered. Their CRD reference is a useful companion to this guide.

While these edges and tunnels can be managed via the CRDs, the operator is capable of exposing kubernetes Service resources to the internet. This is done by creating a ngrok tunnel and edge for the service. The controller will automatically create a ngrok tunnel and edge for the service when the service is created or updated. The controller will also automatically delete the ngrok tunnel and edge when the service is deleted.

TCP LoadBalancer

By default, services of type LoadBalancer are exposed using a ngrok TCP Edge. A reserved address is automatically created for the service and the service's status will be updated with the reserved address. Other projects like external-dns can be used to create a CNAME record for the reserved address automatically.

Example

apiVersion: v1
kind: Service
metadata:
  name: mysite
  annotations:
    k8s.ngrok.com/modules: only-trusted-ips # Optional. The ngrok modules used for this Service
spec:
  allocateLoadBalancerNodePorts: false # ngrok's tunneling technology does not require NodePorts to be allocated.
  loadBalancerClass: ngrok
  type: LoadBalancer
  selector:
    app: mysite
  ports:
    - name: traffic
      port: 9000
      protocol: TCP
      targetPort: 9000

Modules

The k8s.ngrok.com/modules annotation can be used to specify the ngrok modules to use for the service. The following modules are available for TCP services:

IP Restrictions

If other modules are supplied that are not supported by the TCP edge, the controller will ignore them.

TLS LoadBalancer

Example

Note: the k8s.ngrok.com/domain annotation is required to use TLS and will expose the service as a TLS Edge. Once the reserved domain is ready and the TLSEdge is created, the service's status will be updated with the ngrok address. Other projects like external-dns can be used to create a CNAME record for the reserved domain automatically.

apiVersion: v1
kind: Service
metadata:
  name: mysite
  annotations:
    k8s.ngrok.com/modules: only-trusted-ips # Optional. The ngrok modules used for this Service
    k8s.ngrok.com/domain: mysite.mydomain.com # Required to use TLS
    external-dns.alpha.kubernetes.io/hostname: "mysite.mydomain.com"
spec:
  allocateLoadBalancerNodePorts: false # ngrok's tunneling technology does not require NodePorts to be allocated.
  loadBalancerClass: ngrok
  type: LoadBalancer
  selector:
    app: mysite
  ports:
    - name: traffic
      port: 9000
      protocol: TCP
      targetPort: 9000

Modules

The k8s.ngrok.com/modules annotation can be used to specify the ngrok modules to use for the service. The following modules are available for TLS services:

If other modules are supplied that are not supported by the TLS edge, the controller will ignore them.

Manual management via CRDs

While using services is recommended for simplicity, TCP and TLS edges and tunnels can be managed manually via the operator.

(TLS Only) Get a Domain

At least one hostports must be specified when creating a TLSEdge resource, which takes the form <fqdn>:443. The fully qualified domain name must first be reserved either via the ngrok dashboard or the Domain CRD.

Example:

apiVersion: ingress.k8s.ngrok.com/v1alpha1
kind: Domain
metadata:
  name: tlsedgetest-ngrok-app
spec:
  domain: tlsedgetest.ngrok.app

Create the Edge

Create the edge CRD. These resources are fairly similar, and both require you to specify a TunnelGroupBackend. This consists of a list of labels that determine which specific Tunnel should receive traffic from the edge. Both may also specify IP Policies for limiting access to the edge. At the time of writing, these policies must be provided as a reference in the form ipp_<id>.

On top of the options available to TCP Edges, TLS Edges support (and require) a few other options:

(required) hostports: A list of "<fqdn>:443" strings declaring the list of reserved domains for the edge to listen on.
tlsTermination: Configure the TLS Termination behavior. The terminateAt field may be set to upstream to pass the encrypted stream to the Tunnel backend, or edge to terminate the TLS stream at the ngrok edge, and pass plaintext bytes to the Tunnel.
mutualTls: Configure client certificate validation at the edge. Requires a reference to a Certificate Authority.

TCP Example:

apiVersion: ingress.k8s.ngrok.com/v1alpha1
kind: TCPEdge
metadata:
  name: test-edge
spec:
  backend:
    labels:
      app: tcptestedge

TCP Edges currently do not support providing a reserved TCP address. Therefore, one will be allocated upon edge creation. This can be viewed by checking the status of the resource:

$ kubectl get tcpedges test-edge
NAME        ID                                   HOSTPORTS                  BACKEND ID                          AGE
test-edge   edgtcp_2Wg5AzVE878vQoNMP3Z8wONIr76   ["7.tcp.ngrok.io:27866"]   bkdtg_2Wg5Amjb4GiQoV7SAnpEdM0Dg3n   2m35s

TLS Example:

apiVersion: ingress.k8s.ngrok.com/v1alpha1
kind: TLSEdge
metadata:
  name: test-edge
spec:
  hostports:
    - tlstestedge.ngrok.app:443
  backend:
    labels:
      app: tlstestedge
  tlsTermination:
    terminateAt: upstream

Start the Tunnel

Finally, create a Tunnel to receive and forward traffic for your edge.

Important fields:

forwardsTo: The <hostname>:<port> to forward traffic to. This can be any hostname resolvable and accessible from the ingress controller pod.
labels: a map of labels corresponding to the edge to receive traffic for. These must match the labels specified when creating your edge.
backend.protocol: The protocol understood by the backend service. TCP will forward connections to the backend as-is, while TLS will create a TLS connection to the backend first, and then forward the connection stream over that.

Example:

apiVersion: ingress.k8s.ngrok.com/v1alpha1
kind: Tunnel
metadata:
  name: test-tunnel
spec:
  backend:
    protocol: TCP
  forwardsTo: kubernetes.default.svc:443
  labels:
    app: tlsedgetest

Full Example

This is an example of using a TLS Edge to expose the kubernetes control plane via ngrok.

---
apiVersion: ingress.k8s.ngrok.com/v1alpha1
kind: Domain
metadata:
  name: tlsedgetest-ngrok-app
spec:
  # Reserve the tlsedgetest.ngrok.app domain.
  domain: tlsedgetest.ngrok.app
---
apiVersion: ingress.k8s.ngrok.com/v1alpha1
kind: TLSEdge
metadata:
  name: test-edge
spec:
  hostports:
    # Listen for connections on the domain we reserved
    - tlsedgetest.ngrok.app:443
  backend:
    labels:
      app: tlsedgetest
  # Pass the TLS stream on to the backend - let the application do its own TLS
  # handshake.
  tlsTermination:
    terminateAt: upstream
---
apiVersion: ingress.k8s.ngrok.com/v1alpha1
kind: Tunnel
metadata:
  name: test-tunnel
spec:
  # Forward the raw TCP stream to our backend.
  # It will technically contain TLS, and the backend speaks TLS, but we don't
  # want the Tunnel to terminate TLS before forwarding incoming connections.
  # We don't want a TLS turducken.
  backend:
    protocol: TCP
  # Forward to the kubernetes control plane.
  forwardsTo: kubernetes.default.svc:443
  # Listen for connections using the labels from our edge.
  labels:
    app: tlsedgetest

Check the status of your resources:

$ kubectl get domain
NAME                    ID                               REGION   DOMAIN                  CNAME TARGET   AGE
tlsedgetest-ngrok-app   rd_2Wg986lvMqsiB1J5WV5lOcmT21a            tlsedgetest.ngrok.app                  4s
$ kubectl get tlsedge
NAME        ID                                   HOSTPORTS                       BACKEND ID                          AGE
test-edge   edgtls_2Wg989BMmZLWXixStL8BjAxMcxW   ["tlsedgetest.ngrok.app:443"]   bkdtg_2Wg981gcSnxaX5cTL28LWwVg4xD   12s
$ kubectl get tunnel
NAME          FORWARDSTO                   AGE
test-tunnel   kubernetes.default.svc:443   52m

Our domain and edge both have IDs allocated, so we know they've been created successfully!

Edit your kubeconfig and replace the server with https://tlsedgetest.ngrok.app, comment out certificate-authority-data and add insecure-skip-tls-verify: true to your cluster config. This is needed because kubernetes is completing the TLS handshake with its own certificate, which won't be valid for your ngrok domain.

Use kubectl cluster-info to verify that everything is still working:

$ kubectl cluster-info
Kubernetes control plane is running at https://tlsedgetest.ngrok.app
CoreDNS is running at https://tlsedgetest.ngrok.app/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

Agent Endpoints

Agent Endpoints along with Cloud Endpoints are the successors to Edges. Agent Endpoints are ephemeral and tied to the lifetime of the ngrok agent.

Using Agent Endpoints, you can easily expose your Kubernetes services to internet, or use .internal domains to allow other ngrok endpoints to route to them internally without exposing them to the public internet.

The ngrok kubernetes operator ensures that you always have a set of Agent Endpoints according to the configuration of any and all AgentEndpoint custom resources (AgentEndpoint CRD reference). you have applied. You may see the ID's of any agent endpoints within the ngrok dashboard change over time if they are managed by the operator whenever the operator is restarted, but their configuraiton and availability will always be consistent so long as the operator is running.

See the ngrok agent CLI configuration page, for more information about using the CLI to start agent endpoints outside of Kubernetes.

Note: Agent Endpoints are currently in feature-preview for the ngrok Kubernetes operator. You will need to use --version 0.17.0-rc.1 (or newer) when using helm to install or update the operator. See the deployment guide for information about installing the ngrok Kubernetes operator.

HTTP Endpoints

This Agent Endpoint accepts cleartext HTTP connections at http://example-http-domain.ngrok.io, and forwards to the kubernetes service test-service.default on port 8080 over cleartext HTTP. While the public url is http://, the upstream.url does not need to also use http://. You can forward traffic to HTTPS or cleartext HTTP upstream urls.

---
apiVersion: ngrok.k8s.ngrok.com/v1alpha1
kind: AgentEndpoint
metadata:
  name: http-k8s
  namespace: default
spec:
  description: example HTTP agent endpoint # optional description
  metadata: '{"owned-by":"ngrok-operator"}' # optional metadata
  upstream:
    protocol: http1 # optional, allows http1/http2. defaults to http1 for HTTP/HTTPS agent endpoints
    url: test-service.default:8080
  url: http://example-http-domain.ngrok.io

HTTPS Endpoints

This Agent Endpoint accepts HTTPS connections at https://example-https-domain.ngrok.io, terminates TLS, and forwards to the kubernetes service test-service.default on port 8443 over HTTPS. While the public url is https://, the upstream.url does not need to also use https://. You can forward traffic to HTTPS or cleartext HTTP upstream urls.

---
apiVersion: ngrok.k8s.ngrok.com/v1alpha1
kind: AgentEndpoint
metadata:
  name: https-k8s
  namespace: default
spec:
  description: example HTTPS agent endpoint
  metadata: '{"owned-by":"ngrok-operator"}'
  upstream:
    protocol: http2
    url: https://test-service.default:8443
  url: https://example-https-domain.ngrok.io

The following example showcases forwarding traffic to a publicy routable url outside of Kubernetes, but will also work with an HTTP Agent Endpoint.

---
apiVersion: ngrok.k8s.ngrok.com/v1alpha1
kind: AgentEndpoint
metadata:
  name: https-external
  namespace: default
spec:
  description: example HTTPS agent endpoint with external upstream
  metadata: '{"owned-by":"ngrok-operator"}'
  upstream:
    protocol: http2
    url: https://httpbin.org
  url: https://example-https-domain-external-upstream.ngrok.io

The following example showcases how a traffic policy with the terminate-tls action can be used to control TLS termination options. More information and examples about configuring traffic policies on Agent Endpoints can be found in the traffic policy section below.

For HTTPS endpoints, ngrok will already terminate TLS connections for you even if you do not explicitly use this action in your Traffic Policy. If you specify this action in your Traffic Policy without any configuration, you will see no change in behavior for your endpoints.

---
apiVersion: ngrok.k8s.ngrok.com/v1alpha1
kind: AgentEndpoint
metadata:
  name: https-k8s
  namespace: default
spec:
  description: example HTTPS agent endpoint
  metadata: '{"owned-by":"ngrok-operator"}'
  trafficPolicy:
    inline:
      on_tcp_connect:
        - actions:
            - type: terminate-tls
              config:
                min_version: 1.2
                max_version: 1.3
  upstream:
    protocol: http2
    url: https://test-service.default:8443
  url: https://example-https-domain.ngrok.io

TCP Endpoints

TCP Agent Endpoints can be used to forward raw TCP traffic. In order to create a TCP Agent Endpoint, you must first reserve a TCP address through the ngrok dashboard.

---
apiVersion: ngrok.k8s.ngrok.com/v1alpha1
kind: AgentEndpoint
metadata:
  name: tcp-k8s
  namespace: default
spec:
  description: example TCP agent endpoint
  metadata: '{"owned-by":"ngrok-operator"}'
  upstream:
    url: tcp://test-service.default:9000
  url: tcp://5.tcp.ngrok.io:24429 # TCP address reserved from the ngrok dashboard

TLS Endpoints

TLS endpoints enable you to deliver any network service that runs over a TLS-based protocol. TLS endpoints make no assumptions about the wrapped protocol being transported.

---
apiVersion: ngrok.k8s.ngrok.com/v1alpha1
kind: AgentEndpoint
metadata:
  name: tls-k8s
  namespace: default
spec:
  description: test tls agent endpoint
  metadata: '{"owned-by":"ngrok-operator"}'
  upstream:
    url: tls://test-service.default:9000
  url: https://example-tls-domain.ngrok.io

For TLS endpoints, ngrok will not terminate the TLS connection by default and it is up to you to handle TLS termination in your upstream service.

---
apiVersion: ngrok.k8s.ngrok.com/v1alpha1
kind: AgentEndpoint
metadata:
  name: tls-k8s
  namespace: default
spec:
  description: test tls agent endpoint
  metadata: '{"owned-by":"ngrok-operator"}'
  trafficPolicy:
    inline:
      on_tcp_connect:
        - actions:
            - type: terminate-tls
              config:
                min_version: 1.2
                max_version: 1.3
  upstream:
    url: tls://test-service.default:9000
  url: https://example-tls-domain.ngrok.io

Internal Endpoints

Agent Endpoints with a url ending in .internal can be used to create Internal Endpoints.

All internal endpoints end in .internal, and are specific to one ngrok account. Requests may only be forwarded to a internal endpoint on the same account as the the internal endpoint, and the target must be of the same protocol (i.e. an HTTP Endpoint may only forward to an HTTP Internal Endpoint). In addition, the target's traffic policy may only specify policy for the current protocol, for example if forward-internal is used in the on_http_request traffic policy phase, the internal endpoint may only have on_http_request and on_http_response sections in its traffic policy configuration.

1. Create a public Agent Endpoint

The following Agent Endpoint will forward all traffic to an internal Agent Endpoint that will be created next

---
apiVersion: ngrok.k8s.ngrok.com/v1alpha1
kind: AgentEndpoint
metadata:
  name: forward-internal
  namespace: default
spec:
  description: forwards to the following internal agent endpoint
  metadata: '{"owned-by":"ngrok-operator"}'
  trafficPolicy:
    inline:
      on_http_request:
        - actions:
            - type: forward-internal
              config:
                binding: internal
                url: https://example-https-endpoint.internal
  upstream:
    protocol: http1
    url: foo.bar:8080
  url: https://example-forward-internal-aep.ngrok.io

Since the above endpoint is forwarding all traffic to the below internal endpoint, no traffic will be sent to foo.bar:8080 which is the upstream.

2. Create an internal Agent Endpoint

This internal Agent Endpoint is not accessible from the public internet, but may receive traffic from other Cloud and Agent Endpoints using their traffic policy's forward-internal action.

---
apiVersion: ngrok.k8s.ngrok.com/v1alpha1
kind: AgentEndpoint
metadata:
  name: internal
  namespace: default
spec:
  description: example internal agent endpoint
  metadata: '{"owned-by":"ngrok-operator"}'
  upstream:
    protocol: http1
    url: test-service.default:8080
  url: https://example-https-endpoint.internal
  bindings:
    - internal

3. Create a Cloud Endpoint

The following showcases how you could also use a Cloud Endpoint to forward traffic to your internal Agent Endpoint

Cloud Endpoints can created with Kubernetes resources such as with the following example, but they can also be created from the ngrok dashboard and the ngrok API.

Cloud Endpoints enable you to:

Centrally manage your traffic policy configurations in Kubernetes and/or ngrok's cloud service
Load balance traffic to different ngrok agents/operators
Apply different configurations for each path in your application. For example, you may apply OAuth on /app and Compression on /static/css

---
apiVersion: ngrok.k8s.ngrok.com/v1alpha1
kind: CloudEndpoint
metadata:
  name: forward-internal
  namespace: default
spec:
  description: https cloud endpoint forwarding to internal agent endpoint
  metadata: '{"owned-by":"ngrok-operator"}'
  trafficPolicy:
    policy:
      on_http_request:
        - actions:
            - type: forward-internal
              config:
                url: https://example-https-endpoint.internal
  url: https://example-forward-internal-cep.ngrok.io

Traffic Policies with Agent Endpoints

The follwoing example showcases how to inline a traffic policy configuration for an Agent Endpoint.

---
apiVersion: ngrok.k8s.ngrok.com/v1alpha1
kind: AgentEndpoint
metadata:
  name: inline-traffic-policy
  namespace: default
spec:
  description: https agent endpoint with inline traffic policy
  metadata: '{"owned-by":"ngrok-operator"}'
  trafficPolicy:
    inline:
      on_http_request:
        - actions:
            - type: custom-response
              config:
                status_code: 503
                content:
                  <html><body><h1>Service Unavailable</h1><p>Our servers are currently
                  down for maintenance. Please check back later.</p></body></html>
                headers:
                  content-type: text/html
  upstream:
    protocol: http2
    url: https://httpbin.org
  url: https://example-endpoint-tp-inline.ngrok.io

The follwoing example showcases how to supply a traffic policy configuration via reference to an NgrokTrafficPolicy custom resource for an Agent Endpoint. Note that the referenced NgrokTrafficPolicy must be in the same namespace as the AgentEndpoint it is being referenced from.

apiVersion: ngrok.k8s.ngrok.com/v1alpha1
kind: NgrokTrafficPolicy
metadata:
  name: example-policy
  namespace: default
spec:
  policy:
    on_http_request:
      - actions:
          - type: custom-response
            config:
              status_code: 503
              content:
                <html><body><h1>Service Unavailable</h1><p>Our servers are currently
                down for maintenance. Please check back later.</p></body></html>
              headers:
                content-type: text/html

---
apiVersion: ngrok.k8s.ngrok.com/v1alpha1
kind: AgentEndpoint
metadata:
  name: referenced-traffic-policy
  namespace: default
spec:
  description: https agent endpoint with referenced traffic policy
  metadata: '{"owned-by":"ngrok-operator"}'
  trafficPolicy:
    targetRef:
      name: example-policy
  upstream:
    protocol: http2
    url: https://httpbin.org
  url: https://example-endpoint-tp-ref.ngrok.io

TLS and HTTPS

IP Restrictions

Modules

Design​

Reusable​

Composable​

RBAC​

Supported Modules​

Circuit Breaker​

Compression​

Enabled​

Disabled​

Headers​

Request​

Response​

IP Restrictions​

OAuth​

Ngrok Managed OAuth Application​

Google​

User Managed OAuth Application​

Google​

OpenID Connect (OIDC)​

SAML​

TLS Termination​

Mutual TLS​

Webhook Verification​

Configuring Multiple Modules​

Traffic Policy​

Design​

Ingress Composition​

Gateway Composition​

TCP and TLS Edges​

TCP LoadBalancer​

Example​

Modules​

TLS LoadBalancer​

Example​

Modules​

Manual management via CRDs​

(TLS Only) Get a Domain​

Create the Edge​

Start the Tunnel​

Full Example​

Agent Endpoints​

HTTP Endpoints​

HTTPS Endpoints​

TCP Endpoints​

TLS Endpoints​

Internal Endpoints​

1. Create a public Agent Endpoint​

2. Create an internal Agent Endpoint​

3. Create a Cloud Endpoint​

Traffic Policies with Agent Endpoints​

Design

Reusable

Composable

RBAC

Supported Modules

Circuit Breaker

Compression

Enabled

Disabled

Headers

Request

Response

IP Restrictions

OAuth

Ngrok Managed OAuth Application

Google

User Managed OAuth Application

Google

OpenID Connect (OIDC)

SAML

TLS Termination

Mutual TLS

Webhook Verification

Configuring Multiple Modules

Traffic Policy

Design

Ingress Composition

Gateway Composition

TCP and TLS Edges

TCP LoadBalancer

Example

Modules

TLS LoadBalancer

Example

Modules

Manual management via CRDs

(TLS Only) Get a Domain

Create the Edge

Start the Tunnel

Full Example

Agent Endpoints

HTTP Endpoints

HTTPS Endpoints

TCP Endpoints

TLS Endpoints

Internal Endpoints

1. Create a public Agent Endpoint

2. Create an internal Agent Endpoint

3. Create a Cloud Endpoint

Traffic Policies with Agent Endpoints