> ## Documentation Index
> Fetch the complete documentation index at: https://ngrok.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Securing Upstream Connections with TLS & Mutual TLS (mTLS)

> Configure upstream TLS and mutual TLS in Kubernetes to secure connections between ngrok and your backend services.

If your upstream service requires encrypted TLS connections, you need to ensure that requests forwarded from the API Gateway are properly secured using TLS.
Additionally, if your upstream service requires mutual TLS (mTLS), you can configure client certificates to authenticate both the API Gateway and the upstream service.

By configuring Upstream TLS and mTLS, you can:

🔐 Ensure end-to-end encryption between ngrok and your backend services. <br />
⚡ Authenticate API Gateway requests using client certificates. <br />
🚀 Enforce stricter security policies for upstream service communication. <br />

## 🔍 What are the benefits of upstream TLS & mTLS?

Securing upstream connections is essential for protecting sensitive data in transit, ensuring that API requests remain confidential and tamper-proof as they move between services.

Key Benefits:

* **End-to-End Encryption:** Protect sensitive data from client to backend using strong encryption.
* **Client Certificate Authentication:** Ensure only trusted API gateways can connect to your backend.
* **Prevent Unauthorized Access:** Block unauthorized requests by requiring valid client certificates.
* **Ensure Regulatory Compliance:** Meet security standards like HIPAA, PCI DSS, and SOC 2.
* **Enhance Service-to-Service Security:** Secure Kubernetes microservices using mTLS authentication.

## Upstream TLS examples

The following examples showcase how you can configure client certificates to be used by the ngrok Kubernetes Operator when it connects to upstream services.

### 1. Generate certificates

```bash theme={null}
# 1. Generate CA private key (ca.key)
openssl genpkey -algorithm RSA -out ca.key -pkeyopt rsa_keygen_bits:2048

# 2. Generate CA certificate (ca.crt)
openssl req -x509 -new -nodes -key ca.key -sha256 -days 365 -out ca.crt -subj "/CN=ExampleCA"

# 3. Generate server private key (server.key)
openssl genpkey -algorithm RSA -out server.key -pkeyopt rsa_keygen_bits:2048

# 4. Generate server certificate signing request (CSR) (server.csr)
openssl req -new -key server.key -out server.csr -subj "/CN=terminate-tls-example.ngrok.app"

# 5. Generate server certificate (server.crt)
openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt -days 365 -sha256

# 6. Generate client private key (client.key)
openssl genpkey -algorithm RSA -out client.key -pkeyopt rsa_keygen_bits:2048

# 7. Generate client certificate signing request (CSR) (client.csr)
openssl req -new -key client.key -out client.csr -subj "/CN=ExampleClient"

# 8. Generate client certificate (client.crt) with Client Authentication
openssl x509 -req -in client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out client.crt -days 365 -sha256 -extfile <(printf "extendedKeyUsage = clientAuth")
```

These commands will generate the following files:

* CA Files:
  * `ca.key`: CA private key
  * `ca.crt`: CA certificate
* Server Files:
  * `server.key`: Server private key
  * `server.csr`: Server certificate signing request
  * `server.crt`: Server certificate
* Client Files:
* `client.key`: Client private key
* `client.csr`: Client certificate signing request
* `client.crt`: Client certificate (with proper clientAuth extension for mTLS)

### 2. Configure your upstream

If you're testing this with an upstream service that you have. It will need to make use of the following files that were generated.

* `server.crt`: Used to terminate TLS
* `server.key`: Used to terminate TLS
* `ca.crt`: Used to validate client certificates

### 3. Create a Kubernetes Secret for the client certificate

```bash theme={null}
kubectl create secret tls client-cert-secret \
  --cert=client.crt \
  --key=client.key
```

The following examples showcase using the client certificate when connecting to the upstream.

<Tabs>
  <Tab title="AgentEndpoint">
    ### 4. Create an `AgentEndpoint` that uses the client certificate

    ```yaml theme={null}
    apiVersion: ngrok.k8s.ngrok.com/v1alpha1
    kind: AgentEndpoint
    metadata:
      name: example-agent-endpoint
    spec:
      url: https://example-hostname.ngrok.io
      upstream:
        url: https://my-service.my-namespace:8443
      clientCertificateRefs:
      - name: client-cert-secret
        namespace: default
    ```
  </Tab>

  <Tab title="Ingress">
    ❌ `Ingress` resources do not natively support advanced upstream TLS options, so this option requires using Gateway API or the `AgentEndpoint` resources directly.

    You can still use TLS upstream services with `Ingress` resources although you will not be able to configure mutual TLS between the Operator and the upstream service.
    If your service expects HTTPS instead of HTTP, then you can annotate the `Service` like the following example to let the ngrok Kubernetes Operator know which ports are HTTPS and which are HTTP

    ```yaml theme={null}
    apiVersion: v1
    kind: Service
    metadata:
      name: example-service
      labels:
        app: example-app
      annotations:
        k8s.ngrok.com/app-protocols: '{"https-port":"https","http-port":"http"}'
    spec:
      ports:
        - name: https-port
          port: 443
          protocol: TCP
          targetPort: 8443
        - name: http-port
          port: 80
          protocol: TCP
          targetPort: 8080
      selector:
        app: example-app
    ```
  </Tab>

  <Tab title="Gateway API">
    ✅ Gateway API natively supports upstream TLS configuration, and this feature is fully supported by the ngrok Kubernetes Operator.

    ### 4. Create a `Gateway` that uses the client certificate `Secret`

    ```yaml theme={null}
    apiVersion: gateway.networking.k8s.io/v1
    kind: Gateway
    metadata:
      name: example-gateway
      namespace: default
    spec:
      backendTLS:
        clientCertificateRef:
          name: client-cert-secret
      gatewayClassName: ngrok
      listeners:
        - name: example
          hostname: "example-hostname.ngrok.io"
          port: 443
          protocol: HTTPS
    ```

    ### 5. Create an `HTTPRoute` that routes to the upstream

    ```yaml theme={null}
    apiVersion: gateway.networking.k8s.io/v1
    kind: HTTPRoute
    metadata:
      name: example-route
      namespace: default
    spec:
      hostnames:
      - example-hostname.ngrok.io
      parentRefs:
      - group: gateway.networking.k8s.io
        kind: Gateway
        name: example-gateway
        namespace: default
      rules:
      - matches:
        - path:
            type: PathPrefix
            value: /
        backendRefs:
          - group: ""
            kind: Service
            name: example-service
            port: 443
            weight: 1
    ```

    ### 6. Annotate the service

    In order for the Operator to understand what protocol each port of the `Service` expects, add the following annotation to your upstream `Service`.

    ```yaml theme={null}
    apiVersion: v1
    kind: Service
    metadata:
      name: example-service
      labels:
        app: example-app
      annotations:
        k8s.ngrok.com/app-protocols: '{"https-port":"https","http-port":"http"}'
    spec:
      ports:
        - name: https-port
          port: 443
          protocol: TCP
          targetPort: 8443
        - name: http-port
          port: 80
          protocol: TCP
          targetPort: 8080
      selector:
        app: example-app
    ```
  </Tab>
</Tabs>