http-request: call services from Traffic Policy

•

891 words

•Topics:

You’ve been asking for more control over not just where traffic goes, but how you can tie together its logic with the services you’ve already deployed. I’m happy to say we’ve just added a powerful new action to our Traffic Policy engine, http-request.

It lets you make HTTP calls to internal services (via Internal Endpoints) right from your traffic policy so you can validate requests, trigger side chain events, or route traffic dynamically without writing new middleware or deploying another gateway.

If you’ve ever shipped a one-off service just to validate a token or call a webhook, you already know why this matters. And now you can do it all directly from your traffic policy.

Let me show you how it works.

Call internal services as part of your policy

First, I’ll start off by showing you a real example of using the new http-request action. This traffic policy calls an internal auth service to check a token before letting traffic through:

1on_http_request:⋯2  - name: ValidateToken⋯3    actions:⋯4      - type: http-request⋯5        config:⋯6          url: https://auth.internal/validate7          method: POST8          headers:⋯9            content-type: application/json10          body: '{ "token": "${req.headers[\"authorization\"][0]}" }'11          on_error: halt

If the validation fails, ngrok stops traffic without ever reaching your app. You can use CEL directly inside the body, URL, query parameters, or headers.

Use it for auth, logging, or internal routing

You can use the new http-request action for many different use cases:

Auth: Validate incoming requests against an identity service
Logging: Send events or errors to a collector
Chaining: Make requests to internal services to transform or inspect traffic, like “pre-tiering” requests based customer data

Because policies can run on both the on_http_request and on_http_response phases, you have full control before and after your service has handled the request.

Want to call Slack or OpenAI?

The http-request action only supports internal endpoints by design. But what if you want to call an external or public API like Slack or OpenAI?

Easy, you can make them an internal endpoint with the ngrok CLI:

1ngrok http https://hooks.slack.com \2  --url https://slack-hooks.internal \3  --host-header rewrite

This command sets up a ngrok internal endpoint (https://slack-hooks.internal) that forwards traffic to the public Slack API. And don’t forget the --host-header rewrite flag, that’s important so that way the host header is hooks.slack.com not slack-hooks.internal!

Now, from the perspective of your Traffic Policy, Slack is an internal service. You can now create a Cloud Endpoint and call it directly from your policy like this:

1on_http_response:⋯2  - name: NotifySlackOnError⋯3    expressions:⋯4      - res.status_code == 5005    actions:⋯6      - type: http-request⋯7        config:⋯8          url: https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX9          method: POST10          headers:⋯11            content-type: application/json12          body: '{ "text": "🚨 500 error on ${req.url}" }'13      - type: custom-response⋯14        config:⋯15          headers:⋯16            content-type: application/json17          status_code: 20018          body: ${res.body}

What about OpenAI? Same thing, create an internal service that forwards to the OpenAI API:

1ngrok http https://api.openai.com \2  --url https://openai.internal \3  --host-header rewrite

Then update your Cloud Endpoint to use the http-request action to call the chat/completions endpoint with a static prompt:

1on_http_request:⋯2  - name: CallOpenAI⋯3    actions:⋯4      - type: http-request⋯5        config:⋯6          url: https://openai.internal/v1/chat/completions7          method: POST8          headers:⋯9            content-type: application/json10            authorization: "Bearer <YOUR_OPENAI_API_KEY>"11          body: '{ "model": "gpt-4", "messages": [{ "role": "user", "content": "What is the meaning of life?" }] }'12          timeout: 10s13      - type: custom-response⋯14        config:⋯15          headers:⋯16            content-type: application/json17          status_code: 20018          body: ${actions.ngrok.http_request.res.body}

And just like that, you’ve setup your endpoint to call OpenAI directly from your Cloud Endpoint with no OpenAI SDK required.

Add resilience with retry logic

What happens if the service you’re calling fails temporarily? The http-request action supports automatic retries with full control over when and why to retry.

Requests are retried up to three times, to control when and how retries are decided you can create a custom CEL expression on the retry_condition configuration option. Inside of the retry CEL expression you have access to the following variables:

attempts: number of attempts so far
last_attempt.req: the last HTTP request object
last_attempt.res: the last HTTP response object
last_attempt.error: any error object that occurred (with a code and message)

Here’s an example CEL expression that retries only on 500 status codes:

1retry_condition: last_attempt.res.status_code == 500

Want to make your internal auth call more resilient? Add a timeout and retry_condition that safely handles server errors without retrying on expected denials like a 401 or 403:

1on_http_request:⋯2  - name: ResilientAuth⋯3    actions:⋯4      - type: http-request⋯5        config:⋯6          url: https://auth.internal/validate7          method: POST8          headers:⋯9            content-type: application/json10          body: '{ "token": "${req.headers[\"authorization\"][0]}" }'11          timeout: 5s12          retry_condition: last_attempt.res.status_code >= 500 && last_attempt.res.status_code < 60013          on_error: halt

This ensures your auth check won’t fail due to a single blip.

What else can you do with http-request?

Plenty. Chain internal services. Trigger workflows. Log action results. Validate authN. Wire up your own mini service mesh with nothing but YAML. Okay, maybe not everything… but that’s where you come in.

Once you’ve had a chance to play with http-request, let me know how you’re using it. Got feedback? Features you wish it had? I want to hear it all.

Until then, check out our resources:

Happy routing.