# Create embeddings for search and retrieval through AI Gateway

> Turn text into vectors with OpenAI, Gemini, Qwen, Tencent, and Mistral embedding models, using your ngrok access key and credits.

- Date: 2026-10-21
- Shipped by: Brenden Ehlers
- Products: AI Gateway
- URL: https://ngrok.com/changelog/ai-gateway-embeddings

Your embedding requests can now go through the same AI Gateway URL and access key as your chat requests. Managed models bill to your ngrok credits, so you can try OpenAI, Qwen, Tencent, and Gemini embeddings without a separate account for each provider. Every request shows up in Logs with its token usage and cost, and switching models on `/v1/embeddings` is a change to one string.

An embedding turns text into a list of numbers. Texts with similar meanings get similar vectors, so your application can compare them to find the closest match. That's the building block for semantic search, retrieval for RAG, deduplication, and classification. The example below uses it to match a production alert to the runbook most likely to fix it.

## Models and pricing

Embedding requests bill input tokens only, because a vector has no output tokens. These models work on the OpenAI-format `/v1/embeddings` endpoint:

| Model ID                             | Provider         | Price per million input tokens |
| ------------------------------------ | ---------------- | ------------------------------ |
| `openai/text-embedding-3-small`      | OpenAI           | $0.02                          |
| `openai/text-embedding-3-large`      | OpenAI           | $0.13                          |
| `openai/text-embedding-ada-002`      | OpenAI           | $0.10                          |
| `qwen/text-embedding-v4`             | QwenCloud        | $0.07                          |
| `qwen/text-embedding-v3`             | QwenCloud        | $0.07                          |
| `tencent/kinfra-text-embedding-0.6b` | Tencent TokenHub | $0.07                          |
| `tencent/kinfra-text-embedding-4b`   | Tencent TokenHub | $0.084                         |
| `mistral/mistral-embed`              | Mistral AI       | Your own key                   |
| `mistral/codestral-embed`            | Mistral AI       | Your own key                   |
| `qwen/qwen3.7-text-embedding`        | QwenCloud        | Your own key                   |

Priced models bill to your ngrok credits. For models marked **Your own key**, attach your Mistral AI or QwenCloud key, and the provider bills those requests directly.

Gemini embedding models use Google's native request format instead. Call `/v1beta/models/gemini-embedding-2:embedContent`, or `:batchEmbedContents` for several inputs, with Google's Gen AI SDK. `gemini-embedding-2` bills to your credits at $0.20 per million input tokens. `gemini-embedding-001` requires your own Google key. The [Google Gen AI SDK guide](https://ngrok.com/docs/ai-gateway/sdks/google) covers the client setup.

A few limits apply to embedding requests:

- `ngrok/auto` isn't available, because automatic routing picks among chat models.
- A fallback list must name one model. Vectors from different models don't compare, so the gateway refuses a list that mixes models with a `400` instead of corrupting your index.
- Requests accept text input only. Streaming isn't supported.

## Get started

[Create an ngrok account or sign in](https://app.ngrok.ai/). Create an [**access key**](https://app.ngrok.ai/keys) that allows OpenAI, and confirm your account has credits.

Follow the [**setup guide**](https://app.ngrok.ai/getting-started) and select **OpenAI** and **TypeScript**.

Install the OpenAI SDK and TypeScript runner:

```bash
npm install openai && npm install --save-dev tsx
```

Save your ngrok access key in `.env.local`. Keep this file out of version control:

```dotenv
AI_GATEWAY_API_KEY=your_access_key_here
```

This example matches a production alert to the runbook that's most likely to fix it. It embeds the alert and three runbook titles in one request, then compares the alert's vector against each runbook's vector with cosine similarity. The closer the score is to 1, the closer the meaning.

Use an ES module project (`"type": "module"` in `package.json`) for top-level `await`. Save this example as `index.ts`:

```typescript
import OpenAI from "openai";

const client = new OpenAI({
	baseURL: "https://gateway.ngrok.ai/v1",
	apiKey: process.env.AI_GATEWAY_API_KEY,
});

const alert = "checkout-api: 5xx rate at 38% since the latest deploy";
const runbooks = [
	"Roll back a bad deploy of the checkout service",
	"Renew an expired TLS certificate",
	"Add database capacity when connections run out",
];

const response = await client.embeddings.create({
	model: "openai/text-embedding-3-small",
	input: [alert, ...runbooks],
	encoding_format: "float",
});

const vectors = response.data.map((item) => item.embedding);
const alertVector = vectors[0];

function cosineSimilarity(a: number[], b: number[]) {
	let dot = 0;
	let normA = 0;
	let normB = 0;
	a.forEach((value, i) => {
		dot += value * b[i];
		normA += value * value;
		normB += b[i] * b[i];
	});
	return dot / (Math.sqrt(normA) * Math.sqrt(normB));
}

const ranked = runbooks
	.map((title, i) => ({ title, score: cosineSimilarity(alertVector, vectors[i + 1]) }))
	.sort((a, b) => b.score - a.score);

console.log(`Best match: ${ranked[0].title} (${ranked[0].score.toFixed(2)})`);
console.log(`Input tokens: ${response.usage.prompt_tokens}`);
```

Run the example from your project directory:

```bash
node --env-file=.env.local --import tsx index.ts
```

`response.data` holds one embedding per input, in the order you sent them. `response.usage.prompt_tokens` is the token count the request bills on. The output looks something like this. The score and token count shown here are illustrative and vary by model:

```text
Best match: Roll back a bad deploy of the checkout service (0.58)
Input tokens: 42
```

In a real application, embed your documents once, store the vectors, and embed only the incoming query at request time.

To try another model, change `model` to one of the IDs above. Keep `encoding_format: "float"`. The OpenAI SDK asks for base64 by default, and the QwenCloud and TokenHub models return float vectors only. `text-embedding-3-small`, `text-embedding-3-large`, and the QwenCloud models also accept `dimensions` for shorter vectors.

Open [**Logs**](https://app.ngrok.ai/activity), find the request you just sent, and inspect its model, provider, status, token usage, and cost.