> ## Documentation Index
> Fetch the complete documentation index at: https://ngrok.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# SDK Integration

> Connect your SDK to the ngrok AI Gateway.

<Note>
  **Prerequisite**: You need an AI Gateway endpoint before using these SDK guides. Create one using the [dashboard quickstart](/ai-gateway/quickstart) or follow the [manual setup guide](/ai-gateway/guides/creating-endpoints).
</Note>

The AI Gateway works with official and third-party SDKs. Change the base URL in your SDK configuration and you're connected.

## Supported SDKs

<CardGroup cols={2}>
  <Card title="OpenAI SDK" icon="circle-nodes" href="/ai-gateway/sdks/openai">
    Official SDKs for Python, TypeScript, Go, Java, and .NET
  </Card>

  <Card title="Anthropic SDK" icon="message-bot" href="/ai-gateway/sdks/anthropic">
    Official SDKs for Python, TypeScript, Java, Go, Ruby, C#, and PHP
  </Card>

  <Card title="Vercel AI SDK" icon="triangle" href="/ai-gateway/sdks/vercel-ai-sdk">
    Build AI apps with React, Next.js, and streaming
  </Card>

  <Card title="TanStack AI" icon="layer-group" href="/ai-gateway/sdks/tanstack-ai">
    Type-safe AI library for React and Solid
  </Card>

  <Card title="LangChain" icon="link" href="/ai-gateway/sdks/langchain">
    Framework for chains, agents, and RAG
  </Card>

  <Card title="Other SDKs" icon="code" href="/ai-gateway/sdks/other">
    cURL, HTTP clients
  </Card>
</CardGroup>

## Quick start

The pattern is the same for any SDK—just change the base URL.

**Which API key do I use?**

* **AI Gateway API Keys** (recommended): Use your [AI Gateway API Key](/ai-gateway/concepts/api-keys) (format: `ng-xxxxx-g1-xxxxx`). ngrok handles provider keys for you.
* **BYOK (passthrough mode)**: Use your provider API key (for example, `sk-...` from OpenAI). See [Bring Your Own Keys](/ai-gateway/concepts/bring-your-own-keys).

<CodeGroup>
  ```python Python (OpenAI) theme={null}
  from openai import OpenAI

  client = OpenAI(
      base_url="https://your-ai-gateway.ngrok.app/v1",
      api_key="ng-xxxxx-g1-xxxxx"  # Your AI Gateway API Key
  )
  ```

  ```python Python (Anthropic) theme={null}
  import anthropic

  client = anthropic.Anthropic(
      base_url="https://your-ai-gateway.ngrok.app",
      api_key="ng-xxxxx-g1-xxxxx"  # Your AI Gateway API Key
  )
  ```

  ```typescript TypeScript (OpenAI) theme={null}
  import OpenAI from "openai";

  const client = new OpenAI({
    baseURL: "https://your-ai-gateway.ngrok.app/v1",
    apiKey: "ng-xxxxx-g1-xxxxx",  // Your AI Gateway API Key
  });
  ```

  ```typescript TypeScript (Anthropic) theme={null}
  import Anthropic from "@anthropic-ai/sdk";

  const client = new Anthropic({
    baseURL: "https://your-ai-gateway.ngrok.app",
    apiKey: "ng-xxxxx-g1-xxxxx",  // Your AI Gateway API Key
  });
  ```

  ```typescript TypeScript (Vercel AI SDK) theme={null}
  import { createOpenAI } from "@ai-sdk/openai";

  const openai = createOpenAI({
    baseURL: "https://your-ai-gateway.ngrok.app/v1",
    apiKey: "ng-xxxxx-g1-xxxxx",  // Your AI Gateway API Key
  });
  ```

  ```typescript TypeScript (TanStack AI) theme={null}
  import { openai } from "@tanstack/ai-openai";

  const adapter = openai({
    baseURL: "https://your-ai-gateway.ngrok.app/v1",
    apiKey: "ng-xxxxx-g1-xxxxx",  // Your AI Gateway API Key
  });
  ```

  ```python Python (LangChain) theme={null}
  from langchain_openai import ChatOpenAI

  llm = ChatOpenAI(
      base_url="https://your-ai-gateway.ngrok.app/v1",
      api_key="ng-xxxxx-g1-xxxxx",  # Your AI Gateway API Key
  )
  ```
</CodeGroup>

## What works through the gateway

Everything your SDK supports works through the gateway:

| Feature               | Supported               |
| --------------------- | ----------------------- |
| Chat Completions API  | ✅                       |
| Messages API          | ✅ (for Anthropic SDKs)  |
| Responses API         | ✅                       |
| Streaming             | ✅                       |
| Function/tool calling | ✅                       |
| Embeddings            | ✅                       |
| Async clients         | ✅                       |
| Retries               | ✅ (enhanced by gateway) |

## Gateway benefits

* **Automatic failover** - If one provider fails, the gateway tries another
* **Key rotation** - Use multiple provider API keys to avoid rate limits
* **Provider switching** - Change providers without changing code
* **Observability** - Track usage, latency, and errors across all requests

## Using different providers

Use model prefixes to route to specific providers:

<CodeGroup>
  ```python OpenAI SDK theme={null}
  # OpenAI
  client.chat.completions.create(model="openai:gpt-4o", ...)

  # Anthropic
  client.chat.completions.create(model="anthropic:claude-3-5-sonnet-latest", ...)

  # Your self-hosted Ollama
  client.chat.completions.create(model="ollama:llama3.2", ...)
  ```

  ```python Anthropic SDK theme={null}
  message = client.messages.create(
      model="anthropic:claude-3-5-sonnet-latest",
      max_tokens=1024,
      messages=[{"role": "user", "content": "Hello!"}]
  )
  ```
</CodeGroup>

Or let the gateway choose with `ngrok/auto`:

```python theme={null}
client.chat.completions.create(model="ngrok/auto", ...)
```

## Next steps

Choose your SDK guide to get started:

* [OpenAI SDK](/ai-gateway/sdks/openai) - Python, TypeScript, Go, Java, .NET
* [Vercel AI SDK](/ai-gateway/sdks/vercel-ai-sdk) - Next.js, React, streaming
* [TanStack AI](/ai-gateway/sdks/tanstack-ai) - Type-safe AI for React, Solid
* [LangChain](/ai-gateway/sdks/langchain) - Chains, agents, RAG
* [Other SDKs](/ai-gateway/sdks/other) - cURL, HTTP clients