Skip to main content

Documentation Index

Fetch the complete documentation index at: https://ngrok.com/docs/llms.txt

Use this file to discover all available pages before exploring further.

The AI Gateway maintains a catalog of known providers and models. This catalog enables automatic provider inference, model validation, and rich metadata for selection strategies.

​
How the catalog works

When you send a request with a model name like gpt-4o, the gateway:
  1. Looks up the model in the catalog
  2. Identifies the provider (OpenAI)
  3. Routes the request to the appropriate provider endpoint and injects the API key
  4. Applies any configured selection strategies
You can also explicitly specify providers using the provider:model format (for example, openai:gpt-4o).

​
Providers

OpenAI and Anthropic are available with AI Gateway API Keys—no provider account needed. All other providers require you to bring your own key.

​
OpenAI

FieldValue
Provider IDopenai
AliasesopenAI, open-ai, open-AI
Base URLhttps://api.openai.com/v1/
Websiteopenai.com
BYOK RequiredNo
How to use OpenAI →

​
OpenAI models

Model IDDisplay NameContext WindowOutput TokensModalities
gpt-5.4-proGPT-5.4 Pro1,050,000128,000text, image
gpt-5.4GPT-5.41,050,000128,000text, image
gpt-5.3-codexGPT-5.3-Codex400,000128,000text, image
gpt-5.2-codexGPT-5.2-Codex400,000128,000text, image
gpt-5.2-proGPT-5.2 Pro400,000100,000text, image
gpt-5.2GPT-5.2400,000128,000text, image
gpt-5.2-chat-latestGPT-5.2 Chat Latest400,000128,000text, image
gpt-5.1GPT-5.1400,000128,000text, image
gpt-5.1-chat-latestGPT-5.1 Chat Latest256,00032,768text, image
gpt-5GPT-5400,000128,000text, image
gpt-5-miniGPT-5 Mini400,000128,000text, image
gpt-5-nanoGPT-5 Nano400,000128,000text, image
gpt-5-chat-latestGPT-5 Chat400,000128,000text, image
gpt-4.1GPT-4.11,000,000-text, image
gpt-4.1-miniGPT-4.1 mini1,000,000-text, image
gpt-4.1-nanoGPT-4.1 nano1,000,000-text, image
gpt-4oGPT-4o128,00016,384text, image, audio
gpt-4o-miniGPT-4o Mini128,00016,384text, image
o4-miniO4-Mini200,000100,000text
o4-mini-deep-researchO4-Mini-Deep-Research200,000100,000text
o3-proO3-Pro128,000100,000text
o3O3128,000100,000text
o3-miniO3 Mini200,000100,000text
o3-deep-researchO3-Deep-Research200,000100,000text
o1-proO1-Pro200,000100,000text
o1O1128,000100,000text
gpt-4-turboGPT-4 Turbo128,0004,096text, image
gpt-4GPT-48,1928,192text
gpt-3.5-turboGPT-3.5 Turbo (deprecated, retires September 28, 2026)16,3854,096text

​
Anthropic

FieldValue
Provider IDanthropic
AliasesAnthropic
Base URLhttps://api.anthropic.com/v1/
Websiteanthropic.com
BYOK RequiredNo
How to use Anthropic →

​
Anthropic models

Model IDDisplay NameContext WindowOutput TokensModalities
claude-opus-4-6Claude Opus 4.61,000,000128,000text, image
claude-sonnet-4-6Claude Sonnet 4.61,000,00064,000text, image
claude-haiku-4-5Claude Haiku 4.5200,00064,000text, image
claude-sonnet-4-5Claude Sonnet 4.51,000,00064,000text, image
claude-opus-4-5Claude Opus 4.5200,00064,000text, image
claude-opus-4-1Claude Opus 4.1200,00032,000text, image
claude-sonnet-4-0Claude Sonnet 41,000,00064,000text, image
claude-opus-4-0Claude Opus 4200,00032,000text, image
claude-3-haiku-20240307Claude Haiku 3 (deprecated, retires April 20, 2026)200,0004,096text, image

​
Google

FieldValue
Provider IDgoogle
AliasesGoogle, gemini
Base URLhttps://generativelanguage.googleapis.com/v1beta/openai/
Websiteaistudio.google.com
BYOK RequiredYes
How to use Google →

​
Google models

Model IDDisplay NameContext WindowOutput TokensModalities
gemini-2.5-proGemini 2.5 Pro1,048,57665,535text, image, audio, video, file
gemini-2.5-flashGemini 2.5 Flash1,048,57665,535text, image, audio, video, file
gemini-2.5-flash-liteGemini 2.5 Flash-Lite1,048,57665,535text, image, audio, video, file
gemini-2.0-flashGemini 2.0 Flash1,048,5768,192text, image, audio, video, file
gemini-2.0-flash-liteGemini 2.0 Flash-Lite1,048,5768,192text, image, audio, video, file
gemini-3-pro-previewGemini 3 Pro Preview1,000,00065,536text, image, audio, video, file

​
DeepSeek

FieldValue
Provider IDdeepseek
AliasesDeepSeek, deep-seek
Base URLhttps://api.deepseek.com
Websitedeepseek.com
BYOK RequiredYes
How to use DeepSeek →

​
DeepSeek models

Model IDDisplay NameContext WindowOutput TokensModalities
deepseek-reasonerdeepseek-reasoner128,00064,000text
deepseek-chatdeepseek-chat128,0008,192text

​
Groq

FieldValue
Provider IDgroq
Base URLhttps://api.groq.com/openai/v1
Websitegroq.com
BYOK RequiredYes
How to use Groq → Groq provides AI inference powered by their custom LPU (Language Processing Unit) hardware.

​
Groq models

Model IDDisplay NameContext WindowOutput TokensModalities
meta-llama/llama-3.1-8b-instantLlama 3.1 8B Instant131,072131,072text
meta-llama/llama-3.3-70b-versatileLlama 3.3 70B Versatile131,07232,768text
meta-llama/llama-prompt-guard-2-22mLlama Prompt Guard 2 22M512512text
meta-llama/llama-prompt-guard-2-86mLlama Prompt Guard 2 86M512512text
meta-llama/llama-guard-4-12bLlama Guard 4 12B131,0721,024text, image
meta-llama/llama-4-maverick-17b-128e-instructLlama 4 Maverick 17B 128E Instruct131,0728,192text, image
meta-llama/llama-4-scout-17b-16e-instructLlama 4 Scout 17B 16E Instruct131,0728,192text, image
moonshotai/kimi-k2-instruct-0905Kimi K2262,14416,384text
openai/gpt-oss-120bGPT OSS 120B131,072131,072text
openai/gpt-oss-20bGPT OSS 20B131,072131,072text
openai/gpt-oss-safeguard-20bSafety GPT OSS 20B131,07265,536text
qwen/qwen3-32bQwen3-32B131,07240,960text

​
OpenRouter

FieldValue
Provider IDopenrouter
Base URLhttps://openrouter.ai/api/v1/
Websiteopenrouter.ai
BYOK RequiredYes
How to use OpenRouter → OpenRouter is a unified API that provides access to multiple AI models from various providers through a single endpoint.

​
Hyperbolic

FieldValue
Provider IDhyperbolic
Base URLhttps://api.hyperbolic.xyz/v1/
Websitehyperbolic.xyz
BYOK RequiredYes
How to use Hyperbolic → Hyperbolic provides high-performance inference for open-source models.

​
InceptionLabs

FieldValue
Provider IDinceptionlabs
Websiteinceptionlabs.ai
BYOK RequiredYes
How to use InceptionLabs → InceptionLabs develops diffusion-based language models for fast, efficient text generation.

​
Inference.net

FieldValue
Provider IDinference-net
Base URLhttps://api.inference.net/v1/
Websiteinference.net
BYOK RequiredYes
How to use Inference.net → Inference.net provides a distributed inference network for running AI models at scale.

​
Using models from the catalog

​
Simple model reference

Reference models directly by their ID: 
{
  "model": "gpt-4o",
  "messages": [{"role": "user", "content": "Hello"}]
}

​
Explicit provider

Use the provider:model format for explicit routing: 
{
  "model": "openai:gpt-4o",
  "messages": [{"role": "user", "content": "Hello"}]
}

​
Custom providers and models

You can extend the catalog by configuring custom providers in your Traffic Policy. See Custom Providers for configuration details. 
providers:
  - id: "custom-ollama"
    base_url: "https://ollama.internal"
    models:
      - id: "llama3"
      - id: "mistral"

​
Catalog updates

The model catalog is updated periodically to include new models and providers. For immediate access to models not yet in the catalog, add them explicitly to your provider configuration.

​
Aliases reference

Model and provider names are not case-sensitive. For example, gpt-4o, GPT-4o, and Gpt-4O all resolve to the same model. The following aliases are available in addition to the primary IDs listed above.

​
Provider aliases

Provider IDAliases
openaiopenAI, open-ai, open-AI
anthropicAnthropic
googlegemini
deepseekdeep-seek
openrouteropen-router
inceptionlabsinception-labs, inception
inference.netinference-net, inference_net
groqgroqcloud

​
OpenAI model aliases

AliasResolves to
gpt-4-omnigpt-4o
gpt-4o-2024-05-13gpt-4o
gpt-4o-2024-08-06gpt-4o
gpt-4o-2024-11-20gpt-4o
chatgpt-4o-latestgpt-4o
gpt-4o-mini-2024-07-18gpt-4o-mini
gpt-4-turbo-2024-04-09gpt-4-turbo
gpt-4-turbo-previewgpt-4-turbo
gpt-4-1106-previewgpt-4-turbo
gpt-4-0125-previewgpt-4-turbo
gpt-4-0613gpt-4
gpt-4-0314gpt-4
gpt-4-32kgpt-4
gpt-4.1-2025-04-14gpt-4.1
gpt-4.1-mini-2025-04-14gpt-4.1-mini
gpt-4.1-nano-2025-04-14gpt-4.1-nano
gpt-5.1-2025-11-13gpt-5.1
gpt-5.4-2026-03-05gpt-5.4
gpt-5.4-pro-2026-03-05gpt-5.4-pro
gpt-5.2-pro-2025-12-11gpt-5.2-pro
GPT-5.3-Codexgpt-5.3-codex
gpt-5.3-Codexgpt-5.3-codex
gpt5.3-codexgpt-5.3-codex
GPT-5 minigpt-5-mini
GPT-5 nanogpt-5-nano
GPT-5 Chatgpt-5-chat-latest
gpt-3.5-turbo-0125gpt-3.5-turbo
gpt-3.5-turbo-16kgpt-3.5-turbo
o4-mini-2025-04-16o4-mini
o4-mini-deep-research-2025-06-26o4-mini-deep-research
o3-pro-2025-06-10o3-pro
o3-2025-04-16o3
o3-mini-2025-01-31o3-mini

​
Anthropic model aliases

AliasResolves to
claude-opus-4-6-20260205claude-opus-4-6
claude-sonnet-4-6-20260217claude-sonnet-4-6
claude-haiku-4-5-20251001claude-haiku-4-5
claude-sonnet-4-5-20250929claude-sonnet-4-5
claude-opus-4-5-20251101claude-opus-4-5
claude-opus-4-1-20250805claude-opus-4-1
claude-opus-4.1claude-opus-4-1
claude-sonnet-4-20250514claude-sonnet-4-0
claude-sonnet-4claude-sonnet-4-0
claude-opus-4-20250514claude-opus-4-0
claude-opus-4claude-opus-4-0
claude-haiku-3claude-3-haiku-20240307

​
Google model aliases

AliasResolves to
gemini-3gemini-3-pro-preview
gemini-3-progemini-3-pro-preview

​
Meta model aliases

AliasResolves to
llama-3.1-8bllama-3.1-8b-instant
llama-3.3-70bllama-3.3-70b-versatile
llama-4-maverick-17b-128ellama-4-maverick-17b-128e-instruct
llama-4-scout-17b-16ellama-4-scout-17b-16e-instruct

​
Moonshot AI model aliases

AliasResolves to
kimi-k2-instructkimi-k2
kimi-k2-instruct-0905kimi-k2
moonshotai/kimi-k2-instruct-0905kimi-k2