LangChain

Prerequisite: Complete the Quickstart to get an access key. All requests go to https://gateway.ngrok.ai.

LangChain is a framework for building applications with LLMs. The ngrok AI Gateway works with LangChain’s OpenAI-compatible integrations, adding automatic failover and key rotation.

Installation

pip install langchain langchain-openai

npm install langchain @langchain/openai

Basic usage

Configure LangChain to use your AI Gateway. Pass your access key (ng-xxxxx-g1-xxxxx) as the API key—the AI Gateway injects upstream provider credentials from your access key configuration.

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    base_url="https://gateway.ngrok.ai/v1",
    api_key="ng-xxxxx-g1-xxxxx",  # Your access key
    model="gpt-4o"
)

response = llm.invoke("What is the capital of France?")
print(response.content)

import { ChatOpenAI } from "@langchain/openai";

const llm = new ChatOpenAI({
  configuration: {
    baseURL: "https://gateway.ngrok.ai/v1",
  },
  apiKey: "ng-xxxxx-g1-xxxxx",  // Your access key
  model: "gpt-4o",
});

const response = await llm.invoke("What is the capital of France?");
console.log(response.content);

Streaming

Stream responses for real-time output:

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    base_url="https://gateway.ngrok.ai/v1",
    api_key="ng-xxxxx-g1-xxxxx",  # Your access key
    model="gpt-4o"
)

for chunk in llm.stream("Write a poem about AI"):
    print(chunk.content, end="", flush=True)

Using different providers

Route to different providers with model prefixes:

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    base_url="https://gateway.ngrok.ai/v1",
    api_key="ng-xxxxx-g1-xxxxx",  # Your access key
    model="anthropic:claude-3-5-sonnet-latest"  # Use Anthropic through gateway
)

response = llm.invoke("Hello!")

Chains

Build chains that route through the gateway:

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate

llm = ChatOpenAI(
    base_url="https://gateway.ngrok.ai/v1",
    api_key="ng-xxxxx-g1-xxxxx",  # Your access key
    model="gpt-4o"
)

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful assistant that translates {input_language} to {output_language}."),
    ("human", "{input}")
])

chain = prompt | llm

response = chain.invoke({
    "input_language": "English",
    "output_language": "French",
    "input": "Hello, how are you?"
})

print(response.content)

Embeddings

Generate embeddings through the gateway:

from langchain_openai import OpenAIEmbeddings

embeddings = OpenAIEmbeddings(
    base_url="https://gateway.ngrok.ai/v1",
    api_key="ng-xxxxx-g1-xxxxx",  # Your access key
    model="text-embedding-3-small"
)

vector = embeddings.embed_query("Hello world")

RAG applications

Build RAG applications with automatic failover:

from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain_community.vectorstores import FAISS
from langchain_core.prompts import ChatPromptTemplate

llm = ChatOpenAI(
    base_url="https://gateway.ngrok.ai/v1",
    api_key="ng-xxxxx-g1-xxxxx",  # Your access key
    model="gpt-4o"
)

embeddings = OpenAIEmbeddings(
    base_url="https://gateway.ngrok.ai/v1",
    api_key="ng-xxxxx-g1-xxxxx"  # Your access key
)

# Create vector store
texts = ["Document 1 content", "Document 2 content"]
vectorstore = FAISS.from_texts(texts, embeddings)

# Query with RAG
retriever = vectorstore.as_retriever()
docs = retriever.invoke("query")

prompt = ChatPromptTemplate.from_template(
    "Answer based on context: {context}\n\nQuestion: {question}"
)

chain = prompt | llm
response = chain.invoke({"context": docs, "question": "Your question"})

Agents

Build agents that use the gateway:

from langchain_openai import ChatOpenAI
from langchain.agents import create_react_agent, AgentExecutor
from langchain_core.tools import tool

llm = ChatOpenAI(
    base_url="https://gateway.ngrok.ai/v1",
    api_key="ng-xxxxx-g1-xxxxx",  # Your access key
    model="gpt-4o"
)

@tool
def search(query: str) -> str:
    """Search for information."""
    return f"Results for: {query}"

# Create and run agent
# ...

Next steps

LangChain Documentation - Full LangChain reference
Choose a model: Model IDs and provider prefixes
Configure fallback models: Try another model when the first one fails
Choose how to reach providers: Credits, provider keys, and self-run models

Concepts

Getting Started With

SDKs

Guides

Observability

Examples

Reference

API Reference

Installation

Basic usage

Streaming

Using different providers

Chains

Embeddings

RAG applications

Agents

Next steps

​Installation

​Basic usage

​Streaming

​Using different providers

​Chains

​Embeddings

​RAG applications

​Agents

​Next steps

Installation

Basic usage

Streaming

Using different providers

Chains

Embeddings

RAG applications

Agents

Next steps