Vercel AI SDK

Prerequisite: Complete the Quickstart to get an access key. All requests go to https://gateway.ngrok.ai.

The Vercel AI SDK provides a unified interface for building AI applications. It works with the ngrok AI Gateway, adding provider failover, key rotation, and observability to Vercel’s streaming and UI components.

Installation

npm install ai @ai-sdk/openai

Basic usage

Point the OpenAI provider at your AI Gateway endpoint:

import { generateText } from "ai";
import { createOpenAI } from "@ai-sdk/openai";

const openai = createOpenAI({
  baseURL: "https://gateway.ngrok.ai/v1",
  apiKey: "ng-xxxxx-g1-xxxxx",  // Your access key
});

const { text } = await generateText({
  model: openai("gpt-4o"),
  prompt: "What is the meaning of life?",
});

import { generateText } from "ai";
import { createOpenAI } from "@ai-sdk/openai";

const openai = createOpenAI({
  baseURL: "https://gateway.ngrok.ai/v1",
  apiKey: "ng-xxxxx-g1-xxxxx",  // Your access key
});

export const runtime = "edge";

export async function POST(req: Request) {
  const { prompt } = await req.json();
  
  const { text } = await generateText({
    model: openai("gpt-4o"),
    prompt,
  });
  
  return Response.json({ text });
}

Streaming responses

The AI Gateway supports streaming. Use streamText for real-time responses:

import { streamText } from "ai";
import { createOpenAI } from "@ai-sdk/openai";

const openai = createOpenAI({
  baseURL: "https://gateway.ngrok.ai/v1",
  apiKey: "ng-xxxxx-g1-xxxxx",  // Your access key
});

const result = streamText({
  model: openai("gpt-4o"),
  prompt: "Write a poem about AI gateways.",
});

for await (const textPart of result.textStream) {
  process.stdout.write(textPart);
}

Chat interface

Build chat applications with the useChat hook:

app/page.tsx

"use client";

import { useChat } from "ai/react";

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit } = useChat({
    api: "/api/chat",
  });

  return (
    <div>
      {messages.map((m) => (
        <div key={m.id}>
          {m.role}: {m.content}
        </div>
      ))}
      <form onSubmit={handleSubmit}>
        <input value={input} onChange={handleInputChange} />
        <button type="submit">Send</button>
      </form>
    </div>
  );
}

app/api/chat/route.ts

import { streamText } from "ai";
import { createOpenAI } from "@ai-sdk/openai";

const openai = createOpenAI({
  baseURL: "https://gateway.ngrok.ai/v1",
  apiKey: "ng-xxxxx-g1-xxxxx",  // Your access key
});

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: openai("gpt-4o"),
    messages,
  });

  return result.toDataStreamResponse();
}

Using different providers

The AI Gateway routes based on the model name. Use provider prefixes to be explicit:

import { generateText } from "ai";
import { createOpenAI } from "@ai-sdk/openai";

const gateway = createOpenAI({
  baseURL: "https://gateway.ngrok.ai/v1",
  apiKey: "ng-xxxxx-g1-xxxxx",  // Your access key
});

// Use different providers through the same gateway
const openaiResult = await generateText({ model: gateway("openai:gpt-4o"), prompt: "Hello" });
const anthropicResult = await generateText({ model: gateway("anthropic:claude-3-5-sonnet-latest"), prompt: "Hello" });

Model failover

List fallback models in the request. See Configure fallback models.

Environment variables

Set up your environment:

.env.local

AI_GATEWAY_URL=https://gateway.ngrok.ai/v1
AI_GATEWAY_ACCESS_KEY=ng-xxxxx-g1-xxxxx

const openai = createOpenAI({
  baseURL: process.env.AI_GATEWAY_URL,
  apiKey: process.env.AI_GATEWAY_ACCESS_KEY!,
});

Tool calling

The AI Gateway supports function/tool calling:

import { generateText, tool } from "ai";
import { createOpenAI } from "@ai-sdk/openai";
import { z } from "zod";

const openai = createOpenAI({
  baseURL: "https://gateway.ngrok.ai/v1",
  apiKey: "ng-xxxxx-g1-xxxxx",  // Your access key
});

const { text, toolCalls } = await generateText({
  model: openai("gpt-4o"),
  tools: {
    weather: tool({
      description: "Get the weather in a location",
      parameters: z.object({
        location: z.string().describe("The location to get weather for"),
      }),
      execute: async ({ location }) => {
        return { temperature: 72, condition: "sunny" };
      },
    }),
  },
  prompt: "What's the weather in San Francisco?",
});

Error handling

Handle errors gracefully:

import { generateText, APICallError } from "ai";

try {
  const { text } = await generateText({
    model: openai("gpt-4o"),
    prompt: "Hello",
  });
} catch (error) {
  if (error instanceof APICallError) {
    console.error("API Error:", error.message);
    console.error("Status:", error.statusCode);
  }
}

With the AI Gateway’s failover, many errors are handled automatically by retrying with different providers or keys.

Next steps

Vercel AI SDK Documentation - Full SDK reference
Choose a model: Model IDs and provider prefixes
Configure fallback models: Try another model when the first one fails
Choose how to reach providers: Credits, provider keys, and self-run models

Concepts

Getting Started With

SDKs

Guides

Observability

Examples

Reference

API Reference

Installation

Basic usage

Streaming responses

Chat interface

Using different providers

Model failover

Environment variables

Tool calling

Error handling

Next steps

​Installation

​Basic usage

​Streaming responses

​Chat interface

​Using different providers

​Model failover

​Environment variables

​Tool calling

​Error handling

​Next steps

Installation

Basic usage

Streaming responses

Chat interface

Using different providers

Model failover

Environment variables

Tool calling

Error handling

Next steps