Vercel AI SDK - Runpod Documentation

The @runpod/ai-sdk-provider package integrates Runpod Public Endpoints with the Vercel AI SDK. This gives you a streamlined, type-safe interface for text generation, streaming, image generation, and video generation in JavaScript and TypeScript projects. The Vercel AI SDK is a popular open-source library for building AI applications. By using the Runpod provider, you can access Runpod’s Public Endpoints using the same patterns and APIs you’d use with other AI providers like OpenAI or Anthropic.

Why use the Vercel AI SDK?

Unified interface: Use the same generateText, streamText, generateImage, and generateVideo functions regardless of which AI provider you’re using.
Type safety: Full TypeScript support with typed responses and parameters.
Streaming built-in: First-class support for streaming text responses.
Framework integrations: Works seamlessly with Next.js, React, Svelte, and other frameworks.
Provider switching: Easily switch between Runpod and other providers without rewriting your code.

Installation

Install the Runpod provider alongside the Vercel AI SDK:

npm install @runpod/ai-sdk-provider ai

Configuration

Default configuration

The provider reads your API key from the RUNPOD_API_KEY environment variable by default. Import the runpod instance and start using it immediately:

import { runpod } from "@runpod/ai-sdk-provider";

Set the environment variable in your shell or .env file:

export RUNPOD_API_KEY="YOUR_API_KEY"

Custom configuration

For more control, use createRunpod to create a custom provider instance:

import { createRunpod } from "@runpod/ai-sdk-provider";

const runpod = createRunpod({
  apiKey: "YOUR_API_KEY",
  baseURL: "https://api.runpod.ai/v2",
  headers: {
    "X-Custom-Header": "value",
  },
});

Option	Description	Default
`apiKey`	Your Runpod API key	`RUNPOD_API_KEY` env var
`baseURL`	Base URL for API requests	`https://api.runpod.ai/v2`
`headers`	Custom HTTP headers to include with requests	`{}`

Using custom endpoints

You can use your own Serverless endpoints with the AI SDK. This is useful when you’ve deployed a custom model or want to use a specific endpoint you’ve created.

Using endpoint IDs

Pass your Serverless endpoint ID directly as the model identifier:

import { runpod } from "@runpod/ai-sdk-provider";
import { generateText, experimental_generateImage as generateImage } from "ai";

// Use a custom chat endpoint
const { text } = await generateText({
  model: runpod("your-endpoint-id"),
  prompt: "Hello, how are you?",
});

// Use a custom image endpoint
const { image } = await generateImage({
  model: runpod.image("your-image-endpoint-id"),
  prompt: "A beautiful sunset",
});

The SDK resolves your endpoint ID to https://api.runpod.ai/v2/{endpointId} automatically.

Using Console URLs

Copy an endpoint URL directly from the Runpod Console and use it as the model identifier:

import { runpod } from "@runpod/ai-sdk-provider";
import { experimental_generateImage as generateImage } from "ai";

const { image } = await generateImage({
  model: runpod.image("https://console.runpod.io/serverless/user/endpoint/abc123xyz"),
  prompt: "A serene mountain landscape",
});

The SDK extracts the endpoint ID from the Console URL and routes requests to your endpoint.

Text generation

Basic text generation

Use generateText to generate text from a prompt:

import { runpod } from "@runpod/ai-sdk-provider";
import { generateText } from "ai";

const { text, finishReason, usage } = await generateText({
  model: runpod("qwen3-32b-awq"),
  prompt: "Write a Python function that checks if a number is prime:",
});

console.log(text);
console.log(`Tokens used: ${usage.totalTokens}`);

The response includes:

text: The generated text
finishReason: Why generation stopped (stop, length, etc.)
usage: Token counts (promptTokens, completionTokens, totalTokens)

Chat conversations

For multi-turn conversations, pass a messages array instead of a prompt:

import { runpod } from "@runpod/ai-sdk-provider";
import { generateText } from "ai";

const { text } = await generateText({
  model: runpod("qwen3-32b-awq"),
  messages: [
    {
      role: "system",
      content: "You are a helpful coding assistant. Be concise.",
    },
    {
      role: "user",
      content: "How do I read a JSON file in Python?",
    },
  ],
});

console.log(text);

Generation parameters

Control the generation behavior with additional parameters:

const { text } = await generateText({
  model: runpod("qwen3-32b-awq"),
  prompt: "Write a creative story about a robot:",
  temperature: 0.8, // Higher = more creative (0-1)
  maxTokens: 500, // Maximum tokens to generate
  topP: 0.9, // Nucleus sampling threshold
});

Streaming

For real-time output (useful for chat interfaces), use streamText:

import { runpod } from "@runpod/ai-sdk-provider";
import { streamText } from "ai";

const { textStream } = await streamText({
  model: runpod("qwen3-32b-awq"),
  prompt: "Explain quantum computing in simple terms:",
  temperature: 0.7,
});

for await (const chunk of textStream) {
  process.stdout.write(chunk);
}

Streaming with callbacks

You can also use callbacks to handle streaming events:

import { runpod } from "@runpod/ai-sdk-provider";
import { streamText } from "ai";

const result = await streamText({
  model: runpod("qwen3-32b-awq"),
  prompt: "Write a poem about the ocean:",
  onChunk: ({ chunk }) => {
    if (chunk.type === "text-delta") {
      process.stdout.write(chunk.textDelta);
    }
  },
  onFinish: ({ text, usage }) => {
    console.log(`\n\nTotal tokens: ${usage.totalTokens}`);
  },
});

Image generation

Text-to-image

Generate images using models like Flux:

import { runpod } from "@runpod/ai-sdk-provider";
import { experimental_generateImage as generateImage } from "ai";
import { writeFileSync } from "fs";

const { image } = await generateImage({
  model: runpod.image("black-forest-labs-flux-1-dev"),
  prompt: "A serene mountain landscape at sunset, photorealistic",
  aspectRatio: "16:9",
});

// Save the image to a file
writeFileSync("output.png", image.uint8Array);

// Or access as base64
console.log(image.base64);

The response includes:

image.uint8Array: Binary image data
image.base64: Base64-encoded image
image.mimeType: Image MIME type (e.g., image/png)

Image editing

Edit existing images by providing reference images:

import { runpod } from "@runpod/ai-sdk-provider";
import { experimental_generateImage as generateImage } from "ai";

const { image } = await generateImage({
  model: runpod.image("google-nano-banana-edit"),
  prompt: {
    text: "Add modern Scandinavian furniture to this room",
    images: ["https://example.com/empty-room.png"],
  },
  aspectRatio: "16:9",
});

For models that support multiple reference images:

const { image } = await generateImage({
  model: runpod.image("google-nano-banana-edit"),
  prompt: {
    text: "Combine these into an epic band photo",
    images: [
      "https://example.com/drummer.png",
      "https://example.com/guitarist.png",
      "https://example.com/bassist.png",
      "https://example.com/singer.png",
    ],
  },
});

Provider options

Pass model-specific parameters using providerOptions:

const { image } = await generateImage({
  model: runpod.image("black-forest-labs-flux-1-dev"),
  prompt: "A sunset over the ocean",
  providerOptions: {
    runpod: {
      negative_prompt: "blurry, low quality, distorted",
      num_inference_steps: 30,
      guidance: 7.5,
      seed: 42,
      enable_safety_checker: true,
    },
  },
});

Option	Description
`negative_prompt`	Elements to exclude from the image
`num_inference_steps`	Number of denoising steps (higher = more detail)
`guidance`	How closely to follow the prompt (0-10)
`seed`	Seed for reproducible results (-1 for random)
`enable_safety_checker`	Enable content safety filtering
`maxPollAttempts`	Max polling attempts for async generation
`pollIntervalMillis`	Milliseconds between status polls

Video generation

Use experimental_generateVideo to generate videos from text prompts or images. The Runpod provider supports 15 video models, including Sora, Wan, Seedance, and Kling. Video generation is asynchronous—the SDK submits a job, polls for completion, and returns the video URL when ready.

Text-to-video

Generate videos from text prompts:

import { runpod } from "@runpod/ai-sdk-provider";
import { experimental_generateVideo as generateVideo } from "ai";

const { video } = await generateVideo({
  model: runpod.video("alibaba/wan-2.6-t2v"),
  prompt: "A golden retriever running on a sunny beach, cinematic, 4k",
});

console.log(video.url);

The response includes:

video.url: URL to the generated video
video.mediaType: Video MIME type (video/mp4)

Image-to-video

Animate an existing image:

import { runpod } from "@runpod/ai-sdk-provider";
import { experimental_generateVideo as generateVideo } from "ai";

const { video } = await generateVideo({
  model: runpod.video("alibaba/wan-2.6-i2v"),
  prompt: "Animate this scene with gentle camera movement",
  image: new URL("https://example.com/image.png"),
});

console.log(video.url);

Video generation parameters

Control the video generation with additional parameters:

const { video } = await generateVideo({
  model: runpod.video("alibaba/wan-2.6-t2v"),
  prompt: "A serene mountain landscape with flowing water",
  duration: 5,
  aspectRatio: "16:9",
  seed: 42,
});

Video provider options

Pass model-specific parameters using providerOptions:

const { video } = await generateVideo({
  model: runpod.video("alibaba/wan-2.6-t2v"),
  prompt: "A serene mountain landscape with flowing water",
  duration: 5,
  aspectRatio: "16:9",
  providerOptions: {
    runpod: {
      negative_prompt: "blurry, low quality",
      guidance_scale: 7.5,
    },
  },
});

Option	Description
`negative_prompt`	Elements to exclude from the video
`guidance_scale`	How closely to follow the prompt
`num_inference_steps`	Number of inference steps
`style`	Style preset (model-specific)
`maxPollAttempts`	Max polling attempts (default: 120)
`pollIntervalMillis`	Milliseconds between status polls (default: 5000)

Supported models

Text models

Model ID	Description
`qwen3-32b-awq`	Qwen3 32B with AWQ quantization. Good for general text and code generation.
`gpt-oss-120b`	GPT OSS 120B. Supports tool calling.

Image models

Model ID	Description
`black-forest-labs-flux-1-dev`	Flux Dev. High quality, detailed images.
`black-forest-labs-flux-1-schnell`	Flux Schnell. Fast generation, good for prototyping.
`google-nano-banana-edit`	Nano Banana Edit. Supports multiple reference images.
`google/nano-banana-2-edit`	Nano Banana 2 Edit. Image editing with 14 aspect ratios and resolution options (1k/2k/4k).
`bytedance-seedream-4-0-t2i`	Seedream 4.0. Text-to-image with good prompt adherence.
`tongyi-mai/z-image-turbo`	Z-Image Turbo. Fast 6B parameter model with text-to-image support.

Video models

Model ID	Type	Resolution	Aspect Ratios	Duration
`pruna/p-video`	t2v	720p, 1080p	16:9, 9:16	5s
`vidu/q3-t2v`	t2v	720p, 1080p	16:9, 9:16, 1:1	5, 10s
`vidu/q3-i2v`	i2v	720p, 1080p	16:9, 9:16, 1:1	5, 10s
`kwaivgi/kling-v2.6-std-motion-control`	i2v + video	720p	16:9, 9:16, 1:1	5, 10s
`kwaivgi/kling-video-o1-r2v`	i2v	720p	16:9, 9:16, 1:1	3–10s
`kwaivgi/kling-v2.1-i2v-pro`	i2v	720p	16:9, 9:16, 1:1	5, 10s
`alibaba/wan-2.6-t2v`	t2v	720p, 1080p	16:9, 9:16	5, 10, 15s
`alibaba/wan-2.6-i2v`	i2v	720p, 1080p	16:9, 9:16	5, 10, 15s
`alibaba/wan-2.5`	i2v	480p, 720p, 1080p	16:9, 9:16	5, 10s
`alibaba/wan-2.2-t2v-720-lora`	i2v	720p	16:9	5, 8s
`alibaba/wan-2.2-i2v-720`	i2v	720p	16:9	5, 8s
`alibaba/wan-2.1-i2v-720`	i2v	720p	16:9	5s
`bytedance/seedance-v1.5-pro-i2v`	i2v	480p, 720p	21:9, 16:9, 9:16, 1:1, 4:3, 3:4	4–12s
`openai/sora-2-pro-i2v`	i2v	720p, 1080p	16:9, 9:16, 1:1	4, 8, 12s
`openai/sora-2-i2v`	i2v	720p, 1080p	16:9, 9:16, 1:1	4, 8, 12s

For a complete list of available models and their parameters, see the model reference.

Example: Chat application

Here’s a complete example of a simple chat application using streaming:

import { runpod } from "@runpod/ai-sdk-provider";
import { streamText } from "ai";
import * as readline from "readline";

const rl = readline.createInterface({
  input: process.stdin,
  output: process.stdout,
});

const messages: { role: "user" | "assistant"; content: string }[] = [];

async function chat(userMessage: string) {
  messages.push({ role: "user", content: userMessage });

  const { textStream } = await streamText({
    model: runpod("qwen3-32b-awq"),
    system: "You are a helpful assistant.",
    messages,
  });

  let assistantMessage = "";
  process.stdout.write("\nAssistant: ");

  for await (const chunk of textStream) {
    process.stdout.write(chunk);
    assistantMessage += chunk;
  }

  messages.push({ role: "assistant", content: assistantMessage });
  console.log("\n");
}

function prompt() {
  rl.question("You: ", async (input) => {
    if (input.toLowerCase() === "exit") {
      rl.close();
      return;
    }
    await chat(input);
    prompt();
  });
}

console.log('Chat started. Type "exit" to quit.\n');
prompt();

Next steps

Model reference: View all available models and their parameters.
Make API requests: Learn about the REST API for lower-level control.
@runpod/ai-sdk-provider on GitHub: View the source code and contribute.
Vercel AI SDK documentation: Learn more about the AI SDK.

​Why use the Vercel AI SDK?

​Installation

​Configuration

​Default configuration

​Custom configuration

​Using custom endpoints

​Using endpoint IDs

​Using Console URLs

​Text generation

​Basic text generation

​Chat conversations

​Generation parameters

​Streaming

​Streaming with callbacks

​Image generation

​Text-to-image

​Image editing

​Provider options

​Video generation

​Text-to-video

​Image-to-video

​Video generation parameters

​Video provider options

​Supported models

​Text models

​Image models

​Video models

​Example: Chat application

​Next steps

Why use the Vercel AI SDK?

Installation

Configuration

Default configuration

Custom configuration

Using custom endpoints

Using endpoint IDs

Using Console URLs

Text generation

Basic text generation

Chat conversations

Generation parameters

Streaming

Streaming with callbacks

Image generation

Text-to-image

Image editing

Provider options

Video generation

Text-to-video

Image-to-video

Video generation parameters

Video provider options

Supported models

Text models

Image models

Video models

Example: Chat application

Next steps