Model fallbacks now available in Vercel AI Gateway

Vercel's AI Gateway now supports fallback models for when models fail or are unavailable. In addition to safeguarding against provider-level failures, model fallbacks can help with errors and capability mismatches between models (e.g., multimodal, tool-calling, etc.).

Fallback models will be tried in the specified order until a request succeeds or no options remain. Any error, such as context limits, unsupported inputs, or provider outages, can trigger a fallback. Requests are billed based on the model that completes successfully.

This example shows an instance where the primary model does not support multimodal capabilities, falling back to models that do. To use, specify the model fallbacks in models within providerOptions:

1
import { streamText } from 'ai';
2
const result = streamText({
3
  model: 'openai/gpt-oss-120b', // Primary model
4
  prompt: 'Parse the attached PDF for tables and graphs, \
5
  and return the highest performing categories this year',
6
  providerOptions: {
7
    gateway: {
8
      models: [
9
        'google/gemini-2.5-pro',
10
        'anthropic/claude-sonnet-4.5',
11
        'meta/llama-3.1-8b'
12
      ], // Fallback models
13
    },
14
  },
15
})

To have pre-defined provider routing in addition to model routing, specify both models and providers (order or only) in providerOptions:

1
import { streamText } from 'ai';
2
const result = streamText({
3
  model: 'openai/gpt-5-nano', // Primary model
4
  prompt: 'Parse the attached PDF for tables and graphs, \
5
  and return the highest performing categories this year',
6
  providerOptions: {
7
    gateway: {
8
      order: ['vertex', 'cerebras'], // Provider routing order
9
      models: [
10
        'google/gemini-2.5-flash',
11
        'openai/gpt-oss-120b'
12
      ], // Fallback models
13
    },
14
  },
15
})

AI Gateway also includes built-in observability, Bring Your Own Key support, and supports OpenAI-compatible API.