How Do I Survive a Provider Outage?

Taco Tuesday cannot wait for us to write fallback code during lunch. We need to choose the provider and model sequence while every service is healthy.

We will configure provider failover and model fallback while every service is healthy.

These controls cover two failure levels. Provider failover keeps the same model and tries another host. Model fallback moves to a different model after the preferred model has no available provider.

Outcome

Configure both failover layers on the order flow, and know how to read a failover event off a response.

Fast Track

1. Add order and models under providerOptions.gateway in a call
2. Run pnpm outage-drill
3. Compare the healthy attempt log with the captured failure trace

Hands-on exercise

Because you cannot schedule a provider outage for practice, this exercise configures both fallback layers and teaches you to read attempt metadata before an incident occurs.

Requirements:

• A scripts/outage-drill.ts that calls the order-confirmation prompt on anthropic/claude-sonnet-4.6
• order: ["anthropic", "bedrock"]: prefer direct, name the first fallback
• models: ["openai/gpt-5.4-mini"]: use the previous order model if Claude is unavailable across its providers
• Walk modelAttempts and print every attempt: model, provider, success or the error

The wrong way, for contrast, is the try/catch pyramid: catch the Anthropic error, retry Bedrock by hand, catch that, swap models, each with its own SDK and error shape. That approach creates additional integration and maintenance work.

Try It

pnpm outage-drill

On a healthy run, the first provider may succeed:

anthropic/claude-sonnet-4.6 via anthropic — ok
Survived. 1 model tried, 1 provider attempt.

The example below is a previously captured and sanitized failure trace:

anthropic/claude-sonnet-4.6 via anthropic — failed: Internal error
anthropic/claude-sonnet-4.6 via bedrock — ok
Survived. 1 model tried, 2 provider attempts.

The captured trace shows a failed attempt followed by a successful retry. Do not stage a fake provider failure and present it as live output.

Two issues you may encounter:

Your model has one provider. Then order has nothing to reorder and provider failover can't save you; the models array is your entire outage plan. Check the model's page for its provider count; it changes how much you should trust layer one.

The fallback model answers in a different style. Evaluate every backup against the feature's quality, latency, tool, and output requirements. Some features should return a controlled error instead of accepting a poor fallback.

Commit

git commit -m "feat(failover): add provider order and model fallback to the order flow"

Done-When

• Both layers configured: order for providers, models for the deep failure
• The drill script prints the attempt log on a normal day
• You can explain provider failover vs model fallback in one sentence each
• You know how many providers serve your primary model

Solution

import { generateText } from "ai";
import { MENU } from "../lib/menu";
 
const result = await generateText({
  model: "anthropic/claude-sonnet-4.6",
  prompt:
    "Confirm this order back with a total: two Birria Eclipse and one " +
    `Elote Clásico.\n\n${MENU}`,
  providerOptions: {
    gateway: {
      order: ["anthropic", "bedrock"],
      models: ["openai/gpt-5.4-mini"],
    },
  },
});
 
const routing = result.finalStep.providerMetadata?.gateway?.routing as any;
const modelAttempts = routing?.modelAttempts ?? [];
let attempts = 0;
 
for (const m of modelAttempts) {
  for (const p of m.providerAttempts ?? []) {
    attempts++;
    const outcome = p.success ? "ok" : `failed: ${p.error}`;
    console.log(`${m.canonicalSlug} via ${p.provider} — ${outcome}`);
  }
}
 
console.log(
  `Survived. ${modelAttempts.length} model${modelAttempts.length === 1 ? "" : "s"} tried, ` +
  `${attempts} provider attempt${attempts === 1 ? "" : "s"}.`
);

With both fallback layers configured, the order flow can continue through a provider or model failure.

Related Questions

• How do I fail over when a provider is slow, not just down?
• How do I switch models without rewriting my app?
• How do I pin routing to a specific provider?