Will AI Gateway Work with My Existing AI Stack?

Working code has seniority. Replacing a provider SDK before the application needs anything else creates risk without much payoff.

We will start with the client's base URL.

Outcome

Point an OpenAI SDK client at the Gateway and use it to call both OpenAI and Anthropic models.

Fast Track

1. Install the existing client with pnpm add openai
2. Set baseURL: "https://ai-gateway.vercel.sh/v1" and apiKey to your Gateway key
3. Ask for anthropic/claude-sonnet-4.6 and watch the OpenAI SDK serve Claude

Hands-on exercise

The Taco Tuesday assistant already uses the OpenAI SDK. We will preserve its request shape and change the client configuration.

Requirements:

• A scripts/existing-stack.ts using the openai package, not ai
• Construct the client with the Gateway's base URL and your AI_GATEWAY_API_KEY
• First call: the model the app always used, as openai/gpt-5.4-mini (note the creator prefix; that's the one naming change)
• Second call, same client: anthropic/claude-sonnet-4.6

The OpenAI SDK sends an OpenAI-compatible request to the Gateway endpoint. The Gateway can route that request to Claude without changing the client interface.

Try It

pnpm existing-stack

[openai/gpt-5.4-mini via OpenAI SDK]
Order up: two Al Pastor Meteors, one Agua Fresca. That'll be $12.00.

[anthropic/claude-sonnet-4.6 via the same OpenAI SDK]
Two Al Pastor Meteors and today's agua fresca coming up — $12.00 even.

The same client reached models from two providers. Both requests now pass through AI Gateway and appear in its request history.

Two issues you may encounter:

404 or "model not found" with the old model name. Gateway model names include the creator prefix. Change gpt-5.4-mini to openai/gpt-5.4-mini and verify the current string in the model catalog.

Your stack uses a framework rather than a provider SDK. The same approach applies, although the configuration key varies. LangChain, LlamaIndex, LiteLLM, Mastra, and Pydantic AI have documented integrations; most use the framework's existing option for an OpenAI-compatible base URL. In Python, the equivalent option is typically base_url.

Commit

git commit -m "feat(compat): point the legacy OpenAI SDK client at the gateway"

Done-When

• The OpenAI SDK returns a completion through the Gateway with only baseURL/apiKey changed
• The same client served a Claude model
• Both requests show up in your Gateway dashboard usage
• You can name where your framework's base URL setting lives (if you have one)

Solution

import OpenAI from "openai";
 
// The only migration: this constructor used to have no arguments.
const openai = new OpenAI({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  baseURL: "https://ai-gateway.vercel.sh/v1",
});
 
const order =
  "Take this order and confirm it back with a total: two Al Pastor Meteors " +
  "($4.50 each) and one agua fresca ($3.00).";
 
for (const model of ["openai/gpt-5.4-mini", "anthropic/claude-sonnet-4.6"]) {
  const completion = await openai.chat.completions.create({
    model,
    messages: [{ role: "user", content: order }],
  });
 
  console.log(`[${model} via OpenAI SDK]`);
  console.log(completion.choices[0].message.content?.trim());
  console.log();
}

The existing client code remains in place and now sends requests to the Gateway URL.

Related Questions

• How do I migrate off the raw OpenAI SDK?
• How do I switch models without rewriting my app?
• Why use AI Gateway instead of calling providers directly?