seldon
v0
Log inSign up
Docs menu

Gateway

SDKs and frameworks

Use Seldon from the OpenAI SDKs, LiteLLM, LangChain and the Vercel AI SDK by changing only the base URL and API key.

Seldon speaks the OpenAI Chat Completions wire format. Existing clients work by changing two settings: the base URL, https://api.seldon-ai.com/v1, and a Seldon API key. Everything below reads them from SELDON_BASE_URL and SELDON_API_KEY.

OpenAI Python

Python
import os

from openai import OpenAI

client = OpenAI(api_key=os.environ["SELDON_API_KEY"], base_url=os.environ["SELDON_BASE_URL"])

OpenAI Node and TypeScript

TypeScript
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.SELDON_API_KEY,
  baseURL: process.env.SELDON_BASE_URL,
});

LiteLLM

Prefix the model with openai/ so LiteLLM uses its OpenAI-compatible client:

Python
import os

import litellm

response = litellm.completion(
    model="openai/openai/gpt-5.4-nano",
    api_base=os.environ["SELDON_BASE_URL"],
    api_key=os.environ["SELDON_API_KEY"],
    messages=[{"role": "user", "content": "Hello"}],
)

LangChain

Python
import os

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="openai/gpt-5.4-nano",
    base_url=os.environ["SELDON_BASE_URL"],
    api_key=os.environ["SELDON_API_KEY"],
)

Vercel AI SDK

TypeScript
import { createOpenAI } from "@ai-sdk/openai";
import { generateText } from "ai";

const seldon = createOpenAI({
  baseURL: process.env.SELDON_BASE_URL,
  apiKey: process.env.SELDON_API_KEY,
});

const { text } = await generateText({
  model: seldon.chat("openai/gpt-5.4-nano"),
  prompt: "Hello",
});

Use seldon.chat(...) so the SDK calls Chat Completions. Newer AI SDK versions default to the Responses API, which Seldon does not serve.

Verified compatibility

Each checked cell is covered by an automated test on every change to Seldon. Blank cells may work but are not yet covered.

Featureopenai (Python)LiteLLMlangchain-openaiopenai (Node)Vercel AI SDK
Non-streaming completion✓✓✓✓✓
Streaming✓✓✓✓
Tools✓✓
JSON response_format✓✓✓
Vision (image_url)✓✓
usage object✓✓✓
Structured error envelope✓
App attribution (X-Title)✓

Seldon-specific fields

Fields such as provider, models and cache are not part of the OpenAI SDK types. Pass them through the SDK's extra-body option:

Python
response = client.chat.completions.create(
    model="openai/gpt-5.4-nano",
    messages=[{"role": "user", "content": "Hello"}],
    extra_body={"provider": {"sort": "price"}},
)

In the Node SDK, add the fields to the request object; TypeScript may need a type cast. See Models and routing.