OpenAI SDK

Use the official OpenAI Node.js and Python SDKs with Fallbakit's OpenAI-compatible chat completions endpoint.

Fallbakit works with the official OpenAI Node.js and Python SDKs for chat completions. Use a Fallbakit application API key and set the OpenAI SDK base URL to https://api.fallbakit.com/v1.

Node.js

npm install openai

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.FALLBAKIT_API_KEY,
  baseURL: process.env.FALLBAKIT_OPENAI_BASE_URL ?? "https://api.fallbakit.com/v1",
});

const completion = await client.chat.completions.create({
  model: process.env.FALLBAKIT_MODEL ?? "llama3.2",
  messages: [{ role: "user", content: "Write a tiny launch checklist." }],
});

console.log(completion.choices[0]?.message?.content);

Python

python -m pip install openai

import os

from openai import OpenAI

client = OpenAI(
    api_key=os.environ["FALLBAKIT_API_KEY"],
    base_url=os.environ.get("FALLBAKIT_OPENAI_BASE_URL", "https://api.fallbakit.com/v1"),
)

completion = client.chat.completions.create(
    model=os.environ.get("FALLBAKIT_MODEL", "llama3.2"),
    messages=[{"role": "user", "content": "Write a tiny launch checklist."}],
)

print(completion.choices[0].message.content)

Streaming

Node.js streams are async iterables:

const stream = await client.chat.completions.create({
  model: "llama3.2",
  messages: [{ role: "user", content: "Stream a short answer." }],
  stream: true,
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}

Python streams are iterators:

stream = client.chat.completions.create(
    model="llama3.2",
    messages=[{"role": "user", "content": "Stream a short answer."}],
    stream=True,
)

for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="", flush=True)

Fallback Controls

Most requests can rely on the application's configured routing rules. When a request needs explicit Fallbakit controls, send them under extra_body.fallbakit.

Node.js can send the OpenAI-compatible extra request field directly:

await client.chat.completions.create({
  model: "llama3.2",
  messages: [{ role: "user", content: "Summarize local-first routing." }],
  extra_body: {
    fallbakit: {
      fallbackProvider: "openai",
      fallbackModel: "gpt-4o-mini",
    },
  },
});

Python's OpenAI SDK uses extra_body as a request option that merges extra top-level JSON into the request body, so nest Fallbakit's field once:

completion = client.chat.completions.create(
    model="llama3.2",
    messages=[{"role": "user", "content": "Summarize local-first routing."}],
    extra_body={
        "extra_body": {
            "fallbakit": {
                "fallbackProvider": "openai",
                "fallbackModel": "gpt-4o-mini",
            }
        }
    },
)

Supported controls are fallbackProvider, fallbackModel, cloudModelOnly, and localModelOnly. Provider keys stay in Fallbakit application settings and should never be sent in SDK requests.

Node.js

Python

Streaming

Fallback Controls

On this page