OpenAI API/SDKs compatibility

The Opper OpenAI compatibility layer allows you to use Opper models with the OpenAI API and SDKs. This gives you the ability to use any model provided by Opper in any project that uses the OpenAI API/SDKs.

import os
from openai import OpenAI

client = OpenAI(
  base_url="https://api.opper.ai/compat/openai",
  api_key="-", # must not be blank
  default_headers={"x-opper-api-key": os.getenv("OPPER_API_KEY")},
)

response = client.chat.completions.create(
  model="groq/deepseek-r1-distill-llama-70b",
  messages=[
    {"role": "user", "content": "What is the capital of France? Please reverse the name before answering."}
  ],
)

print(response)

import { OpenAI } from "openai";

const client = new OpenAI({
    baseURL: "https://api.opper.ai/compat/openai",
    apiKey: "OPPER_API_KEY",
    defaultHeaders: { "x-opper-api-key": "OPPER_API_KEY" },
});

const completion = await client.chat.completions.create({
    model: "openai/gpt-4o-mini",
    messages: [
        {
            role: "user",
            content: "What is the capital of France? Please reverse the name before answering.",
        },
    ],
});

console.log(completion);

Opper functionality when using the OpenAI API/SDKs

The OpenAI API/SDKs allows the user to provide extra body to the request. This can be used to work with Opper functionality. The extra body arguments supported are:

fallback_models: A list of models to use if the primary model is not available.
tags: A dictionary of tags to add to the request.
span_uuid: The UUID of the span to add to the request.
evaluate: Whether to evaluate the generation or not.

It also allows you to provide fallback models in case of failures.

import os
from openai import OpenAI
from opperai import Opper

opper = Opper()

client = OpenAI(
  base_url="https://api.opper.ai/compat/openai",
  api_key="-", # must not be blank
  default_headers={"x-opper-api-key": os.getenv("OPPER_API_KEY")},
)

with opper.spans.start("reverse-name") as span:
  response = client.chat.completions.create(
    model="gorq/deepseek-r1-distill-llama-70", # This model is not available since provider is called "gorq" and not "groq"
    messages=[
      {"role": "user", "content": "What is the capital of France? Please reverse the name before answering."}
    ],
    extra_body={
      "fallback_models": [
        "groq/deepseek-r1-distill-llama-70b",
      ],
      "tags": {
        "user_id": "123",
      },
      "span_uuid": str(span.uuid),
      "evaluate": False,
    }
  )

print(response)

import { OpenAI } from "openai";
import OpperAI from "opperai";

const opper = new OpperAI();

const client = new OpenAI({
    baseURL: "https://api.opper.ai/compat/openai",
    apiKey: "OPPER_API_KEY",
    defaultHeaders: { "x-opper-api-key": "OPPER_API_KEY" },
});

async function main() {
    const trace = await opper.traces.start({
        name: "node-sdk/using-the-openai-sdk",
        input: "What is the capital of France? Please reverse the name before answering.",
    });

    const completion = await client.chat.completions.create({
        model: "openai/gpt-4o-mini",
        messages: [
            {
                role: "user",
                content: "What is the capital of France? Please reverse the name before answering.",
            },
        ],

        // @ts-expect-error These are Opper specific params.
        // fallback_models: ["openai/gpt-4o-mini"],
        span_uuid: trace.uuid.toString(),
        // evaluate: false,
    });

    await trace.end({ output: { foo: completion.choices[0].message.content } });
}

main();