# Installation Source: https://docs.opper.ai/agents/installation Install the unified opperai SDK for Python or TypeScript The Agent SDK ships inside the unified `opperai` package, so you don't install anything extra. ```bash Python theme={null} pip install opperai # or uv add opperai # or poetry add opperai ``` ```bash TypeScript theme={null} npm install opperai zod # or pnpm add opperai zod ``` | Language | Minimum runtime | Notes | | - | - | - | | Python | 3.10+ | Pydantic optional: `pip install opperai[pydantic]` for typed `output_schema`. | | TypeScript | Node.js 18+ | Requires Zod v4 (`zod@4`). The `zod@3.25.x` dual-mode build is not supported. | ## API key Set `OPPER_API_KEY` in the environment, or pass `client` config explicitly: ```python Python theme={null} from opperai.agent import Agent agent = Agent( name="my-agent", instructions="Be helpful.", client={"api_key": "op-..."}, ) ``` ```typescript TypeScript theme={null} import { Agent } from "opperai"; const agent = new Agent({ name: "my-agent", instructions: "Be helpful.", client: { apiKey: "op-..." }, }); ``` Get a key at [platform.opper.ai](https://platform.opper.ai/). ## Verify ```python Python theme={null} import opperai print(opperai.__version__) ``` ```typescript TypeScript theme={null} import { Agent } from "opperai"; console.log(typeof Agent); // "function" ``` Next: [Quickstart](/agents/quickstart). # Migration & legacy SDKs Source: https://docs.opper.ai/agents/migration Coming from the standalone agent SDKs? Here's where to look The Agent SDK now ships inside the `opperai` package. The previous standalone agent SDKs no longer get updates. They're still installable, and we keep links to the old repos here for anyone on those versions. ## Are you on a legacy version? | Language | Legacy package | Last legacy version | New package | | - | - | - | - | | Python | [`opper-agents`](https://pypi.org/project/opper-agents/) | `<= 0.4.0` | [`opperai`](https://pypi.org/project/opperai/) `>= 2.0` | | TypeScript | [`@opperai/agents`](https://www.npmjs.com/package/@opperai/agents) | `<= 0.8.0` | [`opperai`](https://www.npmjs.com/package/opperai) `>= 4.0` | If you're on a version at or below those, you're on the legacy SDK. ## Migration guides The new repo has a migration guide for each language. They cover the breaking changes: imports, constructors, method renames, schema changes, and MCP config. `opperai` 1.x + `opper-agents` 0.x → `opperai` 2.0 `opperai` 3.x + `@opperai/agents` 0.x → `opperai` 4.0 The headline changes: * `from opper_agents import ...` → `from opperai.agent import ...` (Python) / `from '@opperai/agents'` → `from 'opperai'` (TS) * `agent.process(...)` → `agent.run(...)`; result is `{ output, meta }` * `description=` → `instructions=` on the `Agent` constructor * `Hooks` is now a typed object (Python `Hooks(...)`, TS `Hooks` interface), not individual `@hook` decorators ## Legacy repos (read-only) These repositories are kept up for reference. New work happens in [`opper-ai/opper-sdks`](https://github.com/opper-ai/opper-sdks). | | Python | TypeScript | | - | - | - | | Repo | [`opperai-agent-sdk`](https://github.com/opper-ai/opperai-agent-sdk) | [`opperai-agent-sdk-node`](https://github.com/opper-ai/opperai-agent-sdk-node) | | Package | [`opper-agents`](https://pypi.org/project/opper-agents/) | [`@opperai/agents`](https://www.npmjs.com/package/@opperai/agents) | If you're starting a new project, install `opperai` and follow the [Quickstart](/agents/quickstart). If you're maintaining a project that pins `opper-agents` or `@opperai/agents`, the legacy packages still work, but new features land only in `opperai`. # Agent SDK Overview Source: https://docs.opper.ai/agents/overview Build AI agents with tools, streaming, multi-agent composition, and MCP support The Opper Agent SDK is part of the unified `opperai` SDK and is available for **Python** and **TypeScript** with a matching API. `pip install opperai` `npm install opperai` On the standalone `opper-agents` (Python) or `@opperai/agents` (TypeScript) package? See [Migration & legacy SDKs](/agents/migration) for the upgrade path and links to the legacy repos. `opper-ai/opper-sdks` has the full source for both languages. 13 numbered examples per language. Each one runs against the live API. | Language | Package | Imports | Examples | | - | - | - | - | | Python | `opperai` | `from opperai.agent import Agent, tool, Hooks` | [`python/examples/agents/`](https://github.com/opper-ai/opper-sdks/tree/main/python/examples/agents) | | TypeScript | `opperai` | `import { Agent, tool } from "opperai"` | [`typescript/examples/agents/`](https://github.com/opper-ai/opper-sdks/tree/main/typescript/examples/agents) | ## What an agent is An agent runs a **think → act → observe** loop: 1. **Think**: the LLM is given the goal, instructions, and available tools, and decides what to do next. 2. **Act**: if the model called a tool, the SDK executes it and feeds the result back. 3. **Loop**: repeat until the model produces a final answer (and validates it against `output_schema` if set). You define instructions, tools, and an optional output schema. The SDK handles the rest. ## Minimal agent ```python Python theme={null} from opperai.agent import Agent, tool @tool def get_weather(city: str) -> str: """Get the current weather for a city.""" return f"Sunny, 22°C in {city}" agent = Agent( name="weather-assistant", instructions="You are a helpful weather assistant.", tools=[get_weather], ) result = await agent.run("What's the weather in Paris?") print(result.output) print(result.meta.usage.total_tokens) ``` ```typescript TypeScript theme={null} import { z } from "zod"; import { Agent, tool } from "opperai"; const getWeather = tool({ name: "get_weather", description: "Get the current weather for a city", parameters: z.object({ city: z.string() }), execute: async ({ city }) => `Sunny, 22°C in ${city}`, }); const agent = new Agent({ name: "weather-assistant", instructions: "You are a helpful weather assistant.", tools: [getWeather], }); const result = await agent.run("What's the weather in Paris?"); console.log(result.output); console.log(result.meta.usage.totalTokens); ``` ## Patterns | Pattern | Page | Source example | | - | - | - | | Tools | [Tools](/agents/patterns/tools) | `02_agent_with_tools` | | Structured output | [Structured output](/agents/patterns/structured-output) | `01_agent_with_schema` | | Streaming | [Streaming](/agents/patterns/streaming) | `03_streaming` | | Lifecycle hooks | [Hooks](/agents/patterns/hooks) | `04_hooks_logging`, `05_hooks_timing` | | Multi-agent | [Multi-agent](/agents/patterns/multi-agent) | `07_agent_as_tool`, `08_multi_agent` | | Multi-turn | [Conversation](/agents/patterns/conversation) | `10_conversation` | | MCP | [MCP](/agents/patterns/mcp) | `09_mcp_stdio` | With tracing on, every agent run, LLM call, and tool execution shows up as a structured trace tree in the [Opper dashboard](https://platform.opper.ai/). Tracing is off by default: Opper records usage metadata (model, cost, and latency) but stores no traces until you turn tracing on in [Opper retention](/control-plane/rules/retention), a Control Plane feature. ## When to use it Reach for the Agent SDK when the work involves choosing between tools, chaining steps based on intermediate results, or coordinating multiple specialists. For a single deterministic LLM call, call a model directly through the [chat completions endpoint](/v3-api-reference/compatibility/chat-completions). # Conversation Source: https://docs.opper.ai/agents/patterns/conversation Multi-turn stateful interactions with an agent Source: [`10_conversation.py`](https://github.com/opper-ai/opper-sdks/blob/main/python/examples/agents/10_conversation.py) · [`10-conversation.ts`](https://github.com/opper-ai/opper-sdks/blob/main/typescript/examples/agents/10-conversation.ts) `agent.conversation()` returns a `Conversation` object that tracks message history across turns. Each `.send()` (or `.stream()`) call sees prior context automatically, so you don't manage the items array yourself. ## Multi-turn ```python Python theme={null} agent = Agent(name="note-assistant", instructions="...", tools=[save_note, list_notes]) conv = agent.conversation() await conv.send("My name is Alice. Remember that.") await conv.send("Save a note: buy groceries") r = await conv.send("What's my name, and what notes have I saved?") print(r.output) # references Alice and the note print(len(conv.get_items())) # full message history ``` ```typescript TypeScript theme={null} const agent = new Agent({ name: "note-assistant", instructions: "...", tools: [saveNote, listNotes] }); const conv = agent.conversation(); await conv.send("My name is Alice. Remember that."); await conv.send("Save a note: buy groceries"); const r = await conv.send("What's my name, and what notes have I saved?"); console.log(r.output); console.log(conv.getItems().length); ``` ## Streaming inside a conversation `conv.stream(input)` works the same as `agent.stream(input)`, but the conversation accumulates the streamed turn into its history once `.result()` resolves. ```python theme={null} stream = conv.stream("Summarize everything we discussed.") async for event in stream: if event.type == "text_delta": print(event.text, end="", flush=True) result = await stream.result() ``` ## Resetting `conv.clear()` empties the history and starts fresh. There is no built-in persistence, so serialize `conv.get_items()` / `conv.getItems()` yourself if you need to resume across processes. # Hooks Source: https://docs.opper.ai/agents/patterns/hooks Observe every lifecycle event during an agent run Source: [`04_hooks_logging.py`](https://github.com/opper-ai/opper-sdks/blob/main/python/examples/agents/04_hooks_logging.py), [`05_hooks_timing.py`](https://github.com/opper-ai/opper-sdks/blob/main/python/examples/agents/05_hooks_timing.py) · [`04-hooks-logging.ts`](https://github.com/opper-ai/opper-sdks/blob/main/typescript/examples/agents/04-hooks-logging.ts), [`05-hooks-timing.ts`](https://github.com/opper-ai/opper-sdks/blob/main/typescript/examples/agents/05-hooks-timing.ts) Hooks let you log, time, or measure agent execution without touching the agent logic itself. Pass a `Hooks` object to the agent constructor. ## Lifecycle events | Hook | Fired when | | - | - | | `on_agent_start` / `onAgentStart` | A run begins. | | `on_agent_end` / `onAgentEnd` | A run completes (or fails). | | `on_iteration_start` / `onIterationStart` | A new think→act cycle. | | `on_llm_call` / `onLLMCall` | About to send a request to the model. | | `on_llm_response` / `onLLMResponse` | Got a model response. | | `on_tool_start` / `onToolStart` | About to execute a tool. | | `on_tool_end` / `onToolEnd` | Tool finished (with `duration_ms`). | ## Example ```python Python theme={null} from opperai.agent import Agent, Hooks, tool hooks = Hooks( on_agent_start=lambda ctx: print(f'-> {ctx["agent"]}: {ctx["input"]}'), on_tool_start=lambda ctx: print(f' tool {ctx["name"]}({ctx["input"]})'), on_tool_end=lambda ctx: print(f' -> {ctx["output"]} ({ctx["duration_ms"]:.0f}ms)'), on_agent_end=lambda ctx: print( f'<- done in {ctx["result"].meta.iterations} iterations' if ctx.get("result") else f'<- failed: {ctx.get("error")}' ), ) agent = Agent(name="city-expert", instructions="...", tools=[lookup_city], hooks=hooks) ``` ```typescript TypeScript theme={null} import { Agent, type Hooks } from "opperai"; const hooks: Hooks = { onAgentStart: ({ agent, input }) => console.log(`-> ${agent}: ${input}`), onToolStart: ({ name, input }) => console.log(` tool ${name}(${JSON.stringify(input)})`), onToolEnd: ({ name, output, durationMs }) => console.log(` -> ${JSON.stringify(output)} (${durationMs}ms)`), onAgentEnd: ({ result, error }) => error ? console.log(`<- failed: ${error.message}`) : console.log(`<- done in ${result!.meta.iterations} iterations`), }; const agent = new Agent({ name: "city-expert", instructions: "...", tools: [lookupCity], hooks }); ``` ## Common uses * Logging: print events or ship them to your logger. * Timing: add up `duration_ms` per tool to find bottlenecks (see `05_hooks_timing`). * Custom metrics: push counters or histograms to your metrics backend. * Span enrichment: attach extra metadata to the trace span the SDK already creates. Hooks sit on top of built-in tracing. With tracing on (turn it on in [Opper retention](/control-plane/rules/retention)), every run, LLM call, and tool execution lands in the [Opper dashboard](https://platform.opper.ai/) without extra setup. # MCP Integration Source: https://docs.opper.ai/agents/patterns/mcp Connect to Model Context Protocol servers and use their tools Source: [`09_mcp_stdio.py`](https://github.com/opper-ai/opper-sdks/blob/main/python/examples/agents/09_mcp_stdio.py) · [`09-mcp-stdio.ts`](https://github.com/opper-ai/opper-sdks/blob/main/typescript/examples/agents/09-mcp-stdio.ts) · Applied: [`daily_digest_agent.py`](https://github.com/opper-ai/opper-sdks/blob/main/python/examples/agents/applied_agents/daily_digest_agent.py) [MCP](https://modelcontextprotocol.io/) servers expose tools (filesystem, databases, APIs, etc.) over a standard protocol. The agent SDK can connect to any MCP server and turn its tools into native agent tools. ## Stdio (local process) ```python Python theme={null} from opperai.agent import Agent from opperai.agent.mcp import MCPStdioConfig, mcp agent = Agent( name="fs-assistant", instructions="You have filesystem access via MCP. Be concise.", tools=[ mcp(MCPStdioConfig( name="filesystem", command="npx", args=["-y", "@modelcontextprotocol/server-filesystem", "/tmp"], )), ], ) result = await agent.run("List the files in /tmp.") ``` ```typescript TypeScript theme={null} import { Agent, mcp } from "opperai"; const agent = new Agent({ name: "fs-assistant", instructions: "You have filesystem access via MCP. Be concise.", tools: [ mcp({ name: "filesystem", transport: "stdio", command: "npx", args: ["-y", "@modelcontextprotocol/server-filesystem", "/tmp"], }), ], }); const result = await agent.run("List the files in /tmp."); ``` The SDK starts the server, discovers its tools, and makes them available to the agent. When the run ends, it shuts the server down. ## Streamable HTTP / SSE (remote) Both SDKs also accept `MCPStreamableHTTPConfig` (`MCPSSEConfig` for legacy SSE) for remote servers. Useful for hosted MCPs like [Composio](https://composio.dev/) or [Smithery](https://smithery.ai/): ```python Python theme={null} from opperai.agent.mcp import MCPStreamableHTTPConfig, mcp mcp(MCPStreamableHTTPConfig( name="gmail", url=os.environ["COMPOSIO_GMAIL_MCP_URL"], )) ``` ```typescript TypeScript theme={null} import { mcp } from "opperai"; mcp({ name: "gmail", transport: "streamable-http", url: process.env.COMPOSIO_GMAIL_MCP_URL!, }); ``` In TypeScript, `transport` selects the config type: `"stdio"`, `"streamable-http"`, or `"sse"`. Remote configs also take an optional `headers` object for authentication. The applied [`daily_digest_agent`](https://github.com/opper-ai/opper-sdks/blob/main/python/examples/agents/applied_agents/daily_digest_agent.py) example wires four remote MCP servers (Gmail, Hacker News, Notion, Search) into a single agent. ## Multiple servers Pass multiple `mcp(...)` providers in the same `tools=[...]` list, and the agent merges all of their tools. If two tool names collide, the agent prefixes them with the server name. # Multi-Agent Source: https://docs.opper.ai/agents/patterns/multi-agent Compose specialists by exposing one agent as a tool to another Source: [`07_agent_as_tool.py`](https://github.com/opper-ai/opper-sdks/blob/main/python/examples/agents/07_agent_as_tool.py), [`08_multi_agent.py`](https://github.com/opper-ai/opper-sdks/blob/main/python/examples/agents/08_multi_agent.py) · [`07-agent-as-tool.ts`](https://github.com/opper-ai/opper-sdks/blob/main/typescript/examples/agents/07-agent-as-tool.ts), [`08-multi-agent.ts`](https://github.com/opper-ai/opper-sdks/blob/main/typescript/examples/agents/08-multi-agent.ts) Use `agent.as_tool(...)` (Python) / `agent.asTool(...)` (TS) to wrap a specialist agent as a callable tool, then attach it to a coordinator. The coordinator decides when to delegate and to whom. ## Coordinator + specialist ```python Python theme={null} from opperai.agent import Agent geographer = Agent( name="geographer", instructions="You are a geography expert. Be precise and concise.", ) coordinator = Agent( name="coordinator", instructions=( "You are a helpful assistant. For geography questions, " "use the geography_expert tool. Otherwise answer directly." ), tools=[ geographer.as_tool( name="geography_expert", description="Ask a geography expert. Pass the full question as input.", ), ], ) result = await coordinator.run("What are the three largest countries by area?") ``` ```typescript TypeScript theme={null} import { Agent } from "opperai"; const geographer = new Agent({ name: "geographer", instructions: "You are a geography expert. Be precise and concise.", }); const coordinator = new Agent({ name: "coordinator", instructions: "You are a helpful assistant. For geography questions, " + "use the geography_expert tool. Otherwise answer directly.", tools: [ geographer.asTool({ name: "geography_expert", description: "Ask a geography expert. Pass the full question as input.", }), ], }); const result = await coordinator.run("What are the three largest countries by area?"); ``` The sub-agent's tool input is a single string (the prompt). Its output is `{ output, usage, iterations, ... }`, which is captured in the coordinator's `result.meta.tool_calls`. ## Pipelines Stack multiple specialists under one coordinator (researcher → writer → editor, etc.). The coordinator chooses the order based on its instructions. See `08_multi_agent` for a three-agent editorial pipeline. ## Tracing With tracing enabled, sub-agent calls appear as nested `SubAgent` spans under the coordinator's run. You get the full tree, token usage, and timings for every agent in one place. # Streaming Source: https://docs.opper.ai/agents/patterns/streaming Observe an agent's work in real time Source: [`03_streaming.py`](https://github.com/opper-ai/opper-sdks/blob/main/python/examples/agents/03_streaming.py) · [`03-streaming.ts`](https://github.com/opper-ai/opper-sdks/blob/main/typescript/examples/agents/03-streaming.ts) Use `agent.stream(input)` to receive lifecycle events as the agent runs. After iterating, call `.result()` for the final `RunResult`. ## Iterate events ```python Python theme={null} import sys from opperai.agent import Agent, tool agent = Agent(name="weather", instructions="...", tools=[get_weather]) stream = agent.stream("What's the weather in Paris and Tokyo?") async for event in stream: match event.type: case "iteration_start": print(f"\n[Iteration {event.iteration}]") case "text_delta": sys.stdout.write(event.text) sys.stdout.flush() case "tool_start": print(f"-> {event.name}({event.input})") case "tool_end": print(f"<- {event.name} -> {event.output} ({event.duration_ms:.0f}ms)") result = await stream.result() print("\nFinal:", result.output) ``` ```typescript TypeScript theme={null} import { Agent } from "opperai"; const agent = new Agent({ name: "weather", instructions: "...", tools: [getWeather] }); const stream = agent.stream("What's the weather in Paris and Tokyo?"); for await (const event of stream) { switch (event.type) { case "iteration_start": console.log(`\n[Iteration ${event.iteration}]`); break; case "text_delta": process.stdout.write(event.text); break; case "tool_start": console.log(`-> ${event.name}(${JSON.stringify(event.input)})`); break; case "tool_end": console.log(`<- ${event.name} -> ${JSON.stringify(event.output)} (${event.durationMs}ms)`); break; } } const result = await stream.result(); console.log("\nFinal:", result.output); ``` ## Event types | Event | Fired when | | - | - | | `iteration_start` | A new think→act cycle begins. | | `text_delta` | The model emits a chunk of final text. | | `tool_start` / `tool_end` | A tool is about to run / has completed. | | `result` | The final `RunResult` (also returned by `.result()`). | The full set is exported as `AgentStreamEvent` in TypeScript and lives in `opperai.agent._stream` in Python. # Structured Output Source: https://docs.opper.ai/agents/patterns/structured-output Return validated, typed data from an agent run Source: [`01_agent_with_schema.py`](https://github.com/opper-ai/opper-sdks/blob/main/python/examples/agents/01_agent_with_schema.py) · [`01-agent-with-schema.ts`](https://github.com/opper-ai/opper-sdks/blob/main/typescript/examples/agents/01-agent-with-schema.ts) Pass an `output_schema` to constrain the agent's final answer to a typed shape. The SDK validates the model's output against the schema and returns a typed object. ## Pydantic / Zod ```python Python theme={null} from pydantic import BaseModel from opperai.agent import Agent class Summary(BaseModel): title: str key_points: list[str] sentiment: str # "positive" | "negative" | "neutral" agent = Agent( name="summarizer", instructions="Summarize the input.", output_schema=Summary, ) result = await agent.run("Python 3.12 brings major performance improvements...") summary = result.output # validated Summary instance print(summary.title) print(summary.sentiment) for point in summary.key_points: print("-", point) ``` ```typescript TypeScript theme={null} import { z } from "zod"; import { Agent } from "opperai"; const Summary = z.object({ title: z.string().describe("A short title"), key_points: z.array(z.string()).describe("3-5 key points"), sentiment: z.enum(["positive", "negative", "neutral"]), }); const agent = new Agent({ name: "summarizer", instructions: "Summarize the input.", outputSchema: Summary, }); const result = await agent.run("Python 3.12 brings major performance improvements..."); // result.output is typed — no cast needed console.log(result.output.title); console.log(result.output.sentiment); for (const point of result.output.key_points) console.log("-", point); ``` ## Other schema sources Both SDKs accept raw JSON Schema. Python additionally accepts dataclasses and `TypedDict`. TypeScript accepts any Standard Schema implementation (Valibot, ArkType, etc.). ```python Python theme={null} output_schema = { "type": "object", "properties": { "title": {"type": "string"}, "key_points": {"type": "array", "items": {"type": "string"}}, }, "required": ["title", "key_points"], } ``` ## Without a schema Omit `output_schema` and the agent returns a string in `result.output`. # Tools Source: https://docs.opper.ai/agents/patterns/tools Define functions an agent can call during its loop Source: [`02_agent_with_tools.py`](https://github.com/opper-ai/opper-sdks/blob/main/python/examples/agents/02_agent_with_tools.py) · [`02-agent-with-tools.ts`](https://github.com/opper-ai/opper-sdks/blob/main/typescript/examples/agents/02-agent-with-tools.ts) A tool is a Python or TypeScript function the agent may decide to call. The SDK introspects parameters, sends a JSON Schema to the model, and executes the function when the model picks it. ## Define a tool ```python Python theme={null} from opperai.agent import tool @tool def lookup_product(product_id: str) -> dict: """Look up a product by its ID.""" return {"name": "Headphones", "price": 79.99, "stock": 42} ``` ```typescript TypeScript theme={null} import { z } from "zod"; import { tool } from "opperai"; const lookupProduct = tool({ name: "lookup_product", description: "Look up a product by its ID", parameters: z.object({ product_id: z.string().describe("The product ID (e.g. prod-001)"), }), execute: async ({ product_id }) => { return { name: "Headphones", price: 79.99, stock: 42 }; }, }); ``` In Python, the schema comes from the type hints and the docstring becomes the description. In TypeScript, you pass an explicit Zod schema, any other [Standard Schema](https://standardschema.dev/) (Valibot, ArkType, and so on), or raw JSON Schema. ## Attach tools to an agent ```python Python theme={null} from opperai.agent import Agent agent = Agent( name="shop-assistant", instructions="Use tools to look up products. Be concise.", tools=[lookup_product, check_availability], ) result = await agent.run("Can I order 10 units of prod-002?") ``` ```typescript TypeScript theme={null} import { Agent } from "opperai"; const agent = new Agent({ name: "shop-assistant", instructions: "Use tools to look up products. Be concise.", tools: [lookupProduct, checkAvailability], }); const result = await agent.run("Can I order 10 units of prod-002?"); ``` The agent loops automatically: it calls a tool, observes the result, and either calls another tool or produces a final answer. ## Inspect tool calls Every tool invocation is captured in `result.meta`: ```python Python theme={null} for call in result.meta.tool_calls: print(call.name, call.input, "->", call.output, f"({call.duration_ms:.0f}ms)") ``` ```typescript TypeScript theme={null} for (const call of result.meta.toolCalls) { console.log(call.name, call.input, "->", call.output, `(${call.durationMs}ms)`); } ``` ## Tips * Return JSON-serializable data. Dicts, lists, and primitives work best, since the SDK serializes them for the model. * Keep the docstring or description short. That text is what the model reads when it decides whether to call the tool. * Errors are fine. If the function raises, the SDK reports the error to the model so it can recover or report back. * Tool calls run in parallel by default. If the model issues several tool calls in one turn, the SDK runs them concurrently. Disable this with `parallel_tool_execution=False` (Python) or `parallelToolExecution: false` (TS) on the `Agent`. # Quickstart Source: https://docs.opper.ai/agents/quickstart Install, set your API key, and run your first agent Source: [`opper-ai/opper-sdks`](https://github.com/opper-ai/opper-sdks). The agent layer ships inside the `opperai` package, so there's nothing extra to install. ## Build your first agent ```bash Python theme={null} pip install opperai # or: uv add opperai ``` ```bash TypeScript theme={null} npm install opperai zod # or: pnpm add opperai zod ``` Requirements: Python 3.10+ or Node.js 18+. TypeScript users need Zod v4. ```bash theme={null} export OPPER_API_KEY="your-api-key" ``` Get a key at [platform.opper.ai](https://platform.opper.ai/). You can also pass it explicitly via `client={"api_key": "..."}` (Python) or `client: { apiKey: "..." }` (TypeScript) on the `Agent` constructor. ```python Python theme={null} import asyncio from opperai.agent import Agent, tool @tool def add(a: int, b: int) -> int: """Add two numbers.""" return a + b @tool def multiply(a: int, b: int) -> int: """Multiply two numbers.""" return a * b async def main() -> None: agent = Agent( name="math-agent", instructions="You are a helpful math assistant. Show the steps.", tools=[add, multiply], ) result = await agent.run("What is 5 + 3, then multiplied by 2?") print("Output:", result.output) print("Iterations:", result.meta.iterations) print("Tokens:", result.meta.usage.total_tokens) asyncio.run(main()) ``` ```typescript TypeScript theme={null} import { z } from "zod"; import { Agent, tool } from "opperai"; const add = tool({ name: "add", description: "Add two numbers", parameters: z.object({ a: z.number(), b: z.number() }), execute: async ({ a, b }) => a + b, }); const multiply = tool({ name: "multiply", description: "Multiply two numbers", parameters: z.object({ a: z.number(), b: z.number() }), execute: async ({ a, b }) => a * b, }); const agent = new Agent({ name: "math-agent", instructions: "You are a helpful math assistant. Show the steps.", tools: [add, multiply], }); const result = await agent.run("What is 5 + 3, then multiplied by 2?"); console.log("Output:", result.output); console.log("Iterations:", result.meta.iterations); console.log("Tokens:", result.meta.usage.totalTokens); ``` The agent picks `add(5, 3) = 8`, then `multiply(8, 2) = 16`, then returns. Every tool call is captured in `result.meta.tool_calls` (Python) / `result.meta.toolCalls` (TS). The repo has numbered examples, each showing one pattern. Run them directly: ```bash Python theme={null} git clone https://github.com/opper-ai/opper-sdks cd opper-sdks/python uv run python examples/agents/00_first_agent.py ``` ```bash TypeScript theme={null} git clone https://github.com/opper-ai/opper-sdks cd opper-sdks/typescript npx tsx examples/agents/00-your-first-agent.ts ``` ## What's next Define tools the agent can call. Return validated, typed data instead of free text. Watch the agent work as it happens. Compose specialists into a coordinator. # Completions Source: https://docs.opper.ai/build/gateway/completions The basic gateway call: send messages, pick a model, get a response. The simplest gateway call: a list of messages goes in, a single response comes out. Same shape every model supports. If you haven't set up your SDK yet, start with [Drop-in SDKs](/build/gateway/drop-in-sdks). ## A minimal call ```python Python theme={null} import os from openai import OpenAI client = OpenAI( base_url="https://api.opper.ai/v3/compat", api_key=os.environ["OPPER_API_KEY"], ) r = client.chat.completions.create( model="openai/gpt-5-mini", messages=[{"role": "user", "content": "What's a vector?"}], ) print(r.choices[0].message.content) ``` ```typescript TypeScript theme={null} import OpenAI from "openai"; const client = new OpenAI({ baseURL: "https://api.opper.ai/v3/compat", apiKey: process.env.OPPER_API_KEY!, }); const r = await client.chat.completions.create({ model: "openai/gpt-5-mini", messages: [{ role: "user", content: "What's a vector?" }], }); console.log(r.choices[0].message.content); ``` ```bash cURL theme={null} curl https://api.opper.ai/v3/compat/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPPER_API_KEY" \ -d '{ "model": "openai/gpt-5-mini", "messages": [{"role": "user", "content": "What'\''s a vector?"}] }' ``` ## Pick a model The `model` field is always provider-prefixed: `openai/gpt-5.5`, `anthropic/claude-sonnet-4-6`, `gemini/gemini-3.1-pro-preview`. Browse the full set on the [Models](/capabilities/models) page. You can call any model from any SDK. The Anthropic SDK can talk to a Google model, the OpenAI SDK can talk to a Claude model. The provider prefix decides where the call routes. The SDK only sets the request shape. If you don't pass `model`, the call falls through to your [default-model rule](/control-plane/route), which sets a model per organization or project so you can keep model names out of your code. ## Common parameters | Parameter | What it does | | - | - | | `temperature` | 0 to 2. Lower is more deterministic. | | `top_p` | 0 to 1. Nucleus sampling. Don't use with `temperature` at the same time. | | `max_tokens` | Cap the response length. | | `stop` | A string or array of strings. The model stops as soon as it sees one. | | `frequency_penalty` / `presence_penalty` | -2 to 2. Penalize repeated tokens. | | `n` | How many response choices to generate. Default 1. | Reasoning models (the GPT-5 family, Claude with extended thinking) also accept `reasoning_effort: "low" | "medium" | "high"` to control how much the model "thinks" before answering. ## The response shape ```json Response theme={null} { "id": "chatcmpl_...", "object": "chat.completion", "model": "openai/gpt-5-mini", "created": 1716124800, "choices": [ { "index": 0, "message": { "role": "assistant", "content": "A vector is..." }, "finish_reason": "stop" } ], "usage": { "prompt_tokens": 11, "completion_tokens": 42, "total_tokens": 53 } } ``` The two things you'll read most: * `choices[0].message.content` is the assistant's reply * `choices[0].finish_reason` is why the model stopped (`stop`, `length`, `tool_calls`, `content_filter`) ## What's next Multi-turn chat with message history. Let the model call your code. Stream tokens as they're generated. Get JSON back instead of free text. # Conversations Source: https://docs.opper.ai/build/gateway/conversations Multi-turn chat with message history. The foundation of any chatbot or assistant. A conversation is just a list of messages. You keep adding to it, send the whole list every time, and the model has full context of everything that came before. ## The message shape Every message has a `role` and `content`. | Role | What it is | | - | - | | `system` | Instructions for the model. Usually one message at the start of the conversation. | | `user` | What the user typed. | | `assistant` | What the model said. You append these from previous responses. | | `tool` | The result of a tool call (when using [Tool calling](/build/gateway/tools)). | ## A working multi-turn ```python Python theme={null} import os from openai import OpenAI client = OpenAI( base_url="https://api.opper.ai/v3/compat", api_key=os.environ["OPPER_API_KEY"], ) messages = [ {"role": "system", "content": "You are a concise assistant."}, {"role": "user", "content": "What's a vector?"}, ] # Turn 1 r = client.chat.completions.create(model="openai/gpt-5-mini", messages=messages) reply = r.choices[0].message.content messages.append({"role": "assistant", "content": reply}) print("Assistant:", reply) # Turn 2 — the model remembers turn 1 because we sent the whole list messages.append({"role": "user", "content": "Give me an example in 3D."}) r = client.chat.completions.create(model="openai/gpt-5-mini", messages=messages) print("Assistant:", r.choices[0].message.content) ``` ```typescript TypeScript theme={null} import OpenAI from "openai"; const client = new OpenAI({ baseURL: "https://api.opper.ai/v3/compat", apiKey: process.env.OPPER_API_KEY!, }); const messages: any[] = [ { role: "system", content: "You are a concise assistant." }, { role: "user", content: "What's a vector?" }, ]; let r = await client.chat.completions.create({ model: "openai/gpt-5-mini", messages }); const reply = r.choices[0].message.content; messages.push({ role: "assistant", content: reply }); console.log("Assistant:", reply); messages.push({ role: "user", content: "Give me an example in 3D." }); r = await client.chat.completions.create({ model: "openai/gpt-5-mini", messages }); console.log("Assistant:", r.choices[0].message.content); ``` ## System prompt The first `system` message sets the model's persona and ground rules. It stays in the conversation, and you only write it once. ```python Python theme={null} {"role": "system", "content": "You are a customer support agent for Acme Corp. Always cite a ticket number when referring to a past issue."} ``` Keep it short. Long system prompts cost more on every turn and tend to be ignored. ## Stop reasons The response tells you why the model stopped: | `finish_reason` | Meaning | | - | - | | `stop` | Model finished its answer naturally. | | `length` | Hit `max_tokens`. Response may be truncated. | | `tool_calls` | Model wants to call a tool (see [Tool calling](/build/gateway/tools)). | | `content_filter` | A [check](/control-plane/rules/checks) blocked or modified the output. | ## Keeping history under control Messages add up fast. A few options: * **Trim** older turns when the conversation gets long. Keep the system message and recent N turns. * **Summarize** older turns into a single assistant message ("earlier you discussed: X, Y, Z"). * **Use a separate [structured output](/build/gateway/structured-output) call** to extract just the bits worth remembering, then drop the rest. ## What's next Let the model invoke your tools mid-conversation. Stream the response token-by-token. Send images and documents as message content. The auth and base URL setup. # Drop-in SDKs Source: https://docs.opper.ai/build/gateway/drop-in-sdks Use the OpenAI, Anthropic, Google AI, or TypeSafe SDK against Opper by changing the base URL and API key. The gateway speaks the protocols you already use: the OpenAI, Anthropic, and Google AI APIs, with messages, tools, and multi-turn conversation working exactly as they do natively. If you already use one of those SDKs, keep it. Point it at Opper and swap the API key for an Opper one, and every call now flows through the gateway. You get 700+ models, fallback routing, per-call usage and cost in [Logs](/control-plane/logs), and organization Rules without rewriting anything. The TypeSafe SDKs also work with the gateway for classification and scoring. See [TypeSafe SDKs](#typesafe-sdks) below for setup, or [Classification & scoring](/build/gateway/evaluations) for question types and HTTP examples. The OpenAI SDK's `client.decisions` works too, in OpenAI's Decisions format. ## Setup For the OpenAI, Anthropic, and Google AI SDKs the change is the same: 1. Set the base URL to `https://api.opper.ai/v3/compat` 2. Pass your Opper API key as the SDK's normal `api_key` Opper reads the key from `Authorization: Bearer`, which is what the OpenAI SDK sends. On the routes that serve a vendor's own protocol it reads that protocol's header too — `x-api-key` on the Anthropic Messages endpoints, `x-goog-api-key` on the Google ones. So all three SDKs authenticate with nothing but their normal `api_key`. ## SDK examples ```python Python theme={null} import os from openai import OpenAI client = OpenAI( base_url="https://api.opper.ai/v3/compat", api_key=os.environ["OPPER_API_KEY"], ) response = client.chat.completions.create( model="openai/gpt-5-mini", messages=[{"role": "user", "content": "Hello"}], ) print(response.choices[0].message.content) ``` ```typescript TypeScript theme={null} import OpenAI from "openai"; const client = new OpenAI({ baseURL: "https://api.opper.ai/v3/compat", apiKey: process.env.OPPER_API_KEY!, }); const response = await client.chat.completions.create({ model: "openai/gpt-5-mini", messages: [{ role: "user", content: "Hello" }], }); console.log(response.choices[0].message.content); ``` Works with `chat.completions.create`, `embeddings.create`, and `responses.create`. ```python Python theme={null} import os from anthropic import Anthropic client = Anthropic( base_url="https://api.opper.ai/v3/compat", api_key=os.environ["OPPER_API_KEY"], ) message = client.messages.create( model="anthropic/claude-sonnet-4-6", max_tokens=1024, messages=[{"role": "user", "content": "Hello"}], ) print(message.content[0].text) ``` ```typescript TypeScript theme={null} import Anthropic from "@anthropic-ai/sdk"; const client = new Anthropic({ baseURL: "https://api.opper.ai/v3/compat", apiKey: process.env.OPPER_API_KEY!, }); const message = await client.messages.create({ model: "anthropic/claude-sonnet-4-6", max_tokens: 1024, messages: [{ role: "user", content: "Hello" }], }); if (message.content[0].type === "text") console.log(message.content[0].text); ``` `client.models.list()` also works, but the model list isn't a Messages route, so it reads only the bearer header. On a client you list models from, pass the key as `auth_token` (`authToken` in TypeScript) instead of `api_key`. ```python Python theme={null} import os from google import genai client = genai.Client( api_key=os.environ["OPPER_API_KEY"], http_options={"base_url": "https://api.opper.ai/v3/compat"}, ) response = client.models.generate_content( model="gemini/gemini-3.1-pro-preview", contents="Hello", ) print(response.text) ``` ```typescript TypeScript theme={null} import { GoogleGenAI } from "@google/genai"; const client = new GoogleGenAI({ apiKey: process.env.OPPER_API_KEY!, httpOptions: { baseUrl: "https://api.opper.ai/v3/compat" }, }); const response = await client.models.generateContent({ model: "gemini/gemini-3.1-pro-preview", contents: "Hello", }); console.log(response.text); ``` ## TypeSafe SDKs Keep the [TypeSafe SDK](https://docs.typesafe.ai/sdk), set its base URL to `https://api.opper.ai/v3/compat`, and use your Opper API key. The SDK sends its native `state` and `questions` format to `/v1/systemone` and returns typed answers with probabilities and confidence. Install `typesafe-sdk` for Python or `@typesafe-ai/sdk` for TypeScript: ```python Python theme={null} import os from typesafe_sdk import Noul, TypeSafeClient with TypeSafeClient( base_url="https://api.opper.ai/v3/compat", api_key=os.environ["OPPER_API_KEY"], ) as client: response = client.system_one( model="typesafe/jev-1.13.0", state="I was charged twice. Please refund the duplicate payment.", questions={ "refund_requested": Noul( instructions="Is the customer requesting a refund?" ), }, ) print(response.nouls["refund_requested"].noul) ``` ```typescript TypeScript theme={null} import { noul, TypeSafeClient } from "@typesafe-ai/sdk"; const client = new TypeSafeClient({ baseURL: "https://api.opper.ai/v3/compat", apiKey: process.env.OPPER_API_KEY!, }); const response = await client.systemOne({ model: "typesafe/jev-1.13.0", state: "I was charged twice. Please refund the duplicate payment.", questions: { refund_requested: noul("Is the customer requesting a refund?"), }, }); console.log(response.answers.refund_requested.noul); ``` Your organization needs a card on file. TypeSafe evaluation models use the System One protocol; use the TypeSafe SDK for these models. See [Classification & scoring](/build/gateway/evaluations) for `noul`, `choice`, and `score`, supported model aliases, and gateway limits. OpenAI's `gpt-6-luna` answers System One too, as `openai/gpt-6-luna-decisions`. ## Cross-vendor models For generation models, the `model` field is provider-prefixed: `openai/gpt-5.5`, `anthropic/claude-sonnet-4-6`, `gemini/gemini-3.1-pro-preview`. You can call these generation models from the OpenAI, Anthropic, or Google AI SDK. Your code calling `client.messages.create(...)` with the Anthropic SDK can still talk to a Google model. Browse the full catalog on the [Models](/capabilities/models) page. ## Opper extras For generation requests, the OpenAI, Anthropic, and Google AI SDKs let you send Opper-specific options through an extras field. Use them to attach tags for filtering traces, link a call to a parent span, or skip automatic scoring. TypeSafe System One requests accept only `model`, `state`, and `questions`; the extras below are unsupported on that endpoint. OpenAI Decisions requests accept only `model`, `input`, `questions`, and `safety_identifier`, so the extras are unsupported there too. | Option | Does this | | - | - | | `tags` | A key/value object for grouping calls in traces. | | `span_uuid` | Attach this call to a parent span you created with the Opper SDK. | | `evaluate` | Set to `false` to skip [score check](/control-plane/rules/checks#score-checks) scoring. | ```python Python with extras theme={null} response = client.chat.completions.create( model="openai/gpt-5-mini", messages=[{"role": "user", "content": "Hello"}], extra_body={ "tags": {"user_id": "u_123"}, "evaluate": False, }, ) ``` **Per-request fallbacks: `models`, on chat completions only.** The Chat Completions endpoint reads the same `models` array OpenRouter does: the primary in `model`, the rest in `models`, tried in order when the one before fails for any reason. Omit `model` and the first entry is the primary. A [compliance rule](/control-plane/rules/overview) that blocks an entry removes it from the chain without an error. Which model answered is in `meta.routing.served` and the `X-Opper-Served-Model` header; the response `model` echoes what you sent. ```python theme={null} client.chat.completions.create( model="openai/gpt-5.6-luna", messages=[{"role": "user", "content": "Hello"}], extra_body={"models": ["azure/gpt-5.6-luna", "deepseek-v4.1-flash"]}, ) ``` The other endpoints on this page do not read `models` or `fallback_models`. For a chain you manage in one place, with ranking, versions and rollback, build a [dynamic route](/capabilities/routes/overview) for generation models and call its name as `model` from the OpenAI, Anthropic, or Google AI SDK on supported generation endpoints. To attribute usage and **cost** per customer or team — not just label traces — send tags with the `X-Opper-Tags` header instead, then group spend by tag in analytics. See [Tags & usage attribution](/build/gateway/usage-attribution). ## All the URLs You usually don't need this. Your SDK appends the right path. For reference: | URL | What it mimics | | - | - | | `POST /v3/compat/chat/completions` | OpenAI Chat Completions | | `POST /v3/compat/responses` | OpenAI Responses API | | `POST /v3/compat/embeddings` | OpenAI Embeddings | | `POST /v3/compat/decisions` | [OpenAI Decisions](/build/gateway/evaluations) | | `POST /v3/compat/v1/messages` | Anthropic Messages | | `GET /v3/compat/v1/models` | Anthropic Models list | | `POST /v3/compat/v1beta/interactions` | Google AI Interactions | | `POST /v3/compat/openresponses` | [OpenResponses](https://openresponses.org) (vendor-neutral) | | `POST /v3/compat/v1/systemone` | [TypeSafe System One](/build/gateway/evaluations) | ## Hostnames: global and EU The same gateway is reachable on three hostnames. All of them take the same auth and serve the same models. | Hostname | What it is | | - | - | | `api.opper.ai/v3/compat` | The canonical URL used above. Served from the EU (Stockholm). | | `gw.opper.ai` | Global edge endpoint — TLS terminates at the CDN edge nearest your callers, then rides an accelerated path to the EU backend. | | `eu.gw.opper.ai` | EU-resident twin of `gw` — TLS terminates only in Stockholm, for hard EU-residency requirements. | The `gw` hosts also serve the vendor paths at cleaner base URLs, so SDK config gets shorter: ```text theme={null} OpenAI SDK: base_url = "https://gw.opper.ai/openai/v1" Anthropic SDK: base_url = "https://gw.opper.ai/anthropic" Google AI SDK: base_url = "https://gw.opper.ai/google" ``` `POST https://gw.opper.ai/chat/completions` and `POST https://gw.opper.ai/v1/messages` also work at the bare root, for tools that take only a hostname and no path. ## Frameworks Use Opper's native provider for `streamText` and `generateObject`. No compat layer needed. Point them at the `/v3/compat` base URL. ## What's next Let the model invoke your tools. Send images and documents in messages. Stream the response token-by-token. Get JSON back, validated against a schema. # Classification & scoring Source: https://docs.opper.ai/build/gateway/evaluations Get probabilities, categories, and scores from decision models through OpenAI Decisions or TypeSafe System One. Decision models answer typed questions about an input with probabilities instead of writing text. Use them to detect intent, route a support request, or score urgency, and let the probability decide what your application does next. Opper serves every decision model through two wire formats: | Endpoint | Format | Answers natively | Input | | - | - | - | - | | `POST /v3/compat/decisions` | [OpenAI Decisions](https://developers.openai.com/api/docs/guides/decisions) | OpenAI's `gpt-6-luna` | Text and inline images for `gpt-6-luna`; text for other models | | `POST /v3/compat/v1/systemone` | [TypeSafe System One](https://docs.typesafe.ai/api) | TypeSafe, Opper, Cloudflare, and Berget models | Text or structured context | A model reached through the other format is translated both ways, so you can keep one client and switch models by name. Both endpoints use your project-scoped Opper API key. Provider credentials are managed by Opper. Your organization needs a card on file, and your project's [model access rules](/control-plane/rules/model-access) and provider access requirements apply. ## Question types | Decisions | System One | Use it for | Answer | | - | - | - | - | | `predicate` | `noul` | A yes/no question | The probability of yes, between 0 and 1. | | `choice` | `choice` | Choosing a category | The chosen value, a `confidence`, and a probability for each option. | | `score` | `score` | Scoring against ordered levels | A probability-weighted `score`, a `confidence`, and a probability for each level. Levels start at zero; the score can be fractional. | You can include several questions in one request. Set a project API key from [the platform](https://platform.opper.ai) for the examples below: ```bash theme={null} export OPPER_API_KEY="op-your-project-api-key" ``` ## OpenAI Decisions Send OpenAI's Decisions format to `POST https://api.opper.ai/v3/compat/decisions`: * `model`: `gpt-6-luna`, or the full catalog ID of another decision model, such as `typesafe/jev-1.13.0`. * `input`: a string, or user messages whose `content` is a string or a list of `input_text` and `input_image` parts. Images must be inline base64 `data:` URLs. An image part takes an optional `detail` of `low`, `high`, `auto`, or `original`. * `questions`: a list of `predicate`, `choice`, and `score` questions. Each requires `type` and `instructions` and takes an optional `name`, unique within the request. `choice` takes 2–255 `choices` whose `value` is a string or a boolean; `score` takes 2–10 ordered `levels`, each with a `label`. * `safety_identifier` (optional): accepted for compatibility with OpenAI's SDK, then dropped. Opper neither forwards nor stores it; requests to OpenAI carry Opper's own identifier for your project. Answers come back in question order, each with its question's `name`. ### Classify a support request ```bash theme={null} curl --fail-with-body -sS \ https://api.opper.ai/v3/compat/decisions \ -H "Authorization: Bearer $OPPER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "gpt-6-luna", "input": "I was charged twice for my October invoice. Please refund one of the two payments today.", "questions": [ { "type": "predicate", "name": "refund", "instructions": "Does the customer ask for a refund?" }, { "type": "choice", "name": "team", "instructions": "Which team should handle this?", "choices": [ {"value": "billing", "description": "Payments and invoices"}, {"value": "technical"}, {"value": "sales"} ] }, { "type": "score", "name": "urgency", "instructions": "How urgent is this?", "levels": [{"label": "Low"}, {"label": "Medium"}, {"label": "High"}] } ] }' ``` Example response: ```json theme={null} { "model": "gpt-6-luna", "answers": [ {"type": "predicate", "name": "refund", "probability": 1.0}, { "type": "choice", "name": "team", "choice": "billing", "probabilities": [ {"value": "billing", "probability": 1.0}, {"value": "technical", "probability": 0.0}, {"value": "sales", "probability": 0.0} ], "confidence": 1.0 }, { "type": "score", "name": "urgency", "score": 1.48, "probabilities": [ {"value": 0, "label": "Low", "probability": 0.02}, {"value": 1, "label": "Medium", "probability": 0.48}, {"value": 2, "label": "High", "probability": 0.5} ], "confidence": 0.22 } ], "usage": { "input_tokens": 391, "input_tokens_details": {"cached_tokens": 0, "cache_write_tokens": 0}, "output_tokens": 0, "output_tokens_details": {"reasoning_tokens": 0}, "total_tokens": 391 } } ``` Choose a probability threshold in your application if you need a Boolean decision. Opper returns the probability without applying a threshold. ### Ask about an image `gpt-6-luna` reads images. This request sends an 8×8 red PNG: ```bash theme={null} curl --fail-with-body -sS \ https://api.opper.ai/v3/compat/decisions \ -H "Authorization: Bearer $OPPER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "gpt-6-luna", "input": [ { "role": "user", "content": [ {"type": "input_text", "text": "Describe this image."}, {"type": "input_image", "image_url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAgAAAAICAIAAABLbSncAAAAEklEQVR4nGP4z8CAFWEXHbQSACj/P8Fu7N9hAAAAAElFTkSuQmCC"} ] } ], "questions": [ {"type": "predicate", "name": "red", "instructions": "Is the image predominantly red?"} ] }' ``` An example `answers`: ```json theme={null} [{"type": "predicate", "name": "red", "probability": 0.98}] ``` Other decision models take text only and return `400` for image input. ### Use the OpenAI SDK The OpenAI Python SDK calls this endpoint from version 3.26.0. Point it at Opper: ```python theme={null} import os from openai import OpenAI client = OpenAI( base_url="https://api.opper.ai/v3/compat", api_key=os.environ["OPPER_API_KEY"], ) decision = client.decisions.create( model="gpt-6-luna", input="I was charged twice for my October invoice. Please refund one of the two payments today.", questions=[ {"type": "predicate", "name": "refund", "instructions": "Does the customer ask for a refund?"}, ], ) print(decision.answers[0].probability) ``` ### Declined questions The model can decline a question. That answer is `{"type": "refusal", "name": "..."}` with no probability, and the other answers are unaffected. The request is billed as usual. ### Other decision models Name any other decision model by its full catalog ID, such as `typesafe/jev-1.13.0` or `opper/kev-4b`. Opper translates the request to System One and the answer back, so the response has the same shape. For these models: * Text parts join into one input, and image input returns `400`. * Boolean choice values reach the model as the text `true` and `false`, so a question cannot hold both the string `"true"` and the boolean `true`. * `usage` reports `input_tokens`, `output_tokens`, and `total_tokens`. The support request above, sent with `"model": "typesafe/jev-1.13.0"`, answers: ```json theme={null} { "model": "jev-1.13.0", "answers": [ {"type": "predicate", "name": "refund", "probability": 0.99}, { "type": "choice", "name": "team", "choice": "billing", "probabilities": [ {"value": "billing", "probability": 1}, {"value": "technical", "probability": 0}, {"value": "sales", "probability": 0} ], "confidence": 1 }, { "type": "score", "name": "urgency", "score": 1.65, "probabilities": [ {"value": 0, "label": "Low", "probability": 0}, {"value": 1, "label": "Medium", "probability": 0.34}, {"value": 2, "label": "High", "probability": 0.66} ], "confidence": 0.49 } ], "usage": {"input_tokens": 377, "output_tokens": 69, "total_tokens": 446} } ``` ## TypeSafe System One Send requests to `POST https://api.opper.ai/v3/compat/v1/systemone`. Existing TypeSafe HTTP clients can change their base URL to `https://api.opper.ai/v3/compat` and authenticate with `Authorization: Bearer $OPPER_API_KEY`. Send the native `state` and `questions` format to `/v1/systemone`. Every request contains: * `model`: a decision model ID, such as `typesafe/jev-1.13.0`. * `state`: the context to evaluate, as a string, object, or array. * `questions`: a nonempty object of named questions. Each question requires `type` and `instructions`. `choice` requires a `criteria` object with 1–255 categories. `score` requires an ordered `criteria` array with 2–10 levels. `noul` does not require criteria; you can optionally describe the `true` and `false` outcomes in a `criteria` object. Answers use the same keys as your questions. ### Detect a refund request ```bash theme={null} curl --fail-with-body -sS \ https://api.opper.ai/v3/compat/v1/systemone \ -H "Authorization: Bearer $OPPER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "typesafe/jev-1.13.0", "state": "I was charged twice for my subscription. Please return the duplicate payment.", "questions": { "refund_requested": { "type": "noul", "instructions": "Is the customer requesting a refund?" } } }' ``` Example response: ```json theme={null} { "model": "jev-1.13.0", "answers": { "refund_requested": { "type": "noul", "noul": 0.98 } }, "usage": { "input_tokens": 287, "output_tokens": 21 } } ``` ### Route a support request ```bash theme={null} curl --fail-with-body -sS \ https://api.opper.ai/v3/compat/v1/systemone \ -H "Authorization: Bearer $OPPER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "typesafe/jev-1.13.0", "state": { "subject": "Cannot authenticate", "message": "Our integration started returning 401 errors after rotating our API key." }, "questions": { "team": { "type": "choice", "instructions": "Choose the team best suited to resolve this request.", "criteria": { "finance": "Charges, payments, and invoices", "engineering": "API and integration issues", "account_management": "Contracts and subscriptions" } } } }' ``` An example `answers.team`: ```json theme={null} { "type": "choice", "choice": "engineering", "confidence": 1.0, "probabilities": { "engineering": 1.0, "account_management": 0.0, "finance": 0.0 } } ``` ### Score urgency ```bash theme={null} curl --fail-with-body -sS \ https://api.opper.ai/v3/compat/v1/systemone \ -H "Authorization: Bearer $OPPER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "typesafe/jev-1.13.0", "state": "Our production checkout is unavailable. Every customer payment has failed for the last 20 minutes.", "questions": { "urgency": { "type": "score", "instructions": "Rate the operational urgency.", "criteria": [ "Low: informational request with no operational impact", "Medium: degraded functionality with a workaround", "High: production outage preventing customer payments" ] } } }' ``` An example `answers.urgency`: ```json theme={null} { "type": "score", "score": 2.0, "confidence": 1.0, "probabilities": { "0": 0.0, "1": 0.0, "2": 1.0 }, "legend": { "0": "Low: informational request with no operational impact", "1": "Medium: degraded functionality with a workaround", "2": "High: production outage preventing customer payments" } } ``` With these three levels, scores range from 0 to 2. ### Use the TypeSafe SDK The [TypeSafe SDK](https://docs.typesafe.ai/sdk) works with Opper unchanged: set its base URL to `https://api.opper.ai/v3/compat` and use your Opper API key. In Python (`typesafe-sdk`): ```python theme={null} import os from typesafe_sdk import Noul, TypeSafeClient with TypeSafeClient( base_url="https://api.opper.ai/v3/compat", api_key=os.environ["OPPER_API_KEY"], ) as client: response = client.system_one( model="typesafe/jev-1.13.0", state="I was charged twice. Please refund the duplicate payment.", questions={ "refund_requested": Noul(instructions="Is the customer requesting a refund?"), }, ) print(response.nouls["refund_requested"].noul) ``` The TypeScript SDK (`@typesafe-ai/sdk`) works the same way; see [Drop-in SDKs](/build/gateway/drop-in-sdks#typesafe-sdks). ### OpenAI's model on System One `gpt-6-luna` answers System One requests as `openai/gpt-6-luna-decisions`, text only. Opper sends structured `state` and `instructions` as compact JSON text and adds a `noul` question's `true` and `false` criteria to its instructions. A `choice` needs at least two categories. If the model declines a question, the request returns `422` naming it, and the call is billed because OpenAI bills it. Answers and token counts can vary between calls. ## Choose a model Discover decision models with: ```bash theme={null} curl --fail-with-body -sS "https://api.opper.ai/v3/models?type=evaluation" ``` A bare model name means OpenAI's model on Decisions (`gpt-6-luna`) and TypeSafe's on System One (`jev-1.13.0`, `jev-latest`, and `jev-preview` resolve to TypeSafe catalog entries). Use the full catalog ID for every other model on either endpoint, such as `typesafe/jev-1.13.0`, `opper/kev-4b`, or `openai/gpt-6-luna-decisions`. The qualified aliases `typesafe/jev-latest` and `typesafe/jev-preview` are supported too. Aliases follow Opper's approved catalog mapping. They do not automatically follow a new upstream TypeSafe release. The response's `model` identifies the upstream model that answered. Jev 1.13 has a 64k total context limit, with a separate 32k limit for the state plus the longest question. TypeSafe enforces these limits using its own tokenizer. Decision models, the catalog IDs `GET /v3/models?type=evaluation` lists (such as `typesafe/jev-1.13.0` and `openai/gpt-6-luna-decisions`), answer only on these two endpoints. Chat Completions and the other generation protocols do not accept them. A bare `gpt-6-luna` on those protocols is OpenAI's chat model, which works as usual. Both decision endpoints are synchronous; streaming, tools, and fields outside each format are unsupported. ## Usage, cost, and traces `gpt-6-luna` on Decisions returns OpenAI's response body unchanged, including its `usage` details. Translated Decisions responses carry `usage.input_tokens`, `usage.output_tokens`, and `usage.total_tokens`; System One responses carry `usage.input_tokens` and `usage.output_tokens`. Opper does not add a response envelope. Billing uses the selected endpoint's catalog input and output token prices, including any long-context surcharge the catalog lists. See the [model catalog](https://opper.ai/models) for current pricing. Add `-i` to a curl call to see these response headers: | Header | Meaning | | - | - | | `X-Opper-Cost` / `X-Generation-Cost` | The call's cost in USD. | | `X-Generation-Id` | Opper's generation ID. | | `X-Opper-Trace-Id` | The trace UUID. | Usage and billing are recorded for successful calls and for questions the model declined. Full input/output traces depend on your [Opper retention rule](/control-plane/rules/retention); in traces, each inline image appears as a placeholder with its size. Provider data policies are configured separately through [model access](/control-plane/rules/model-access); EU gateway hosting does not make every provider EU-hosted. ## Errors | Status | Meaning | | - | - | | `400` | An invalid Decisions request, image input for a text-only model, an unknown, unavailable, or wrong-type model, or an upstream bad request. | | `401` | Missing or invalid project API key. | | `402` | Your organization has no card on file, or billing access is blocked. | | `403` | Model access rules or required provider access prevent the request. | | `413` | A Decisions request body over 50 MiB, or an oversized request the upstream provider rejected. | | `422` | An invalid System One request, unsupported question types or fields, an upstream validation failure, or a question the model declined on System One. | | `429` | Upstream rate limiting. For `gpt-6-luna`, Opper makes one attempt and passes OpenAI's `Retry-After` header through. | | `502` | An upstream service or authentication failure prevented completion. | | `503` | The provider or model configuration is unavailable. | | `504` | The request timed out. | | `529` | TypeSafe is temporarily overloaded. | See the API reference for [Decisions](/v3-api-reference/compatibility/create-decision) and [System One](/v3-api-reference/compatibility/systemone), [OpenAI's Decisions guide](https://developers.openai.com/api/docs/guides/decisions), or [TypeSafe's API documentation](https://docs.typesafe.ai/api) for the native protocols. # Prompt caching Source: https://docs.opper.ai/build/gateway/prompt-caching Reuse a large, stable prompt prefix across calls to cut cost and latency on Anthropic models. Opt-in, one field. When you send the same long prefix — a big system prompt, a document, a tool catalog — on call after call, prompt caching lets the provider store that prefix and skip re-processing it next time. On Anthropic (Claude) models this is a real, billable feature: a **cache read costs 0.1×** the normal input price, so a reused prefix gets \~90% cheaper and comes back faster. It isn't free, though. Writing the cache costs **1.25×** input, and Anthropic's cache entries expire after **5 minutes** (extendable to 1 hour). So caching only pays off when the same prefix comes back **within the cache lifetime**. If your calls are minutes apart with no reuse, you'd pay the 1.25× write premium every time and never collect the discount. Because of that trade-off, **Opper leaves prompt caching off by default** and lets you turn it on per request. You decide, because only you know whether your prefix repeats. This page is about **Anthropic** prompt caching. **OpenAI** and **Gemini** cache automatically, server-side, at no extra cost and with no flag — there's nothing to enable. The `cache_control` field below is a no-op on those providers, so the same request stays portable across models. ## Enable it: top-level `cache_control` Add a top-level `cache_control` to the request body. Opper places a single **moving breakpoint** on the largest cacheable prefix and advances it as the conversation grows — the "automatic" mode, best for multi-turn chats and agent loops. ```python OpenAI SDK theme={null} import os from openai import OpenAI client = OpenAI( base_url="https://api.opper.ai/v3/compat", api_key=os.environ["OPPER_API_KEY"], ) resp = client.chat.completions.create( model="anthropic/claude-sonnet-4-6", messages=[ {"role": "system", "content": LONG_STABLE_SYSTEM_PROMPT}, {"role": "user", "content": "What changed in the latest revision?"}, ], extra_body={"cache_control": {"type": "ephemeral"}}, ) ``` ```typescript OpenAI SDK theme={null} import OpenAI from "openai"; const client = new OpenAI({ baseURL: "https://api.opper.ai/v3/compat", apiKey: process.env.OPPER_API_KEY, }); const resp = await client.chat.completions.create({ model: "anthropic/claude-sonnet-4-6", messages: [ { role: "system", content: LONG_STABLE_SYSTEM_PROMPT }, { role: "user", content: "What changed in the latest revision?" }, ], // @ts-expect-error — Opper extension, passed through to the model cache_control: { type: "ephemeral" }, }); ``` ```bash cURL theme={null} curl https://api.opper.ai/v3/compat/chat/completions \ -H "Authorization: Bearer $OPPER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "anthropic/claude-sonnet-4-6", "cache_control": { "type": "ephemeral" }, "messages": [ { "role": "system", "content": "…long stable system prompt…" }, { "role": "user", "content": "What changed in the latest revision?" } ] }' ``` The top-level field works on every Anthropic-routable compat endpoint: `chat/completions`, `v1/messages`, `responses`, `openresponses`, and the Google-shape `interactions` / `generateContent`. ### Extend the lifetime to 1 hour Add a `ttl`. Useful when your prefix is reused over minutes rather than seconds (a long research session, a slow agent). ```json theme={null} { "cache_control": { "type": "ephemeral", "ttl": "1h" } } ``` ## Precise placement: per-block `cache_control` For fine control over exactly where the cached prefix ends, put `cache_control` directly on a content block instead of the top level. Anthropic allows up to **four** such breakpoints. This is the same syntax the Anthropic API uses natively, so existing code works unchanged. Per-block placement is supported on **`chat/completions`** (OpenAI shape) and **`v1/messages`** (Anthropic shape): ```python OpenAI SDK theme={null} resp = client.chat.completions.create( model="anthropic/claude-sonnet-4-6", messages=[ { "role": "user", "content": [ { "type": "text", "text": BIG_DOCUMENT, # cache everything up to here "cache_control": {"type": "ephemeral"}, }, {"type": "text", "text": "Summarize the section on billing."}, ], } ], ) ``` ```python Anthropic SDK theme={null} from anthropic import Anthropic client = Anthropic( base_url="https://api.opper.ai/v3/compat", api_key=os.environ["OPPER_API_KEY"], ) message = client.messages.create( model="anthropic/claude-sonnet-4-6", max_tokens=1024, system=[ { "type": "text", "text": LONG_STABLE_SYSTEM_PROMPT, "cache_control": {"type": "ephemeral"}, # cache the system prompt } ], messages=[{"role": "user", "content": "What changed in the latest revision?"}], ) ``` If you set both a top-level `cache_control` and explicit per-block breakpoints, the **per-block breakpoints win** — Opper won't add its automatic one on top. ## When it's worth it A large, stable prefix (system prompt, document, tool catalog) reused across back-to-back calls — agent loops, multi-turn chat, batch jobs over one context. One-shot calls, or calls spaced further apart than the cache lifetime. You'd pay the 1.25× write premium with no read to recoup it. Every response reports what actually happened via its usage: `cache_creation_input_tokens` (written at 1.25×) and `cache_read_input_tokens` (read at 0.1×). Watch those to confirm caching is paying off — reads should dominate writes once your prefix starts repeating. ## Reference * **Field:** `cache_control` — top-level (automatic) or on a content block (explicit). * **Value:** `{ "type": "ephemeral" }`, optionally `"ttl": "1h"` (default is 5 minutes). * **Activates on:** Anthropic-family models. No-op on OpenAI / Gemini (they cache automatically). * **Pricing:** cache write 1.25× input, cache read 0.1× input. * **Breakpoints:** up to 4 explicit per-block; or 1 automatic top-level. # Rerank Source: https://docs.opper.ai/build/gateway/rerank Rank retrieved documents by relevance to a query before passing them to a model. `POST /v3/rerank` scores a list of documents against a query and returns the most relevant first. Use it after retrieving candidate passages, then pass the highest-ranked results into your model's context. Authenticate with a project API key. Reranking needs a card on file; organizations without one get `402`. ## Rerank documents Send a model, a query, and up to 1,000 document strings. Set `top_n` to limit the results and `return_documents` to include their text. ```bash cURL theme={null} curl https://api.opper.ai/v3/rerank \ -H "Authorization: Bearer $OPPER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "bge-reranker-v2-m3", "query": "How do I reset my password?", "documents": [ "Billing and invoices", "Use the forgot-password link to reset your password", "Release notes" ], "top_n": 1, "return_documents": true }' ``` ```python Python theme={null} import os import requests response = requests.post( "https://api.opper.ai/v3/rerank", headers={"Authorization": f"Bearer {os.environ['OPPER_API_KEY']}"}, json={ "model": "bge-reranker-v2-m3", "query": "How do I reset my password?", "documents": [ "Billing and invoices", "Use the forgot-password link to reset your password", "Release notes", ], "top_n": 1, "return_documents": True, }, timeout=65, ) response.raise_for_status() for result in response.json()["results"]: print(result["index"], result["relevance_score"], result["document"]["text"]) ``` ```typescript TypeScript theme={null} const response = await fetch("https://api.opper.ai/v3/rerank", { method: "POST", headers: { Authorization: `Bearer ${process.env.OPPER_API_KEY}`, "Content-Type": "application/json", }, body: JSON.stringify({ model: "bge-reranker-v2-m3", query: "How do I reset my password?", documents: [ "Billing and invoices", "Use the forgot-password link to reset your password", "Release notes", ], top_n: 1, return_documents: true, }), }); if (!response.ok) throw new Error(await response.text()); const { results } = await response.json(); for (const result of results) { console.log(result.index, result.relevance_score, result.document.text); } ``` The response contains `results`, a generation `id`, the `model` endpoint that served the request, and `usage`. Each result's `index` points to its original position in your `documents` array, even after sorting. For this example, the password-reset passage at index `1` ranks first. | Field | Behavior | | - | - | | `model` | Required. A rerank model name or a fully qualified endpoint ID. | | `query` | Required. A nonblank search query. | | `documents` | Required. An array of 1–1,000 strings. Providers may impose additional document-length limits. | | `top_n` | Optional integer, at least 1. Omit to return all documents. | | `return_documents` | Defaults to `false`. Set to `true` to include `results[].document.text`; otherwise use `results[].index` to retrieve the original text yourself. | | `top_k` | Compatibility alias for `top_n`. Zero means unspecified. If both fields have positive values, they must match; conflicting values return `400`. | ## Choose a model List available rerank models with `GET /v3/models?type=rerank`: ```bash theme={null} curl "https://api.opper.ai/v3/models?type=rerank" \ -H "Authorization: Bearer $OPPER_API_KEY" ``` A bare name, such as `bge-reranker-v2-m3` or `rerank-v3.5`, selects from that model's pooled endpoints. If an endpoint fails and another eligible endpoint exists, Opper tries the next one. A fully qualified ID, such as `berget/bge-reranker-v2-m3` or `aws/cohere/rerank-v3.5`, pins one endpoint and does not fall back. The project's model access rules and provider entitlements apply to every candidate, including fallback endpoints. The response's `model` identifies the endpoint that answered. ## Usage and cost `usage.total_tokens` contains the provider's reported token count. Rerank tokens are recorded as input tokens; no output text tokens are generated. Providers that do not report tokens return `0`, which does not mean the call is free. Billing follows the selected endpoint's catalog price: per input token for Berget BGE, or per call for Bedrock Cohere. `usage.cost` and the `X-Generation-Cost` response header report the cost in USD. A successful request is billed for the endpoint that served it. Limiting the returned results with `top_n` does not limit the candidate documents sent for scoring. ## Errors | Status | Meaning | | - | - | | `400` | Invalid input, an unknown or wrong-type model, or conflicting `top_n` and `top_k` values. | | `401` | Missing or invalid project API key. | | `402` | Your organization has no card on file, or billing access is blocked. | | `403` | Model access rules or a required provider agreement prevent the request. | | `429` | The upstream provider rate-limited the request and no fallback succeeded. | | `502` | An upstream service failure prevented completion. | | `504` | The request timed out. | Use the rerank URL directly: `https://api.opper.ai/v3/rerank`. The `/v3/compat` base URL used for chat completions does not expose a rerank route. See the [API reference](/v3-api-reference/rerank/rerank-documents) for the complete request and response schema. # Provider server-side tools Source: https://docs.opper.ai/build/gateway/server-tools Use Anthropic, OpenAI, and Google server-side tools (web search, code execution, grounding) through Opper. Server-side tools are tools the *provider* runs for you — Anthropic's `web_search`, OpenAI's `code_interpreter`, Google's `googleSearch`, and so on. Unlike [function tools](/build/gateway/tools), there's no round-trip: the model invokes the tool, the provider runs it, and the result comes back in the same response. Opper forwards these tools verbatim to each provider on that provider's native endpoint. Surcharges (e.g. \$10 / 1k web searches) bill as the provider reports them. ## Where each tool lives Server-side tools are endpoint-specific — each provider only accepts its own tools on its own compatibility endpoint. Send them anywhere else and you get a 400 with a hint pointing at the right endpoint. | Provider | Endpoint | Tools | | - | - | - | | Anthropic | `POST /v3/compat/v1/messages` | `web_search_*`, `web_fetch_*`, `bash_*`, `computer_*`, `memory_*`, `text_editor_*`, `mcp_toolset`, `advisor_*` | | OpenAI | `POST /v3/compat/responses` | `web_search`, `file_search`, `code_interpreter`, `image_generation` | | Google | `POST /v3/compat/v1beta/models/{model}:generateContent` | `googleSearch`, `codeExecution`, `urlContext`, `googleSearchRetrieval` | Function tools (your own `{type:"function",...}` definitions) work on every endpoint — the restriction is just for provider-native server tools. ## Examples ```bash Anthropic web_search theme={null} curl https://api.opper.ai/v3/compat/v1/messages \ -H "Authorization: Bearer $OPPER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "anthropic/claude-sonnet-4-6", "max_tokens": 1024, "messages": [{"role": "user", "content": "What were the top tech headlines today?"}], "tools": [{ "type": "web_search_20250305", "name": "web_search", "max_uses": 3 }] }' ``` ```bash OpenAI web_search theme={null} curl https://api.opper.ai/v3/compat/responses \ -H "Authorization: Bearer $OPPER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "openai/gpt-5.5", "input": "What were the top tech headlines today?", "tools": [{"type": "web_search"}] }' ``` ```bash Google grounding theme={null} curl "https://api.opper.ai/v3/compat/v1beta/models/gemini/gemini-3.8-flash:generateContent" \ -H "Authorization: Bearer $OPPER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "contents": [{"role":"user","parts":[{"text":"Who won the latest F1 race?"}]}], "tools": [{"googleSearch": {}}] }' ``` ## Responses Each provider's native response shape comes back verbatim, so the official Anthropic / OpenAI / Google SDKs parse it without changes: * **Anthropic** — `content[]` contains `server_tool_use`, `web_search_tool_result` (with the result URLs + encrypted content), and `text` blocks whose `citations[]` reference the sources the model actually grounded on. * **OpenAI Responses** — `output[]` contains `web_search_call` items (with the search queries) and `message.content[].output_text.annotations[]` `url_citation` entries. * **Google** — `candidates[0].groundingMetadata` carries `webSearchQueries`, `groundingChunks[].web` (URI + title), `groundingSupports`, and `searchEntryPoint`. ### Compact responses Server-tool payloads can get heavy (Anthropic `web_search` results carry full snippets + per-result `encrypted_content`). Send `X-Opper-Compact-Response: true` to strip the bandwidth-heavy fields while preserving citation URLs and titles. Tradeoff: a compact response **can't be replayed** to extend a cited multi-turn conversation, because Anthropic citation continuity needs the `encrypted_index` / `encrypted_content` that compact mode drops. ## Routing through Bedrock and Vertex When you route Claude through a cloud platform (`aws/claude-*`, `vertexai/claude-*`), the supported subset shrinks. Opper rejects with a clear 400 when you ask for something the cloud doesn't carry. | Tool | Direct Anthropic | Vertex AI Claude | Bedrock Claude | | - | - | - | - | | `web_search_*` | ✓ | ✓ | ✗ | | `web_fetch_*` | ✓ | ✗ | ✗ | | `bash_*` / `computer_*` / `memory_*` / `text_editor_*` | ✓ | ✓ | ✓ | | `code_execution_*` | ✓ | ✗ | ✗ | | `mcp_toolset` / `advisor_*` | ✓ | ✗ | ✗ | Source: Anthropic's [Features not supported](https://platform.claude.com/docs/en/api/claude-on-vertex-ai) lists on each platform's docs page. ## Provider docs For the full parameter set on each tool (filters, citation options, region settings, etc.), consult the provider's own reference: * [Anthropic — tool reference](https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-reference) * [OpenAI — Responses API tools](https://platform.openai.com/docs/guides/tools) * [Google — generateContent tools](https://ai.google.dev/gemini-api/docs/grounding) ## See also * **[Web search](/build/gateway/web-search)** — the portable `opper:web_search` tool: one tool entry that works on every endpoint, routes to native when supported and to Opper's engine when not. * **[server-tools-compare](https://github.com/opper-ai/opper-cookbook/tree/main/examples/server-tools-compare)** — runnable cookbook example: fans the same question out to all three providers in parallel and shows queries, citations, cost, and the compact-mode size delta side by side. # Service tiers Source: https://docs.opper.ai/build/gateway/service-tiers Trade speed for price per request: flex for a discount, priority for faster tokens. One field, the same as on OpenAI. Several providers sell the same model at more than one processing tier. **Flex** runs your request on spare capacity at a discount (20% to 50% off, depending on the provider), in exchange for slower and less predictable responses. **Priority** puts your request ahead of standard traffic for faster tokens, at a premium. Opper exposes these tiers with the same `service_tier` field OpenAI uses, on every provider that offers them. | Tier | What you get | Typical price | | - | - | - | | `flex` | Spare capacity. Responses can queue for minutes, and requests can be refused when capacity runs out. | 50% to 80% of standard | | `priority` (alias `fast`) | Scheduled ahead of standard traffic, or served on faster hardware: lower latency, faster output. | 1.5x to 2x standard | | `ultrafast` | OpenAI's fastest tier, on selected models. | About 6x standard | | `default` | The standard tier. Same as leaving the field out. | Standard | ## Request a tier Add `service_tier` to the request body: ```python OpenAI SDK theme={null} import os from openai import OpenAI client = OpenAI( base_url="https://api.opper.ai/v3/compat", api_key=os.environ["OPPER_API_KEY"], ) resp = client.chat.completions.create( model="gpt-5.4-mini", service_tier="flex", messages=[{"role": "user", "content": "Summarize this report."}], timeout=600, # flex requests can queue, allow a long timeout ) print(resp.service_tier) # "flex" ``` ```typescript OpenAI SDK theme={null} import OpenAI from "openai"; const client = new OpenAI({ baseURL: "https://api.opper.ai/v3/compat", apiKey: process.env.OPPER_API_KEY, timeout: 600_000, // flex requests can queue, allow a long timeout }); const resp = await client.chat.completions.create({ model: "gpt-5.4-mini", service_tier: "flex", messages: [{ role: "user", content: "Summarize this report." }], }); console.log(resp.service_tier); // "flex" ``` ```bash cURL theme={null} curl https://api.opper.ai/v3/compat/chat/completions \ -H "Authorization: Bearer $OPPER_API_KEY" \ -H "Content-Type: application/json" \ --max-time 600 \ -d '{ "model": "gpt-5.4-mini", "service_tier": "flex", "messages": [{ "role": "user", "content": "Summarize this report." }] }' ``` The field works on `chat/completions`, `responses`, `openresponses` and `v1/messages`. On `v1/messages`, Anthropic's `speed: "fast"` is accepted too and means `priority`. `auto`, `default`, `standard`, `standard_only` and `scale` all mean the standard tier, so clients that send them (Codex does, for example) keep working. Any other value is a `400`. ### Or name the tier endpoint Each tier is its own endpoint in the catalog, so you can also call it by id, with no `service_tier` field: ```json theme={null} {"model": "openai:flex/gpt-5.4-mini", "messages": [...]} ``` Tier endpoint ids follow `[:]:/`, for example `openai:priority/gpt-5.4`, `gemini:flex/gemini-3.8-flash` or `vertexai:eu:priority/gemini-3.6-flash`. Tier endpoints are never picked for a request that doesn't ask for a tier: calling `gpt-5.4-mini` without `service_tier` always runs at the standard tier. ## What happens when the tier isn't available | You ask for | Opper tries | If none answers | | - | - | - | | `flex` | the flex endpoint | the request fails (for example `429` or `503`). It never silently runs at the full price. | | `priority` | the priority endpoint, then the standard one | standard is the fallback, billed at the standard price | | `ultrafast` | ultrafast, then priority, then standard | standard is the fallback | If the model has no endpoint at the tier you asked for, the request runs at the standard tier and the response says `default`. Asking for `flex` never fails just because a model lacks it. ### Fallbacks Each tier is its own model id (`openai:flex/gpt-5.4-mini`), so it goes into your usual fallbacks like any other model: * **The [`models` array](/capabilities/models#fallbacks-and-backup-chains)** (Chat Completions): flex first, the same model at standard if flex has no capacity. ```json theme={null} { "model": "openai:flex/gpt-5.4-mini", "models": ["openai/gpt-5.4-mini"], "messages": [...] } ``` * **A [dynamic route](/capabilities/routes/overview)**, on every endpoint: put both endpoints in a [Pool](/capabilities/routes/pool) ordered **Cheapest first**. The flex endpoint ranks first on price and the standard one takes over when flex is out of capacity. Name the tier endpoints rather than setting `service_tier` in a chain: with `service_tier` set, every entry moves to its own tier endpoint when it has one (as on OpenAI and OpenRouter), so the fallback above would run on flex too. ## See which tier served the request You are billed at the tier that actually ran. Providers sometimes serve a priority request at the standard tier when they're out of priority capacity; Opper reads that back from the provider and bills the standard price. * Header: `X-Opper-Served-Service-Tier: flex | priority | ultrafast | default`, on every response. * Body: `service_tier` on `chat/completions`, `responses` and `openresponses` when you asked for a tier or named a tier endpoint. Chat streams carry it on every chunk; `meta.routing.served.service_tier` names it too. * On a stream, the header is written before the provider confirms the tier, so treat the final chunk (or `response.completed`) as the authoritative answer. ## Timeouts for flex Flex requests can wait in the provider's queue for minutes before the first token (Google quotes 1 to 15 minutes). Opper waits up to 10 minutes for the first token of a flex request instead of its usual first-token limit, so set your client timeout to at least `600` seconds. Use flex for background work: evaluations, enrichment, batch-like jobs that still need a synchronous answer. ## Find the models that offer a tier Which models sell a tier changes as providers add them, so the catalog is the source of truth. Every tier endpoint in [`GET /v3/models`](/v3-api-reference/models/list-models) carries its tier as `service_tier`, its own `pricing` and its `region`. Filter with `?service_tier=`: ```bash theme={null} # Every model you can run at flex curl "https://api.opper.ai/v3/models?service_tier=flex&limit=0" # Priority endpoints that run in the EU curl "https://api.opper.ai/v3/models?service_tier=priority&inference_location=EU&limit=0" # One model's endpoints: standard and every tier curl "https://api.opper.ai/v3/models?model=kimi-k3&limit=0" ``` `service_tier` takes `flex`, `priority` (or `fast`), `ultrafast` and `default` (the standard endpoints), comma-separated. It combines with the other filters on the endpoint, such as `provider`, `model` and the data-handling filters. To list the ids with their prices: ```bash theme={null} curl -s "https://api.opper.ai/v3/models?service_tier=flex&limit=0" \ | jq -r '.models[] | "\(.id)\t\(.region)\tin \(.pricing.input[0])\tout \(.pricing.output[0])"' ``` The Models page in the [Opper platform](https://platform.opper.ai) shows the same endpoints, each with a tier mark and its own price. Tier endpoints follow your [compliance rules](/control-plane/rules/overview) like any other endpoint. If your project only allows EU processing, add `inference_location=EU` to the query above to see the tiers you can use. # Streaming Source: https://docs.opper.ai/build/gateway/streaming Stream chat responses token-by-token over Server-Sent Events. Streaming shows the response as it's generated. Set `stream: true` and you get an iterator of chunks instead of waiting for the whole response. This works the same way it does in the OpenAI SDK. If you've streamed an OpenAI completion before, you've already written this code. ## A working example ```python Python theme={null} import os from openai import OpenAI client = OpenAI( base_url="https://api.opper.ai/v3/compat", api_key=os.environ["OPPER_API_KEY"], ) stream = client.chat.completions.create( model="openai/gpt-5-mini", messages=[{"role": "user", "content": "Write a haiku about Tuesday."}], stream=True, ) for chunk in stream: delta = chunk.choices[0].delta.content if delta: print(delta, end="", flush=True) print() ``` ```typescript TypeScript theme={null} import OpenAI from "openai"; const client = new OpenAI({ baseURL: "https://api.opper.ai/v3/compat", apiKey: process.env.OPPER_API_KEY!, }); const stream = await client.chat.completions.create({ model: "openai/gpt-5-mini", messages: [{ role: "user", content: "Write a haiku about Tuesday." }], stream: true, }); for await (const chunk of stream) { const delta = chunk.choices[0].delta.content; if (delta) process.stdout.write(delta); } process.stdout.write("\n"); ``` ```bash cURL theme={null} curl https://api.opper.ai/v3/compat/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPPER_API_KEY" \ -d '{ "model": "openai/gpt-5-mini", "messages": [{"role": "user", "content": "Write a haiku about Tuesday."}], "stream": true }' ``` ## What's in each chunk Each streamed chunk looks like a normal chat completion, except: * `choices[0].delta` carries the new content (instead of `choices[0].message`). * `delta.content` is the next piece of text. May be empty on some chunks. * `delta.tool_calls` carries fragments of a tool call as the model assembles it (see below). * `finish_reason` is `null` while streaming. The final chunk sets it to `stop`, `length`, `tool_calls`, or `content_filter`. ## Streaming tool calls When the model decides to call a tool, the arguments arrive as JSON fragments across multiple chunks. Concatenate them as they come in, then parse once the call is complete. ```python Python theme={null} import json from collections import defaultdict stream = client.chat.completions.create( model="openai/gpt-5-mini", messages=messages, tools=tools, stream=True, ) # Accumulate tool args by index args_by_index = defaultdict(str) tool_call_by_index = {} for chunk in stream: for tc in chunk.choices[0].delta.tool_calls or []: tool_call_by_index[tc.index] = tc if tc.function and tc.function.arguments: args_by_index[tc.index] += tc.function.arguments # After the loop, each entry in args_by_index is a complete JSON string for index, tc in tool_call_by_index.items(): if tc.function and tc.function.name: parsed = json.loads(args_by_index[index]) print(f"{tc.function.name}({parsed})") ``` This lets you show a "calling search…" UI as the call assembles, rather than waiting for the model to finish. ## Errors mid-stream If something goes wrong after streaming has started (a [check](/control-plane/rules/checks) rejects the output, the upstream model fails, a timeout fires), the stream ends with an error chunk. Always handle exceptions around the iteration. ## What's next The full tool-use round trip. Multi-turn chat with message history. Stream toward a schema-validated JSON object. # Structured output Source: https://docs.opper.ai/build/gateway/structured-output Constrain a response to a JSON Schema. Get back validated, typed JSON instead of free text. When you want a response you can parse rather than a sentence, set `response_format` to a JSON Schema. The model produces JSON that matches it, and the gateway validates the response against the schema before returning it. This is the standard OpenAI-compatible parameter, so it works from any SDK you point at Opper. Reach for structured output to parse a receipt, extract fields from a contract, classify a support ticket, or any job where you describe the shape you want back. ## A working example ```python Python theme={null} import os, json from openai import OpenAI client = OpenAI( base_url="https://api.opper.ai/v3/compat", api_key=os.environ["OPPER_API_KEY"], ) r = client.chat.completions.create( model="openai/gpt-5-mini", messages=[ {"role": "user", "content": "Pick a category for: 'My reset link is broken.'"} ], response_format={ "type": "json_schema", "json_schema": { "name": "ticket", "schema": { "type": "object", "properties": { "category": {"type": "string"}, "priority": {"type": "string"}, "confidence": {"type": "number"}, }, "required": ["category", "priority", "confidence"], }, }, }, ) result = json.loads(r.choices[0].message.content) print(result["category"], result["priority"]) ``` ```typescript TypeScript theme={null} import OpenAI from "openai"; const client = new OpenAI({ baseURL: "https://api.opper.ai/v3/compat", apiKey: process.env.OPPER_API_KEY!, }); const r = await client.chat.completions.create({ model: "openai/gpt-5-mini", messages: [ { role: "user", content: "Pick a category for: 'My reset link is broken.'" }, ], response_format: { type: "json_schema", json_schema: { name: "ticket", schema: { type: "object", properties: { category: { type: "string" }, priority: { type: "string" }, confidence: { type: "number" }, }, required: ["category", "priority", "confidence"], }, }, }, }); const result = JSON.parse(r.choices[0].message.content!); console.log(result.category, result.priority); ``` ```bash cURL theme={null} curl https://api.opper.ai/v3/compat/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPPER_API_KEY" \ -d '{ "model": "openai/gpt-5-mini", "messages": [{"role": "user", "content": "Pick a category for: '\''My reset link is broken.'\''"}], "response_format": { "type": "json_schema", "json_schema": { "name": "ticket", "schema": { "type": "object", "properties": { "category": {"type": "string"}, "priority": {"type": "string"}, "confidence": {"type": "number"} }, "required": ["category", "priority", "confidence"] } } } }' ``` The `content` field comes back as a JSON string. Parse it once to get a typed object. If the model can't produce something that matches the schema, the call fails with a validation error rather than silently returning malformed JSON. The Anthropic and Google AI SDKs reach the same behaviour through their own structured-output options. Whichever SDK you use, the gateway validates the result against your schema. See [Drop-in SDKs](/build/gateway/drop-in-sdks). ## Writing the schema The schema is plain JSON Schema. A handful of patterns cover almost everything you'll need. ### Enums (one of a fixed set) When the answer is one of a fixed set of values, use an enum. The model is far more reliable on these than on free strings. ```json theme={null} "priority": { "type": "string", "enum": ["low", "medium", "high"] } ``` ### Optional fields List required fields in the `required` array. Anything left out can come back as `null`. ```json theme={null} { "type": "object", "properties": { "name": { "type": "string" }, "email": { "type": "string" } }, "required": ["name"] } ``` ### Arrays ```json theme={null} "items": { "type": "array", "items": { "type": "string" } } ``` ### Nested objects ```json theme={null} { "type": "object", "properties": { "merchant": { "type": "string" }, "items": { "type": "array", "items": { "type": "object", "properties": { "description": { "type": "string" }, "price": { "type": "number" } }, "required": ["description", "price"] } }, "total": { "type": "number" } }, "required": ["merchant", "items", "total"] } ``` ### Field descriptions Describe ambiguous fields. The model reads descriptions as guidance. ```json theme={null} "priority": { "type": "string", "enum": ["low", "medium", "high"], "description": "Use 'high' when the user can't access their account." } ``` ## Structured output from images, PDFs, and audio Structured output composes with multimodal input. Put the image, PDF, or audio in the message content and set `response_format` to the shape you want back. See [Vision & PDFs](/build/multimodal/vision-pdfs) for how to attach media. ## When to use what | Need | Reach for | | - | - | | Typed JSON from a model | `response_format: json_schema` (this page) | | Model that calls your code mid-conversation | [Tool calling](/build/gateway/tools) | | Multimodal input with structured output | [Vision & PDFs](/build/multimodal/vision-pdfs) + `response_format` | ## What's next When the model should call your code, not just answer. Send images and documents, get structure back. Stream the response as it's generated. Inspect calls and manage the rules Opper enforces. # Tool calling Source: https://docs.opper.ai/build/gateway/tools Let the model decide when to call functions you define. The standard tool-use round-trip. Tools turn a chat from "the model writes you a sentence" into "the model can do things." You hand it a list of functions it's allowed to call. It decides if and when to call them, you run the actual code, and you feed the result back into the conversation. Opper's tool calling uses the standard OpenAI-compatible `tools` array, so anything you've written against OpenAI, Anthropic, or other compat SDKs works unchanged. It works with every model in the catalog that has the `tools` capability. ## The round trip A tool call always follows the same shape: 1. You send a chat completion request with a `tools` list. 2. The model responds with a `tool_use` (it wants to call one). 3. You run the tool in your code. 4. You send the result back to the model as a `tool_result` message. 5. The model uses the result to write its final response. ## A working example A weather assistant. ```python Python theme={null} import os, json from openai import OpenAI client = OpenAI( base_url="https://api.opper.ai/v3/compat", api_key=os.environ["OPPER_API_KEY"], ) # 1. Declare the tool tools = [{ "type": "function", "function": { "name": "get_weather", "description": "Get the current weather for a city.", "parameters": { "type": "object", "properties": { "city": {"type": "string", "description": "City name, e.g. 'Stockholm'"} }, "required": ["city"], }, }, }] # 2. Your local implementation def get_weather(city: str) -> str: return f"Sunny, 22°C in {city}" # 3. First call. Model decides to use the tool. messages = [{"role": "user", "content": "What's the weather in Stockholm?"}] r = client.chat.completions.create( model="openai/gpt-5-mini", messages=messages, tools=tools, ) msg = r.choices[0].message messages.append(msg) # 4. Run the tool, append the result for call in (msg.tool_calls or []): args = json.loads(call.function.arguments) result = get_weather(**args) messages.append({ "role": "tool", "tool_call_id": call.id, "content": result, }) # 5. Second call. Model uses the result to answer. r = client.chat.completions.create( model="openai/gpt-5-mini", messages=messages, tools=tools, ) print(r.choices[0].message.content) ``` ```typescript TypeScript theme={null} import OpenAI from "openai"; const client = new OpenAI({ baseURL: "https://api.opper.ai/v3/compat", apiKey: process.env.OPPER_API_KEY!, }); const tools = [{ type: "function" as const, function: { name: "get_weather", description: "Get the current weather for a city.", parameters: { type: "object", properties: { city: { type: "string", description: "City name, e.g. 'Stockholm'" }, }, required: ["city"], }, }, }]; function getWeather(city: string) { return `Sunny, 22°C in ${city}`; } const messages: any[] = [{ role: "user", content: "What's the weather in Stockholm?" }]; let r = await client.chat.completions.create({ model: "openai/gpt-5-mini", messages, tools, }); const msg = r.choices[0].message; messages.push(msg); for (const call of msg.tool_calls ?? []) { const args = JSON.parse(call.function.arguments); const result = getWeather(args.city); messages.push({ role: "tool", tool_call_id: call.id, content: result, }); } r = await client.chat.completions.create({ model: "openai/gpt-5-mini", messages, tools, }); console.log(r.choices[0].message.content); ``` ## Control when tools fire The `tool_choice` parameter tells the model how aggressively to use tools. | Value | Behavior | | - | - | | `"auto"` *(default)* | The model decides. Calls a tool when it makes sense, otherwise answers in text. | | `"none"` | Tools are ignored. The model has to answer from what it knows. | | `"required"` | The model must call one of the tools. Use when the answer can only come from a tool. | | `{"type": "function", "function": {"name": "X"}}` | Force a specific tool. | ## Parallel calls The model can ask for several tool calls in one turn. The `tool_calls` array on the response can have more than one entry. Run them all (in parallel if they're independent), then send back one `tool` message per call, each with its `tool_call_id`. ## Streaming tool arguments When you stream a tool call with `stream: true`, the arguments arrive as JSON fragments in `delta.tool_calls[].function.arguments`. Concatenate them as they come in, then parse once the call is complete. Useful for showing "I'm calling search…" UI as the call assembles. See [Streaming](/build/gateway/streaming). ## Tools vs structured output Tools and [structured output](/build/gateway/structured-output) look similar but do different things. | | Tools | Structured output | | - | - | - | | **Goal** | Have the model trigger an action | Get a single typed JSON object back | | **Round-trips** | At least two (call, result, follow-up) | One | | **Multiple results per turn** | Yes, parallel calls allowed | No, one object | | **Best for** | Agents, search, data lookups, side effects | Extraction, classification, parsing | If you have a clear input → output and don't need the model to drive control flow, reach for [structured output](/build/gateway/structured-output), not tools. ## What's next Provider-run tools — Anthropic web\_search, OpenAI code\_interpreter, Google grounding — no round-trip needed. Multi-turn chat with message history. Stream tokens and tool arguments as they arrive. Get JSON back without the tool round-trip. # Tags & usage attribution Source: https://docs.opper.ai/build/gateway/usage-attribution Attach key:value tags to calls, then slice usage and cost by customer, team, or environment — via the analytics API, the CLI, or the dashboard. When one Opper account serves many customers, teams, or features, per-model spend isn't enough — you want to know *whose* usage it was. Tags solve that: attach up to 8 `key:value` pairs to any call (`customer_id:acme`, `team:eu`, `env:prod`), and they're recorded on the call's billing and metrics rows. From there you can group spend and token counts by any tag key. Every call already carries built-in tags like `model`, so `group_by=model` works with no setup at all. Custom tags add your own dimensions on top. There are two ways to attach tags. They differ only in *where* the tags travel — pick whichever your client can send: | Mechanism | Use it when | | - | - | | [`X-Opper-Tags` header](#tag-with-the-x-opper-tags-header) | You call the gateway and can set request headers (drop-in SDKs, LiteLLM, curl). | | [URL session prefix](#tag-with-the-url) | All you control is a base URL — no headers, no body (coding agents, fixed integrations). | ## Tag with the `X-Opper-Tags` header Works on every gateway (`/v3/compat`) call. The value is comma-separated `key:value` pairs; spaces after commas are fine, and sending the header more than once accumulates pairs. ```bash curl theme={null} curl https://api.opper.ai/v3/compat/chat/completions \ -H "Authorization: Bearer $OPPER_API_KEY" \ -H "X-Opper-Tags: customer_id:acme, team:eu" \ -H "Content-Type: application/json" \ -d '{ "model": "openai/gpt-5-mini", "messages": [{"role": "user", "content": "Hello"}] }' ``` ```python OpenAI SDK theme={null} import os from openai import OpenAI client = OpenAI( base_url="https://api.opper.ai/v3/compat", api_key=os.environ["OPPER_API_KEY"], default_headers={"X-Opper-Tags": "customer_id:acme, team:eu"}, ) response = client.chat.completions.create( model="openai/gpt-5-mini", messages=[{"role": "user", "content": "Hello"}], ) ``` ```yaml LiteLLM theme={null} model_list: - model_name: claude-haiku litellm_params: model: openrouter/anthropic/claude-haiku-4-5 api_base: https://api.opper.ai/v3/compat api_key: os.environ/OPPER_API_KEY extra_headers: {"X-Opper-Tags": "tenant:acme,env:prod"} ``` Header values are taken literally — no escaping needed. A value may contain `:` (only the first colon splits key from value) but not `,`. A malformed header is rejected with a `400` rather than silently dropped, so a templating bug on your side can't quietly produce untagged (unattributable) usage. ## Tag with the URL Some clients let you configure nothing but a base URL — a coding agent's `ANTHROPIC_BASE_URL`, a vendor integration with a single "endpoint" field. For those, tags can ride in the URL path itself: ``` https://api.opper.ai/v3/session/sess_/:/:/... ``` Everything after the tag segments is routed exactly like `/v3/compat/...`, so the client's SDK appends its usual path (`/chat/completions`, `/v1/messages`, …) and every request through that base URL carries the tags: ```python theme={null} import os, uuid from openai import OpenAI session = f"sess_{uuid.uuid4()}" # lowercase UUIDv4 with a sess_ prefix client = OpenAI( base_url=f"https://api.opper.ai/v3/session/{session}/customer_id:acme/team:eu", api_key=os.environ["OPPER_API_KEY"], ) ``` Two things are specific to this mechanism: * **`session_id` comes free.** The `sess_` segment (a lowercase UUIDv4 you generate) is recorded as a `session_id` tag on every call, so you can also pull usage for one session: `GET /v2/analytics/usage?session_id=sess_...`. This is how `opper launch` accounts for each coding-agent session. * **Percent-encode tag values.** Values are URL path segments and are decoded exactly once — encode any character that would break a path (`/` → `%2F`, and so on). If a request carries both URL tags and an `X-Opper-Tags` header, the URL's value wins for any key present in both. ## Limits and validation The same rules apply to both mechanisms. Invalid tags fail the request with a `400` — they are never silently dropped. | Rule | Limit | | - | - | | Tags per call | 8 | | Key format | Starts with a letter; then letters, digits, `_`, `.`, `-`; max 64 characters (`^[a-zA-Z][a-zA-Z0-9_.-]{0,63}$`) | | Value size | 256 bytes | | Reserved keys | Keys starting with `opper.`, and `session_id` (set by the URL session prefix) | ## Slice usage by tag ### Analytics API `GET /v2/analytics/usage` aggregates cost (and any metadata fields you ask for) over time, and `group_by=` splits every time bucket by that tag's values: ```bash theme={null} curl -s "https://api.opper.ai/v2/analytics/usage?group_by=customer_id&fields=total_tokens&granularity=day" \ -H "Authorization: Bearer $OPPER_API_KEY" ``` ```json theme={null} [ { "time_bucket": "2026-08-10T00:00:00Z", "cost": "1.8421", "total_tokens": 261404, "customer_id": "acme" }, { "time_bucket": "2026-08-10T00:00:00Z", "cost": "0.4114", "total_tokens": 60912, "customer_id": "globex" }, { "time_bucket": "2026-08-10T00:00:00Z", "cost": "0.0231", "total_tokens": 4102, "customer_id": null } ] ``` Untagged usage shows up with the group key `null`, so the rows always sum to your total spend — nothing hides just because it wasn't tagged. | Parameter | What it does | | - | - | | `group_by` | Tag keys to split by. Repeat the param or pass a comma-separated list (`group_by=customer_id,env` returns one row per combination). Built-in tags like `model` work too. | | `fields` | Metadata fields to sum per row: `total_tokens`, `prompt_tokens`, `completion_tokens`, … | | `granularity` | Bucket size: `minute`, `hour`, `day` (default), `month`, `year`. | | `from_date` / `to_date` | Time range (inclusive / exclusive). Defaults to the current month. | | `session_id` | Narrow to one URL-prefix session (`sess_...`). | ### CLI The [Opper CLI](/developer-tools/cli) wraps the same endpoint: ```bash theme={null} opper usage list --group-by=customer_id ``` ### Dashboard The usage page at [platform.opper.ai](https://platform.opper.ai) has a **Tag** filter that scopes the whole page — spend chart, breakdowns, attribution, CSV export — to one `key:value` tag. The dropdown lists your organization's recently seen tags, so anything you tag shows up there automatically. In [Logs](/control-plane/logs), each call's tags appear in its details, and the **Tag** filter lists only the calls that carry one. The dashboard Tag filter is rolling out and may not be visible on your account yet. The analytics API and CLI above work today. ## What's next Point your existing OpenAI, Anthropic, or Google SDK at the gateway. Tags on spans make individual traces filterable, too. LiteLLM, Vercel AI SDK, and more — with per-tenant attribution built in. Per-token pricing for every model the gateway serves. # Web search Source: https://docs.opper.ai/build/gateway/web-search Portable web search via the opper:web_search server-side tool. One tool entry that works on every compat endpoint, with native server-side search where the routed model supports it and Opper-managed fallback where it does not. Server-side web search comes in two flavors at Opper. [Provider-native shapes](/build/gateway/server-tools) forward verbatim to the routed provider: `web_search_20250305` on Anthropic, `{type:"web_search"}` on OpenAI Responses, `{googleSearch:{}}` on Google. The `opper:web_search` tool documented here is the portable cross-provider shape. It gives you one tool entry, identical response artifacts, and no per-provider branching in your code. Pick the canonical shape when you want a single tool entry that works regardless of which model the request lands on. Pick a native shape when you want the model's provider-specific search behavior and are happy to author per-provider request bodies. ## Quick start ```bash Anthropic-shape (/v3/compat/v1/messages) theme={null} curl https://api.opper.ai/v3/compat/v1/messages \ -H "Authorization: Bearer $OPPER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "anthropic/claude-sonnet-4-6", "max_tokens": 1024, "messages": [{"role": "user", "content": "What were the top tech headlines this week?"}], "tools": [{ "type": "opper:web_search", "freshness": "w", "max_uses": 3 }] }' ``` ```bash OpenAI-shape (/v3/compat/chat/completions) theme={null} curl https://api.opper.ai/v3/compat/chat/completions \ -H "Authorization: Bearer $OPPER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "openai/gpt-5.1", "messages": [{"role": "user", "content": "What were the top tech headlines this week?"}], "tools": [{ "type": "opper:web_search", "freshness": "w", "max_uses": 3 }] }' ``` ```bash OpenAI-shape (/v3/compat/responses) theme={null} curl https://api.opper.ai/v3/compat/responses \ -H "Authorization: Bearer $OPPER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "openai/gpt-5.1", "input": "What were the top tech headlines this week?", "tools": [{ "type": "opper:web_search", "freshness": "w", "max_uses": 3 }] }' ``` ```bash Google-shape (/v3/compat/.../generateContent) theme={null} curl "https://api.opper.ai/v3/compat/v1beta/models/gemini/gemini-3.8-flash:generateContent" \ -H "Authorization: Bearer $OPPER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "contents": [{"role": "user", "parts": [{"text": "What were the top tech headlines this week?"}]}], "tools": [{ "type": "opper:web_search", "freshness": "w", "max_uses": 3 }] }' ``` ## Engine selection The `engine` field controls how Opper routes the search. | Value | Behavior | | - | - | | `auto` (default) | Use the model's native server tool if the routed model+endpoint supports one. Fall back to Opper's engine otherwise. | | `native` | Require native. Returns `400` if the routed model has no native web search on the called endpoint. | | `opper` | Always use Opper's engine, regardless of native availability. Opper picks the backend. Useful when you want a uniform response shape across every model. | | `jina` | Pin Opper's engine to the Jina backend. | | `exa` | Pin Opper's engine to the Exa backend. Exa supports a true publish-date `freshness` window. | `jina` and `exa` are backend pins within Opper's engine: like `opper` they always run the server-side search (never native), but force a specific provider. Availability depends on server-side configuration. Pinning a backend that isn't configured returns a tool-call error. Today, native web search is available on: * Anthropic Claude on `/v3/compat/v1/messages` (both direct `anthropic/` and Vertex `vertexai/claude-*` routes) * OpenAI on `/v3/compat/responses` for Responses-API models (`gpt-5*` family, gpt-5-search-api, etc.) * Google Gemini on `/v3/compat/v1beta/models/{model}:generateContent` `engine: "auto"` will route to native on those combinations and to Opper's engine for everything else (Mistral, DeepSeek, Alibaba Qwen, Groq, etc.). ## Parameters | Parameter | Type | Description | | - | - | - | | `type` | string | Required. Must be `"opper:web_search"`. | | `engine` | string | `auto` \| `native` \| `opper` \| `jina` \| `exa`. Defaults to `auto`. | | `freshness` | string | Recency filter. `d` (24h), `w` (7d), `m` (30d), `y` (1y). Empty = no filter. | | `max_results` | int | Results per individual search. 0 uses the engine default. | | `max_total_results` | int | Cumulative cap across every search in the request. 0 disables. | | `max_uses` | int | Maximum number of searches the model may run in this request. | | `search_context_size` | string | Per-result snippet length in characters. `very_low` (\~1k) \| `low` (\~5k) \| `medium` (\~10k, default) \| `high` (\~30k) \| `full` (full extracted page, capped \~50k). | | `allowed_domains` | string\[] | Restrict results to these domains. | | `excluded_domains` | string\[] | Exclude results from these domains. | | `country` | string | Two-letter ISO-3166 alpha-2 code (e.g. `"se"`, `"us"`) to bias results regionally. | ## Citations and where they land The response shape matches whatever endpoint you called, so your code reads citations from the same place regardless of `engine`. * **Anthropic-shape**: `content[]` contains `server_tool_use`, `web_search_tool_result` (URLs + snippets), and `text` blocks whose `citations[]` (`web_search_result_location`) reference the sources. See [server-side tools](/build/gateway/server-tools#responses) for the field-by-field shape. * **OpenAI-shape (Responses)**: `output[]` contains `web_search_call` items with the search queries and a `message` whose `content[].output_text.annotations[]` carry `url_citation` entries. * **OpenAI-shape (Chat Completions)**: `choices[0].message.annotations[]` carry the `url_citation` entries, mirroring the Responses shape. * **Google-shape**: `candidates[0].groundingMetadata` carries `webSearchQueries`, `groundingChunks[].web` (URI + title), and `groundingSupports`. On the native route (`engine: "auto"` landing on a model with a native server tool) citations are anchored per statement: each one points at the exact span of prose the model grounded on, exactly as the provider emits them. On Opper's engine route (`engine: "opper"`, `"jina"`, or `"exa"`) citations are block-level: every source the search returned is attached to the answer text. You get the same fields in the same place, but without per-sentence character offsets, and the Google `searchEntryPoint` (provider-hosted "Search Suggestions" HTML) is not emitted. ## Cost and usage Every response includes a tool-cost breakdown under `usage.opper.cost.tools.web_search`: ```json theme={null} "usage": { "input_tokens": 2224, "output_tokens": 511, "server_tool_use": { "web_search_requests": 3 }, "opper": { "cost": { "tokens": 0.0255, "tools": { "total": 0.03, "web_search": { "count": 3, "cost": 0.03, "unit": 0.01 } }, "total": 0.0555 } } } ``` `count` is the number of searches the model actually ran, `unit` is the per-search cost, `cost` is the total. `engine` is also surfaced when the request routed through Opper's engine. ## Pricing Opper's engine is billed per search. Native searches (`engine: "auto"` landing on a model with a native tool, or `engine: "native"`) are billed at the routed provider's rate and passed through — the exact cost still shows up under `usage.opper.cost.tools.web_search`. | Engine | Search index | Data residency | Price | | - | - | - | - | | `opper` / `jina` | [Jina](https://jina.ai/) | EU — search processing stays in-region | \$0.01 / search | | `exa` | [Exa](https://exa.ai/) | Global (US-based) | \$0.007 / search | | `native` → Anthropic | Brave 1 | Provider infrastructure 2 | provider rate, ≈ \$10 / 1,000 searches — see [Anthropic](https://platform.claude.com/docs/en/agents-and-tools/tool-use/web-search-tool) | | `native` → OpenAI | Bing 1 | Provider infrastructure 2 | provider rate, ≈ \$10 / 1,000 calls — see [OpenAI](https://developers.openai.com/api/docs/guides/tools-web-search) | | `native` → Gemini | Google Search | Provider infrastructure 2 | provider rate, ≈ \$14 / 1,000 queries (Gemini 3) — see [Google](https://ai.google.dev/gemini-api/docs/grounding) | 1 The underlying index for the native providers is as publicly reported, not officially confirmed by the provider. Native provider prices change independently of Opper; the linked pages are authoritative. More Opper engine backends will be added over time. 2 Data residency reflects where the search *runs*. Native searches execute on the provider's own infrastructure — Opper doesn't control their region, and only the Jina backend offers EU residency. This is separate from result *localization*: use the `country` parameter (above) to bias results toward a geography, which does not change where the search runs. ## In the trace With tracing on, each server-side search appears in the [trace view](https://platform.opper.ai) as a `server_side_tool` step nested under the turn that requested it, showing the query, the results it returned, and the per-search cost — so the search is never a black box. ## Full example The quick start keeps the tool entry minimal. This example sets every parameter on one request so you can see them in one place. Only `type` is required; everything else is optional and falls back to the engine or handler default. ```bash theme={null} curl https://api.opper.ai/v3/compat/v1/messages \ -H "Authorization: Bearer $OPPER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "anthropic/claude-sonnet-4-5", "max_tokens": 1024, "messages": [{ "role": "user", "content": "What shipped in the latest Go release?" }], "tools": [ { "type": "opper:web_search", "engine": "auto", "freshness": "m", "max_results": 5, "max_total_results": 20, "max_uses": 3, "search_context_size": "medium", "allowed_domains": ["go.dev"], "excluded_domains": ["reddit.com"], "country": "us" } ] }' ``` The [OpenAPI spec](https://api.opper.ai/v3/openapi.yaml) (`CanonicalWebSearchTool` schema) is the authoritative definition of every field, its type, and its default. ## See also * [Provider server-side tools](/build/gateway/server-tools): native tool shapes per provider, routing rules, and the Vertex/Bedrock capability matrix. * [POST /v3/tools/web/search](/v3-api-reference/tools/web-search): call search as a standalone REST endpoint, with no model and no agentic loop. # Audio Source: https://docs.opper.ai/build/multimodal/audio Speech, transcription, music and sound effects through one gateway and key. The gateway covers audio with three endpoints: * **Text to speech**: `POST /v3/audio/speech` turns text into spoken audio. * **Speech to text**: `POST /v3/audio/transcriptions` turns audio into a transcript. * **Music and sound effects**: `POST /v3/audio/generations` turns a prompt into a score, a song, a sound effect or a loop of ambience. See [Music and sound effects](#music-and-sound-effects). All are provider-agnostic: `model` selects the provider, a small set of fields is normalized for you, and anything model-specific goes in `parameters`. You can also [clone a custom voice](#custom-voices-cloning) from a short sample and reuse it. For live, two-way voice conversations, use [Realtime](/build/realtime/quickstart) instead. ## Text to speech Pass a `model` and the `input` text. The response carries the audio inline as base64. Add `"store": true` to also keep a copy in [Files](/build/multimodal/files) and get a `file_id` + presigned `url`. ```bash cURL theme={null} curl -sX POST https://api.opper.ai/v3/audio/speech \ -H "Authorization: Bearer $OPPER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "openai/tts-1", "input": "Hello from Opper.", "voice": "alloy", "format": "mp3" }' ``` ```python Python theme={null} import os, base64, requests r = requests.post( "https://api.opper.ai/v3/audio/speech", headers={"Authorization": f"Bearer {os.environ['OPPER_API_KEY']}"}, json={ "model": "openai/tts-1", "input": "Hello from Opper.", "voice": "alloy", "format": "mp3", }, ) r.raise_for_status() audio = r.json()["audio"] open("speech.mp3", "wb").write(base64.b64decode(audio["b64_json"])) print("saved speech.mp3") ``` ```json Response theme={null} { "id": "spch_...", "model": "openai/tts-1", "created": 1716124800, "audio": { "b64_json": "SUQzBAAAAAA...", "mime_type": "audio/mpeg" }, "usage": { "cost": 0.0009, "characters": 17 } } ``` | Field | What it does | | - | - | | `model` | Required. The TTS model, e.g. `openai/tts-1`. | | `input` | Required. The text to speak. | | `voice` | Provider voice; empty uses the model's default. See [Finding voices](#finding-voices). | | `format` | `mp3` (default), `wav`, `opus`, `aac`, `flac`, or `pcm`. | | `speed` | `0.25`–`4.0` where the model supports it. | | `store` | Persist the audio and return a `file_id` + `url`. Defaults to false; a stored file is permanent until deleted. Refused on zero-data-retention projects. | | `parameters` | Opaque per-provider passthrough. | Speech is billed per input character. ### Finding voices `GET /v3/audio/voices?model=` lists every voice a text-to-speech model accepts as `voice`: the provider's built-in voices (`type: preset`) first, then your project's [cloned voices](#custom-voices-cloning) on the same provider (`type: custom`). Pass the `id` of any entry as `voice`. For ElevenLabs and Mistral, the built-in voices are read from the provider itself, so a voice they add shows up without any change on our side. Each comes with a name, gender, accent and languages, and ElevenLabs voices carry a `preview_url` you can listen to before you pick. For other providers, the list is the voices the model declares. `meta.presets_source` says which one you got (`provider` or `catalog`). ```bash theme={null} curl -s "https://api.opper.ai/v3/audio/voices?model=elevenlabs/eleven_v3&gender=female" \ -H "Authorization: Bearer $OPPER_API_KEY" \ | jq '.data[] | {id, name, accent, languages, preview_url}' ``` ```json theme={null} { "id": "hpp4J3VqNfWAUOO0d1Us", "name": "Bella - Professional, Bright, Warm", "accent": "american", "languages": ["en"], "preview_url": "https://..." } ``` | Filter | What it does | | - | - | | `model` | Required for this listing. A text-to-speech model id. | | `type` | `preset` for built-in voices only, `custom` for your clones only. | | `language` | A language code such as `de` or `en-GB`. `en` also matches `en-US` and `en-GB`. | | `gender` | As the provider labels it, for example `female` or `male`. | | `limit`, `offset` | Paging. Up to 1000 per page. | The entry marked `"default": true` is the voice used when you leave `voice` out. | Provider | Notes | | - | - | | OpenAI, Gemini, xAI | A short fixed list. A voice outside it returns a `400`. | | Azure Speech | The voice name selects both the speaker and the locale, for example `en-GB-SoniaNeural`, so `language=fr-CA` finds every French (Canada) voice. Standard voices are on `azure-speech/neural-tts`, HD voices on `azure-speech/neural-hd-tts`. | | Mistral Voxtral | Presets such as `en_paul_neutral` and `gb_jane_neutral`, in several moods per speaker. | | ElevenLabs | ElevenLabs' built-in voices. Voices in your own ElevenLabs account are not available here, because Opper calls ElevenLabs with its own account: [clone the voice](#custom-voices-cloning) instead. | | Fish Audio | Not listed. Pass a Fish voice **id** as `voice`; names are not accepted. Browse voices with audio samples and ids in the [Fish voice library](https://docs.slng.ai/models/voices/fish-audio). Set a voice explicitly for consistent results. | #### Voices for a specific language Filter by language to find voices made for it. This lists every French (Canada) voice on Azure, and works the same for any locale such as `de-AT`, `sv-SE` or `ja-JP`: ```bash theme={null} curl -s "https://api.opper.ai/v3/audio/voices?model=azure-speech/neural-tts&language=fr-CA" \ -H "Authorization: Bearer $OPPER_API_KEY" \ | jq -r '.data[].id' ``` Most multilingual models will also read text in another language with any voice, but with a foreign accent. For native-sounding speech, pick a voice made for that language or regional variety. ### Expressiveness A voice can sound flat by default. How you add expression depends on the provider: * **ElevenLabs** exposes voice controls through `parameters.voice_settings`: lower `stability` for a more dynamic, emotional read, higher `style` to exaggerate delivery, and `use_speaker_boost`. The `elevenlabs/eleven_v3` model also understands inline audio tags in the `input` text — `[excited]`, `[whispers]`, `[sarcastic]`, `[laughs]` — for fine-grained emotion. * **Mistral Voxtral** has no style knobs; it mirrors the delivery of the voice it speaks as. For a lively result, use — or clone — a lively-sounding voice. ```bash Voice settings (ElevenLabs) theme={null} curl -sX POST https://api.opper.ai/v3/audio/speech \ -H "Authorization: Bearer $OPPER_API_KEY" -H "Content-Type: application/json" \ -d '{ "model": "elevenlabs/eleven_multilingual_v2", "input": "We did it — the results are in!", "voice": "voice_abc123", "parameters": { "voice_settings": { "stability": 0.3, "style": 0.8, "use_speaker_boost": true } } }' ``` ```bash Emotion tags (eleven_v3) theme={null} curl -sX POST https://api.opper.ai/v3/audio/speech \ -H "Authorization: Bearer $OPPER_API_KEY" -H "Content-Type: application/json" \ -d '{ "model": "elevenlabs/eleven_v3", "input": "[excited] We did it! [whispers] and no one even noticed.", "voice": "voice_abc123" }' ``` Audio tags count toward billed input characters, and `eleven_v3` bills at a higher rate — great for a hero voice, more than you need for bulk narration. Use `GET /v3/audio/models` to discover the exact `parameters` a model accepts (its `params.tts.parameters` list). ## Custom voices (cloning) Clone a voice from a short reference sample and reuse it by an opaque id. `POST /v3/audio/voices` takes a `model` (which selects the provider to clone on), an `audio` reference — a `file_id`, https URL, or base64 data-URI of a few seconds of speech — and an optional `name`, `languages`, and `gender`. It returns a `voice_...` id you pass as `voice` on `/v3/audio/speech`, exactly like a built-in voice. ```bash cURL theme={null} # 1. Clone — returns { "id": "voice_...", "provider": "mistral", ... } curl -sX POST https://api.opper.ai/v3/audio/voices \ -H "Authorization: Bearer $OPPER_API_KEY" -H "Content-Type: application/json" \ -d '{ "model": "mistral/voxtral-mini-tts-2603", "name": "my-voice", "audio": "file_abc123", "languages": ["de"] }' # 2. Speak with it — pass the voice_... id as `voice` curl -sX POST https://api.opper.ai/v3/audio/speech \ -H "Authorization: Bearer $OPPER_API_KEY" -H "Content-Type: application/json" \ -d '{ "model": "mistral/voxtral-mini-tts-2603", "input": "Hallo!", "voice": "voice_..." }' ``` ```python Python theme={null} import os, requests base = "https://api.opper.ai" h = {"Authorization": f"Bearer {os.environ['OPPER_API_KEY']}"} voice = requests.post(f"{base}/v3/audio/voices", headers=h, json={ "model": "mistral/voxtral-mini-tts-2603", "name": "my-voice", "audio": "file_abc123", # or an https URL / data-URI "languages": ["de"], }).json() speech = requests.post(f"{base}/v3/audio/speech", headers=h, json={ "model": "mistral/voxtral-mini-tts-2603", "input": "Hallo!", "voice": voice["id"], # voice_... }).json() ``` ```json Response (create) theme={null} { "id": "voice_033Qq...", "object": "voice", "name": "my-voice", "provider": "mistral", "languages": ["de"], "created": 1716124800, "expires_at": 1718716800 } ``` Manage your voices — all scoped to your project: | Method | Path | What it does | | - | - | - | | `GET` | `/v3/audio/voices` | List your project's cloned voices (newest first, paginated). Add `?model=` to also list that model's built-in voices, see [Finding voices](#finding-voices). | | `GET` | `/v3/audio/voices/{id}` | Fetch one voice. | | `DELETE` | `/v3/audio/voices/{id}` | Remove it — also deletes it at the provider. | | Field (create) | What it does | | - | - | | `model` | Required. Selects the provider to clone on (e.g. `mistral/voxtral-mini-tts-2603`, `elevenlabs/eleven_multilingual_v2`). | | `audio` | Required. The reference sample: a `file_id`, https URL, or base64 data-URI. | | `name` | A human label for the voice. | | `languages` | Language hints, e.g. `["de"]`. | | `gender` | Optional voice gender hint. | | `parameters` | Opaque per-provider passthrough for create-voice fields we don't normalize. | **A clone carries the voice, not a language.** Cloning captures timbre and accent; the output language comes from the text you synthesize with a multilingual model — so one clone can speak any supported language. Recording in the target language gives the most native accent. A few things to know: * **Provider binding.** A voice is bound to the provider you cloned it on and only works with that provider's TTS models. Using it with another provider's model returns a `400`. * **Isolation & privacy.** Voices are scoped to your project — never visible to or usable by another project — and the reference sample is not stored. * **Expiry.** Some providers remove clones after a while. When they do, `expires_at` is returned. Treat it as the provider's estimate, not a deadline: it is the earliest date the provider may remove the voice, and a voice often keeps working well past it. A voice stays in your list for as long as the provider still has it. Once the provider has removed it, it drops out of the list and a speak returns a clear "no longer available" error. Create a new one from the same sample. Providers that don't expire voices omit `expires_at`. * **Sample length.** A few seconds is enough for Mistral Voxtral (zero-shot). ElevenLabs works best with \~1–2 minutes of clean audio (avoid more than \~3 minutes); recording quality matters more than length. ## Speech to text Pass a `model` and an `audio` source: a `file_id` from a previous generation or upload, an https URL, or a data-URI (max 25 MB decoded synchronously; up to 100 MB in [async mode](#long-recordings-async)). A `file_id` lets you transcribe audio you already have on Opper, such as a clip you uploaded or speech you generated with `"store": true`, without re-sending the bytes. See [Files](/build/multimodal/files) for how files work, lifecycle, and storage quotas. ```bash Transcribe a file_id theme={null} curl -sX POST https://api.opper.ai/v3/audio/transcriptions \ -H "Authorization: Bearer $OPPER_API_KEY" -H "Content-Type: application/json" \ -d '{ "model": "openai/whisper-1", "audio": "file_abc123" }' ``` ```bash cURL theme={null} curl -sX POST https://api.opper.ai/v3/audio/transcriptions \ -H "Authorization: Bearer $OPPER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "openai/whisper-1", "audio": "https://example.com/meeting.mp3", "language": "en" }' ``` ```python Python theme={null} import os, requests r = requests.post( "https://api.opper.ai/v3/audio/transcriptions", headers={"Authorization": f"Bearer {os.environ['OPPER_API_KEY']}"}, json={ "model": "openai/whisper-1", "audio": "https://example.com/meeting.mp3", "language": "en", }, ) r.raise_for_status() print(r.json()["text"]) ``` ```json Response theme={null} { "id": "trsc_...", "model": "openai/whisper-1", "created": 1716124800, "text": "Full transcript here...", "language": "en", "duration": 42.5, "segments": [ { "start": 0.0, "end": 3.2, "text": "Full transcript here..." } ], "usage": { "cost": 0.004, "seconds": 42.5 } } ``` | Field | What it does | | - | - | | `model` | Required. The transcription model, e.g. `openai/whisper-1`. | | `audio` | Required. A `file_id`, https URL, or data-URI to transcribe. | | `language` | ISO-639-1 hint (e.g. `en`); the provider validates it. | | `prompt` | Context or vocabulary hint to bias the transcript. | | `diarize` | Request speaker labels on segments. Rejected on models that don't support diarization. | | `stream` | Return the transcript live as Server-Sent Events. Supported where `params.stt.stream` is true; a 400 otherwise. Mutually exclusive with `async`. See below. | | `async` | Run as a background job for long recordings — returns `202` with a status URL to poll instead of blocking. See below. | | `parameters` | Opaque per-provider passthrough. | Transcription is billed per provider-reported audio duration. Segment and word timestamps are returned when the model provides them. ### Live transcript (streaming) Set `stream: true` to receive the transcript as it's produced — as Server-Sent Events instead of one JSON body — for a live transcript while the audio is processed. The stream emits `transcript.text.delta` chunks, then a terminal `transcript.text.done` with the full transcript and usage, then `data: [DONE]`. Streaming is supported on models whose `params.stt.stream` is `true` (discover them via [`GET /v3/audio/models`](#discover-models)) — e.g. `mistral/voxtral-mini-2602`; other models return a `400`. `stream` and `async` are mutually exclusive. ```bash cURL theme={null} # -N keeps curl from buffering, so events print as they arrive curl -N -sX POST https://api.opper.ai/v3/audio/transcriptions \ -H "Authorization: Bearer $OPPER_API_KEY" -H "Content-Type: application/json" \ -d '{ "model": "mistral/voxtral-mini-2602", "audio": "file_abc123", "stream": true }' ``` ```python Python theme={null} import os, json, requests with requests.post( "https://api.opper.ai/v3/audio/transcriptions", headers={"Authorization": f"Bearer {os.environ['OPPER_API_KEY']}"}, json={"model": "mistral/voxtral-mini-2602", "audio": "file_abc123", "stream": True}, stream=True, ) as r: for line in r.iter_lines(): if not line or not line.startswith(b"data: "): continue data = line[len(b"data: "):] if data == b"[DONE]": break event = json.loads(data) if event["type"] == "transcript.text.delta": print(event["delta"], end="", flush=True) elif event["type"] == "transcript.text.done": print("\n\n", event["usage"]) ``` ```text Event stream theme={null} data: {"type":"transcript.text.delta","delta":"Good morning and "} data: {"type":"transcript.text.delta","delta":"welcome!"} data: {"type":"transcript.text.done","id":"trsc_...","text":"Good morning and welcome!","language":"en","duration":101,"usage":{"cost":0.00505,"seconds":101}} data: [DONE] ``` Billing is identical to the synchronous call (per provider-reported audio duration). For long recordings that don't need a live transcript, use [async](#long-recordings-async) instead; for two-way live voice, use [Realtime](/build/realtime/quickstart). ### Long recordings (async) A synchronous request is a good fit up to roughly an hour of audio. For longer recordings, set `async: true`: the request returns immediately with a `202` and a `status_url`. Poll it until the job is `completed`, then fetch the transcript from the returned `url`. Async accepts the same `audio` sources as the synchronous call (`file_id`, https URL, or data-URI). Async also raises the decoded-audio limit from 25 MB to **100 MB** (the synchronous cap stays 25 MB). To reach the full 100 MB, pass a `file_id` or https URL — a data-URI that large would need a \~133 MB request body. This lines up with the [Files](/build/multimodal/files) per-file limit, so a stored `file_id` transcribes in full. ```bash Submit + poll theme={null} # 1. Submit — returns { "id": "gen_...", "status_url": ".../v3/artifacts/{id}/status" } curl -sX POST https://api.opper.ai/v3/audio/transcriptions \ -H "Authorization: Bearer $OPPER_API_KEY" -H "Content-Type: application/json" \ -d '{ "model": "mistral/voxtral-mini-2602", "audio": "file_abc123", "diarize": true, "async": true }' # 2. Poll — 202 while processing, then 200 with a presigned url to the transcript JSON curl -s https://api.opper.ai/v3/artifacts/{id}/status \ -H "Authorization: Bearer $OPPER_API_KEY" ``` `mistral/voxtral-mini-2602` is an EU-hosted batch transcription model (speaker diarization, word timestamps, up to 3 h per request) — a good fit for the async flow. ## Music and sound effects `POST /v3/audio/generations` generates audio from a prompt with any model tagged `music` or `sound`. It runs synchronously and returns the audio inline as base64, like speech. The same request also answers on `POST /v3/audio/music`. ### A sound effect Describe one action per request, as the material and distance you want to hear, and set `duration_seconds` to the length of that action. ```bash cURL theme={null} curl -sX POST https://api.opper.ai/v3/audio/generations \ -H "Authorization: Bearer $OPPER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "elevenlabs/eleven_text_to_sound_v2", "prompt": "a single hammer strike on an iron anvil, close, dry", "duration_seconds": 1 }' ``` ```python Python theme={null} import os, base64, requests r = requests.post( "https://api.opper.ai/v3/audio/generations", headers={"Authorization": f"Bearer {os.environ['OPPER_API_KEY']}"}, json={ "model": "elevenlabs/eleven_text_to_sound_v2", "prompt": "a single hammer strike on an iron anvil, close, dry", "duration_seconds": 1, }, ) r.raise_for_status() open("anvil.mp3", "wb").write(base64.b64decode(r.json()["audio"]["b64_json"])) ``` ```json Response theme={null} { "id": "snd_...", "model": "elevenlabs/eleven_text_to_sound_v2", "created": 1790781677, "audio": { "b64_json": "SUQzBAAAAAA...", "mime_type": "audio/mpeg" }, "usage": { "cost": 0.002, "duration_seconds": 1 } } ``` For ambience and hums, add `"loop": true` on a model with the `audio_loop` capability: the result repeats without a seam, so you can tile it under a whole scene. ### Music with timed sections A plain `prompt` with `duration_seconds` gives you a track. When the music has to fit a picture, send `sections` instead: a timed plan where each section has its own `duration_ms`, `prompt` and `styles`. The sections set the total length. This needs a model with the `audio_sections` capability. ```bash cURL theme={null} curl -sX POST https://api.opper.ai/v3/audio/generations \ -H "Authorization: Bearer $OPPER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "elevenlabs/music_v2_5", "sections": [ { "duration_ms": 4000, "prompt": "120 BPM, playful marimba intro, sound from the first beat", "styles": ["marimba", "instrumental"] }, { "duration_ms": 4000, "prompt": "the same instruments, fuller, ending on a resolved chord", "styles": ["marimba", "pizzicato strings", "instrumental"] } ] }' ``` ```python Python theme={null} import os, base64, requests r = requests.post( "https://api.opper.ai/v3/audio/generations", headers={"Authorization": f"Bearer {os.environ['OPPER_API_KEY']}"}, json={ "model": "elevenlabs/music_v2_5", "sections": [ { "duration_ms": 4000, "prompt": "120 BPM, playful marimba intro, sound from the first beat", "styles": ["marimba", "instrumental"], }, { "duration_ms": 4000, "prompt": "the same instruments, fuller, ending on a resolved chord", "styles": ["marimba", "pizzicato strings", "instrumental"], }, ], }, ) r.raise_for_status() open("score.mp3", "wb").write(base64.b64decode(r.json()["audio"]["b64_json"])) ``` A section's `prompt` describes the music and is never sung. Words to be sung go only in that section's `lyrics`; leave it out for instrumental music. `negative_styles` lists what a section should avoid, for example `["vocals"]`. Some models take either a top-level `prompt` or `sections`, and put `instrumental` in the section styles rather than the top-level field; they answer 400 saying so. On ElevenLabs music, `seed` works only with `sections`: a `prompt` with a `seed` answers 400. | Field | What it does | | - | - | | `model` | Required. A `music` or `sound` model from [Discover models](#discover-models). | | `prompt` | What to generate. Required unless `sections` are sent. | | `duration_seconds` | Requested length, up to 600. Omit it to let the model choose; not sent with `sections`. | | `sections` | A timed plan: `duration_ms`, `prompt`, `styles`, `negative_styles`, `lyrics` per section. Needs `audio_sections`. | | `loop` | Seamless repeat for ambience. Needs `audio_loop`. | | `instrumental` | Music only: no vocals. | | `format` | `mp3`, `wav`, `opus` or `pcm`. Omit it for the model's native format. | | `seed` | For more repeatable output, where the model supports it. ElevenLabs music takes it only with `sections`, and ElevenLabs sound effects not at all. | | `async` | Run in the background and poll a `status_url` (202), for long tracks. Works like [long transcriptions](#long-recordings-async). | | `store` | Also save the result to [Files](/build/multimodal/files) and return a `file_id`. | | `parameters` | Provider fields, forwarded as-is in the provider's own names. | A value a model cannot honor (a length, a format, `sections` or `loop` on a model without the capability) answers 400 naming the limit, rather than being ignored. `parameters` is never checked by Opper; the provider validates it. Music and sound effects are billed per minute of audio or per generation, depending on the model; the model entry's `pricing` says which. **Getting usable results** * **One action per sound effect.** "A hammer tap" and "a saw stroke" are two requests; mix them yourself. * **Put section boundaries on the tempo's phrase grid.** At 120 BPM, multiples of 2 or 4 seconds. Boundaries between phrases get moved to the nearest phrase. * **Keep one instrument palette across sections** and vary the intensity. Changing instruments between sections sounds like a different track. * **Do silences, drops and breakdowns in your mix,** not in the prompt. Models often play through a section that asks for "drums out". * **Measure what you got** (length, loudness per second) before cutting picture to it. ## Discover models `GET /v3/audio/models` lists the audio models available, tagged `tts`, `stt`, `music` or `sound`, with their voices, formats and capabilities. Filter with `?type=music` or `?type=sound`, or by capability with `?capability=audio_sections` or `?capability=audio_loop`. Each model's `params.tts` / `params.stt` block also carries a `parameters` list of the native passthrough keys it accepts (e.g. ElevenLabs' exact `output_format` preset, or a `language_code`), so you can discover the model-specific knobs for `parameters` rather than guessing: ```bash theme={null} # all audio models, or filter by type curl -s "https://api.opper.ai/v3/audio/models?type=tts" \ -H "Authorization: Bearer $OPPER_API_KEY" ``` ## What's next Live, two-way voice over WebSocket. Which models do speech, transcription, music and sound effects. Inspect calls and manage provider, region, and spend rules. Generate video from a prompt or image. # Files Source: https://docs.opper.ai/build/multimodal/files Reusable storage for media. Upload once, reference by id across image, audio, and video calls. Files are how media moves between calls on Opper. Upload a file (or let a generation store its output) and you get back a `file_` — a reusable handle you can pass as input to later calls instead of re-uploading or re-encoding bytes. One uploaded image can seed a video; one generated image can be edited by the next call; a generated audio clip can be fed straight to transcription. ## Two ways files appear * **You upload them.** `POST /v3/files` (multipart) returns a `file_` for reference media — an image to animate, a source video to edit, an audio clip to transcribe. * **Generations store them when asked.** The [image](/build/multimodal/images), [audio](/build/multimodal/audio), and [video](/build/multimodal/video) endpoints save their output to Files when you pass `store: true`, and return a `file_id` alongside the result. Nothing is stored otherwise. ## Using a `file_id` as input A `file_id` is accepted anywhere a media source is — next to an http(s) URL or a data-URI: | Endpoint | Fields that accept a `file_id` | | - | - | | [`POST /v3/images`](/build/multimodal/images) | `image`, `mask`, `reference_images` | | [`POST /v3/audio/transcriptions`](/build/multimodal/audio) | `audio` | | [`POST /v3/videos`](/build/multimodal/video) | `image`, `video`, `reference_images` | ```bash theme={null} # 1. Upload a reference image curl -sX POST https://api.opper.ai/v3/files \ -H "Authorization: Bearer $OPPER_API_KEY" \ -F "file=@cat.jpg" # → { "id": "file_abc123", ... } # 2. Use it as the seed for a video curl -sX POST https://api.opper.ai/v3/videos \ -H "Authorization: Bearer $OPPER_API_KEY" -H "Content-Type: application/json" \ -d '{ "model": "xai/grok-imagine-video", "prompt": "the cat blinks slowly", "image": "file_abc123" }' ``` ## Lifecycle Files are **permanent until you delete them** — uploads and stored generation outputs alike. There is no default expiry: a `file_id` you get today keeps working until you call `DELETE /v3/files/{id}`. If you want a file to clean itself up, opt into an expiry: * **On upload** — pass `ttl_seconds` (a positive integer, up to 100 years) as a form field and the file expires that many seconds from now. * **Later** — `PATCH /v3/files/{id}` with `{"ttl_seconds": 3600}` sets or reschedules the expiry from the moment of the call; `{"ttl_seconds": null}` clears it and makes the file permanent again. ```bash theme={null} # Give an existing file one hour to live curl -sX PATCH https://api.opper.ai/v3/files/file_abc123 \ -H "Authorization: Bearer $OPPER_API_KEY" -H "Content-Type: application/json" \ -d '{ "ttl_seconds": 3600 }' # Changed your mind — make it permanent again curl -sX PATCH https://api.opper.ai/v3/files/file_abc123 \ -H "Authorization: Bearer $OPPER_API_KEY" -H "Content-Type: application/json" \ -d '{ "ttl_seconds": null }' ``` Expired files are removed by a background sweep, within the hour of their expiry. ### Files and retention policy [Retention](/control-plane/rules/retention) rules govern telemetry (traces and generation recordings), not your stored files. A 30-day retention rule does not delete files. The one exception is **zero data retention**: a zero-day scope cannot hold files at all. Uploads are rejected, `store: true` on a generation is refused, and enabling zero-day on a scope that already holds files permanently deletes them, after an explicit confirmation of the file count. ## Quotas Each organization has a storage quota — a total byte budget (it can vary by plan) and a cap on the number of files. Uploads and stored generation outputs draw from the same budget. * **Uploads** that would exceed the quota are rejected with `413`. * **Generated outputs** (`store: true`) are refused with `413` when the quota is already full. If it fills up while the call runs, the call still succeeds and returns the result inline without storing it, and the response signals the skip. Since files never expire on their own, the quota is what bounds your storage: delete files you no longer need, or give short-lived ones a `ttl_seconds` so they clean themselves up. Larger budgets are tied to your plan. ## Operations | Operation | Endpoint | | - | - | | Upload a file | [`POST /v3/files`](/v3-api-reference/files/upload-file) | | List files | [`GET /v3/files`](/v3-api-reference/files/list-files) | | Get metadata | [`GET /v3/files/{id}`](/v3-api-reference/files/get-file) | | Get a download URL | [`GET /v3/files/{id}/content`](/v3-api-reference/files/get-file-content) | | Set or clear an expiry | [`PATCH /v3/files/{id}`](/v3-api-reference/files/update-file) | | Delete a file | [`DELETE /v3/files/{id}`](/v3-api-reference/files/delete-file) | ## What's next Generate and edit images; feed a `file_id` for image-to-image. Seed a video from an uploaded or generated image. Transcribe an audio `file_id`. How the modalities fit together. # Images Source: https://docs.opper.ai/build/multimodal/images Generate and edit images with one endpoint across every image model. `POST /v3/images` is a deterministic, provider-agnostic image endpoint. You pass a `model` and a `prompt`; Opper runs the job and returns the image inline. The same call works across every image model (OpenAI GPT Image, Google Imagen, xAI, Pruna, and more) — `model` and `prompt` are owned by Opper, a small set of high-level parameters is normalized for you, and anything model-specific goes in `parameters`. Image generation is **synchronous by default**: the response comes back on the same request with the image as base64 (plus a stored `file_id` + presigned `url` when you pass `"store": true`). Slow jobs (the top quality tiers on the newest models) can run in the background instead, see [Heavy jobs](#heavy-jobs-async). For looking at images instead of making them, see [Vision & PDFs](/build/multimodal/vision-pdfs). ## Generate an image ```bash cURL theme={null} curl -sX POST https://api.opper.ai/v3/images \ -H "Authorization: Bearer $OPPER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "openai/gpt-image-2.5-flare", "prompt": "a wide-angle photo of a hot air balloon over green hills at dawn", "size": "1024x1024" }' ``` ```python Python theme={null} import os, base64, requests r = requests.post( "https://api.opper.ai/v3/images", headers={"Authorization": f"Bearer {os.environ['OPPER_API_KEY']}"}, json={ "model": "openai/gpt-image-2.5-flare", "prompt": "a wide-angle photo of a hot air balloon over green hills at dawn", "size": "1024x1024", }, ) r.raise_for_status() img = r.json()["data"][0] open("balloon.png", "wb").write(base64.b64decode(img["b64_json"])) print("saved balloon.png") ``` OpenAI's current pair: **`gpt-image-2.5-flare`** for fast everyday generation and editing, **`gpt-image-2.5-sunburst`** when an edit has to land exactly — same rates, slower. Both take two tiers above `high`, `xhigh` and `max`, and bill from the tokens the provider reports rather than a fixed per-image table, so cost climbs steeply with the tier: `max` runs about 16x `medium`. The response mirrors OpenAI's `images/generations` shape, plus gateway extras: ```json theme={null} { "id": "img_...", "model": "openai/gpt-image-2.5-flare", "created": 1716124800, "data": [ { "b64_json": "iVBORw0KGgo...", "mime_type": "image/png", "revised_prompt": "..." } ], "usage": { "cost": 0.013, "images": 1 } } ``` Pass `"store": true` to also store each image, so you get a reusable `file_id` and a presigned `url` alongside the base64, or `"response_format": "url"` to store it and return only the stored URL instead of the base64. Without either, nothing is stored. The `file_id` is a reusable handle — pass it as the `image` on a later edit or as the seed `image` on a [video](/build/multimodal/video) call, no re-upload. See [Files](/build/multimodal/files) for how files work, lifecycle, and storage quotas. ## Parameters `model` and `prompt` are required. The rest are optional; the high-level ones are normalized across providers, and `parameters` is forwarded verbatim to the provider, which validates it. | Field | What it does | | - | - | | `n` | Number of images to generate (default 1, max 4). | | `size` | Pixel size `WxH` for pixel-sized models (GPT Image, FLUX). Validated against the model's sizes. | | `aspect_ratio` | `16:9`, `1:1`, etc. for aspect-ratio models (Imagen, Pruna, xAI). Bridged to/from `size` per model. | | `quality` | Normalized tier: `low` \| `medium` \| `high` (default `medium`), plus `xhigh`, `max`, `auto` and `4k` where a model declares them. Rejected on models that declare no quality tiers, and on a tier the chosen model doesn't list. | | `response_format` | `b64_json` (default) or `url` (returns the stored URL only). | | `store` | Persist the output to storage and return a `file_id` + `url`. Defaults to false; a stored file is permanent until deleted. Refused on zero-data-retention projects. | | `async` | Run the job on the background worker: returns `202` with a URL to poll instead of blocking. One image per request. See [Heavy jobs](#heavy-jobs-async). | | `parameters` | Opaque per-provider passthrough for anything not normalized above. | Model-specific knobs go in `parameters` — for example GPT Image's `background` (`auto` | `opaque` | `transparent`), or a diffusion model's `seed` / `negative_prompt`. They're forwarded verbatim and validated by the provider. See [Discover models](#discover-models) for how to list the keys a given model accepts. ## Edit and image-to-image Pass a source image alongside the prompt. Each input accepts an http(s) URL, a data-URI, or a `file_id` from a previous generation or upload. Models that only do text-to-image reject these with a clear `400`. | Field | Use for | | - | - | | `image` | The source image to edit or transform. | | `mask` | An inpaint mask (`image/*`) marking the region to change. | | `reference_images` | Style or subject reference images. | ```bash theme={null} curl -sX POST https://api.opper.ai/v3/images \ -H "Authorization: Bearer $OPPER_API_KEY" -H "Content-Type: application/json" \ -d '{ "model": "openai/gpt-image-2.5-flare", "prompt": "put the balloon over a snowy mountain range", "image": "file_abc123" }' ``` ## Heavy jobs (async) A synchronous call is bounded by a **3-minute** server deadline. Nearly every generation fits: most models return in 5–30 s. The exception is the top quality tiers on the newest models — `max` on GPT Image 2.5 takes a minute or more. A job that overruns the deadline comes back as a `504` and is **not** billed. Set `async: true` to hand that job to the background worker. The call returns straight away with a `202`, an `id` and a `status_url`. Poll the status URL: `202` while the job is `processing`, then `200` with `status: "completed"`, a presigned `url`, the `mime_type`, the billed `usage`, and a reusable `file_id` when you passed `"store": true`. Async runs one image per request, so leave `n` at 1. ```bash Submit + poll theme={null} # 1. Submit — returns { "id": "gen_...", "status_url": ".../v3/artifacts/{id}/status" } curl -sX POST https://api.opper.ai/v3/images \ -H "Authorization: Bearer $OPPER_API_KEY" -H "Content-Type: application/json" \ -d '{ "model": "openai/gpt-image-2.5-sunburst", "prompt": "an oil painting of a lighthouse in a winter storm", "quality": "max", "async": true }' # 2. Poll: 202 while processing, then 200 with the image url curl -s https://api.opper.ai/v3/artifacts/{id}/status \ -H "Authorization: Bearer $OPPER_API_KEY" ``` ## Discover models `GET /v3/images/models` reports the models you can use and their capabilities. Each model's `params.image` block lists the normalized axes it accepts — `sizes`, `aspect_ratios`, `resolutions`, `qualities` — plus a `parameters` list of the native passthrough keys (e.g. `seed`, `negative_prompt`, `personGeneration`) you can put in `parameters`, so you can discover a model's knobs instead of guessing. ```bash theme={null} curl -s "https://api.opper.ai/v3/images/models" \ -H "Authorization: Bearer $OPPER_API_KEY" ``` ## What's next Send images into a model instead of generating them. Turn a generated image into video. Which models generate images, and at what sizes. Inspect generations and manage spend rules. # OCR Source: https://docs.opper.ai/build/multimodal/ocr Extract text and structure from PDFs and images as per-page markdown, through one gateway and key. `POST /v3/ocr` turns a document into per-page markdown synchronously. It's provider-agnostic: `model` selects the engine, the document source is normalized for you, and anything model-specific goes in `parameters`. Pass a `model` and a `document`. The `document` is one of four shapes, selected by `type`: | `document.type` | Fields | Use for | | - | - | - | | `document_url` | `document_url` | A PDF or image at an https URL | | `image_url` | `image_url` | An image at an https URL | | `base64` | `content`, `document_name` | Inline bytes you already hold | | `file` | `file_id` | A `file_` from [Files](/build/multimodal/files) | A `file_id` lets you OCR a document you already have on Opper without re-sending the bytes. Upload it to [Files](/build/multimodal/files) with `purpose: ocr_input` (PDFs and images), then reference it here. Files count toward your organization's storage quota; zero-data-retention projects can't hold files. ```bash cURL theme={null} curl -sX POST https://api.opper.ai/v3/ocr \ -H "Authorization: Bearer $OPPER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "mistral/mistral-ocr-latest", "document": { "type": "document_url", "document_url": "https://example.com/report.pdf" } }' ``` ```python Python theme={null} import os, requests r = requests.post( "https://api.opper.ai/v3/ocr", headers={"Authorization": f"Bearer {os.environ['OPPER_API_KEY']}"}, json={ "model": "mistral/mistral-ocr-latest", "document": {"type": "document_url", "document_url": "https://example.com/report.pdf"}, }, ) r.raise_for_status() pages = r.json()["pages"] print(pages[0]["markdown"]) ``` ```bash OCR a file_id theme={null} curl -sX POST https://api.opper.ai/v3/ocr \ -H "Authorization: Bearer $OPPER_API_KEY" -H "Content-Type: application/json" \ -d '{ "model": "opper/docling-latest", "document": { "type": "file", "file_id": "file_abc123" } }' ``` ```json Response theme={null} { "id": "ocr_...", "model": "mistral/mistral-ocr-latest", "pages": [ { "index": 0, "markdown": "# Report\n\nThe quarter...", "dimensions": { "width": 612, "height": 792 } } ], "usage": { "cost": 0.002, "pages_processed": 1 } } ``` | Field | What it does | | - | - | | `model` | Required. The OCR model, e.g. `mistral/mistral-ocr-latest` or `opper/docling-latest` (the older `docling/docling-latest` still resolves to the same model). | | `document` | Required. The source to read — a `document_url`, `image_url`, `base64` content, or a `file_id`. | | `pages` | 0-based page indices to process; omitted reads every page. | | `include_image_base64` | Return images embedded in the document as base64. | | `parameters` | Opaque per-provider passthrough — e.g. Docling's `ocr_engine`, `lang`, `table_mode`, `do_formula_enrichment`, or Mistral OCR 4.1's `include_blocks` and `confidence_scores_granularity` (see [Structured blocks](#structured-blocks-mistral-ocr-4-1)). | OCR is billed per page processed. The extracted markdown preserves structure — headings, tables, and lists. ### Languages (Docling) Docling accepts canonical ISO 639-1 language codes in `parameters.lang` (e.g. `["sv", "en"]`) with `parameters.ocr_engine` set to `tesseract` or `easyocr`; Opper maps them to each engine's own codes: ```bash theme={null} curl -sX POST https://api.opper.ai/v3/ocr \ -H "Authorization: Bearer $OPPER_API_KEY" -H "Content-Type: application/json" \ -d '{ "model": "opper/docling-latest", "document": { "type": "file", "file_id": "file_abc123" }, "parameters": { "ocr_engine": "tesseract", "lang": ["sv", "en"] } }' ``` ### Structured blocks (Mistral OCR 4.1) `mistral/mistral-ocr-4-1` (also reachable as `mistral/mistral-ocr-latest`) can return the page's layout in reading order, not just markdown. Opt in through `parameters`: | `parameters` key | Effect | | - | - | | `include_blocks` | Return a `blocks` array per page — each block has a `type` (`title`, `table`, `equation`, `signature`, `text`, …), a bounding box (`top_left_x`, `top_left_y`, `bottom_right_x`, `bottom_right_y`), and its `content`. | | `confidence_scores_granularity` | `"page"` for an aggregate page score, `"word"` for per-word scores. Adds a `confidence_scores` object to each page. | ```bash theme={null} curl -sX POST https://api.opper.ai/v3/ocr \ -H "Authorization: Bearer $OPPER_API_KEY" -H "Content-Type: application/json" \ -d '{ "model": "mistral/mistral-ocr-4-1", "document": { "type": "document_url", "document_url": "https://example.com/report.pdf" }, "parameters": { "include_blocks": true, "confidence_scores_granularity": "page" } }' ``` ```json Response (excerpt) theme={null} { "pages": [ { "index": 0, "markdown": "# Report\n\n...", "blocks": [ { "type": "title", "top_left_x": 64, "top_left_y": 41, "bottom_right_x": 166, "bottom_right_y": 56, "content": "Report" } ], "confidence_scores": { "average_page_confidence_score": 0.97, "minimum_page_confidence_score": 0.32 } } ] } ``` `blocks` and `confidence_scores` are only present when requested. Earlier Mistral models (`mistral/mistral-ocr-2512`) and Docling ignore these keys and respond as before. ## Discover models `GET /v3/ocr/models` lists the OCR models available, each with its `price_per_page`: ```bash theme={null} curl -s "https://api.opper.ai/v3/ocr/models" \ -H "Authorization: Bearer $OPPER_API_KEY" ``` ## What's next Upload once with `purpose: ocr_input`, reuse by `file_id`. Reason over a document with an LLM instead of extracting it. Which models do OCR. Inspect calls and manage provider, region, and spend rules. # Multimodality Source: https://docs.opper.ai/build/multimodal/overview Beyond text: images, audio, and video through the same gateway and key. The gateway isn't text-only. The same endpoint and API key reach models that see images and PDFs, generate images, speak and transcribe, and produce video — each with the routing and governance you get on every call, and listed in [Logs](/control-plane/logs) with its usage and cost. ## Modalities Send images and documents into a model as message content, with optional structured output. Generate and edit images with `POST /v3/images`. GPT Image, Imagen, FLUX, and more. Speech, transcription, music and sound effects with `/v3/audio/speech`, `/v3/audio/transcriptions` and `/v3/audio/generations`. Generate video from a prompt or reference image with `POST /v3/videos`. Two-way voice over WebSocket. OpenAI, xAI, and Gemini behind one protocol. The full catalog, with each model's input and output modalities marked. ## Discovering what a model can do Each modality has a discovery endpoint that reports the models available and their capabilities, so you don't have to hardcode a list: | Endpoint | Returns | | - | - | | `GET /v3/images/models` | Models for `POST /v3/images`, with sizes, aspect ratios, and edit support. | | `GET /v3/audio/models` | Speech (`tts`), transcription (`stt`), `music` and `sound` models, with voices, formats and capabilities. | | `GET /v3/videos/models` | Models for `POST /v3/videos`, with resolutions, aspect ratios, and max duration. | ```bash theme={null} curl -s "https://api.opper.ai/v3/images/models" \ -H "Authorization: Bearer $OPPER_API_KEY" ``` ## Input, output, and storage Generation endpoints (`/v3/images`, `/v3/audio/speech`, `/v3/audio/generations`, `/v3/videos`) return the result inline as base64 by default. Pass `"store": true` to also persist a copy to [Files](/build/multimodal/files) and get a reusable `file_id` and a presigned `url`. That stored output can be fed straight into a later call, such as an `image` into a video generation or an audio `file_id` into a transcription, without re-encoding. Stored outputs are permanent until you delete them; `store: true` is refused on zero-data-retention projects. # Video Source: https://docs.opper.ai/build/multimodal/video Generate video from a text prompt or reference image with a single async endpoint. `POST /v3/videos` is a deterministic, provider-agnostic video endpoint. You pass a `model` and a `prompt`; Opper runs the job and gives you a URL to poll. The same two-step flow works across every video provider (Google Veo, xAI Grok-Imagine, Alibaba Wan, Pruna, and more) — only the contents of `parameters` differ per model. Video generation is **asynchronous**: submitting returns immediately with an `id`, and you poll for the result. A clip typically takes from \~30 seconds to a few minutes depending on the model, resolution, and duration. ## How it works 1. **Submit** — `POST /v3/videos` with `model` + `prompt` → `202 { id, status_url }` 2. **Poll** — `GET {status_url}` until `status` is `completed` or `failed` 3. **Download** — fetch the presigned `url` from the completed response (valid 1 hour) ## Run your first generation Create a project-scoped runtime API key in the [Opper dashboard](https://platform.opper.ai). ```bash theme={null} curl -sX POST https://api.opper.ai/v3/videos \ -H "Authorization: Bearer $OPPER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "xai/grok-imagine-video", "prompt": "a hot air balloon drifting over green hills at dawn", "parameters": { "duration": 6, "aspect_ratio": "16:9", "resolution": "720p" } }' # → 202 { "id": "gen_...", "status_url": "https://api.opper.ai/v3/artifacts/gen_.../status" } ``` ```python theme={null} import os, time, requests BASE = "https://api.opper.ai" headers = {"Authorization": f"Bearer {os.environ['OPPER_API_KEY']}"} r = requests.post(f"{BASE}/v3/videos", headers=headers, json={ "model": "xai/grok-imagine-video", "prompt": "a hot air balloon drifting over green hills at dawn", "parameters": {"duration": 6, "aspect_ratio": "16:9", "resolution": "720p"}, }) r.raise_for_status() job = r.json() print("submitted:", job["id"]) ``` ```bash theme={null} # Poll (repeat until status is "completed" or "failed") curl -s https://api.opper.ai/v3/artifacts/gen_XXX/status \ -H "Authorization: Bearer $OPPER_API_KEY" # processing → { "status": "processing", "provider": "xai", "model": "xai/grok-imagine-video" } # done → { "status": "completed", "mime_type": "video/mp4", "url": "https://...presigned..." } # Download the finished clip curl -s "" -o video.mp4 ``` ```python theme={null} status_url = job["status_url"] while True: s = requests.get(status_url, headers=headers).json() if s["status"] == "completed": data = requests.get(s["url"]).content # presigned; no auth header needed open("video.mp4", "wb").write(data) print("saved video.mp4", len(data), "bytes") break if s["status"] == "failed": raise SystemExit(f"generation failed: {s.get('error')}") time.sleep(10) ``` ## Models and parameters `model` and `prompt` are the only fields Opper owns. Everything inside `parameters` is **forwarded verbatim** to the provider, which validates and rejects it — so the accepted keys depend on the model. Common examples: | Model | Example `parameters` | | - | - | | `xai/grok-imagine-video` | `{ "duration": 6, "aspect_ratio": "16:9", "resolution": "720p" }` | | `pruna/wan-t2v` | `{ "resolution": "720p", "aspect_ratio": "16:9" }` | | `vertexai/veo-3.1-fast-generate-001` | `{ "durationSeconds": 4, "resolution": "720p", "generateAudio": false }` | To discover what's available programmatically — including each model's accepted `aspect_ratios`, `resolutions`, and `max_duration` — call [`GET /v3/videos/models`](/v3-api-reference/video/list-video-models): ```bash theme={null} curl -s "https://api.opper.ai/v3/videos/models" \ -H "Authorization: Bearer $OPPER_API_KEY" ``` Each item's `capabilities` tells you the input modality: `video_generation` is text-to-video, `image_to_video` needs a reference image, and `video_editing` is video-to-video. ## Reference image and video Alongside `prompt`, you can pass a top-level `image` (image-to-video) or `video` (video-to-video). Each is an http(s) URL, a data-URI, or a `file_id`. * **`image`** — supported by `image_to_video` models, such as `vertexai/veo-3.1-generate-001`, `vertexai/veo-3.1-fast-generate-001`, `pruna/wan-i2v`, `xai/grok-imagine-video`. * **`video`** — supported by `video_editing` models, such as `pruna/vace`. A `file_id` from a previous [image generation](/build/multimodal/images) or upload works as the seed `image` here — no re-upload. See [Files](/build/multimodal/files) for how files work, lifecycle, and storage quotas. ```bash Seed a video from a file_id theme={null} curl -sX POST https://api.opper.ai/v3/videos \ -H "Authorization: Bearer $OPPER_API_KEY" -H "Content-Type: application/json" \ -d '{ "model": "xai/grok-imagine-video", "prompt": "the cat blinks slowly", "image": "file_abc123" }' ``` ```bash theme={null} # image-to-video (xAI): a public, server-fetchable image URL curl -sX POST https://api.opper.ai/v3/videos \ -H "Authorization: Bearer $OPPER_API_KEY" -H "Content-Type: application/json" \ -d '{ "model": "xai/grok-imagine-video", "prompt": "the cat slowly turns its head and blinks", "image": "https://example.com/cat.jpg", "parameters": { "duration": 6, "resolution": "720p" } }' ``` Source-format rules differ per provider: | Provider | `image` / `video` accepts | | - | - | | Veo | base64 / data-URI / `gs://` URI only — an http URL is not fetched. (A `gs://` object must be readable by Opper's Vertex service account, so base64/data-URI is the reliable choice.) | | Pruna, xAI | a **public, server-fetchable** URL only (a data-URI or private URL won't work). | ## Reference * [Generate a video](/v3-api-reference/video/create-video) — `POST /v3/videos` * [List video models](/v3-api-reference/video/list-video-models) — `GET /v3/videos/models` # Vision & PDFs Source: https://docs.opper.ai/build/multimodal/vision-pdfs Send images and PDF documents to the model as part of a chat message. Put an image or a PDF into a chat message and the model can read it. It's the same `messages` array, with a richer `content` field — no separate endpoint. Vision and PDF are model *capabilities*, so you send the media to a regular chat model that supports them. Not every model does. Filter the [catalog](/capabilities/models) by capability — or call `GET /v3/models?capability=vision` (images) or `?capability=pdf` (documents) — to find ones that accept each. The Claude, Gemini, and GPT families support both. ## Images Two ways to send an image: a hosted URL or inline base64. ```python Python theme={null} import os, base64 from openai import OpenAI client = OpenAI( base_url="https://api.opper.ai/v3/compat", api_key=os.environ["OPPER_API_KEY"], ) # Hosted URL r = client.chat.completions.create( model="openai/gpt-5-mini", messages=[{ "role": "user", "content": [ {"type": "text", "text": "What's in this picture?"}, {"type": "image_url", "image_url": {"url": "https://example.com/cat.jpg"}}, ], }], ) print(r.choices[0].message.content) # Inline base64 with open("cat.jpg", "rb") as f: b64 = base64.b64encode(f.read()).decode() r = client.chat.completions.create( model="openai/gpt-5-mini", messages=[{ "role": "user", "content": [ {"type": "text", "text": "What's in this picture?"}, {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{b64}"}}, ], }], ) ``` ```typescript TypeScript theme={null} import OpenAI from "openai"; import { readFileSync } from "node:fs"; const client = new OpenAI({ baseURL: "https://api.opper.ai/v3/compat", apiKey: process.env.OPPER_API_KEY!, }); // Hosted URL let r = await client.chat.completions.create({ model: "openai/gpt-5-mini", messages: [{ role: "user", content: [ { type: "text", text: "What's in this picture?" }, { type: "image_url", image_url: { url: "https://example.com/cat.jpg" } }, ], }], }); console.log(r.choices[0].message.content); // Inline base64 const b64 = readFileSync("cat.jpg").toString("base64"); r = await client.chat.completions.create({ model: "openai/gpt-5-mini", messages: [{ role: "user", content: [ { type: "text", text: "What's in this picture?" }, { type: "image_url", image_url: { url: `data:image/jpeg;base64,${b64}` } }, ], }], }); ``` Use the URL form when the image is already on the web. Use base64 for local files or anything not publicly reachable. ## PDFs PDFs work the same way. The model reads both the text and any embedded images (charts, diagrams, scanned pages). ```python Python theme={null} import base64 with open("contract.pdf", "rb") as f: b64 = base64.b64encode(f.read()).decode() r = client.chat.completions.create( model="openai/gpt-5-mini", messages=[{ "role": "user", "content": [ {"type": "text", "text": "Summarize the key clauses in this contract."}, {"type": "file", "file": {"file_data": f"data:application/pdf;base64,{b64}"}}, ], }], ) print(r.choices[0].message.content) ``` ## Free text or structured output | Need | Reach for | | - | - | | Show an image and ask a free-text question about it | A plain message (this page) | | Extract structured fields from an image or PDF (a receipt, an invoice, a form) | [Structured output](/build/gateway/structured-output) with `response_format` | | Run a multi-turn conversation about an uploaded document | A plain message (this page) | | Batch process documents into a database | [Structured output](/build/gateway/structured-output) | Add `response_format` when you want typed JSON out of an image or PDF. Leave it off when the model just needs to talk about the media. ## What's next Multimodal input with typed JSON output. Multi-turn chat. Works with image and PDF messages too. Which models accept which input types. # Build on Opper Source: https://docs.opper.ai/build/overview One gateway, every model and modality. Bring the SDK you already use. Everything you build runs through one gateway: a single endpoint and key in front of [700+ models](/capabilities/models), compatible with the OpenAI, Anthropic, and Google AI APIs. Pick the surface that matches what you're building — the routing, governance, and tracing are the same underneath. ## Where to start Text generation through the SDK you already use. Messages, tools, structured output, streaming, and web search. Images, audio, and video. Vision input, image generation, speech, transcription, and clips. Two-way voice over WebSocket. OpenAI, xAI, and Gemini behind one protocol. Ask several models the same question in parallel, then consolidate or compare. Yes/no probabilities, category choices, and ordered scores through OpenAI Decisions or TypeSafe System One. The full catalog, with EU-hosted and multimodal models marked. ## Pick by what you're building Use the [gateway](/build/gateway/drop-in-sdks). It handles multi-turn conversations, [tool calling](/build/gateway/tools), and image inputs out of the box, and you can use the OpenAI, Anthropic, or Google AI SDK you already know. Use [structured output](/build/gateway/structured-output). Attach the file, describe the fields you want with a JSON Schema, and get a clean, validated object back. Use [structured output](/build/gateway/structured-output). Describe the output shape with `response_format` and the model fills it in. For classification or scoring with probabilities and confidence, use a [decision model](/build/gateway/evaluations) through OpenAI Decisions or TypeSafe System One. Point your SDK at the gateway. Change the base URL and the API key, and your code keeps working—now with Opper's Logs and Rules. See [Drop-in SDKs](/build/gateway/drop-in-sdks). Use [Realtime](/build/realtime/quickstart). Open a websocket, stream microphone audio in, and stream voice out. Use the [Video](/build/multimodal/video) endpoint. Submit a prompt (and optionally a reference image), then poll for the result. Use [Roundtable](/build/roundtable/overview). Send one question to a panel of models and get back a consolidated answer or a side-by-side comparison. ## What's the same everywhere Whichever surface you use: * You can call any of the [700+ models](/capabilities/models) Opper supports. * Your [Rules](/control-plane/rules/overview) apply automatically. * Every call shows up in [Logs](/control-plane/logs) with its usage and cost, and is traced in full once you [turn tracing on](/control-plane/rules/retention). * Your API key is the same. You can mix them in one app. A common setup has the gateway powering the user-facing conversation while a background structured-output call parses uploads or summarizes context. # Realtime protocol Source: https://docs.opper.ai/build/realtime/protocol Full WebSocket protocol reference for two-way voice conversations with Opper. The Opper Realtime API lets you build voice-to-voice applications. Your customer speaks, the model responds with audio in real time, and tool calls flow back and forth without you holding the connection together yourself. The same protocol works across OpenAI, xAI, and Gemini, so you switch upstream models by changing a string. It also drives **live speech-to-text** — audio in, streaming transcript out — for transcription-only models like Mistral's EU-hosted Voxtral (see [Live transcription](#live-transcription-speech-to-text)). ## How it works You open a WebSocket to `wss://api.opper.ai/v3/realtime`, send a `session.start` frame that picks the model and configures the agent, then stream microphone audio in and play assistant audio out as it arrives. Opper resolves the provider from the model id, handles the upstream handshake (including provider quirks like xAI's client-secret exchange), meters usage as you go, and gives you one event vocabulary so your client code stays the same no matter which model you target. ``` ┌──────────┐ audio.append ┌──────────┐ provider-specific ┌────────────┐ │ Client │ ───────────────▶ │ Opper │ ──────────────────▶ │ Upstream │ │ (browser │ │ Realtime │ │ (OpenAI / │ │ or your │ ◀─────────────── │ proxy │ ◀────────────────── │ xAI / │ │ backend) │ audio.delta │ │ normalized frames │ Gemini) │ └──────────┘ └──────────┘ └────────────┘ ``` ## Just trying it out? The [Realtime quickstart](/build/realtime/quickstart) is the fastest way to a working session. It has one runnable code block per environment (server or browser), and you'll be live in under five minutes. For a production-shaped example with browser UI, microphone capture, audio playback, tool calls, and provider switching, see the [brainstorm-time cookbook](https://github.com/opper-ai/opper-cookbook/tree/main/examples/brainstorm-time). ## Authentication There are two patterns, depending on whether your client runs server-side or in a browser. ### Server-side: bearer token Server-to-server clients (Node, Python, Go, anything that runs in your backend) connect directly with a project-scoped runtime API key in the `Authorization` header. The [quickstart](/build/realtime/quickstart) uses this pattern. ```typescript theme={null} new WebSocket("wss://api.opper.ai/v3/realtime", { headers: { Authorization: `Bearer ${process.env.OPPER_API_KEY}` }, }); ``` Management keys (`op-mak-…`) are rejected. Only project-scoped runtime keys can open realtime sessions. ### Browser: ephemeral tickets Browsers can't set an `Authorization` header on the native `WebSocket` constructor. To open a realtime session directly from a browser, your backend mints a single-use ticket and the browser redeems it on the upgrade request. **Step 1: server-side mint.** Your backend hits `POST /v3/realtime-sessions` with its normal API key. The response carries a short-lived `client_secret` and an `expires_at` timestamp. ```typescript theme={null} // On your backend (e.g. /api/realtime-ticket) const resp = await fetch("https://api.opper.ai/v3/realtime-sessions", { method: "POST", headers: { "Authorization": `Bearer ${process.env.OPPER_API_KEY}`, "Content-Type": "application/json", }, body: JSON.stringify({ config: { model: "openai/gpt-realtime-2", voice: "marin", instructions: "You are a concise voice assistant.", tools: [/* ... */], }, ttl_seconds: 60, }), }); const { client_secret, expires_at, ws_url } = await resp.json(); // Return client_secret + ws_url to the browser ``` **Step 2: browser redeem.** The browser opens the WebSocket carrying the ticket in the `Sec-WebSocket-Protocol` subprotocol header. Prefer this form. Credentials in the URL query string end up in access logs, browser history, and `Referer` headers, while the subprotocol header is request-only and stays out of those places. ```typescript theme={null} // In the browser, recommended const ws = new WebSocket( "wss://api.opper.ai/v3/realtime", [`opper-ticket.${clientSecret}`], ); ws.onopen = () => { // The ticket already carries the bound config. For fields that weren't // pre-bound at mint time, send them now: ws.send(JSON.stringify({ type: "session.start", config: { modalities: ["audio"] }, })); }; ``` A query-parameter fallback is also accepted for environments that can't set a subprotocol (uncommon: most clients support the second `WebSocket` constructor argument): ```typescript theme={null} // Fallback only const ws = new WebSocket(`wss://api.opper.ai/v3/realtime?ticket=${clientSecret}`); ``` **Ticket properties:** * Single-use. Replays return `401`. * Default TTL 60 seconds, max 5 minutes. ### Pre-binding for security Any field you populate in `config` at mint time is locked for that ticket. The browser can't override it on `session.start`, and it can't change it later with `session.update` either. This is the main safety guarantee. A leaked ticket can only open the session your backend authorized. It can't pivot to a more expensive model or a different system prompt, and it can't unlock the field by updating it after start. At minimum, bind `model`. Tighter setups also bind `instructions`, `tools`, and `voice`. Fields left zero stay open for the browser to set in `session.start`. To force a boolean off (for example, to stop the browser from enabling output transcription), list the field in `locked_fields`. Without it, a zero value in `config` doesn't lock anything, so the browser could still flip the bool on. A ticket with `locked_fields` but no `config` is also valid: the listed fields are force-set to their zero value regardless of what the browser sends. Unknown field names are rejected at mint time with `400 Bad Request`. ```typescript theme={null} // Forbid the browser from enabling output transcription { config: { model: "openai/gpt-realtime-2", output_transcription: false, }, locked_fields: ["output_transcription"], } ``` ```typescript theme={null} // Server-side mint with strong binding. Browser cannot change any of these { config: { model: "openai/gpt-realtime-2", voice: "marin", instructions: "...locked system prompt...", tools: [/* customer-approved tools only */], reasoning_effort: "low", }, ttl_seconds: 30, } ``` ## Preflight rejections Before the WebSocket upgrade, Opper checks balance, spend caps, whether a card is on file, and concurrency caps. Failures come back as HTTP status codes rather than opaque WS closes, so you can react to them: | Status | Meaning | | - | - | | `401 Unauthorized` | Missing key, non-runtime key, or project-less key | | `402 Payment Required` | Balance exhausted, spend cap hit, or no card on file | | `429 Too Many Requests` | Concurrent-session cap reached for this project | | `503 Service Unavailable` | Realtime endpoint not configured for the requested provider | ## Session lifecycle A session goes through four phases. Most of the time you only care about the middle two. **1. Connect.** WebSocket upgrade. A successful response is `101 Switching Protocols`. After this the connection is open, but the agent isn't running yet. **2. Configure (`session.start`).** Send this once as the first message after upgrade. The config selects the model, voice, system instructions, tool list, VAD parameters, and optional transcription toggles. Opper validates the config against the resolved model's capabilities and rejects anything unsupported before dialing the upstream. The same validation runs on every later `session.update`, so unsupported `modalities`, `voice`, or `reasoning_effort` values return an error mid-stream. **3. Interact.** Stream audio in (`audio.append` + `audio.commit` if not using server VAD), send text turns (`text.input`), receive `audio.delta` / `text.delta` frames, handle `tool.call` events, return `tool.result`. **4. Close.** Either side can close the WebSocket. On a client-initiated close, Opper performs a final billing flush and tears down the upstream connection. On a server-initiated close (caps, idle timeout, balance exhaustion), you'll see a closing event (`session.terminating`) with a structured reason before the connection ends. ## Configuring the session Everything goes in `config` on the first `session.start` frame: | Field | Type | Notes | | - | - | - | | `model` | string | **Required.** Provider-prefixed model id, e.g. `openai/gpt-realtime-2`. See [Per-provider notes](#per-provider-notes). | | `voice` | string | Provider-specific voice name. Optional, model falls back to its default. | | `instructions` | string | System prompt for the agent. | | `modalities` | string\[] | Output modalities. OpenAI realtime accepts a single value (`["audio"]` or `["text"]`); xAI and Gemini accept either form. Default depends on model. | | `turn_detection` | object | `{ type: "server_vad", threshold, prefix_padding_ms, silence_duration_ms }`. Defaults are server-side. | | `tools` | object\[] | Tools the model can call, each `{ name, description, parameters }` with a JSON Schema for `parameters`. Flat, without the `function` wrapper used in [gateway tool calling](/build/gateway/tools). | | `reasoning_effort` | string | `gpt-realtime-2` only. `minimal` / `low` / `medium` / `high` / `xhigh`. | | `input_transcription` | bool | Surface `transcript.committed` events with the user's speech transcribed. Off by default. | | `input_transcription_model` | string | OpenAI only. Optional selector. Falls back to `gpt-4o-mini-transcribe`. | | `output_transcription` | bool | Surface `text.delta` events with the assistant's spoken words. Off by default. | ## Event vocabulary All events are JSON over text WebSocket frames. ### Client → server | Event | Purpose | | - | - | | `session.start` | First frame. Opens the session with config. | | `session.update` | Update the config mid-session (provider-dependent; Gemini ignores this). | | `audio.append` | Append base64-encoded PCM16 mic audio at the session's input sample rate. | | `audio.commit` | Mark end of speech (only needed when not using server VAD). | | `audio.clear` | Discard buffered mic audio that hasn't been committed yet. | | `text.input` | Send a typed user message. | | `image.input` | Send a still image as a user message. `image_url` accepts a base64 data URI (`data:image/png;base64,...`) on every vision-capable realtime model; OpenAI also accepts https URLs, while Gemini Live requires inline data and rejects https with a clear error. Optional `text` rides on the same message item so the model sees the question alongside the image. Optional `image_detail` (`"auto"` / `"low"` / `"high"`) is honored by OpenAI and ignored by Gemini. Only supported on realtime models whose capabilities include `vision` — the gateway responds with an `error` event otherwise. Image bytes are not stored in the session trace recording. | | `response.create` | Force the model to respond now (when not using server VAD). | | `response.cancel` | Cancel an in-flight response. | | `tool.result` | Return a tool result the model asked for via `tool.call`. | ### Server → client | Event | Purpose | | - | - | | `session.started` | Confirmation that the upstream session is live. Carries `session_id`, `input_sample_rate`, `output_sample_rate`, `audio_format`. | | `audio.delta` | Assistant audio chunk (base64-encoded PCM16). Play as it arrives. | | `text.delta` | Streaming assistant text. Only fires if `output_transcription` is on (or the model emits text natively). | | `transcript.committed` | User speech transcript, after VAD commits a turn. Only fires if `input_transcription` is on. | | `speech.started` / `speech.stopped` | VAD signals for the user's microphone. Use these to interrupt playback when the user barges in. | | `response.started` / `response.completed` | Lifecycle markers for an assistant turn. | | `tool.call` | Model wants to call a tool. You must reply with a matching `tool.result`. | | `session.terminating` | Server is closing the session. Carries a structured reason (see below). | | `session.ended` | Final frame before the upstream connection closes. | | `error` | Provider or protocol error mid-session. | Termination reasons on `session.terminating.error.code`: `session_timeout`, `idle_timeout`, `balance_exhausted`, `project_spend_cap_hit`, `org_spend_cap_hit`, `billing_not_supported`. ## Live transcription (speech-to-text) The same WebSocket also powers **transcription-only** models — audio in, streaming text out, no audio response. This is the low-latency, EU-hosted path for live captions or dictation when you don't need the model to talk back. Pick a transcription realtime model and request text output: ```json theme={null} { "type": "session.start", "config": { "model": "mistral/voxtral-mini-transcribe-realtime-2602", "modalities": ["text"] } } ``` Then stream microphone audio with `audio.append` exactly as you would for a voice session. The transcript arrives incrementally on `text.delta` and is finalized on `transcript.committed` at each segment boundary: ```typescript theme={null} switch (ev.type) { case "text.delta": // partial transcript, word by word caption += ev.delta; break; case "transcript.committed": // finalized segment console.log("final:", ev.transcript); break; } ``` There is no `audio.delta` — these models don't speak — and `voice`, `tools`, and the audio-output fields don't apply. Requesting `modalities: ["audio"]` returns an `error`. Diarization isn't available on the realtime path; for speaker labels use batch [transcription](/build/multimodal/audio) with `diarize: true`. `mistral/voxtral-mini-transcribe-realtime-2602` is EU-hosted (GDPR-resident) and billed per minute of input audio. ## Tool calls Tools follow the same round trip as [gateway tool calling](/build/gateway/tools): declare them in `config.tools` with a JSON Schema for parameters. When the model decides to call one, you get a `tool.call` event. Reply with `tool.result` and the model continues the conversation. Opper handles the provider wire format internally, so your handler is the same across upstreams. ```typescript theme={null} case "tool.call": const result = await runTool(ev.tool_name, ev.tool_arguments); ws.send(JSON.stringify({ type: "tool.result", tool_call_id: ev.tool_call_id, tool_result: result, })); break; ``` While a tool is executing, the upstream model is paused. You'll hear a silence in the audio stream proportional to your tool's latency. For tools that take more than a second or two, plan the UX around it with something like a "looking that up..." cue. ## Per-provider notes The protocol is the same across providers, but each upstream has quirks worth knowing. **Models:** `openai/gpt-realtime-2`, `openai/gpt-realtime-1.5`, `openai/gpt-realtime`, `openai/gpt-realtime-mini`. **Voices:** `alloy`, `ash`, `ballad`, `coral`, `echo`, `sage`, `shimmer`, `verse`, `marin`, `cedar` (latest models). **Sample rate:** 24 kHz symmetric (in/out). **Reasoning effort:** `gpt-realtime-2` accepts `reasoning_effort` and bills `reasoning` tokens. Other models ignore it. **Transcription:** input transcription runs in a parallel pipeline upstream, so the user transcript can arrive *after* the assistant has started responding. Enabling `input_transcription` adds a separate per-minute bill line (`input_transcription_ms`) at the rate of the transcription model. **Transcription is set-at-start.** `input_transcription` and `input_transcription_model` cannot be toggled via `session.update` on OpenAI. Mid-session changes return an error. Request transcription at `session.start` or not at all. xAI and Gemini fold transcription into their per-minute audio rate and don't have this restriction. **Models:** `xai/grok-voice-latest`. **Voices:** `ara`, `eve`, `leo`, `rex`, `sal`. **Sample rate:** 24 kHz symmetric. **Billing:** flat per-minute rates for audio in / audio out. Input and output transcription are folded into the per-minute rate; toggling them on doesn't cost extra. **Connect latency:** xAI requires a three-step handshake (client-secret exchange, WS dial, session.updated). Expect \~1s before `session.started` lands. **Models:** `gemini/gemini-3.1-flash-live-preview`. **Voices:** `Puck`, `Charon`, `Kore`, `Fenrir`, `Aoede`, `Leda`, `Orus`, `Zephyr`. **Sample rate:** **asymmetric**. 16 kHz input, 24 kHz output. Both rates are reported in `session.started`. Read `input_sample_rate` for mic capture and `output_sample_rate` for playback. **Tools:** supported; translated to Gemini's `toolResponse` format internally. **Mid-turn close:** Gemini reports usage only at turn boundaries. If you close mid-sentence, Opper waits briefly for the trailing usage frame before tearing down. **Models:** `mistral/voxtral-mini-transcribe-realtime-2602`. Transcription-only — audio in, text out, no audio response. See [Live transcription](#live-transcription-speech-to-text). **Sample rate:** 16 kHz input (PCM16). No audio output. **Output:** streaming `text.delta` plus a `transcript.committed` at each segment boundary. Set `modalities: ["text"]`; `voice` and `tools` don't apply, and `diarize` isn't supported on the realtime path. **Region:** EU-hosted (GDPR-resident). **Billing:** per minute of input audio (`audio_input_per_minute`); there is no output-audio charge. ## Session limits | Limit | Default | Trigger | | - | - | - | | Concurrent sessions per project | 5 | `429` at preflight | | Max session duration | 30 minutes | `session.terminating` with `session_timeout` | | Idle timeout (no client traffic) | 60 seconds | `session.terminating` with `idle_timeout` | | Session-start grace period | 10 seconds | WebSocket close if `session.start` never arrives | ## Billing Realtime sessions bill incrementally as the conversation runs. Per-model rates are published on `GET /v3/models`. Look for `audio_input`, `audio_output`, `audio_input_per_minute`, `audio_output_per_minute`, and `input_transcription_per_minute` on the relevant model. ## Tracing Each session is listed in [Logs](/control-plane/logs) with its usage and cost. With tracing on (set in [Opper retention](/control-plane/rules/retention)), its generation records also appear in the [Trace explorer](/control-plane/trace), filterable by `model` or `session_id`. ## See also * [Realtime protocol reference](/v3-api-reference/realtime/protocol): event vocabulary quick-ref for the API reference section. * [brainstorm-time cookbook](https://github.com/opper-ai/opper-cookbook/tree/main/examples/brainstorm-time): full working voice app with browser UI, tools, and provider switching. * [Models](/capabilities/models): the full list of supported realtime model ids. # Realtime quickstart Source: https://docs.opper.ai/build/realtime/quickstart Open your first voice-to-voice session in under five minutes. This is the shortest path to a working voice loop. Pick the tab that matches where your code runs, copy the snippet, and swap in your API key. For the full protocol (events, config fields, per-provider notes, billing, transcription, tool flow), see the [Realtime protocol](/build/realtime/protocol). ## Run your first session Create a project-scoped runtime API key in the [Opper dashboard](https://platform.opper.ai). Server-side clients connect directly with a bearer token. This is the quickest way to check that the endpoint works. ```bash theme={null} npm install ws ``` ```typescript theme={null} import WebSocket from "ws"; const ws = new WebSocket("wss://api.opper.ai/v3/realtime", { headers: { Authorization: `Bearer ${process.env.OPPER_API_KEY}` }, }); ws.on("open", () => { ws.send(JSON.stringify({ type: "session.start", config: { model: "openai/gpt-realtime-2", voice: "marin", instructions: "You are a concise voice assistant.", }, })); }); ws.on("message", (raw) => { const ev = JSON.parse(raw.toString()); if (ev.type === "session.started") { console.log(`live: ${ev.session_id} @ ${ev.output_sample_rate}Hz`); } if (ev.type === "audio.delta") { // ev.audio is base64-encoded PCM16. Pipe to your audio player. } }); ``` Browsers can't set an `Authorization` header on the native `WebSocket` constructor. Mint a single-use ticket from your backend, then redeem it from the browser. **Step A. Your backend mints the ticket:** ```typescript theme={null} // POST from your trusted server (Node, Python, anything) const resp = await fetch("https://api.opper.ai/v3/realtime-sessions", { method: "POST", headers: { Authorization: `Bearer ${process.env.OPPER_API_KEY}`, "Content-Type": "application/json", }, body: JSON.stringify({ config: { model: "openai/gpt-realtime-2", voice: "marin", instructions: "You are a concise voice assistant.", }, }), }); const { client_secret } = await resp.json(); // Return client_secret to the browser. ``` **Step B. The browser redeems it via subprotocol header:** ```typescript theme={null} const ws = new WebSocket( "wss://api.opper.ai/v3/realtime", [`opper-ticket.${clientSecret}`], ); ws.onopen = () => { // Bound fields (model, voice, instructions) are already locked in // by the ticket. Send any remaining session.start fields here. ws.send(JSON.stringify({ type: "session.start", config: {} })); }; ws.onmessage = (e) => { const ev = JSON.parse(e.data); // Handle session.started, audio.delta, tool.call, etc. }; ``` Tickets are single-use and expire in 60 seconds by default. Whatever fields you populate in the mint request are **locked**, so the browser can't override them. See [Pre-binding for security](/build/realtime/protocol#pre-binding-for-security). On a successful open: ```json theme={null} { "type": "session.started", "session_id": "sess_...", "input_sample_rate": 24000, "output_sample_rate": 24000, "audio_format": "pcm16" } ``` From here, stream audio in with `audio.append` (base64-encoded PCM16 at `input_sample_rate`), and the assistant's audio comes back in `audio.delta` frames at `output_sample_rate`. If a tool fires you get a `tool.call` and reply with `tool.result`. When the session ends you get `session.terminating` followed by a clean WebSocket close. ## Switch providers The protocol is the same across providers. Change one string: | Provider | Model id | Notes | | - | - | - | | OpenAI | `openai/gpt-realtime-2` | Reasoning effort supported. 24 kHz symmetric. | | xAI | `xai/grok-voice-latest` | Per-minute billing. 24 kHz symmetric. | | Gemini | `gemini/gemini-3.1-flash-live-preview` | **Asymmetric** sample rates: 16 kHz in, 24 kHz out. | See [Per-provider notes](/build/realtime/protocol#per-provider-notes) for voice lists and quirks. ## What's next Every config field, event, and per-provider note. The full list of supported realtime model IDs. `POST /v3/realtime-sessions` request and response schema. A complete browser voice app with microphone capture and tool calls. # Roundtable Source: https://docs.opper.ai/build/roundtable/overview Ask several models the same question in parallel, then consolidate or compare their answers. Roundtable is in **beta**. Its request and response shapes may change without the usual deprecation window. `POST /v3/roundtable` sends one prompt to several models in parallel and brings their answers back together. Instead of trusting a single model, you get a panel: ask three models the same question and either merge their responses into one consolidated answer or compare them side by side. It runs on the same gateway, key, routing, and tracing as every other call. Use it when an answer is high-stakes or subjective and a second (and third) opinion is worth the extra cost: architecture decisions, evals and grading, content review, or any "let's see what a few good models think" moment. ## How it works 1. **Fan out** — your `input` goes to every model in `models`, in parallel. 2. **Resolve** — depending on the resolution mode, the results are consolidated into one answer, returned as-is, or voted into a single choice. 3. **Return** — you get the consolidated `data`, aggregated `meta` (models used, total cost, duration), and the per-model `model_results`. ## Resolution modes | Mode | What you get | | - | - | | `summary` (default) | A consolidation model merges every answer into one `data`. Best when you want a single, reconciled response. | | `fast` | No consolidation — `data` is null and you read `model_results` directly. Best when you want to compare answers yourself. | | `multiple_choice` | Each model picks from the `choices` you supply; the result is the consolidated winner. Best for classification or voting. | In `summary` mode you can override the consolidation model with `summary_model`. In `multiple_choice` mode, pass the literal options in `choices`. ## Shared options These apply across every model in the panel: | Field | What it does | | - | - | | `instructions` | A system prompt shared by all model calls. | | `output_schema` | A JSON Schema applied to every model and the consolidation, so each answer comes back validated. | | `stream` | Stream progress over SSE — per-model deltas, then the consolidation. | | `max_retries` / `timeout_ms` | Per-model retry count and timeout covering all retries. | Each entry in `models` can also override `temperature`, `max_tokens`, `top_p`, and `reasoning` effort for that model alone. ## What's next Run your first roundtable in a few lines. The full request and response schema. Pick the models for your panel. Constrain every answer to a schema. # Roundtable quickstart Source: https://docs.opper.ai/build/roundtable/quickstart Ask three models one question and get a consolidated answer. Roundtable is in **beta**. Its request and response shapes may change without the usual deprecation window. ## Ask a panel Send one `input` to several `models`. With the default `summary` resolution, a consolidation model merges their answers into a single `data`. ```bash cURL theme={null} curl -sX POST https://api.opper.ai/v3/roundtable \ -H "Authorization: Bearer $OPPER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "What is the best database for a multi-tenant SaaS, and why?", "models": [ { "name": "openai/gpt-5.5" }, { "name": "anthropic/claude-sonnet-4-6" }, { "name": "gemini/gemini-3.1-pro-preview" } ] }' ``` ```python Python theme={null} import os, requests r = requests.post( "https://api.opper.ai/v3/roundtable", headers={"Authorization": f"Bearer {os.environ['OPPER_API_KEY']}"}, json={ "input": "What is the best database for a multi-tenant SaaS, and why?", "models": [ {"name": "openai/gpt-5.5"}, {"name": "anthropic/claude-sonnet-4-6"}, {"name": "gemini/gemini-3.1-pro-preview"}, ], }, ) r.raise_for_status() out = r.json() print("consolidated:", out["data"]) print("total cost:", out["meta"]["total_cost"]) for m in out["model_results"]: print(m["model"], "→", m["data"][:80]) ``` ```json Response theme={null} { "id": "rt_...", "data": "The panel broadly favors PostgreSQL with row-level security...", "meta": { "resolution": "summary", "models_used": ["openai/gpt-5.5", "anthropic/claude-sonnet-4-6", "gemini/gemini-3.1-pro-preview"], "total_cost": 0.021, "total_duration_ms": 5210, "summary_model": "openai/gpt-5.5" }, "model_results": [ { "index": 0, "model": "openai/gpt-5.5", "data": "...", "cost": 0.006 }, { "index": 1, "model": "anthropic/claude-sonnet-4-6", "data": "...", "cost": 0.008 }, { "index": 2, "model": "gemini/gemini-3.1-pro-preview", "data": "...", "cost": 0.007 } ] } ``` ## Vote on a choice For classification or voting, set `resolution` to `multiple_choice` and pass the options in `choices`. Each model picks one and the result is the consolidated winner. ```bash theme={null} curl -sX POST https://api.opper.ai/v3/roundtable \ -H "Authorization: Bearer $OPPER_API_KEY" -H "Content-Type: application/json" \ -d '{ "input": "Is this review positive, negative, or neutral? \"It works, but support was slow.\"", "models": [ { "name": "openai/gpt-5-mini" }, { "name": "anthropic/claude-haiku-4-5" }, { "name": "gemini/gemini-3.8-flash" } ], "resolution": "multiple_choice", "choices": ["positive", "negative", "neutral"] }' ``` ## Compare without consolidating To read each model's answer yourself with no consolidation step, set `resolution` to `fast`. `data` comes back null and you use `model_results` directly. Add an `output_schema` to constrain every model — and the consolidation — to the same JSON Schema, and pass `instructions` as a system prompt shared across the panel. Set `stream: true` to receive per-model deltas over SSE. See the [overview](/build/roundtable/overview) and [API reference](/v3-api-reference/roundtable/create-roundtable). # Custom models Source: https://docs.opper.ai/capabilities/custom-models Bring your own key: register your own provider deployments and API keys, then call them like any other Opper model. You aren't limited to Opper's hosted models. Register your own provider deployment, for example a private Azure OpenAI deployment, with its API key. It then appears in the catalog alongside built-in models and gets the same routing, governance, and tracing. ## Register a custom model The quickest way is in the dashboard. Open **Settings → Models → Custom** in [platform.opper.ai](https://platform.opper.ai) and add a model: | Field | What to enter | | - | - | | **Name** | A label, e.g. "My Custom Model". | | **Identifier** | How you'll call it, e.g. `example/my-gpt4`. | | **Type** | The provider type (Azure, OpenAI-compatible, and so on). | | **Credentials** | The provider's API key, token, or service-account JSON. | | **API URL** | Your endpoint, e.g. `https://my-deployment.openai.azure.com/`. | Save it, then call the model by its identifier like any other. Custom models are scoped to your project. The identifier prefix (e.g., `example/`) is the namespace. ### Register from the API or CLI If you'd rather automate it: ```shell theme={null} curl -X POST https://api.opper.ai/v2/custom-models \ -H "Content-Type: application/json" \ -H "Authorization: Bearer ${OPPER_API_KEY}" \ -d '{ "name": "example/my-gpt4", "provider": "azure", "model_id": "gpt4-production", "api_key": "my-api-key-here", "params": { "api_base": "https://my-gpt4-deployment.openai.azure.com/", "api_version": "2024-06-01" } }' ``` ```shell theme={null} opper models create example/my-gpt4 azure/gpt4-production my-api-key-here \ '{"api_base": "https://my-gpt4-deployment.openai.azure.com/", "api_version": "2024-06-01"}' ``` ## What's next The full catalog of built-in models. How routing and model selection fit together. # Models Source: https://docs.opper.ai/capabilities/models Every model Opper supports, with EU-hosted options marked. Opper supports 700+ models from 50+ providers through one gateway and key. Browse the full, always-current catalog — with pricing, context windows, capabilities, and EU-hosted models marked — at [opper.ai/models](https://opper.ai/models). The complete model catalog. Copy a model id to use it from the SDK or any drop-in client. Call generation models by their provider-prefixed name (`openai/gpt-5.5`, `anthropic/claude-sonnet-4-6`, `gemini/gemini-3.1-pro-preview`) through the compatible SDKs. See [Drop-in SDKs](/build/gateway/drop-in-sdks). Models for other tasks use their dedicated endpoints. ## Evaluation models Decision models provide yes/no probabilities, category choices, and ordered scores. Discover them with `GET /v3/models?type=evaluation`, then call `POST /v3/compat/decisions` (OpenAI's Decisions format) or `POST /v3/compat/v1/systemone` (TypeSafe's System One format) with your project API key. Every decision model answers on both; see [Classification & scoring](/build/gateway/evaluations). Evaluation models cannot be called through Chat Completions. ## Multimodal models Beyond text, the catalog includes models that accept images, audio, and video, and models that generate images, speech, and video. Filter the [catalog](https://opper.ai/models) by capability — `vision`, `audio`, `video`, `image gen` — to see what each model supports. For the modality-specific endpoints, a discovery call returns the usable models and their capabilities at runtime, so you don't have to hardcode a list: | Endpoint | Models for | | - | - | | `GET /v3/images/models` | [Image generation](/build/multimodal/images) (`POST /v3/images`) | | `GET /v3/audio/models` | [Speech & transcription](/build/multimodal/audio) (`POST /v3/audio/speech`, `/v3/audio/transcriptions`) | | `GET /v3/videos/models` | [Video generation](/build/multimodal/video) (`POST /v3/videos`) | For sending images and PDFs into a text model, see [Vision & PDFs](/build/multimodal/vision-pdfs). ## Fallbacks and backup chains There are two ways to get a fallback chain. **Per request, on Chat Completions.** Send the primary in `model` and the rest in `models`, the same shape OpenRouter uses: ```json theme={null} {"model": "openai/gpt-5.6-luna", "models": ["azure/gpt-5.6-luna", "deepseek-v4.1-flash"], "messages": [...]} ``` Entries are tried in order when the one before fails for any reason. An entry can be a pinned id, a bare pool name (its providers are tried in turn) or a legacy alias. A [compliance rule](/control-plane/rules/overview) that blocks an entry removes it from the chain without an error. `meta.routing.attempts` lists every attempt with its status and `meta.routing.served` names the model that answered. Only `/v3/compat/chat/completions` reads `models`. **Managed on the gateway, for every endpoint.** Build a [dynamic route](/capabilities/routes/overview), give it the models you want to try, deploy it, and call the route's name as `model` from any SDK, on every endpoint. This is the option to reach for when the chain should live in one place, with ranking, a version history and rollback. The gateway tries the models in the route and the first one that answers wins; a model that is down or rate-limited is passed over, and in a Pool so is one that a [compliance rule](/control-plane/rules/overview) blocks for your organization — neither comes back as an error. ```text theme={null} production-main → Pool: azure/gpt-5.6-luna, melious/deepseek-v4.1-flash ranked per request on price, latency or throughput; the first that answers wins ``` Two nodes answer a request: * A [**Pool**](/capabilities/routes/pool) holds several models and re-ranks them on every request on live numbers — cheapest, fastest, highest throughput, or a blend. Use it when any of the models is acceptable and you want the gateway to pick. * A **Model** node calls one model, with fallbacks in the fixed order you wrote down. Use it when the order matters — "this model, and only if it fails, that one". Either can sit on the other's fallback edge, so a route can say "try this pool, then that model". Every attempt is recorded on the trace, so you can see which model actually answered and why the others were passed over. Two things this buys you: * **Resilience.** If the primary provider is down or rate-limited — or, in a Pool, a compliance rule takes it out of play — the call continues to the next model instead of failing. * **Swap models without touching code.** Point your app at the route name, then reorder or repoint the route in the platform — no deploy. Routes are part of the [Control Plane](/control-plane/overview). To put your own provider deployments into a route, register them as [custom models](/capabilities/custom-models) first. Model **aliases** (Settings → Models), a stable name mapping to an ordered list of models, still resolve on every call for organizations that use them. New fallback chains should be built as dynamic routes: they give you ranking, per-attempt traces, and the same skip-on-compliance behaviour. ## What's next Pick the model per request: pools, branches, splits, classifiers. Register your own model deployments and API keys. Use Opper as the provider for your editor, agent, or CLI. # Dynamic routes Source: https://docs.opper.ai/capabilities/routes/overview Build a graph that picks the model per request, deploy it, and call it like a model. A dynamic route is a graph you author that decides, per request, which model answers. Instead of naming one model in your code, you name the route. The graph can branch on the request, split traffic, ask a classifier, or rank a set of models on live numbers — and you change it without touching code. The route editor: Start, a Pool node and End on the canvas, with the pool's inspector on the right ## Call a route A deployed route is a model name. Pass `dynamic/` wherever a model goes, such as the chat completions and Messages endpoints or the SDKs: ```bash theme={null} curl https://api.opper.ai/v3/compat/chat/completions \ -H "Authorization: Bearer $OPPER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "dynamic/support-router", "messages": [{"role": "user", "content": "My invoice is wrong."}] }' ``` The response's `meta.routing` says what happened: `requested` is the route, `route` names the version that ran and the node that answered, and `served` is the model that produced the answer. ## Build one Open **Models → Routes** in [platform.opper.ai](https://platform.opper.ai) and create a route. A new route is three nodes — Start, a Model and End — which already works: the model answers and the walk ends. * Click the **+** on an edge to add a node there. Under **Split traffic** you get **If / Else**, **Traffic split** and **Classifier**; below them, **Add Model** and **Add Pool**. * Select a node to edit it in the inspector on the right. * **Save draft** keeps your work without changing traffic. **Publish** deploys it. Routes can also be created, simulated, deployed and rolled back from code through the [Management API](/control-plane/management-api#deploy-a-dynamic-route-from-code), with a management key carrying the `dynamic_routes:write` scope. The editor and the API save the same graph. | Node | What it does | | - | - | | **Model** | Calls one model. Add fallback models to try, in a fixed order, when it fails. | | **Pool** | Holds several models and ranks them on every request — cheapest, fastest, highest throughput, or a weighted blend. See [Pool node](/capabilities/routes/pool). | | **Branch** (If / Else) | Evaluates conditions in order; the first one that is true wins, and a default branch catches the rest. | | **Split** (Traffic split) | Sends a percentage of requests down each branch. The weights must add up to 100. | | **Classifier** | Asks a model which of your labelled cases the request belongs to, up to ten, and takes that branch. | | **Observe** | Scores the answer with an LLM judge after the response has been sent, and attaches the score to the trace. | Model and Pool are the nodes that answer: a request stops at the first one that produces an answer. Each also has a **fallback** edge, taken when every attempt on it failed, so a route can say "try this pool, then that model". Observe nodes can only follow an answer. ## Conditions Branch conditions are [CEL](https://cel.dev) expressions over the request. `metadata` is the request's metadata map, `auth` the caller, and `input` the request itself: ``` metadata.tier == "enterprise" has(metadata.locale) && metadata.locale.startsWith("sv") ``` Anything that does not compile is rejected when you publish, never at request time. ## Draft and deployed Editing changes the draft. Publishing snapshots it as a numbered version and points the route at it. Requests only ever run the active version, so a half-finished edit cannot reach traffic, and a bad deploy is undone by pointing the route back at the previous version. **Simulate** sends a sample request through the draft and shows the path it takes — which branch matched, which model answered and, for a pool, the order its models would be tried in. ## What's next Rank several models per request on price, latency or throughput. The same ordering for the pool behind a bare model name, no graph needed. # Pool node Source: https://docs.opper.ai/capabilities/routes/pool Rank several models per request on price, latency or throughput, and try them in that order. A pool node holds several models and decides, on every request, which one to try first. The first model that answers wins; if all of them fail, the route takes the pool's fallback edge. A model node with fallbacks is a fixed order you wrote down when you built the route. A pool re-ranks on numbers that track what providers are doing right now. A pool's inspector: two GPT OSS 120B endpoints with a score beside each, the order chips and the three weight bars ## Add a pool Click the **+** on an edge and choose **Add Pool**. A pool can go at the end of a path, in an empty branch, or on another node's fallback edge — anywhere an answer is expected. A new pool has two rows to fill, because two is the smallest set that can be ordered; **Add model** takes it up to ten. Each row is a model picker that offers individual provider endpoints only — `groq/gpt-oss-120b`, not the bare `gpt-oss-120b` — since a bare name is already a pool of its own, and a pool ranks single providers. A model another row already holds is not offered again. ## Choose the order | Order | Tries first | | - | - | | **Cheapest first** | The lowest catalog list price. | | **Fastest first** | The lowest median time to first token, measured over the last day. | | **Highest throughput** | The most completion tokens per second, measured over the last day. | | **Weighted** | A blend of the three, on the weights you set. | The three bars under the chips are the weights, and each chip is a preset for them: **Cheapest first** is cost at 100 and nothing else. Drag a second bar up and the order becomes **Weighted** by itself. Only the ratios matter. Each metric is scaled against a typical value first, so "cost 50, latency 25" means price counts twice as much as latency. A weight of zero drops that metric from the blend, so a model with no latency number still ranks on a blend that only weighs cost. Price is published, so every model ranks on it from the start. Latency and throughput are measured from recent traffic across Opper, not just yours; a model that has seen little of it has no number yet and sorts last until it does. A pool that is new to you ranks fully on **Cheapest first** from day one. ## The score beside each model Each row shows the model's score for the chosen order right now: the sort key evaluated for that model, lower tried first, from the same live numbers a request uses. The rows stay in the order you added them — the score tells you the order the runtime will use. A model the order cannot score reads **no score** and sorts last. Hover the chip for the numbers behind it: list price, time to first token, throughput and how many calls were measured. ## Failover The ranking is a preference, not a filter. A model that sorts last is still tried when everything ahead of it fails. When every model in the pool has failed, the route takes the pool's **fallback** edge — a fallback model, or End, in which case the caller gets the last error. The trace of each call records the order the pool tried, and the response's `meta.routing` names the model that answered. ## The same order as a routing rule The [Routing rule](/control-plane/rules/routing) applies this ordering to the pool behind a bare model name — every provider serving `gpt-oss-120b`, with no graph involved. A pool node applies it to the models you list. Both use the same vocabulary and the same ranking, so a rule and a pool that ask for the same order rank identically on identical numbers. # Keep AI inference in the EU (EU data residency) Source: https://docs.opper.ai/control-plane/eu-data-residency Enforce EU-only inference and storage for every AI model call with one organization-wide Model access rule, or choose an EU route per call on any plan. Opper itself is hosted in Stockholm. Opper is hosted in the EU, in Stockholm. Where a model call runs depends on the route it uses, and you can keep inference in the EU in two ways: * **Per call, on any plan.** Call an EU route, such as `azure/gpt-5.5` (Sweden) or `mistral/mistral-large-2512` (France). That call runs in the EU. * **For your whole organization, on the Control Plane plan.** A [Model access](/control-plane/rules/model-access) rule with **Inference location** and **Storage location** set to EU. It is an account-level lock, not a filter in a model picker: Opper refuses every call to a route outside the EU with `403` before the request leaves Opper, whichever API key, SDK, or app sends it. ## Where your data is processed | Layer | Where | What it covers | | - | - | - | | **Opper platform** | Stockholm, Sweden: AWS `eu-north-1`, with a second production environment at evroc | The gateway, traces, usage records, and stored files | | **The model call** | Where the route runs, for example Sweden for `azure/gpt-5.5` or the United States for `anthropic/claude-sonnet-4-6` | The prompt and response, while the provider processes them | Every route in the catalog records two locations: * **Inference location**: where the model runs. * **Storage location**: where the provider keeps any content it retains, such as logs or content held for moderation. A route that retains nothing passes any storage requirement. A route that retains content without a recorded storage location counts as `GLOBAL` and fails an EU storage requirement. ## Enforce EU-only for your organization In the [Opper platform](https://platform.opper.ai), open **Rules → Model access**: Select **Set an org allowlist**, or edit the existing one. Set **Inference location** to **EU** and **Storage location** to **EU**. Use **EEA** instead to also allow Norway, Iceland, and Liechtenstein, or pick individual countries. Select the match counter to review the models that remain, then select **Save changes**. The rule applies from the next request. The same rule through the [Management API](/control-plane/management-api#manage-rules-from-code): ```bash theme={null} curl -X POST https://api.opper.ai/management/v1/controls/rules \ -H "Authorization: Bearer $OPPER_MANAGEMENT_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "id": "'$(uuidgen | tr A-Z a-z)'", "kind": "comply", "enabled": true, "scope": {"type": "org"}, "config": {"type": "allowlist", "allowlist": {"provider": {"locations": {"inference": {"include": ["EU"]}, "storage": {"include": ["EU"]}}}}} }' ``` ### What the rule covers * **Every model call**: text generation, embeddings, image generation, speech, transcription, video, OCR, realtime voice, and rerank. * **Every API key and project** in the organization. A project override can narrow the rule further, but cannot allow a location the organization rule excludes. * **Routes without a recorded location.** A route with no recorded inference location, or one that retains content with no recorded storage location, counts as `GLOBAL` and is refused. Inference location says where a model runs, not who operates it. Several EU routes are run by companies headquartered outside the EU, such as Azure and AWS in Sweden. To restrict by the operator's or the model maker's home country as well, add **Provider country** or **Maker country** to the same rule. For a gateway that is European-owned end to end, on a European cloud, see [Sovereign AI](/control-plane/sovereign-ai). ## What a blocked call returns A call to a route outside the EU is refused with `403` before anything reaches the provider. On chat completions: ```json theme={null} { "error": { "code": "permission_error", "message": "model \"anthropic/claude-sonnet-4-6\" is blocked by the active model allowlist: inference location US is outside EU" } } ``` Other endpoints return the same message in their own error shape. See [What a blocked call returns](/control-plane/rules/model-access#what-a-blocked-call-returns). ## Bare model names mix regions A bare model name is served by a pool of routes, and a pool can span regions. In September 2026, `claude-sonnet-4-6` was served from these five routes: | Route | Inference location | Under an EU inference and storage rule | | - | - | - | | `anthropic/claude-sonnet-4-6` | United States | Skipped | | `vertexai/claude-sonnet-4-6` | Global | Skipped | | `vertexai/claude-sonnet-4-6-eu` | EU | Used | | `aws/claude-sonnet-4-6-eu` | Sweden | Used | | `azure/claude-sonnet-4-6` | Sweden | Skipped: retains content with no recorded storage location | Under that rule, a call to `claude-sonnet-4-6` runs only on the routes that pass and skips the rest. Without a rule, it can run on any of them. To keep a single call in the EU without a rule, pin an EU route such as `aws/claude-sonnet-4-6-eu`. ## Find EU routes * **Browse:** [opper.ai/models/eu](https://opper.ai/models/eu), or the machine-readable [opper.ai/models/eu.json](https://opper.ai/models/eu.json). * **Query the catalog** without an API key: ```bash theme={null} curl "https://api.opper.ai/v3/models?type=llm&inference_location=EU&storage_location=EU&limit=0" ``` `inference_location` and `storage_location` match the same way as the Rules page, so the result is what a rule with the same settings allows. `limit=0` returns every match and `total` gives the count. Add `&training=no&logging=none` for routes that are both EU and [zero data retention](/control-plane/zero-data-retention). ## EU and zero data retention together EU residency and zero data retention are separate requirements, and one rule can hold both: set the locations above and set **Provider data policy** to **Zero data retention**. See [Enforce zero data retention](/control-plane/zero-data-retention) for what that policy covers. ## Related Store no content at Opper and allow only providers that neither train on nor log it. Every allowlist field, error body, and override. Certifications, hosting, DPA, and sub-processors. Route across EU models by price, latency, or throughput. # Logs Source: https://docs.opper.ai/control-plane/logs Every provider call a project makes, with its model, provider, status, tokens, cost and latency, plus its input and output when a retention rule stores them. **Logs** lists every call a project makes to a model provider, newest first. Each row is the usage record Opper writes for every call on every plan, so Logs works without tracing: it shows which provider served a call, how long it took, what it cost, and why it failed. A call's input and output appear in its details when a [retention rule](#inputs-outputs-and-retention) stored them. Open a project in [platform.opper.ai](https://platform.opper.ai) and select **Logs**. The Logs list with Status, Provider, API key, Tag and Range filters above rows showing status, duration, time, model, region, key, input and output tokens, and cost ## The log list Each row is one provider call. When Opper retries a call or fails over to another provider, every attempt gets its own row, so one request can show up as several (see [Routing](#routing)). | Column | Shows | | - | - | | **Status** | The call's HTTP status, with a dot for how it ended. See [Status](#status). | | **Duration** | From Opper sending the call to the provider until the whole response was read. | | **Time** | When the call was made, in your local time. | | **Model** | The model that ran, beside the mark of the provider that served it. | | **Region** | Where the serving endpoint runs, such as **EU** or **GLOBAL**. Empty for a call through your own provider key. | | **Key** | The API key the call came in on. A key that has since been deleted reads **Deleted key**. | | **Input tokens**, **Output tokens** | Tokens in and out. Hover either one for the full split, including reasoning and cache tokens. | | **Cost** | What the call cost. Hover it for the breakdown. | Scroll down to load older calls. The list does not update by itself: select the refresh button at the right of the filter bar to load new calls. With a preset range such as **Last hour**, refresh also moves the range up to now. Press `j` and `k` or the arrow keys to move through the list, `Enter` to open a log, and `Esc` to close it. ### Filter the list | Filter | Narrows the list to | | - | - | | **Status** | **Success**, **Error** or **Blocked** calls. See [Status](#status). | | **Provider** | One provider. The menu lists the providers this project's calls went to in the last 90 days. | | **API key** | Calls made with one of the project's keys. | | **Tag** | Calls that carry one `key=value` tag. The menu groups tags by key. You can also filter from a log's details: see [Filter by tag](#filter-by-tag). | | **Range** | **Last hour**, **Last 24 hours**, **Last 7 days**, **Last 30 days**, or days you pick. Without a range, the list goes back through every call the project has made. | Filters search the project's whole history, or the whole range, not only the rows already loaded. A combination that matches very few calls can run out of time before it fills a page; the list then says **These filters took too long**. Pick a shorter range, or change or clear a filter. ### Filter by tag The **Tag** menu groups tags by key, with each value listed under its key. The Tag filter open below its pill, listing Any, then tags grouped by key: app with support-bot and triage, then customer with its values The menu lists only tags from calls that a [retention rule](#inputs-outputs-and-retention) stored. On the Gateway plan, or with retention off, it can be empty even though your calls carry tags. The menu also covers the whole organization, so a tag that only another project sends finds no logs here. You can also filter from a log's details, which show the call's [tags](/build/gateway/usage-attribution) at the top. Select a tag there to show only the logs that carry it, whether or not the menu lists it. The tag stays highlighted with an **×**, and the details stay open on the same call. Select the tag again, or clear the **Tag** filter, to show every log. A tag Logs cannot filter on shows as plain text in the details and cannot be selected: one whose value is longer than 256 bytes or ends in a space. ## Log details Select a row to open its details beside the list, or as a drawer on a narrow screen. The page URL carries the log's ID (`?log=gen_…`), so you can share a link that opens the same call. The details of one log: the model and log ID, the Duration, First token, Tokens and Cost figures, the latency bar, and the call's input as JSON with the output collapsed From the top, the details show: * **The model**, with a status dot, when the call ran, and the log ID. Use the copy button to copy the ID. * **Tags** you sent with the call, as `key=value`. Select one to filter the list by it; see [Filter by tag](#filter-by-tag). * **Duration**, **First token**, **Tokens** and **Cost**. * For a streamed call, a bar that splits the duration at the first token: **to first token**, then **then streamed** for the rest of the response. * **Input and output**, when they were stored. See [Inputs, outputs and retention](#inputs-outputs-and-retention). * **Tokens**: the total, and a bar that splits it into **Input** (cached reads, uncached input and cache writes) and **Generated** (output, and reasoning for reasoning models). * **Routing**: which endpoint served the call, and why. See [Routing](#routing). * **Technical details**, below. The lower half of a log's details: the Tokens breakdown into uncached input, output and reasoning, Routing with pool, selected endpoint, method and selection rank, and Technical details with status, retention, model ID, API key and streaming | Technical detail | Shows | | - | - | | **Status** | How the call ended, with its HTTP status. | | **Retention** | Until when the call's input and output are kept, and under which rule. See [What each log shows](#what-each-log-shows). | | **Model ID** | The exact model ID the provider was called with, such as `bedrock/eu.anthropic.claude-opus-5-5`. | | **API key** | The key the call came in on. | | **Session** | The session ID, for a call made through a [URL session prefix](/build/gateway/usage-attribution#tag-with-the-url). | | **Capability** | The kind of call, shown when it is not a text completion. | | **Streaming** | Whether the response was streamed. | | **Refusal** | The refusal code, for a call Opper blocked. | ### Status The list shows each call's HTTP status. The details spell out what it means, and the **Status** filter groups calls into three outcomes. | Details say | When | Status filter | | - | - | - | | **Completed** | The call succeeded (below 400). | Success | | **Rate limited by provider** | The provider answered 429. | Error | | **Client closed the connection** | Your client disconnected before the response finished (499). | Error | | **No response from provider** | The provider never answered (599). | Error | | **Provider error** | The provider failed with another 5xx status. | Error | | **Rejected by provider** | The provider refused the request with another 4xx status. | Error | | **Blocked by your rules** | Opper refused the call under your [Rules](/control-plane/rules/overview) before any provider saw it. | Blocked | | **Agreement required** | The route needs an [agreement](/control-plane/rules/model-access#routes-that-require-an-agreement) your organization does not have. | Blocked | A blocked call never reached a provider, so it has no output. Its **Refusal** row shows the refusal code. ### Routing When a call names a bare model, such as `claude-opus-5-5` rather than `aws/claude-opus-5-5`, a [model pool](/control-plane/rules/routing) serves it, and **Routing** explains how the pool placed the call: | Row | Shows | | - | - | | **Pool** | The model name the pool serves, and how many providers it held for this call. | | **Selected** | The endpoint that served the call, with its provider and region. | | **Method** | What put the first provider first. | | **Selection rank** | Where this attempt sat in the order: **1st choice**, or **2nd choice** and so on after the providers ahead of it failed. | | **Cut off** | **No first token before the deadline** when Opper gave up waiting for this attempt and moved on. | **Method** is one of: | Method | Meaning | | - | - | | **Your pool order** | Your [routing rule](/control-plane/rules/routing) ranked the pool, for example **Your pool order: Cheapest first**. | | **Session affinity** | The provider that served this session before went first, ahead of any pool order. | | **Your BYOK key** | A provider you hold your [own key](/capabilities/custom-models) for went first. | | **Round robin** | No routing rule applied, your rule is set to round robin, or it had no numbers yet to rank on. | | **Only member** | The pool had one provider to choose from. | A call that fails over leaves a row for each attempt: the failed attempts first, then the one that answered, with a **Selection rank** of 2nd choice or later. A provider retried after a 429 or 5xx writes a second row at the same rank. A call that names a specific provider's model, and a response served from cache, show only **Selected**. So do calls made before Opper started recording pool placement, so a missing **Pool** row does not mean the call was pinned. ## Inputs, outputs and retention Logs holds two kinds of data, kept for different lengths of time: * **The usage record**: everything in the list and the details except the input and output. Opper writes it for every call on every plan, including when retention is off, and keeps it for 5 years. It contains no prompts or responses. * **The input and output**: stored only when a [retention rule](/control-plane/rules/retention) covers the call's project, kept for that rule's period of 1 to 30 days, then deleted. | | Gateway | Control Plane and Enterprise | | - | - | - | | Every call in Logs, with model, provider, status, tokens, cost, latency, routing and tags | Yes | Yes | | Each call's input and output | Not stored | Stored for 1 to 30 days while retention is on | ### Keep inputs and outputs with Control Plane Retention rules are a Control Plane feature. On the Gateway plan, every call still appears in Logs with its full usage record, but its input and output are never stored. In their place, each log's **Input and output** section explains what Control Plane keeps, with an **Upgrade to Control Plane** button. The Input and output section on the Gateway plan: Keep what each call sent and received, with a Control Plane badge, the platform fee and an Upgrade to Control Plane button To see what your application actually sent and received, upgrade to [Control Plane](https://opper.ai/pricing), then turn on [Opper retention](/control-plane/rules/retention). Each new call's input and output then appear in Logs, and in [Traces](/control-plane/trace), for up to 30 days. On Control Plane with retention off for the project, the same section shows a **Turn on retention** button that opens **Opper retention** in Rules. ### Retention counts from when the call was made Whether a call's input and output were stored is decided when the call runs, by the retention rule in effect then, and recorded on its log. Changing the rule later does not change what a call already has: * **Turning retention on** stores calls made after you save. Earlier calls stay **Not stored**. * **Turning retention off, shortening it, or moving to Gateway** deletes nothing early. Stored inputs and outputs stay until the expiry they got when the call was made, and the **Retention** row adds **off now for new calls**. ### What each log shows | You see | Why | | - | - | | The input and output, and **Retention: Until** a date | Stored under the rule named beside it, such as **30-day rule**, until that date (UTC). | | **Keep what each call sent and received**, with **Upgrade to Control Plane** | The organization is on Gateway, which does not store inputs and outputs. | | **Turn on retention** as a button | The organization is on Control Plane, but no retention rule stores this project's calls. | | **Not stored. Retention was off when this call was made.** | Retention is on now, but was not for this call. | | **A zero data retention rule covers this project** | A [zero data retention](/control-plane/rules/retention#existing-zero-data-retention-rules) rule applies, so nothing is stored by design. | | **Stored under a 7-day retention rule and deleted on** a date | The input and output were stored, and their retention period has ended. | ### How inputs and outputs are shown * Messages render as a chat. Use the `{}` toggle to see the raw JSON; a call with no chat shape always shows JSON. * A failed call shows its error text above the input. * Images, audio and files sent inline are replaced with a placeholder; the media itself is not shown. * An input or output over 1 MB is cut to its first 1 MB and marked **Shortened**. * A blocked call has no output, since it never reached a provider. ## Logs and traces Logs and [Traces](/control-plane/trace) answer different questions: | | Logs | Traces | | - | - | - | | A row is | One provider call | One request, as a tree of spans | | Available on | Every plan | Control Plane, with retention on | | Without retention | Every call, with its usage record | Nothing is recorded | | Shows | Provider, routing, status, tokens and cost for each attempt | Nested LLM calls, tool calls, custom spans and rule activity | | Input and output | While a retention rule keeps them | While a retention rule keeps them | Use Logs to find a call by provider, key or outcome, and Traces to follow everything a request did. Both read the same stored input and output, under the same retention rule. ## Where to go next Turn on retention so new calls keep their input and output. Follow a request through its model calls, tool calls and rules. Tag calls so they are easy to find in Logs and to attribute in Usage. Choose which provider a model pool tries first. # Management API Source: https://docs.opper.ai/control-plane/management-api Provision projects, runtime API keys, members, dynamic routes and rules programmatically, from CI or infrastructure-as-code. The Management API is the provisioning surface for your organization. Creating a project, minting a runtime key for it, inviting the people who work in it, deploying a dynamic route, writing a compliance rule, and retiring any of them are the things you would otherwise do by hand in the platform UI. Here they are HTTP endpoints, so onboarding a service, rolling out a routing change or tearing down a stale environment can live in CI or Terraform instead of a runbook. It is a control-plane API: it manages the containers your model calls run in, not the calls themselves. Sending a prompt still goes through the [v3 API](/v3-api-reference/compatibility/chat-completions) with a runtime key. The Management API and management keys are available on every plan. A plan limits the features, not the API: writing a rule of a kind your plan does not include answers `402` with the code `plan_required` (see [Manage rules from code](#manage-rules-from-code)). Organizations without a card on file are capped at 60 requests per minute and 3 active management keys; adding a card lifts both. ## Authentication The Management API uses its own credential — a **management key**, prefixed `op-mak-`. Mint one in the platform UI under **Settings → API keys**, then send it as a bearer token: ```bash theme={null} curl https://api.opper.ai/management/v1/projects \ -H "Authorization: Bearer $OPPER_MANAGEMENT_API_KEY" ``` A runtime `op-…` key is not accepted here — it will be rejected with `403`. The two credentials are deliberately different things: | | Runtime key (`op-…`) | Management key (`op-mak-…`) | | - | - | - | | Used for | Model calls on the v3 API | Provisioning on the Management API | | Bound to | A single project | The whole organization | | Created by | The Management API, or the UI | The platform UI | | Carries | No scopes | An explicit scope list | A management key can create and delete projects across your entire organization. Store it as a secret in your CI provider — never in application config alongside runtime keys, and never in a client. ## Scopes Every management key carries an explicit list of scopes, chosen when you mint it. A request whose key lacks the scope an endpoint requires is rejected with `403` and a `required_scope` field naming what was missing. | Scope | Grants | | - | - | | `projects:read` | List and fetch projects | | `projects:write` | Create and update projects | | `projects:delete` | Delete projects, and with them their runtime keys | | `apikeys:read` | List a project's runtime keys (prefixes only) | | `apikeys:write` | Mint and delete runtime keys | | `dynamic_routes:read` | List and fetch dynamic routes and their versions | | `dynamic_routes:write` | Create, edit, simulate, deploy, roll back and delete routes | | `controls:read` | List rules | | `controls:write` | Create, update and delete rules | | `members:read` | List members and invitations | | `members:write` | Invite people, revoke invitations and remove members | Grant the narrowest set that does the job. A CI pipeline that only rotates keys needs `apikeys:write` and nothing else. ## Response shape Every response is an envelope with `meta` and `data`. Single resources leave `meta` empty; collections use it for `total_count`. ```json theme={null} { "meta": { "total_count": 2 }, "data": [ { "uuid": "…", "name": "checkout-service", "description": "", "created_at": "2026-08-01T09:12:44Z" } ] } ``` ## Provision a project and a runtime key The common pipeline is two calls: create the project, then mint a key inside it. ```bash theme={null} curl -X POST https://api.opper.ai/management/v1/projects \ -H "Authorization: Bearer $OPPER_MANAGEMENT_API_KEY" \ -H "Content-Type: application/json" \ -d '{"name": "checkout-service"}' ``` Project names are unique within an organization — reusing one returns `409`. The response carries the project `uuid` you need for the next call. ```bash theme={null} curl -X POST https://api.opper.ai/management/v1/projects/$PROJECT_UUID/api-keys \ -H "Authorization: Bearer $OPPER_MANAGEMENT_API_KEY" \ -H "Content-Type: application/json" \ -d '{"name": "ci-deploy"}' ``` The full `op-…` key is returned **once**, in this response, and is never retrievable again. Write it to your secret store in the same step that creates it. Listing keys later returns only their prefixes. Because the secret is shown once, a blind retry is expensive: it mints a second key while the first one's secret is already lost. Pass an `idempotency_key` and a retry answers `200` with the existing key's metadata and `meta.idempotency_replayed: true`, minting nothing. The secret is not returned again, so if you never captured it, revoke that key and create a replacement under a new `idempotency_key`. The minted key is an ordinary runtime key, scoped to that project. Hand it to the service as `OPPER_API_KEY` and call the v3 API with it. ## Lifecycle notes * **Updates are partial.** `PATCH` a project with `name`, `description`, or both; omit a field to leave it unchanged, and send `"description": null` to clear it. A body with neither field is rejected rather than treated as a no-op. * **Deleting a project deletes its keys.** The project is soft-deleted, but every runtime key bound to it is permanently destroyed and stops working immediately. * **Retention is not set here.** It is a control-plane rule — see [Opper retention](/control-plane/rules/retention). * **Creates take an `idempotency_key`.** Minting a runtime key and inviting a member both accept one, so a pipeline that never saw a response can retry without creating a second object. A replay answers `200` with the original and `meta.idempotency_replayed: true`; the same key with different arguments is `409`, and so is a replay of something deleted since. Keys are 1 to 128 letters, digits, dots, underscores, colons or hyphens, scoped to the management key that used them. ## Manage members and invitations Projects and runtime keys cover the services that call models; members cover the people. Inviting a colleague from the same pipeline that creates their project gives them a real membership, so their usage shows up under their own name in the platform instead of behind a shared key. ```bash theme={null} curl -X POST https://api.opper.ai/management/v1/invites \ -H "Authorization: Bearer $OPPER_MANAGEMENT_API_KEY" \ -H "Content-Type: application/json" \ -d '{"email": "ada@example.com", "idempotency_key": "onboard-ada"}' ``` Opper emails the invitation, the same one the **Members** settings page sends. The response also carries the acceptance `token` and `invite_url`, so a pipeline can post the link to its own channel instead. `email_sent` says whether the mail was queued; when it is `false` the invitation still exists and delivering `invite_url` is up to you. `token` and `invite_url` let whoever holds them join your organization. Handle them like a secret and do not write them to build logs. ```bash theme={null} curl "https://api.opper.ai/management/v1/invites?pending=true" \ -H "Authorization: Bearer $OPPER_MANAGEMENT_API_KEY" ``` `pending=true` lists the invitations not yet accepted and `pending=false` the accepted ones; leave it out for both. An invitation is valid for four weeks. `GET /management/v1/members` lists everyone who has joined, with their effective role and an `is_owner` flag. ### Invite behavior * **Inviting twice resends.** A second invite for an address that already has a live invitation does not create another one. It emails the invitee again, keeps the same token (so a link already in their inbox still works), moves the expiry four weeks out and answers `200` instead of `201`. * **Retries are safe with an `idempotency_key`.** A repeat with the same key answers `200` with the original invitation and `meta.idempotency_replayed: true`, sends no email and leaves the expiry alone. Reusing a key with a different address or role is `409`. The key is 1 to 128 letters, digits, dots, underscores, colons or hyphens. Leave it out when you actually want to resend. * **Existing members are refused.** Inviting an address that already belongs to a member answers `409`. * **Choosing a role depends on the plan.** On Control Plane and Enterprise, pass `role` with a built-in role (`admin`, `developer`, `viewer`). Custom roles are Enterprise only: on Enterprise, `role` can also be one of your custom roles. A role the organization does not have is `422`. On other plans leave `role` out: sending one is refused with `403` rather than ignored. A new invitation with no `role` gives the invitee the plan's default member role. A management key can assign any role up to `admin`. Ownership is never granted by invitation. See [Roles and permissions](/control-plane/roles-and-permissions) for what each role can do and which plans can assign roles. ### Remove a member ```bash theme={null} curl -X DELETE https://api.opper.ai/management/v1/members/$USER_UUID \ -H "Authorization: Bearer $OPPER_MANAGEMENT_API_KEY" ``` `$USER_UUID` is the member's `uuid` from `GET /management/v1/members`. Removal takes effect immediately and revokes what the membership issued in this organization: the member's runtime API keys, the OAuth authorizations behind them and their agent connections. It is not an account deletion; the person keeps their Opper account and their access to any other organization. The organization's owner cannot be removed (`409`). Transfer ownership in the platform first. To withdraw an invitation that has not been accepted, delete it with `DELETE /management/v1/invites/{id}` and its link stops working. Deleting an accepted invitation only removes the record of it; remove the member to take away their access. Member management needs a management key. It is deliberately not exposed to agents connected to Opper through OAuth, so an assistant acting on a person's behalf cannot change who belongs to the organization. ## Deploy a dynamic route from code A [dynamic route](/capabilities/routes/overview) is a graph the gateway walks per request to pick the model. The editor and this API save the same graph, so a route built in the UI can be exported, versioned in git and redeployed from a pipeline. The smallest useful route is a Pool node with two models: the first that answers wins, and a model a compliance rule blocks is skipped. ```bash theme={null} curl -X POST https://api.opper.ai/management/v1/dynamic-routes \ -H "Authorization: Bearer $OPPER_MANAGEMENT_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "name": "support-router", "deploy": true, "note": "initial", "graph": { "elements": [ {"id": "START", "type": "start", "outputs": {"next": {"elementId": "pool"}}}, {"id": "pool", "type": "pool", "properties": {"order": "cost", "models": [ {"model": "azure/gpt-5.6-luna"}, {"model": "melious/deepseek-v4.1-flash"} ]}, "outputs": {"success": {"elementId": "END"}, "fallback": {"elementId": "END"}}}, {"id": "END", "type": "end", "outputs": {}} ] } }' ``` `deploy: true` validates the draft and publishes it as version 1 in one call. Leave it out to save a draft and deploy later with `POST /management/v1/dynamic-routes/support-router/deploy`. ```bash theme={null} curl -X POST https://api.opper.ai/management/v1/dynamic-routes/support-router/simulate \ -H "Authorization: Bearer $OPPER_MANAGEMENT_API_KEY" \ -H "Content-Type: application/json" \ -d '{"input": {"messages": [{"role": "user", "content": "My invoice is wrong."}]}}' ``` No provider is called. The response lists the nodes visited and the model that would answer. The route is a model id: pass `dynamic/support-router` as `model` on any endpoint, with a runtime key. The response's `meta.routing` says which version ran and which model served. Every deploy is a numbered version. `POST …/versions/{n}/rollback` re-deploys an older graph as a new version, so the history stays linear. ## Manage rules from code [Rules](/control-plane/rules/overview) are rows the same API can read and write: retention, spend limits, model access, checks and routing defaults. A rule created here is enforced from the next request, exactly like one saved on the Rules page. Each rule has a `kind` (`guard`, `observe`, `route`, `comply`), a `scope` (the organization or a set of projects; a budget rule can also cap members by role or by name) and a `config` whose shape follows the kind; the reference documents each. ```bash theme={null} curl -X POST https://api.opper.ai/management/v1/controls/rules \ -H "Authorization: Bearer $OPPER_MANAGEMENT_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "id": "'$(uuidgen | tr A-Z a-z)'", "kind": "comply", "enabled": true, "scope": {"type": "org"}, "config": {"type": "allowlist", "allowlist": {"provider": {"locations": {"inference": {"include": ["EU"]}, "storage": {"include": ["EU"]}}}}} }' ``` `id`, `kind` and `scope` are fixed once created; change anything else with `PATCH`. A zero-day retention rule over a scope that still holds stored files is refused with `409` until you resend with `confirm_file_deletion: true`. Rules are a Control Plane feature. On a plan without it, creating or updating a rule answers `402`, and the message names the rule kind and your current plan: ```json theme={null} {"error": {"type": "payment_required", "code": "plan_required", "message": "saving comply rules requires the control_plane plan; this organization is on \"gateway\". Upgrade to control_plane to enable this feature."}} ``` Listing and deleting rules is never plan gated. Downgrading to the Gateway plan deletes every rule after you confirm it. ## Revoke a leaked management key If a management key is exposed, disable it with the key itself: ```bash theme={null} curl -X POST https://api.opper.ai/management/v1/management-keys/_self/disable \ -H "Authorization: Bearer $LEAKED_KEY" ``` Possession of the token is the authorization: no scope is required and the call is never rate limited, so a leaked key can always be shut off. This disables the key the request authenticated with and every token issued under it. ## Endpoints List, create, fetch, update, and delete projects. List, mint, and delete a project's runtime keys. List and remove members, and send, list and revoke invitations. Create, simulate, deploy, version and roll back routes. List, create, update and delete rules. # Observe & Control Source: https://docs.opper.ai/control-plane/overview Inspect model calls, enforce organization rules, and route traffic without changing application code. Once an AI feature reaches production, you need to understand what it is doing and change its operating boundaries safely. **Observe & Control** brings those tools together in the Opper platform. ## Observe what happened See every provider call with its model, provider, status, tokens, cost, and latency, on every plan. Inspect the model call, nested tool calls, cost, latency, and rule activity behind a request. Manage files used as model inputs and generation outputs. ## Control what is allowed [**Rules**](/control-plane/rules/overview) is the organization-level workspace for the policies Opper enforces on model calls. Turn Opper tracing on or off and choose how long to keep traces. Set a monthly organization budget and optional project caps. Restrict the providers, regions, countries, and models your apps can use. Score quality or flag, redact, and block content. Choose which provider a model pool tries first. Default-model and response-caching rules are set through the [Management API](/control-plane/management-api#manage-rules-from-code): Set a default model for an organization or project. ## How rules apply Organization rules set the default for every project. Project overrides can tighten that default, but cannot loosen it. * Model-access rules intersect, so a model must be available at both levels. * The shortest applicable retention period wins. * Project spend caps must fit inside the organization budget. * Checks can run across the organization or only on selected projects. When trace storage is enabled, rule activity appears on the relevant [trace](/control-plane/trace). In the playground, turn on **Project controls** to test a call with its rules applied. # Roles and permissions Source: https://docs.opper.ai/control-plane/roles-and-permissions What each role in an Opper organization can do, which plans can assign roles, and how to invite a member with a role or change it later. Every member of an Opper organization has one role. The role decides what the member can see and do in the Opper platform and with the API keys they own. Any member can see their own role, and the role of every other member, under **Settings**, then **Users**. ## Built-in roles Opper has four built-in roles. Each role includes everything the role before it can do. | Role | Summary | | - | - | | **Viewer** | Read-only. Can look at the organization's projects and usage. Cannot run models or change settings. | | **Developer** | Can do what a Viewer can, and also run models, configure dynamic routes, manage their own API keys and view rules. | | **Admin** | Can do what a Developer can, and also manage the organization: its projects, rules, members, keys and billing. | | **Owner** | Can do what an Admin can, and also transfer ownership and delete the organization. | ## What each role can do ### Workspace | Resource | Permission | Viewer | Developer | Admin | Owner | | - | - | - | - | - | - | | Projects | View projects | Yes | Yes | Yes | Yes | | Projects | Create, edit and delete projects | No | No | Yes | Yes | | Dynamic routes | Inspect dynamic routes | Yes | Yes | Yes | Yes | | Dynamic routes | Configure dynamic routes | No | Yes | Yes | Yes | | Models & route execution | Run models and routes | No | Yes | Yes | Yes | | Personal API keys | Manage own API keys | No | Yes | Yes | Yes | | Project API keys | Manage project API keys | No | No | Yes | Yes | **Run models and routes** covers calls made with the member's API keys and calls made in the playground. **Configure dynamic routes** covers everything that changes a [dynamic route](/capabilities/routes/overview): creating, editing, deploying, rolling back and deleting it. ### Observability & governance | Resource | Permission | Viewer | Developer | Admin | Owner | | - | - | - | - | - | - | | Usage | View usage | Yes | Yes | Yes | Yes | | Trace metadata | View trace metadata | Yes | Yes | Yes | Yes | | Trace inputs & outputs | View trace inputs and outputs | No | No | Yes | Yes | | Rules | View rules | No | Yes | Yes | Yes | | Rules | Configure rules | No | No | Yes | Yes | | Audit logs | View audit logs | No | No | Yes | Yes | A member with **View trace metadata** but without **View trace inputs and outputs** can open [Traces](/control-plane/trace) and [Logs](/control-plane/logs), but does not see prompts, responses or other captured content. ### Organization | Resource | Permission | Viewer | Developer | Admin | Owner | | - | - | - | - | - | - | | Members | Manage members and assign roles | No | No | Yes | Yes | | Roles & permissions | Create and edit roles | No | No | Yes | Yes | | SSO group mappings | Map SSO groups | No | No | Yes | Yes | | Single sign-on setup | Set up and enforce single sign-on | No | No | Yes | Yes | | Management credentials | Manage management credentials | No | No | Yes | Yes | | Billing & plans | View billing | No | No | Yes | Yes | | Billing & plans | Manage billing and plans | No | No | Yes | Yes | | Ownership | Transfer ownership | No | No | No | Yes | | Organization | Delete the organization | No | No | No | Yes | Management credentials are the management keys used with the [Management API](/control-plane/management-api). **View billing** decides whether the organization's balance, spend and budget are included when one of the member's API keys calls `GET /v3/me`. In the platform, every member can open **Settings**, then **Billing & Credits**, and see the balance, spend and recent activity. Only a role with **Manage billing and plans** can add credit, change auto-recharge or change the plan. A role gives access to a feature only when the organization's plan includes that feature. For example, setting up single sign-on requires the Enterprise plan. ## Who can view and change rules * **View rules:** Owner, Admin and Developer. Viewer cannot view rules. * **Change rules:** Owner and Admin. Developer and Viewer cannot change rules. In the tables above, changing rules is the **Configure rules** permission. On Enterprise, a [custom role](#custom-roles) can hold either permission. For example, an Enterprise organization can create a role that can run models but cannot view rules. A member whose role can view rules but cannot change them sees the [Rules](/control-plane/rules/overview) page marked **Read only**. Every field on the page is disabled. A member whose role cannot view rules: * Does not see the **Rules** link under **Observe & Control** in the sidebar. * Sees a panel titled **Rules are managed by your administrators** in place of the Rules page, and in place of the **Effective policy** section on the dashboard. The organization's rules apply to every member's calls, whether or not the member can view them. If you need more access to rules than your role gives you, ask an Owner or Admin of your organization to [change your role](#change-a-role). Rules can also be read and changed through the [Management API](/control-plane/management-api#manage-rules-from-code) by anyone who holds a management key with the matching scope. Only a member whose role holds **Manage management credentials** can create management keys. ## Which plans can choose roles | Plan | Roles you can assign | | - | - | | **Control Plane** | Admin, Developer or Viewer. You choose the role when you invite a member and can change it later. | | **Enterprise** | Admin, Developer or Viewer, plus your own custom roles. You choose the role when you invite a member and can change it later. | | **All other plans** | None. Every invited member joins as an Admin, and a member's role cannot be changed. | ### Custom roles Custom roles are available on the Enterprise plan only. A custom role can hold any subset of the permissions an Admin holds. It can never hold the two Owner-only permissions: **Transfer ownership** and **Delete the organization**. Use a custom role when a built-in role holds more than you want a member to have. For example, a custom role can run models without being able to configure dynamic routes. To create one, your role must hold **Create and edit roles**. Open **Settings**, then **Roles & permissions**, and select **Create role**. Enter a **Role name**, select the permissions the role should hold, and select **Save role**. You can only grant permissions that your own role holds. The role editor uses the group, resource and permission names from the tables above. It lists every permission in those tables except **Set up and enforce single sign-on** and the two Owner-only permissions. A custom role is assigned the same way as a built-in role: in the invite dialog, or in the role dropdown on the **Users** page. ## The Owner * The person who creates an organization is its Owner. * An organization has one Owner. * Owner cannot be chosen in an invitation or in the role dropdown. It moves only when the current Owner transfers ownership. * The Owner cannot leave the organization or be removed from it. Transfer ownership first. ### Transfer ownership Only the Owner can transfer ownership, and only to someone who is already a member of the organization. 1. Open **Settings**, then **Users**. 2. In the row of the member who should become the Owner, open the actions menu and select **Make owner**. 3. Select **Make owner** again to confirm. The transfer takes effect immediately. You stay a member of the organization, with a role other than Owner. Check the **Users** page to see which role you now have. ## Invite a member with a role To invite members, your role must hold **Manage members and assign roles** (Owner and Admin do). 1. Open **Settings**, then **Users**. 2. Select **Invite user**. 3. Enter one email address in each row. 4. On Control Plane and Enterprise, choose a role in the **Role** column of each row. You can only choose roles whose permissions your own role also holds. On all other plans the **Role** column is locked to **Admin**. 5. Check the role shown next to each email address. 6. Select **Send invitations**. Check the role next to every email address before you select **Send invitations**. Each row has its own role, and the person joins with the role their row shows. Until the person accepts, they appear on the **Users** page as a pending invite. The **Users** page does not show which role a pending invitation carries. If you sent an invitation with the wrong role, remove the invitation and send a new one, or change the role after the person has joined. Members can also be invited from code. See [Manage members and invitations](/control-plane/management-api#manage-members-and-invitations) in the Management API guide. ## Change a role Changing a member's role requires the Control Plane or Enterprise plan, and a role that holds **Manage members and assign roles** (Owner and Admin do). 1. Open **Settings**, then **Users**. 2. In the member's row, open the role dropdown. 3. Select the new role. The change is saved when you select the role, and applies immediately. If a member's row shows the role as a label with no dropdown, one of these applies: * The member is the Owner. Use [Transfer ownership](#transfer-ownership) instead. * The member's role is managed by your identity provider through single sign-on. The label shows **SSO** next to the role. * The member's current role holds a permission that your own role does not hold. * Your organization's plan does not include role choice, or your own role does not hold **Manage members and assign roles**. The dropdown only lists roles whose permissions your own role also holds. ## Permission identifier reference This section is a reference for Enterprise organizations that use custom roles. It lists the identifier of each permission in the tables above. | Identifier | Permission | | - | - | | `projects:read` | View projects | | `projects:write` | Create, edit and delete projects | | `dynamic_routes:read` | Inspect dynamic routes | | `dynamic_routes:write` | Configure dynamic routes | | `inference:execute` | Run models and routes | | `personal_keys:manage` | Manage own API keys | | `project_keys:manage` | Manage project API keys | | `usage:read` | View usage | | `traces:read` | View trace metadata | | `traces:content:read` | View trace inputs and outputs | | `controls:read` | View rules | | `controls:write` | Configure rules | | `audit:read` | View audit logs | | `members:write` | Manage members and assign roles | | `roles:write` | Create and edit roles | | `sso:groups:write` | Map SSO groups | | `sso:write` | Set up and enforce single sign-on | | `management_keys:write` | Manage management credentials | | `billing:read` | View billing | | `billing:write` | Manage billing and plans | | `ownership:transfer` | Transfer ownership (Owner only) | | `organization:delete` | Delete the organization (Owner only) | A custom role that holds **Manage billing and plans** can also view billing. Management API keys have their own list of [scopes](/control-plane/management-api#scopes). Some scopes share a name with a permission above, but a scope limits what a key can do and a permission limits what a member can do. # Default model Source: https://docs.opper.ai/control-plane/route Set the model a call uses when it doesn't name one, per organization or project, with a route rule in the Management API. A default-model rule picks the model for calls that don't pass `model`. Set it once for an organization or project, then change it without touching code. A typical use: your app runs on `openai/gpt-5-mini` everywhere, then you move a single high-stakes project to `anthropic/claude-opus-5` by editing one rule. ## Set a default model Default-model rules are managed through the [Management API](/control-plane/management-api) with a management key that has the `controls:write` scope. The Rules page in the platform does not edit them. Create a `route` rule whose config holds the model in `default_model`. Send `default_model` without a `type` field: ```bash theme={null} curl -X POST https://api.opper.ai/management/v1/controls/rules \ -H "Authorization: Bearer $OPPER_MANAGEMENT_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "id": "'$(uuidgen | tr A-Z a-z)'", "kind": "route", "name": "Default model", "enabled": true, "scope": {"type": "org"}, "config": {"default_model": "openai/gpt-5-mini"} }' ``` | Scope | `scope` value | Applies to | | - | - | - | | Organization | `{"type": "org"}` | Every call in your organization. | | Projects | `{"type": "projects", "project_uuids": ["…"]}` | Calls made with those projects' API keys. | The most specific rule wins: a project default overrides the organization default. Setting `"default_model": null` on a project rule records an explicit "no default" for that project. To change the model later, send the new `config` with `PATCH /management/v1/controls/rules/{uuid}`. List existing rules with `GET /management/v1/controls/rules`. See the [rules reference](/v3-api-reference/management/list-control-rules) for every field. Rules are a Control Plane feature. On the Gateway plan, creating the rule answers `402` with `plan_required`. ### The default must be allowed at its scope A default model has to pass [model access](/control-plane/rules/model-access) at the same scope. If it doesn't, the API refuses the rule with `400`: ``` default_model "anthropic/claude-opus-5" is not allowed by the comply allowlist at this scope — relax the allowlist or pick a different model ``` The reverse is checked too: saving a model-access change that would exclude an existing default model is refused with `400` until you change or remove that default. ## Callers can still pick a model Passing an explicit `model` always takes precedence over the default, as long as model access allows it. Use this for per-call trade-offs: cheaper models for background jobs, higher-quality ones for user-facing replies. If the requested model isn't allowed, the call fails with `403` and says which condition failed; it doesn't fall back to the default. See [what a blocked call returns](/control-plane/rules/model-access#what-a-blocked-call-returns). A call with no `model` and no default-model rule in its scope is refused; the Messages endpoint, for example, answers `400` with `model is required`. Opper does not pick a model for you. Keep model names out of application code. Set a default per scope and switch models by editing the rule. To choose between several models per request, use a [dynamic route](/capabilities/routes/overview) instead. # Checks Source: https://docs.opper.ai/control-plane/rules/checks Score quality or flag, redact, and block content in model calls. Checks inspect model inputs and outputs. Use them to measure response quality, detect policy violations, remove sensitive data, or stop unsafe content before it continues through your application. Saved checks with their engine, direction, action, and scope Each row shows what the check reads, the engine it uses, the action it takes, its scope, and its activity over the last 24 hours. ## Start from a template When you have no checks, Rules shows a gallery of common starting points. Select a template to add a pre-filled draft, then review it before saving. The check template gallery After you save your first check, open **New check** to find the same templates and custom options. The New check picker with templates and a description pane | Starting point | Engine | Default action | | - | - | - | | PII | LLM | Flag | | Toxicity | LLM | Flag | | Secrets | LLM | Block | | Email, phone, credit card, SSN, API key | Regex | Redact | | Answer quality | LLM | Score | | Custom LLM check | LLM | Flag | | Custom regex check | Regex | Redact | The description pane explains what each template detects and what its default action will do. The Answer quality template and its score action ## Choose an action The action determines when a check runs and whether it can change the call. | Action | Result | When it runs | | - | - | - | | **Score** | Grades a stored output and flags scores below the threshold. It never changes the response. | After the call | | **Flag** | Records a violation and allows the call to continue. | During the call | | **Redact** | Replaces matching content before the call continues. | During the call | | **Block** | Rejects the call with an error. | During the call | Score checks are stored and run differently from enforcement checks. Changing a saved check between **Score** and another action creates a new check, so its previous activity history is not carried over. ## Choose an engine ### LLM checks Use an LLM check when the rule depends on meaning or context—for example tone, intent, toxicity, or whether an answer addressed the question. An LLM check with an action, direction, criteria, and scope | Field | What to configure | | - | - | | **Applies to** | Input, output, or both. | | **Criteria** | Instructions for the classifier, up to 4,096 characters. Required for Flag, Redact, and Block. Score has separate criteria rules. | | **Scope** | The whole organization or selected projects. | Opper chooses and manages the classifier model. LLM checks use a failover chain and are billed at a flat rate for checks, regardless of which model handles the request. LLM checks fail open. If the classifier is unavailable or returns an invalid result, Opper skips the check and allows the model call to continue. The failure is recorded on the trace when trace storage is enabled. ### Regex checks Use a regex check for predictable formats such as email addresses, credit card numbers, SSNs, and API keys. Regex checks do not call a model and cannot use the Score action. A regex check with patterns and a replacement value Select a pattern chip to edit and test it. The regex pattern editor with validation and a match tester | Field | What to configure | | - | - | | **Describe what to detect** | Optionally describe the pattern in plain language, then select **Generate** for a first draft. | | **Name** | The label shown in the check summary. | | **Regex** | The expression to match. Syntax is validated as you type. | | **Flags** | `i` for case-insensitive, `m` for multiline, and `s` for dotall. Global matching is always enabled. | | **Test** | Sample content used to highlight and count matches before you save. | Test the pattern with examples that should match and examples that should not. ### Redaction behavior When the action is **Redact**, a regex check replaces each match with `***` by default. An LLM check uses `[REDACTED]`. Enter a replacement value if you want a different result. If an LLM check detects a violation but cannot identify the exact text to replace, Opper flags the call instead of changing its content. This can happen with broad findings such as toxicity. ## Score checks Choose **Score** to evaluate response quality after a call completes. A score check with score type, threshold, sampling, and criteria ### Score type | Type | Result | | - | - | | **Scale** | Returns a value from 0 to 1 and flags results below your threshold. Criteria are optional; without them, the judge performs a general quality evaluation. | | **Binary** | Returns 0 or 1 and flags a result of 0. Criteria are required. | ### Sampling Sampling controls how many outputs the judge evaluates and therefore how much the check costs. | Strategy | Behavior | | - | - | | **All** | Evaluate every output. | | **Rate** | Evaluate 1 in every *N* outputs. New checks default to 1 in 10. | | **Adaptive** | Evaluate every output up to a limit per 1-hour or 24-hour window, then gradually reduce the sampling rate as volume grows. | Adaptive sampling keeps evaluation costs more predictable during traffic spikes. If its counter is unavailable, Opper evaluates every output rather than silently skipping them. Score checks need stored outputs. They do not run when effective retention is unset or set to 0 days. See [Opper retention](/control-plane/rules/retention). The judge model is managed by Opper and billed at the same flat rate for checks. Disabling a score check stops queued evaluations before the next judge call begins. ## Set the scope Run a check across the organization or limit it to selected projects. | Scope | Applies to | | - | - | | **Organization** | Every project. | | **Projects** | Only the projects you select. | If you choose **Projects**, select at least one project before saving. ## When checks overlap Every matching check runs; there is no priority order. If several checks apply to the same call, the strictest result wins. One blocking result is enough to reject the call. Rules highlights checks with the same name and overlapping scope so you can confirm that the overlap is intentional. Opper evaluates up to **32 checks per request**, counting input and output together. Any additional checks are skipped. ## Disable a check or review activity Use a row's toggle to stop a saved check without deleting its configuration. Disabled checks stop immediately and are excluded from validation until you enable them again. The activity label covers the last 24 hours: | Label | Meaning | | - | - | | `draft` | Added but not yet saved | | `off` | Saved but disabled | | `N matches · 24h` | Number of matches for a Flag, Redact, or Block check | | `N evaluations · 24h` | Number of outputs evaluated by a Score check | Start a new enforcement check with **Flag**. Review its matches before switching to **Block** or **Redact**. For LLM checks, changing the action does not change the price—the same classifier runs either way. # Model access: provider and model allowlist Source: https://docs.opper.ai/control-plane/rules/model-access Allow or block providers, service routes, makers, countries, locations, and models for your organization, and require provider data practices such as zero data retention. Enforced on every call. Model access is an organization-wide allowlist for AI models. It decides which providers, service routes, model makers, countries, inference and storage locations, and models your calls can use, and which provider data practices they must meet, such as zero data retention. Opper enforces it on every call: a call to a model outside the allowlist is refused with `403` before anything reaches a provider. Start with an organization allowlist, then narrow it for projects with stricter requirements. Model access is a Control Plane feature. With no model-access rule, no models are excluded by model access. For the two most common setups, see [Enforce zero data retention](/control-plane/zero-data-retention) and [Keep AI inference in the EU](/control-plane/eu-data-residency). ## Choose a provider data policy In **Rules → Model access**, use **Provider data policy** to choose which provider practices are acceptable for your calls. Provider data policy set to Zero data retention, with No training and No logging checked and No provider moderation unavailable pending an agreement | Option | Effect on eligible models | | - | - | | **No data requirements** | Applies no filter for provider training, logging, or moderation practices and hides the requirements checklist. Other model-access filters still apply. | | **Zero data retention** | Selects **No training** and **No logging**, allowing only models that meet both requirements. **No provider moderation** is a separate requirement. | | **Custom** | Opens the checklist so you can choose individual requirements. Models must meet every selected requirement. | **No data requirements** leaves those practices unconstrained. It does not request providers that train on, log, or moderate your content. The displayed policy follows your selections. For example, selecting only **No training** shows **Custom**; selecting both **No training** and **No logging** shows **Zero data retention**, even if you also require **No provider moderation**. Clearing all three returns to **No data requirements** and hides the checklist. Opening an empty **Custom** editor and cancelling it by selecting **No data requirements** does not create a rule. ### What each checkbox requires Checking a box adds a requirement; clearing it removes that requirement. An unchecked box does not require the opposite practice. Requirements inherited from the organization still apply to projects. | Requirement | Models that remain eligible | | - | - | | **No training** | Models whose provider policy rules out using requests or responses for training. Providers that offer training on an opt-in basis are also excluded. | | **No logging** | Models whose providers do not store request or response content in logs. This does not cover provider caching, content retained for moderation, or Opper trace storage. | | **No provider moderation** | Models whose providers do not run content moderation classifiers on requests. This requires Opper approval under a signed agreement. | These requirements filter models using provider policies recorded in Opper's catalog. They do not change provider settings. If the catalog does not establish a provider's policy for a selected requirement, its models do not pass that requirement. A policy you have not selected does not exclude models. The **Zero data retention** preset means **No training** plus **No logging**. It does not guarantee that a provider never stores content: caching and content retained for moderation are outside those two requirements. Configure Opper's own tracing and storage separately under [Opper retention](/control-plane/rules/retention). Tools and services your application uses outside Opper keep their own retention, so review them separately. ### Routes that require an agreement Selecting **Zero data retention** needs no agreement. Some of the routes it allows still require one before your organization can call them: routes whose IDs begin with `azure-zdr` or `vertexai-zdr`, such as `azure-zdr/gpt-5.6-luna` and `vertexai-zdr/gemini-3.7-flash-eu`. Opper approves access to each of these providers separately, under a signed agreement. Until then, the model catalog marks their routes **Agreement required**, and a call to one is refused with `403` and the code `entitlement_required` before anything reaches the provider. The two providers handle content differently, so a route that requires an agreement does not by itself mean provider moderation is off. Opper's catalog records these practices for the current routes: | Routes | Training | Request logging | Provider moderation | | - | - | - | - | | `azure-zdr` | No training | No logging | Off | | `vertexai-zdr` | No training | No logging | On, with no moderation retention | To decide whether you need one of these routes, start from your requirements: * **OpenAI models on Azure.** Ordinary Azure routes may offer the same model in a location that meets your requirements, with no training or request logging and no additional agreement. They run provider moderation, which may retain flagged content. Decide whether that meets your retention requirements, or whether you need **No provider moderation**. For Azure ZDR routes, the agreement makes your organization responsible for content moderation. * **Gemini.** In Opper's current catalog, every Gemini route that needs no agreement is recorded with abuse-monitoring logging, so none of them meets **No logging**. For Gemini without provider request logging, you may need a `vertexai-zdr` route. * **Location.** Both providers have EU and global routes, but not every model has both. Choose a route that meets your processing-location requirement, and use **Inference location** in your allowlist to enforce it. Provider caching is recorded separately. The `vertexai-zdr` routes, for example, are recorded with implicit caching, which **No logging** does not cover. Route approval also does not change what Opper itself stores: configure tracing and storage under [Opper retention](/control-plane/rules/retention). Each route's details in the model catalog show its recorded training, logging, and moderation practices, and the [models page](https://opper.ai/models) lists each route's price. When a call is refused with `entitlement_required`, the error message links to this page. Where the catalog supports it, the message also names a route for the same model in the same location that needs no agreement, or says that every such route is recorded with request logging. Opper does not switch the call to another route for you. To request access, select **Contact us** in the route's details in the model catalog, or email [support@opper.ai](mailto:support@opper.ai) with your organization, the models or routes you need, your required processing location, and your retention requirements. We can help identify an appropriate route and arrange the agreement where needed. Provider data policies are part of Rules, which are a Control Plane feature; approval for a route is granted separately and does not change your rules. ### Arrange approval for No provider moderation Contact Opper **before selecting No provider moderation** to arrange a signed agreement accepting responsibility for content moderation. Until Opper approves your organization, the checkbox is unavailable and shows **Agreement required**. Use the small **Contact us** button beneath the requirement to open the support flow. Custom provider data requirements with No provider moderation disabled, an Agreement required notice, and the Contact us button After approval, you can select the requirement. It concerns provider moderation classifiers; it does not remove a model's built-in safety behavior or your [Opper checks](/control-plane/rules/checks). ## Build an organization allowlist If you do not have an organization allowlist yet, select **Set an org allowlist**. Filter the catalog by any combination of these fields: | Field | Use it to | | - | - | | **Providers** | Allow selected providers, such as Anthropic or OpenAI. | | **Blocked providers** | Block providers even if another filter would include them. | | **Service routes** | Allow specific ways a provider serves models. | | **Blocked routes** | Exclude specific service routes. | | **Inference location** | Limit where inference runs, using countries or broader areas such as the EU or EEA. | | **Storage location** | Limit where providers retain content. | | **Maker** | Choose the companies that trained the models, regardless of who serves them. | | **Maker country** | Limit where the model maker is legally based. | | **Provider country** | Limit where the company operating the service route is ultimately owned. | | **Models** | Allow only specific catalog models. | A model must match every field you set. Leave a field empty when you do not want to filter on it. Blocked providers and routes take precedence. Provider data requirements apply alongside these filters, so a model must satisfy both. The match counter updates as you edit. Select it to review the exact models that will remain available before you save. Select **Save changes** to apply your draft, or **Discard** to return to the saved rules. Changing a selector or checkbox alone does not change live model access. A dialog listing the catalog models that match the allowlist If the same provider appears in both **Providers** and **Blocked providers**, the rule cannot be saved. The pickers hide already-selected values to help prevent that conflict. ## When Opper adds a provider or route The allowlist is checked against the live catalog on every call, so it also decides what happens to providers and routes that Opper adds later. There is no separate setting for new providers. The fields you set decide: | Your rule sets | A newly added provider or route is | | - | - | | **Providers** | Blocked if it comes from a provider company not on the list. A new route from a company already on the list is allowed. | | **Models** | Blocked unless its route ID without the provider prefix, or one of its aliases, is on the list. For example, `claude-sonnet-4-6` admits a new `azure/claude-sonnet-4-6`, but not `aws/claude-sonnet-4-6-eu`. | | Only **Blocked providers** or **Blocked routes** | Allowed, unless you block it too. | | **Inference location**, **Storage location**, **Maker**, a country field, or a provider data requirement | Allowed as soon as the catalog records facts for it that meet every field you set. | Facts the catalog does not have count as a failure. A route with no recorded inference location counts as `GLOBAL` and fails an EU or country filter, and a data practice that has not been researched fails that requirement. To keep new providers out until you have reviewed them, set **Providers**. ## What model access covers Model access applies to every model call: text generation and embeddings, image generation, speech, transcription, video, OCR, realtime voice, and rerank. A disallowed model is refused before anything reaches a provider. Opper does not silently switch to a different model. How a call is checked depends on how it names the model: | The call names | What happens when the allowlist excludes it | | - | - | | A pinned model, such as `anthropic/claude-sonnet-5` | Refused with `403`. | | A bare model name, such as `claude-sonnet-5`, served by a pool of providers | Excluded providers are skipped and the call runs on an allowed one. If none are left, the call is refused with `403`. | | A [dynamic route](/capabilities/routes/overview), `dynamic/` | A Pool node skips excluded models; if it has none left, it takes its fallback edge, or fails with `403` when it has none. A Model node whose model is excluded ends the route with `403` and the code `route_compliance_blocked`. | | A `models` fallback list | Excluded entries are removed from the list before the first attempt, without an error. If every entry is excluded, the call is refused with `403`. | A [default-model rule](/control-plane/route) must point to a model that the same scope can use. The API refuses a default model that the allowlist excludes, and refuses an allowlist change that would exclude an existing default model, with `400`, until you change or remove that default. ## What a blocked call returns Every refusal is HTTP `403`, sent before the request reaches a provider. That includes streaming requests: the error comes back before the stream opens. The message names the model and the condition that failed. It does not name the rule; the rule's name and scope are recorded on the call's [trace](/control-plane/trace). The body follows the shape of the endpoint you called: | Endpoint | Body | | - | - | | `/v3/compat/chat/completions` | `{"error": {"code": "permission_error", "message": "…"}}` | | `/v3/compat/v1/messages` | `{"type": "error", "error": {"type": "permission_error", "message": "…"}}` | | `/v3/compat/responses` | `{"error": {"code": "model_not_allowed", "message": "…"}}` | | `/v3/compat/openresponses` | `{"error": {"type": "permission_error", "code": "model_not_allowed", "message": "…"}}` | | Gemini-compatible `generateContent` | `{"error": {"code": 403, "status": "permission_error", "message": "…"}}` | | `/v3/compat/embeddings` | `{"error": {"type": "permission_error", "message": "…"}}` | | Image, audio, video, and OCR endpoints | `{"error": "…"}` | A [dynamic route](/capabilities/routes/overview) whose Model node is excluded fails with the code `route_compliance_blocked`, and its message names the node rather than the condition. For example, a chat completion that pins a US-hosted model under an EU-only rule returns: ```json theme={null} { "error": { "code": "permission_error", "message": "model \"anthropic/claude-sonnet-5\" is blocked by the active model allowlist: inference location US is outside EU" } } ``` The part after the colon names the condition that failed: | Condition | Message ends with | | - | - | | Inference location outside the allowed area | `inference location US is outside EU` | | No recorded inference location | `inference location is not recorded for this route, which counts as GLOBAL (allowed: EU)` | | Provider not on your **Providers** list | `provider "mistral" is not on the allowlist (allowed: anthropic, openai)` | | Provider or route blocked | `provider "xai" is excluded`, or `service route "…" is excluded` | | Model not on your **Models** list | `model "gpt-5.5" is not pinned (pinned: claude-sonnet-5)` | When a `models` fallback list names pinned routes and every one is excluded, their distinct conditions are joined with `; `. If the list names bare model names instead, the first entry's `has no allowed members` message comes back. Two other refusals use their own messages: * **A bare model name with no allowed providers left:** `model "claude-sonnet-5" has no allowed members under the current model allowlist`. * **A route that requires an agreement:** code `entitlement_required`. See [Routes that require an agreement](#routes-that-require-an-agreement). If Opper cannot read your rules, it refuses every call rather than letting one through unchecked, with a message that ends `the model rules could not be read, so every model is refused until they can be; retry the request`. To check before you call, list models with your API key and `GET https://api.opper.ai/v3/models?include=policy`. Each model gets a `policy` object with `allowed`, `blocked_by` (`org`, `project`, or `entitlement`), and `reason`: the condition that failed, as in the part of a `403` message after the colon. `reason` is left out when `blocked_by` is `entitlement`. ## Narrow access for a project Select **Narrow model access for a project** to add an override. A project override intersects with the organization rule: it can remove access, but it cannot restore a provider, location, service route, or model that the organization rule excluded. Organization data requirements are inherited too. When the organization sets provider data requirements, the project's policy selector is locked and the inherited checkboxes stay checked and unavailable for editing. Add requirements that the organization has not set to narrow access further. For example, if the organization requires **No training**, a project can add **No logging**, but cannot allow providers that fail the organization's training requirement. The override summary shows the effective result, not just the fields you added: ``` PROJECT NARROWS ORG RULE BY EFFECTIVE MODELS sqlassistant providers → anthropic, openai 45 models of 77 ``` While you edit an override, values unavailable at the organization level are dimmed. A project provider picker with values excluded by the organization rule dimmed To make one of those values available, broaden the organization rule first. You can edit both levels in the same draft and save them together. An override can leave a project with **0 available models**. Opper warns you but allows the save because denying all model calls may be intentional. ## If older organization rules exist The runtime uses only the newest enabled organization allowlist. Older organization allowlists have no effect and do not intersect with it. Rules shows the active allowlist and tells you when older rules are still present so you can remove them. Project overrides still intersect with the active organization allowlist. ## If a legacy Zero Data Retention rule is active An [older Zero Data Retention rule](/control-plane/rules/retention#existing-zero-data-retention-rules) can still lock the provider policy and its requirements. To replace an organization-level legacy rule: 1. Open **Rules** in the Opper platform and find the **Opper retention** section. 2. In the legacy-rule notice, select **Replace with new settings**. This prepares a draft with tracing off at 0-day retention and **No training** and **No logging** selected under **Model access**. Other allowlist filters are kept. 3. Review the requirements and matched models under **Model access**, then select **Save changes** to apply the replacement. Select **Discard** to keep the existing rule instead. After replacement, tracing and provider data requirements can be edited separately. The replacement does not add **No provider moderation**; that requirement still needs the agreement and approval described above. The legacy **Moderation retention restricted** label concerns content retained for moderation; it does not mean that no provider moderation classifier runs. Set shared requirements, such as **No training** or EU inference locations, at the organization level. Use project overrides for workloads that need an even smaller model set. # Rules Source: https://docs.opper.ai/control-plane/rules/overview Set organization-wide limits for data, spend, model access, checks, and routing. Rules gives you one place to decide how Opper handles every model call in your organization. Set a retention policy, cap spend, limit which models are available, and inspect inputs and outputs—all without changing application code. The Rules page showing spend limits and Opper retention Open **Rules** from **Observe & Control** in the Opper platform. The page is also available at `platform.opper.ai//rules` on plans that include Rules. Whether you can view and change rules depends on your role in the organization. See [Who can view and change rules](/control-plane/roles-and-permissions#who-can-view-and-change-rules). Rules can also be managed from code: the [Management API](/control-plane/management-api#manage-rules-from-code) lists, creates, updates and deletes the same rules, with a management key carrying the `controls:write` scope. ## What you can configure Turn Opper tracing on or off and choose how long to keep traces. Set a monthly organization budget and optional project caps. Restrict the providers, regions, countries, and models your apps can use. Score quality or flag, redact, and block content. Choose which provider a model pool tries first. Rules apply to LLM calls made through the **Opper v3 API**, including calls from the [compatibility gateway](/build/gateway/drop-in-sdks) and the playground. Spend limits also cover image, speech, and video generation. Model access covers every model type: embeddings, image generation, speech, transcription, video, OCR, realtime voice, and rerank as well. Checks do not currently apply to those modalities. The deprecated v2 API is not covered by Rules. ## Organization rules and project overrides Start with an organization rule to set the default for every project. Add a project override when a workload needs a stricter policy. | Rule | Organization setting | Project override | | - | - | - | | **Opper retention** | Default retention period | A shorter period | | **Spend limits** | Total monthly budget | A cap within the organization budget | | **Model access** | Models available across the organization | A narrower set of models | | **Checks** | Runs for every project | Runs only for selected projects | | **Routing** | The order a model pool is tried in | A different order, or round robin | Project overrides cannot loosen an organization rule. For example, a project cannot use a provider blocked at the organization level or retain data for longer than the organization allows. Routing is the exception: a project's order replaces the organization's rather than narrowing it. Overrides for retention, spend, model access, and routing appear directly below their organization rule. For checks, choose the scope while editing the check. ## Save a set of changes You can edit several sections before applying them. Nothing changes until you select **Save changes**. The unsaved-changes bar listing the edited sections The save bar lists every section with unsaved changes. Select **Discard** to reload the saved configuration, or **Save changes** to apply the entire draft. If something needs attention, the bar links to the affected field and disables saving until the problem is fixed. A check that cannot be saved until at least one project is selected ## When no rules are configured Spend limits and Opper retention before any rules are configured With no applicable enabled rules: * Opper records usage metadata, which [Logs](/control-plane/logs) lists, but does not store trace content such as inputs, outputs, or spans. * Usage is uncapped. * Every model in the catalog is available. * No checks run. See [Opper retention](/control-plane/rules/retention) for the difference between trace content and the usage metadata Opper keeps for billing. ## Link to a rule Each section has a stable URL fragment: | Fragment | Opens | | - | - | | `#retention` | Opper retention | | `#spend` | Spend limits | | `#model-access` | Model access | | `#checks` | Checks | | `#rule-` | A specific check | When trace storage is enabled, checks and model-access decisions appear as events on the relevant [trace span](/control-plane/trace). Rules still enforce when effective retention is zero or unset, but no trace content is stored. # Opper retention Source: https://docs.opper.ai/control-plane/rules/retention Choose whether Opper stores traces and how long to keep them. **Opper retention** controls whether Opper stores traces of your calls and how long those traces remain available. Open **Rules** and use the **tracing** toggle to turn trace storage on or off for the organization. Retention rules are a Control Plane feature. On the Gateway plan, Opper stores no inputs or outputs, but every call still appears in [Logs](/control-plane/logs) with its usage record. Provider data requirements are separate. Use [Model access](/control-plane/rules/model-access) to choose which provider data policies your calls must meet. Turning Opper tracing off does not change model access, and selecting a provider's zero data retention policy does not turn Opper tracing off. ## Turn tracing on or off With **tracing On**, Opper stores spans, inputs, and outputs for the effective retention period. Set **trace retention** to a whole number from **1 to 30 days**. Opper retention with tracing On, a 7-day trace retention period, and the option to shorten retention for a project With **tracing Off**, switching off an enabled organization retention rule sets it to **0 days**. New calls do not store trace content, and project rules cannot turn tracing back on while that organization-wide zero-day rule applies. Usage and cost are still recorded. Opper retention with tracing Off and a notice that retention is 0 days for every project while usage and cost are still recorded Turning tracing back on restores the previous valid period if you switched it off in the same page session. Otherwise, it uses **7 days**. You can change that period before saving. Select **Save changes** to apply your draft, or **Discard** to return to the saved settings. A new trace retention period applies to calls made after you save. Existing traces keep the expiry date they received when they were created; turning tracing off does not immediately delete those traces. ### When no organization retention rule is enabled The toggle also shows **Off** when the organization rule is missing or disabled. This is different from an enabled **0-day** rule: a project can still store traces if it has its own enabled positive retention period. With no applicable enabled retention rule, Opper records usage metadata but does not store traces. Removing the organization rule therefore does not enforce zero-day retention across every project. Check the explanation beside the toggle to see whether retention is zero, the rule is disabled, or no organization rule exists. ## Shorten retention for a project With organization tracing on, select **Shorten retention for a project** and choose a project. Set its period from **0 to 30 days**. The effective period is always the shortest enabled retention rule that applies. For example, if the organization keeps traces for 30 days and a project has a 7-day override, that project keeps traces for 7 days. A project cannot extend the organization period. Set a project to **0 days** to stop storing its new traces while other projects continue tracing. ## Files and response caching Positive trace retention periods apply to traces, not stored files. Files keep their own lifetime: they remain until deleted unless you set a file expiry. See [Files and retention policy](/build/multimodal/files#files-and-retention-policy). An enabled **0-day** retention rule has additional effects in its scope: * File uploads are rejected and generated outputs are not persisted as stored files. * Existing files are scheduled for permanent deletion. If files already exist, saving the rule asks you to confirm the number that will be deleted. * Response caching is skipped. Requests that explicitly ask to use or refresh the cache, or set a positive cache lifetime, are rejected. These effects follow Opper's zero-day retention setting. A provider data policy in **Model access** does not apply them by itself. ## Usage metadata is retained separately Opper keeps a separate usage record for billing and reporting. It contains cost, tokens, latency, status, provider, model, and any `X-Opper-Tags` used for [usage attribution](/build/gateway/usage-attribution). It does not contain prompts or responses. Usage records are retained for **5 years**, including when tracing is off. Trace retention rules do not shorten this period. [Logs](/control-plane/logs) lists these records one call at a time, and shows each call's input and output while its retention period lasts. Tags are kept with usage records. Avoid putting sensitive data in tags; include only the identifiers you need for attribution. ## Checks without stored traces Checks that act during the request still enforce when effective retention is zero or unset. **Block** rejects matching calls and **Redact** replaces matching content. **Flag** still evaluates, but has no stored span on which to record its result. [Score checks](/control-plane/rules/checks#score-checks) need stored outputs and pause until the affected calls have a positive retention period. ## Existing zero data retention rules Organizations that saved an older combined **zero data retention** rule still have that rule enforced. It forces retention to **0 days** and restricts model access; the tracing toggle is locked while the organization rule is active. Select **Replace with new settings** to prepare a replacement: tracing stays off in **Opper retention**, and the provider restrictions move to **Model access**. Save the changes to apply the replacement. You can then manage Opper storage and provider requirements separately. Existing project-scoped zero data retention rules remain in the override list as **0 days** with a **ZDR** badge and continue to apply to those projects. # Routing Source: https://docs.opper.ai/control-plane/rules/routing Choose which provider a model pool tries first: cheapest, fastest, highest throughput, or a weighted blend. Routing sets the order Opper tries the providers behind a **model pool**. Set it once for the organization, then override it for projects with different priorities. The Routing section with a weighted pool order: cost 50, latency 25, throughput 25 ## What a model pool is Many models in the catalog are served by more than one provider. Asking for a model by its bare name — `gpt-oss-120b` rather than `cerebras/gpt-oss-120b` — reaches the whole pool of providers that serve it. Opper tries one of them first and falls over to the rest if it fails. With no routing rule, the pool is tried in **round robin**: every call goes to the next provider in turn, which spreads load evenly but ignores everything the providers differ on — price, time to first token, throughput. A routing rule states the preference once, and every pooled call in its scope follows it. Pinned calls are unaffected. A call that names a specific provider's model gets exactly that model. ## Choose an order | Order | Tries first | | - | - | | **Round robin** | The next provider in turn. The default when no rule is set. | | **Cheapest first** | The lowest catalog list price, input and output blended at the ratio real traffic runs at. | | **Fastest first** | The lowest median time to first token, measured over the last day of traffic. | | **Highest throughput** | The most completion tokens per second, measured over the last day. | | **Weighted** | A blend of the three, on the weights you set. | Price is published, so every provider ranks on it from the start. Latency and throughput are measured from recent traffic across Opper, not just yours, so a newly added provider has no number until it has served some calls; it sorts last until then and stays in the pool for failover. ## Set the weights The three bars are always drawn, and each order chip is a preset for them: **Cheapest first** is cost at 100 and nothing else. Drag a second bar up and the order becomes **Weighted** by itself; a blend that collapses to one bar reads as that metric's preset again. Only the ratios matter. Each metric is scaled against a typical value first — a dollar of price, a second of latency and a rate of throughput are not comparable otherwise — so "cost 50, latency 25" means what it says: price counts twice as much as latency, and throughput does not count at all. A weight of zero drops the metric from the blend entirely. A provider with no latency number still ranks on a blend that only weighs cost. ## Override the order for a project Select **Override for a project** to add an override. A project's own order replaces the organization's, so a project can choose round robin to switch an inherited order off, or pick a different order for a different priority. ## Where you see the result Every pooled chat completion reports how it was routed in its `meta.routing` object (other endpoints name the served model in the `X-Opper-Served-Model` and `X-Opper-Served-Provider` headers): `served` names the provider and model that ran, `pool_size` how many could have, and `pool_rank` where the served one stood in the order Opper computed for that call. `pool_source` says what decided the head — `rule`, `affinity`, or `round_robin` — and `pool_order` names the rule's order when one applied. A conversation that has already been served by one provider keeps going to it while its prompt cache is warm, which outranks the routing rule. A rule change therefore shows up on new conversations first; an existing one follows within an hour. # Spend limits Source: https://docs.opper.ai/control-plane/rules/spend Set a monthly organization budget and optional project caps. Spend limits protect you from unexpected usage. Set one budget for the organization, then give individual projects smaller caps where you need tighter control. A $2,000 organization budget with a $500 project cap ## Set the monthly budget | Field | What it controls | | - | - | | **Monthly budget** | The maximum organization spend for the calendar month. | | **Warn at** | The percentage at which the usage indicator turns amber. The default is 80%. | The usage indicator shows current month-to-date spend: * Green below the warning threshold * Amber at or above the warning threshold * Red at 100% **Warn at** changes the indicator on this page; it does not send an email or notification. ## What happens when you reach a limit The call that reaches or crosses the limit completes. Opper blocks the next call with **HTTP 402 Payment Required**, so final spend can be slightly higher than the configured amount. Budget limits reset at the start of each calendar month in **UTC**. Adding credit does not clear a budget block because the limit applies to spend, not account balance. To resume calls sooner, raise or remove the budget; Opper re-evaluates the block as soon as you save. A low account balance can also return HTTP 402. Adding credit clears a balance-related block, but not a monthly budget block. ## Add project caps Select **Cap spend for a project** to limit one project's monthly spend. Project usage counts toward the organization budget—it is not an additional allowance. When you save, Opper checks that: * The organization has at most one enabled monthly budget. * Each project has at most one enabled project cap. * Every project cap is less than or equal to the organization budget. These checks work in both directions. If an existing project cap is $500, for example, you cannot lower the organization budget below $500 until you reduce the project cap. ## What counts toward a limit Spend limits cover every billable model call, including: * LLM and embedding calls through the v3 API and compatibility gateway * Image, speech, and video generation * Judge calls made by [score checks](/control-plane/rules/checks#score-checks) Use the organization budget as the hard ceiling for your account. Use project caps to contain workloads that may spike independently. # Sovereign AI: a European-owned AI gateway (Opper on evroc) Source: https://docs.opper.ai/control-plane/sovereign-ai Opper on evroc is a sovereign AI gateway: the same Opper on evroc, the Swedish cloud, paired with European-owned model providers hosted in Europe. For public sector, defense, and regulated teams. Access by onboarding. Opper on evroc is a sovereign AI gateway: the same Opper, running as its own instance on evroc, the Swedish cloud. Pair it with model providers that are European-owned and host their models in Europe, and your data stays with European companies from your app to the model. It is for teams where EU residency is not enough and policy requires European ownership at every hop, such as government and the public sector, defense, critical infrastructure, and regulated industries. It does for Opper what a government cloud region does for the large US clouds: the same product, run separately for workloads that cannot sit on shared commercial infrastructure. Access is by onboarding, not self-service sign-up. [Contact us](https://opper.ai/contact) and we set up your organization on the evroc instance. More at [opper.ai/sovereign-ai](https://opper.ai/sovereign-ai). ## European at every hop | Hop | Company | Country | | - | - | - | | **Gateway** | Opper Technology AB, your processor under a DPA governed by Swedish law, ISO/IEC 27001:2022 certified | Sweden | | **Cloud** | evroc | Sweden | | **Model provider** | A European-owned provider that hosts its models in Europe | Europe | ## What you can do Everything you build on Opper works on the evroc instance: * **The gateway**: one OpenAI-compatible API for chat completions, structured output, tool calling, streaming, and fallbacks across models. * **Rules**, on Control Plane, set once for the whole organization: a monthly spend limit, how long traces are kept, which models and providers are allowed, checks that redact or block sensitive content before a model sees it, and the order providers are tried in. * **Usage** for every call, and **traces** once you turn tracing on, kept on the same instance. The instance has its own account, API keys, and address. Projects build against the same API on both instances, so moving between them means changing the base URL and the API key: | | Main platform | Opper on evroc | | - | - | - | | **Platform** | `platform.opper.ai` | `evroc.on.opper.ai` | | **API** | `https://api.opper.ai` | `https://api.evroc.on.opper.ai` | | **Runs on** | AWS Stockholm (`eu-north-1`) | evroc, Sweden | | **Access** | Self-service sign-up | Onboarding | ```bash theme={null} curl https://api.evroc.on.opper.ai/v3/compat/chat/completions \ -H "Authorization: Bearer $OPPER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "mistral/mistral-large-2512", "messages": [{"role": "user", "content": "Hello"}] }' ``` Use an API key from the evroc instance. Keys from the main platform do not work there. ## Keep every call sovereign ### Sovereign routes A route is sovereign when the company operating it is based in Europe and the model runs in Europe. That covers Mistral's own models and open-weight models on European hosts such as Berget, evroc, GreenPT, Regolo, and Scaleway. Call any of them by its route ID, such as `mistral/mistral-large-2512`. The [model catalog](/v3-api-reference/models/list-models) records both facts for every route: `provider.owner_country` is where the operator's ultimate parent is based, and `compliance.inference_location` is where the model runs. It needs no API key. ### Enforce it for your organization On Control Plane, one [Model access](/control-plane/rules/model-access) rule makes sovereign routes the only ones your organization can call. In **Rules → Model access**, set **Inference location** and **Provider country** to the same area, such as **EEA**. The [Management API](/control-plane/management-api#manage-rules-from-code) takes the same areas and country codes. This rule also admits the UK and Switzerland: ```bash theme={null} curl -X POST https://api.evroc.on.opper.ai/management/v1/controls/rules \ -H "Authorization: Bearer $OPPER_MANAGEMENT_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "id": "'$(uuidgen | tr A-Z a-z)'", "kind": "comply", "enabled": true, "scope": {"type": "org"}, "config": {"type": "allowlist", "allowlist": {"provider": {"locations": { "inference": {"include": ["EEA", "GB", "CH"]}, "jurisdiction": {"include": ["EEA", "GB", "CH"]} }}}} }' ``` `jurisdiction` is the **Provider country** field. Both fields match by containment, so `EEA` admits a route in Sweden. A call pinned to a route outside the rule is refused with `403` before it reaches a provider, and a bare model name skips the providers outside it. ## Other markets and custom deployments Opper on evroc is a dedicated instance for Europe. The same Opper can run as a dedicated instance for another market, inside your own cloud account (AWS, Azure, Google Cloud, or a national sovereign cloud), or on your own hardware, with the same API on every option. If you need sovereign AI outside Europe or on your own infrastructure, [contact us](https://opper.ai/contact). ## Need Claude, GPT, or Gemini? Most teams need EU residency rather than sovereignty. The main Opper platform runs in AWS Stockholm and serves Claude, GPT, and Gemini from EU data centers on AWS, Azure, and Google Cloud. See [Keep AI inference in the EU](/control-plane/eu-data-residency). ## Related Enforce EU-only inference and storage for every call on the main platform. Every allowlist field, including provider country and inference location. Store no content at Opper and allow only providers that neither train on nor log it. Certifications, hosting, DPA, and sub-processors. # Traces Source: https://docs.opper.ai/control-plane/trace See the full tree behind every call — model calls, tool calls, and every rule that fired. Tracing records what happened inside every call—the model call, any nested tool or LLM calls, and any rules that ran. Each call becomes a **trace**: a tree of **spans** you can open and inspect. A typical use: figure out why one request was slow or wrong — which model ran, how long each step took, what a tool returned, and what a judge scored it. Without an applicable enabled retention rule, Opper records usage metadata — model, cost, and latency — but does not store traces. To capture spans, inputs, and outputs, turn on tracing in [Opper retention](/control-plane/rules/retention). [Logs](/control-plane/logs) lists every call either way. ## What you see in the dashboard Open **Traces** in [platform.opper.ai](https://platform.opper.ai). Three views build on each other. **The trace explorer** lists recent traces, each row showing the time, total duration, the trace name with its step composition (e.g. *28 steps · 8 llm · 11 tool*), cost and tokens, and a preview of the input and output. A status dot flags failures, and you can search by trace name or filter by **Status**, **Duration**, and **Cost**. The Traces explorer listing recent traces with duration, step composition, cost, tokens, and an input/output preview **The span tree** opens when you click a trace. The left pane lists every span as a row with its name and duration, drawn as a timeline bar so you can see where time went and which calls fanned out from which. Generations (`llm`) and tool calls (`web_search`, `web_fetch`, …) nest under the step that ran them. The header sums up the whole trace — **duration**, **tokens**, and **cost**. A trace's span tree with nested LLM and tool-call spans, each showing its duration, beside the selected span's input and output **The span detail** pane shows everything for the selected span: its span ID, parent, where it starts within the trace, its duration, and — with retention on — the full input and output (rendered as chat, or as raw JSON with the `{}` toggle). Click into any span to drill down through the breadcrumb at the top. The span detail pane for a web_fetch span showing span ID, parent, start offset, input, and output ### Quick actions A few actions sit in the header of each view: | Action | What it does | | - | - | | **Open in playground** | Loads the trace into the [playground](https://platform.opper.ai) as an editable conversation so you can replay and iterate on it (see below). | | **Copy span** | Copies the span's full JSON (input, output, metadata, timings) to your clipboard, handy for sharing a repro or pasting into an issue. | | **Open trace in new tab** | Opens the full trace on its own page (via the ↗ icon), so you can keep it open while you work or share a direct link to it. | ### Replay a trace in the playground **Open in playground** reconstructs the trace as an editable conversation — every system, user, and assistant turn, in order — so you can pick up a real production run and iterate on it. A trace loaded into the playground as a conversation, with a Run from here button on a message and a model switcher in the side panel From here you can: * **Run from here** — hover any message and replay the conversation from that point, so you can change one turn and see how the run plays out without rerunning everything before it. * **Switch the model** — pick a different model from the **Model** dropdown to see how another model handles the same input, and adjust **Parameters** (temperature, max tokens), **Controls**, and **Tools** alongside it. * **View code**, **Compare**, or **Save** the result — none of this touches production, so it's a safe place to debug a bad trace or tune a prompt. ## The trace model A trace is a tree of spans. The **root span** names the trace; every other span points to its parent through `parent_id`, which is how the tree is built. Spans of type `generation` are the LLM calls; tool and function calls appear as their own spans nested under the step that invoked them. | Field | What it holds | | - | - | | `name` | Human-readable label for the step. The root span's name becomes the trace name. | | `type` | Span kind — e.g. `generation` (an LLM call), `function`, or `call`. | | `input` / `output` | The data into and out of the step. Stored only with retention on. | | `start_time` / `end_time` | UTC timestamps; the dashboard derives a duration from them. | | `parent_id` | The parent span, used to build the tree. Empty on the root span. | | `error` | Error message if the step failed. | | `meta` / `tags` | Arbitrary metadata you attach for filtering and context. | | `score` | Numeric score on the span. | | `metrics` | Custom measurements, each a `dimension` + `value` (+ optional `comment`). | Generation spans carry extra detail — the `model` that ran, `total_tokens`, the `instructions` used, and, when a [score check](/control-plane/rules/checks#score-checks) runs, the judge's `observations` and per-criterion `scorer_context`. The trace itself rolls up its spans: `name`, `input`, `output`, `start_time`/`end_time`, `duration_ms`, `status`, `span_count`, and `total_tokens`, plus the ordered `spans` tree and any `events`. ## Retention Retention rules attach at the org or project level, so you can set different policies for different projects — a longer period on a staging project while you debug, or zero-day retention on a sensitive production one, and so on. Set them in [Opper retention](/control-plane/rules/retention). See [Core concepts](/overview/concepts) for how this fits into the request path. ## Instrument custom spans Most spans are created for you. To trace work that happens outside a single Gateway call — a multi-step pipeline, a background job, your own retrieval step — create spans yourself. A span with no `parent_id` starts a new trace; pass `parent_id` to nest a child under it. ```python Python theme={null} from datetime import datetime, timezone from opperai import Opper opper = Opper() # Root span — starts a new trace span = opper.spans.create( name="my-pipeline", start_time=datetime.now(timezone.utc).isoformat(), input="Starting the pipeline", meta={"userId": "u-123"}, ) # Child span — nested under the root via parent_id step = opper.spans.create( name="retrieve-context", parent_id=span.id, start_time=datetime.now(timezone.utc).isoformat(), ) # Close a span with its output and end time opper.spans.update( step.id, output="Found 5 documents", end_time=datetime.now(timezone.utc).isoformat(), ) print(f"Span: {span.id}, Trace: {span.trace_id}") ``` ```typescript TypeScript theme={null} import { Opper } from "opperai"; const opper = new Opper(); // Root span — starts a new trace const span = await opper.spans.create({ name: "my-pipeline", start_time: new Date().toISOString(), input: "Starting the pipeline", meta: { userId: "u-123" }, }); // Child span — nested under the root via parent_id const step = await opper.spans.create({ name: "retrieve-context", parent_id: span.id, start_time: new Date().toISOString(), }); // Close a span with its output and end time await opper.spans.update(step.id, { output: "Found 5 documents", end_time: new Date().toISOString(), }); console.log(`Span: ${span.id}, Trace: ${span.trace_id}`); ``` ```bash cURL theme={null} # Root span — starts a new trace curl -X POST https://api.opper.ai/v3/spans \ -H "Authorization: Bearer $OPPER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "name": "my-pipeline", "start_time": "2026-06-17T10:00:00Z", "input": "Starting the pipeline", "meta": {"userId": "u-123"} }' # Child span — pass parent_id to nest it curl -X POST https://api.opper.ai/v3/spans \ -H "Authorization: Bearer $OPPER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "name": "retrieve-context", "parent_id": "SPAN_ID", "start_time": "2026-06-17T10:00:01Z" }' # Close a span with its output and end time curl -X PATCH https://api.opper.ai/v3/spans/SPAN_ID \ -H "Authorization: Bearer $OPPER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "output": "Found 5 documents", "end_time": "2026-06-17T10:00:05Z" }' ``` ## Read traces programmatically List recent traces, then fetch one to walk its span tree. ```python Python theme={null} from opperai import Opper opper = Opper() traces = opper.traces.list(limit=5) for t in traces.data: print(f"{t.id} - {t.name or '(unnamed)'} ({t.span_count} spans)") trace = opper.traces.get(traces.data[0].id) for s in trace.spans: indent = " " if not s.parent_id else " " print(f"{indent}{s.name} ({s.id[:8]}...)") ``` ```typescript TypeScript theme={null} import { Opper } from "opperai"; const opper = new Opper(); const traces = await opper.traces.list({ limit: 5 }); for (const t of traces.data) { console.log(`${t.id} - ${t.name ?? "(unnamed)"} (${t.span_count} spans)`); } const trace = await opper.traces.get(traces.data[0].id); for (const s of trace.spans) { const indent = s.parent_id ? " " : " "; console.log(`${indent}${s.name} (${s.id.slice(0, 8)}...)`); } ``` ```bash cURL theme={null} # List recent traces curl https://api.opper.ai/v3/traces?limit=5 \ -H "Authorization: Bearer $OPPER_API_KEY" # Get one trace with its full span tree curl https://api.opper.ai/v3/traces/TRACE_ID \ -H "Authorization: Bearer $OPPER_API_KEY" ``` ## Events and feedback on spans As a call runs, Opper records rule and routing activity as **events** on the relevant span. Each event has a `kind`—`guardrail`, `observe`, `route`, `comply`, or `feedback`—plus a source and data payload. These backend event names remain for API compatibility even though the platform now presents them through Rules. You can also write your own events — for example, capturing thumbs-up/down feedback from your app — with `POST /v3/spans/{id}/events`, and read them back with `GET /v3/spans/{id}/events`. Use `meta` and `tags` on your custom spans to record context like user, tenant, or feature flag. It makes traces far easier to filter and group later. To attribute usage and **cost** by those same dimensions, tag the calls themselves — see [Tags & usage attribution](/build/gateway/usage-attribution). ## Where to go next Score every response against criteria you write. Results land on the trace. Turn on retention so traces store full inputs and outputs. Create, update, and read spans directly over the API. How traces fit with the Gateway, projects, and Rules on a call. # Enforce zero data retention (ZDR) Source: https://docs.opper.ai/control-plane/zero-data-retention Zero data retention on Opper: store no prompts or outputs at Opper and allow only providers that neither train on nor log your content. Choose ZDR routes per call on any plan, or enforce ZDR org-wide with Rules. Zero data retention (ZDR) on Opper has two parts, and you can enforce both for your whole organization: * **Opper stores no content.** With an enabled 0-day [Opper retention](/control-plane/rules/retention) rule, Opper stores no prompts, outputs, or traces for any project. It keeps only a usage record with no prompts or responses in it. * **The provider neither trains on nor logs your content.** The **Zero data retention** provider data policy in [Model access](/control-plane/rules/model-access) allows only model routes whose provider neither trains on nor logs request or response content. A call to any other route is refused with `403` before the request leaves Opper. The two settings are independent: turning Opper tracing off does not restrict providers, and the provider policy does not turn tracing off. Set both. ## What each plan includes | | Gateway | Control Plane | Enterprise | | - | - | - | - | | **Opper storage** | Usage metadata only. No prompts, outputs, or traces are stored. | None by default. Turn tracing on for 1 to 30 days, or enforce 0 days for the organization or a project. | Custom retention. | | **Provider ZDR** | Choose a ZDR route in each call. Nothing stops another call from using a route that logs. | Enforced org-wide by a Model access rule. Other routes are refused. | Enforced org-wide, with custom terms. | Rules, including retention and model access, are a Control Plane feature. On the Gateway plan you get ZDR by choosing the route in every call; see [Choose a ZDR route per call](#choose-a-zdr-route-per-call). ## Enforce ZDR for your organization In the [Opper platform](https://platform.opper.ai), open **Rules**: In **Opper retention**, switch tracing **Off** and save. This leaves the organization with an enabled **0-day** rule: new calls store no traces, and projects cannot turn tracing back on. If the toggle already shows Off because the organization rule is missing or disabled, turn it on and off again in the draft before saving, so that an enabled 0-day rule is saved. In **Model access**, set **Provider data policy** to **Zero data retention**. This selects **No training** and **No logging**. Check the match counter to see which models remain, then select **Save changes**. Some providers run content moderation classifiers that can retain flagged content. **No provider moderation** excludes them, and needs a signed agreement with Opper first. See [Arrange approval for No provider moderation](/control-plane/rules/model-access#arrange-approval-for-no-provider-moderation). Both rules apply from the next request. To manage them from code, use the [Management API](/control-plane/management-api#manage-rules-from-code). ## What ZDR means, term by term Providers describe data handling in different words. This is how common terms map to what Opper records for each route and checks at request time: | Term you may see | On Opper | Allowed under the ZDR policy | | - | - | - | | No training on your data | **No training** | Yes, required | | Request or response logging | **No logging** means the provider keeps no content in logs | Only routes with no logging | | Abuse monitoring (for example, 30-day retention for abuse review) | Recorded as `abuse_monitoring` logging | No, it fails **No logging** | | Content moderation | Recorded separately. **No provider moderation** is its own requirement. | Yes, unless you also require **No provider moderation** | | Prompt caching | Recorded separately as the route's caching | Yes; **No logging** does not cover caching | For example, in Opper's catalog in September 2026: | Route | Where inference runs | Training | Logging | Under the ZDR policy | | - | - | - | - | - | | `anthropic/claude-sonnet-5` | United States | No | Abuse monitoring | Refused | | `openai/gpt-5.5` | United States | No | Abuse monitoring | Refused | | `aws/claude-sonnet-4-6-eu` | Sweden | No | None | Allowed | | `azure/gpt-5.5` | Sweden | No | None | Allowed | | `mistral/mistral-large-2512` | France | No | None | Allowed | A few providers require a signed agreement before your organization can call their ZDR routes, such as `azure-zdr` and `vertexai-zdr`. See [Routes that require an agreement](/control-plane/rules/model-access#routes-that-require-an-agreement). ### What ZDR does not cover * **Provider caching.** Some routes cache prompts implicitly. The route's caching is recorded separately and **No logging** does not exclude it. * **Content kept for moderation.** Routes that run provider moderation may retain flagged content. Require **No provider moderation** to exclude them. * **Opper's usage record.** Opper keeps a record of each call's cost, tokens, latency, status, provider, model, and any `X-Opper-Tags` for 5 years, for billing. It contains no prompts or responses, so keep personal data out of tags. See [usage metadata](/control-plane/rules/retention#usage-metadata-is-retained-separately), and [Logs](/control-plane/logs) to browse the records call by call. * **Services outside Opper.** Tools and services your application calls directly keep their own retention. A 0-day Opper retention rule also changes what Opper does with files and caching in its scope: file uploads are rejected, existing files are deleted after you confirm, and response caching is skipped. See [Files and response caching](/control-plane/rules/retention#files-and-response-caching). ## Fallbacks and pools under ZDR Opper never falls back to a route that fails your policy. A bare model name such as `claude-sonnet-5` runs only on the providers in its pool that pass, and a pinned route such as `anthropic/claude-sonnet-5` that fails is refused with `403` rather than switched. Pools in dynamic routes and `models` fallback lists skip failing entries in the same way; a Model node in a dynamic route instead ends the route with `403`. See [What model access covers](/control-plane/rules/model-access#what-model-access-covers) for each case and [What a blocked call returns](/control-plane/rules/model-access#what-a-blocked-call-returns) for the error bodies. ## Find routes that qualify * **Query the catalog.** `GET /v3/models` needs no API key. `training=no` with `logging=none` is the same filter as the ZDR policy, so this returns the LLM routes the policy allows (`limit=0` returns every match; `total` gives the count). Routes marked **Agreement required** still need the agreement: ```bash theme={null} curl "https://api.opper.ai/v3/models?type=llm&training=no&logging=none&limit=0" ``` Each model's `compliance` object records its `training`, `logging`, `moderation`, and `caching`. * **Browse:** [opper.ai/models](https://opper.ai/models) labels each route **ZDR**, **monitored**, or **retained**. The labels are close to the policy but not identical: the policy checks training and request logging, while the labels also count content a provider can hold for moderation. Some routes labelled monitored, such as Azure routes, meet the policy. Use the query above for the exact set the policy allows. * **Check your own rules.** With your API key, add `include=policy` to see whether each model is allowed for your organization and why not, or `include=route` for the evidence behind each route's data-handling record. ## Choose a ZDR route per call Without rules, on any plan, you get provider-side ZDR on a call by naming a route that meets it, for example `mistral/mistral-large-2512`, which neither trains on nor logs your content and runs no provider moderation: ```bash theme={null} curl https://api.opper.ai/v3/compat/chat/completions \ -H "Authorization: Bearer $OPPER_API_KEY" \ -H "Content-Type: application/json" \ -d '{"model": "mistral/mistral-large-2512", "messages": [{"role": "user", "content": "Hello"}]}' ``` Opper stores no prompts or outputs either, unless you have turned tracing on (Control Plane). What per-call routing cannot do is stop a different call from naming a route that logs. For that, enforce the policy with a Model access rule. ## Related Enforce EU-only inference and storage locations. Every allowlist field and provider data requirement. Turn trace storage on or off and set how long traces are kept. Certifications, hosting, DPA, and sub-processors. # CLI Source: https://docs.opper.ai/developer-tools/cli The official Opper command for the terminal: sign in, manage models, view traces, generate images, and launch coding agents through Opper The Opper CLI is the official `opper` command for the terminal. It gives you direct platform access to manage models, inspect traces, track usage, and generate images. It can also launch coding agents (Claude Code, Claude Desktop, OpenCode, Copilot CLI, Codex, Codex Desktop, Hermes, OpenClaw, pi) and route their inference through Opper. ## Install ```bash theme={null} npm i -g @opperai/cli ``` Or run any command on demand without installing: ```bash theme={null} npx @opperai/cli ``` Requires Node.js ≥ 20.12. ### Migrate from the retired Go CLI The Go CLI and its Homebrew distribution are retired. Use the npm package above for the current CLI. Both versions install an `opper` executable, so remove the old Homebrew installation before installing the replacement: ```bash theme={null} brew uninstall opper-ai/oppercli/opper brew untap opper-ai/oppercli npm i -g @opperai/cli opper --help ``` If you installed the Go CLI manually or with `go install`, check `which -a opper` and remove the old executable from your PATH. Confirm that `opper --help` shows the current CLI before continuing. Keep `~/.oppercli` while migrating. On `opper login`, the current CLI can import its named API keys and base URLs into `~/.opper/config.json` once, provided the new configuration does not already exist and migration has not already been attempted. The legacy file is preserved. If you already configured the current CLI, use `opper config add` to add any remaining keys instead. Command arguments can differ between versions; check `opper --help` when updating scripts. ## Quick start ```bash theme={null} opper login # OAuth device flow opper whoami # confirm the active account opper launch claude # launch Claude Code through Opper ``` ## What you can do | Command group | What it does | | - | - | | `login` / `logout` / `whoami` | OAuth device flow and account inspection. | | `ask` | Ask a question grounded on the installed Opper [skills](/developer-tools/skills). Useful for "how do I…" questions about the platform, SDKs, or the CLI itself. | | `call` | Legacy. Executes an Opper function by name through the `/call` endpoint, with `--model` and `--stream`. | | `launch ` / `agents configure ` / `agents list` / `agents remove ` | Launch coding agents through Opper (Claude Code, Claude Desktop, OpenCode, Copilot CLI, Codex, Codex Desktop, Hermes, OpenClaw, pi). `agents configure` sets an agent up with your Opper key without launching it. `agents remove` removes the Opper integration without uninstalling the agent itself. | | `editors` | Wire AI code editors (OpenCode) to use Opper as a provider. | | `skills` | Install, update, list, or uninstall bundled Opper [skills](/developer-tools/skills). | | `models`, `traces`, `usage` | Manage every Opper platform resource directly. | | `image generate` | Generate images from a prompt. | | `config` | Manage stored API keys and slots. | For full flags on any subcommand: `opper --help`. Or run `opper` with no arguments for an interactive menu. ## Launch coding agents through Opper `opper launch ` routes a coding agent through Opper. Every model call from the agent goes through your Opper account, so you get the same cost tracking and model routing, with each call in [Logs](/control-plane/logs) and full [traces](/control-plane/trace) once tracing is on. | Agent | Slug | How Opper plugs in | | - | - | - | | [Claude Code](/integrations/coding-agents/claude-code) | `claude` | `ANTHROPIC_BASE_URL` / `ANTHROPIC_AUTH_TOKEN` env vars | | [Claude Desktop](/integrations/personal-agents/claude-desktop) | `claude-desktop` | writes a third-party-inference (`deploymentMode: "3p"`) profile into `~/Library/Application Support/Claude-3p/` (macOS) / `%LOCALAPPDATA%\Claude-3p\` (Windows); quits and reopens the GUI app to apply | | [OpenCode](/integrations/coding-agents/opencode) | `opencode` | provider block in `~/.config/opencode/opencode.json` | | [GitHub Copilot CLI](/integrations/coding-agents/github-copilot-cli) | `copilot` | browser sign-in, allowed model picker, usage and renewal commands; requires Node.js 22+ on macOS/Linux | | [Codex](/integrations/coding-agents/codex) | `codex` | passes an Opper provider to each run with `-c` flags; `~/.codex/config.toml` is left unchanged | | Codex Desktop | `codex-desktop` | macOS only: writes an `opper-desktop` provider into `~/.codex/config.toml` and opens the Codex/ChatGPT desktop app (if it's already running, quit and reopen it); applies to new local Codex tasks | | [Hermes](/integrations/personal-agents/hermes) | `hermes` | isolated `HERMES_HOME=~/.opper/hermes-home/` so your real `~/.hermes/` is never touched | | [OpenClaw](/integrations/personal-agents/openclaw) | `openclaw` | `opper` provider entry in `~/.openclaw/agents/main/agent/models.json`; defaults to running `gateway start` (background daemon) | | [pi](/integrations/coding-agents/pi) | `pi` | `opper` provider entry in `~/.pi/agent/models.json` | Anything after the agent name is forwarded to the agent's CLI verbatim (Claude Desktop and Codex Desktop are GUI-only and do not accept extra arguments): ```bash theme={null} opper launch claude -- --print "Hello" ``` To remove an agent's Opper integration without uninstalling the agent itself: ```bash theme={null} opper agents remove claude-desktop # works for any registered adapter ``` For a full Claude Code walkthrough — model alias mapping, the `/model` picker, per-project config, and CI — see [Run Claude Code on any model](/guides/claude-code-router). ## Ask the CLI a question `opper ask ""` runs an Opper agent grounded on the locally-installed Opper [skills](/developer-tools/skills). The answer streams to your terminal, then prints token and request counts. ```bash theme={null} opper skills install # one-time opper ask "how do I make a call?" opper ask --model anthropic/claude-opus-4-7 "compare the v2 and v3 APIs" ``` Use `ask` to get unstuck without leaving the terminal. The model has the same skill packs available as the agents you launch through `opper launch`. ## Authentication Auth state lives in `~/.opper/config.json` as a list of slots, each holding an API key and base URL. Use `--key ` on any command to pick which slot to use. Resolution at request time: `OPPER_API_KEY` env var > slot named by `--key` (or `default`). ```bash theme={null} opper config add staging --base-url https://... opper models list --key staging ``` ## Source and reference View on GitHub The repository README is the source of truth for releases, supported agents, and configuration details. For exact arguments and flags, `opper --help` is always authoritative. # MCP server Source: https://docs.opper.ai/developer-tools/mcp Connect a coding agent to your Opper account so it can create projects and API keys, check models and make calls, with your approval in the browser Opper has an MCP server at `https://api.opper.ai/mcp`. Connect a coding agent to it and the agent can set up Opper for you: create a project, create an API key, check that a model is available and make a first call. You approve the connection once in the browser and choose what the agent may do. Sign up needs no credit card. Free models work before you add a card, so an agent can take you from no account to a working call without you opening the dashboard. This is not the documentation search server. `https://docs.opper.ai/mcp` lets an agent search these docs, needs no sign-in and cannot see or change your account. Use that one when an agent needs to look something up, and the server on this page when it needs to do something. ## Connect your agent Add a remote (HTTP) MCP server with the URL `https://api.opper.ai/mcp`. Leave out any API key and any scope list. The client opens a browser window where you sign in and choose the permissions. The server speaks MCP over Streamable HTTP and signs you in with OAuth. It keeps no session, so there is nothing to reconnect after a deploy. ### OpenCode Tested with OpenCode 1.18. Add this to `opencode.json`: ```json theme={null} { "mcp": { "opper": { "type": "remote", "url": "https://api.opper.ai/mcp", "enabled": true } } } ``` Then authenticate and restart the TUI: ```bash theme={null} opencode mcp auth opper opencode mcp list # should report "connected" ``` ### Other clients Any MCP client that supports remote servers with OAuth sign-in can use the URL above. OpenCode is the one we have tested end to end so far. ## What happens when you connect 1. Your agent asks to connect and your browser opens `platform.opper.ai`. 2. You sign in, or create an account if you do not have one. No card is asked for. 3. You pick the organization and what the agent may do. **Read only** is selected to begin with. **Build and manage** adds projects, API keys, rules, routes and model calls. Deleting projects is a separate checkbox. 4. The agent continues on its own. Nothing is pasted into the conversation. The authorization link is valid for ten minutes. If sign-up takes longer, ask the agent to start the connection again. ## Permissions You grant permissions per connection. The agent never gets more than you ticked, and adding one later needs a new approval in the browser. | Permission | Lets the agent | | - | - | | `account:read` | See which account and organization it is connected to, the plan and billing status, and check whether a model is available to you | | `projects:read`, `projects:write`, `projects:delete` | List, create and edit, or delete projects | | `apikeys:read`, `apikeys:write` | List API keys (prefix only), or create and delete them | | `controls:read`, `controls:write` | Read or change rules. Changing rules needs a plan that includes them | | `dynamic_routes:read`, `dynamic_routes:write` | Read, or build, simulate, deploy and roll back custom routes | | `runtime:call` | Make real model calls with one of your project's keys. Normal costs and rules apply | | `runtime:read` | Read usage, costs and traces | ## What works without an account Two tools need no sign-in, so an agent can look around first: * `list_models` searches the public model catalog. * `get_guide` returns short recipes for getting started, connecting an app, structured output, rules and debugging. Every other tool answers with a sign-in challenge, which is what makes the client open the browser. ## API keys When the agent creates an API key, the secret stays on the server by default. The agent gets the key's ID and can make calls with it through the `call_model` tool. That keeps the secret out of the model conversation. To use the key in your own application, create one under **Settings, API keys** in the dashboard and set it as `OPPER_API_KEY`. An agent can ask for the secret when it creates a key, but anything a tool returns can end up in the conversation, so only do that with a client you trust to handle it privately. ## Staying in control * See and disconnect your agents at [platform.opper.ai/mcp/connections](https://platform.opper.ai/mcp/connections). A disconnected agent stops working on its next request. * Disconnecting an agent does not delete API keys it created. Delete those under **Settings, API keys**. * The agent acts as you, in the one organization you picked. If you leave that organization, its connections stop working. * Billing, adding a card and managing members stay in the dashboard. # Skills Source: https://docs.opper.ai/developer-tools/skills Markdown files that give AI code editors context about how to use Opper, install once and activate automatically Skills are markdown files that give AI code editors context about how to use Opper. They work with any editor that supports the [Agent Skills](https://agentskills.io) standard, including Claude Code, Cursor, Cline, GitHub Copilot, OpenAI Codex, and Windsurf. When installed, skills sit in your editor's skills directory. The editor reads a short description of each skill at startup and loads the full documentation only when your task matches its domain, so it doesn't waste context. ## Installation ```bash theme={null} # All Opper skills (recommended) npx skills add opper-ai/opper-skills # Or pick what you need npx skills add opper-ai/opper-skills/opper npx skills add opper-ai/opper-skills/opper-mcp npx skills add opper-ai/opper-skills/opper-cli npx skills add opper-ai/opper-skills/opper-sdks npx skills add opper-ai/opper-skills/opper-api npx skills add opper-ai/opper-skills/opper-multimodal ``` You can also install the bundled set through the [Opper CLI](/developer-tools/cli), either headless or globally installed: ```bash theme={null} npx @opperai/cli skills install # no install needed opper skills install # if @opperai/cli is installed globally ``` To skip the install, give any AI coding assistant this prompt. It fetches the skills live and walks you through setup: ``` Use curl to download, read and follow: https://skills.opper.ai ``` ## Available skills | Skill | What it covers | | - | - | | **opper** | The entry point. Works out what you're trying to do with Opper, routes to the right skill below, and owns setup, verification, and follow-up | | **opper-mcp** | The [Opper MCP server](/developer-tools/mcp): connecting a coding agent to Opper, setting up projects and runtime keys, managing Control Plane rules, inspecting usage and traces, and testing models. Preferred for agent-assisted setup | | **opper-cli** | The `opper` command-line tool: sign-in, traces, usage, models, image generation, launching coding agents through Opper, installing skills, configuring AI editors | | **opper-sdks** | The unified `opperai` packages for Python and TypeScript, including the Agent SDK (tools, streaming, multi-agent, MCP, conversation) | | **opper-api** | The Opper REST API: Gateway and Rules concepts, models, OpenAI/Anthropic/Google/OpenResponses-compatible endpoints under `/v3/compat`, and migration from other LLM gateways | | **opper-multimodal** | Everything beyond text: image, speech, music, and video generation, transcription and OCR, the `/v3/files` storage API, vision and PDF input, and realtime voice | ## Try it out Once installed, your editor will activate the right skill automatically. Try prompts like: * "Sign me in to Opper from the terminal and launch Claude Code through it." * "Make this Python script return structured output through Opper." * "Build an Opper agent in TypeScript with a `get_weather` tool." * "Wrap this pipeline in an Opper trace so I can see each step." * "What models does Opper support, and how do I migrate this OpenRouter code?" ## Updating ```bash theme={null} # If installed via npx skills npx skills update # If installed via the Opper CLI npx @opperai/cli skills update # or `opper skills update` if installed globally # If cloned manually cd opper-skills && git pull ``` ## Source View on GitHub # Claude Code clone Source: https://docs.opper.ai/guides/claude-code-clone Build a tiny coding agent that reads, writes, and runs code in a loop, using the gateway and tool calling. A coding agent is just a tool-calling loop with the right tools. Give the model `read_file`, `write_file`, and `run_command`, then keep calling it: as long as it returns tool calls, you run them and feed the results back. It writes code, runs it, reads the error, fixes it, and goes again until the task is done. Conversation state persists, so follow-up requests build on what it already did.
python app.py

You › Build a fizzbuzz script and run it for n=15.

Tool › write\_file("fizzbuzz.py")
→ Wrote 545 bytes

Tool › run\_command("python fizzbuzz.py")
→ /bin/sh: python: command not found

Tool › run\_command("python3 fizzbuzz.py")
→ 1 · 2 · Fizz · 4 · Buzz · Fizz · 7 · 8 · Fizz · Buzz · 11 · Fizz · 13 · 14 · FizzBuzz

Bot › Done. Wrote fizzbuzz.py, recovered from the missing `python`, and verified the output for n=15.

You › Now add a test that fizzbuzz(5) ends with "Buzz", and run it.

Tool › read\_file("fizzbuzz.py")
→ def fizzbuzz(n): ...

Tool › write\_file("test\_fizzbuzz.py")
→ Wrote 184 bytes

Tool › run\_command("python3 -m unittest -v")
→ Ran 1 test · OK

Bot › Added test\_fizzbuzz.py and it passes. fizzbuzz(5) ends with "Buzz".

Notice the loop adapting: `python` wasn't found, so it retried with `python3`; on the follow-up it read the existing file before writing the test. You don't script either move, the model reacts to what the tools return. ## The agent ```python app.py theme={null} import os, json, subprocess from openai import OpenAI client = OpenAI(base_url="https://api.opper.ai/v3/compat", api_key=os.environ["OPPER_API_KEY"]) WORKDIR = "workspace" os.makedirs(WORKDIR, exist_ok=True) def read_file(path): p = os.path.join(WORKDIR, path) return open(p).read() if os.path.exists(p) else f"(no file at {path})" def write_file(path, content): with open(os.path.join(WORKDIR, path), "w") as f: f.write(content) return f"Wrote {len(content)} bytes to {path}" def run_command(command): r = subprocess.run(command, shell=True, cwd=WORKDIR, capture_output=True, text=True, timeout=30) return (r.stdout + r.stderr).strip() or "(no output)" TOOLS = {"read_file": read_file, "write_file": write_file, "run_command": run_command} specs = [ {"type": "function", "function": {"name": "read_file", "description": "Read a file in the workspace.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}}, "required": ["path"]}}}, {"type": "function", "function": {"name": "write_file", "description": "Create or overwrite a file in the workspace.", "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}}}, {"type": "function", "function": {"name": "run_command", "description": "Run a shell command in the workspace and return its output.", "parameters": {"type": "object", "properties": {"command": {"type": "string"}}, "required": ["command"]}}}, ] SYSTEM = ( "You are a coding agent. Use the tools to read, write, and run code in the " "workspace until the task is done. Verify your work, then give a short summary." ) messages = [{"role": "system", "content": SYSTEM}] def send(user_message): messages.append({"role": "user", "content": user_message}) # Keep going while the model wants to call tools. while True: r = client.chat.completions.create(model="anthropic/claude-sonnet-4-6", messages=messages, tools=specs) msg = r.choices[0].message messages.append(msg) if not msg.tool_calls: return msg.content for call in msg.tool_calls: args = json.loads(call.function.arguments) print(f" · {call.function.name}") messages.append({"role": "tool", "tool_call_id": call.id, "content": TOOLS[call.function.name](**args)}) print(send("Write fizzbuzz.py with a fizzbuzz(n) function, then run it for n=15.")) print(send("Now add a test that fizzbuzz(5) ends with 'Buzz', and run it.")) ``` ```typescript app.ts theme={null} import OpenAI from "openai"; import { readFileSync, writeFileSync, existsSync, mkdirSync } from "node:fs"; import { execSync } from "node:child_process"; import path from "node:path"; const client = new OpenAI({ baseURL: "https://api.opper.ai/v3/compat", apiKey: process.env.OPPER_API_KEY }); const WORKDIR = "workspace"; if (!existsSync(WORKDIR)) mkdirSync(WORKDIR); const TOOLS: Record string> = { read_file: ({ path: p }) => existsSync(path.join(WORKDIR, p)) ? readFileSync(path.join(WORKDIR, p), "utf8") : `(no file at ${p})`, write_file: ({ path: p, content }) => { writeFileSync(path.join(WORKDIR, p), content); return `Wrote ${content.length} bytes to ${p}`; }, run_command: ({ command }) => { try { return execSync(command, { cwd: WORKDIR, timeout: 30000 }).toString().trim() || "(no output)"; } catch (e: any) { return ((e.stdout?.toString() || "") + (e.stderr?.toString() || "")).trim() || String(e); } }, }; const specs = [ { type: "function" as const, function: { name: "read_file", description: "Read a file in the workspace.", parameters: { type: "object", properties: { path: { type: "string" } }, required: ["path"] } } }, { type: "function" as const, function: { name: "write_file", description: "Create or overwrite a file in the workspace.", parameters: { type: "object", properties: { path: { type: "string" }, content: { type: "string" } }, required: ["path", "content"] } } }, { type: "function" as const, function: { name: "run_command", description: "Run a shell command in the workspace and return its output.", parameters: { type: "object", properties: { command: { type: "string" } }, required: ["command"] } } }, ]; const SYSTEM = "You are a coding agent. Use the tools to read, write, and run code in the " + "workspace until the task is done. Verify your work, then give a short summary."; const messages: any[] = [{ role: "system", content: SYSTEM }]; async function send(userMessage: string) { messages.push({ role: "user", content: userMessage }); // Keep going while the model wants to call tools. while (true) { const r = await client.chat.completions.create({ model: "anthropic/claude-sonnet-4-6", messages, tools: specs }); const msg = r.choices[0].message; messages.push(msg); if (!msg.tool_calls) return msg.content; for (const call of msg.tool_calls) { const args = JSON.parse(call.function.arguments); console.log(` · ${call.function.name}`); messages.push({ role: "tool", tool_call_id: call.id, content: TOOLS[call.function.name](args) }); } } } console.log(await send("Write fizzbuzz.js with a fizzbuzz(n) function, then run it with node for n=15.")); console.log(await send("Now add a test that fizzbuzz(5) ends with 'Buzz', and run it with node.")); ``` Run it: ```bash Python theme={null} pip install openai export OPPER_API_KEY="your-api-key" python app.py ``` ```bash TypeScript theme={null} npm install openai export OPPER_API_KEY="your-api-key" npx tsx app.ts ``` ## How it works * **The model drives the loop.** Each turn it either calls tools or returns a final answer. You run the tools, feed the results back, and loop *while it keeps calling tools*, no fixed step count. * **It recovers on its own.** Tool output (including errors) goes straight back to the model, so it adapts: a failed command, a syntax error, a missing file. You don't script the recovery. * **State persists.** `messages` holds the whole history, so a follow-up like "now add a test" builds on the code the agent already wrote. * **Three tools go a long way.** `read_file`, `write_file`, and `run_command` are enough to edit and test a project. Add more (search, git, HTTP) and the agent gets more capable. Swap `model` for any of the [700+ models](/capabilities/models) without touching the loop. `run_command` executes shell commands, so sandbox it before pointing it at anything real: a container, a throwaway directory, or an allowlist of commands. The example keeps everything inside a `workspace` folder. ## What's next The tool-use round trip this loop is built on. A batteries-included agent framework with tools, streaming, and multi-agent. Message history and multi-turn patterns. Score the agent's runs and catch regressions. # Claude Code Router: Run Claude Code on Any Model Source: https://docs.opper.ai/guides/claude-code-router Route Claude Code through Opper instead of a local proxy and run it on 700+ models — one API key, provider rates, model fallbacks, and EU-hosted routes. Claude Code talks to `api.anthropic.com` by default, but it ships with built-in gateway support: point `ANTHROPIC_BASE_URL` at Opper and the same harness runs on any of [700+ models](/capabilities/models) — Anthropic, OpenAI, Google, Kimi, DeepSeek, and more — with one API key, one bill at provider rates, configurable fallbacks, and per-session cost tracking. Unlike proxy tools such as [claude-code-router](https://github.com/musistudio/claude-code-router), there is nothing to run on your machine: no local server, no router config file to maintain. Claude Code speaks its native protocol directly to Opper's [Anthropic-compatible endpoint](/v3-api-reference/compatibility/create-message), and the routing happens server-side in the gateway. ## Prerequisites * Node.js ≥ 20.12 (`npx` works without a global install) * [Claude Code](https://claude.com/claude-code) installed * An Opper account — free to create; the login step below opens sign-up if you don't have one ## Quick start ```bash theme={null} npx @opperai/cli login # one-time OAuth sign-in in your browser npx @opperai/cli launch claude ``` `launch claude` sets `ANTHROPIC_BASE_URL` and `ANTHROPIC_AUTH_TOKEN` for the session and starts Claude Code — no config files, nothing else to run. Install globally with `npm i -g @opperai/cli` if you prefer `opper` on your PATH; verify your login anytime with `opper whoami`. If you were previously signed in to Claude Code with an Anthropic account, run `/logout` inside Claude Code once so cached credentials don't conflict with the gateway, then confirm with `/status` that the Opper base URL is active. ## Switch the model Pass any model from the [catalog](https://opper.ai/models) with `--model`, using full model IDs: ```bash theme={null} opper launch --model openai/gpt-5.4 claude opper launch --model alibaba:global/kimi-k2.7-code claude ``` Anything after the agent name is forwarded to Claude Code verbatim: ```bash theme={null} opper launch claude --resume ``` ## Manual configuration (without the CLI) Claude Code's own environment variables work directly against Opper's Anthropic-compatible endpoint. Get an API key (`opper config get default` prints the key for your default credential), then: ```bash theme={null} export ANTHROPIC_BASE_URL=https://api.opper.ai/v3/compat export ANTHROPIC_AUTH_TOKEN=your-api-key export ANTHROPIC_DEFAULT_OPUS_MODEL=anthropic/claude-opus-5 export ANTHROPIC_DEFAULT_SONNET_MODEL=anthropic/claude-sonnet-5 export ANTHROPIC_DEFAULT_HAIKU_MODEL=gemini/gemini-3.5-flash claude ``` The `ANTHROPIC_DEFAULT_*` variables map Claude Code's model aliases, and any full ID from the catalog works in each slot — so `/model sonnet` can resolve to a non-Anthropic model entirely: | Variable | Controls | | - | - | | `ANTHROPIC_DEFAULT_OPUS_MODEL` | The `opus` alias | | `ANTHROPIC_DEFAULT_SONNET_MODEL` | The `sonnet` alias (main model) | | `ANTHROPIC_DEFAULT_HAIKU_MODEL` | The `haiku` alias | | `CLAUDE_CODE_SUBAGENT_MODEL` | Model used for subagents | `ANTHROPIC_SMALL_FAST_MODEL` is deprecated in Claude Code; use `ANTHROPIC_DEFAULT_HAIKU_MODEL` for the background model. To route a single project through Opper while others keep using your Anthropic subscription, put the same keys in that project's `.claude/settings.local.json` instead of your shell profile: ```json theme={null} { "env": { "ANTHROPIC_BASE_URL": "https://api.opper.ai/v3/compat", "ANTHROPIC_AUTH_TOKEN": "your-api-key", "ANTHROPIC_DEFAULT_SONNET_MODEL": "anthropic/claude-sonnet-5" } } ``` ### Show gateway models in the /model picker Claude Code can populate its `/model` picker from the gateway's [model list](/v3-api-reference/compatibility/list-models): ```bash theme={null} export CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY=1 ``` Discovered models appear in the picker labeled "From gateway"; any catalog ID also works directly via `--model` or the env vars above. Alternatively, pin a single custom entry with `ANTHROPIC_CUSTOM_MODEL_OPTION` (plus `_NAME` and `_DESCRIPTION`). ## Claude Agent SDK and CI The same environment variables route the [Claude Agent SDK](https://docs.claude.com/en/api/agent-sdk/overview) through Opper. In CI — GitHub Actions, for example — set the base URL in the workflow and keep the key in a secret: ```yaml theme={null} env: ANTHROPIC_BASE_URL: https://api.opper.ai/v3/compat ANTHROPIC_AUTH_TOKEN: ${{ secrets.OPPER_API_KEY }} ``` ## Keep inference in the EU Opper's gateway is hosted exclusively in the EU — the only thing that leaves is the model call itself, and you control where that goes. Pick an EU-hosted model from the [catalog](https://opper.ai/models) (marked in the list, e.g. `mistral/mistral-large-2512`) and your coding sessions never leave the EU. See [Keep data in the EU](/overview/gateway#keep-data-in-the-eu) for how routing restrictions are enforced at the gateway. On a network that only allows egress to the gateway, also set `CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1` — it turns off Claude Code's nonessential background traffic to Anthropic and third-party services (version checks, telemetry, error reports), which would otherwise show up as blocked connections in egress monitoring. ## Coming from OpenRouter or claude-code-router? * **From OpenRouter** — same mechanism, different base URL: set `ANTHROPIC_BASE_URL=https://api.opper.ai/v3/compat` and pass your Opper key as `ANTHROPIC_AUTH_TOKEN`. You gain EU-hosted routing, inference at provider rates, and per-call [tracing](/control-plane/trace) when you turn it on (Control Plane). * **From claude-code-router** — the open-source proxy runs locally and needs its own config file and process. With Opper the router runs server-side: the two env vars above are the entire setup, and they work anywhere Claude Code runs, including CI. ## What changes compared to Anthropic direct * **Billing.** Routed sessions are billed as API usage through Opper at [provider rates](https://opper.ai/pricing). A Claude Pro or Max subscription does not apply while a gateway credential is active. * **Model IDs.** The `opus` / `sonnet` / `haiku` aliases resolve through the `ANTHROPIC_DEFAULT_*` mapping; for everything else use full IDs from the catalog. * **Feature parity.** Some Anthropic-endpoint conveniences ([prompt caching](/build/gateway/prompt-caching) among them) depend on the route serving your request. Anthropic documents the gateway mechanism but does not officially support non-Claude models in Claude Code, so expect polish to vary by model — frontier models and strong open-weight tool-callers work well in practice. * **Reliability.** Set a [dynamic route](/capabilities/routes/overview) (`dynamic/`) as the model in `--model` or an `ANTHROPIC_DEFAULT_*` variable to get a [fallback chain](/capabilities/models#fallbacks-and-backup-chains) (Control Plane): if the primary model fails or a provider has an outage, the call falls through to the next model in the route. ## Troubleshooting * **"Model not found"** — use the full model ID exactly as listed in the catalog; aliases only resolve through the `ANTHROPIC_DEFAULT_*` mapping. * **Auth errors** — run `opper whoami`; re-run `opper login` if the session expired. If Claude Code was previously signed in to an Anthropic account, `/logout` once and check `/status` shows the Opper base URL. * **Context length errors** — context windows differ per model; check the catalog and pick a model sized for your session. * **Checking spend** — every routed session is metered with per-call cost, latency, and token accounting under [Logs](/control-plane/logs) in your Opper dashboard. # Customer support bot Source: https://docs.opper.ai/guides/customer-service-bot Build a multi-turn support bot on the gateway with tool calling. The model looks up orders when it needs to. A support bot needs a natural back-and-forth with the user and the ability to look things up in your systems. Here's one built on the [gateway](/build/gateway/drop-in-sdks) with tool calling: it asks for an order ID and email, then calls a `get_order_status` tool to answer.
python app.py

You › Hi, where is my order?

Bot › I can check that. Please share your order ID and the email you used to place the order.

You › Order 123123, email [santa@clau.se](mailto:santa@clau.se)

Tool › get\_order\_status(order\_id="123123", email="[santa@clau.se](mailto:santa@clau.se)")
→ found · out for delivery · ETA 2024-11-08

Bot › Your order 123123 is out for delivery, arriving around 2024-11-08 to Snowy Mountain 123, Greenland. Items: Large wooden sled and Reindeer harness (x2).

You › Great, what did I actually order?

Bot › Your order 123123 contains a Large wooden sled and a Reindeer harness (x2).

The bot asks for the ID and email, looks the order up once it has them, and still knows the contents on the next turn because the history is kept. Here's the whole thing. ## The bot ```python app.py theme={null} import os, json from openai import OpenAI client = OpenAI(base_url="https://api.opper.ai/v3/compat", api_key=os.environ["OPPER_API_KEY"]) # A tiny order "database" ORDERS = { "123123": { "email": "santa@clau.se", "status": "out for delivery", "eta": "2024-11-08", "items": ["Large wooden sled", "Reindeer harness (x2)"], "address": "Snowy Mountain 123, Greenland", } } def get_order_status(order_id: str, email: str) -> dict: order = ORDERS.get(order_id) if not order or order["email"].lower() != email.lower(): return {"found": False, "reason": "No order matches that ID and email."} return {"found": True, **{k: order[k] for k in ("status", "eta", "items", "address")}} tools = [{ "type": "function", "function": { "name": "get_order_status", "description": "Look up an order's status and contents by order ID and the email used to place it.", "parameters": { "type": "object", "properties": { "order_id": {"type": "string"}, "email": {"type": "string"}, }, "required": ["order_id", "email"], }, }, }] system = ( "You are a friendly customer support agent for a sleigh supply store. " "Use the get_order_status tool to answer questions about orders. " "Ask for the order ID and email if you need them. Keep replies short." ) def reply(messages): # Loop until the model answers in text instead of calling a tool. while True: r = client.chat.completions.create(model="openai/gpt-5-mini", messages=messages, tools=tools) msg = r.choices[0].message messages.append(msg) if not msg.tool_calls: return msg.content for call in msg.tool_calls: args = json.loads(call.function.arguments) result = get_order_status(**args) messages.append({"role": "tool", "tool_call_id": call.id, "content": json.dumps(result)}) messages = [{"role": "system", "content": system}] while True: user = input("User: ") if user.lower() in {"quit", "exit"}: break messages.append({"role": "user", "content": user}) print("Assistant:", reply(messages)) ``` ```typescript app.ts theme={null} import OpenAI from "openai"; import readline from "node:readline/promises"; const client = new OpenAI({ baseURL: "https://api.opper.ai/v3/compat", apiKey: process.env.OPPER_API_KEY }); // A tiny order "database" const ORDERS: Record = { "123123": { email: "santa@clau.se", status: "out for delivery", eta: "2024-11-08", items: ["Large wooden sled", "Reindeer harness (x2)"], address: "Snowy Mountain 123, Greenland", }, }; function getOrderStatus(orderId: string, email: string) { const o = ORDERS[orderId]; if (!o || o.email.toLowerCase() !== email.toLowerCase()) return { found: false, reason: "No order matches that ID and email." }; const { status, eta, items, address } = o; return { found: true, status, eta, items, address }; } const tools = [{ type: "function" as const, function: { name: "get_order_status", description: "Look up an order's status and contents by order ID and the email used to place it.", parameters: { type: "object", properties: { order_id: { type: "string" }, email: { type: "string" } }, required: ["order_id", "email"], }, }, }]; const system = "You are a friendly customer support agent for a sleigh supply store. " + "Use the get_order_status tool to answer questions about orders. " + "Ask for the order ID and email if you need them. Keep replies short."; async function reply(messages: any[]) { // Loop until the model answers in text instead of calling a tool. while (true) { const r = await client.chat.completions.create({ model: "openai/gpt-5-mini", messages, tools }); const msg = r.choices[0].message; messages.push(msg); if (!msg.tool_calls) return msg.content; for (const call of msg.tool_calls) { const args = JSON.parse(call.function.arguments); const result = getOrderStatus(args.order_id, args.email); messages.push({ role: "tool", tool_call_id: call.id, content: JSON.stringify(result) }); } } } const rl = readline.createInterface({ input: process.stdin, output: process.stdout }); const messages: any[] = [{ role: "system", content: system }]; while (true) { const user = await rl.question("User: "); if (["quit", "exit"].includes(user.toLowerCase())) break; messages.push({ role: "user", content: user }); console.log("Assistant:", await reply(messages)); } ``` Run it: ```bash Python theme={null} pip install openai export OPPER_API_KEY="your-api-key" python app.py ``` ```bash TypeScript theme={null} npm install openai export OPPER_API_KEY="your-api-key" npx tsx app.ts ``` ## How it works * The `tools` array describes `get_order_status` to the model. It calls the tool only when it needs order data, and asks for the ID and email first when they're missing. * The `reply` loop runs the model, executes any tool call, feeds the result back, and repeats until the model returns plain text. That's the standard [tool round trip](/build/gateway/tools). * The whole conversation, including the system prompt and tool results, lives in `messages`, so the bot remembers the order across turns. See [Conversations](/build/gateway/conversations). * Swap `model` for any of the [700+ models](/capabilities/models) without touching the rest of the code. With tracing turned on (Control Plane), every call is traced in the platform. Add a [score check](/control-plane/rules/checks#score-checks) on this bot to score replies and catch regressions. ## What's next The full tool-use round trip. Multi-turn chat and message history. Block or redact sensitive content before it reaches the model. Score every reply and watch quality over time. # Natural language to SQL Source: https://docs.opper.ai/guides/language-to-sql Turn plain-English questions into SQL you can run, using the gateway and a typed schema. This is a clean fit for [structured output](/build/gateway/structured-output): you send the database schema and a question, and you get back a JSON object with the query. Because the output is schema-constrained, you can pull the `sql` field straight out and execute it. The example uses the OpenAI SDK pointed at the gateway, so the same code runs against any model. The example uses an in-memory SQLite database so it runs as-is.
python app.py

You › Who are the two highest-paid people in Engineering?

SQL › SELECT name, salary FROM employees WHERE department = 'Engineering' ORDER BY salary DESC LIMIT 2;

→ \[('Dan', 151000), ('Alice', 145000)]

## The assistant ```python app.py theme={null} import os, json, sqlite3 from openai import OpenAI client = OpenAI( base_url="https://api.opper.ai/v3/compat", api_key=os.environ["OPPER_API_KEY"], ) # A sample database db = sqlite3.connect(":memory:") db.executescript(""" CREATE TABLE employees (id INTEGER, name TEXT, department TEXT, salary INTEGER, hire_date TEXT); INSERT INTO employees VALUES (1,'Alice','Engineering',145000,'2021-03-01'), (2,'Bjorn','Engineering',132000,'2022-07-15'), (3,'Chloe','Sales',98000,'2020-01-10'), (4,'Dan','Engineering',151000,'2019-11-20'), (5,'Eva','Sales',105000,'2023-02-05'); """) def schema(con): rows = con.execute("SELECT sql FROM sqlite_master WHERE type='table'").fetchall() return "\n".join(r[0] for r in rows) QUERY_SCHEMA = { "type": "object", "properties": { "reasoning": {"type": "string", "description": "Brief reasoning about how to answer."}, "sql": {"type": "string", "description": "A single valid SQLite query."}, }, "required": ["reasoning", "sql"], } def to_sql(question, db_schema): r = client.chat.completions.create( model="openai/gpt-5-mini", messages=[ {"role": "system", "content": "Given a SQLite schema and a question, write one SQLite query that answers it. Return only the query in `sql`."}, {"role": "user", "content": f"Schema:\n{db_schema}\n\nQuestion: {question}"}, ], response_format={"type": "json_schema", "json_schema": {"name": "query", "schema": QUERY_SCHEMA}}, ) return json.loads(r.choices[0].message.content) question = "Who are the two highest-paid people in Engineering? Show name and salary." out = to_sql(question, schema(db)) print("SQL:", out["sql"]) print("Rows:", db.execute(out["sql"]).fetchall()) ``` ```typescript app.ts theme={null} import Database from "better-sqlite3"; import OpenAI from "openai"; const client = new OpenAI({ baseURL: "https://api.opper.ai/v3/compat", apiKey: process.env.OPPER_API_KEY!, }); // A sample database const db = new Database(":memory:"); db.exec(` CREATE TABLE employees (id INTEGER, name TEXT, department TEXT, salary INTEGER, hire_date TEXT); INSERT INTO employees VALUES (1,'Alice','Engineering',145000,'2021-03-01'), (2,'Bjorn','Engineering',132000,'2022-07-15'), (3,'Chloe','Sales',98000,'2020-01-10'), (4,'Dan','Engineering',151000,'2019-11-20'), (5,'Eva','Sales',105000,'2023-02-05'); `); const schema = () => db.prepare("SELECT sql FROM sqlite_master WHERE type='table'").all().map((r: any) => r.sql).join("\n"); const question = "Who are the two highest-paid people in Engineering? Show name and salary."; const r = await client.chat.completions.create({ model: "openai/gpt-5-mini", messages: [ { role: "system", content: "Given a SQLite schema and a question, write one SQLite query that answers it. Return only the query in `sql`." }, { role: "user", content: `Schema:\n${schema()}\n\nQuestion: ${question}` }, ], response_format: { type: "json_schema", json_schema: { name: "query", schema: { type: "object", properties: { reasoning: { type: "string" }, sql: { type: "string" } }, required: ["reasoning", "sql"], }, }, }, }); const out = JSON.parse(r.choices[0].message.content!); console.log("SQL:", out.sql); console.log("Rows:", db.prepare(out.sql).all()); ``` Run it: ```bash Python theme={null} pip install openai export OPPER_API_KEY="your-api-key" python app.py ``` ```bash TypeScript theme={null} npm install openai better-sqlite3 export OPPER_API_KEY="your-api-key" npx tsx app.ts ``` ## How it works * The schema forces the model to return a `sql` string (plus its reasoning). With [structured output](/build/gateway/structured-output) you parse the response once and run `out["sql"]` directly, no regex. * Passing the live `schema(db)` means the model writes queries against your real tables, not guessed ones. * The `reasoning` field is optional but useful: it gives you a plain-English explanation of why the query looks the way it does, which shows up on the call's trace once you turn [tracing](/control-plane/trace) on. ## Make it more robust * **Retry on error.** Catch SQLite exceptions, send the error message back as input, and ask for a corrected query. * **Read-only safety.** Run the generated SQL against a read-only connection so a bad query can't modify data. * **Score it.** Add a [score check](/control-plane/rules/checks#score-checks) that verifies the query ran and returned rows. ## What's next JSON Schema, enums, and field descriptions. Set a default model per project without touching code. Stream the query as it generates. Score generated SQL automatically. # Research assistant Source: https://docs.opper.ai/guides/research-assistant Answer questions with live web search, then return a grounded answer with citations using structured output. This assistant searches the live web with Opper's built-in web tool, then uses [structured output](/build/gateway/structured-output) to write a grounded answer that cites its sources. Two steps: search, then one structured call through the gateway.
python app.py

You › What is Mistral's largest open-weight model?

Tool › web\_search("Mistral largest open-weight model")
→ 5 results from the web

Bot › Mistral's largest open-weight model is Mistral Medium 3.5, a 128B-parameter model for reasoning, coding, and instruction-following.

→ sources: \["mindstudio.ai/blog/what-is-mistral-medium-3-5..."]

Search results go in, a typed answer with sources comes out. ## The assistant ```python app.py theme={null} import os, json, requests from openai import OpenAI KEY = os.environ["OPPER_API_KEY"] client = OpenAI(base_url="https://api.opper.ai/v3/compat", api_key=KEY) def web_search(query: str) -> list[dict]: r = requests.post( "https://api.opper.ai/v3/tools/web/search", headers={"Authorization": f"Bearer {KEY}"}, json={"query": query}, ) r.raise_for_status() return r.json()["results"] ANSWER_SCHEMA = { "type": "object", "properties": { "answer": {"type": "string", "description": "A concise answer grounded in the search results."}, "sources": {"type": "array", "items": {"type": "string"}, "description": "URLs of the results you actually used."}, }, "required": ["answer", "sources"], } def research(question: str): results = web_search(question) r = client.chat.completions.create( model="openai/gpt-5-mini", messages=[ {"role": "system", "content": "Answer the question using only the provided search results. Cite the URLs you used in sources."}, {"role": "user", "content": f"Question: {question}\n\nResults:\n{json.dumps(results)}"}, ], response_format={"type": "json_schema", "json_schema": {"name": "answer", "schema": ANSWER_SCHEMA}}, ) return json.loads(r.choices[0].message.content) if __name__ == "__main__": data = research("What is Mistral's largest open-weight model?") print(data["answer"]) print("Sources:", data["sources"]) ``` ```typescript app.ts theme={null} import OpenAI from "openai"; const KEY = process.env.OPPER_API_KEY!; const client = new OpenAI({ baseURL: "https://api.opper.ai/v3/compat", apiKey: KEY }); async function webSearch(query: string) { const r = await fetch("https://api.opper.ai/v3/tools/web/search", { method: "POST", headers: { Authorization: `Bearer ${KEY}`, "Content-Type": "application/json" }, body: JSON.stringify({ query }), }); if (!r.ok) throw new Error(`search failed: ${r.status}`); return (await r.json()).results; } async function research(question: string) { const results = await webSearch(question); const r = await client.chat.completions.create({ model: "openai/gpt-5-mini", messages: [ { role: "system", content: "Answer the question using only the provided search results. Cite the URLs you used in sources." }, { role: "user", content: `Question: ${question}\n\nResults:\n${JSON.stringify(results)}` }, ], response_format: { type: "json_schema", json_schema: { name: "answer", schema: { type: "object", properties: { answer: { type: "string" }, sources: { type: "array", items: { type: "string" } } }, required: ["answer", "sources"], }, }, }, }); return JSON.parse(r.choices[0].message.content!); } const data = await research("What is Mistral's largest open-weight model?"); console.log(data.answer); console.log("Sources:", data.sources); ``` Run it: ```bash Python theme={null} pip install openai requests export OPPER_API_KEY="your-api-key" python app.py ``` ```bash TypeScript theme={null} npm install openai export OPPER_API_KEY="your-api-key" npx tsx app.ts ``` ## How it works * **Search.** `web_search` calls Opper's hosted web tool (`POST /v3/tools/web/search`) and returns a list of `{title, url, snippet}` results. No separate search API key to manage. * **Answer.** The [structured output](/build/gateway/structured-output) call gets the question plus those results and returns a typed answer. Because the output is schema-constrained, `answer` and `sources` come back clean after a single parse. * **Grounding.** The model only sees the results you pass in, so it answers from the search rather than from memory. If a fact isn't in the results, it can say so. Pass more or fewer results to trade cost for coverage, and add a [score check](/control-plane/rules/checks#score-checks) that verifies every answer actually cites a source. ## What's next The schema-constrained call behind the answer step. The portable web tool, across every model. Let the model call your code directly. Score answers for grounding and citation quality. # n8n Source: https://docs.opper.ai/integrations/automation/n8n Install the Opper community node and use any Opper model in your n8n AI workflows [n8n](https://n8n.io) is a workflow automation platform with AI nodes built in. Opper ships a [verified community node](https://docs.n8n.io/integrations/community-nodes/installation/verified-install/), so Opper models appear as a native chat-model sub-node rather than a generic HTTP request. **Requirements:** n8n 1.94.0 or later, Node.js 20.15+. ## Setup Open the nodes panel on the canvas (**+**, or press **n**) and search for **Opper**. Verified community nodes appear under **More from the community** at the bottom of the panel. Select **Opper Chat Model**, then select **Install**. Only instance owners and admins can install; once installed, every member can use it in their workflows. The npm package behind it is `@opperai/n8n-nodes-opper`, if you would rather install it from **Settings → Community Nodes**. Create an **Opper API** credential: | Field | Value | | - | - | | **API Key** | Your Opper API key (starts with `op-`) | | **Base URL** | `https://api.opper.ai/v3/compat` (default) | The credential test validates connectivity through Opper's OpenAI-compatible `GET /models` endpoint. Add an **AI Agent** node (or **Basic LLM Chain**, or anything else that accepts a language-model connection) and attach **Opper Chat Model** as its model sub-node. An n8n canvas with Opper Chat Model connected to the Chat Model input of an AI Agent node, and the nodes panel listing Opper Chat Model alongside the built-in providers ## Choosing a model The Opper Chat Model node lists the catalog fetched with your credential, filtered by your [model access rules](/control-plane/rules/model-access). Agent nodes need tool calling — pick a model the [catalog](/capabilities/models) marks with the `tools` capability. The Opper Chat Model parameters panel with an Opper credential selected and the Model dropdown open on EU-hosted evroc models Switching models is a dropdown change rather than a rebuild, so the same workflow can run on Claude, GPT, Gemini, Mistral or an EU-sovereign open-weight model. Picking a model such as `evroc/zai-org/GLM-5.2` keeps inference inside the EU without touching anything else in the workflow. Under **Options** you can set temperature, top-p, frequency and presence penalties, max tokens, timeout, retries, and a JSON response format. Every call is recorded with its token usage and cost, and traced in full once you turn tracing on in the [Control Plane](/control-plane/overview). ## Attributing usage per workflow n8n instances usually run many workflows against one credential. Tag the traffic so you can tell them apart in Opper analytics — see [Tags and usage attribution](/build/gateway/usage-attribution). ## Troubleshooting n8n loads community nodes at startup. Restart the instance after installing. Verified community nodes can be hidden per instance. On n8n Cloud, an instance owner toggles them in the Cloud Admin Panel; self-hosted instances control it with environment variables. Note also that only owners and admins can install a node — members can only use one that is already installed. It calls `GET /models`. A failure usually means the key is wrong, or the base URL has an extra path segment — it should be exactly `https://api.opper.ai/v3/compat`. ## Related Split usage per workflow or tenant. Every app that runs on Opper. # LibreChat Source: https://docs.opper.ai/integrations/chat/librechat Add Opper as a custom endpoint in LibreChat, the self-hosted open-source AI chat platform [LibreChat](https://www.librechat.ai) is an open-source AI chat platform (MIT, [source](https://github.com/danny-avila/LibreChat)) — a self-hosted, multi-user ChatGPT-style interface with conversation branching, presets, and file uploads. It takes any OpenAI-compatible provider as a custom endpoint in `librechat.yaml`. ## Setup Add Opper under `endpoints.custom`: ```yaml theme={null} endpoints: custom: - name: 'Opper' apiKey: '${OPPER_API_KEY}' baseURL: 'https://api.opper.ai/v3/compat' models: default: ['anthropic/claude-sonnet-4-6', 'openai/gpt-5.5'] fetch: true titleConvo: true modelDisplayLabel: 'Opper' ``` Then set the key in your `.env` and restart: ```bash theme={null} OPPER_API_KEY=your-opper-api-key ``` `fetch: true` makes LibreChat pull the catalog from `GET /v3/compat/models` at startup, so the picker fills itself. Set it to `false` and rely on `models.default` if you'd rather pin an explicit list. Use the `${VAR}` env-var syntax rather than pasting the key into `librechat.yaml` — the file is usually committed alongside your deployment. ## Choosing a model With `fetch: true`, LibreChat lists everything your key can reach, filtered by your [model access rules](/control-plane/rules/model-access). The catalog is large, so a curated `models.default` list is often the nicer experience for end users. ## Multi-user usage LibreChat is typically multi-tenant. Forward per-user tags with the `X-Opper-Tags` header to break usage down by user or workspace in Opper analytics — see [Tags and usage attribution](/build/gateway/usage-attribution). ## Troubleshooting LibreChat reads `librechat.yaml` at startup only. Restart the container or process after editing it. Set `fetch: false` and list the models you want in `models.default`. The `${OPPER_API_KEY}` reference is resolved from LibreChat's environment, not your shell. Confirm it's in the `.env` the process actually loads. ## Related Split usage per user or workspace. The other self-hosted chat workspace. # Open WebUI Source: https://docs.opper.ai/integrations/chat/open-webui Add Opper as an OpenAI-compatible connection in Open WebUI [Open WebUI](https://openwebui.com) is a self-hosted AI workspace — chat, RAG over your documents, and tools, for a whole team. It is built around standard protocols and connects to any server that implements the OpenAI Chat Completions API. ## Setup In Open WebUI, go to **Admin Settings → Connections → OpenAI**, then click **Add Connection**. | Field | Value | | - | - | | **URL** | `https://api.opper.ai/v3/compat` | | **API Key** | Your Opper API key | Opper's catalog is large. Under **Model IDs (Filter)**, add the IDs you want your users to see — type each one and click **+**. Save. Or set it at startup with environment variables: ```bash theme={null} OPENAI_API_BASE_URL=https://api.opper.ai/v3/compat OPENAI_API_KEY=your-opper-api-key ``` Running Open WebUI in Docker? The gateway is a public endpoint, so no `host.docker.internal` rewriting is needed — that caveat only applies to model servers on your host machine. ## Choosing a model Open WebUI verifies a connection by calling `/models`, which Opper implements, so auto-detection works and the picker fills itself. Because the catalog runs to hundreds of entries, the **Model IDs (Filter)** allowlist is strongly recommended — the same advice Open WebUI gives for any large gateway. ## Multi-user usage Open WebUI is multi-user by design. Forward per-user tags with the `X-Opper-Tags` header to break usage down in Opper analytics — see [Tags and usage attribution](/build/gateway/usage-attribution). ## Troubleshooting Verification calls `/models` with a bearer token. If it errors while chat completions succeed, add your model IDs to the **Model IDs (Filter)** allowlist manually and carry on. Expected without a filter. Add an allowlist under **Model IDs (Filter)**, or narrow the key's project with [model access rules](/control-plane/rules/model-access) so the filter happens server-side. Check the key is an Opper key (they start with `op-`) and that it's on the OpenAI connection, not Ollama. ## Related Filter the catalog server-side, per project. The other self-hosted chat platform. # Aider Source: https://docs.opper.ai/integrations/coding-agents/aider Point Aider at Opper to pair-program in your terminal on any model in the catalog [Aider](https://aider.chat) is an open-source AI pair programming tool for the terminal (Apache-2.0, [source](https://github.com/Aider-AI/aider)). It edits code in your local git repo and commits as it goes. Aider treats any OpenAI-compatible endpoint as an OpenAI provider, so two environment variables point it at Opper. ## Setup ```bash theme={null} export OPENAI_API_BASE=https://api.opper.ai/v3/compat export OPENAI_API_KEY=your-opper-api-key aider --model openai/anthropic/claude-sonnet-4-6 ``` On Windows, use `setx OPENAI_API_BASE ...` and `setx OPENAI_API_KEY ...` instead. The `openai/` prefix is Aider's own routing marker — it tells Aider to use its OpenAI-compatible client. Everything after it is the Opper model ID, which carries its own provider prefix. So `anthropic/claude-sonnet-4-6` on Opper becomes `openai/anthropic/claude-sonnet-4-6` on the Aider command line. ## Choosing a model Any model ID from the [catalog](/capabilities/models), prefixed with `openai/`. To make it stick, put it in `.aider.conf.yml` at the root of your repo: ```yaml theme={null} model: openai/anthropic/claude-sonnet-4-6 ``` ## Troubleshooting Aider needs the `openai/` prefix to route through its OpenAI-compatible client. Without it, Aider tries to resolve the name against its built-in model list and fails. `OPENAI_API_KEY` must hold your **Opper** key, not an OpenAI one. Opper keys start with `op-`. Aider keeps a static metadata map for models it knows. The warning is cosmetic and the model still works; set `--no-show-model-warnings` to silence it. ## Related Browse the catalog for model IDs and capabilities. Every app that runs on Opper. # Claude Code Source: https://docs.opper.ai/integrations/coding-agents/claude-code Run Anthropic's terminal coding agent on any model in the Opper catalog [Claude Code](https://claude.com/product/claude-code) is Anthropic's agentic coding tool for the terminal. It reads your whole repository, plans and applies changes across files, runs commands and tests, and iterates on failures. It ships with built-in gateway support, so pointing `ANTHROPIC_BASE_URL` at Opper runs the same harness on any model in the catalog — not just Claude. ## Setup One command, no config files: ```bash theme={null} npx @opperai/cli launch claude ``` This sets `ANTHROPIC_BASE_URL` and `ANTHROPIC_AUTH_TOKEN` for the session and starts Claude Code. Pin a model with `--model `. To wire it by hand instead: ```bash theme={null} export ANTHROPIC_BASE_URL=https://api.opper.ai/v3/compat export ANTHROPIC_AUTH_TOKEN=your-opper-api-key claude ``` Claude Code appends `/v1/messages` itself and talks to Opper's Anthropic-compatible surface. Everything else about the agent is unchanged. ## Full guide Claude Code has more configuration surface than any other integration here — model alias mapping, the `/model` picker, per-project settings, CI usage, and migrating from OpenRouter. All of it lives in the dedicated guide: Manual setup, alias mapping, per-project config, CI, and troubleshooting. ## Related The same agent inside Anthropic's desktop app. Every app that runs on Opper. # Codex Source: https://docs.opper.ai/integrations/coding-agents/codex Run OpenAI's Codex CLI on any model in the Opper catalog [Codex](https://openai.com/codex/) is OpenAI's open-source coding agent for the terminal (Apache-2.0, [source](https://github.com/openai/codex)). It works from natural-language instructions to edit code across your project, run commands, and complete tasks end to end. Codex reads its model providers from a config file, so adding Opper as one provider lets you run it on Claude, Gemini, DeepSeek, or any other model in the catalog. ## Setup One command, no config files: ```bash theme={null} npx @opperai/cli launch codex --model anthropic/claude-opus-4-8 ``` This starts Codex with Opper passed as its provider through `-c` flags, for that run only, so `~/.codex/config.toml` is left unchanged. `--model` is optional. To configure it by hand, add the provider to `~/.codex/config.toml`: ```toml theme={null} model_provider = "opper" web_search = "disabled" [model_providers.opper] name = "Opper" base_url = "https://api.opper.ai/v3/compat" env_key = "OPPER_API_KEY" wire_api = "responses" ``` `wire_api = "responses"` sends Codex's requests to Opper's Responses endpoint. `web_search = "disabled"` turns off Codex's default cached web search, which the gateway can't serve across providers. The CLI passes both on every run. Then export your key and start Codex: ```bash theme={null} export OPPER_API_KEY=your-opper-api-key codex --model anthropic/claude-opus-4-8 ``` ## Choosing a model Pass any Opper model ID with `--model`, or set a default in `config.toml`: ```toml theme={null} model = "anthropic/claude-sonnet-4-6" ``` Codex needs tool calling and streaming — pick a model the catalog marks with the `tools` capability. Browse IDs in the [model catalog](/capabilities/models). ## Troubleshooting Check that `OPPER_API_KEY` is exported in the shell running Codex, and that `env_key` in `config.toml` names that same variable. Opper keys start with `op-`. Copy the model ID exactly from the [catalog](/capabilities/models) — it includes the provider prefix, as in `anthropic/claude-sonnet-4-6`. If the error says `web_search is not supported`, Codex is requesting its default cached web search, which the gateway can't serve for those models. Add `web_search = "disabled"` at the top level of `config.toml`. `model_provider = "opper"` and `web_search = "disabled"` have to sit at the top level of `config.toml`, not inside the `[model_providers.opper]` table. TOML tables capture every key that follows them. ## Related Every Codex call, with model, latency, tokens, and cost. Every app that runs on Opper. # DeepSeek Harness Source: https://docs.opper.ai/integrations/coding-agents/deepseek-harness Add Opper as a custom provider in DeepSeek's open-source agent harness [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) (`dsh`) is DeepSeek's open-source agent harness, released under MIT and built so that everything is a plugin — the model adapter, the tool registry, the sandbox, and the agent loop itself. It runs as a local web UI or headless from the terminal, and accepts any OpenAI-compatible endpoint as a custom provider. ## Setup ### Web UI ```bash theme={null} npx @deepseek-ai/dsh web ``` Open the UI (served at `http://127.0.0.1:3080` by default), go to **Settings → Models → Add a custom provider**, and fill in: | Field | Value | | - | - | | **Provider ID** | `opper` (lowercase; permanent once saved) | | **Base URL** | `https://api.opper.ai/v3/compat` | | **API protocol** | OpenAI (chat completions) | | **API key** | Your Opper API key | Under **Model catalog**, click **Fetch available models** — `dsh` queries Opper's OpenAI-compatible `GET /models` endpoint and lists the full catalog. Select the models you want, save, then pick one in the model picker to make it the default for new sessions. ### Config file, and headless mode The Models page writes to `$DSH_HOME/settings.yaml` (default `~/.dsh`). To configure the same thing by hand — for example for one-shot headless jobs: ```yaml theme={null} # $DSH_HOME/settings.yaml agent-default-model: provider: opper model: anthropic/claude-sonnet-4-6 llm-pi-ai: providers: opper: apiKeyEnv: OPPER_API_KEY api: openai-completions baseURL: https://api.opper.ai/v3/compat models: - id: anthropic/claude-sonnet-4-6 - id: tensorx/deepseek/deepseek-v4-flash ``` ```bash theme={null} export OPPER_API_KEY=your-opper-api-key npx @deepseek-ai/dsh --profile headless "Summarize this repository" ``` Models entered by hand are treated as text-only; for a vision model, add `input: [text, image]` to its entry. The API key itself never lands in `settings.yaml` — the UI stores it in `$DSH_HOME/.credentials.yaml`, and `apiKeyEnv` reads it from the environment instead. ## Troubleshooting Check the base URL has no trailing path — `dsh` appends `/models` itself. It should be exactly `https://api.opper.ai/v3/compat`. Hand-written model entries default to text-only. Add `input: [text, image]` under that model's entry in `settings.yaml`. The provider ID is permanent once saved. Delete the provider and re-add it with the correct ID. ## Related Browse the catalog for model IDs and capabilities. Every app that runs on Opper. # Droid Source: https://docs.opper.ai/integrations/coding-agents/droid Add Opper as a custom model provider in Factory's coding agent [Droid](https://factory.ai) is Factory's coding agent for the terminal and IDEs. It supports bring-your-own-key custom models, so any OpenAI-compatible endpoint can serve it — including Opper. ## Setup Add a `customModels` entry to `~/.factory/settings.json`: ```json theme={null} { "customModels": [ { "model": "anthropic/claude-sonnet-4-6", "displayName": "Claude Sonnet 4.6 (Opper)", "baseUrl": "https://api.opper.ai/v3/compat", "apiKey": "${OPPER_API_KEY}", "provider": "generic-chat-completion-api", "maxOutputTokens": 16384 } ] } ``` ```bash theme={null} export OPPER_API_KEY=your-opper-api-key droid ``` Add one entry per model you want in the picker. `provider` must be `generic-chat-completion-api` for an OpenAI-compatible endpoint. Factory also reads a legacy snake\_case format (`custom_models`, `base_url`) from `~/.factory/config.json`. Where both exist, `settings.json` wins — use it for new setups. ## Choosing a model Droid lists whatever you put in `customModels`; there is no catalog fetch. Pull the IDs you want from the [model catalog](/capabilities/models) and add an entry each. ## Troubleshooting Check you edited `~/.factory/settings.json` and not the legacy `config.json`, and that `customModels` is camelCase. `${OPPER_API_KEY}` is expanded from the environment, so the variable has to be exported in the shell that starts Droid. Opper keys start with `op-`. Raise `maxOutputTokens` for that entry. The value above is a conservative default, not the model's limit. ## Related Browse the catalog for model IDs and capabilities. Every app that runs on Opper. # GitHub Copilot CLI Source: https://docs.opper.ai/integrations/coding-agents/github-copilot-cli Use Copilot CLI with your Opper sign-in, allowed models, and personal allowance Run [GitHub Copilot CLI](https://github.com/github/copilot-cli) through Opper. Sign in with your Opper account, choose from the models your organization allows, and track your usage from the terminal. ## Start Copilot Requires **Node.js 22 or newer** and macOS or Linux. Use Opper CLI **0.1.41 or newer**. ```bash theme={null} npm install -g @opperai/cli@latest opper launch copilot ``` Opper opens browser sign-in when needed and launches Copilot with your allowed models. Use your organization's SSO when configured. An existing valid sign-in is reused. You can also run without installing the Opper CLI globally: ```bash theme={null} npx @opperai/cli@latest launch copilot ``` ## Inside Copilot | Command | What it does | | - | - | | `/model` | Choose an allowed model, pool, or dynamic route. | | `/opper-usage` | See your allowance, usage, reset date, and credential expiry when supplied by the server. | | `/opper-login` | Approve renewal in your browser and continue the same session with the same account and organization. | | `/footer` | Toggle the **Custom** Opper allowance display and native context-window display. | | `/context` | Inspect the conversation's context usage. | If your credential expires during a request, run `/opper-login`, complete browser approval, and retry your message. Failed requests are not replayed automatically. Opper warns when a known credential expiry is within 24 hours. The Opper slash commands use Copilot's experimental extension support, enabled by the launcher. ## Choose what to list The launcher refreshes your allowed catalog each time and lists tool-capable models with supported context limits. To show only pools and routes: ```bash theme={null} opper launch copilot --kinds pools,routes ``` Use `--kinds models`, `--kinds pools`, `--kinds routes`, or `--kinds all`. Your last model and type filter are remembered. ## Optional project attribution New personal agent credentials work at organization scope. Your personal or role allowance applies across your personal keys in that organization. To attribute a session to a project and apply its policies: ```bash theme={null} opper projects list opper --project-uuid launch copilot ``` Existing project-bound credentials retain their binding until renewed. Organization billing details are shown only when you have permission. The launcher uses an isolated Copilot profile. Existing Copilot settings are untouched; footer preferences apply to the current launch. Resuming a previous launch with `--resume` or `--continue` is not supported yet. This flow routes inference through Opper; GitHub-connected features are not included. ## Related Sign in with Opper from the editor. More commands for working with Opper. # Goose Source: https://docs.opper.ai/integrations/coding-agents/goose Configure Goose to use Opper as its OpenAI-compatible provider [Goose](https://goose-docs.ai) is an open agent from the Agentic AI Foundation (Apache-2.0, [source](https://github.com/aaif-goose/goose)). It runs as a desktop app or from the terminal, uses your local toolchain, and extends through MCP. Goose's OpenAI provider takes a custom host, so it points at Opper without needing a dedicated provider. ## Setup Run the configuration wizard: ```bash theme={null} goose configure ``` Then choose **Configure Providers → OpenAI**, and supply: | Prompt | Value | | - | - | | **API key** | Your Opper API key | | **Host** | `https://api.opper.ai` | | **Base path** | `v3/compat/chat/completions` | Goose splits the endpoint into a host and a path, unlike most apps here. Putting the full `https://api.opper.ai/v3/compat` in the host field and leaving the default base path produces a 404 — the request would go to `https://api.opper.ai/v3/compat/v1/chat/completions`. The same thing set by environment variable, or in Goose's `config.yaml`: ```yaml theme={null} OPENAI_API_KEY: your-opper-api-key OPENAI_HOST: https://api.opper.ai OPENAI_BASE_PATH: v3/compat/chat/completions ``` ## Choosing a model Goose asks for a model during `goose configure`. Use a full model ID from the [catalog](/capabilities/models), such as `anthropic/claude-sonnet-4-6`. ## Troubleshooting Almost always `OPENAI_BASE_PATH`. It has to be `v3/compat/chat/completions` with the host set to the bare `https://api.opper.ai` — no scheme or path duplicated across the two fields. `OPENAI_API_KEY` must hold your Opper key. Opper keys start with `op-`. Only if you actually run a LiteLLM proxy in front of Opper. Don't configure both the OpenAI provider and the LiteLLM provider against the same endpoint — pick one. See [LiteLLM](/integrations/frameworks/litellm) for the proxy setup. ## Related Browse the catalog for model IDs and capabilities. Every app that runs on Opper. # OpenCode Source: https://docs.opper.ai/integrations/coding-agents/opencode Run SST's open-source terminal coding agent on any model in the Opper catalog [OpenCode](https://opencode.ai) is SST's open-source terminal coding agent (MIT, [source](https://github.com/anomalyco/opencode)). It reads a provider block from its config file, so Opper slots in as one more provider. ## Setup There are three ways in, and they differ in one thing: where the model list comes from. **Just a key.** OpenCode ships with the [models.dev](https://models.dev) registry, which already lists Opper, so there is nothing to configure: ```bash theme={null} export OPPER_API_KEY=your-opper-api-key opencode ``` Opper appears in `/model` straight away and no config file is written. That registry is one global list, identical for everyone, so it cannot carry anything specific to your key. Your [pools](/capabilities/models), your deployed [dynamic routes](/capabilities/routes/overview) and your project's [model access rules](/control-plane/rules/model-access) are all invisible to it. The two CLI commands below fetch the catalog scoped to your key instead, and OpenCode merges what they write with the registry rather than replacing it, so you keep both lists. **Signed in through the CLI**, which suits a single workstation: ```bash theme={null} npx @opperai/cli login npx @opperai/cli launch opencode ``` The key is stored by the CLI and passed to OpenCode for you, and the model list is refreshed on every launch. **With an environment variable** instead, which suits rollouts, CI and shared machines: ```bash theme={null} export OPPER_API_KEY=your-opper-api-key npx @opperai/cli editors opencode opencode ``` `editors opencode` writes the config without launching anything, and refreshes the list whenever you re-run it. The key itself is never written to disk: the config carries `{env:OPPER_API_KEY}`, which OpenCode resolves from the environment each run, so keep it exported. | | models.dev | `launch` / `editors` | | - | - | - | | Setup | export a key | one command | | Model list | one global list | the catalog scoped to your key | | Model access rules applied | no | yes | | Pools | no | yes | | Dynamic routes | no | yes | | Refreshed | with OpenCode's registry cache | every run | ### What lands in the config The model list is fetched from `/v3/compat/models` when the command runs, so it is scoped to your API key. You get the concrete models your project's [model access rules](/control-plane/rules/model-access) allow, the [pools](/capabilities/models) that load-balance a bare name across providers, and your deployed [dynamic routes](/capabilities/routes/overview) as `dynamic/`. `launch` refreshes the list on every run and `editors` refreshes it whenever you re-run it, so new models and newly deployed routes appear without editing anything. If no key is available the command still succeeds, falling back to a small built-in list. ### By hand To wire it manually instead, add an `opper` provider to `~/.config/opencode/opencode.json`. A hand-written block is a fixed list, so pools and dynamic routes will not appear and you maintain the models yourself: ```json theme={null} { "$schema": "https://opencode.ai/config.json", "provider": { "opper": { "npm": "@ai-sdk/openai-compatible", "name": "Opper", "options": { "baseURL": "https://api.opper.ai/v3/compat", "apiKey": "{env:OPPER_API_KEY}" }, "models": { "anthropic/claude-sonnet-4-6": { "name": "Claude Sonnet 4.6" }, "openai/gpt-5.5": { "name": "GPT-5.5" } } } } } ``` ```bash theme={null} export OPPER_API_KEY=your-opper-api-key opencode ``` If `opencode.json` already exists, merge the `opper` block into it rather than overwriting the file — replacing it wholesale drops any other providers you have configured. ## Choosing a model Use `/model` inside a session to switch. The picker lists whatever the provider block exposes; model IDs follow the `provider/model` convention from the [catalog](/capabilities/models). ## Troubleshooting `OPPER_API_KEY` has to be exported in the shell that launches OpenCode. Opper keys start with `op-`. OpenCode caches the models.dev registry at `~/.cache/opencode/models.json` and a long-lived install can be holding a copy from before Opper was listed. Refresh it: ```bash theme={null} curl -sSfo ~/.cache/opencode/models.json https://models.dev/api.json ``` Running `opper launch opencode` also sidesteps the cache, because it writes the provider block itself. `opper launch opencode` merges its entry alongside your existing providers rather than replacing the file, so re-running it restores the Opper block without touching the rest. The command falls back to a small built-in list when it cannot reach the catalog, which usually means no key was available when it ran. Run `opper login`, or export `OPPER_API_KEY`, then run `opper editors opencode` again. Neither the models.dev registry nor a hand-written provider block can carry them, because both are fixed lists that know nothing about your key. Run `opper editors opencode` to add the catalog scoped to your key, which includes both. ## Related Every agent the CLI can launch. Every app that runs on Opper. # pi Source: https://docs.opper.ai/integrations/coding-agents/pi Run the pi terminal coding agent on Opper, with native provider support and session tracing [pi](https://pi.dev) is a terminal-based coding agent. It has the deepest Opper integration of any agent here: a first-party provider package that adds the full catalog, automatic session span tracing, and seven multimodal tools. ## Setup One command, no config files: ```bash theme={null} npx @opperai/cli launch pi ``` This adds an `opper` provider entry to `~/.pi/agent/models.json` — idempotently, alongside any existing providers — and starts pi. ### Native provider package For the full integration, install `@opperai/pi-provider`: ```bash theme={null} pi install npm:@opperai/pi-provider export OPPER_API_KEY=your-opper-api-key ``` This gives you three things the plain provider block doesn't: * **The whole catalog** in the `/model` picker, not a hand-listed subset. * **Automatic session span tracing** — once you turn tracing on, every turn in a pi session lands under one trace in Opper. * **Seven multimodal tools** registered directly in pi: image generation, image analysis, text-to-speech, speech-to-text, web search, web fetch, and embeddings. ### Load only the provider or only the tools Both halves load by default. To take one, name the extensions explicitly in `~/.pi/agent/config.json`: ```json theme={null} { "packages": [ { "source": "npm:@opperai/pi-provider", "extensions": ["extensions/provider.ts", "extensions/tools.ts"] } ] } ``` Drop either entry from `extensions` to load just the other half. ## Choosing a model Use `/model` inside a session. With the provider package installed, the picker lists the full [catalog](/capabilities/models). ## Environment variables | Variable | Description | | - | - | | `OPPER_API_KEY` | Your Opper API key | | `OPPER_AGENT_NAME` | Override the framework name sent in headers (default: `pi-code`) | | `PI_IMAGE_SAVE_MODE` | Default image save mode: `tmp`, `project`, `global`, or `custom` | ## Related Session spans from pi, grouped per conversation once tracing is on. Every app that runs on Opper. # Voidleap Code Source: https://docs.opper.ai/integrations/coding-agents/voidleap-code Pick Opper from the built-in provider list in Voidleap Code [Voidleap Code](https://voidleap.com) is a desktop agentic IDE. Opper ships inside it as a first-class provider, so there is no base URL to override — pick Opper from the app's own provider list and paste a key. ## Setup In Voidleap Code, go to **Settings → Providers → Add Provider**. Select **Opper** from the list. The endpoint is already configured. Create one at [platform.opper.ai](https://platform.opper.ai/settings/api-keys) and paste it in. The model list is fetched from Opper once the key is saved, filtered by your [model access rules](/control-plane/rules/model-access). ## Related Scope which models a key can reach. Every app that runs on Opper. # Cline Source: https://docs.opper.ai/integrations/editors/cline Add Opper as an OpenAI Compatible provider in Cline [Cline](https://cline.bot) is an open-source AI coding agent for VS Code (Apache-2.0, [source](https://github.com/cline/cline)). It plans, edits across files, and runs commands, with an approval step before each action. ## Setup Click the gear icon in the Cline sidebar. Choose **OpenAI Compatible** as the API Provider. | Field | Value | | - | - | | **Base URL** | `https://api.opper.ai/v3/compat` | | **API Key** | Your Opper API key | Enter any model ID from the [catalog](/capabilities/models) — for example `anthropic/claude-sonnet-4-6`. ## Choosing a model Cline is an agent, so the model needs tool calling. Pick a model the [catalog](/capabilities/models) marks with the `tools` capability. Cline tracks input, output, and cache tokens per task in its own UI. The same calls appear in Opper's [Logs](/control-plane/logs) with cost attached, and as full [traces](/control-plane/trace) once you turn tracing on. ## Troubleshooting Check the key is an Opper key (they start with `op-`) and that you selected **OpenAI Compatible**, not **OpenAI**. Copy the model ID exactly from the [catalog](/capabilities/models), including the provider prefix. The model likely doesn't support tool calling well. Switch to a model the catalog marks with the `tools` capability. ## Related Same setup, different VS Code agent. Every app that runs on Opper. # Continue.dev Source: https://docs.opper.ai/integrations/editors/continue Add Opper to Continue.dev for VS Code and JetBrains [Continue.dev](https://continue.dev) is an open-source AI code assistant for VS Code and JetBrains. It reads its models from a YAML config, so Opper slots in as an OpenAI-compatible provider. ## Setup Add Opper to `~/.continue/config.yaml`: ```yaml theme={null} models: - name: Claude Sonnet 4.6 (Opper) provider: openai model: anthropic/claude-sonnet-4-6 apiKey: your-opper-api-key apiBase: https://api.opper.ai/v3/compat roles: - chat - edit ``` Add one entry per model you want available. `provider: openai` selects Continue's OpenAI-compatible client; `apiBase` redirects it to Opper. `config.yaml` holds the key in plaintext. It lives in your home directory rather than a repo, but treat it like any other credential file — don't copy it into a project or commit it. ## Roles Continue assigns models to roles — `chat`, `edit`, `apply`, `autocomplete`, `embed`, `rerank`. List the roles each entry should serve. A fast, cheap model is usually the right choice for `autocomplete`, and a frontier model for `chat` and `edit`. ## Troubleshooting Continue reloads `config.yaml` on save. If it's still missing, check the indentation — `models` is a top-level list. `apiKey` must hold your Opper key, and `apiBase` has to be set. Without `apiBase`, Continue sends the key to OpenAI. ## Related Browse the catalog for model IDs and capabilities. Every app that runs on Opper. # Cursor Source: https://docs.opper.ai/integrations/editors/cursor Point Cursor at Opper as a custom OpenAI base URL, and add Opper skills to your project [Cursor](https://cursor.com) is Anysphere's AI code editor. It has two independent things worth wiring to Opper: the model endpoint it calls, and the skills that teach it how to build with Opper. ## Run Cursor on Opper models Cursor takes a custom OpenAI base URL in its settings: **Settings → Models**, then enable **Override OpenAI Base URL**. ``` https://api.opper.ai/v3/compat ``` Paste an Opper key from [platform.opper.ai](https://platform.opper.ai/settings/api-keys) into the OpenAI API key field, then add the model IDs you want — for example `anthropic/claude-sonnet-4-6`. Verify the key to activate the override. Cursor's BYOK path drives chat and inline edits. Some Cursor-specific features — Tab completion and Composer's agent mode among them — run on Anysphere's own infrastructure and ignore the override. That's a Cursor product decision, not a gateway limitation. ## Add Opper skills to your project Separately from the endpoint, you can give Cursor context about Opper so it writes correct Opper code. Install the Opper [skills](/developer-tools/skills) in your project: ```bash theme={null} npx skills add opper-ai/opper-skills ``` Cursor reads each skill's short description at startup and loads the full skill only when your task matches it. ## Troubleshooting Cursor needs each model ID added by hand under **Settings → Models** — it doesn't fetch the catalog. Copy IDs from the [catalog](/capabilities/models). Expected. The base URL override covers chat and edits; Tab runs on Anysphere's own service. Cursor verifies by making a real call against the first model in your list. If that model ID is wrong or your [model access rules](/control-plane/rules/model-access) deny it, verification fails even though the key is valid. ## Related What each Opper skill covers, and how to keep them updated. Every app that runs on Opper. # GitHub Copilot Source: https://docs.opper.ai/integrations/editors/github-copilot Use Opper models in VS Code chat with browser sign-in and personal usage tracking The [Opper extension for VS Code](https://marketplace.visualstudio.com/items?itemName=opperai.opper-vscode) adds your allowed Opper models to the chat model picker, including agent mode. Inline completions remain on GitHub's service. ## Setup Install **Opper** from the VS Code Marketplace, or run: ```bash theme={null} code --install-extension opperai.opper-vscode ``` Open the Command Palette and run **Opper: Sign In**. Complete browser approval using your Opper account and your organization's SSO when configured. No Opper CLI installation is needed. Already have an API key? Open **Opper: Account and Models** and choose the API-key option. Credentials are stored in VS Code's SecretStorage. Open Chat and choose an Opper model. The list is fetched for your credential and respects your organization's model access policies. ## Models, pools, and routes The picker includes allowed models, load-balancing pools, and deployed [dynamic routes](/capabilities/routes/overview). Use **Opper: Choose What to List (Models, Pools, Routes)** to control which types appear, and **Opper: Refresh Models** to fetch the latest list. Manage model access and compliance policies in Opper. Agent mode requires a model or route that advertises tool support. ## Usage and sign-in * **Opper: Show Budget** shows your personal or role allowance, spend, remaining amount, and reset date when available. Project usage is shown when your credential is project-bound; organization billing requires permission. * Hover over the Opper status-bar item for an allowance summary, or click it to open the budget view. * **Opper: Renew Sign-in** opens browser approval for a replacement credential. Renewing does not reset your allowance. New personal agent logins can work at organization scope without choosing a project. Existing project-bound credentials keep their binding until renewed. Managed Copilot installations may require an administrator to enable the **Bring Your Own Language Model Key in VS Code** policy. ## Related Use your Opper sign-in from the terminal. Configure your organization's allowed models. # JetBrains AI Assistant Source: https://docs.opper.ai/integrations/editors/jetbrains Add Opper as an OpenAI-compatible provider in IntelliJ, PyCharm, and every JetBrains IDE [JetBrains AI Assistant](https://www.jetbrains.com/ai/) brings chat and agent features into IntelliJ IDEA, PyCharm, WebStorm, GoLand, and the rest of the JetBrains IDE family. Its bring-your-own-key mode accepts any OpenAI-compatible provider. ## Setup **Settings → Tools → AI Assistant → Providers & API keys**, then add an **OpenAI-compatible** provider. | Field | Value | | - | - | | **Base URL** | `https://api.opper.ai/v3/compat` | | **API Key** | Your Opper API key | | **Model** | Any ID from the [catalog](/capabilities/models), e.g. `anthropic/claude-sonnet-4-6` | Click **Continue**. AI Chat and the integrated agents now run on Opper — no JetBrains AI subscription required. ## Choosing a model The model is named per provider entry rather than fetched, so add one provider entry per model you want to switch between. Which AI Assistant features work depends on what the model supports — agents need tool calling. ## Troubleshooting Check the key is an Opper key (they start with `op-`) and that you chose the OpenAI-compatible provider type rather than OpenAI. Copy the model ID exactly from the [catalog](/capabilities/models), including the provider prefix. A few AI Assistant features are tied to JetBrains' own service rather than the configured provider. Chat and the integrated agents are the ones BYOK covers. ## Related Browse the catalog for model IDs and capabilities. Every app that runs on Opper. # Kilo Code Source: https://docs.opper.ai/integrations/editors/kilo-code Add Opper as an OpenAI Compatible provider in Kilo Code [Kilo Code](https://kilo.ai) is an open-source AI agent for VS Code (Apache-2.0, [source](https://github.com/Kilo-Org/kilocode)). It combines planning, editing, and command execution with a mode system for different kinds of work. ## Setup Click the gear icon in the Kilo Code panel. Select **OpenAI Compatible** from the provider list. | Field | Value | | - | - | | **Base URL** | `https://api.opper.ai/v3/compat` | | **API Key** | Your Opper API key | | **Model** | Any ID from the [catalog](/capabilities/models), e.g. `anthropic/claude-sonnet-4-6` | ## Troubleshooting Check the key is an Opper key (they start with `op-`) and that the provider is **OpenAI Compatible**, not **OpenAI**. Copy the model ID exactly from the [catalog](/capabilities/models), including the provider prefix. Kilo Code fetches models from `GET /v3/compat/models`. An empty list usually means your [model-access rules](/control-plane/rules/model-access) have narrowed the catalog to nothing for this key. ## Related Same setup, different VS Code agent. Every app that runs on Opper. # Zed Source: https://docs.opper.ai/integrations/editors/zed Add Opper as an openai_compatible provider in Zed, via the agent panel or settings.json [Zed](https://zed.dev) is a high-performance code editor from Zed Industries (GPL-3.0, [source](https://github.com/zed-industries/zed)) with an agent panel built in. It supports OpenAI-compatible providers both through the UI and through `settings.json`. ## Setup ### Through the UI Go to **Agent Settings → LLM Providers → Add Provider**, and supply the base URL `https://api.opper.ai/v3/compat` along with your Opper API key. ### Through settings.json ```json theme={null} { "language_models": { "openai_compatible": { "opper": { "api_url": "https://api.opper.ai/v3/compat", "available_models": [ { "name": "anthropic/claude-sonnet-4-6", "display_name": "Claude Sonnet 4.6 (Opper)", "max_tokens": 1000000, "capabilities": { "tools": true, "images": true } }, { "name": "openai/gpt-5.5", "display_name": "GPT-5.5 (Opper)", "max_tokens": 1050000, "capabilities": { "tools": true, "images": true } } ] } } } } ``` Don't put the API key in `settings.json`. Zed reads it from the provider settings UI, or from an environment variable derived from the provider name — `OPPER_API_KEY` for a provider named `opper`. `tools` and `images` live under a `capabilities` object on each model, not at the top level of the entry. Both default to on when omitted. ## Choosing a model Zed doesn't fetch the catalog — each model you want has to be listed in `available_models`. Pull the IDs and context windows from the [model catalog](/capabilities/models). `max_tokens` is the model's **context window**, and Zed uses it to drive the token counter and decide when to truncate. Setting it too low silently costs you usable context, so take the real figure from the catalog rather than assuming a provider's older limit. ## Troubleshooting Zed reloads `settings.json` on save. If the provider is still missing, the JSON is likely malformed — check that `openai_compatible` sits inside `language_models`. The key isn't reaching Zed. Set it in the provider settings UI, or export `OPPER_API_KEY` before launching Zed from the terminal — a Zed started from the Dock won't see a variable set in your shell rc. Set `"capabilities": { "images": true }` on that model's entry — a bare `"images"` key at the top level of the entry is ignored. Check the model actually supports vision in the [catalog](/capabilities/models) too. ## Related Browse the catalog for model IDs and context windows. Every app that runs on Opper. # LiteLLM Source: https://docs.opper.ai/integrations/frameworks/litellm Use Opper as the upstream router for a self-hosted LiteLLM gateway [LiteLLM](https://docs.litellm.ai) is a self-hosted LLM gateway. Point it at Opper as its upstream router and every team behind your LiteLLM proxy gets the full Opper catalog through one Opper API key, with Opper handling provider routing, failover, and EU compliance. ## Setup Opper's model IDs use the same `provider/model` convention as OpenRouter, so LiteLLM's OpenRouter-compatible provider works today: ```yaml theme={null} model_list: - model_name: claude-haiku litellm_params: model: openrouter/anthropic/claude-haiku-4-5 api_base: https://api.opper.ai/v3/compat api_key: os.environ/OPPER_API_KEY ``` A first-class `opper/` provider for LiteLLM is [in review upstream](https://github.com/BerriAI/litellm/pull/36075). Once merged, the config becomes `model: opper/anthropic/claude-haiku-4-5` with no `api_base` override, and cost tracking works on every dispatch path including streaming. ## Cost tracking Every Opper response includes `usage.cost` (USD). LiteLLM's OpenRouter path reads it into its spend tracking, so LiteLLM spend logs show Opper's actual billed cost instead of estimates from a static price map. ## Model discovery `GET https://api.opper.ai/v3/compat/models` returns the models your key can use — filtered by your organization's compliance rules, with per-token pricing and per-model compliance metadata (region, ZDR status, GDPR residency). Sync your `model_list` from it, or filter it first, for example to EU-resident, zero-data-retention models only. ## Per-tenant usage attribution Forward up to 8 tags per request with the `X-Opper-Tags` header, then group usage and cost by tag in Opper analytics: ```yaml theme={null} model_list: - model_name: claude-haiku litellm_params: model: openrouter/anthropic/claude-haiku-4-5 api_base: https://api.opper.ai/v3/compat api_key: os.environ/OPPER_API_KEY extra_headers: {"X-Opper-Tags": "tenant:acme,env:prod"} ``` ```bash theme={null} curl -s "https://api.opper.ai/v2/analytics/usage?group_by=tenant&fields=total_tokens" \ -H "Authorization: Bearer $OPPER_API_KEY" ``` Tag limits, the other ways to attach tags, and more ways to slice usage are covered in [Tags and usage attribution](/build/gateway/usage-attribution). ## Related Group spend by tenant, environment, or app. IDs, pricing, and compliance metadata. # Mastra Source: https://docs.opper.ai/integrations/frameworks/mastra Use Opper as a first-class provider in Mastra, the TypeScript agent framework [Mastra](https://mastra.ai) is a TypeScript framework for building agents, workflows, and RAG applications. Opper is a **first-class provider in Mastra's model router** — there is no base URL to configure and no compatibility shim to set up. Prefix a model with `opper/` and Mastra handles the rest. ## Setup Set your key in the environment: ```bash theme={null} # .env OPPER_API_KEY=your-opper-api-key ``` Then name any Opper model on an agent: ```ts theme={null} import { Agent } from "@mastra/core/agent"; const agent = new Agent({ id: "my-agent", name: "My Agent", instructions: "You are a helpful assistant", model: "opper/anthropic/claude-sonnet-5" }); const response = await agent.generate("Hello!"); const stream = await agent.stream("Tell me a story"); for await (const chunk of stream) { console.log(chunk); } ``` Mastra picks up `OPPER_API_KEY` automatically — no client construction, no `baseURL`. ## Choosing a model Model IDs are the catalog ID with an `opper/` prefix: `opper/anthropic/claude-sonnet-5`, `opper/openai/gpt-5.5`, `opper/xai/grok-4.6`. Mastra's router ships a snapshot of the catalog with context windows and per-token pricing, sourced from [models.dev](https://models.dev). Because that list is a snapshot, a model added to Opper very recently may not appear in Mastra's router yet. Name it explicitly with the object form below, or check the [Opper catalog](/capabilities/models) for what's live. ## Routing per request Mastra accepts a function for `model`, so you can pick per request — cheap model for routine work, frontier model for the hard path: ```ts theme={null} const agent = new Agent({ id: "dynamic-agent", name: "Dynamic Agent", model: ({ requestContext }) => requestContext.task === "complex" ? "opper/anthropic/claude-opus-5" : "opper/openai/gpt-5.6-luna" }); ``` To move that decision off your code entirely, deploy a [dynamic route](/capabilities/routes/overview) and name it once instead. ## Custom headers The object form takes explicit configuration, which is also how you attach [usage tags](/build/gateway/usage-attribution): ```ts theme={null} const agent = new Agent({ id: "custom-agent", name: "custom-agent", model: { url: "https://api.opper.ai/v3/compat", id: "opper/anthropic/claude-sonnet-5", apiKey: process.env.OPPER_API_KEY, headers: { "X-Opper-Tags": "app:support-bot,env:prod" } } }); ``` Mastra routes through Opper's OpenAI-compatible `/chat/completions` surface. Opper features that live outside that surface — [structured output](/build/gateway/structured-output) beyond `response_format`, [realtime](/build/realtime/quickstart), and the [audio](/build/multimodal/audio) and [image](/build/multimodal/images) endpoints — need a direct call rather than the Mastra agent. ## Related Move model choice out of application code. The other TypeScript toolkit with an Opper provider. # Vercel AI SDK Source: https://docs.opper.ai/integrations/frameworks/vercel-ai-sdk Use any Opper model with the Vercel AI SDK through the official provider package The [Vercel AI SDK](https://ai-sdk.dev/) is a TypeScript toolkit for building AI-powered applications. Opper publishes an official provider package, so any Opper model works with `generateText`, `streamText`, `embed`, and tool calls. ## Setup ```bash theme={null} npm install @opperai/ai-sdk-provider ai ``` ```ts theme={null} import { opper } from '@opperai/ai-sdk-provider'; import { generateText } from 'ai'; const { text } = await generateText({ model: opper('openai/gpt-5.5'), prompt: 'Hello!', }); ``` Pass any Opper model string: `opper('anthropic/claude-sonnet-4-6')`, `opper('gemini/gemini-3.5-flash')`, `opper('default')`, and so on. ## Session tracing Wrap your model with `opperSpan` to group every call in a conversation under one trace in Opper. Traces are stored once you turn [tracing](/control-plane/trace) on; without it, each call still appears in [Logs](/control-plane/logs) with its token usage and cost. ```ts theme={null} import { wrapLanguageModel, generateText } from 'ai'; import { opper, opperSpan } from '@opperai/ai-sdk-provider'; const { middleware, handle } = opperSpan({ name: 'my-chat-session' }); const model = wrapLanguageModel({ model: opper('openai/gpt-5.5'), middleware, }); const { text } = await generateText({ model, prompt: 'Hello!' }); await handle.end(text); // closes the span ``` ## Related Grouped spans, per session. The OpenAI, Anthropic, and Google AI SDKs against Opper. # Integrations Source: https://docs.opper.ai/integrations/overview Run the coding agents, editors, chat apps, and automation platforms you already use on any model in the Opper catalog Most AI apps let you point them at a different inference endpoint. Opper's gateway speaks the OpenAI, Anthropic, and Gemini wire formats, so once an app can take a base URL and an API key, it runs on any of the [models in the catalog](/capabilities/models) — with one key, [EU or US hosting](/overview/security), and every call recorded in [Logs](/control-plane/logs) with its token usage and cost, and traced in full once you turn [tracing](/control-plane/trace) on. Nine agents need no configuration at all. The [Opper CLI](/developer-tools/cli) sets them up for you: ```bash theme={null} npx @opperai/cli launch claude ``` ## One base URL Every integration on this page points at the same endpoint: ``` https://api.opper.ai/v3/compat ``` It serves all three compatibility surfaces from that one URL — OpenAI Chat Completions and Responses, the Anthropic Messages API, and the Gemini `generateContent` API. Apps that append their own path (`/chat/completions`, `/v1/messages`, `/v1beta/models/{model}:generateContent`) work unchanged; see [Drop-in SDKs](/build/gateway/drop-in-sdks) for the full surface. Model IDs use the `provider/model` convention — `anthropic/claude-sonnet-4-6`, `openai/gpt-5.5`, `gemini/gemini-3.5-flash`. Apps that support model discovery call `GET /v3/compat/models` and populate their own picker, filtered by your [model access rules](/control-plane/rules/model-access), so a model your policy denies is never offered. ## Launch from the CLI For these agents, one command provisions the config and starts the app. Nothing to edit by hand: | Agent | Command | What it writes | | - | - | - | | [Claude Code](/integrations/coding-agents/claude-code) | `opper launch claude` | `ANTHROPIC_BASE_URL` / `ANTHROPIC_AUTH_TOKEN` for the session | | [Claude Desktop](/integrations/personal-agents/claude-desktop) | `opper launch claude-desktop` | third-party-inference profile, then restarts the app | | [Codex](/integrations/coding-agents/codex) | `opper launch codex` | provider passed per run with `-c` flags; `~/.codex/config.toml` is left unchanged | | Codex Desktop | `opper launch codex-desktop` | `opper-desktop` provider in `~/.codex/config.toml`, then opens the Codex/ChatGPT app (macOS) | | [GitHub Copilot CLI](/integrations/coding-agents/github-copilot-cli) | `opper launch copilot` | browser sign-in, then a per-run Copilot profile with your allowed models | | [OpenCode](/integrations/coding-agents/opencode) | `opper launch opencode` | provider block in `~/.config/opencode/opencode.json` | | [Hermes](/integrations/personal-agents/hermes) | `opper launch hermes` | isolated `HERMES_HOME`, your own config untouched | | [OpenClaw](/integrations/personal-agents/openclaw) | `opper launch openclaw` | `opper` provider in `~/.openclaw/agents/main/agent/models.json` | | [pi](/integrations/coding-agents/pi) | `opper launch pi` | `opper` provider in `~/.pi/agent/models.json` | Pin a model with `--model`, and remove any integration later with `opper agents remove ` — it unwires Opper without uninstalling the app. ## Coding agents Anthropic's terminal coding agent, on any model OpenAI's open-source coding CLI SST's open-source terminal coding agent Terminal agent with a native Opper provider DeepSeek's plugin-first agent harness GitHub's terminal agent, no Copilot plan needed AI pair programming in your terminal Open agent from the Agentic AI Foundation Factory's agent for terminal and IDEs Desktop agentic IDE with Opper built in ## Editors and IDEs Copilot Chat and agent mode, via the Opper VS Code extension The AI code editor Open-source AI coding agent for VS Code Open-source AI agent for VS Code The high-performance code editor IntelliJ, PyCharm, and every JetBrains IDE Open-source assistant for VS Code and JetBrains ## Personal agents Anthropic's agent for knowledge work Anthropic's macOS and Windows desktop app AI copilot inside your Obsidian vault Personal agent on WhatsApp, Telegram, and Discord Nous Research's open terminal agent ## Chat The open-source AI chat platform Self-hosted AI workspace ## Automation Workflow automation with a native Opper node ## Frameworks and gateways Not apps — libraries and proxies you build on or run in front of Opper. TypeScript toolkit with an official Opper provider TypeScript agent framework with a first-class `opper/` provider Use Opper as the upstream router for a self-hosted gateway Opper is listed on [models.dev](https://models.dev), the open catalog many tools read their model list from. If an app pulls its providers from there, Opper may already be in its picker — with model IDs, context windows, and pricing filled in — before you configure anything. ## Why route an app through Opper | | Direct to the provider | Through Opper | | - | - | - | | **Model access** | Whatever that provider serves | The whole [catalog](/capabilities/models), switchable mid-session | | **Keys** | One per provider | One Opper key | | **Residency** | Wherever the provider runs | Pin EU-hosted models when you need to | | **Reliability** | Single upstream | Pools spread a model across every provider serving it | | **Cost control** | Per-provider dashboards | [Spend rules](/control-plane/rules/spend) and one bill at provider rates | | **Visibility** | Whatever the app shows you | Every call in [Logs](/control-plane/logs) with model, latency, tokens, and cost; full [traces](/control-plane/trace) once tracing is on | | **Policy** | Enforced per provider, if at all | [Model access](/control-plane/rules/model-access) and [retention](/control-plane/rules/retention) rules, centrally | ## Your app isn't listed Most of these integrations are the same three fields. If your app has a setting for a custom OpenAI-compatible provider, it already works: ``` https://api.opper.ai/v3/compat ``` If the app expects a host and a path separately, the host is `https://api.opper.ai` and the path is `v3/compat/chat/completions`. Create one at [platform.opper.ai](https://platform.opper.ai/settings/api-keys). Keys are scoped to a project, so use a separate one per app if you want usage split out. Use a full model ID such as `anthropic/claude-sonnet-4-6`. Apps that fetch `GET /v3/compat/models` will list the catalog for you. Tag the traffic with `X-Opper-Tags` to break usage down per app — see [Tags and usage attribution](/build/gateway/usage-attribution). ## Next steps Model IDs, hosting regions, and retention metadata. Launch agents, inspect traces, and manage models from the terminal. # Claude Cowork Source: https://docs.opper.ai/integrations/personal-agents/claude-cowork Route Anthropic's knowledge-work agent through Opper's Anthropic-compatible gateway [Claude Cowork](https://claude.com/product/cowork) is Anthropic's agent for knowledge work — the same agentic engine as Claude Code, but built for non-technical tasks inside the Claude desktop app, with no terminal required. It works across your local files, folders, and applications while you steer. Cowork supports third-party inference providers, so it can run on any model in the Opper catalog rather than Claude alone. ## Setup In Cowork's **Help** menu, click **Enable Developer Mode**, then quit and restart the app. A new **Developer** menu appears in the menu bar. From the **Developer** menu, choose **Configure Third-party inference**, then **Gateway (Anthropic-compatible)**. | Field | Value | | - | - | | **Gateway base URL** | `https://api.opper.ai/v3/compat` | | **Gateway API key** | Your Opper API key | | **Gateway auth scheme** | `bearer` | Click **Apply locally** to try it on your own machine, or **Export** to ship the same configuration as an MDM profile (`.mobileconfig` for macOS, `.reg` for Windows) for a team-wide rollout. ## Choosing a model Add models to Cowork by their Opper IDs and they appear in its picker. With no [model-access rule](/control-plane/rules/model-access) set, your key reaches the whole catalog; a rule narrows it. A reasonable starting set: | Use case | Models | | - | - | | **Claude Sonnet** across major clouds | `aws/claude-sonnet-4-6-eu` · `azure/claude-sonnet-4-6` · `gcp/claude-sonnet-4-5-eu` | | **Claude Opus** when you need it | `aws/claude-opus-4-7` · `azure/claude-opus-4-7` · `gcp/claude-opus-4-7-eu` | | **Cheaper routine work** (open-weight) | `evroc/gpt-oss-120b` | Once a model is added on the Opper side, it shows up in the Cowork picker on your next conversation. You can switch mid-session without losing chat history. ## Verify Send any message. The status indicator at the bottom-left should read **`Cowork 3P | Gateway`** — that's how you know the request routed through Opper rather than going to Anthropic directly. From there, calls appear in [Logs](/control-plane/logs) with model, latency, tokens, and cost, and as full [traces](/control-plane/trace) once you turn tracing on. ## Troubleshooting Cowork needs a full quit and restart after enabling Developer Mode — closing the window isn't enough. Cowork lists the models you entered in the third-party inference dialog — it does not fetch the catalog. Add at least one model ID there. If a model you added is rejected, check your [model-access rules](/control-plane/rules/model-access) aren't denying it. The gateway config didn't apply. Re-open **Developer → Configure Third-party inference** and confirm the auth scheme is `bearer`. ## Related Anthropic's desktop app, wired by the Opper CLI. Pin residency and retention for knowledge work. # Claude Desktop Source: https://docs.opper.ai/integrations/personal-agents/claude-desktop Route Anthropic's desktop app — chat and Claude Code alike — through Opper [Claude Desktop](https://claude.com/download) is Anthropic's macOS and Windows desktop app. Both the chat and Claude Code surfaces inside it run through whichever inference profile is active, so pointing that profile at Opper covers the whole app. ## Setup One command, no config files: ```bash theme={null} npx @opperai/cli launch claude-desktop ``` The CLI uses Claude Desktop's documented third-party-inference mode (`deploymentMode: "3p"`). It writes an Opper profile to `~/Library/Application Support/Claude-3p/configLibrary/` on macOS, or `%LOCALAPPDATA%\Claude-3p\configLibrary\` on Windows, pointing at Opper's Anthropic-compatible endpoint. If Claude Desktop is already running, it is quit and reopened so the change takes effect immediately. The picker shows `claude-opus-5` (the default), `claude-sonnet-5`, `claude-haiku-4-5`, and `claude-fable-5-1`. Pass `--model ` to make a different one the default. Claude Desktop only accepts Claude-backed models, so the ID has to be a `claude-*` pool or a `provider/claude-*` route; dynamic routes (`dynamic/...`) are rejected. ## Reverting ```bash theme={null} npx @opperai/cli agents remove claude-desktop ``` This flips `deploymentMode` back to `"1p"` and removes the Opper profile entry. Other tools' third-party profiles and your own preferences are left untouched. On first launch, macOS will prompt "opper would like to control Claude". Accept this so the CLI can quit and reopen the app cleanly. Available on macOS and Windows — Anthropic does not ship a Linux build of Claude Desktop. ## Related Anthropic's knowledge-work agent, configured in-app. The same agent in the terminal. # Hermes Source: https://docs.opper.ai/integrations/personal-agents/hermes Run Nous Research's open terminal agent on any model in the Opper catalog [Hermes](https://hermes-agent.nousresearch.com/) is an open-source terminal agent from Nous Research. It registers named custom providers, and discovers the available models from each one automatically. ## Setup One command, no config files: ```bash theme={null} npx @opperai/cli launch hermes ``` The CLI uses an isolated `HERMES_HOME=~/.opper/hermes-home/`, so your real `~/.hermes/` is untouched. To wire Opper into your existing Hermes setup, add your API key and register Opper as a named custom provider, then point the `model` block at it: ```bash theme={null} echo 'OPPER_API_KEY=your-api-key' >> ~/.hermes/.env ``` ```yaml theme={null} # ~/.hermes/config.yaml model: provider: opper default: anthropic/claude-opus-4-7 custom_providers: - name: opper base_url: https://api.opper.ai/v3/compat key_env: OPPER_API_KEY ``` ## Choosing a model Switch mid-session: ``` /model custom:opper:anthropic/claude-opus-4-7 /model custom:opper:gemini/gemini-3.5-flash ``` Hermes discovers the available models from Opper automatically — the model picker lists the full catalog without any per-model configuration. ## Troubleshooting Hermes calls `GET /v3/compat/models` to populate it. An empty list usually means your [model-access rules](/control-plane/rules/model-access) have narrowed the catalog to nothing for this key. `key_env` names the variable Hermes reads, so `OPPER_API_KEY` has to exist in `~/.hermes/.env` or in the environment. ## Related Every agent the CLI can launch. Every app that runs on Opper. # Copilot for Obsidian Source: https://docs.opper.ai/integrations/personal-agents/obsidian-copilot Pick Opper from the built-in BYOK provider list in Copilot for Obsidian [Copilot for Obsidian](https://www.obsidiancopilot.com) is an AI copilot that works inside your Obsidian vault — chat with your notes, search across them, and generate new ones (AGPL-3.0, [source](https://github.com/logancyang/obsidian-copilot), by Brevilabs). Opper ships inside the plugin as a built-in provider choice, so there's no base URL to type: the endpoint comes prefilled. ## Setup In Obsidian, go to **Settings → Copilot → BYOK → Add a provider**. Type `Opper` and select it. The base URL is prefilled. Create one at [platform.opper.ai](https://platform.opper.ai/settings/api-keys) and paste it in. The model list is fetched from Opper, filtered by your [model access rules](/control-plane/rules/model-access). ## Choosing a model Copilot for Obsidian uses models for two distinct jobs — chat and embeddings for vault search. Pick a chat model from the [catalog](/capabilities/models), and an embedding model separately if you want semantic search over your notes. Vault contents are personal data. If that matters for you, pin an EU-hosted model and set a [retention rule](/control-plane/rules/retention) on the project this key belongs to. ## Related Control what's stored, per project. Every app that runs on Opper. # OpenClaw Source: https://docs.opper.ai/integrations/personal-agents/openclaw Run the OpenClaw personal agent on Opper, across WhatsApp, Telegram, and Discord [OpenClaw](https://openclaw.ai) is an open-source personal AI assistant built on pi. It runs on WhatsApp, Telegram, Discord, and other chat apps, and takes any OpenAI-compatible endpoint as a custom provider. ## Setup One command, no config files: ```bash theme={null} npx @opperai/cli launch openclaw ``` This adds an `opper` provider entry to `~/.openclaw/agents/main/agent/models.json` and defaults to running `gateway start` as a background daemon. To wire it by hand, add Opper as a custom provider in `~/.openclaw/config.json`. This is the top-level config; the CLI writes the equivalent entry into the per-agent `models.json` above. Either location works — use the manual one when you want the provider available to every agent, or want the model list under version control: ```json theme={null} { "env": { "OPPER_API_KEY": "your-api-key" }, "agents": { "defaults": { "model": { "primary": "opper/anthropic/claude-opus-4-7" } } }, "models": { "providers": { "opper": { "baseUrl": "https://api.opper.ai/v3/compat", "apiKey": "${OPPER_API_KEY}", "api": "openai-completions", "models": [ { "id": "anthropic/claude-opus-4-7", "name": "Claude Opus 4.7 (Opper)" }, { "id": "anthropic/claude-sonnet-4-6", "name": "Claude Sonnet 4.6 (Opper)" }, { "id": "mistral/mistral-large-2512", "name": "Mistral Large 3 (Opper)" }, { "id": "vertexai/gemini-3.5-flash-eu", "name": "Gemini 3.5 Flash EU (Opper)" } ] } } } } ``` ## Choosing a model Switch at any time: ```bash theme={null} openclaw models set opper/anthropic/claude-sonnet-4-6 ``` The `opper/` prefix names the provider; everything after it is the Opper model ID. The custom-provider route gives you full model access with a single API key, but no session span tracing. OpenClaw is built on pi, so if you want traces grouped per session it's worth looking at the native [pi provider package](/integrations/coding-agents/pi) — check its docs for whether your OpenClaw version can load pi extensions. ## Related The agent OpenClaw is built on, with native tracing. Every app that runs on Opper. # What is Opper? Source: https://docs.opper.ai/overview/about Opper is one connection to every AI model, with the tools you need to ship AI to real users. Opper sits between your app and the AI models you use. One API key gets you 700+ models from every major provider, plus the observability and controls to run them in production. Without it, you end up gluing the pieces together yourself: a separate SDK and key for each provider, and monitoring you add after something breaks. Opper is that whole layer behind one API. Switch models by changing a string, and every call is governed and metered by default. ## Build, observe, and control One API for every model. Send a request, get a response. Switch models with a config change. Inspect logs and traces, and manage checks, budgets, model access, retention, and routing. ## The Gateway One API that covers every modality and every major model. * **Any model, same code.** Choose `openai/gpt-5.5`, `anthropic/claude-sonnet-4-6`, `gemini/gemini-3.1-pro-preview`, or any of 700+ others by changing a string. * **Beyond text.** Send images and PDFs in, and generate images, speech, and video, transcribe audio, and run realtime voice — all through the same API. See [Multimodality](/build/multimodal/overview). * **Use your existing SDK.** Already on OpenAI, Anthropic, or Google AI? Point it at Opper and your code stays the same. * **Or write no code at all.** Point an app you already use — Claude Code, Cursor, Zed, n8n — at Opper instead. See [Integrations](/integrations/overview). * **One key.** Replaces a folder of provider credentials. ## Observe & Control Once your AI is in front of real users, you need to understand what it is doing and set clear operating boundaries. Opper lets you inspect calls and manage the rules enforced by the gateway: | | What it does | | - | - | | **Data retention** | Sets how long request and response content is kept, or keeps none of it. | | **Spend limits** | Caps what an organization can spend in a calendar month. | | **Model access** | Limits which providers, regions, and models can be used. | | **Checks** | Scores responses against your criteria, and blocks or redacts content. | | **Routing** | Chooses which provider a model pool tries first. | | **Default model** | Picks the model for calls that don't name one, set through the Management API. | The first five live on the organization-level [Rules](/control-plane/rules/overview) page. Each rule applies to the whole organization or selected projects. You can review rule activity in traces when trace storage is enabled. [Explore Observe & Control →](/control-plane/overview) ## Where Opper runs Opper runs in Stockholm, on AWS with a second production environment at evroc, and your data stays in-region. On the Gateway plan it stores only metadata, not your prompts or responses; on Control Plane it stores traces only once you turn tracing on, for the period you choose. Only the model call itself reaches the provider, and you can pin which providers and countries are allowed. The full picture is on the [Security](/overview/security) page. ## Get going Run your first call, then read the Platform section in order to understand how it all fits. Five minutes to your first call. Gateway, projects, Rules, and what happens on a call. One API, 700+ models, routing, and EU enforcement. # Core concepts Source: https://docs.opper.ai/overview/concepts How Opper connects model calls, projects, rules, and traces. Opper is one API in front of every major AI model, wrapped in a layer that governs and records every call passing through it. You write ordinary model calls. Opper routes each one to a model, applies your rules, and records what happened. Three ideas cover almost everything: the **Gateway** runs model calls, **projects** organize your applications, and **Rules** define what those calls may do. ## Gateway and Rules The [AI Gateway](/overview/gateway) is the request path. Your app calls it, it picks a model, runs the request, and returns the result. One API for [text generation](/build/gateway/drop-in-sdks), [multimodality](/build/multimodal/overview) — image, audio, and video — and [realtime voice](/build/realtime/quickstart), in front of 700+ models. In [Rules](/control-plane/rules/overview), you decide which models can run, what content gets blocked, how responses are scored, how long call content is kept, and what the organization can spend. The Gateway enforces those rules automatically, so policy stays outside your application code. You can block a category of content, restrict a project to EU providers, or cap monthly spend without shipping a new version of your app. ## Projects Your account is an **organization**. Inside it, each app or environment is a **project** with its own API key and its own isolated data. Rules apply at one of these two levels. An organization rule applies everywhere; a project rule narrows it. You can tighten a policy for one project but never loosen what the organization allows, so a rule such as “EU providers only” continues to hold across every project. ## What happens on a call Every call runs the same path, and you can replay all of it afterward: 1. The Gateway identifies your project from the API key. 2. [Checks](/control-plane/rules/checks) inspect the input, a [default-model rule](/control-plane/route) fills in the model if you didn't name one, and [model access](/control-plane/rules/model-access) confirms the model is allowed. 3. The model runs. 4. [Checks](/control-plane/rules/checks) inspect the output, and a score check grades it afterwards. 5. The call is recorded. Without a retention rule that's metadata only (model, cost, latency), listed in [Logs](/control-plane/logs); with a retention period above 0 days, it's the full **trace**. A trace is the full tree behind one request: the model call, any tool calls, and every rule that fired. Without a [retention rule](/control-plane/rules/retention) Opper keeps only metadata. To store full traces (with inputs and outputs), turn tracing on in **Rules → Opper retention** on the Control Plane plan. That's also what lets a [score check](/control-plane/rules/checks#score-checks) grade a call, since it needs the content to judge. ## Where to go next One API, 700+ models, routing, and EU enforcement. The request path in detail. Inspect logs and traces, and manage the rules applied in production. # AI Gateway Source: https://docs.opper.ai/overview/gateway One endpoint in front of 700+ models. Drop in your SDK, fall back across models with dynamic routes, keep data in the EU, and see one bill across every provider. The AI Gateway is the request path. Your app sends a call, the gateway picks a model, runs it, and returns the result. One gateway and API key, with [700+ models](/capabilities/models) behind it. The gateway is hosted in the EU; provider inference locations depend on the selected model. It replaces wiring up each provider yourself, so there are no per-provider SDKs, keys, billing, or rate limits to manage. To swap a model, change a string. ## What the gateway gives you Reach 700+ models from 50+ providers, including OpenAI, Anthropic, Google, and Mistral, through a single endpoint and key. Not just text — generate images, speech, and video, transcribe audio, and run realtime voice. Keep the OpenAI, Anthropic, or Google AI SDK you already use. Change the base URL, nothing else. Point your app at a dynamic route and change its models in the platform, no deploy. Register your own provider deployments and API keys. They work like any other model. Hosted in the EU. Pin an EU route per call, or restrict every call to EU-only or zero-retention providers at the gateway. Every response carries its cost. Spend across all providers in one place. Every call is metered and listed in [Logs](/control-plane/logs): cost, latency, and tokens. Control Plane adds full traces. ## One API, every model Address any model as `provider/model`. The same call works whether you point it at OpenAI, Anthropic, or a model hosted in the EU. ```bash theme={null} openai/gpt-5 # OpenAI anthropic/claude-opus-4-7 # Anthropic mistral/mistral-large-2512 # Mistral, EU-hosted ``` Some models are served by more than one provider (for example the same Claude model via Anthropic, Azure, or Bedrock). Opper balances across them for you, so a single name keeps working even when one provider is busy. To call a model, point your SDK at `https://api.opper.ai/v3/compat` — see [Drop-in SDKs](/build/gateway/drop-in-sdks). The same endpoint covers text, [structured output](/build/gateway/structured-output), [tool calling](/build/gateway/tools), and [multimodal input](/build/multimodal/vision-pdfs), with the same routing, governance, and tracing on every call. The gateway isn't text-only. Dedicated endpoints generate [images](/build/multimodal/images) (`POST /v3/images`), [speech and transcripts](/build/multimodal/audio) (`POST /v3/audio/*`), and [video](/build/multimodal/video) (`POST /v3/videos`), and [realtime voice](/build/realtime/quickstart) runs over a WebSocket — all behind the same key and governance. See [Multimodality](/build/multimodal/overview). For classification and scoring, [decision models](/build/gateway/evaluations) answer in OpenAI's Decisions format at `POST /v3/compat/decisions` and in TypeSafe's System One format at `POST /v3/compat/v1/systemone`, returning probabilities, categories, and scores through the same project key. ## Pin and switch models without touching code Keep model names out of your application. Set a default once and change it without a deploy: * **[Default model](/control-plane/route)** sets the model for calls that don't name one, per organization or project. Edit one rule to move a project to a different model. * **[Custom models](/capabilities/custom-models)** let you register your own provider deployments and call them like any other model in the catalog. Callers can still pass an explicit `model` for per-call trade-offs, as long as [model access](/control-plane/rules/model-access) allows it. ## Bring your own keys You can go beyond Opper's hosted models. Register your own provider deployments and API keys (for example a private Azure OpenAI deployment), and they show up in the catalog alongside built-in models with the same routing, governance, and tracing. ```bash theme={null} curl -X POST https://api.opper.ai/v2/custom-models \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPPER_API_KEY" \ -d '{ "name": "example/my-gpt", "provider": "azure", "model_id": "gpt-production", "api_key": "your-provider-key", "params": {"api_base": "https://your-deployment.openai.azure.com/", "api_version": "2024-06-01"} }' ``` Call it like any other model: `example/my-gpt`. See [Custom models](/capabilities/custom-models) for the full setup. ## Keep data in the EU The gateway is hosted exclusively in the EU. The only thing that leaves is the model call itself, and you control where that goes. With a [model access](/control-plane/rules/model-access) rule you can restrict every call to: * EU-only routes, by where inference runs and where providers store content (see [EU data residency](/control-plane/eu-data-residency)) * Specific regions or countries * Providers that meet your data requirements, including the Zero data retention policy Opper storage is configured separately. Opper stores no traces until a retention rule turns tracing on. To stop storing new traces after that, switch tracing off in [Opper retention](/control-plane/rules/retention) so an enabled **0-day** retention rule applies. Usage records are still kept, and existing traces keep their original expiry dates. See that page for the effects on files and response caching. If you also need provider data restrictions, configure the provider data policy in [Model access](/control-plane/rules/model-access). Set both controls to meet both requirements: turning Opper tracing off does not restrict providers, and selecting a provider data policy does not turn Opper tracing off. Anything that breaks a rule is rejected at the gateway before it reaches a provider. See [Security](/overview/security) for more. ## One bill, full visibility Every response includes its `cost`, computed the same way across all 700+ models. The gateway aggregates spend and usage across providers, so you get one bill and one place to watch it. Every call is metered (cost, latency, tokens) and shows up in **Analytics** for spend over time. Turn tracing on with a [retention rule](/control-plane/rules/retention) (Control Plane) and you also get the full [trace](/overview/concepts) for each call, with inputs, outputs, and every step. ## How the Gateway and Rules work together The Gateway runs your requests. [Rules](/control-plane/rules/overview) defines the boundaries it enforces. | | AI Gateway | Rules | | - | - | - | | **Role** | Runs every request | Defines what requests may do | | **You use it by** | Making model calls | Configuring policies in the platform | | **Examples** | Fallbacks, provider access, usage metering | Retention, spend limits, model access, checks, routing | Configure a rule once; the Gateway applies it to every matching call. ## Security The gateway also enforces data residency. The next page covers how Opper is hosted and protected. EU hosting, sub-processors, encryption, and deletion. The full catalog. EU-hosted models marked. Ready to build? Text, multimodal, and voice. Use the OpenAI, Anthropic, or Google AI SDK you already have. # Quickstart Source: https://docs.opper.ai/overview/quickstart Make your first Opper call in under five minutes. You'll get an API key, pick an API, and run one call. The rest of the docs build on this. ## Make your first call Sign in at [platform.opper.ai](https://platform.opper.ai) and [create a key](https://platform.opper.ai/settings/api-keys). Each key belongs to one project, so make a new project for each app or environment. Sign-up needs no card, and the free models work right away. To call a premium model such as `openai/gpt-5-mini` below, add a card and credits under **Billing**. Put the key in your shell: ```bash theme={null} export OPPER_API_KEY="your-api-key" ``` Point your SDK at the gateway and make a call. Here it is with the OpenAI SDK — the same base-URL trick works with the Anthropic and Google AI SDKs too. ```python Python theme={null} import os from openai import OpenAI client = OpenAI( base_url="https://api.opper.ai/v3/compat", api_key=os.environ["OPPER_API_KEY"], ) response = client.chat.completions.create( model="openai/gpt-5-mini", messages=[{"role": "user", "content": "Hello!"}], ) print(response.choices[0].message.content) ``` ```typescript TypeScript theme={null} import OpenAI from "openai"; const client = new OpenAI({ baseURL: "https://api.opper.ai/v3/compat", apiKey: process.env.OPPER_API_KEY!, }); const response = await client.chat.completions.create({ model: "openai/gpt-5-mini", messages: [{ role: "user", content: "Hello!" }], }); console.log(response.choices[0].message.content); ``` ```bash cURL theme={null} curl https://api.opper.ai/v3/compat/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPPER_API_KEY" \ -d '{ "model": "openai/gpt-5-mini", "messages": [{"role": "user", "content": "Hello!"}] }' ``` The same trick works with the Anthropic and Google AI SDKs. See [Drop-in SDKs](/build/gateway/drop-in-sdks) for all three. Want typed JSON back instead of free text? Add `response_format` — see [Structured output](/build/gateway/structured-output). Open [platform.opper.ai](https://platform.opper.ai) to see your call, the model that ran it, the cost, and the latency. With tracing turned on (a Control Plane feature), click into the most recent trace to also see the inputs, outputs, and any rules that fired. ## What to do next A short guide to text, multimodal, and voice. Browse the 700+ models Opper supports and how to call them. Use the OpenAI, Anthropic, or Google AI SDK against Opper. Inspect calls, set rules, and manage routing. # Security and compliance Source: https://docs.opper.ai/overview/security ISO/IEC 27001:2022 certified and hosted in Stockholm. Certifications, plans with SSO and audit logs, retention defaults, DPA, sub-processors, and the controls that keep data in the EU. Opper is certified to ISO/IEC 27001:2022 and hosted in the EU, in Stockholm. Opper does not train on your data, keeps prompts and outputs in traces only for the retention period set in Rules (none on the Gateway plan), and can enforce EU-only inference and zero data retention for every call in your organization. Files you upload or store are kept until you delete them. Certificates, policies, and the current sub-processor list are on the trust center at [trust.opper.ai](https://trust.opper.ai). Security covers the gateway and the platform. The [AI Gateway](/overview/gateway) decides where calls can go at request time, and the platform is how everything underneath is hosted and protected. This page covers both. ## Certifications | Standard | Status | | - | - | | **ISO/IEC 27001:2022** | Certified. Certificate 160926-1, valid through 15 September 2029. | | **GDPR** | Opper is based in Stockholm and acts as your processor under a Data Processing Agreement. Standard Contractual Clauses are available on request. See [DPA and contact](#dpa-and-contact). | | **SOC 2** | Opper does not hold a SOC 2 report. Its information security management system is certified to ISO/IEC 27001:2022 instead. | The certificate and Opper's security policies are available at [trust.opper.ai](https://trust.opper.ai). ## EU-hosted by default Opper's primary production environment runs in AWS Stockholm (`eu-north-1`), with a second production environment at evroc, also in Stockholm. Opper's own data (traces, usage records, stored files, and backups) is hosted there. The one thing that leaves Opper's footprint is the model call itself, and you can constrain even that at the gateway. Use [model access](/control-plane/rules/model-access) rules to: * Restrict calls to EU routes, by inference location and by where providers store content * Pin specific countries that calls can route to * Require provider data practices, including the Zero data retention policy The gateway refuses any call that violates these rules before it leaves the platform. See [Keep AI inference in the EU](/control-plane/eu-data-residency) and [Enforce zero data retention](/control-plane/zero-data-retention). If your policy requires European ownership at every hop, not only EU hosting, Opper also runs as a separate sovereign instance on evroc, the Swedish cloud. See [Sovereign AI](/control-plane/sovereign-ai). ## We don't train on your data Opper never uses customer data to train models, and never shares it with providers for training. ## What each plan includes | | Gateway | Control Plane | Enterprise | | - | - | - | - | | **Prompts and outputs stored by Opper** | None. Usage metadata only. | None by default. Turn tracing on for 1 to 30 days, or enforce 0 days for the organization or a project. | Custom retention | | **Zero data retention** | ZDR routes, chosen per call | Enforced org-wide by rule | Enforced org-wide, custom terms | | **Model access** (allowlist, EU-only) | No rules; you choose the model in each call | Organization allowlist and project overrides | Custom | | **Checks** (flag, redact, block, score) | No | Yes | Custom | | **Spend limits** | Prepaid balance, auto top-up | Organization budget and project caps | Custom | | **DPA** | Standard DPA | Standard DPA | Custom terms | | **SSO / SAML** | No | No | Yes | | **Audit logging** | No | No | Yes | | **Role-based access** | No | Built-in roles (Admin, Developer, Viewer) | Built-in and custom roles | | **Custom hosting regions** | No | No | Yes | | **SLA** | No | Add-on | Yes | On every plan, Opper keeps a usage record for each call (cost, tokens, latency, status, provider, model, and any `X-Opper-Tags`, with no prompts or responses) for 5 years, for billing. [Logs](/control-plane/logs) lists these records call by call. See [opper.ai/pricing](https://opper.ai/pricing) for prices. ## Configure Opper storage and provider data requirements Opper trace storage is controlled by [Opper retention](/control-plane/rules/retention). With no applicable enabled retention rule, Opper records usage metadata but does not store traces. With tracing on, spans, inputs, and outputs are kept for the effective retention period, up to 30 days. Check your organization's saved rules to see each project's effective period. Provider data requirements are set separately, in [Model access](/control-plane/rules/model-access). Neither setting configures the other: turning Opper tracing off does not restrict providers, and selecting a provider data policy does not turn Opper tracing off. [Enforce zero data retention](/control-plane/zero-data-retention) walks through setting both. ## One sub-processor for every model Most AI vendors require a separate DPA amendment for every model provider you turn on. Opper doesn't. Opper is your one AI sub-processor, so you can add or remove models without rewriting contracts. * **Opper's platform sub-processors**, with their purpose and location, are listed on the trust center at [trust.opper.ai](https://trust.opper.ai). * **Model providers** are listed at [opper.ai/providers](https://opper.ai/providers), with each provider's locations, training and logging practices, and DPA status. You choose which ones your calls can reach with [model access](/control-plane/rules/model-access). With an EU-only model access rule, model calls run only on EU routes and Opper's own storage stays in Stockholm, so prompts and outputs are processed in the EU. ## Encryption * At rest: AWS RDS with KMS-managed keys. Uploaded files use S3 SSE-S3. * In transit: TLS on every public endpoint. * Backups: encrypted in AWS Backup. Daily snapshots kept 5 weeks, weekly snapshots 14 months. Only Opper engineers can restore. ## Data isolation Each organization's data is isolated at the application layer. Uploaded files live in a private S3 bucket with objects segregated per organization. Service-to-service traffic is restricted to a private AWS VPC. ## Deletion * Delete a project → all associated traces and events are removed. * Set [Opper retention](/control-plane/rules/retention) to 0 → new calls do not store traces. Existing traces keep their original expiry dates. * Uploaded files persist until you delete them (`DELETE /v3/files/{id}`, or an expiry you set). Scopes with an enabled 0-day Opper retention rule can't hold files: uploads are rejected, and existing files are scheduled for permanent deletion after you confirm the file count and save. A provider data policy in Model access does not trigger file deletion by itself. ## DPA and contact Standard DPA and Standard Contractual Clauses are available on request. Contact [hello@opper.ai](mailto:hello@opper.ai). * Full DPA: [opper.ai/data-processing-agreement](https://opper.ai/data-processing-agreement) * Trust center, sub-processors, and security policies: [trust.opper.ai](https://trust.opper.ai) ## Controls that put this in your code Enforce EU-only inference and storage for every call. Store no content at Opper and allow only providers that neither train on nor log it. Restrict which providers, regions, and models calls can reach. Choose how long content is kept, or keep none of it at all. Block or redact sensitive content before the model sees it. Cap what an organization can spend in a calendar month. ## Models See which models you can reach, and which ones are EU-hosted. The full catalog, with EU-hosted models marked. Use Opper as the provider for your editor, agent, or CLI. # Generate audio Source: https://docs.opper.ai/v3-api-reference/audio/create-audio-generation post /v3/audio/generations Generate audio from a text prompt with any audio generation model from `GET /v3/audio/models`: music (`type=music`) or sound effects and ambience (`type=sound`). By default runs synchronously and returns it inline as base64 (200). Set `async: true` to run it on the background worker and get a 202 with a status URL to poll instead; use it for long tracks, which can take minutes. `model` is required, and `prompt` too unless `sections` are sent; `duration_seconds`, `instrumental`, `format` (omit it for the model's native format) and `seed` are normalized, and a value a model cannot honor answers 400 naming the limit. `sections` (a timed plan: each section's `duration_ms`, `prompt` and `styles`) needs a model with the `audio_sections` capability, and `loop` one with `audio_loop`; a model without the capability answers 400 rather than ignoring the field. Everything in `parameters` is forwarded verbatim to the provider. Lyrics are returned when the provider produces them. Set `store: true` to also save the output to /v3/files and get a reusable `file_id`; nothing is stored otherwise. Music, songs, sound effects and ambience. See the [Audio guide](/build/multimodal/audio#music-and-sound-effects) for runnable examples, timed sections and loops. # Generate speech Source: https://docs.opper.ai/v3-api-reference/audio/create-speech post /v3/audio/speech Convert text to spoken audio synchronously and get it back inline as base64. `model` and `input` are required; `voice`, `format` (mp3 default), and `speed` are normalized and validated against the model's declared capabilities, and everything in `parameters` is forwarded verbatim to the provider. Set `store: true` to also save the output to /v3/files and get a reusable `file_id`; nothing is stored otherwise. See the [Audio guide](/build/multimodal/audio) for runnable examples and voices. # Transcribe audio Source: https://docs.opper.ai/v3-api-reference/audio/create-transcription post /v3/audio/transcriptions Transcribe audio to text. By default runs synchronously and returns the transcript inline (200); `audio` accepts a `file_`, an https URL, or a base64 data-URI (max 25MB decoded). Set `stream: true` to receive the transcript live as Server-Sent Events (`transcript.text.delta` chunks then a final `transcript.text.done`, terminated by `data: [DONE]`) — supported on streaming models like `mistral/voxtral-mini-2602`, a 400 otherwise. Set `async: true` to run on the background worker and get a 202 with a status URL to poll instead — use it for long recordings that can exceed the synchronous timeout; it accepts the same `audio` sources (an inline data-URI is staged server-side and deleted after transcription) and lifts the decoded-audio cap to 100MB (a `file_` or https URL is the best way to reach that; a data-URI that large needs a ~133MB request body). `stream` and `async` are mutually exclusive. `model` and `audio` are required; `language` and `prompt` are passed through as hints; `diarize` requests speaker labels (400 on models without diarization support); everything in `parameters` is forwarded verbatim to the provider. See the [Audio guide](/build/multimodal/audio) for runnable examples. # Create Voice Source: https://docs.opper.ai/v3-api-reference/audio/create-voice post /v3/audio/voices Clone a custom voice from a short reference sample (a few seconds of speech). `model` selects the provider to clone on (e.g. `mistral/voxtral-mini-tts-2603`) — the returned voice works with any of that provider's TTS models; `audio` is the reference sample as a `file_`, an https URL, or a base64 data-URI; `languages` and `gender` are optional hints. Returns an opaque `id` (`voice_…`) to pass as `voice` on POST /v3/audio/speech. The voice is scoped to your project — it is not visible to or usable by other projects — and the reference audio is not stored. Paid feature. # Delete Voice Source: https://docs.opper.ai/v3-api-reference/audio/delete-voice delete /v3/audio/voices/{id} Delete one of the project's custom voices. Removes it upstream at the provider too; the reference sample was never stored. # Get Voice Source: https://docs.opper.ai/v3-api-reference/audio/get-voice get /v3/audio/voices/{id} Get one of the project's custom voices by id. # List audio models Source: https://docs.opper.ai/v3-api-reference/audio/list-audio-models get /v3/audio/models List the models usable with the `/v3/audio` endpoints: a `type=tts,stt,music,sound` view of the model catalog. Same shape and query params as `/v3/models` (search, `include`, pagination), but always scoped to audio models. Narrow to one family with `type=tts` (POST /v3/audio/speech), `type=stt` (POST /v3/audio/transcriptions), `type=music` (POST /v3/audio/music) or `type=sound` (POST /v3/audio/generations). `params.tts` lists each speech model's `voices`, `default_voice`, and `max_length`; `params.stt` lists each transcription model's `languages`, `formats`, and whether it supports `diarize`. Discovery endpoint for [speech, transcription, music and sound effects](/build/multimodal/audio). Filter by kind with `?type=tts`, `?type=stt`, `?type=music` or `?type=sound`. # List Voices Source: https://docs.opper.ai/v3-api-reference/audio/list-voices get /v3/audio/voices Without `model`: the project's custom cloned voices, newest first. With `model` (a text-to-speech model id): every voice that model takes as `voice` on /v3/audio/speech, which is the provider's preset voices (`type: preset`) followed by the project's clones on that provider (`type: custom`). Presets are read from the provider when it can list them (`meta.presets_source: provider`, refreshed every few hours) and otherwise come from the model's declared voices (`meta.presets_source: catalog`). Only voices every account on the provider can use are listed as presets; no other customer's voices are ever shown. Paginated with the shared `meta`+`data` shape. # Chat Completions Source: https://docs.opper.ai/v3-api-reference/compatibility/chat-completions post /v3/compat/chat/completions OpenAI-compatible chat completions endpoint. Supports streaming via the stream parameter. # Count Tokens Source: https://docs.opper.ai/v3-api-reference/compatibility/count-message-tokens post /v3/compat/v1/messages/count_tokens Anthropic Messages token-counting compatible endpoint. Returns the number of input tokens a Messages request would consume, without creating a message. Mirrors Anthropic's POST /v1/messages/count_tokens (same request body as the Messages endpoint, minus the required max_tokens). The model resolves exactly as it does on the Messages endpoint, so every model a message accepts can be counted, and one a message would reject fails the same way (404 for an unknown model, 403 for one the allowlist or an entitlement denies). Claude models are counted exactly by their own backend, in the model's region. Every other model, and a dynamic route, gets Opper's estimate in the same shape: characters / 4 over the system prompt, messages and tool declarations, plus 1 (within about 20% on English prose), with a flat amount per image and 3,000 tokens per PDF page. The `X-Opper-Count-Source` response header says which. # Count Tokens Source: https://docs.opper.ai/v3-api-reference/compatibility/count-tokens post /v3/compat/v1beta/models/{model}:countTokens Google-compatible countTokens endpoint for pre-flight context-window sizing. The request body matches `:generateContent`. NOTE: Opper has no in-process Gemini tokenizer and returns a chars/4 heuristic (±~20% on English prose) — sufficient for "does my prompt fit?" checks but not for exact cost projection. # Decisions Source: https://docs.opper.ai/v3-api-reference/compatibility/create-decision post /v3/compat/decisions OpenAI Decisions-compatible synchronous evaluation. Send `model`, `input` (a string, or user messages with `input_text` and inline base64 `input_image` parts) and typed `questions` (`predicate`, `choice`, `score`); `answers` come back in question order with probabilities and confidence; a question the model declines comes back as a `refusal` answer. A bare model name is OpenAI's, e.g. `gpt-6-luna`, which is called natively and answers with OpenAI's body unchanged. Other decision models are addressed by their full catalog ID, e.g. `typesafe/jev-latest`, `opper/kev-4b` or `greenpt/metis` (list available endpoints with `GET /v3/models?type=evaluation`); those are reached by translation to System One and accept text input only, except `empiriolabs/aplomb-1`, which takes the images too. GreenPT Metis receives all 1–32 questions together in one upstream request. Translation preserves question order, probabilities and confidence, but joins text-message boundaries and does not expose native impact, score legends or additional answer metadata; use the System One endpoint for the native contract. Billed on input tokens. Uses a project-scoped Opper API key, Comply policies and provider entitlements. Cost and generation ID are returned in headers. Streaming is unsupported. See the [classification and scoring guide](/build/gateway/evaluations) for working examples, model discovery, and usage and billing behavior. # Embeddings Source: https://docs.opper.ai/v3-api-reference/compatibility/create-embeddings post /v3/compat/embeddings OpenAI-compatible embeddings endpoint. # Interactions Source: https://docs.opper.ai/v3-api-reference/compatibility/create-interaction post /v3/compat/v1beta/interactions Google Interactions API compatible endpoint. Supports streaming via the stream parameter. # Messages Source: https://docs.opper.ai/v3-api-reference/compatibility/create-message post /v3/compat/v1/messages Anthropic Messages API compatible endpoint. Supports streaming via the stream parameter. # Responses Source: https://docs.opper.ai/v3-api-reference/compatibility/create-response post /v3/compat/responses OpenAI Responses API compatible endpoint. Supports streaming via the stream parameter. # Generate Content Source: https://docs.opper.ai/v3-api-reference/compatibility/generate-content post /v3/compat/v1beta/models/{model}:generateContent Google Gemini-compatible generateContent endpoint — the drop-in surface for SDKs calling `models.generate_content()`. The request body is the Gemini `GenerateContentRequest` shape (camelCase `contents` / `systemInstruction` / `generationConfig`); the response is `candidates[].content.parts[]` + `usageMetadata`. The `{model}` path segment accepts a bare Gemini id (e.g. `gemini-2.5-flash`, defaults to the Gemini-direct provider) or an Opper-canonical id (e.g. `vertexai/gemini-2.5-flash`). # List Models Source: https://docs.opper.ai/v3-api-reference/compatibility/list-models get /v3/compat/models OpenAI Models API compatible endpoint. Returns everything the caller can put in `model` — in OpenAI's models.list shape — so tools that point an OpenAI-compatible base URL at `/v3/compat` (gateways, n8n, LangChain, etc.) can discover the live catalogue without hardcoding model IDs. Every entry carries `opper.kind`, which names what it is: - `model` — a concrete catalog row (LLM or embedding), e.g. `anthropic/claude-sonnet-4.5`. Pinning this id pins that provider. - `pool` — a bare model name that load-balances across providers, e.g. `claude-sonnet-4.5`. `opper.members` lists the catalog rows it routes over. Listed only for names that genuinely fan out (two or more routable members). - `dynamic_route` — one of the calling org's deployed routing graphs, addressed as `dynamic/`. `opper.version` is the deployed version a call would execute. Org-scoped to the API key; draft-only routes are not listed. No pricing is reported — the graph picks the model per request — but `context_length` and `opper.max_output_tokens` ARE, as the smallest any reachable candidate offers: a floor is a guarantee rather than a guess, and clients that must size a context before calling would otherwise have to invent one. Omitted when any candidate cannot be resolved here (e.g. an org-scoped BYOK model). Narrow with `?type=` (comma-separated, e.g. `?type=pool,dynamic_route`); omitting it returns all three kinds, with concrete models first (their long-standing order is unchanged), then pools, then routes. Entries additionally carry OpenRouter-shaped `pricing` (USD per token) and `context_length`, plus an `opper` block holding the entry's `type` (`llm` or `embedding` — this list carries both, and `capabilities` cannot tell them apart), the model's `capabilities` (`text`, `tools`, `vision`, `pdf`, `structured_output`, … — the same vocabulary as `/v3/models`), its `max_output_tokens`, selectable effort controls under `opper.reasoning` (`supported` and optional `default`), and a compliance summary (`region`, `country`, `zdr`, `gdpr_residency`, `verification`) so a client can declare what a model can do and filter on residency/ZDR criteria in one call; `zdr` is the tri-state retention facts object (`logging`, `moderation`, `caching`, `training`, `subprocessors`; `null` = not established), not a summary word — derive what you need from it; full data-handling detail lives on `/v3/models?include=route`. A pool reports a capability or compliance property only when EVERY member has it, takes the smallest `context_length` / `max_output_tokens` across members, and prices at the worst case. Pool effort controls are the intersection of all allowed members' declared ladders; `supported: []` means no portable selectable effort, even if members think internally. The default is present only if all members agree on a supported value. Concrete rows without declared effort controls and dynamic routes omit `opper.reasoning`. `none` is an explicit off-switch only when listed; omitting an effort leaves provider defaults in control. These fields do not change routing. This means a capability check, a residency filter or a pre-call cost estimate can never be flattered by one lucky member. `dynamic_route` entries carry no capabilities for the same reason they carry no pricing. The whole listing is scoped to the caller's comply allowlist: denied models, denied pool members, pools with no permitted member, and routes whose every model node is denied are all absent. Featured models first, then ID-ascending; no pagination, matching OpenAI. # OpenResponses Source: https://docs.opper.ai/v3-api-reference/compatibility/openresponses post /v3/compat/openresponses OpenResponses-compatible endpoint with multi-provider support and opper: extension items. # Stream Generate Content Source: https://docs.opper.ai/v3-api-reference/compatibility/stream-generate-content post /v3/compat/v1beta/models/{model}:streamGenerateContent Streaming variant of generateContent. Pass `?alt=sse` (as Google's SDKs do) to receive Server-Sent Events; each `data:` line is a partial `GenerateContentResponse`. The request body matches `:generateContent`. # System One evaluation Source: https://docs.opper.ai/v3-api-reference/compatibility/systemone post /v3/compat/v1/systemone System One-compatible synchronous evaluation. Send `model`, `state` and named typed `questions` (`noul`, `choice`, `score`). Native model names such as `jev-latest` resolve to the TypeSafe catalog endpoint; fully-qualified `typesafe/...` IDs pin that endpoint. Other decision models require their full catalog ID, for example `opper/kev-4b` or `greenpt/metis`; list available endpoints with `GET /v3/models?type=evaluation`. GreenPT Metis jointly evaluates 1–32 questions in one request; state is required but may be null, and instructions may be omitted or null. Its nullable descriptions, probabilities, confidence, score legends and native `impact` metadata are preserved. EmpirioLabs Aplomb (`empiriolabs/aplomb-1`) takes 1–128 questions and images, audio and video as media nodes anywhere in `state`: an object with `type` `image`, `audio` or `video` and either a public `https://` `url` or base64 `data` with its `mime`. Other providers retain their own validation contracts. Returns native `model`, `answers` and token `usage`. Uses a project-scoped Opper API key, Comply policies and provider entitlements. Cost and generation ID are returned in headers. Streaming, chat generation and tool calling are unsupported. See the [classification and scoring guide](/build/gateway/evaluations) for working examples, model discovery, and usage and billing behavior. # Delete a file Source: https://docs.opper.ai/v3-api-reference/files/delete-file delete /v3/files/{id} Delete a file (removes the stored bytes). See the [Files guide](/build/multimodal/files). # Get file metadata Source: https://docs.opper.ai/v3-api-reference/files/get-file get /v3/files/{id} Get a file's metadata by id. See the [Files guide](/build/multimodal/files). # Get a file download URL Source: https://docs.opper.ai/v3-api-reference/files/get-file-content get /v3/files/{id}/content Returns a short-TTL presigned download URL for the file's bytes. Returns a short-TTL presigned download URL for the file's bytes. See the [Files guide](/build/multimodal/files). # List files Source: https://docs.opper.ai/v3-api-reference/files/list-files get /v3/files List files for the project, newest first. Paginated with the shared `meta`+`data` shape. Optionally filter by `type`, `purpose`, or a `q` substring search. See the [Files guide](/build/multimodal/files). # Set or clear a file's expiry Source: https://docs.opper.ai/v3-api-reference/files/update-file patch /v3/files/{id} Set or clear a file's expiry. `ttl_seconds` is the only mutable field: a positive integer schedules the file to expire that many seconds from now; an explicit `null` clears a previously set expiry (files are permanent by default). See the [Files guide](/build/multimodal/files). # Upload a file Source: https://docs.opper.ai/v3-api-reference/files/upload-file post /v3/files Upload a file (multipart/form-data) and get back a `file_`. The id can be passed as the `image`/`video`/`reference_images` source on POST /v3/videos. `purpose` defaults to `reference_media` (accepts common image/video types). The MIME type is detected from the bytes; size and per-org quota are enforced. Files are permanent until deleted; pass `ttl_seconds` to schedule an expiry. Returns a `file_` you can pass as a media source on later calls. See the [Files guide](/build/multimodal/files). # Delete generation Source: https://docs.opper.ai/v3-api-reference/generations/delete-generation delete /v3/generations/{id} Delete a specific recorded generation. # Get generation Source: https://docs.opper.ai/v3-api-reference/generations/get-generation get /v3/generations/{id} Get a specific recorded generation by ID. # List generations Source: https://docs.opper.ai/v3-api-reference/generations/list-generations get /v3/generations List recorded HTTP request/response generations with pagination. # Generate an image Source: https://docs.opper.ai/v3-api-reference/images/create-image post /v3/images Generate one or more images. By default runs synchronously and returns them inline (200). Set `async: true` to run a single image on the background worker and get a 202 with a status URL to poll instead — use it for slow models (e.g. gpt-image high, imagen ultra) that can exceed the synchronous timeout. `model` and `prompt` are required; `n` (max 4; async is single-image), `size`/`aspect_ratio`, `quality`, and `style` are normalized, and everything in `parameters` is forwarded verbatim to the provider. `image`/`mask`/`reference_images` accept an http(s) URL, a data-URI, or a `file_` for edit / image-to-image. `response_format` defaults to `b64_json`. Set `store: true` to also save the outputs to /v3/files and get a reusable `file_id`; nothing is stored otherwise. See the [Images guide](/build/multimodal/images) for runnable examples, edit / image-to-image, and model discovery. # List image models Source: https://docs.opper.ai/v3-api-reference/images/list-image-models get /v3/images/models List the models usable with `POST /v3/images` — scoped to the `image_generation` capability. Unlike the video/audio model lists (which scope by `type`), this scopes by capability, because `/v3/images` routes on the `image_generation` capability: the result spans `type:image` models AND hybrid `type:llm` chat-image models (e.g. Nano Banana) that produce images natively. Same shape and query params as `/v3/models` (search, `include`, pagination). `params.image` lists each model's accepted `sizes` (pixel-size models) or `aspect_ratios` (aspect-ratio models) and `qualities`. Discovery endpoint for [image generation](/build/multimodal/images). # Mint an API key Source: https://docs.opper.ai/v3-api-reference/management/create-api-key post /management/v1/projects/{uuid}/api-keys Mints a runtime API key for the project. The full key is returned only on initial creation when return_secret is not false; the user MCP tool defaults false. Reuse idempotency_key with identical arguments after uncertain results. A replay returns existing metadata without a secret; changed arguments or a deleted key return 409. Requires the `apikeys:write` scope. # Create a control rule Source: https://docs.opper.ai/v3-api-reference/management/create-control-rule post /management/v1/controls/rules Creates a rule with a client-generated UUID. Setting enabled applies it immediately. Zero-day retention or ZDR may permanently delete files; a 409 warning requires explicit confirm_file_deletion acknowledgement. Read rules again to inspect effective policy. Requires the `controls:write` scope. # Create a dynamic route Source: https://docs.opper.ai/v3-api-reference/management/create-dynamic-route post /management/v1/dynamic-routes Creates a route draft. Omit graph to seed a complete default graph, then use data.draft.graph as a starting point. Empty graphs are invalid. Omit deploy or set deploy=false to keep the route unpublished; deploy=true changes live routing immediately. Review data.draft.validation, save changes with update_dynamic_route_draft, then use simulate_dynamic_route without version_number to test the draft before deploy_dynamic_route. Invoke a deployed route with model=dynamic/; invocation_model contains the exact string. Requires the `dynamic_routes:write` scope. Graph format: an object with an `elements` array. Each element has a unique `id`, a `type`, optional `properties`, and `outputs`, an object mapping output names to `{"elementId":"target-node-id"}`. Include `id: START, type: start` and `id: END, type: end`; `input` and `output` are not node types. A minimal deployable graph (replace the model with an available catalog ID permitted by your policy): ```json {"elements":[{"id":"START","type":"start","outputs":{"next":{"elementId":"primary"}}},{"id":"primary","type":"model","properties":{"model":"cerebras/gpt-oss-120b"},"outputs":{"success":{"elementId":"END"},"fallback":{"elementId":"END"}}},{"id":"END","type":"end","outputs":{}}]} ``` Supported node types and properties: - `start`: the START node; `outputs.next` selects the first node. - `end`: the END node; `outputs` is `{}`. - `model`: `properties.model` is a catalog model ID. `outputs.success` targets END or an observe node; `outputs.fallback` targets another model, pool, or END. - `pool`: `properties.models` contains 2-10 `{model,metrics}` candidates; `properties.expression` is a numeric CEL expression such as `model.cost`, sorted ascending. Uses the same success/fallback outputs as model. - `conditional`: `properties.cases` contains `{id,expression}` entries; each expression is a CEL predicate over `metadata`, `auth`, and `input`. Connect an output for each case ID and a `default` output. The first true case wins. - `percentage`: `properties.buckets` contains at least two `{id,weight}` entries with positive integer weights totaling 100. Connect an output for each bucket ID. - `classifier`: `properties.cases` contains 2-10 `{id,label,instructions}` entries with unique labels, plus optional `context` and `model` properties. Connect an output per case ID; no default output. Classification failures fail the route. - `observe`: `properties.criteria` defines an asynchronous evaluation, with optional `score_type` (`score` or `binary`), `model`, `threshold`, and `sample_rate` (both greater than 0 and at most 1). Place after model/pool success or another observe node; `outputs.next` leads to another observe node or END. Graphs must be acyclic. Every node except END accepts at most one incoming edge; use separate model nodes on separate branches. Deployment requires each path from START to reach a model or pool before END. The route name is the model identifier on chat completions, responses, messages and native calls. # Invite a member Source: https://docs.opper.ai/v3-api-reference/management/create-invite post /management/v1/invites Invites an address into the organization and emails the invitation. The response carries the acceptance token and link as well, so a caller that would rather deliver it itself can, and `email_sent` reports whether the mail was queued. A live invitation for the same address is resent rather than duplicated: the token stays valid, the expiry moves out, and the reply is 200 instead of 201. Send `idempotency_key` to retry safely after an uncertain result. A replay answers 200 with the same invitation and `meta.idempotency_replayed`, mails nobody and leaves the expiry where it was; the same key with different arguments is 409. Omit it when you mean to resend. `role` is available on Enterprise; elsewhere the invitee joins with the plan's default member role. Treat `token` and `invite_url` as secrets: anyone holding one can join the organization. Requires the `members:write` scope. # Create a project Source: https://docs.opper.ai/v3-api-reference/management/create-project post /management/v1/projects Creates a project in the calling organization. Requires the `projects:write` scope. # Delete an API key Source: https://docs.opper.ai/v3-api-reference/management/delete-api-key delete /management/v1/projects/{uuid}/api-keys/{id} Permanently deletes a runtime API key. Requires the `apikeys:write` scope. # Delete a control rule Source: https://docs.opper.ai/v3-api-reference/management/delete-control-rule delete /management/v1/controls/rules/{uuid} Deletes a rule and stops its enforcement; inherited rules still apply. Requires the `controls:write` scope. # Delete a dynamic route Source: https://docs.opper.ai/v3-api-reference/management/delete-dynamic-route delete /management/v1/dynamic-routes/{name} Soft-deletes the route. Calls to `dynamic/` fail from the next request on. Requires the `dynamic_routes:write` scope. # Delete a project Source: https://docs.opper.ai/v3-api-reference/management/delete-project delete /management/v1/projects/{uuid} Soft-deletes the project and permanently deletes every runtime API key bound to it. Requires the `projects:delete` scope. # Deploy the draft Source: https://docs.opper.ai/v3-api-reference/management/deploy-dynamic-route post /management/v1/dynamic-routes/{name}/deploy Validates and deploys the current draft as a new immutable version, changing live routing. Returns the deployed version in meta.deployed_version. Requires the `dynamic_routes:write` scope. # Disable the calling management key Source: https://docs.opper.ai/v3-api-reference/management/disable-management-key post /management/v1/management-keys/_self/disable Emergency revocation for a leaked token: disables the management API key the request authenticated with, and every token under it. Possession of the token is the authorization — no scope is required, and this works regardless of the organization's plan. # Get a dynamic route Source: https://docs.opper.ai/v3-api-reference/management/get-dynamic-route get /management/v1/dynamic-routes/{name} Returns the route, draft graph, validation issues and active version. Requires the `dynamic_routes:read` scope. # Get a version Source: https://docs.opper.ai/v3-api-reference/management/get-dynamic-route-version get /management/v1/dynamic-routes/{name}/versions/{version_number} Returns a historical deployed version, including its graph and layout. Requires the `dynamic_routes:read` scope. # Get a project Source: https://docs.opper.ai/v3-api-reference/management/get-project get /management/v1/projects/{uuid} Fetches one project by UUID. Requires the `projects:read` scope. # List a project's API keys Source: https://docs.opper.ai/v3-api-reference/management/list-api-keys get /management/v1/projects/{uuid}/api-keys Lists the runtime API keys bound to a project. Only the key prefix is returned — the full key is shown once, at creation. Requires the `apikeys:read` scope. # List control rules Source: https://docs.opper.ai/v3-api-reference/management/list-control-rules get /management/v1/controls/rules Lists rules in the calling organization. Project filters include inherited organization rules and matching function rules. Allowlist rules include effective_count and parent_allowlist; route rules may include allowed_model_ids. Requires the `controls:read` scope. # List a route's versions Source: https://docs.opper.ai/v3-api-reference/management/list-dynamic-route-versions get /management/v1/dynamic-routes/{name}/versions Lists immutable deployed-version summaries for a route. Requires the `dynamic_routes:read` scope. # List dynamic routes Source: https://docs.opper.ai/v3-api-reference/management/list-dynamic-routes get /management/v1/dynamic-routes Lists routes in the calling organization, including whether drafts differ from the deployed version. Requires the `dynamic_routes:read` scope. meta.default_classifier_chain is the model chain a classifier uses when it does not pin one. meta.default_first_token_timeout_ms is the first-token timeout a model or pool node runs on when its arguments state none. # List invitations Source: https://docs.opper.ai/v3-api-reference/management/list-invites get /management/v1/invites Lists the organization's invitations. Pass `pending=true` for the unaccepted ones, `pending=false` for the accepted. Requires the `members:read` scope. # List members Source: https://docs.opper.ai/v3-api-reference/management/list-members get /management/v1/members Lists every member of the calling organization with their effective role, and marks the one member ownership resolves to. Requires the `members:read` scope. # List projects Source: https://docs.opper.ai/v3-api-reference/management/list-projects get /management/v1/projects Lists every project in the calling organization. Requires the `projects:read` scope. # Remove a member Source: https://docs.opper.ai/v3-api-reference/management/remove-member delete /management/v1/members/{user_uuid} Removes the person's membership in the calling organization and revokes the credentials that membership issued: their runtime API keys in this organization, the OAuth authorizations behind those keys, and their agent connections scoped to it. This is not a user deletion. The account remains, as does everything they hold in any other organization. The organization's owner cannot be removed (409); transfer ownership in the platform first. Requires the `members:write` scope. # Copy a version into the draft Source: https://docs.opper.ai/v3-api-reference/management/restore-dynamic-route-version-to-draft post /management/v1/dynamic-routes/{name}/versions/{version_number}/restore-to-draft Replaces the current draft with a historical version without changing live routing. Review the returned draft validation, then simulate the draft by omitting version_number before explicitly deploying. Requires the `dynamic_routes:write` scope. # Revoke an invitation Source: https://docs.opper.ai/v3-api-reference/management/revoke-invite delete /management/v1/invites/{id} Deletes an invitation, so its link stops working. Revoking an accepted invitation removes the record of the acceptance, not the membership it produced. Requires the `members:write` scope. # Roll back to a version Source: https://docs.opper.ai/v3-api-reference/management/rollback-dynamic-route post /management/v1/dynamic-routes/{name}/versions/{version_number}/rollback Publishes a historical version as a new deployed version and changes live routing immediately. Returns the new version in meta.deployed_version. Requires the `dynamic_routes:write` scope. The draft is left unchanged. # Simulate a request through the route Source: https://docs.opper.ai/v3-api-reference/management/simulate-dynamic-route post /management/v1/dynamic-routes/{name}/simulate Omit version_number or set it to null to simulate the current draft, including an unpublished route. To simulate history, supply a positive version_number from list_dynamic_route_versions. Zero is invalid and does not mean draft. Minimal draft call: {"name":"support","version_number":null}. Optional input and metadata supply request context; overrides.model_outcomes maps model/pool node IDs to success or fallback, and overrides.classifier_cases maps classifier node IDs to case IDs. For example {"name":"support","version_number":null,"overrides":{"model_outcomes":{"primary":"success"}}} tests the documented example graph. May call a live classifier when no override is provided; does not deploy or invoke the selected final model. Inspect data.status and data.path for the routing verdict. Fix draft issues with update_dynamic_route_draft before deployment. Requires the `dynamic_routes:write` scope. # Update a control rule Source: https://docs.opper.ai/v3-api-reference/management/update-control-rule patch /management/v1/controls/rules/{uuid} Updates name, enabled, config or schema_version; omit fields to keep them and use name:null to clear a name. Config is replaced, not merged. ID, kind and scope are immutable. An empty update is rejected. Enabling zero-day retention or ZDR can permanently delete files and may require acknowledgement after a 409 warning. Requires the `controls:write` scope. # Replace the draft Source: https://docs.opper.ai/v3-api-reference/management/update-dynamic-route-draft put /management/v1/dynamic-routes/{name}/draft Saves draft changes without deploying. Omitted description, graph and layout are preserved; an empty description clears it. Supply the complete graph to replace it. Structural errors reject the save; inspect meta.issues and data.draft.validation for remaining issues before deployment. Use simulate_dynamic_route without version_number to test the saved draft; deploy_dynamic_route performs full deployment validation. Requires the `dynamic_routes:write` scope. Graph format: an object with an `elements` array. Each element has a unique `id`, a `type`, optional `properties`, and `outputs`, an object mapping output names to `{"elementId":"target-node-id"}`. Include `id: START, type: start` and `id: END, type: end`; `input` and `output` are not node types. A minimal deployable graph (replace the model with an available catalog ID permitted by your policy): ```json {"elements":[{"id":"START","type":"start","outputs":{"next":{"elementId":"primary"}}},{"id":"primary","type":"model","properties":{"model":"cerebras/gpt-oss-120b"},"outputs":{"success":{"elementId":"END"},"fallback":{"elementId":"END"}}},{"id":"END","type":"end","outputs":{}}]} ``` Supported node types and properties: - `start`: the START node; `outputs.next` selects the first node. - `end`: the END node; `outputs` is `{}`. - `model`: `properties.model` is a catalog model ID. `outputs.success` targets END or an observe node; `outputs.fallback` targets another model, pool, or END. - `pool`: `properties.models` contains 2-10 `{model,metrics}` candidates; `properties.expression` is a numeric CEL expression such as `model.cost`, sorted ascending. Uses the same success/fallback outputs as model. - `conditional`: `properties.cases` contains `{id,expression}` entries; each expression is a CEL predicate over `metadata`, `auth`, and `input`. Connect an output for each case ID and a `default` output. The first true case wins. - `percentage`: `properties.buckets` contains at least two `{id,weight}` entries with positive integer weights totaling 100. Connect an output for each bucket ID. - `classifier`: `properties.cases` contains 2-10 `{id,label,instructions}` entries with unique labels, plus optional `context` and `model` properties. Connect an output per case ID; no default output. Classification failures fail the route. - `observe`: `properties.criteria` defines an asynchronous evaluation, with optional `score_type` (`score` or `binary`), `model`, `threshold`, and `sample_rate` (both greater than 0 and at most 1). Place after model/pool success or another observe node; `outputs.next` leads to another observe node or END. Graphs must be acyclic. Every node except END accepts at most one incoming edge; use separate model nodes on separate branches. Deployment requires each path from START to reach a model or pool before END. # Update a project Source: https://docs.opper.ai/v3-api-reference/management/update-project patch /management/v1/projects/{uuid} Updates a project's name and/or description. Omit a field to leave it unchanged; send `"description": null` to clear it. An empty body is rejected. Retention is not settable here — it is a control-plane setting, managed through `/management/v1/controls/rules`. Requires the `projects:write` scope. # List models Source: https://docs.opper.ai/v3-api-reference/models/list-models get /v3/models Endpoint to retrieve model info. Optional `include` adds extra fields per item: `maker` (underlying-weights creator, e.g. `meta` for Llama), `route` (data-handling / legal block — requires an Opper API key), `benchmarks` (public scores from artificialanalysis.ai; absent for realtime, OCR, image-gen, and other specialty models), `policy` (requires an Opper API key — annotates each item with the caller's effective comply allowlist verdict: `{"allowed": bool, "blocked_by": "org"|"project"}`; the list itself stays unfiltered). Comma-separated, e.g. `?include=maker,benchmarks`. `inference_location`, `storage_location`, `training`, `logging` and `moderation` filter the list the way a model access rule on the Rules page does, so the result is what a rule with the same settings would allow. Values within one parameter are alternatives; the parameters combine. For models that run in the EU use `?inference_location=EU`; for the Rules page's Zero data retention preset use `?training=no&logging=none`. # Extract text (OCR) Source: https://docs.opper.ai/v3-api-reference/ocr/extract-text post /v3/ocr Extract text from a document or image synchronously and get it back as per-page markdown. `model` and `document` are required; `document` is one of `{type:"document_url", document_url}`, `{type:"image_url", image_url}`, `{type:"base64", content, document_name}` (PDF or image), or `{type:"file", file_id}` for a `file_` from POST /v3/files. `pages` selects 0-based page indices; `include_image_base64`/`image_limit`/`image_min_size` control embedded-image extraction; everything in `parameters` is forwarded verbatim to the provider (e.g. Docling's `ocr_engine`, `lang`, `table_mode`, `do_formula_enrichment`). Billed per page processed. See the [OCR guide](/build/multimodal/ocr) for runnable examples, document sources (URL / base64 / `file_id`), and language options. # List OCR models Source: https://docs.opper.ai/v3-api-reference/ocr/list-ocr-models get /v3/ocr/models List the models usable with `POST /v3/ocr` — a `type=ocr` view of the model catalog. Same shape and query params as `/v3/models` (search, `include`, pagination). Each item's `price_per_page` describes the per-page cost. See the [OCR guide](/build/multimodal/ocr) for runnable examples and per-page pricing. # Mint realtime ticket Source: https://docs.opper.ai/v3-api-reference/realtime/create-realtime-session Server-to-server endpoint that mints a short-lived, single-use ticket for browser realtime sessions. ``` POST /v3/realtime-sessions ``` Returns a single-use `client_secret` your browser uses to open a `/v3/realtime` WebSocket. Browsers can't set an `Authorization` header on a WebSocket constructor, so this endpoint exists to bridge the gap: the customer's backend authenticates with its normal API key, optionally pre-binds session config fields, and returns the ticket to the browser. See the [Realtime voice guide](/capabilities/realtime#browser-ephemeral-tickets) for the end-to-end flow. ## Authentication Standard bearer auth with a project-scoped runtime API key. Management keys (`op-mak-…`) are rejected. ``` Authorization: Bearer ``` ## Request body | Field | Type | Description | | - | - | - | | `config` | object | Optional. Any non-zero fields are **bound** to the ticket and cannot be overridden when the browser sends `session.start` or `session.update`. See [session config reference](/capabilities/realtime#configuring-the-session). | | `locked_fields` | string\[] | Optional. Names of config fields that must be enforced **even when their value in `config` is the zero value** — use this to force a boolean off (e.g. `["output_transcription"]` with `config.output_transcription = false` prevents the browser from enabling it). Field names match the JSON tag on the config schema. A `locked_fields` list with **no `config` at all** is a valid "unconditionally forbid these" shape: each listed field is force-set to its zero value. Unknown field names are rejected with `400`. | | `ttl_seconds` | integer | Optional. Ticket lifetime in seconds. Default `60`, max `300`. Values outside the range are clamped. | ### Pre-binding Bound fields win over whatever the browser sends — this is the safety guarantee tickets provide. Recommended minimum binding: `model`. Tighter setups also bind `instructions`, `tools`, and `voice`. ```json theme={null} { "config": { "model": "openai/gpt-realtime-2", "voice": "marin", "instructions": "You are a concise voice assistant.", "tools": [ { "name": "lookup_order", "description": "...", "parameters": { /* ... */ } } ] }, "ttl_seconds": 60 } ``` ## Response | Field | Type | Description | | - | - | - | | `client_secret` | string | Opaque single-use ticket. Treat as a short-lived bearer credential and discard after one WebSocket upgrade. | | `expires_at` | string (RFC3339) | UTC timestamp after which the ticket is rejected even if unused. | | `ws_url` | string | Optional. Pre-built WebSocket URL including the ticket query string. May be empty. | ```json theme={null} { "client_secret": "Z2VtaW5pLWFkLWhvY...", "expires_at": "2026-05-13T15:32:00Z", "ws_url": "wss://api.opper.ai/v3/realtime?ticket=Z2VtaW5pLWFkLWhvY..." } ``` ## Errors | Status | Cause | | - | - | | `400 Bad Request` | Invalid JSON; pre-bound `model` is unknown or not a realtime model; `locked_fields` contains a name that doesn't match any config field (typo guard — error names the offending value). | | `401 Unauthorized` | Missing, non-runtime, or project-less API key. | | `503 Service Unavailable` | Realtime tickets not enabled on this deployment. | ## Redemption The browser presents the ticket on the WebSocket upgrade. Two transports are accepted; **prefer the subprotocol header** — credentials in the URL query string end up in access logs, browser history, and `Referer` headers, while the subprotocol header is request-only. **Recommended (subprotocol header):** ``` GET /v3/realtime Sec-WebSocket-Protocol: opper-ticket. ``` ```typescript theme={null} new WebSocket("wss://api.opper.ai/v3/realtime", [`opper-ticket.${clientSecret}`]); ``` **Fallback (query parameter)** — only for environments that can't set a subprotocol: ``` GET /v3/realtime?ticket= ``` Replays return `401`. ## See also * **[Realtime protocol](/v3-api-reference/realtime/protocol)** — full WebSocket event vocabulary. * **[Realtime voice guide](/capabilities/realtime)** — pre-binding, lifecycle, billing. # Realtime protocol Source: https://docs.opper.ai/v3-api-reference/realtime/protocol WebSocket event vocabulary for the Opper Realtime API. See the full guide for end-to-end usage. The Opper Realtime API is a bidirectional WebSocket protocol — it can't be fully represented in OpenAPI, so this page documents the event vocabulary. For the end-to-end journey (authentication, session lifecycle, tool flow, per-provider notes, billing), see the **[Realtime voice guide](/capabilities/realtime)**. ## Connection ``` wss://api.opper.ai/v3/realtime ``` Two authentication paths are accepted on the upgrade request: * **Server-side (bearer token):** `Authorization: Bearer `. * **Browser (ephemeral ticket):** `Sec-WebSocket-Protocol: opper-ticket.` subprotocol header (recommended), or `?ticket=` query parameter (fallback — bearer credentials in URLs end up in access logs). Tickets are minted by [`POST /v3/realtime-sessions`](/v3-api-reference/realtime/create-realtime-session) and are single-use. Successful upgrade returns `101 Switching Protocols`. The first frame the client sends must be `session.start`. ## Client → server events | Event | Required fields | Purpose | | - | - | - | | `session.start` | `config.model` | Open the session. Must be the first frame. | | `session.update` | `config` | Update session config mid-stream. Capability validation re-runs after any ticket overlay — unsupported `modalities` / `voice` / `reasoning_effort` values are rejected before reaching the upstream. **OpenAI rejects mid-session changes** to `input_transcription` and `input_transcription_model` (one-shot at start). Gemini ignores `session.update` entirely. | | `audio.append` | `audio` (base64 PCM16) | Stream a chunk of microphone audio. | | `audio.commit` | — | Mark end of speech turn (when not using server VAD). | | `audio.clear` | — | Discard buffered uncommitted audio. | | `text.input` | `text` | Send a typed user message. | | `response.create` | — | Force a model response now. | | `response.cancel` | — | Cancel an in-flight response. | | `tool.result` | `tool_call_id`, `tool_result` | Return a function call result the model requested. | ### `session.start` config | Field | Type | Notes | | - | - | - | | `model` | string | Required. Provider-prefixed id, e.g. `openai/gpt-realtime-2`, `xai/grok-voice-latest`, `gemini/gemini-3.1-flash-live-preview`. | | `voice` | string | Provider-specific voice id. | | `instructions` | string | System prompt. | | `modalities` | string\[] | Output modalities. OpenAI realtime accepts a single value — `["audio"]` or `["text"]`, not both. xAI and Gemini accept either form. Values outside the resolved model's capabilities are rejected before the upstream is dialed. | | `turn_detection` | object | `{ type, threshold, prefix_padding_ms, silence_duration_ms }`. | | `tools` | object\[] | Function-calling schema. | | `reasoning_effort` | string | `gpt-realtime-2` only. `minimal` / `low` / `medium` / `high` / `xhigh`. | | `input_transcription` | bool | Surface `transcript.committed` events. Off by default. | | `input_transcription_model` | string | OpenAI only — selects the transcription model. | | `output_transcription` | bool | Surface assistant-speech `text.delta` events. Off by default. | ## Server → client events | Event | Fields | Purpose | | - | - | - | | `session.started` | `session_id`, `input_sample_rate`, `output_sample_rate`, `audio_format` | Upstream session is live. | | `audio.delta` | `audio` (base64 PCM16) | Assistant audio chunk at `output_sample_rate`. | | `text.delta` | `delta` | Streaming assistant text (when transcription opt-in is on, or model emits text). | | `transcript.committed` | `transcript` | User-speech transcript (when `input_transcription` is on). | | `speech.started` / `speech.stopped` | — | VAD events for the user's microphone. | | `response.started` / `response.completed` | — | Lifecycle markers for an assistant turn. | | `tool.call` | `tool_call_id`, `tool_name`, `tool_arguments` | Function call request from the model. | | `session.terminating` | `error.code`, `error.message` | Server is closing the session. See termination codes below. | | `session.ended` | — | Final frame before the upstream WS closes. | | `error` | `error.code`, `error.message` | Provider or protocol error. **Mid-session errors:** `error.code` is a stable Opper code for protocol-level rejections (`unsupported_modalities`, `invalid_config`, `provider_error`, etc.) or a pass-through provider code for upstream-originated errors. Treat unknown codes as recoverable and surface `error.message` to the user. **Do not** enumerate — new codes can land as new providers are added. | ### Termination codes `session.terminating.error.code` is a closed set; you can enumerate against it: | Code | Cause | | - | - | | `session_timeout` | Session exceeded the 30-minute max duration. | | `idle_timeout` | No client traffic for 60 seconds. | | `balance_exhausted` | Organization balance is empty. | | `project_spend_cap_hit` | Project hit its configured spend cap. | | `org_spend_cap_hit` | Organization hit its configured spend cap. | | `billing_not_supported` | Account on a plan that doesn't support realtime billing. | ## Sample payloads Every event is a JSON object over a text WebSocket frame. Field shapes below are the source of truth for client implementations and agent code generation — match these exactly. ### Client → server ```json session.start theme={null} { "type": "session.start", "config": { "model": "openai/gpt-realtime-2", "voice": "marin", "instructions": "You are a concise voice assistant.", "modalities": ["audio"], "turn_detection": { "type": "server_vad", "threshold": 0.5, "prefix_padding_ms": 300, "silence_duration_ms": 500 }, "tools": [ { "name": "lookup_weather", "description": "Get the current weather for a city.", "parameters": { "type": "object", "properties": { "city": { "type": "string" } }, "required": ["city"] } } ], "reasoning_effort": "low", "input_transcription": true, "input_transcription_model": "gpt-4o-mini-transcribe", "output_transcription": false } } ``` ```json session.update theme={null} { "type": "session.update", "config": { "voice": "verse", "instructions": "Be terser." } } ``` ```json audio.append theme={null} { "type": "audio.append", "audio": "" } ``` ```json text.input theme={null} { "type": "text.input", "text": "What's the weather in Stockholm?" } ``` ```json tool.result theme={null} { "type": "tool.result", "tool_call_id": "call_abc123", "tool_result": { "temperature_c": 12, "conditions": "rain" } } ``` ### Server → client ```json session.started theme={null} { "type": "session.started", "session_id": "sess_xyz", "input_sample_rate": 24000, "output_sample_rate": 24000, "audio_format": "pcm16" } ``` ```json audio.delta theme={null} { "type": "audio.delta", "audio": "" } ``` ```json text.delta theme={null} { "type": "text.delta", "delta": "It's currently raining" } ``` ```json transcript.committed theme={null} { "type": "transcript.committed", "transcript": "what's the weather in stockholm" } ``` ```json speech.started theme={null} { "type": "speech.started" } ``` ```json response.started theme={null} { "type": "response.started" } ``` ```json tool.call theme={null} { "type": "tool.call", "tool_call_id": "call_abc123", "tool_name": "lookup_weather", "tool_arguments": { "city": "Stockholm" } } ``` ```json session.terminating theme={null} { "type": "session.terminating", "error": { "code": "idle_timeout", "message": "No client traffic for 60 seconds." } } ``` ```json error theme={null} { "type": "error", "error": { "code": "unsupported_modalities", "message": "config.modalities contains \"text\" but model \"fake/audio-only\" does not support text output" } } ``` ## Preflight rejections Before the WebSocket upgrade, the endpoint returns standard HTTP status codes: | Status | Cause | | - | - | | `401 Unauthorized` | Missing, non-runtime, or project-less API key. | | `402 Payment Required` | Balance exhausted, spend cap hit, or plan doesn't support realtime. | | `429 Too Many Requests` | Concurrent-session cap reached for this project (default 5). | | `503 Service Unavailable` | Realtime endpoint not configured for the requested provider. | ## See also * **[Realtime voice guide](/capabilities/realtime)** — full journey including authentication, session lifecycle, tool flow, billing, per-provider notes, and a working TypeScript example. * **[brainstorm-time cookbook](https://github.com/opper-ai/opper-cookbook/tree/main/examples/brainstorm-time)** — complete end-to-end voice app. # Rerank documents Source: https://docs.opper.ai/v3-api-reference/rerank/rerank-documents post /v3/rerank Reorder `documents` by relevance to `query` and get them back most-relevant first. `model`, `query` and `documents` are required; `top_n` caps how many results come back (integer >=1; default: all). `top_k` is a compatibility alias (zero means unspecified); positive values must match when both fields are supplied, and `return_documents` echoes each document's text alongside its score. Up to 1000 documents per call. Billed per call or per token depending on the endpoint: `usage.cost` is what was charged, `usage.total_tokens` the provider's count when it reports one. `model` accepts either form, and which one you send decides the routing: - a **bare model name** (e.g. `rerank-v3.5`) routes over that model's pooled endpoints and **falls over to the next one** when an endpoint fails — the case this matters for is provider rate limits, where one endpoint's quota is exhausted and another's is not. - a **fully-qualified id** (e.g. `aws/cohere/rerank-v3.5`) pins that exact endpoint and never falls over. The response's `model` is the endpoint that actually served the call, which for a pooled name is not necessarily the preferred one. List what is available with `GET /v3/models?type=rerank`. The project's Comply model allowlist applies to every endpoint a bare name can route to, not only the one you named: an endpoint it forbids is never tried, not even as a fallover target. A pinned id the allowlist forbids, or a pool it empties, is refused with `403` rather than substituted. A provider that requires a signed agreement needs the organization's grant, as everywhere else: no grant, no call, and never as a fallover target. See the [rerank guide](/build/gateway/rerank) for examples, model selection, and billing behavior. # Roundtable Source: https://docs.opper.ai/v3-api-reference/roundtable/create-roundtable post /v3/roundtable Send a query to multiple models in parallel and combine results using a configurable resolution strategy (summary, fast, multiple_choice). This endpoint is in beta. Its request and response shapes may change without the usual deprecation window. # Create span Source: https://docs.opper.ai/v3-api-reference/spans/create-span post /v3/spans Create a new trace span. # Update span Source: https://docs.opper.ai/v3-api-reference/spans/update-span patch /v3/spans/{id} Update an existing trace span. # Health check Source: https://docs.opper.ai/v3-api-reference/system/health-check get /health Returns server health status. # OpenAPI spec (JSON) Source: https://docs.opper.ai/v3-api-reference/system/openapi-spec-json get /v3/openapi.json Returns the OpenAPI specification in JSON format. # OpenAPI spec (YAML) Source: https://docs.opper.ai/v3-api-reference/system/openapi-spec-yaml get /v3/openapi.yaml Returns the OpenAPI specification in YAML format. # Web fetch Source: https://docs.opper.ai/v3-api-reference/tools/web-fetch post /v3/tools/web/fetch Fetch a URL and return its content as markdown. This endpoint is in beta. Its request and response shapes may change without the usual deprecation window. # Web search Source: https://docs.opper.ai/v3-api-reference/tools/web-search post /v3/tools/web/search Search the web and return results with title, URL, and snippet. This endpoint is in beta. Its request and response shapes may change without the usual deprecation window. # Generate a video Source: https://docs.opper.ai/v3-api-reference/video/create-video post /v3/videos Submit a deterministic video-generation job. Returns immediately with a generation id and a status URL to poll; the video is produced asynchronously. `model` and `prompt` are required; everything in `parameters` is forwarded verbatim to the provider. `image`/`last_image`/`video`/`reference_images` accept an http(s) URL, a data-URI, or a `file_` (resolved per provider). Set `store: true` to also save the output to /v3/files and get a reusable `file_id` on the status poll; nothing is stored otherwise. Poll the returned `status_url` (GET /v3/artifacts/{id}/status) for a presigned download URL when complete. Video generation is **asynchronous**: this returns `202` with an `id` and a `status_url`. Poll `GET /v3/artifacts/{id}/status` until `completed`, then download from the presigned `url`. See the [Video guide](/build/multimodal/video) for the end-to-end flow, reference image/video inputs, and per-provider parameters. # List video models Source: https://docs.opper.ai/v3-api-reference/video/list-video-models get /v3/videos/models List the models usable with `POST /v3/videos` — a `type=video` view of the model catalog. Same shape and query params as `/v3/models` (search, `include`, pagination), but always scoped to video models. Inspect each item's `capabilities` to tell input modality apart: `video_generation` is text-to-video, `image_to_video` needs a source image, and `video_editing` is video-to-video. `params.video` lists each model's accepted `aspect_ratios`, `resolutions`, and `max_duration`. Discovery endpoint for [video generation](/build/multimodal/video). Filter with `?capability=image_to_video` and search with `?q=`.