Calls

What is a Call?

A Call is a call to a LLM to perform a generation. With the Opper SDKs, calls can be configured to use a specific model, prompt, schema. Note that all calls are automatically logged as traces in Opper.

Behind the scenes, the Opper API works to construct prompts for the model to generate outputs that optimially completes the call. Prompts are always completely visible in the trace view.

Make a Call

from opperai import AsyncOpper

opper = AsyncOpper()

async def main():
    result, _ = await opper.call(
        name="respond",
        input="What is the capital of Sweden?")

    print(result)
    # The capital of Sweden is Stockholm.

asyncio.run(main())

import Client from "opperai";

const client = new Client();

(async () => {
    const { message } = await client.call({
        name: "respond",
        input: "What is the capital of Sweden?",
    });

    console.log(message);
    // The capital of Sweden is Stockholm.
})();

curl -X POST https://api.opper.ai/v1/call \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "name": "respond",
    "input": "What is the capital of Sweden?",
  }'

# Expected output:
# {
#   "message": "The capital of Sweden is Stockholm."
# }

Opper also support multi modal inputs and outputs. For examples of this, check out the Examples folder in the Python SDK and Node SDK respectively.

Call with Instructions

Instructions are useful for instructing the model of how to respond to a given input. We typically recommend keeping them small and concise and let the schemas be expressive.

from opperai import AsyncOpper

opper = AsyncOpper()

async def main():
    result, _ = await opper.call(
        name="respond",
        input="What is the capital of Sweden?", 
        instructions="Given a question, produce an answer in Swedish"
    )

    print(result)
    # Huvudstaden i Sverige är Stockholm.

asyncio.run(main())

import Client from "opperai";

const client = new Client();

(async () => {
    const { message } = await client.call({
        name: "respond",
        input: "What is the capital of Sweden?",
        instructions: "Answer questions in Swedish",
    });

    console.log(message);
    // Huvudstaden i Sverige är Stockholm.
})();

curl -X POST https://api.opper.ai/v1/call \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "name": "respond",
    "input": "What is the capital of Sweden?",
    "instructions": "Answer questions in Swedish"
  }'

# Expected output:
# {
#   "message": "Huvudstaden i Sverige är Stockholm."
# }

Use Schemas

Calls are preferably defined to return a specific JSON schema. While this is JSON under the hood, we recommend using Pydantic for Python and Zod for Typescript to facilitate building accurate schemas:

from opperai import AsyncOpper
from pydantic import BaseModel

opper = AsyncOpper()

class Room(BaseModel):
    beds: int
    seaview: bool
    description: str

async def main():
    result, _ = await opper.call(
        name="extractRoom",
        input="Room at Grand Hotel with 2 beds and a view to the sea.", 
        output_type=Room
    )

    print(result)
    # Room(beds=2, seaview=True, description="Room at Grand Hotel with 2 beds and a view to the sea")

asyncio.run(main())

import Client from "opperai";

const client = new Client();

const outputSchema = {
    type: "object",
    $schema: "https://json-schema.org/draft/2020-12/schema",
    properties: {
        room_count: {
            type: "number",
            description: "The number of rooms"
        },
        view: {
            type: "string",
            description: "The view from the room"
        },
        bed_size: {
            type: "string",
            description: "The size of the bed"
        },
        hotel_name: {
            type: "string",
            description: "The name of the hotel"
        }
    },
    required: ["room_count", "view", "bed_size", "hotel_name"]
};

(async () => {
    // Use json_payload from the call
    const { json_payload } = await client.call({
        name: "extractRoom",
        input: "The Grand Hotel offers a luxurious suite with 3 spacious rooms, each providing a breathtaking view of the ocean. The suite includes a king-sized bed, an en-suite bathroom, and a private balcony for an unforgettable stay.",
        output_schema: outputSchema,
    });

    console.log(json_payload);
    // Expected output: { room_count: 3, view: 'ocean', bed_size: 'king-sized', hotel_name: 'The Grand Hotel' }
})();

curl -X POST https://api.opper.ai/v1/call \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "name": "extractRoom",
    "input": "Room at Grand Hotel with 2 beds and a view to the sea.",
    "output_schema": {
      "type": "object",
      "properties": {
        "beds": {
          "type": "integer",
          "description": "The number of beds in the room"
        },
        "seaview": {
          "type": "boolean",
          "description": "Whether the room has a sea view"
        },
        "description": {
          "type": "string",
          "description": "A brief description of the room"
        }
      },
      "required": ["beds", "seaview", "description"]
    },
  }'

# Expected output:
# {
#   "message": "Room(beds=2, seaview=True, description='Room at Grand Hotel with 2 beds and a view to the sea')"
# }

Specifying Model

On any call you can specify which model to use. Opper has support for a lot of models, and loves to add additional ones. If no model is specified, Opper reverts to azure/gpt-4o. To specify a different model:

from opperai import AsyncOpper

opper = AsyncOpper()

async def main():
    result, _ = await opper.call(
        name="respond",
        input="What is the capital of Sweden?", 
        model="anthropic/claude-3.5-sonnet"
    )

    print(result)
    # The capital of Sweden is Stockholm.

asyncio.run(main())

import Client from "opperai";

// Your API key will be loaded from the environment variable OPPER_API_KEY if not provided
const client = new Client();

(async () => {
    const { message } = await client.call({
        name: "respond",
        input: "What is the capital of Sweden?",
        model: "anthropic/claude-3.5-sonnet",
    });

    console.log(message);
    // The capital of Sweden is Stockholm.
})();

curl -X POST https://api.opper.ai/v1/call \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "name": "respond",
    "input": "What is the capital of Sweden?",
    "model": "anthropic/claude-3.5-sonnet"
  }'

# Expected output:
# {
#   "message": "The capital of Sweden is Stockholm."
# }

See Models for more information.

Specifying Examples

Providing examples to a call is a great way to show how you want outputs to look like given some inputs. This will help the model reason without having to improve the prompt. The number of examples is limited to 10.

from opperai import AsyncOpper
from opperai.types import Example

opper = AsyncOpper()

async def main():
    result, _ = await opper.call(
        name="GetFirstWeekday",
        instructions= "Extract the first weekday mentioned in the text",
        examples= [
            Example(
                input= "Today is Tuesday, yesterday was Monday",
                output= "Monday",
            ),
            Example(
                input= "Sunday, Saturday and Friday are the best days of the week",
                output= "Friday",
            ),
            Example(
                input= "It is Saturday that is the best day of the week, next to Tuesday",
                output= "Tuesday",
            ),
        ],
        input="Do you want to come by on this Thursday or this Tuesday?", 
    )

    print(result)
    # Tuesday

asyncio.run(main())

import Client from "opperai";

// Your API key will be loaded from the environment variable OPPER_API_KEY if not provided
const client = new Client();

(async () => {
    const { message: weekday } = await client.call({
        name: "GetFirstWeekday",
        instructions: "Extract the first weekday mentioned in the text",
        // Maximum of 10 examples
        examples: [
            {
                input: "Today is Tuesday, yesterday was Monday",
                output: "Monday",
            },
            {
                input: "Sunday, Saturday and Friday are the best days of the week",
                output: "Friday",
            },
            {
                input: "It is Saturday that is the best day of the week, next to Tuesday",
                output: "Tuesday",
            },
        ],
        input: "Do you want to come by on this Thursday or this Tuesday?",
    });
    console.log("First weekday: ", weekday);
    // First weekday: Tuesday
})();

For a more extensive example of using examples by populating dataset and automatically retrieving them in the call, see Guiding output with examples.

Read our Cookbook for additional examples: https://github.com/opper-ai/opper-cookbook