The Opper API enables you to leverage large language models to complete a wide variety of tasks. These completions can produce nearly any type of output — from plain text and structured data to images and audio. You can also supply a wide range of input formats, including text, structured inputs, images, and audio.
How it works
The task completion api works by taking a detailed specification of a task in a declarative form defined in code. Upon receiving the task, the Opper platform will prompt models to complete it as per the specification. To be able to optimize it, it will create a generative function for each task that holds revisions, metrics and observations. To achieve reliability, it will facilitate a process for optimally interacting with models from data of what has worked. It may perform retries if models fail, and utilise past generations as examples as guidance. Most of this is configurable, but the defaults aims to be good for many cases.
Defining tasks
Tasks are defined in a declarative manner, where you specify input data, output structures and optional configuration on how to complete the task. The Opper API returns completions as specified and builds optimal prompts for completing the task.
Here is a simple example of a task:
from opperai import Opper
import os
from pydantic import BaseModel, Field
from typing import List
opper = Opper(http_bearer=os.getenv("OPPER_API_KEY", ""))
# Input schema with field descriptions
class KBQueryInput(BaseModel):
facts: List[str] = Field(description="Standalone facts to answer from")
question: str = Field(description="The question to answer")
# Output schema with field descriptions
class KBQueryOutput(BaseModel):
thoughts: str = Field(description="Elaborate step-by-step reasoning")
answer: str = Field(description="Concise answer to the question")
# Task definition and completion run
response = opper.call(
name="mini_kb_query",
instructions="Given the list of bullet-point facts, answer the question.",
input_schema=KBQueryInput,
output_schema=KBQueryOutput,
input={
"facts": [
"Jupiter is the largest planet in the Solar System.",
"The Great Red Spot is a giant storm on Jupiter.",
"Saturn possesses the most extensive ring system in the Solar System."
],
"question": "Which planet hosts the Great Red Spot?"
},
)
print(response.json_payload)
# {'thoughts': "From the facts provided, I know that Jupiter is the largest planet in the Solar System and that the Great Red Spot is a giant storm. This clearly links the Great Red Spot to Jupiter. Facts about Saturn's ring system are unrelated to the question. Therefore, the planet hosting the Great Red Spot is Jupiter.", 'answer': 'Jupiter'}
The completion of this task yields:
{
span_id: "067929e0-6478-43cb-9a5f-6df4bc792a76",
message: Null
json_payload: {
'thoughts': "The question asks about the planet that hosts the Great Red Spot. By analyzing the provided facts, we see that the fact 'The Great Red Spot is a giant storm on Jupiter' directly connects Jupiter to the Great Red Spot. Hence, the answer can be derived confidently.",
'answer': "Jupiter"
},
audio: Null,
cached: False,
images: Null
}
Task completions are heavily influenced by schemas, so make sure to invest time in describing the input and output data. Additionally, tasks can complete into images and audio, as well as take images and audio as input. For more information, see the API reference and examples
For streaming output, see the stream endpoint. Note that it is currently not possible to combine streaming with output schemas. All other options are the same
Choosing model(s)
By default, Opper chooses a default model for processing tasks. You may however specify models yourself.
Choose a model by adding the model parameter to the task definition:
model = "openai/gpt-4.1-nano"
Specify multiple models in a list to traverse them until the completion finishes:
model = [
"openai/gpt-4.1-nano", # first model to try
"openai/gpt-4o-mini", # second model to try
]
It is also possible to pass model specific configuration:
model = [{
"name": "openai/gpt-4o-mini", # the model name
"options": {
"temperature": 0.1 # the options for the model
}
}]
Different models are good at different things. Finding out which model works best for each task is a late stage optimization.It is generally recommended to keep the default model in place and switch to a more powerful model only if necessary, or to a smaller one if it performs equally well.
Extending schemas
Schemas can be extended to be more expressive and hold detailed requirements or prompts:
Literals and enums can be useful for doing classifications:
class KBQueryOutput(BaseModel):
thoughts: str = Field(
description="Elaborate step-by-step reasoning"
)
classification: Literal["easy", "medium", "hard"] = Field(
description="The difficulty of the question"
)
answer: str = Field(
description="Concise answer to the question",
)
Regex patterns can be useful to enforce some pattern of the content:
class KBQueryOutput(BaseModel):
thoughts: str = Field(
description="Elaborate step-by-step reasoning"
)
answer: str = Field(
description="Concise answer to the question starting with 'The answer to the question is '",
pattern=r"^The answer to the question is [A-Za-z0-9\s]+$"
)
Detailed schemas are a great way to constrain the model to complete what you want. It will minimize the need for doing evaluations and runtime tests. Be mindful of not over constraining the schema however, for example by requiring the model to yield information that is not possible. Make sure to give the model an out by allowing it to return null values and similar.
Using examples
You can also provide examples to the task that show examples of successful completions by appending the examples parameter.
Here is an example where we provide an example of how to handle the situation if there are no facts for the question:
examples = [{
"input": KBQueryInput(
facts=[
"Jupiter is the largest planet in the Solar System.",
"The Great Red Spot is a giant storm on Jupiter.",
"Saturn possesses the most extensive ring system in the Solar System."
],
question="How many planets are in the Solar System?"
),
"output": KBQueryOutput(
thoughts="To determine the answer, I reviewed the provided facts. The facts discuss Jupiter, its Great Red Spot, and Saturn's extensive ring system, but they do not specify how many planets are in the Solar System. Without relevant information, the question cannot be answered based on these facts alone",
answer="The answer to the question is unknown"
),
}]
Running this on another question which has no relevant facts:
"question": "What is the name of Earth's moon?"
Yields:
{
'thoughts' : "To determine the answer, I examined the provided facts. The facts focus on details about Jupiter and Saturn, including Jupiter's size, the Great Red Spot, and Saturn's ring system. However, there is no mention of Earth's moon or its name in the given facts. Without relevant information in the facts, I cannot answer the question.",
'answer': "The answer to the question is unknown"
}
The optimal number of examples is typically between 3-10, and they have more effect if they are relevant for the input case. For extended use of examples, we recommend looking into
Opper functions that allows for centrally managing examples and adding relevant ones to the completion request.
It is possible to add metadata to task completions to capture things like customer information or environment, by appending the tags parameter.
Here is an example:
tags = {
"user": "company_123",
"env": "production"
}
Tags are useful for filtering traces, but also for billing and analytics.
Inspecting completions
All completions are automatically logged and can be viewed at https://platform.opper.ai. There you can observe inputs, outputs and even AI observations on the task completion.
You can observe the following information
- Task inputs and outputs
- Model and tokens used
- The low level prompt and generation from the model
- A fast AI evaluation of the task completion
Looking at your completion data - i.e. the input and outputs - is incredibly valuable to understand what is working and not. Opper also supports tying task completions to higher level hierarchical traces, which makes it easy to debug multi step completions that are common in agents or chatbots. See Traces for more information
Managing tasks at scale
As tasks grows in complexity and utility it may be preferred to have their configuration managed out of code. To allow for this, tasks can be managed server side and be retrieved when needed.
Here we create a function that does the answer task from above, where it is also configured to pull examples from the functions dataset.
Create function
class KBQueryInput(BaseModel):
facts: List[str] = Field(
description="Standalone facts to answer from"
)
question: str = Field(
description="The question to answer"
)
class KBQueryOutput(BaseModel):
thoughts: str = Field(
description="Elaborate step-by-step reasoning"
)
answer: str = Field(
description="Concise answer to the question starting with 'The answer to the question is '",
pattern=r"^The answer to the question is [A-Za-z0-9\s ]+$"
)
function = opper.functions.create(
name="mini_kb_query2",
instructions="Given the list of bullet-point facts, answer the question.",
input_schema=KBQueryInput.model_json_schema(),
output_schema=KBQueryOutput.model_json_schema(),
configuration={
"invocation.few_shot.count": 3
}
)
Get function
We can also retrieve the function by its id
function = opper.functions.get(function_id="eb5f283f-f9a4-469f-9530-85ad2a1cf0e9")
Call function
And perform a completing with the function by passing only the inputs:
response = opper.functions.call(
function_id=function.id,
input={
"facts": [
"Jupiter is the largest planet in the Solar System.",
"The Great Red Spot is a giant storm on Jupiter.",
"Saturn possesses the most extensive ring system in the Solar System."
],
"question": "What planet has the largest ring system?"
}
)
Which again yields:
{
'thoughts': "To answer the question about which planet has the largest ring system, I reviewed the provided facts. The first fact mentions that Jupiter is the largest planet in the Solar System. The second fact describes the Great Red Spot on Jupiter. The third fact explicitly states that Saturn possesses the most extensive ring system in the Solar System. Therefore, based on the information provided, Saturn is the planet with the largest ring system.",
'answer': "The answer to the question is Saturn"
}
With functions, tasks can be managed completely server side, allowing for versioning, improving datasets, choosing models and more. It also allows for accessing these through rest, expanding it to more languages than what is supported with SDKs
