Tracing

What is Tracing?

Tracing is essentially logging, but where related or subsequent logs is associated with eachother. This drastically simplifies debugging and understanding complex interactions. This is especially relevant for LLM applications, as they often gets inplemented with a series of steps and calls.

Calls create Spans

Opper automatically traces all function calls and index retrievals. Our traces page offers a detailed view of each trace, making it simple to understand complex flows made of many LLM calls.

Connect Spans into a Trace

To place spans into the same trace you have to create a hierarchy. Here is how you can make spans part of a higher level span:

from opperai import AsyncOpper

opper = AsyncOpper() 

async def main():
    async with opper.traces.start("high_level_span") as high_level_span:
        # Calls in the higher level span

        async with opper.traces.start("subspan") as sub_span:            
            # Calls in the lower level span
            pass
    
asyncio.run(main())

from opperai import Opper, trace

opper = Opper()

# function name becomes span name
@trace
def high_level_function():
    do_things()

import Client from "opperai";

const client = new Client();

(async () => {
    // Start parent trace
    const trace = await client.traces.start({
        name: "my_high_level_span",
        input: { some: "input given to", to: "the trace" },
    });

    // Start the span and provide the input
    const span = await trace.startSpan({
        name: "my_lower_level_span",
        input: { some: "input given to", to: "the span" },
    });

    const { message, span_id } = await client.call({
        name: "name_of_my_call",
        input: "what is the capital of sweden",
        parent_span_uuid: span.uuid, // add to span of trace
    });

    console.log(message);
    // The capital of Sweden is Stockholm

    // End the span and provide the output
    await span.end({
        output: { foo: "bar" },
    });

    await trace.end({ output: { foo: "bar" } });
})();

Adding metrics to a Span

You can attach metrics to a span and visualize it in our UI. This is very useful for enabling a feedback loop from your users or applications to yourself.

from opperai import AsyncOpper

opper = AsyncOpper()

async def main():
    async with opper.traces.start("high_level_span") as high_level_span:
        # your calls

        await high_level_span.save_metric("thumbs_down", 0.5, "User pressed thumbs down button")

asyncio.run(main())

import Client from "opperai";

const client = new Client();

(async () => {
    // Start parent trace
    const trace = await client.traces.start({
        name: "node-sdk/tracing-manual",
        input: { some: "input given to", to: "the trace" },
    });

    // Start the span and provide the input
    const span = await trace.startSpan({
        name: "node-sdk/tracing-manual/span",
        input: { some: "input given to", to: "the span" },
    });

    const { message, span_id } = await client.call({
        name: "node-sdk/call/basic",
        input: "what is the capital of sweden",
        parent_span_uuid: span.uuid, // add to span of trace
    });

    // A metric and/or comment can be saved to the span
    await span.saveMetric({
        dimension: "accuracy",
        score: 1,
        comment: "This is a comment",
    });

    console.log(message);

    // End the span and provide the output
    await span.end({
        output: { foo: message },
    });

    await trace.end({ output: { foo: message } });

Investigate Traces

Traces show up in the portal https://platform.opper.ai/traces. There you can see all traces for your project, including token use, evaluations etc.

Viewing a trace

Click on a trace to view all calls in a trace. Under each call is a generation where you can see the prompt, model used etc.

More information

See more extensive examples in our Python and Node.js SDKs, who both have an example folder with runnable examples.