Datasets

A dataset is a collection of input/output pairs that represent ideal executions of a Function. Datasets establish ground truth by capturing successful examples of a task.

You can find datasets in the UI under each function and use them in two ways:

• As examples in prompts to guide model behavior and output quality, see "Enable few shot prompting" in Calls
• As test cases to evaluate the quality of your function with current or different instructions and models, see Evaluations

A dataset entry

Here's an example dataset entry for a function that counts occurrences of a letter in a string:

Each dataset entry has three key fields:

• input - The test case value passed to the function
• output - The raw model response given current settings
• expected - The ideal output for this input. If not set, defaults to output

All dataset entries must conform to the function's output schema to be valid. The input/expected pairs establish ground truth, while output preserves the original model response.

Using this example as few-shot prompting would demonstrate letter counting to the model. As a test case, it verifies the function counts letters correctly.

Building Your Dataset

Upload JSON Files

Upload existing JSON datasets directly through the function's dataset page.

Add from Traces

When viewing traces, click "add to dataset" to save examples. Add both successful cases for testing/few-shot prompting and failures with corrected expected outputs to track issues.

Add with API/SDK

Use our SDKs to automatically populate datasets from your application. For example, capture edited drafts as expected outputs to build a feedback loop and test suite.

Managing Entries

Edit dataset entries by clicking on them and modifying the expected field using the pen icon.

Using Your Dataset

• Run evaluations to test function performance, see Evaluations
• Include examples in calls to guide model behavior, see "Enable few shot prompting" in Calls