View results

Review the simulation results dashboard and each scenario's conversation, set up usage, and upload a Trajectory to evaluate tool calls.

1. Review the results dashboard

①Results screen layout

When the simulation finishes, the results screen opens. It has four sections, from top to bottom.

Section	What it shows
`Evaluation Result`	Pass Rate, number of `Pass` and `Fail`, and pass rate per evaluation item (`Pass rate by condition`)
`Usage`	`Total Tokens` and `Total Cost` for this Set. Expand `View Detail` to see cost per stage (`Cost by Stage`)
`Representative Failures`	Scenarios that failed to meet 80% or more of their conditions
`All Scenarios`	Evaluation item results for each scenario

②Pass verdict

A scenario is Pass only if it passes every evaluation item. If any item fails, it's Fail.

Evaluation item	What it checks
`Must Mention`	Whether the response stated the values it must mention
`Expected Behavior`	Whether the agent followed the behavior required by the policy
`Expected Actions`	Whether the agent called the right tools with the right arguments

Expected Actions counts toward the verdict only after Trajectory analysis finishes. Until then, it shows Not evaluated.
Pass Rate is the percentage of scenarios that are Pass.
In Pass rate by condition, conditions met is the number of conditions met, and scenarios passed is the number of scenarios that passed that item entirely.
Because a scenario must meet every condition to pass, the Pass Rate can be lower than the per-item pass rates.

③View representative failures

Representative Failures shows up to three failed scenarios that didn't meet 80% or more of their conditions.

The top of each card shows the Task and the number of unmet conditions, such as 6 of 7 conditions not met.
Below, unmet conditions are listed by evaluation item: missing values for Must Mention, behavior not followed for Expected Behavior, and incorrectly called tools for Expected Actions.
If there are many items, they're collapsed, as in +1 more. Click to expand.
Click View Detail to open that scenario's details panel.
If every scenario passed, or no scenario failed 80% or more of its conditions, a message appears instead of cards.

1-1Check evaluation results
1. Check the Pass Rate and the number of Pass and Fail.
2. Check the pass rate for each evaluation item in Pass rate by condition.
See Pass verdict for the judgment criteria.
1-2Check usage
1. Check Total Tokens and Total Cost.
2. Click View Detail.
3. Check the cost per stage in Cost by Stage.
- AI Agent cost is included in the total only after you set unit prices.
- See Usage totals for details.
1-3Check representative failures
1. In Representative Failures, review the scenarios that didn't meet 80% or more of their conditions.
2. If there are many unmet conditions, expand them with +1 more.
3. Open that scenario's details panel with View Detail.
1-4Check all scenarios
1. Check each scenario's results in All Scenarios.
2. Click a row to open the details panel.
- Each cell shows Pass or Fail and (passed/total).
- The Expected Actions column appears after Trajectory analysis finishes.

2. View results by scenario

①Details panel

Click a row in All Scenarios, or click View Detail in Representative Failures, to open the details panel on the right.

Tab	Contents
`Result`	Conversation, verdict for each evaluation item, and usage
`Scenario`	The scenario used for evaluation, its raw JSON, and scenario generation usage

Must Mention values are highlighted in green in the conversation.
Click a passed value in the Must Mention card to jump to where it appears in the conversation.
Click the same value again to jump to the next occurrence.
Expected Actions shows Not Evaluated until Trajectory analysis finishes.

2-1Open the panel
1. Click a scenario row in All Scenarios.
2. Review the conversation in the details panel.
2-2Check evaluation items
1. Check the verdict for each value in Must Mention.
2. Click a passed value to highlight where it appears in the conversation.
3. Check the verdict for each behavior in Expected Behavior.
2-3Check usage
1. Check this scenario's usage in Usage.
See Usage totals for the breakdown by stage.
2-4Review the scenario
1. Click the Scenario tab.
2. Review the scenario used for evaluation.

3. Set up usage

①Usage totals

The Usage total includes the following usage.

Stage	When it's added to the total
`Scenario Generation`	When scenarios are generated
`LLM User`	When the conversation ends
`AI Agent`	When the conversation ends and unit prices are set
`Judge Model`	When scoring finishes

Expand View Detail to see cost per stage (Cost by Stage), split into Generation, Simulation, and Evaluation.
If unit prices aren't set, AI Agent cost is left out of the total and the message AI Agent usage is pending setup and excluded from totals. appears. In Cost by Stage, AI Agent shows Pending setup, and shares are calculated without AI Agent (Shares exclude AI Agent).
Right after you save unit prices, AI Agent usage is being calculated and excluded from totals for now. appears. Once the calculation finishes, it's added to the total.
Scenarios still being scored or with scoring errors don't include Judge Model usage.

②AI Agent unit prices

Set AI Agent unit prices in Set up Usage. Cost is calculated by reading token counts from the evaluated agent's responses.

Field	What to enter
`Input Tokens`	The path to the input token count in the response (e.g., `response_1.usage.prompt_tokens`).
`Output Tokens`	The path to the output token count in the response
`Input token cost ($)`	Price per 1 million input tokens
`Output token cost ($)`	Price per 1 million output tokens

Only Admins can use Set up Usage. Other users see Contact your admin to set it up.
Settings are saved per Application and apply to every Set in that Application.
When you save, the AI Agent cost for the results you're viewing is recalculated.
For multi-turn conversations, tokens from all turns are added together.

3-1Open the settings dialog
1. In the message under Usage, click Set up Usage. You can also expand View Detail and click it in the AI Agent row.
Only Admins can configure this.
3-2Enter token paths
1. In Input Tokens, enter the path to the input token count.
2. You can also pick the path from a sample response with Pick from response.
3. Enter Output Tokens the same way.
If the path is correct, the value appears below, such as Mapped: 1240.
3-3Enter unit prices
1. Enter the input and output prices per 1 million tokens.
2. Check the estimated cost per turn in the box below.
3. Click Save.
- This applies to every Set in the same Application.
- See AI Agent unit prices for what each field means.

4. Upload a Trajectory

①Trajectory

A Trajectory is a record of the tools the agent called during the simulation. Tool calls can't be determined from the conversation log alone, so you upload them separately.

You can upload one only in Sets created with a Tool Schema.
Upload it from the results screen after the simulation has fully finished.
.json and .jsonl files are accepted.

②Analysis status

Status	What you see
Before upload	Yellow message `Tool call evaluation isn't included yet.`
Analyzing	Blue message `Analyzing trajectory.` `Upload Trajectory` is disabled
Analysis complete	The message disappears and `Expected Actions` results are added. The button changes to `Re-upload Trajectory`
Analysis failed	Red message with `Retry` and `Upload Again` buttons

When analysis finishes, the Pass and Fail counts and the Pass Rate are recalculated.
You can't delete a Set during analysis.
To reevaluate with a different file, click Re-upload Trajectory. Uploading a new file replaces the current Expected Actions results.

4-1Upload the file
1. Check the yellow message.
2. Click Upload Trajectory.
3. Drag and drop a file, or click to select one.
4. Click Upload.
4-2Check analysis results
1. During analysis, a blue message appears.
2. When analysis finishes, the Expected Actions pass rate appears.
3. To reevaluate with a different file, click Re-upload Trajectory.
The Pass and Fail counts and the Pass Rate are also recalculated to include tool call results.
4-3Handle analysis failures
1. If a red message appears, check the file format.
2. Click Retry to analyze again.
3. To upload a different file, click Upload Again.

Frequently asked questions

Why is there no Upload Trajectory button?Because the Set was created without a Tool Schema.

Cause: Tool call evaluation is available only in Sets created with a Tool Schema. The Trajectory column in the list shows No Tool Schema.
Solution: To evaluate tool calls too, create a new Set with a Tool Schema.

Why does the total cost look lower than it should?Because AI Agent cost isn't included.

Cause: If AI Agent unit prices aren't set, that cost is left out of the total.
Solution: Have an Admin set unit prices in Set up Usage. See AI Agent unit prices for how.

Why don't I see Set up Usage?Because only Admins can configure it.

Cause: If you're not an Admin, you see a message asking you to contact your admin instead of the button.
Solution: Ask your Workspace Admin to set the unit prices.

Why hasn't the cost changed after I saved unit prices?Because it's still being calculated.

Cause: When you save unit prices, the AI Agent cost is recalculated. Calculating appears in the meantime.
Solution: Check again in a moment.

Why does Expected Actions show Not Evaluated?Because Trajectory analysis hasn't finished.

Cause: Tool calls can't be determined from the conversation log alone.
Solution: Upload a Trajectory on the results screen. The verdict appears once analysis finishes.

What should I do if Trajectory analysis fails?Make sure the file is .json or .jsonl, then click Retry.

Cause: Only .json and .jsonl files can be analyzed as Trajectories.
Solution: Check the file, then click Retry, or upload the file again with Upload Again.