Eval History

Review the evaluation results generated by Run Eval and rerun items that errored.

1. View the results list

①Results screen layout

The results screen shows the results of one Task in one Run as a table. Click a Task name on the Runs screen to open it.

The top of the screen shows Runs > Run name and the Task name.
Each row in the table is one Benchmark Set data item, identified by the first column, ID.
The table columns are ordered as data values, Output, then scores.

Column group	Example column name	Content
Input	`query`	The value sent to the Application
Reference	`expected_response`	The ground truth a Metric refers to when scoring
Metadata	`query_author`	Additional information not used in evaluation
Output	`response_Gemini-Pro`	The response generated by a Version
Score	`Average score_Gemini-Pro` `Context Recall_score_Gemini-Pro`	The average score and the score for each Metric
Judgment rationale	`Context Recall_reason_Gemini-Pro`	Why the Metric gave the score

Use Sort and Filter to show only the rows you want. Use Column Visibility to hide columns.
Results are stored with the values at evaluation time. Metadata, however, shows the latest values.

Evaluation Core Concept explains Input, Reference, and Metadata.

②Average score

Average score is the average of a row's Metric scores for the same Version. The column name is Average score_<Version>.

Metrics that errored are excluded from the average.
Rows with no fetched Output show -.
In Filter, set an Average Score range to show only low-scoring rows.

③Results by Version

If you turn on 2 or more Versions in a Run, the results are split by Version.

Table: Each Version gets its own Output and score columns. The Version name is appended to each column name (e.g., response_Gemini-Pro, response_RAG-v1.2).
Row Detail: Version tabs appear in the top right. Click a tab to see the results for each Version.

You can compare the scores of different Versions for the same data side by side. Run Eval explains how to select Versions.

1-1Open the results screen
1. Click a Task name on the Runs screen to open the results screen. If this is your first time, read Results screen layout first.
2. Check the Run and Task names at the top of the screen.
1-2Check scores
1. Check the Output of each Version for each data row.
2. Check the Metric score cells and the Average score.
3. Use Sort and Filter to show only the rows you want.
4. Click Export to export the results.

2. View Row Detail

①Row Detail

Row Detail is a panel that shows one data item's content and results on a single screen. Click a row in the table to open it.

Location	What you see
Left	`Run Name`, `Subgroup`, `Task` `Input`, `Reference`, `Metadata`
Right	Version tabs, `Latency`, `Token Usage` `Output`, `Results by Metric`

Click a Metric in Results by Metric to see its Score and Reason.
Use the list or arrows at the top of the panel to move to the previous or next row.

2-1View row details
1. Click a data row in the list to open Row Detail.
2. Check Input, Reference, and Metadata.
3. Check Output and Results by Metric.
4. Click a Version tab to see another Version's results.
Version tabs appear when you evaluate 2 or more API Versions. See Results by Version.

3. Check errors

①Fetch and Evaluation

An evaluation runs in two stages. Evaluation starts only after Fetch finishes.

Stage	What it does	In-progress indicator
Fetch	Calls the Version's API and fetches Outputs.	`Fetching is in progress (n%)`
Evaluation	Metrics score the fetched Outputs.	`Evaluation is in progress (n%)`

The in-progress indicator appears in the progress bar at the top of the results screen. When the progress bar disappears, both stages are done.

②Error types

Errors fall into two types depending on the stage where they occurred.

Error	Meaning	How it looks in the table
Fetch error	The Output couldn't be fetched.	The Output cell is shown in red (e.g., `[E-504] API response timeout`). The row's score cells show `-`.
Evaluation error	The Output exists, but scoring failed.	The score cell shows `Error` in red.

If there are errors, the Error Status Bar appears at the top of the screen. The bar shows the count of both error types (e.g., Fetch errors: 2 · Evaluation errors: 2).

3-1Check the Error Status Bar
1. Check the error counts in the Error Status Bar at the top of the screen. There are two error types.
2. Find the items that errored in the red cells.
3. To close the bar, click X on the right.
Even if you close it with X, the bar reappears when you return to this screen.

4. Rerun failed items

①Retry and Retry All

You can rerun items that errored.

Method	What it reruns
The retry icon in a red cell	That one item
`Retry All` in the `Error Status Bar`	All items that errored

When you click Retry All, the confirmation dialog shows how many items will rerun (Fetch items, Evaluation items).
Fetch error items are fetched again first. Then Evaluation runs.
Items whose Output is fetched again are also rescored by their Metrics.

4-1Run Retry All
1. Click Retry All in the Error Status Bar.
2. Click Retry in the confirmation dialog.
3. Check the rerun status in the progress bar at the top.
4. When the rerun finishes, check the results again.
- Items that need Fetch are processed first.
- Then Evaluation runs.
- Retry and Retry All describes the order in detail.

Frequently asked questions

Why did the Error Status Bar appear at the top of the screen?Because some items errored during evaluation.

Cause: The bar appears when a response couldn't be fetched or an evaluation failed for some items. Read Error types for details.
Solution: Click Retry All to rerun the failed items.

Why does the Error Status Bar come back after I close it with X?Because X only hides the bar temporarily.

Cause: If errors remain, the bar reappears when you return to the results screen.
Solution: To clear the bar, rerun the failed items with Retry All.

Why don't scores appear right after I click Retry All?Because Fetch finishes first, and then Evaluation runs.

Cause: If some items need their responses fetched again, evaluation starts only after that finishes.
Solution: Wait until the progress bar at the top disappears, then check the results again. Fetch and Evaluation explains the two stages.

Why can't I see another Version's results in Row Detail?Switch the Version tab to check. Each Version tab shows only that Version's results.

Cause: When you evaluate 2 or more API Versions, Row Detail splits into tabs by Version.
Solution: Click a Version tab in the top right to see each Version's results.