Dashboard

Analyze completed Red Teaming Run results with Run Summary cards, Taxonomy and Strategy heatmaps, and direct Sample comparison.

1. Open and save a dashboard

①Runs shown on the dashboard

The dashboard compares the results of completed Runs side by side. When you first open it, Current Batch shows the most recent Batch.

Item	Rule
`Domain`	Only one Domain appears per screen. Dashboards are separated by Domain.
`Run`	You can select only Runs in that Domain.
Number of Runs	Select up to 6. If you bring in more than 6 with Compare on the Runs screen, the top 6 in the list are selected.

Runs with Errors don't appear on the dashboard. Seeds with Errors don't appear in the results either.

②Saved dashboards

Click Save Dashboard to save the current Domain and Run selection as a new tab in the Dashboards list on the left.

A saved dashboard is a bookmark for quickly returning to a screen you view often. It doesn't freeze the results at a point in time.
You can also edit or remove Runs or the Domain within a saved dashboard.

1-1Open the dashboard
1. In the left menu, click Dashboard under Red Teaming.
2. Check that Current Batch shows the most recent Batch.
3. Select a Domain.
4. In Run, select up to 6 Runs to compare.
The rules for selectable Runs are in Runs shown on the dashboard.
1-2Save the dashboard
1. Click Save Dashboard in the top right.
2. Enter a Dashboard Page Name.
3. Click Complete.
4. Check the new tab in the Dashboards list on the left.
A saved dashboard is a bookmark and doesn't freeze results.

2. Check Run Summary cards

①Run Summary cards

Each selected Run gets one card. There are up to 6 cards, and the layout changes with the number of Runs.

Item	Description
ASR	Shown to two decimal places. The color varies by value, using the same color ranges as the heatmap.
`Unsafe`	Number of attempts judged Unsafe
Run name	The Run title
`Config`	Run settings such as Algorithm and Seed Set
`Target`	The evaluation target model and version
`Resource`	Time taken and cost

2-1Read the cards
1. Check that there's one card per Run.
2. On each card, check the ASR, Unsafe count, run settings, and cost. The items are listed in Run Summary cards.

3. Analyze heatmaps

①Read the heatmaps

Heatmaps show, for each Run, which Taxonomies and Strategies led to successful attacks.

Heatmap	Rows	Columns
`Taxonomy Heatmap`	Taxonomy	Run
`Strategy Heatmap`	Strategy	Run
`Strategy × Taxonomy`	Strategy	Taxonomy

Cells use the [ASR] (Unsafe/Total) format. Cell color is based on the ASR value; the higher it is, the closer to red.
Only the top 5 Taxonomies are shown at first. To see all of them, click View {n} Taxonomy.
Click a cell to apply its conditions to the Direct Sample comparison filters below.
Strategy × Taxonomy shows which Strategy was most effective for a given Taxonomy.

②`-` cells

Taxonomies or Strategies that don't overlap across the selected Runs appear as -. Click What does "-" mean? above the heatmap to open an explanation.

3-1Taxonomy and Strategy heatmaps
1. In Taxonomy Heatmap, check the attack success rate for each Run.
2. Read the [ASR] (Unsafe/Total) value in each cell.
3. A - cell is an item that doesn't overlap across the selected Runs.
4. To see all Taxonomies, click View {n} Taxonomy.
5. Read Strategy Heatmap the same way.
The cell format and the meaning of - are explained in Read the heatmaps.
3-2Strategy × Taxonomy
1. In Strategy × Taxonomy, find the most effective Strategy for each Taxonomy.
2. Click a cell you want to look at in detail.
Clicking a cell takes you to Direct Sample comparison.

4. Direct Sample comparison

①How comparison works

Compare side by side how each Run responded to the same Seed. Use this to see how results for the same query differ by run settings.

The Taxonomy and Strategy filters default to All. Only values that actually exist in the selected Runs appear as options.
If you clicked a heatmap cell, that cell's conditions are filled into the filters automatically.
The Seed list on the left shows only Seeds that match the filters. The first Seed is selected at first, and you can search within the list.
If you ran multiple Iterations, only the last Iteration is shown.

4-1Compare responses by Seed
1. Select the Taxonomy and Strategy filters.
2. Select a Seed from the list on the left.
3. On the right, compare each Run's verdict, attack query, and response.
If there are multiple Iterations, only the last Iteration is shown.

Frequently asked questions

Why can't I select the Run I want to compare?Because the Run is in a different Domain from the current dashboard.

Cause: Dashboards are separated by Domain, and you can select only Runs in the same Domain.
Solution: Change Domain to that Run's Domain.

Why do only some of the Runs I brought in with Compare appear?Because the dashboard shows up to 6 Runs.

Cause: If you bring in more than 6, only the top 6 in the list are selected.
Solution: Select 6 or fewer Runs to compare.

Why isn't a Run with Errors on the dashboard?Because Runs with incomplete results are excluded from comparison.

Cause: Runs with incomplete results are left out of comparison. The same goes for Seeds with Errors.
Solution: Open Run Detail on the Runs screen to check it.

Why do I see - in a heatmap cell?Because the item doesn't overlap across the selected Runs.

Cause: A Taxonomy or Strategy that exists in only one Run can't be compared with other Runs.
Solution: Check the explanation in What does "-" mean? above the heatmap.

Why do I see only 5 Taxonomies?Because only the top 5 are shown at first.

Cause: When there are many Taxonomies, the rest are collapsed to keep the screen short.
Solution: Click View {n} Taxonomy.

Why did the results in my saved dashboard change?Because a saved dashboard is a bookmark.

Cause: Results aren't frozen when you save; the latest results are shown each time you open it.
Solution: If you need results from a specific point in time, download them with Export in Run Detail.