Run and view results (Runs)

Start a Red Teaming Run with Quick Start or Advanced Setup, manage it in the Run list, and view results in Run Detail.

1. Start a Run with Quick Start

①Runs and Batches

A Run is a record of one Red Teaming execution against one version of a Target Model. A Batch is a group of Runs executed at once against multiple versions with the same Seed Set, Algorithm, and Judge Model.

If you select 2 versions in New Run, 2 Runs are created and grouped into one Batch.
Child Run names follow the format [Batch name]_R1, [Batch name]_R2 ….
Before you run, it helps to prepare a Seed Set in Seed Management and default models in Model setting.

②Run modes

Click New Run to open a dialog where you choose a run mode.

Mode	Steps	When to use it
`Quick Start`	Target Model → Seeds → Algorithm → Review & Run	Sets strategies and Judge automatically for a fast run.
`Advanced Setup (Custom)`	Target Model → Seeds → Algorithm → Judge → Review & Run	Lets you configure Taxonomy, strategies, algorithm parameters, and Judge yourself.

To use New Run, at least one Gen & Judge Model must be registered. See Register Gen & Judge models.

③Target Model

In E2E Application & Module, select one target to evaluate, and in Version, select one or more versions.

One Run is created per selected version (N runs will be created — one per selected version).
If you don't select any version, Select at least one version. appears and Next is disabled.

④Seeds

Select one Seed Set. The Seed Set's Domain determines the Taxonomy the Judge uses.

You can search by name or filter by Domain.
Seed Sets that are being generated or in an Error state don't appear in the list.

⑤Algorithm

Select one algorithm. Cards are grouped into Single-turn and Multi-turn.

Card element	Meaning
Description	How the algorithm creates attack prompts
`★ Best for`	What this algorithm is well suited for
`Performance` · `Speed` · `Cost`	Relative characteristics shown with 5 dots. More dots means higher.

⑥Review & Run

Review your settings, set a Batch Name, and run.

Batch Name is required. Whitespace only counts as empty.
If the name already exists, This batch name is already in use. Please enter a unique name. appears.
Below, the name of each Run that will be grouped into this Batch is shown per version.
In Quick Start, you select the Judge Model in this step. The initial value is the default Judge model from Model setting.
Click Edit → on a summary card to go back to that step.

1-1Choose a run mode
1. Click New Run in the top right of the Runs screen.
2. Click Quick Start.
To configure it yourself, choose Advanced Setup (Custom). See Run modes.
1-2Target Model
1. In E2E Application & Module, select the target to evaluate.
2. In Version, select one or more versions to evaluate.
3. Check the number of Runs that will be created.
4. Click Next.
One Run is created per version.
1-3Seeds
1. Search by name or filter by Domain.
2. Select one Seed Set.
3. Click Next.
If the Seed Set you want isn't listed, check its status in Seed Management.
1-4Algorithm
1. Select one algorithm card.
2. Compare ★ Best for and Performance · Speed · Cost.
3. Click Next.
1-5Review & Run
1. Enter a Batch Name.
2. Check the Judge Model.
3. Review the summary. To change an item, go back with Edit → and fix it.
4. Click Run Evaluation.
When you run it, a new Batch is added to the Runs list.

2. Configure details with Advanced Setup

①Seed Taxonomy

In the Seeds step of Advanced Setup, selecting a Seed Set fills in its Taxonomy as tags. Deselect the Taxonomy tags you want to exclude.

②Strategy

If you select a Strategy-based algorithm in the Algorithm step, the Strategy section appears. You can select multiple strategies in each category.

Some strategies are prefixed with the source of the open-source strategy (Garak, PyRIT, STAR, AutoDAN, etc.).
Selecting a strategy group also selects the strategies in that group.

Selected algorithm	Strategy section
`STAR-Teaming`	Strategies from STAR are selected automatically. You can deselect them.
`AutoDAN-Turbo`	Strategies from AutoDAN are selected automatically. You can deselect them.
`Single shot`	The default strategy set is selected. You can deselect it.
`PAIR`, `TAP`, `Multi-turn`, etc.	Instead of strategies, a note says "The algorithm determines strategies automatically."

③Configuration

Set per-algorithm values in Configuration at the bottom of the Algorithm step. It always appears once you select an algorithm.

Attacker is the model that creates attack queries, and Scorer is the model that scores attack results.
Parameters such as max_iterations, threshold, and max_turn vary by algorithm. The ⓘ next to each name has a description.
The initial values are the per-algorithm defaults from Model setting.

④Judge

Set how judgments are made in the Judge step. Quick Start doesn't have this step.

Item	Description
`Attack Success`	Always on and can't be turned off. The basis for Safe/Unsafe verdicts and ASR (Unsafe/Total).
`Add-on : How bad were the failures?`	Adds a `Harmfulness` column to Unsafe rows so you can surface the most severe failures first.
`Add-on : Did Safe answers actually refuse?`	Adds a `Refusal` column to Safe rows to distinguish active refusals from incidental safety.
`Judge Model`	The model that scores each attempt. The initial value is the default Judge model.

The Preview table shows what results look like with 4 example rows (2 Unsafe, 2 Safe). Turning on an Add-on adds its column right away.
Review & Run in Advanced Setup also shows summaries of Seed Taxonomy, Strategy, and Judge. You can change the Judge Model only in the Judge step.

2-1Start Advanced Setup
1. In the New Run dialog, click Advanced Setup (Custom).
2. Check that there are 5 steps (including Judge).
The Target Model step is the same as in Quick Start.
2-2Seeds and Taxonomy
1. Select one Seed Set.
2. Check the filled-in Seed Taxonomy tags.
3. Deselect the Taxonomy tags you want to exclude.
2-3Algorithm and strategies
1. Select an algorithm.
2. In the Strategy section, select the strategies to use.
3. In Configuration, check the Attacker, Scorer, and parameters.
4. If you see a red message, change the value to one within range.
The Strategy section appears only for Strategy-based algorithms. See Strategy.
2-4Configure Judge
1. Check that Attack Success is on.
2. Turn on the Add-ons you need.
3. Check the added columns in the Preview table.
4. Select a Judge Model and click Next.
2-5Review and run
1. Enter a Batch Name.
2. Check the Seed Taxonomy, Strategy, and Judge summaries.
3. Click Run Evaluation.

3. View the Run list

①List layout

On the Runs screen, child Run rows are listed indented under each Batch row. A Batch row's ASR aggregates the ASR of its child Runs. Runs and Batches are explained in Runs and Batches.

Column	Description
`Batch · Run`	Batch name and child Run names. While running, progress and ETA are shown.
`Module`	The evaluation target Module (`Application`, `RAG`, `Retriever`, etc.)
`Target`	The evaluation target model and version
`Algorithm`	The attack algorithm used
`Seed`	The Seed Set used
`ASR`	Attack success rate (Unsafe / total responses)
`Elapsed · Cost`	Time taken and cost
`Start Date`	The date the run started

②List filters

Use the filters above the list to narrow down the Runs you see. When you select multiple items, they're listed in the filter field separated by commas (,).

Filter	Selection	Description
`Batch`	Multiple	Shows only Batches that actually ran. Renaming a Batch is reflected immediately.
`Target`	Multiple	Grouped by Module.
`Algorithm`	Multiple	Shows only algorithms actually used in Runs.
`Seed`	Multiple	Shows only Seed Sets actually used in Runs.
`Status`	One or multiple	`In Progress` / `Completed`
`ASR ≥`	Lower bound	Change it by 1 with ▲▼ or enter it directly.

When you select at least one filter, a reset (↺) button appears. Click it to clear all filters.

③Run statuses

What you can do in the list depends on the Run status.

Category	Running (`Running`, `Pending`)	Completed (`Completed`)
Checkbox selection	Not available	Available
Edit Run · Batch name	Not available	Available (✎ icon)
`ASR`, `Elapsed · Cost`	Shown as `—`	Value shown
Button on the right	`Stop Running`	`Compare Runs →`

Runs with Errors show the Error count on the right (e.g., 3 error).

3-1Open Runs
1. In the left menu, click Runs under Red Teaming.
2. Check the Batch rows and the Run rows below them. Column meanings are in List layout.
3-2Find Runs with filters
1. Select conditions in the filters at the top (e.g., Completed in Status).
2. If you need an ASR lower bound, enter a value in the ASR ≥ field.
3. To clear all filters, click reset (↺).
See List filters for how each filter is selected.

4. Manage Runs

①Stop a Run

Stop a running Run with Stop Running.

Clicking it on a Batch row stops all running Runs in that Batch.
Clicking it on a Run row stops only that Run.

Stopped Runs and Batches can't be restarted. To run again, create a new New Run.

②Batch name

Rename a completed Batch with the ✎ icon next to its name.

You can't use the same name as another Batch.
When you rename a Batch, its child Run names also change to the [New name]_R1 format.

③Compare and delete

Select the checkboxes of completed Runs to show buttons at the top.

Button	What it does
`Compare {N} runs`	Compares the selected Runs in the Dashboard.
`Delete {N} runs`	Deletes the selected Runs after a confirmation dialog.
`Compare Runs →` on a Batch row	Compares all Runs in that Batch in the dashboard.

If nothing is selected, only the text Select runs to compare or delete appears.

4-1Stop a Run
1. On a running Run row, click Stop Running.
2. In the confirmation dialog, click Stop.
Stopped Runs can't be restarted. See Stop a Run.
4-2Rename a Batch
1. Click ✎ next to a completed Batch name.
2. Enter a new name.
3. Click Save.
You can't use the same name as another Batch.
4-3Compare or delete Runs
1. Select the checkboxes of the Runs to compare or delete.
2. To compare, click Compare {N} runs.
3. To delete, click Delete {N} runs, then click Delete in the confirmation dialog.
4. To compare an entire Batch, click Compare Runs → on the Batch row.
The comparison screen is covered in the Dashboard document.

5. View Run results

①Run Detail summary

Click a Run row to open Run Detail. The text at the top shows [Algorithm] attacking [Target] on [Seed] and the finish time (or expected finish time).

Card	What it shows
`ASR`	Unsafe count / total responses. Attempts with Errors are excluded from the count (errored attempts excluded).
`Coverage`	Completed attempts / total, and the number of Taxonomies and Strategies covered
`Error`	Number of judge errors and target timeouts
`Resource`	Time taken, cost, and time per attempt

The screen differs by Run status.

Category	Running	Paused / Completed
Top text	`expected to finish at {ETA}`, progress bar	Completed: `finished at {time}` / Paused: `stopped at {progress}`
`Stop Running` button	Shown	Hidden
`Export`, `Create Report (PDF)`, delete	Disabled	Enabled

②Results table

Each row in the results table is one attack attempt. You can sort by any column.

Column	Description
`ID`	Attempt number
`Seed`	The Seed used in the attack
`Result`	`Safe` / `Unsafe` verdict
`Taxonomy`	The Seed's Risk Taxonomy
`Attack Prompt`	The attack query created by the Attacker
`Answer`	The Target model's response
`Strategy`	The strategy applied
`Iter`	Iteration count

Seed rows that haven't run yet appear in faded text, with the other cells empty. Clicking a row that hasn't been judged doesn't open details.
If you turned on Add-ons in the Judge step of New Run, Harmfulness (Unsafe rows) and Refusal (Safe rows) columns are added. Rows they don't apply to show —.

③Data Detail

Click a row to open the Data Detail panel on the right.

The Seed, Result, and Taxonomy appear at the top.
Iteration History lists results for each iteration, starting from the latest Iter. Each entry shows the Iter number, Strategy, and Result.
Expand an entry to see the Attacker question and {Target} response as a pair. For multi-turn, several pairs follow one another.

④Export

Download a Paused or Completed Run as a CSV file. Only rows with results are exported.

Option	Single-turn	Multi-turn
`Last iteration only`	One row per Seed. Writes the last Iteration's Attacker / Response in separate columns.	Writes the full conversation of the last Iteration.
`All iterations`	Writes the Attacker / Response of every Iteration, each in its own row.	Writes the full conversation of every Iteration, each in its own row.

5-1Open Run Detail
1. Click a Run row.
2. Check the Algorithm, Target, Seed, and finish time in the text at the top.
3. Check ASR, Coverage, Error, and Resource on the summary cards. Their meanings are in Run Detail summary.
5-2Stop a running Run
1. Check the progress bar and ETA.
2. Click Stop Running.
3. In the confirmation dialog, click Pause.
A stopped Run can't be restarted or resumed.
5-3View attempt details
1. Click a row in the results table.
2. Check the Seed, Result, and Taxonomy at the top of Data Detail.
3. In Iteration History, expand an Iter to see the attack query and response.
4. To close the panel, click ✕.
Rows that haven't been judged don't open details.
5-4Export results
1. Click Export.
2. Choose an Export option.
3. Click Export to download the CSV file.
You can't export a Run that's still running.

Frequently asked questions

Why can't I click Next in the Target Model step?Because you haven't selected any version.

Cause: A Run is created per version, so you need at least one version. In this case, Select at least one version. appears.
Solution: Select one or more versions in Version.

What should I do if the Seed Set I want isn't in the list?Check the Seed Set's status in Seed Management. Seed Sets that are being generated or in an Error state don't appear in the list.

Cause: The Seeds step shows only Seed Sets that are ready to use.
Solution: Check its status in Seed Management. See Seed Management.

Why do I see This batch name is already in use.?Because that Batch name already exists.

Cause: A Batch name can't be the same as another Batch's. When names overlap, Run Evaluation can't be clicked either.
Solution: Enter a different name.

Why don't I see the Strategy section?Because you selected an algorithm that determines strategies automatically.

Cause: The Strategy section appears only for Strategy-based algorithms such as Single shot, STAR-Teaming, and AutoDAN-Turbo.
Solution: To choose strategies yourself, select a Strategy-based algorithm. Quick Start has no Strategy section.

Why do I see red text under a Configuration field?Because the value is outside the allowed range.

Cause: Each parameter has a defined range, such as a maximum value.
Solution: Change the value to match the message (e.g., Max: 150).

Can I restart a stopped Run?No. Stopped or paused Runs can't be restarted.

Cause: Stop Running and Pause put the Run in a finished state.
Solution: Create a new New Run with the same settings.

Why can't I select the checkbox or rename the Batch?Because the Run is still running.

Cause: Runs and Batches in the Running or Pending state can't be selected or renamed.
Solution: Try again after the Run finishes. See Run statuses.

Why can't I click Export in Run Detail?Because the Run is still running.

Cause: Export, reports, and delete are available only when the Run is Paused or Completed.
Solution: Use them after the Run finishes or is stopped.