Create and configure a Task

In Evaluation Lab, set a Target Model, Judge, and Objectives to create a Red Teaming Task for testers to attack.

1. View the Task list

①Task

A Task is a single red team event where people attack a model directly. It consists of a Target Model, a Judge, and a list of Objectives.

Component	Meaning
Target Model	The model testers attack
Judge	How tester submissions are scored
Objective	The attack goal testers must achieve

The structure and terms of a Task are explained in Playground Core Concept.
Admins create Tasks, and testers attack from the Annotation Queue described in Run tester attacks.

②Task status

The Evaluation Lab list is split into Active, Closed, and All tabs. The number next to each tab is the Task count.

Status	Meaning
`Active`	Testers can attack and submit. A Task is in this status as soon as it's created.
`Closed`	Hidden from testers and not accepting new submissions. You can still view results.

Column	Meaning
`Created at`	Creation date
`Task Name` · `Description`	Name and description
`Target Model`	The model under attack
`Objective Count`	Number of Objectives
`Status`	`Active` / `Closed`

1-1Open the list
1. In the left menu, click Evaluation Lab under Playground.
2. Check ongoing Tasks in the Active tab.
3. See finished Tasks in the Closed tab.
See Task status for what each status means.
1-2Open a Task
1. Click a Task row to open Task Detail.
Task Detail is explained in View results and manage Tasks.

2. Enter basic info

①Task Setup

Click New Task to open the three-step creation screen (Task Setup → Judge & Objective Configuration → Confirm). In the first step, enter the basic info.

Field	Required	Description
`Task Name`	Required	The Task name. If it's empty, you can't move to the next step.
`Description`	Optional	The Task description
`Target Model`	Required	The model version to attack

Target Model lists only models that support multi-turn conversations. Connect models in Application Setting (Settings > Application > Agent Connection). See Integrate the Application API for how.
You can't change the Target Model after creating the Task.

2-1Open the creation screen
1. Click New Task at the top right of the list.
2. Check that the step indicator shows Task Setup.
2-2Enter basic info
1. Enter a Task Name.
2. Enter a Description if needed.
3. Select a Target Model.
4. Click Next.
You can't change the Target Model after creation. See Task Setup.

3. Set the Judge and Objectives

①Judge Type

A Judge is how tester submissions are scored. The Judge you choose here becomes the default for the whole Task.

Judge Type	Who scores	When to use it
`Self Evaluation`	Testers mark each submission as attack success/failure themselves	When the criteria are simple or human judgment is needed
`Preset Rubric`	An LLM scores with prebuilt criteria from the rubric library	When you want fast scoring with general safety criteria
`Custom Rubric`	An LLM scores with criteria you write or adapt from a preset	When you need criteria tailored to your service

For Preset Rubric and Custom Rubric, you also choose the Judge Model that does the scoring.

②Objective

An Objective is a failure type testers must draw out. A Task can have multiple Objectives.

Field	Required	Description
`Attack Objective`	Required	The goal name (e.g., `Steal personal information`).
`Description`	Optional	The Objective description
`Note to Tester`	Optional	The response expected when the attack succeeds. Visible to testers.

Add Objectives with New Objective, and delete them with the trash icon.
With Judge Edit, each Objective can use a Judge different from the default. In that case, you also set the values below.

Field	Description
`Attack Success Threshold`	How many Rubric criteria must be met for the attack to count as a success (e.g., `1 of 3 rubric`).
`Show Rubric to Testers`	Whether to show the Rubric and Threshold in testers' Chat Playground

3-1Choose a Judge
1. Choose a scoring method in Judge Type.
2. If you score with a Rubric, select a model from the Judge Model list. Hover over a model to see its model ID on the right.
3. For Custom Rubric, enter the criteria yourself.
4. In Attack Success Threshold, choose how many criteria must be met for the attack to count as a success.
See Judge Type for how the methods differ.
3-2Enter Objectives
1. Enter an Attack Objective.
2. Enter a Note to Tester for testers.
3. To add another Objective, click New Objective.
Note to Tester tells testers what a successful attack looks like.
3-3Per-Objective Judge
1. On the Objective that should use a different Judge, click Judge Edit.
2. Choose an Attack Success Threshold.
3. Use Show Rubric to Testers to decide whether testers see the Rubric.
4. Click Next.
Once there's even one submission, you can't change the Judge.

4. Confirm and create

①Confirm

In the last step, review what you entered and create the Task.

The top shows the Task Name, Description, and Target Model.
Below, each Objective shows its description, Note to Tester, Judge Type, and Judge Model.
If something needs fixing, go back with Back.
Click Create Task to create the Task in Active status. It appears in testers' Annotation Queue right away.

4-1Review and create
1. Review the basic info.
2. Review the Objective and Judge settings.
3. Click Create Task.
A new Task becomes Active right away and is visible to testers.

Frequently asked questions

Why doesn't clicking Next take me to the next step?Because Task Name is empty.

Cause: Task Name is required, so if it's empty you see the message Enter a task name.
Solution: Enter a Task Name and click again.

Can I score each Objective with different criteria?Yes. You can change the Judge per Objective.

Cause: The Task's Judge Type is the default, and you can set a separate one on an Objective with Judge Edit.
Solution: Set the Judge and Attack Success Threshold in Judge Edit.

Can I change the Target Model of a Task I already created?No. You can't change it after creating the Task.

Cause: This keeps submission results comparable against the same model.
Solution: Create a new Task. Before creating it, make sure to check the model in the Confirm step.