Fail Case

Find turns with Metric violations in the list, and view the full conversation of the session each turn belongs to.

1. View the Fail Case list

The Fail Case list shows only turns with at least one Fail Case. Turns that passed every Rubric don't appear.

Column	Description
`Session ID`	ID of the session the turn belongs to
`Turn ID`	Turn ID
`Time Stamp`	Time of the turn, based on the User Input.
`User Input`	The question the user entered
`App Output`	The AI service's final answer
`Metric: Fail Case`	The Metrics and Rubrics judged as violations in this turn

Turns from the same session are grouped into one card.
The most recent sessions appear at the top. Turns within a session appear in conversation order.
Each page shows 50 sessions.

The Metric: Fail Case column uses the format Metric name: Rubric name.

Example	Meaning
`Safety: Profanity`	The Profanity Rubric in the Safety Metric was judged a violation
`Character identity: Tone, Childish`	Two Rubrics in the Character identity Metric were judged violations

1-1Open the list
1. Click Fail Case in the left menu.
2. Check the column headers.
See Fail Case list for what each column means.
1-2Check violating turns
1. Check the violating turns inside the session card.
2. Check the violations in Metric: Fail Case.
3. Use the page numbers at the bottom to see more sessions.
See Fail Case notation for how to read it.

If the list is long, narrow it down with filters and search.

In the Metric filter, select the Rubrics you want to see. Rubrics are grouped by Metric.
If you select several, turns matching any of them are shown.
Search looks in Session ID, User Input, and App Output.
Text matching your search term is highlighted.

2-1Apply a filter
1. Click the filter next to Metric.
2. Select the Rubrics you want to see.
3. The filter button shows your selection.
4. To undo it, click the reset icon.
2-2Search
1. Enter a Session ID or text in the search box.
2. Only matching sessions stay in the list.

Click a row in the list to open the Session panel on the right. Details are shown per session, not per turn.

Area	Contents
Header	Session ID, `Copy Link`, close
Summary	Total turns and the number of turns with Fail Cases
`Fail cases only`	When on, shows only turns with Fail Cases
Turns	Turn number and ID, time, response time (`Latency`), Fail Cases, conversation

3-1Open a session
1. Click a row in the list.
2. Check the Session ID in the Session panel.
3-2Review the conversation
1. Check the total turns and Fail Case turns in the summary.
2. Check the Fail Cases and conversation for each turn.
3. Turn on Fail cases only to show only Fail Case turns.
3-3Share a link
1. Click Copy Link and send the link to your teammates.
Opening the link opens the Session panel with this session already found.

When a Fail Case occurs, an email goes to the recipients set for the Metric. You set recipients in Evaluation settings.

Item	Contents
Subject	`[DATUMO Fail Alert - Turn ID] - Metric name`
`Metric Name`	The Metric with the violation
`Fail Reason`	The Rubric judged as a violation
`User Input`, `App Output`	The question and answer in that turn

4-1Check the email
1. Check the turn ID and Metric in the email subject.
2. Check the violation in the body.
4-2Open details
1. Click Go to Detail.
2. The Fail Case screen opens with that session open.

Why is the Fail Case screen empty?Because there's no Integration, or there are no Fail Cases yet.

Cause: Without an Integration, you see There are no integrations yet. With no violating turns, you see There are no Fail Cases yet.
Solution: If there's no Integration, connect one with Go to Integration Setting. See Set up an Integration for how.

Where do I see turns that passed?You can see them in the Session panel.

Cause: The list shows only turns with Fail Cases.
Solution: Click a row to open the Session panel. When Fail cases only is off, all turns in the session appear.

What should I do if turns from some time periods are missing from the list?Check the Gen & Judge model and Metric status for that period.

Cause: Turns from periods with no Gen & Judge model or with the Metric turned off aren't evaluated. Turns not picked by Sampling aren't evaluated either.
Solution: Check the model and Metric status for that period. Turns that weren't evaluated aren't evaluated later.

Why do I get no results after applying a filter?Because no turns match the conditions you selected.

Cause: When you use a filter and a search term together, only turns that meet both conditions appear.
Solution: Clear the filter with the reset icon, or delete the search term.