Application API integration

①What integration enables

Once integrated, Evaluation, Red Teaming, and Observability call your service API directly, so you can run evaluation, attack, and observation jobs. Finish Create an Application first.

You integrate in Settings > Application > Agent Connection. Click Application Setting at the bottom of the left menu to open it. For the screen layout, see the Agent Connection screen in Create an Application.

②Integration steps

Integrate in the order below. The Key structure you define in step 1 is the basis for every later step.

①1-1. Create a Module

1. Click + New Module.

2. Select a Module Type. You can select multiple Module Types.

3. Click Add.

②1-2. Edit a Module

1. Click the edit (pencil) icon at the top right of the Module.

2. Edit the Module name, Input Keys, and Output Keys.

3. Add Keys if needed.

③1-3. Delete a Module

1. Click the Delete button.

2. Type Delete module in the field. The Delete button becomes active.

3. Click Delete. The Module is deleted.

①3-1. Enter integration details

1. Click API Integration on the Version row.
2. Enter the details below.

②3-2. Write the Request Body

1. Write the request body in JSON.
2. Insert the Input Keys from step 1 in {{key_name}} format. During evaluation, they're replaced with the actual values from the Benchmark Set.

{
  "model": "gpt-4o",
  "messages": [
    { "role": "system", "content": "You are a helpful assistant." },
    { "role": "user", "content": "{{user_query}}" }
  ],
  "temperature": 0.2
}

In the example above, {{user_query}} is replaced at run time with the user_query value of each Benchmark row.

③3-3. Test the connection

1. Enter a sample value for each variable in Test Connection on the right.
2. Click Test Connection.
3. Check that a valid response comes back.

Catching auth or format errors here prevents mass failures during evaluation.

④3-4. Save

If you close the dialog without saving, you're asked whether to save.

①4-1. Turn on Multi-turn

1. Turn on the Multi-turn toggle on the left of the API Integration dialog.
2. The Request Body is filled with the default example below.

{
  "messages": [
    ...{{chat_history}},
    { "role": "user", "content": "" }
  ]
}

• The previous conversation goes in place of ...{{chat_history}}. It must appear exactly once in the Request Body.
• The object right after it is the question sent in the current turn.
• "content" is empty. When you select an input variable in 4-2, it's filled in, like "{{query}}".

②4-2. Set up Turn Definition

In Turn Definition, define which values fill one turn of the previous conversation. With the default example, you only need to fill the two cells in the {{content}} row.

1. In the User Turn cell of the {{content}} row, select an input variable (e.g., {{query}}).
2. In the Assistant Turn cell of the same row, enter the path to the agent's answer. If you don't know the path, click Pick from response, select the answer in the response, and click Select.
3. If you need more fields, add them with + Add Variable. They're also added to the Request Body automatically.

You can't move on to the next step while any cell is empty.

③4-3. Test a two-turn connection

Test Connection for Multi-turn has Turn 1 and Turn 2 fields.

1. Enter a first question in Turn 1 and a follow-up question in Turn 2.
   (e.g., My name is Datumo. → What was my name?)
2. Click Test Connection.
3. Check that the Turn 2 response remembers what was said in Turn 1.

The test counts as complete only when both turns succeed. To rerun a failed turn, hover over its result and click the retry icon.

①5-1. Enter response paths

1. Enter a path in the response for each Output Key. If you don't know the path, use Pick from response to select a value from the response.

2. Check that the value under Extracted Value below the path is correct.
3. Click Save.

• If a path is wrong, a red message appears and you can't save.
• If you haven't tested the connection, you're asked whether to test when you save.

②5-2. Token usage and cost (optional)

Set this up only if you want to see costs. Once set, costs appear in the Usage tab of the Run Report and in Agent Simulation.

1. Under Token Usage & Cost at the bottom of the same dialog, enter the Input Tokens and Output Tokens paths. You can also select them with Pick from response.
2. Enter the price per 1M tokens in Input token cost ($) and Output token cost ($).
3. When the estimated cost appears in the box below, click Save.

Common token paths are listed below.

• To set it up later, click Go to Setting on the Version row. To change it, click Edit Token.
• For Runs that already ran, setting it up in the Run Report recalculates the cost. See Eval Report.