> For the complete documentation index, see [llms.txt](https://docs.datumo.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.datumo.com/documentation/documentation-en/get-start/how-to-use/5.-connect-application-api.md).
# 5. Connect Application API
## Application API Integration
Connect the actual AI service API to the Module and Version of the created Application. Once integration is complete, the Evaluation, Redteaming, and Observability modules can directly call the service to perform evaluation, attack, and observation tasks.
***
### Integration Overview
Application API integration proceeds in the following four steps. Each step follows sequentially, and the Key structure defined in Step 1 becomes the standard for all subsequent steps.
1. **Module setup:** Define the Input/Output Keys of the module to be evaluated
2. **Version setup:** Register versions for Before-After comparison
3. **API integration by version:** Connect the real API endpoint for each version
4. **Output Key mapping by API:** Automatically map API responses to Output Keys
## Module
A Module is a functional unit that makes up an Application. You can select and configure the Module Type according to the specific function you want to evaluate.
**Module Key**
Each Module has `Input Key`and `Output Key`.
* `Input Key` : This is the input value passed to the Module.
* `Output Key` : This is the output value generated after the Module runs.
### 1. Create Module
`+ New Module` Select
After selecting Module Type, `Add` click the button. Multiple Module Types can be selected.
### 2. Edit Module
Click the edit icon at the upper right of each Module.
You can edit the Module name and Input/Output keys.
* `+Add Input Key` Select it to add an Input Key.
* `+Add Output Key` Select it to add an Output Key.
However, if even one of the Input keys or Output keys is edited, all version settings belonging to that Application must be redone from the beginning.
### 3. Delete Module
`Delete` Select the button.
Type "Delete module" in the input field `Delete` the button becomes active, and `Delete` selecting the button deletes the Module.
***
## Version
The DATUMO platform can manage a single Application across multiple Versions. Whenever there are changes such as prompt modifications, model changes, or search logic improvements, registering a new Version allows you to compare Before and After under the same evaluation criteria.
### 1. Create Version
#### Version operation tips
* **Separate Versions by change unit**: If you change multiple things at once, you cannot isolate which change contributed to the score. Splitting Versions by a single change unit as much as possible is the basis of quantitative analysis.
* **Unify naming conventions**: If the team agrees on Version naming rules in advance, time-series comparisons on the dashboard become easier.
* **Prefer deactivation over deletion**: Rather than deleting failed Versions, preserve them in a deactivated state so regression tests and reproduction verification remain possible later.
### 2. API integration by Version
Each Version must be connected to a callable AI service API in order to run evaluation, attacks, and observation. You can assign a separate endpoint to each Version, making it possible to apply the same evaluation set simultaneously across multiple environments (staging, production, new model).
#### Enter integration information
In the Version list, from the target Version's **API Integration** area, enter the following information for E2E or each Module.
**Endpoint information**
* **Method**: `GET`, `POST`, `PUT` Select one (most LLM APIs `POST`)
* **URL**: Full address of the API to call
**Header** Add request headers such as authentication token and Content-Type as key-value pairs. Sensitive information (API Key, Bearer Token) can be `Secret` registered as an option and stored and transmitted safely in a masked state.
**Request Body** Write the request body template in JSON format. The **Input Keys defined in Step 1 can be inserted as variables in the format `{{key_name}}` .**They will be automatically replaced with the actual values from the benchmark set during evaluation.
```
{
"model": "gpt-4o",
"messages": [
{ "role": "system", "content": "You are a helpful assistant." },
{ "role": "user", "content": "{{user_query}}" }
],
"temperature": 0.2
}
```
In the example above, `{{user_query}}` part is replaced at runtime with the `user_query` value from each row of the benchmark set. The variable name must exactly match the Input Key name defined in Step 1.
#### Connection test
Before saving the settings, **Test Call** button lets you check whether the actual API is called normally. If you fill in test input values arbitrarily and make the call, the response code (`200 OK` etc.) and Raw Response are displayed on the screen. Catching authentication or body-format errors at this stage can greatly reduce mass failures that occur during evaluation runs later.
***
### 4. Output Key mapping by API
Even if the API is called successfully, DATUMO needs to be told separately **which value in the response**should be treated as the evaluation target. **At this stage, based on the actual response structure received from the Test Call in Step 3,**map the Output Key and the path in the response JSON one-to-one
#### JSONPath-based mapping
Enter the path of the response JSON next to each Output Key [JSONPath](https://goessner.net/articles/JsonPath/) notation. For example, if the response of the OpenAI ChatCompletion API is as follows,
```
{
"id": "chatcmpl-xxx",
"choices": [
{
"message": {
"role": "assistant",
"content": "Refunds can be requested in My Page > Order History."
},
"finish_reason": "stop"
}
],
"usage": { "total_tokens": 142 }
}
```
`answer` The Output Key is `$.choices[0].message.content` If you receive search results together in a RAG service `retrieved_docs` The Output Key is `$.context.documents`and want to specify according to the response structure, or if you want to analyze token usage separately, `$.usage.total_tokens` you can also add a Key for that path.
#### Mapping verification
After finishing the mapping **Preview** button, you can preview which values each Output Key actually extracts from the Test Call response. Be sure to verify that the intended value is extracted correctly before saving. Incorrectly mapped Keys can create noise across all evaluation results, making it impossible to trace the cause of score changes.
#### Handling missing Output Keys
An API response may not always contain a value for a specific Output Key. For example, when there are no search results, `retrieved_docs`, and the `tool_calls`in a turn without tool calls. **Default Value** specification or **Allow Null** options can be set so that evaluation runs do not stop.
***
### After integration is complete
Once all four steps are complete, the Version will be **Integration complete** status, and the following tasks are now possible.
* **Run evaluation in the Evaluation module**: [E-5. Run Eval](https://app.gitbook.com/o/Bis7EnmqiVzacnbqBO3C/s/JTwnb351jHLY7t48GvYX/~/edit/~/changes/DMRf5OO7Z5z4Xpgr0Hqc/evaluation/how-to-work/5.-run-eval)You can select the Application and Version shown here to perform benchmark-set-based evaluation.
* **Run attack in the Redteaming module**: [R-2. Run Auto Redteaming](https://app.gitbook.com/o/Bis7EnmqiVzacnbqBO3C/s/JTwnb351jHLY7t48GvYX/~/edit/~/changes/48/redteaming/how-to-work/2.-auto-redteaming)You can automatically generate and execute adversarial scenario attacks against the Application shown here.
* **Collect real-user conversations in the Observability module**: [O-2. Scenario Setting](https://app.gitbook.com/o/Bis7EnmqiVzacnbqBO3C/s/JTwnb351jHLY7t48GvYX/~/edit/~/changes/48/observability/how-to-work/2.-scenario-setting)You can collect and observe real traffic from the Application shown here.
***
### 🔗 Related documents
* G-1. Create Workspace — Create a Workspace to perform evaluation work
* G-2. User Management — Invite users to the Workspace and grant permissions
* G-3. Register Gen & Judge models — Configure the Gen/Judge models used for evaluation
* G-4. Create Application — Register the Application to be evaluated