Platform Docs
Try DatumoEval

Create an Application

Create an Application to evaluate, set up its Modules and Versions, and edit or delete Applications and Modules on the Agent Connection screen.

1. Create an Application

①What is an Application

An Application is the basic unit of evaluation and monitoring. Each AI service you run maps to one Application. You register Applications inside a Workspace.

②Where to start and permissions

LocationHowVisible to
Application selector in the left menu+ New ApplicationOwner, Admin, Workspace Admin
Workspace General Setting+ Add ApplicationOwner, Admin, Workspace Admin

③Fields

FieldDescription
Application NameA name that identifies it within the Workspace
TypeApplication type (e.g., Knowledge-based QA)
DomainService domain (e.g., Finance)
LanguageKorean (KR), English (EN), ETC
Input/Output KeyDefaults to Input Key query and Output Key response. Each needs at least one key.
Module ManagementAdd Modules to evaluate pipeline components separately. Optional.

Keys can be of type string, list, boolean, number, or json.

  1. 1-1Open the creation dialog

    1. Open the Application selector in the left menu.
    2. Click + New Application.

    You can also open it with + Add Application in Workspace General Setting.

  2. 1-2Enter details

    1. Enter an Application Name.
    2. Select a Type and a Domain.
    3. Select a Language.
    4. Check the Input/Output Key.

  3. 1-3Add Modules

    1. To evaluate Modules separately, click + Add Module and select a Module Type.

    You can also add Modules after creating the Application in 3. Manage Modules.

  4. 1-4Finish

    1. Click Complete.
    2. Choose what to do next. Go to Application Setting takes you to Agent Connection, where you connect the API.

    Explore Criteria takes you to the evaluation criteria screen.

2. Agent Connection

①Screen layout

Click Application Setting at the bottom of the left menu to open Settings > Application > Agent Connection. Editors and Viewers don't see this menu.

AreaContents
Application cardType, Domain, Language, Input Key, Output Key, the Available in Workspace switch, and the edit button
Version ListList of Application Versions. Shows the integration step and Token&Cost settings.
Module ListKeys and Version list for each Module. Add one with + New Module.

②Version status

IndicatorMeaning
Red API IntegrationAPI integration is required. Click it to open the integration dialog.
Output MappingAfter API integration, map the response.
ReadyIntegration and mapping are done, and the Version is ready for evaluation.
ButtonAction
+ New Version from ScratchCreates a new Version with no existing settings.
+ New Version from LatestCreates a Version from the most recent Version's settings. Multi-turn settings and Turn Definition are copied too.

  1. 2-1Open the screen

    1. Check the basic details and Keys on the Application card.
    2. Check the integration status of each Version in Version List.

  2. 2-2Integrate the API

    1. Click the red API Integration. Application API integration explains how to integrate.

    To run an evaluation, finish API Integration and Output Mapping for each Version.

  3. 2-3Add a Version

    1. Click + New Version from Scratch or + New Version from Latest.
    2. Integrated Versions show Ready and their Token&Cost settings.

    Use ⋮ on a Version row to duplicate or delete it.

  4. 2-4Hide from the Workspace

    1. Turn off the Available in Workspace switch. The Application is hidden from the Application selector.

    Hiding an Application doesn't delete it. Turn the switch back on to show it again.

  5. 2-5Edit or delete an Application

    1. Click the edit button at the top right of the card.
    2. Change the details and click Save.
    3. To delete it, click Delete, then click Delete again in the confirmation dialog.

    You can't undo deleting an Application. All its Versions, Modules, and evaluation results are deleted.

3. Manage Modules

Add, edit, or delete Modules in Module List. Changing a Key means you have to set up that Module's Versions again, so the full steps are covered under Module setup in Application API integration.

Frequently asked questions

Why don't I see Application Setting in the left menu?Because your role in this Workspace is Editor or Viewer.
Cause
Application Setting is visible only to Workspace Admins and above.
Solution
Ask a Workspace Admin to change your role.
Why can't I click Complete or Save?Because a required field is empty or a Key is missing.
Cause
You need at least one Input Key and one Output Key. In the edit dialog, the button is also disabled when nothing has changed.
Solution
Fill in the empty fields and keep at least one Key.
Why can't I run an evaluation?Because the Version's API integration and Output Mapping aren't finished.
Cause
A Version that still shows a red API Integration can't be used for evaluation.
Solution
Follow Application API integration to finish the integration.
What should I do if my Application doesn't appear in the selector?Turn on the Available in Workspace switch in Agent Connection.
Cause
Applications with the switch off are hidden from the Workspace list.
Solution
Turn the switch on in Agent Connection.

Last updated