Set up an Integration

Connect your AI service's log store, convert logs into turns with parser code, and integrate them with Observability.

1. Start an Integration

①Integration

An Integration is the connection setting that brings your AI service's user logs into Datumo Platform. Each Application uses one Integration at a time.

AWS CloudWatch logs are currently supported.
Setup has two steps. In step 1, you verify the connection details. In step 2, you run the parser code.
Each time you change the settings, the version goes up by 1. Previous settings stay in Integration History.

The log hierarchy and evaluation units are explained in Observability Core Concept.

②What you need

Item	Description
AWS Region	The region where the logs are stored
Access Key, Secret Key	AWS keys that can access that Region
Log Path	The path to load logs from (e.g., `/aws/lambda/my-service`).
Parser code	Python code that splits logs into turns

Ask your infrastructure team for the keys and path.

1-1Open settings
1. Click Setting at the bottom of the left menu.
2. Click the Observability tab.
1-2Start a new Integration
1. Under Integration, click + New Integration.
If there's no Integration, the Fail Case screen shows There are no integrations yet.

2. Enter connection details

①Basic Setting fields

Field	Description
`Region`	The AWS Region where the logs are stored. Select it from the list.
`Access Key` / `Secret Key`	AWS keys for accessing the Region. Input is masked.
`Log Path`	The path to load logs from

②Test Connection

Test Connection tries a real connection with the details you entered. If it succeeds, the loaded logs appear in Integrated Logs.

It loads up to the 100 most recent logs.
The loaded logs are used in step 2 to check the parser code output.
Once the connection succeeds, the input fields are locked.

If the connection fails, an error message appears.

Error	Cause	What to check
`Invalid access credentials`	The keys are wrong or lack access to the Region	Key values and Region permissions
`Log path not found`	The path you entered doesn't exist	The path and its format
`No logs found`	Connected, but there are no logs	The path and the time range

2-1Enter connection details
1. Select a Region.
2. Enter the Access Key and Secret Key.
3. Enter the Log Path.
2-2Test the connection
1. Click Test Connection.
2. Check the loaded logs in Integrated Logs.
3. Click Next.
Test Connection turns on only after you fill in all four fields.
2-3Handle a failed connection
1. Check the error message.
2. Click Set up again and fix the values.
See Test Connection for the cause of each error.

3. Run the parser code

①Parser code

Parser code is Python code that finds user questions and AI answers in raw logs and groups them into turns. Log formats differ by service, so you write it yourself.

See Check Parser Guide on the screen for how to write it.
The code must return JSON.

Error	Cause
`Json Format Error`	The code ran, but the output isn't valid JSON.
`SyntaxError (Compile Error)`	The code has a syntax error and doesn't run.
`TypeError (Runtime Error)`	An operation used incompatible data types during execution.

②Mapping Preview

Mapping Preview shows how the parser output maps to the evaluation columns.

Column	Meaning
`session_id`	ID of the session the turn belongs to
`turn_id`	Turn ID
`timestamp`	Message time
`type`	`User Input` or `App Output`
`message`	Message content

Rows with empty required values are highlighted in red and excluded from evaluation.
If no rows can be evaluated, Complete is turned off.
Check that User Input and App Output alternate. If they're out of order, the evaluation results will be wrong.

3-1Run the code
1. Check the raw logs in Integrated Logs on the left.
2. Enter your code in Parser Code (Python).
3. Click Run.
See Go to Docs in Check Parser Guide for how to write the code.
3-2View results
1. Check the returned JSON in Parser Code Output.
2. Check the column mapping in Mapping Preview.
3. Red rows have empty required values and are excluded from evaluation.
See Mapping Preview for what each column means.
3-3Edit the code
1. If the output isn't what you expect, click Edit.
2. Fix the code and click Run again.
3-4Complete the integration
1. Click Complete.
- The completion time and the user who completed it are recorded on the Integration card.
- If you close the window without saving, a confirmation dialog opens.

4. Manage Integrations

①Integration info card

Field	Meaning
`Integrated at`	When `Complete` was last clicked
`Integrated by`	Who last clicked `Complete`
`latest logged at`	Time of the most recent integrated turn
`total turns`	Total number of integrated turns
`Ingestion Settings`	Integration Type, Region, Log Path

Integrated at is when setup was finished. latest logged at is when logs actually came in.
You can add a version description of up to 255 characters in the description field.

②Switch to a new Integration

If the log format or storage location changes, create a new version with Switch to new integration.

The settings window opens prefilled with the previous Region, Log Path, and parser code.
Access Key and Secret Key are left empty for security. Enter them again.
When you finish, the version goes up by 1. The previous settings move to Integration History.
If there's no Gen & Judge model, the button is turned off.

③Disconnect an Integration

To disconnect, click Disconnect at the top right of the Integration card. The button sits between Integration History and Switch to new integration.

Clicking Disconnect opens a confirmation window. Nothing is disconnected yet at this point.
Click Disconnect in the confirmation window to disconnect. The buttons can't be clicked while the request is in progress.
When it's done, the Integration disconnected. notice appears and the Integration card changes to There are no integrations.

After disconnecting	What happens
Log collection	No new logs are collected through this Integration.
Reconnecting	A disconnected Integration can't be turned back on. Create a new one with `+ New Integration`.
Past results	Evaluation results from before the disconnect are kept.
Settings	The sampling probability and Metric settings stay as they are.

You can disconnect regardless of the log collection status.
If disconnecting fails, an error notice appears and the confirmation window stays open. Click Disconnect again.

4-1Check the details
1. Check the version and integration details on the Integration card.
2. Check the Region and Log Path in Ingestion Settings.
See Integration info card for what each field means.
4-2Add a description
1. Enter a description for this version in the description field and click Save.
4-3View history
1. Click Integration History.
2. Review settings from past versions, newest first.
4-4Create a new version
1. Click Switch to new integration.
2. Enter the Access Key and Secret Key again.
3. Finish by following the same order as steps 2–3.
The previous settings move to Integration History.
4-5Disconnect
1. Click Disconnect.
2. Click Disconnect in the confirmation window.
3. Check that the Integration card has changed to the empty state.
A disconnected Integration can't be turned back on. Create a new one with + New Integration.

Frequently asked questions

Why can't I click Test Connection?Because some fields are still empty.

Cause: The button turns on only after you enter the Region, Access Key, Secret Key, and Log Path.
Solution: Fill in the empty fields.

Why do I see Invalid access credentials?Because the keys are wrong or don't have access to the Region.

Cause: Even a valid key fails if it lacks access to the selected Region.
Solution: Check the key values and Region permissions with your infrastructure team.

Why do I see No logs found?Because the connection works, but there are no logs at the path.

Cause: The path may be empty, or it may be the wrong path.
Solution: Check the Log Path and the time range again.

Why can't I click Complete in the parser step?Because there are no rows that can be evaluated.

Cause: Rows with empty required values are excluded from evaluation. If every row is empty, you see There are no valid rows available for evaluation.
Solution: Fix the parser code with Edit and run it again.

Why can't I click Switch to new integration?Because there's no Gen & Judge model.

Cause: If no Gen & Judge models are registered, the button is turned off.
Solution: Register a model first in Go to Gen & Judge Model Setting.

What should I do if I'm connected but don't see any data?Check whether latest logged at updates. If it doesn't, check your log collection.

Cause: latest logged at is the time of the last integrated turn.
Solution: Check whether latest logged at updates. If it doesn't, check your log collection.