Evaluating a Co-Worker

Evaluations let you grade a Co-Worker’s answers against expected results using a labelled dataset. Because a Co-Worker runs as an agentic flow, you evaluate it with an In-House DeepEval evaluation whose inference source is the agent itself.

1. In the sidebar, open Evaluations, then the In-House tab. (If you do not see it, ask your administrator.)

2. Click + and choose DeepEval.

3. Basic Information — enter a Name (lowercase letters, numbers, and hyphens; up to 50 characters) and optional Tags.

4. Metrics — tick one or more; at least one is required.

5. Inference Source — choose Agentic Flow Agent.

6. Select Agent — pick your Co-Worker. Items marked (Draft only) have no published version; toggle Use Draft Version to test the unpublished draft.

7. Evaluation Model — choose the judge model (Model Category, then Model Type) that grades the answers.

8. Dataset — select a dataset with an input column and an expected_output column (the ground truth). If you have none, create one first.

9. Click Save, open the evaluation, go to the Runs tab, and click Run Evaluation. Use Refresh to update the status.

10. Open the completed run to review the Overall Metrics card and the per-row table — Input, Expected Output, and each metric’s result with an explanation. Use Download to export the results.

Note

Evaluations, the full metric catalogue, and datasets are documented under Foundry → Evaluations and Foundry → Datasets. This page covers only the Co-Worker-specific path — an In-House DeepEval evaluation with the Agentic Flow Agent inference source pointed at your Co-Worker. To grade a model directly rather than an agent, use the Bedrock evaluation family instead.

Tip

To build an evaluation dataset from real traffic, open a trace on the Traces tab and use Add to Dataset.