Evaluations let you grade a Coworker’s answers against expected results using a labelled dataset. Because a Coworker runs as an agentic flow, you evaluate it with an In-House DeepEval evaluation whose inference source is the agent itself.
- In the sidebar, open Evaluations, then the In-House tab. (If you do not see it, ask your administrator.)
- Click + and choose DeepEval.
- Basic Information — enter a Name (lowercase letters, numbers, and hyphens; up to 50 characters) and optional Tags.
- Metrics — tick one or more; at least one is required.
- Inference Source — choose Agentic Flow Agent.
- Select Agent — pick your Coworker. Items marked (Draft only) have no published version; toggle Use Draft Version to test the unpublished draft.
- Evaluation Model — choose the judge model (Model Category, then Model Type) that grades the answers.
- Dataset — select a dataset with an
inputcolumn and anexpected_outputcolumn (the ground truth). If you have none, create one first. - Click Save, open the evaluation, go to the Runs tab, and click Run Evaluation. Use Refresh to update the status.
- Open the completed run to review the Overall Metrics card and the per-row table — Input, Expected Output, and each metric’s result with an explanation. Use Download to export the results.
Note
Evaluations, the full metric catalogue, and datasets are documented under Foundry → Evaluations and Foundry → Datasets. This page covers only the Coworker-specific path — an In-House DeepEval evaluation with the Agentic Flow Agent inference source pointed at your Coworker. To grade a model directly rather than an agent, use the Bedrock evaluation family instead.
Tip
To build an evaluation dataset from real traffic, open a trace on the Traces tab and use Add to Dataset.