LLM Traces
==========

The LLM Traces tab records every LLM call made during an agent's conversations. It is the primary tool for debugging unexpected agent behaviour, verifying prompt construction, and understanding the cost and latency profile of individual executions.

LLM Traces are available on all three agent types — Chatbot, Agentic Flow, and Voice Agent — and share the same interface across all three.

Access is permission-gated: ``canViewChatTraces``, ``canViewFlowTraces``, or ``canViewVoiceTraces`` depending on agent type.

.. image:: /images/cmd-llm-traces-list.png
   :alt: LLM Traces list
   :align: center
   :width: 100%

|

Traces List
-----------

The list shows one row per trace. A trace represents a single top-level LLM call or agent execution recorded by the observability backend (Langfuse).

Columns:

* **Item ID** — unique identifier for the trace.
* **Name** — the name of the trace, typically reflecting the step or component that generated it.
* **Input** — the input sent to the LLM, shown as a truncated JSON preview. Click the cell to expand the full value.
* **Output** — the LLM's response, shown as a truncated JSON preview.
* **Created At** — timestamp when the trace was recorded.

Filters:

* **Created on** — date range picker.
* **Tags** — filter by trace tags. Tag options are derived dynamically from the tags present in the current data set.

Search: by trace Name.

**Add to Dataset** action (permission-gated with ``canCreateDatasetItems`` plus the relevant per-type trace permission) appears in the row's action menu. This allows you to save a trace directly into a Dataset for use in evaluations. See Section 12 — Foundry Resources for details on Datasets and Evaluations.

Trace Detail View
-----------------

Clicking any row opens the full trace detail.

.. image:: /images/cmd-llm-trace-detail.png
   :alt: Trace detail — header card, observations timeline, observations table
   :align: center
   :width: 100%

|

**Header card**

The top of the detail view shows aggregate metrics for the entire trace:

* **Latency** — total elapsed time in seconds from trace start to completion.
* **Total Cost** — combined cost across all LLM calls within this trace.
* **Usage** — aggregate token counts displayed as ``input → output (∑ total)``. Hovering or clicking this value opens a detailed breakdown popover showing:

  * Input Usage total
  * Output Usage total
  * Total Usage (sum of input + output across all observations in the trace)

  The popover also shows the number of generations (individual LLM calls) aggregated into these totals.

**Trace Preview**

Below the header, the Trace Preview section renders the top-level input and output of the trace in a readable format — the prompt that initiated the trace and the final response returned.

**Observations Timeline**

If the trace contains multiple observations (individual LLM sub-calls, tool calls, or processing steps), an **Observations Timeline** is rendered as a visual timeline showing each observation in sequence with its relative start time and duration. This makes it straightforward to see which step took the most time.

**Observations Table**

Below the timeline, each observation is listed as a table row with its own input, output, and token usage details. This provides the full per-call breakdown needed to diagnose issues such as:

* A specific node sending an unexpectedly large prompt
* A tool call returning an error that the LLM then tried to recover from
* Token usage spiking on a particular step

Best Practices
--------------

* When an agent produces a wrong or unexpected answer, open the most recent trace for that conversation and inspect the Trace Preview first — the prompt content usually reveals whether the issue is in the system prompt, retrieved context, or user input handling.
* Use the Observations Table to isolate which component (which node in a flow, which LLM call in a chain) is responsible for high latency or unexpected token consumption.
* Use **Add to Dataset** to capture interesting traces — both failures and successful examples — to build up evaluation datasets for regression testing. A well-curated dataset makes it much easier to detect regressions after prompt or model changes.
* Filter by date range after making a prompt change to compare trace patterns before and after the change — look for differences in output length, token counts, and cost.