Traces and spans
person concept tool org talk claim — click a node to jump to its page; hover an arrow for the relation
Traces and spans are the fundamental telemetry units used to capture and structure observability data for AI systems, produced according to the OpenTelemetry standard and forming the backbone of LLM Observability pipelines.
Role in observability
Dat Ngo (Arize) positions traces and spans as core infrastructure: it is telemetry—not application code or agent harnesses themselves—that audits agent behavior, making traces "a big fundamental part of observability." (5:31, 5:37) Concretely, instrumented systems create OpenTelemetry traces and spans that power structured views of agent execution. (5:23)
Jason Lopatecki (Arize) adds a practical framing: "traces are pretty nice logs as well," positioning them not as exotic infrastructure but as a natural evolution of familiar logging patterns. (6:12)
Traces as input to self-improving agents
A key architectural pattern described by Lopatecki is pulling production traces directly into a code repository as files, enabling coding agents (agent harnesses) to operate on them. Because coding harnesses are highly effective when working with file-based inputs, serializing traces to files becomes "the key unlock" for agents to diagnose real production failures and produce fixes. (11:36) Skills within these agent systems pull trace data down as temporary files into the repository to facilitate this. (11:36)
Layering evals on top of traces
Evals serve as a pre-processing layer over raw traces: online evaluations run against production traces attach structured failure signals directly to the trace record ("eval are kind of like data on the trace itself"), giving coding agents richer, pre-interpreted context rather than requiring them to reason from raw telemetry alone. (30:19)