Multi-span evals
person concept tool org talk claim — click a node to jump to its page; hover an arrow for the relation
A type of evals|evaluation within LLM observability platforms in which the assessment of a system's behavior requires data drawn from multiple components or spans, rather than a single isolated span.
Core concept
Dat Ngo (Arize) introduces multi-span evals as a distinct evaluation mode within an Observability|observability platform: an eval that "requires data across many different components" rather than operating on a single trace segment. (source)
Relationship to span evals and trajectory evals
Multi-span evals occupy a middle position in a taxonomy of evaluation granularity:
- Span evals operate on a single span — one discrete unit of execution — and contrast directly with multi-span evals, which aggregate across several such units. (source)
- Trajectory evals go further still: where multi-span evals draw from some components, trajectory evals look "over all of the spans in total," making them a broader, holistic assessment of an entire run. (source)
The practical implication is that some evaluation questions — for example, whether an agent's reasoning was consistent across retrieval, tool use, and generation steps — cannot be answered by inspecting any single span and require the cross-component aggregation that multi-span evals provide.
Points of disagreement
No alternative perspectives on multi-span evals appear in the available material; the concept is presented descriptively by Dat Ngo without recorded challenge or elaboration from other speakers.