Observability
person concept tool org talk claim — click a node to jump to its page; hover an arrow for the relation
Observability, in the context of AI agent engineering, is the discipline of answering "what's happening in the thing that I've built" — capturing the internal states, traces, and telemetry of non-deterministic systems that cannot be fully understood through code inspection alone. Both Dat Ngo and Jason Lopatecki of Arize position observability as a foundational requirement for building and maintaining reliable LLM-powered systems.
Core Function: Understanding Non-Deterministic Systems
Dat Ngo argues that observability is especially critical because AI systems are inherently non-deterministic: when you fix a perceived issue, "you might have actually produced like two or three regressions that you didn't really know about." This unpredictability makes passive inspection of source code insufficient — "code doesn't audit agents or harnesses, it's actually the telemetry that does that."
Components of Observability
Observability for LLM systems is composed of several layered primitives:
- Traces and spans: "traces is a big fundamental part of observability" — the structural backbone for capturing execution paths through agents and tool calls.
- OpenTelemetry: Ngo notes that "everything we do is through open telemetry", grounding LLM observability in the established open-source standard.
- Evals at multiple granularities:
- Span evals: "evals can be run on a single kind of component", evaluating individual steps.
- Multi-span evals: "we also have like multi-span evals", spanning sequences of operations.
- Trajectory evals: "if you want to look over all of the spans in total, that's something like a trajectory eval".
- Session-level evals: "there's that session level eval", capturing the full user interaction arc.
The Shift: From Human Dashboards to Agent-Consumed Telemetry
Jason Lopatecki articulates a structural transformation in what observability is for: "observability used to be for humans. Used to be a UI you click, a graph you click." In the agentic era, telemetry becomes the input signal that autonomous agents consume to detect and fix problems — "telemetry is like this smoke thrown off of your system that can allow these agents to go make fixes."
This shift has a practical implication for instrumentation volume. Lopatecki advocates for dramatically more aggressive logging: "you're going to trace 10 times more. You're going to log 10 times more" — because agents can process that volume to determine what code path was taken, whereas humans cannot.
Points of Disagreement
No direct disagreements between speakers appear in the available material. Both Ngo and Lopatecki are Arize advocates and present complementary rather than conflicting views: Ngo focuses on the structural components (traces, evals, OpenTelemetry) while Lopatecki focuses on the evolving role of observability data as fuel for self-improving agentic systems.