Self-Improving Systems
person concept tool org talk claim — click a node to jump to its page; hover an arrow for the relation
Systems designed to autonomously detect problems, investigate root causes, and generate fixes without direct human intervention — closing the loop from signal to pull request. Jason Lopatecki of Arize frames the core question as: "how do I build systems that autonomously fix themselves." 3:03
Core Architecture
Online Evals are a foundational component, adding a continuous monitoring layer that feeds signals into the self-improvement loop. 4:57 The overall pipeline runs from observed signal through automated investigation to a proposed code change, with evals serving as the trigger mechanism that initiates downstream agent activity.
Role of Human Engineers
As agents handle initial investigation and fix generation, the engineer's function shifts fundamentally: "your job kind of moves from responder to reviewer." 15:30 Rather than triaging alerts and diagnosing problems manually, engineers evaluate and approve agent-produced outputs.
Scope and Limitations
Lopatecki distinguishes between what autonomous systems can handle independently versus what still requires human involvement. Small, bounded changes — the one-line fix — represent the ideal autonomous case, but "a lot of times the fixes are bigger… and the bigger it is the more likely a human's involved." 15:08 This scoping reality means self-improving systems function most effectively as a triage and quick-fix layer rather than a wholesale replacement for human engineering judgment.
Implementation Requirements
Lopatecki is explicit that naive integration is insufficient: "it's not just like point Claude Code at the data." 27:33 Effective self-improving agents require well-designed skills — structured retrieval functions that fetch observability data and organize it into files the agent can reason over. Raw data dumps passed directly to the model do not produce reliable results; the data pipeline into the agent is itself a first-class engineering concern.
Points of Disagreement
No competing speakers are represented in the available material. Lopatecki's own framing contains an internal tension: he advocates for autonomous self-improvement 3:03 while simultaneously emphasizing that larger fixes require human drivers 15:08 and that the engineering work to make agents effective is non-trivial 27:33 — qualifying the "autonomous" claim considerably in practice.