Harness Engineering: How to Build Software When Humans Steer, Agents Execute — Ryan Lopopolo, OpenAI

Ryan Lopopolo, Member of Technical Staff at OpenAI, presents the concept of harness engineering: a discipline for structuring codebases, processes, and tooling so that coding agents can do the full job of software engineering with minimal human intervention. Drawing on nine months of building software exclusively with agents at OpenAI, Lopopolo argues that implementation is no longer the scarce resource — human time and model context window are — and that engineers must shift toward systems thinking, delegation, and codifying non-functional requirements so agents can act on them autonomously. The talk is followed by a Q&A with Vibhu Sapra of Latent Space.

Key takeaways

Implementation is no longer scarce: "code is free" to produce, refactor, and delete; the human role shifts to defining, prioritizing, and unblocking work 2:06
The three scarce resources in the agent era are human time, human/model attention, and model context window 5:10
Every human code-review comment that repeats itself is evidence of a context failure in the harness; fix it systematically, not ad hoc 38:13
Lint errors, test failures, and reviewer agents are all just prompts — mechanisms for injecting context into the agent at the right moment 15:34
Lopopolo advocates against overloading agent instructions upfront; instructions should be surfaced just-in-time as the agent encounters the relevant code 25:04
Code is a disposable build artifact; the LLM is a "fuzzy compiler" and the harness provides the constraints and optimization passes 41:51
Every time a human must type "continue" to the agent is a harness failure 33:31

The Shift in Engineering Economics

Lopopolo opens with the claim that coding agents have made implementation abundant: each engineer now has access to "five, 50, or 5,000 engineers worth of capacity 24/7" constrained only by GPU capacity and token budgets 2:36. This eliminates the classic P3-never-gets-done problem — with infinite parallel execution, all P3s can be kicked off simultaneously 5:54.

He argues the "AGI pill" is simply believing models are capable of producing every line of code ever needed, including deciding when to refactor or delete it 4:17. Large-scale migrations, historically blocked for months, can now be completed by firing off 15 agents concurrently 8:09. The consequence: human engineers should reorient toward systems thinking, system design, and delegation 3:02.

Codifying Non-Functional Requirements

A single patch may require 500 implicit decisions about non-functional requirements (error handling, timeouts, type safety, component decomposition), and models have seen every possible choice during training 8:52. Lopopolo's stance: it is the engineer's job to write these requirements down explicitly so the agent has a clear definition of "an acceptable merged patch" 9:17.

Practical techniques he uses:

Custom ESLint rules wired into every package in the PNPM workspace to catch patterns like missing retries/timeouts on network calls 12:59
Structural tests on the source code itself (e.g., enforcing files ≤ 350 lines) to keep the codebase context-efficient 14:01
Actionable lint/test error messages that include remediation steps rather than bare failure notices, so the agent can self-heal 14:25
Reviewer agents triggered on every push, primed with persona-specific documentation (front-end architect, reliability engineer, etc.), that surface P2+ issues as PR comments the implementation agent must address 12:03

Lopopolo notes that all of these mechanisms — AGENTS.md files, rules files, skills, lint messages, reviewer agents — are ultimately just prompts injected at different points in the workflow 15:34.

The Harness Architecture

Lopopolo's team uses OpenAI Codex as the primary entry point, not a custom shell. Rather than spawning Codex inside a pre-configured environment, they write skills that teach Codex how to launch the app, spin up the local observability stack, and attach Chrome DevTools Protocol via a local CLI daemon 20:32. This outside-in approach means all local dev tooling is designed for agent invocation first.

Key architectural choices:

750 PNPM packages in a monorepo, isolated by business-logic domain or stack layer, so the agent can scope most changes to a single directory subtree 34:33
5–10 skills (not hundreds), deliberately kept narrow and improved iteratively rather than broadened — infrastructure changes frequently, and skills hide that complexity from the agent 22:50
Structural/wholesome tests (separate from lints) that assert package privacy, dependency edges, de-duplicated Zod schemas, and single canonical async helpers — targeting the agent's observed tendency to optimize for local package coherence over shared utilities 21:46

Lopopolo endorses depending on first-party harnesses (Claude Code, Codex) because the labs post-train their models in the context of those harnesses (e.g., the apply_patch tool, bash quoting semantics are in the post-training loop), providing leverage not available in custom wrappers 26:43.

Process: "Garbage Collection Day" and Closing the Feedback Loop

With engineers producing 3–5 PRs per day on a three-person team, merge conflicts became the primary bottleneck, caused by long PR lifetimes caused by human code review 37:05. Lopopolo's solution:

Dedicate Fridays ("garbage collection day") to categorically eliminating every class of slop observed during the week rather than fixing individual instances 37:49
Map repeated review comments to missing documentation; add that documentation to the repository in a single canonical location 38:34
Wire that documentation into reviewer agents and/or failing tests so the agent receives it as automated prompt injection and self-heals on the next push 38:19

This is described as the core loop of harness engineering: human review comment → context failure diagnosis → documentation → automated prompt injection → reduced human review load 38:27.

On code review philosophy: Lopopolo warns against making every review comment mandatory, as it creates a failure mode where the coding agent is "bullied by all of the reviewers" — the system should bias toward acceptance, not perfection 29:55.

The Future: Autonomous Product Advancement

Lopopolo's stated goal is a system where he can provide a token budget, a quarter or year's roadmap, and success/reliability metrics, and have agents continuously advance the product without his hands on the wheel 43:37. He frames his remaining role as:

Defining work and success criteria
Triaging user feedback, pages, and production issues
Writing runbooks and acceptance criteria that agents can act on

He describes software engineering maturity as a "pentagonal personality chart" — as one capability area (e.g., implementation) matures, new weak spots (e.g., automated QA smoke testing of built artifacts before distribution) emerge and require the same harness-building investment 44:09.

On plan mode: Lopopolo recommends against approving agent plans without reading them line-by-line, as approving an unread plan encodes instructions you may not want followed. His preferred posture is to skip planning altogether and drop a well-specified ticket directly to the agent 40:49. Token usage splits roughly one-third each across planning/ticket curation, documentation/implementation, and CI-running agents 40:07.