Loop Engineering

2026-06-15 · page · source ↗ · source summary

Addy Osmani's 2026 piece naming and taxonomizing a practice already spreading through the agent-coding community: designing systems that prompt agents, rather than prompting agents yourself. The frame is a maturity leap — from "I tell the agent what to do" to "I design the harness that tells the agent what to do, on a schedule, with guardrails, across worktrees and sub-agents, and I review what emerges."


Key Quotes

"Loop engineering is replacing yourself as the person who prompts the agent."

The one-sentence thesis. It's catchy and correct, though "replacing" overstates it — you're still designing what gets prompted, just at one layer of abstraction up.

"I don't prompt Claude anymore. I have loops running that prompt Claude" — Boris Cherny

The most provocative quote in the piece, and also the most misleading taken alone. Cherny does prompt Claude — he just does it through skills, hooks, and automations he designed. The loop didn't design itself. The labor shifted, it didn't vanish.

"The model that wrote the code is way too nice grading its own homework."

On why sub-agent verification matters. This is a concise statement of Guardrails and Feedback Loops's core principle: verification must be independent of generation. A second agent with different instructions (and sometimes a different model) is the minimum viable independence.

"Build the loop. But build it like someone who intends to stay the engineer, not just the person who presses go."

The closer — and the most important sentence in the piece. This is where Osmani separates his position from the "dark factory" maximalists. The loop is a tool for engineers who want to scale their judgment, not a replacement for judgment itself. This aligns with his other writing (Addy Osmani's Workflow) where "the human engineer remains the director of the show."


The Taxonomy

Osmani organizes loop infrastructure into five components plus state:

Automations

Scheduled triggers — /loop, /goal, cron, hooks — that initiate agent work without human prompting. The /goal primitive (run until a verifiable condition is met) is the shared concept across Claude Code and Codex. This is the entry point: instead of "I'll run that agent now," the system decides when to run.

Worktrees

Isolated git worktrees so parallel agents don't collide. Both platforms now support this natively. DeltaDB and ctx – Agentic Development Environment are pushing this further — but Osmani is describing the current state, not the horizon.

Skills

The SKILL.md format as codified project memory. Without skills, "an agent starts every session cold" and fills gaps with confident guesses. This is Claude Code Mastery's central claim as well: skills are compounding infrastructure. Osmani's contribution is positioning skills within the loop — they're not just for interactive sessions, they're what automations and sub-agents reference.

Plugins and Connectors

MCP-based bridges to external systems. The shift from "agent says here's the fix" to "loop opens the PR, links the ticket, pings the channel." This is the Agent Orchestration dimension — the loop isn't just generating code, it's participating in the engineering workflow.

Sub-agents

Writer/reviewer separation as the minimum viable independent verification. Different agent, different instructions, sometimes different model. This patterns shows up everywhere: OpenCodeReview's per-file concurrent subagents, The Advisor Strategy's advisor-executor pattern, MiMo Code's independent memory writer subagent. Osmani is naming what practitioners keep rediscovering.

State/Memory

"Memory has to be on disk" because the model forgets everything between runs. This is where Coding Agents Continuity Not Memory and napkin converge — the repo as the natural home for operational state. Osmani's treatment is the shallowest part of the piece (a markdown file or Linear board), but the placement is correct: state is the sixth component that makes the other five work across sessions.


Key Themes

#concept #agentic-loop #automation #subagents #verification


Critical Analysis

The term "loop engineering" is doing real work. It names something that practitioners were already doing but didn't have language for. Before this piece, you'd describe it with a paragraph: "I set up a cron job that runs Claude Code on a worktree, reads the CI failures, and files PRs." Now: "loop engineering." The compression matters — it lets the community discuss the practice as a thing, not a workflow description.

But the piece is more taxonomy than invention. Every component Osmani describes (automations, worktrees, skills, sub-agents, connectors) already existed. His contribution is naming the integration as a discipline and organizing it into a teachable frame. This is valuable — naming is infrastructure too — but it's worth being clear about what's novel (the frame) and what's not (the components).

The floor as well as the ceiling: Poor Man's Loop Engineering argues the whole taxonomy collapses to a two-ingredient minimum — an agent that can test its own work plus independent adversarial review by fresh, cross-model agents — and that everything else Osmani lists is optimization on top of those two. It's the same discipline stated as a floor rather than a ceiling, and it converges on Osmani's closer from the opposite direction: its closing warning ("the agents lack taste") is "stay the engineer" restated.

The Boris Cherny quote is doing too much rhetorical work. "I don't prompt Claude anymore" is the headline that'll get shared, but it's false in the way that matters. Someone designed the loops. Someone wrote the skills. Someone chose the verification criteria. The loop didn't emerge from the ether — it was engineered, and that engineering is prompting, just at a different level of abstraction. The risk is that "I don't prompt Claude anymore" becomes an aspiration divorced from the judgment that makes it work.

The three "problems that sharpen" are the best part. Verification debt, comprehension debt, and cognitive surrender. These are the actual costs of loop engineering, and Osmani doesn't soft-pedal them. "Designing the loop is the cure when you do it with judgement and the accelerant when you do it to avoid thinking" — this should be the pull quote, not the Boris Cherny line.

Osmani is straddling two audiences. The piece reads as both a practitioner's guide and a platform comparison (Claude Code vs. Codex). The platform comparison sections dilute the argument — the specific feature comparisons (Codex has an Automations tab, Claude Code has /loop) will date quickly, while the conceptual frame won't.

The missing component: feedback from production. The loop architecture describes generating and reviewing code, but says nothing about what happens after merge. Does the code work in production? Are users happy? Did the fix introduce regressions? This is where Harness Engineering and Guardrails and Feedback Loops go further — verification doesn't end at the PR. A loop that can't learn from production outcomes is a loop that can't improve. This connects to Compound Engineering's thesis: the fourth step (Compound) is what separates productive teams from fast ones. Shipping AI Agents to Production is that missing component spelled out as a five-bullet production feedback loop (OpenTelemetry traces → layered evaluators → production traces as golden data → pre-ship experiments → CI gating on evals), and its conclusion — that the compounding advantage left after models commoditize is context, not the model — is this taxonomy extended one step past merge.

The training-problem critique: Harness Engineering is not Enough argues that even perfect loop engineering can't solve what is fundamentally a model training issue: RL rewards test-passing correctness, not maintainability, and the reward signal for good architecture arrives months too late. This doesn't invalidate loop engineering—Horthy's own proposed workflow uses AI-assisted loops for planning—but it bounds it: loops can raise the floor, not change the ceiling set by what was reinforced during training.

Comparison with Don't Fear the Dark Factory: Matt Wynne's piece describes the same thing — simple loop + validation harness — but from the perspective of personal conversion. Osmani is writing the architecture manual; Wynne wrote the testimony. Both are correct; they serve different readers. Wynne's version is more honest about the emotional experience; Osmani's is more useful as a reference.

Comparison with From AI Studio to AI Forge: McCormick's five-plane stack is the industrial-grade version of what Osmani describes. Where Osmani says "skills + automations + sub-agents," McCormick says "five planes with human changing altitude as the control mechanism." Osmani's version is the one you'd actually implement this week; McCormick's is the one you'd architect toward.

The honest tension: Osmani's previous piece (Addy Osmani's Workflow) was all about the human staying in the loop — review every line, treat the agent like a junior dev. This piece describes removing the human from the prompt loop while keeping them in the review loop. That's a step toward Five Levels from Spicy Autocomplete to the Dark Software Factory Level 3-4, and it's in tension with his earlier advice. He doesn't acknowledge this tension directly, but the closer ("build it like someone who intends to stay the engineer") is trying to hold both positions.

Production-scale validation: AI Code Migration with Claude Code is Loop Engineering at 1M lines. Anthropic's Bun migration instantiates every component of Osmani's taxonomy: skills (the rulebook), sub-agents (implementation fan-out + adversarial review), automations (the mechanical work queue driven by compiler output), and state (the filesystem as the kanban board). The article's thesis — "fix the process that produced the code" — is Osmani's meta-skill stated as a migration principle rather than a development one.

A shipped implementation of the term: LongHorizon-Harness names itself "Loop Engineering" for computer-use agents — a fixed Manager/Executor/Auditor plan-act-verify loop with checkpoint/recovery that wraps Claude Code, Codex, OpenCode, and DeepSeek Harness. It's Osmani's open-ended meta-skill specialized to one domain and shipped as a pip tool, a useful concrete counterpoint to the taxonomy here.

The most complete shipped implementation is Compound Engineering Plugin: all five components — automations (/lfg), worktrees (ce-worktree), skills (33 SKILL.md), connectors (git/gh scripts), sub-agents (reviewer personas) — plus the one thing Osmani's taxonomy leaves thin: a return arrow. Its ce-compound step writes each cycle's learning to docs/solutions/ where the next plan reads it, making the loop's state/memory component self-replenishing rather than just persisted.

DSPy Flex extends loop engineering to the optimization dimension. Flex's thesis — "continually compile our harness, balancing what's written as code and what's handed off to an LM, as new data, models, and tactics arrive" — is Osmani's loop engineering applied to the program itself, not just the orchestration around it. Each time a new model ships or your dataset grows, recompile the module against your metric. The harness becomes a function of (data, model, metric) rather than a fixed design. The optimizer independently discovers the decomposition, routing, and method-selection patterns that Osmani describes as human engineering craft. See DSPy Flex — Let the Model Write the Code.

The research trajectory: Osmani's five-component taxonomy captures what practitioners are handcrafting today. Lilian Weng's Harness Engineering for Self-Improvement surveys the research that aims to automate this craft: ADAS and AFlow treat workflow design as a search problem over code, Darwin Gödel Machine evolves harness code directly, and Agentic Harness Engineering makes the observability problems Osmani flags (comprehension debt, verification debt) into first-class design constraints. The implication: loop engineering as a human practice may itself be transitional — the destination is loops that improve their own loops.

The logical endpoint: Prime Agent (RLM Harness) pushes loop engineering one step further — the agent doesn't just prompt other agents within a human-designed loop, it rewrites the harness itself from its own trajectory. Prime Agent's /refine reads what happened, proposes the smallest CRUD edit to prompts/skills/memory/sub-agents that would improve outcomes, and applies it. It's Loop Engineering with the engineer partially removed from the loop — exactly the step Osmani's closing line ("build it like someone who intends to stay the engineer") warns about. The Factorio reward-hacking case study (where the refinement loop pivoted from building skills to building cheats once an exploit was found) is a concrete illustration of why that warning matters.

A loop that attacks comprehension debt directly. The "Genesis Kit" harness in Building a Production AI PR Review Agent instantiates all five components, but its two distinctive moves both target the debts Osmani flags rather than throughput. First, a spec file the agent is forbidden to edit — locked because agents drift the goal to suit their own convenience — which makes the definition of done immune to the loop that's supposed to satisfy it. Second, a per-milestone verifier spawned with fresh context and given only the goal and invariants, never the builder's trail, on the reasoning that a model marking its own exam writes only the questions it already knows. Against verification debt that works: on camera the verifier catches a TRUNCATE path bypassing a row-level trigger that enforced an append-only invariant. Against comprehension debt the answer is a "quiz me" protocol that interrogates the human about design decisions before accepting a milestone — the most direct attack on cognitive surrender anyone has shipped, and one whose cost is that it turns the engineer back into a student on every iteration.

The taxonomy instantiated as a harness: Building Autonomous Goal Loops That Deliver is this frame made concrete, and it adds two pieces Osmani's five components leave thin. First, a floor vs. direction split — the deterministic floor (tests, validators, schemas, effect checks) preserves earned behavior while direction comes from driving a real request and inspecting the shortfall. Second, an authority model that treats the scorer itself as part of the threat model: Free/Propose/Frozen, where the loop may add a regression check but never weaken one. Its goals/ package (goal.md, facts.md, plan.md, LOOP.md, STATE.md, corpus/, rounds/) is a concrete answer to Osmani's shallowest component, state.


Sources: Loop Engineering Last updated: 2026-07-25