Designing agentic loops

2026-05-14 · source · source ↗ · raw capture · wiki page

Simon Willison argues that a critical new skill for maximizing LLM-based coding tools is designing agentic loops — carefully crafting the tools and iterative processes that coding agents use to brute-force their way to solutions. He defines an LLM agent as something that "runs tools in a loop to achieve a goal."

The Joy and Danger of YOLO Mode

Coding agents like Claude Code and Codex CLI default to requiring human approval for each command, which "dramatically reduces their effectiveness." The alternative — YOLO mode (automatic approval) — is described as "so dangerous" but essential for productive results.

Willison quotes Solomon Hykes: "An AI agent is an LLM wrecking its environment in a loop."

Three key YOLO risks:

  1. Malicious shell commands deleting or corrupting data
  2. Exfiltration attacks stealing source code or secrets from environment variables
  3. Using the machine as a proxy to attack other targets

Mitigation strategies:

He cites Anthropic's own documentation on "Safe YOLO mode" recommending --dangerously-skip-permissions in an internet-restricted container, linking to their .devcontainer reference implementation with a firewall script that limits access to a list of trusted hosts.

Tool Selection for the Loop

Rather than leaning on MCP (Model Context Protocol), Willison prefers shell commands — coding agents are "really good at running shell commands." He creates an AGENTS.md file with example commands rather than extensive documentation. His example: providing one shot-scraper invocation as a template, which is "enough for the agent to guess how to swap out the URL and filename."

He notes that capable LLMs already know how to use tools like Playwright (Python) and ffmpeg, and since they operate in a loop, "they can usually recover from mistakes."

Tightly Scoped Credentials

Two recommendations: 1. Use test/staging environments where damage is contained 2. Set tight budget limits on any credential that can spend money

Case study: He created a dedicated Fly.io organization with a $5 budget and a scoped API key to let Claude Code experiment with Dockerfiles for optimizing cold-start times — a project where he first recognized "designing an agentic loop" as an important skill.

When to Use Agentic Loops

Best for problems with "clear success criteria" involving trial and error. He suggests the thought "ugh, I'm going to have to try a lot of variations here" is a strong signal to try this approach.

Examples given: - Debugging — failing tests investigated by agents that can run them - Performance optimization — benchmarking SQL queries, adding/dropping indexes in isolation - Upgrading dependencies — with a solid test suite, agents can handle breaking changes - Optimizing container sizes — iterating on Dockerfiles and base images

A common thread: automated tests massively amplify the value of coding agents. LLMs themselves are great for building those tests.

Freshness of the Field

Willison emphasizes this is new — Claude Code was released in February 2025. He frames naming the concept as a way to "help us have productive conversations about it," noting there's "so much more to figure out."

Links Referenced