Working with coding agents: rules, skills and a sandbox

I work with Claude Code on robot software, on digging facts out of old code, and on this site, which an agent maintains from my notes, photos and clips. What I’ve learned is that a good model is not enough. The agent is only as reliable as the harness around it: the loop, the tools, the context and the checks it runs inside. That harness is engineering work like any other, and it matters as much as the model.

The harness is a choice, not a given

Model and harness are two separate choices, and the default pairing isn’t automatically the best one. I wrote about this on LinkedIn, on the HarnessTax study from UC Berkeley and Arena: 21 model and harness combinations, including Claude Code, Codex CLI and Pi, on SWE-bench Lite and Terminal-Bench 2.0.

• In 9 of 12 comparisons, the highest success rate came from a harness other than the model provider’s own.
• GPT-5.6 Sol scored 83.3% on Terminal-Bench in Pi against 78.9% in Codex, at roughly half the cost.
• The sample is small, 30 tasks with 3 runs each, and most of the accuracy gaps aren’t statistically significant. The cost gaps are the more reliable signal.

The takeaway I keep: measure cost per solved task, not accuracy alone, and treat the harness as something to engineer and compare, not something that comes with the model.

A rulebook in every repo

Every repo gets an AGENTS.md: what the project is, a map of where things live, the commands, and the rules. It’s the briefing I’d give a new colleague, written once instead of repeated in every chat.

On top of that, one global file holds how I want any agent to work, in any repo:

• Think before coding. State assumptions, and ask when something is unclear instead of picking silently.
• Simplicity first. The minimum code that solves the problem. No features, abstractions or configurability nobody asked for.
• Surgical changes. Every changed line traces back to the request. Don’t “improve” the code next door.
• Goal-driven. Turn the task into a check that can pass or fail, then loop until it passes.
• Never commit unless asked. Never force push. Never commit secrets.

They’re written down because “use good judgement” isn’t an instruction an agent can check itself against.

Skills for work I repeat

A skill is a procedure in Markdown that the agent loads when the task matches: the steps, the formats, the traps. Three I use:

• Ingest. I dump raw material (notes, photos, clips, links) and the skill routes each piece to the right place on this site in the house style. Media goes through one pipeline that re-encodes and strips location data; raw files never reach the public folder.
• Publish. Check, summarise what will go live, and commit only after I say so.
• Repo summarizer. Turns a pile of repos into fact sheets: timeline, architecture diagrams drawn from the real code, my verified contributions, results and cited papers. It fans out one sub-agent per repo in parallel and assembles the result.

Facts need provenance

An agent writing about my work will happily round a number up or credit me with someone else’s commit. So the repo summarizer has hard rules:

• Credit only verified commits. Match on my surname, not my first name. A first-name match once credited a colleague’s commits to me; now the rule says so explicitly.
• Mark inferences as inferences. “Verified in code” and “my reading” are kept apart in every section.
• Never invent numbers, authors or URLs. No source, no link. A number read off a plot is marked approximate.
• Scrub secrets. A pattern scan for keys, tokens, passwords and internal addresses, checked by hand, before anything leaves the machine.

Every number on this site comes from those fact sheets. When a field needs a value the material doesn’t have, the agent writes TODO: and asks.

Schemas as guardrails

The site’s content types have strict schemas: a project summary is at most 70 characters, a key fact at most 10, a caption at most 60. When the agent’s text is too long, the build fails, and the rule is to shorten the text, never to loosen the schema. One check, npm run check, verifies media and builds the whole site, and nothing is committed until it passes.

The same idea works in robot code: give the agent a test that fails, not a description of what “good” means.

A sandbox with separate capabilities

The agent runs in a container, and each capability is granted on its own:

• Files: only the project folders I mount.
• Docker: off by default. When a task really needs it, I start the session with host Docker on purpose.
• SSH: through the forwarded agent. Private keys are never mounted.
• Models: local LLMs come from the host’s Ollama; the container never runs a second one.

Network access doesn’t imply file access, and file access doesn’t imply Docker. Keeping them separate is what lets me give an agent real work without giving it the machine.

What still goes wrong

• It fills gaps. Without a rule against it, a missing fact becomes a plausible one. The TODO: rule and the provenance rules exist because of this.
• It over-builds. Ask for a fix, get a framework. The simplicity rule and small, reviewable diffs keep that in check.
• It needs a reviewer. I read every diff before it’s committed.

What I’m building next

Today the loop still closes through me: the agent works, the checks run, I review. What I’m building is a closed-loop agentic workflow with a cost budget: the agent plans, acts, runs the checks that decide pass or fail, and iterates on its own until the task passes or the budget is spent. The budget is part of the design, not an afterthought, so the measure is the same as in the study: cost per solved task, with the harness and the model chosen to keep it low.