Four days, one feature, and the rails to do it again

In one screen

What happened. An audit of 233 pages produced two shortlists. Design boards went through 21 feedback rounds with Phil. A consolidated prototype became the pixel reference. Stage 1, a static header behind a flag, went live on day three. A PRD for the WordPress-managed menu converged over ten adversarial rounds, was approved from a plain-English pack, converted to 34 sized stories, and built by otto in one day. It was verified on staging and released to production in ten minutes with a rehearsed rollback.

Why the autonomous part worked. The design was frozen and byte-comparable before the PRD existed. An approach memo surfaced the server prerequisites. The PRD carried dated decisions, 18 invariants, and acceptance criteria that quote command output. A panel with a second model family converged it. Story one built the test harness. Verification and the release checklist were stories in the PRD, not afterthoughts. Blocks were resolved by fixing root causes, and every decision was written back into the PRD.

What it cost. About $980 in API spend by the evening of day three, 97 percent of it Fable running routine agents, plus a day of subscription usage for the build. On the first day only about 90 of 438 minutes were agent work; the rest was waiting for humans and relaying feedback by hand.

The proposal. A nine-stage workflow, Feature Rails, with seven human touchpoints and everything else run by agents, living in otto as an otto feature front door plus an otto-feature skill. It reuses what already exists (grill-me, design-shotgun, otto-plan, prd-task-sizer, the release gate) and adds the six things this project had to hand-build: recorded approvals, measured budgets, environment preflight, account rotation, a block-resolution command, and a release stage. otto itself needs a dozen small fixes first, and they make a good first PRD to run through otto.

One addition from outside. A post by Geoffrey Huntley on the morning of the release describes the artefact this project never produced: a high-level design document that the agent is forced to keep current after the build, self-hosted in the application, with the human's intent on top. A reply extends it to a document exported to the team at the end, derived from what the agent actually did, to lower cognitive debt. The rails adopt both, and the design gate becomes the event that starts the next stage.

The four days

Artefacts and gates

Seventeen artefacts were produced in a chain, and nine gates sat between them. The chain is worth reading because the rails below are mostly this chain made explicit, with the two missing records added.

Why otto succeeded

This is the answer to "how did we prepare otto". Eleven practices, each with the evidence that it mattered.

1. The design was decided, consolidated and frozen before the PRD. The prototype's build script asserted link counts and rebuilt byte for byte, so the PRD could demand parity with a file rather than a picture. Story after story reported "parity 0" against that bundle, and the final walker render is byte-identical to stage 1 in production.
2. An approach memo ran before any PRD. It surfaced the things that would have blocked a build mid-run: PHP 7.2 in production, the 1,000 form-field limit that would silently drop menu items, the Dante walker skipping WordPress's own hook, Cloudflare purge on save. Each became a prerequisite or a story.
3. Decisions were numbered, dated and baked in. Seven owner questions went out with recommended defaults; the answers came back by number in one message, were written to memory as "do not re-ask", and became requirements with acceptance criteria. The PRD's own decisions table, D1 to D24, records rejected alternatives.
4. Product invariants gave the executor rules it could not argue with. Eighteen of them: byte-identical legacy output, hidden items never render, every link returns 200 without redirect, SVGs sanitised and URL icons never fetched, one header per request. Stories cite them by id.
5. Every acceptance criterion was a runnable command with quoted output. The form was fixed: a command ending in echo EXIT=$?, the expected result, counts marked "quoted", a live probe for integration-critical stories, and a design-artefact bullet naming the bundle region and frames. The executor could not pass a story by asserting; it had to show.
6. The PRD converged through a panel, not self-review. Four rounds of a single model checking its own work stayed at one or two blockers. Round five used six lenses plus a second model family, deduplication, and three refuters voting on each finding; 49 of 55 findings stood and the revision that followed cleared the blockers. Declines were logged with reasons so the checker stopped re-raising them.
7. Sizing split the work into single-context stories. Twenty-three stories became 34 with letter suffixes, dependencies re-pointed, and human-owned or post-flip stories parked so otto would not run them under a default executor.
8. Story one built the feedback loop. otto detected no test loop for a PHP theme, so the first story created the runner, the lint, and the three-runtime harness, and proved the lint against a broken file. Every later story had something to fail against.
9. Verification and release were stories too. Baselines, screenshot comparison, link-state checks, Lighthouse, the launch gate script and the human release checklist with evidence slots were all in the PRD. The release on day four followed a checklist otto had written and rehearsed on staging.
10. Isolation kept the blast radius small. The build ran in a git worktree on its own branch; staging and production stayed untouched until the gate; the production deploy was proven crawl-neutral on five pages before the menu was activated.
11. Blocks were treated as information, not failures. Each block arrived with a measured cause, a discriminating experiment and lettered options. The operator checked the evidence, then chose between amending a budget with a measured number, moving a gate to where the tool could be fixed, adding a root-cause story, or fixing the environment. Every choice was written back into the PRD with a date, and the re-run took 8 minutes instead of 60 because the note said what still stood.
12. The model policy was explicit before the build started. Opus by default, Sonnet for checked mechanical work, Fable for the hardest problems, a second model family for independent review. The build ran entirely on Opus and cost nothing beyond subscription usage.

Where time and money went

Money. The session's API cost stood at $980.85 on the evening of day three, of which $949.47 was Fable. Almost all of it went on routine agents (auditors, board builders, verification lenses, PRD rounds) that the later model policy assigns to Opus or Sonnet. The day-four build ran on Opus through Max subscription accounts, so the most productive day was the cheapest.

Honest misses

Grouped by who could have prevented them. The agent-side ones are mine.

Feature Rails

The workflow below is the chain above made explicit, with the two missing records added, the hand-built pieces turned into fixtures, and the human touchpoints reduced to seven decisions a non-developer can make. Stages with a highlighted border need a person.

Needs a human decision at the gate. Everything else is agent-run, with escalation cards for the exceptions listed under Guardrails. Each approval is an event: the state file advances and the next stage starts without a further prompt. One artefact cuts across stages 3 to 9, the living design document described after the guardrails.

The nine stages

Phil's walkthrough

What the North Star looks like from the driver's seat. Seven touchpoints, none of which require reading a PRD or opening a terminal beyond the first line.

1. Start. Phil opens Claude Code in the project and types one line./otto-feature "Add a rate-alerts page under Personal, with an email signup"The agent asks eight to twelve questions, one at a time, each with a suggested answer he can accept with a word. It shows a one-screen brief and asks "is this what you mean?"
2. Audit. If the feature touches existing pages, a shortlist arrives on a private URL fifteen minutes later. He ticks the rows he agrees with and presses approve on the page.
3. Design. A board with four variants arrives. He rates each, writes comments, marks which notes apply site-wide, and submits from the page. Rounds stack on the same URL until he presses sign off. A consolidated prototype follows with launch and full modes.
4. Stress tests. He gets a short card: what was checked, what was fixed automatically, and anything that needs a decision (rare).
5. Review pack. A five-minute page in plain English with numbered questions and suggested answers. He answers by number and presses go. That is the point at which cost starts.
6. Build. Silence, apart from a digest and any escalation card. A card looks like: "Story 15 needs a decision. The performance target was set on a slower server than production. Options: accept as measured (recommended), re-measure, or fix the content shift. Reply with a letter."
7. Release. "Ready on staging, 81 of 81 checks, rollback rehearsed. Say go to release." He says go, and ten minutes later the finished-product card arrives with the live URL, the evidence, and the rollback button.

Guardrails

The rails refuse to do these without a named human, and record who said yes:

• Any change to production, including database edits and cache purges outside the release checklist.
• Taking over a shared environment that holds someone else's deploy.
• Waiving a non-functional target (performance, accessibility, size) rather than fixing the cause.
• Merging content branches that have not been signed off.
• Spending past an agreed budget, or escalating the model tier for routine work.
• Restarting a session while agents are running.

The document that comes after

On 25 September, a few hours before the release, Geoffrey Huntley posted a pattern for the moment after an intense agent-assisted build: "Before AI, we would start with a HLDD. After AI, I am increasingly finding that the HLDD comes afterwards." The mechanics: "pre-commit hooks and skills that force the agent to automatically update the HLDD, and you self-host the HLDD in the application as documentation. Top of fold is still the human crafted why, measurement of success and how it will be measured. Below the fold is the how," with the author's own intent as editor remarks on each subsection. One reply, the one Stevan picked out, goes a step further: a skill that exports a design document to the team at the end, derived from the HLDD and what the agent actually did, with diagrams, so the team understands what the change was and why. Its stated goal is to lower cognitive debt, Margaret-Anne Storey's term for the debt that "lives in the brains of developers and affects their lived experiences and abilities to go fast or to make changes".

The same day a public repository shipped the pattern end to end: one canonical architecture document with owner intent, observable success criteria, implemented components, data flows, trust boundaries and qualified acceptance gaps; a maintain-hldd skill discoverable by both Codex and Claude; a staged pre-commit check that refuses a source change without a substantive design-document change, where whitespace and dates alone do not count; the same check in CI on every commit; and a read-only reader inside the app, bundled at build time so it works offline.

Why this project needed it

The build left behind 35 commits, about a dozen PHP modules, more than a thousand automated checks, and a Tools page. The documents that exist are for the agent (a 280 KB PRD whose run copy is gitignored), for the release (the evidence file), or for other executors (the spec index, a module table nobody would read for pleasure). Nothing tells Phil's team in five pages how the menu system works, why it is shaped that way, and what to do when it misbehaves. That gap is cognitive debt, and it is the debt that grows fastest precisely when otto is productive: the more stories pass unattended, the less anyone on the team understands.

What the rails adopt

• A living HLDD from stage 3 onward. Top of fold, human-written and owner-edited: the why, what success looks like and how it is measured, lifted from the brief and the decisions log. Below the fold, agent-maintained: the approved design and its reasons, implemented components, data flows, seams and invariants, trust boundaries (the sanitiser, capability gates, what is never fetched), operations (the Tools page, flush and purge, rollback), and qualified gaps (parked stories, waivers, follow-ups). Editor remarks are allowed anywhere. Diagrams are Mermaid so they render in the repository and in the app.
• The design gate writes the first section. When the teammate approves a design, the approval is not only a URL in approved.json; the chosen variant, the rounds it took and the reasons become the HLDD's design section. The same approval event advances the state file and starts the stress tests and the approach memo. Design is a gate that triggers work, as Stevan read it, and it also seeds the document.
• A guard in the loop. otto's commit verification already checks that a story committed something; it gains the rule that a story touching source paths must also make a substantive change to the HLDD sections it affects, or the story does not pass. The executor prompt gets a maintain-hldd convention alongside the existing spec-index one.
• Self-hosted where the operators are. For a WordPress theme that means a read-only Tools page, "About this navigation", rendering the bundled Markdown next to the seed and activate buttons that already carry explanatory copy. For an application it is a documentation route.
• An export at stage 9. otto hldd export derives the team document from the HLDD, the progress log, the operator decisions and the evidence: what changed, why, the diagrams, how to operate it, how to roll it back. That is the finished-product card grown up, and it is what Phil's team reads instead of the PRD.

For Navigation 2026 now. The seed material exists: the PRD's overview and decisions, the spec index, the Tools page copy, the release evidence and the three verdicts. A one-story job produces docs/HLDD-navigation.md with three diagrams (render seam and fragment filter, cache generation and invalidation, the menu-item field model) and an in-admin reader, and it is a good candidate for the first run of the rails on this codebase.

Putting it in otto

otto today is a 2,201-line bash loop with a planner, an adversarial checker, an executor prompt, a stall watchdog and a status file, plus three skills: an index, otto-plan and otto-orchestrate. Its smoke tests pass 44 of 44. The rails need four kinds of change, in this order of value.

A front door

An otto feature <slug> command that owns a resumable state file, otto/features/<slug>/feature.json, with the current stage, artefact paths and hashes, and approvals. The conversational stages (intake, design feedback, review pack) live in an otto-feature skill because bash cannot ask questions; the mechanical stages call the existing commands. otto-orchestrate already sketches this shape and becomes the skill's later half.

Fields the JSON is missing

Commands the run needed and improvised

• otto approve <gate> writes an approval with the artefact hash.
• otto preflight <env> checks the environment manifest and captures baselines on that host.
• otto resolve <story> waive|amend|add-story|env backs up the PRD, records the decision, resets block counters and relaunches.
• otto release <env> runs the release steps and fills the evidence slots.
• otto hldd check|export enforces the design-document guard and exports the team document at the end.
• An account pool: tokens, usage read per iteration, thresholds, and a switch that stops, checkpoints, re-pins and relaunches. Two scripts in the CTW repo did this by hand and were never exercised.
• OTTO_NOTIFY_CMD for a push to a phone, because a terminal notice waited 87 minutes.

Fixes to the loop itself

What to reuse rather than build

• grill-me for intake: one question at a time with a recommended answer. Add the artefact it lacks, decisions.md.
• design-shotgun for boards. Its binaries are not installed on the devbox, so the boards were hand-built; either install it or adopt the hand-built board generator, which already has stars, comments and copy-JSON, and add direct submission plus the scope field.
• otto-plan and prd-task-sizer as they are, with the panel workflow from day three as the check loop.
• autoplan and the plan-review lenses as panel members; they already score design, engineering and developer experience.
• The release gate and checklist written by NAV-014 and NAV-019, generalised: baselines keyed by host, a consent-cookie option for hosts running Cookiebot, and a flush-then-fresh-hit purge proof instead of an unmeasured TTL.
• ship, land-and-deploy and canary do not fit CTW's deploy script with its typed confirmation; the release block needs a per-project deploy adapter, which the checklist already describes.

Building it with otto

Improving otto with otto is feasible and is the right first test of the rails, because the repo is small, private, and has a fake-executor smoke suite that can serve as the feedback loop. The shape of the PRD:

How to run it safely. otto would be editing its own running script, so the run uses a pinned copy through OTTO_HOME against a worktree of the repo. The smoke suite plus shellcheck is declared as the loop, every story adds one fake scenario, and the skills are installed only after merge because installed skills go live immediately. Expect the checker's rubric to produce noise on a bash repo; the panel should be told the domain. Model routing per the policy: Opus for the loop and skill code, Fable for the teammate-facing copy and the skill prose, Sonnet for mechanical scenarios, gpt-6-astra as the checker.

Templates to lift verbatim

These already exist in the docs folder and the worktree, and they are the fixtures the rails should ship with. Paths are under docs/nav-audit-2026-09-22/ unless noted.

How this was produced

Eight Opus agents worked in parallel from a condensed reader over the 7,770-record session transcript (31.8 MB, 22 to 25 September), the docs folder, the build worktree, and otto's source at commit 4bab194. Six covered the transcript by phase, one mapped the artefact chain, one mapped otto's internals and re-ran its smoke tests. Their reports, about 13,700 words with line references, are saved in the docs folder next to this page's source (retro-sources/) and were synthesised by the session's own model. Where two reports counted differently, the artefact-chain count was used. Every figure above traces to a transcript line, a file, or a command output in those reports. The design-document section draws on Geoffrey Huntley's post of 25 September 2026 (read through a tweet mirror; the replies are behind X's login wall, so only the reply Stevan quoted is used), the same-day pull request in vivary-dev/Vivary-New that implements the pattern, and Margaret-Anne Storey's definition of cognitive debt as quoted by Simon Willison in February 2026.