Closed general

End-of-run readout should lead with build + decisions, not agent interactions

Drew Owens · 2mo ago ·closed by Drew Owens

The problem

The final readout a three-musketeers run produces is dominated by agent-interaction mechanics — round 1 found N, round 2 fixed them, the adversary re-probed, the verifier passed the flows. That is the process, and it is interesting once. But the person reading the readout is deciding whether to ship a thing, and the process tells them almost nothing about the thing. Real example, a Session Monitor build with token-minting + encrypted backups: the readout led with "Round 1: 25 findings, 2 Critical → Round 2: all 25 dead → Round 3: 1 new Critical" and buried what actually mattered — that denying your last owner would flip the service anonymous, and that a fix had silently reverted an earlier fix while both suites stayed green.

What the end-of-run readout should lead with instead

Reorder it so the top of the readout answers the three questions a human actually has:

  1. What was built — the concrete deliverable as it exists now. Files/endpoints/CLI added, what a user can now do, versions, and the independently-verified test state (not the builder's self-report — the counts an outside run reproduces). One paragraph, not a per-round timeline.

  2. What decisions the human must make — the open calls the loop cannot make on its own. "This is auth code; the last round's fixes have not themselves been reviewed — one more pass, or ship?" "Backups are opt-in via backup install; wire them into the installer or leave opt-in?" These are the reason the human is reading at all. They are currently scattered or missing.

  3. Key decisions made DURING the build, driven by adversary/review findings — the non-obvious calls the loop made under pressure, each tied to the finding that forced it. "Split 'is auth configured?' from 'which tokens resolve' because the denylist fix made the token map emptiable at runtime and three call sites read empty-as-open." "Made link expiry mandatory after a permanent fleet-access URL was found to survive in Chat cards." This is the design rationale a future maintainer needs, and it is exactly what the current round-by-round framing scatters across five sections.

The round-by-round detail (finding tables, who-found-what, convergence) is still valuable as an appendix below the fold, for anyone auditing the process. Just stop leading with it.

Why it matters

The readout is the hand-off artifact. Led by process, it reads as "the agents were busy." Led by build + decisions + rationale, it reads as "here is what you have, here is what is yours to decide, and here is why it is shaped the way it is" — which is the only version that helps someone actually ship or reject the work. It also front-loads the dangerous findings (the fail-open, the silent revert) instead of averaging them into a finding count, so a skimming reader still sees the thing that would hurt them.

Concretely

Add a required readout template to the skill, roughly:

## What was built
<deliverable, versions, independently-verified test state>

## Decisions for you
<the open calls only a human can make, with the recommendation and the cost of each>

## Key decisions made during the build
<non-obvious design calls, each -> the finding that forced it>

---
## Process appendix (per-round findings, convergence, who-found-what)

Reported from a real multi-round run (Session Monitor 0.13.0).

1 Reply

Drew Owens · 2mo ago

Fixed in 0.2.0. Phase 6 now carries a required readout template that leads with (1) What was built, including the independently-verified test state rather than the builder's self-report, (2) Decisions for you, with a recommendation and the cost of each option and dangerous findings named first, and (3) Key decisions made during the build, each tied to the finding or wrong plan assumption that forced it. The per-round tables, who-found-what, and convergence story moved to a Process appendix below the fold. Phase 7 (present-to-user) mirrors the same order, and a guideline now states the rule outright: the readout is the hand-off artifact, so it leads with build + decisions, not agent interactions. (Drew, via Claude)

Log in to reply.