Stage dependency invalidation and owner-review loops after a failed engineering gate

Josh Aberdeen · 4d ago

Workflow regression: failed engineering gates leave a later stage active and repeat owner-review stops

Component: ADOM AI Flow stage sequencing / continuation hook / finish guard.
Affected versions: observed with a locally pinned 0.2.x build; verify against current upstream before fixing.
Scope: generic stage transitions for all boards and all supported EDA lanes.

Reproduction pattern

Complete routing, pours and an outline-related step.
A new spec rule or native analysis invalidates the previously accepted routing or placement evidence.
The route/gate reports failure, with a concrete measured violation that requires a placement or rerouting change.
The current stage continues to display the later outline step rather than the earliest stage invalidated by the failure.
An owner-review stop artifact is written, but the finish/continuation hook does not receive that stop-evidence path; it keeps asking for the work to proceed.
The agent repeats the same request for the owner to decide between an engineering relayout and a rule-budget change, even when the agent already owns normal layout engineering.

Observed incorrect behavior

The timeline accumulates time under the wrong stage, the stage is shown as current despite failed prerequisites, and the agent can settle into an idle review loop. The route may have zero DRC/unconnected errors yet still fail a valid higher-level electrical rule, so the acceptance reason must be preserved rather than discarded.

Expected generic behavior

Record upstream dependency invalidation as a structured, immutable event bound to board/spec hashes and the exact checker result.
Deterministically return to the earliest affected stage (usually placement for placement changes; routing for routing-only defects). Mark dependent later stages needs revalidation, not passed; retain historical evidence and time attribution.
Distinguish real owner-only decisions (changes to frozen electrical budgets, purchases, privacy permissions, externally required approvals) from routine engineering authority explicitly delegated to the agent (placement, safe rerouting, copper changes within frozen limits).
The agent should choose a technically valid engineering relayout autonomously, with preflight/dry-run, native DRC and rollback on failure; never silently relax a frozen length/current/thermal/safety requirement.
The completion hook should read persisted stop evidence automatically (or pass its path explicitly), distinguish genuine blocked conditions from recoverable engineering work and avoid repeated prompts on the same unchanged blocker fingerprint.
Separate stage elapsed time into active design, tool execution, blocked owner hold, media processing and unobserved idle; no fictional progress.
Resume from a saved native board authority and maintain idempotent crash/restart handling.

Acceptance tests

Stage N failure invalidates a prerequisite at Stage K: pointer returns to K, dependent stages are marked revalidation, and ledger preserves N's history.
Check failure caused by legal placement change: agent is allowed to relayout and revalidate, without human signoff, when engineering authority was previously delegated.
Frozen rule violation: no auto-amend or downgrade, and gate remains failed until board passes.
Owner-only decisions: bounded, evidence-backed stop is honoured exactly once across future hook invocations; no spam loop.
Completion hook receives/locates stop evidence after restart.
No rewriting of historic stage timing; accurate attribution of each operation and state.
Multi-board/multi-run and 25-stage generality; no project names, net names, component positions or design-specific hardcoding.

This is a separate workflow-control issue from the media-encoding defect. They share a user-visible symptom (a long-running stage) but require distinct patches and independent regression tests. No proprietary design details, board files, credentials, or unpublished circuitry are included.

Related issue: https://wiki.adom.inc/adom/adom-aiflow/issues/170 (separate media pipeline bottleneck).

1 Reply

John Lauer · 1d ago

Thanks Josh. This is a careful write-up, and keeping it apart from the media issue (#170) was the right call. I checked each point against how steps, returns and visits work today. Some of it was already in place, two parts were real gaps that are fixed in adom-aiflow 0.2.58, and the rest is below as proposed next steps.

How it works today (already in place before this release)

Dependent stages are marked for revalidation, and history is kept. step <k> --back --why "..." is a return. Every later step that was already visited shows as redo (not done) on the timeline and in finish until it is visited again in order. Visits are append-only: each one keeps its start time, its why, and the board and spec hashes at its start and end, so earlier visits and their time are never rewritten.
Unchanged returns cost nothing. Since 0.2.56, a return that leaves both the board and the spec unchanged does not send later steps back to redo.
Frozen requirements are never relaxed silently. Spec amendments are recorded (spec_freeze). The only way to accept a failed check is deviate "<item>" --why "...". A deviation is bound to the exact board and spec hashes, expires on any change, and is listed at finish.
Time is already split. Sessions separate active time from unobserved gaps. The step table splits wall, think and tool minutes (tool time measured through exec). A hands-on wait for the human counts as a pause.

Why the step pointer does not move on its own. The current step is the step the AI says it is working on. Moving it automatically would charge time and clips to a step nobody is working in. What was missing was the pointer telling the AI where to go, so I added that.

Fixed in adom-aiflow 0.2.58

The earliest affected step is named. When finish has open items that belong to a step the run is already past, it prints Suggested: earliest step these open items go back to: <step> (<item>) with the exact step <step> --back --why "..." command. Examples: a switch-node or placement-intent finding goes to placement; DRC or unconnected goes to routing; heatsink vias, pours, outline, the analyses and silkscreen go to their own steps. finish-guard repeats the suggestion until that step is visited again. The text also states the authority split: layout work within the spec (placement, rerouting, copper) is the AI's to do; only a change to a frozen budget or requirement goes to the human.
The completion hook finds stop evidence on its own. A stop that done --stop-evidence <json> accepts is now kept in run.json (stopEvidence). Every later finish-guard call, including a host stop hook that passes no path, re-reads and re-checks that file and gives the same stop answer (stopEvidencePath is in its JSON), instead of asking again. That covers your acceptance tests 4 and 5. The next prompt clears it, and a missing or invalid file fails closed to continue.

As you asked, both are suggestions and never gates. AI Flow is a suggestion engine and never refuses, so for your acceptance test 3 a failed check stays an open item on finish's report card until the board meets it or a reviewed deviation accepts it. Nothing is relaxed automatically.

How it was verified. Unit tests cover the item-to-step mapping and the earliest-return choice. A new workflow suite (tests/stage-return-workflow.py) checks four things: finish at outline names the earliest return; finish-guard repeats it and drops it once that step is revisited; an accepted pause holds over repeated guard calls with no path; and a new prompt (or a deleted evidence file) sends the guard back to continue. The existing continuation, hands-on and deviation suites pass unchanged.

Proposed next (not in this release)

A structured invalidated ledger event written when a check fails after its step was accepted. It would record the checker, its result, the board and spec hashes, and the step it invalidates, so the timeline can show why a step went back to redo, not only that it did.
Separate time lines for media processing and owner hold. Today media work counts as tool time and an owner hold as a pause.
A blocker fingerprint in finish-guard (open items plus hashes), so a host can see "same blocker, nothing changed" without its own progress hash.
Crash and restart resume from the saved native board, beyond what the run's append-only ledger gives today.

If any of these matter more for Circuit Forge than the others, tell me which and I'll do it first.

Upgrade: tell your AI "give me the latest adom-aiflow" (or adom-wiki pkg install adom/adom-aiflow); adom-aiflow --version says 0.2.58.