Closed PR #5 open feature request

Feature: Hydrogen Runtime Recovery with bounded stop-loop diagnosis and safe continuation

Josh Aberdeen · ·closed by Colby Knox

Feature proposal: Hydrogen Runtime Recovery, stop/loop detection and safe continuation

Circuit Forge proposes a new opt-in hydrogen-runtime-recovery skill for adom/hydrogen-bootstrap, reusing Hydrogen Doctor, Bridge triage, existing Claude Code background sessions and native AI Flow authority.

Observed engineering need

An AI Flow task can be unfinished while the bound Hydrogen agent becomes idle, or when a mandatory program is missing, an optional app is unavailable, a service is unhealthy, a dependency is misconfigured, an authentication session expires or identical tool failures repeat.

Normal waits and long CAD operations must not be misclassified as loops. The recovery engine must never create simultaneous board-writing agents, override electrical gates, or present a restarted agent as proof of engineering progress.

Proposed implementation

  • Deterministic progress/stall classifier and read-only Hydrogen Doctor checks.
  • Guarded executable recovery with native run/board hashes, exclusive local lock, prior-attempt budget and session/cwd identity checks.
  • Verified background-session ownership-transfer helper for Claude versions that create a new job ID on resume.
  • An opt-in dispatcher with dry-run default, cooldown and bounded recovery.
  • Source-agnostic stage handling: tested on synthetic 3,000 work-item cases, not tied to one PCB, stage or customer.
  • New skill packaging in the Hydrogen Bootstrap installer, including its executable tools/tests; no monitor or auto-repair gets enabled on install.
  • General installer fix: a clean installation with no legacy hd-* skills to purge must not exit non-zero under set -e.

Validation

116/116 Python tests pass, including stopped and busy-agent cases, genuine-vs-false loops, repeated observation deduplication, optional-vs- mandatory tool absence, broken services/auth, native revision fencing, non-destructive handoffs, and retry limits.

A fresh-home sandbox install confirms all eight companion Python tools and tests are delivered; all 116 tests also pass from the installed sandbox. Bootstrap package lint: zero errors. The candidate includes no private CAD designs, credentials, user workstation paths or customer telemetry.

Historical review: the ADOM history API returned exactly 200 revisions and no pagination interface. We compared all 199 adjacent revision pairs. The exact original repository commit cannot be asserted from this limited source history; the boundary is explicitly marked unknown in the attached audit. This is a requested new feature, not a claim of a historical code defect.

Safety and scope

POSIX locking/process collection is currently tested under Linux/WSL. Mac and non-WSL Windows execution require maintainer-native acceptance. The optional installer integration must not enable recovery automatically. Owner holds, interactive sign-ins, native PCB safety/DRC gates and public contribution approval remain mandatory. The issue and branch PR do not modify any running Hydrogen instance.

See attached sanitized evidence ZIP for the full accessible revision matrix, the exact proposed diff, test logs and stated limitations.

hydrogen-runtime-recovery-PUBLIC-EVIDENCE.zip

1 Reply

Josh Aberdeen mentioned this in pull request #4 ·
Colby Knox · ·edited

Thanks for the detailed write-up. We're not going to merge this into hydrogen-bootstrap: bootstrap installs onto every Hydrogen machine, and this is built around the AI Flow PCB run and can stop and resume Claude sessions, which we'd want people to opt into themselves.

Two places this fits well:

  • AI Flow itself. Most of what this watches for (an unfinished stage, a board that stopped changing, a gate that keeps failing) is AI Flow's territory. Issues and PRs on adom/adom-aiflow are welcome, and that's where a recovery step could reach everyone running it.
  • Your own flow. If you'd rather run it your way, clone AI Flow (adom-wiki repo clone adom/adom-aiflow), make your changes, and publish it as your own wiki package that anyone can install with adom-wiki pkg install. Proper forking on the wiki is coming soon.

On the installer change: that line sits inside an if block, so under set -e a clean install continues; we tested it and it doesn't stop the script.

Going idle partway through a run is a real problem, and we'll look at catching it in Hydrogen's AI Threads panel.

Colby Knox closed this ·
Drew Owens mentioned this in pull request #5 ·

Log in to reply.