Non-blocking video evidence: skip invalid clips now; propose durable background worker

Josh Aberdeen · 4d ago

Proposal: Make AI Flow video evidence durable and non-blocking

Target: adom/adom-aiflow (Adom Wiki, MIT).
Reviewed baseline: 0.2.57, commit 083ebd94b24153d7463f70c1910f111415e63fc7.
Affected path: crates/adom-aiflow/src/main.rs, shared clip_artifacts_with().

Defect

AI Flow can correctly flag a recording as blank or truncated, yet still synchronously run FFmpeg to produce the 10× and action cuts. This happens inside the CLI operation that invokes capture recovery or stage work, and so a long/invalid recording can block routing, DRC, placement or other engineering work even when the actual engineering computation is finished. Historical source inspection confirms synchronous conversion predates the blank detector and remains present in the reviewed release.

Part A: small tested correction (patch attached)

When a clip's validation verdict indicates blank, truncated or otherwise invalid footage:

Retain the original recording and capture metadata.
Retain the contact sheet, suspect flag and truthful append-only ledger entries.
Emit a machine-readable clip-encode-skipped event including the reason.
Return before invoking expensive FFmpeg conversions.
Keep existing behavior for valid changing footage.

The attached unified patch is against the immutable 0.2.57 baseline and includes three synthetic media regressions (blank, truncated, healthy motion). It is generic to the shared helper, not a board- or stage-specific workaround.

Evidence: Baseline's two existing media tests passed. Patched candidate: five targeted media tests passed; 401 CLI tests passed, zero failed, five ignored; entire Rust workspace: 644 passed, zero failed, 11 ignored across 24 test groups. Forward/reverse patch --dry-run -p1 passed.

Patch SHA-256: 77d1db2cd80b3d27e296448e0040b70c38c69e2f3cff2f71bb269f7f8ad0dfd4.

Part B: permanent generic architecture (design review requested)

Keep all media processing off the engineering critical path, including valid-video conversion:

Record each raw capture as an immutable, hash-bound receipt. Put a versioned, deterministic job ID into a durable run-local outbox.
A separate low-priority, bounded media worker validates inputs, skips invalid recordings, encodes only valid footage into temporary files and atomically publishes verified outputs.
Use atomic claims, fencing/lease recovery, idempotent retries, job-sidecar results and a single-writer reconciler into run.jsonl and run.json. Media workers must never race the engineering ledger.
No automatic network publication; local-only by default. Preserve original recordings, contact sheets, screenshots and failure explanations.
Keep routing, DRC, placement, pours, current and thermal gates independent of optional encoding. recut supports reprocessing without rerunning engineering. compose/deliver explicitly enforce any required evidence completeness.
Expose separate engineering, queue, media processing, failure and retry times on the timeline.

Required failure-injection tests: blank/truncated/missing-FFmpeg, healthy derivatives, multi-hour recordings, kill worker mid-encode/publish, full disk, concurrent workers, duplicate stop/recut, stale source hashes, crash/restart recovery, legacy-run compatibility, bounded resource usage and a blocked FFmpeg that does not delay any engineering stage.

Scope honesty

Part A is implemented and tested in the attached patch. Part B is an architectural proposal, not implemented code. Part A alone does not remove synchronous encoding for healthy recordings.

Questions for maintainers

Prefer an adom-aiflow evidence-worker subcommand or a separate first-party binary?
Preferred run-ledger single-writer/locking boundary for durable result reconciliation?
Should optional media errors remain warnings until explicit compose/deliver evidence-policy gates?

No customer data, proprietary designs, project snapshots, private device addresses or credentials are included.

AI-FLOW-v0.2.57-blank-evidence-guard.patch

2 Replies

Josh Aberdeen · 4d ago

Related but independently tracked: Stage dependency invalidation, stale timeline stages and repeat owner-review stops after engineering gate failures are documented in https://wiki.adom.inc/adom/adom-aiflow/issues/171 . The two defects should remain separately tested and reviewed; both contribute to misleading stage duration.

John Lauer · 1d ago

Hi Josh,

Thank you for this one, and for shipping Part A as a tested patch with a clear line between what was implemented and what was proposed. That made it easy to review.

What was wrong. You were right. clip_artifacts_with() flagged a blank or truncated clip, then made the 10x cut and the action cut anyway, in the same command that ran the engineering step. Those cuts are a full decode and re-encode. On a long take that held routing, DRC or a tour for minutes, for footage the flag had already kept out of the video.

What changed in adom-aiflow 0.2.58.

Part A, essentially your patch: once a clip is flagged blank or truncated, it keeps its raw file, its contact sheet, its suspect flag and its ledger lines. The ledger gets a clip-encode-skipped event with the reason, and the function returns before any ffmpeg cut. A later recut whose check passes clears the flag and makes the cuts, as before. Your three regressions (blank, truncated, healthy motion) are in the test suite, slightly reshaped around a shared fixture helper.
A first piece of Part B: big clips are left out of the engineering path entirely. If a clip is over AIFLOW_CLIP_DEFER_MB (default 1024 MB; 0 turns this off), a step, tour, finish or deliver does not check it, sheet it or cut it. The capture is marked mediaDeferred, the ledger gets clip-media-deferred, and a NOTE names recut. recut handles deferred clips, and so does compose, which processes them before it builds the video. That way nothing that needs the cuts silently goes without them.
The blank check now decodes the clip once, where it used to seek eight times. On a multi-GB file that is the difference between one pass and up to eight (this came out of #180 and #181).
One small related change: when a step has too little disk for its clip, it now records the step without the clip and prints a NOTE. Before, it stopped the command with an ERROR. A clip problem is never a refusal.

How it was verified. There are unit tests for your three cases, for the deferral rule (size, the recut/compose override, and 0 to disable), and for the one-pass sampler on a real ffmpeg fixture. The full cargo test run and all the workflow suites pass.

What is still open. Healthy clips under the deferral size are still cut synchronously when they stop. A typical step clip is small, so that costs a few seconds, but the durable background worker from your Part B (outbox, leases, a single-writer reconciler into run.jsonl/run.json) is not built. On your questions: we would lean towards an adom-aiflow subcommand over a separate binary, so the worker always matches the rules of the run's own binary. We also agree that media problems should stay notes all the way through. AI Flow never refuses, so even compose and deliver would report missing evidence instead of gating on it. I am leaving the worker design open for now, rather than half-building it.

Upgrade: tell your AI "give me the latest adom-aiflow" (or adom-wiki pkg install adom/adom-aiflow); adom-aiflow --version says 0.2.58.

Colby Knox marked this in progress · 1d ago

Colby Knox removed the in progress marker · 1d ago