Closed general

pup owns the desktop_record_* / desktop_recorder_* verbs; ab core must own whole-desktop recording (native WGC + loopback audio) and refuse desktop_ prefix claims from bridges

John Lauer · 2mo ago ·closed by John Lauer

John, 2026-09-05, after the recorder failed on arav-rog: "if pup has somehow taken over those desktop* verbs that's a disaster cuz that is not how its supposed to work". Filing on his ask.

What the code does today

  • pup-bridge 2.0.396 owns the whole-desktop recorder. src/bridge.json claims the verb prefixes desktop_recorder_ and desktop_record_ (next to pup_ and browser_) and lists desktop_recorder_open, desktop_record_start, desktop_record_stop, desktop_record_status, desktop_record_list, desktop_recorder_close as its verbs. src/server.js implements them ("DESKTOP recording: one global recorder"): the HUD is a Chrome window in a separate _recorder profile, capture is Chrome getDisplayMedia to .webm.
  • ab core owns only the native per-window recorder: record_window_start / record_window_stop / record_window_status (wgc_record.rs, Windows Graphics Capture + Media Foundation to MP4, no picker, background-capturable). Nothing in ab implements desktop_record_start.
  • ab's own docs say the opposite. skills/VERBS.md lists desktop_record_* and desktop_recorder_* under ab-core "Capture & recording"; skills/RECORDING.md calls it "Adom Bridge's consent-gated desktop recorder" and explicitly says tab recording is pup's feature, "not this skill". A reader (John included) comes away believing ab records the desktop and pup records tabs. The wire says pup does both.

Why it matters (measured tonight)

On arav-rog the puppeteer bridge cannot spawn (npm install failed for cache bridge, 496 attempts, in backoff: #142). Every desktop_record_* and desktop_recorder_* verb on that box therefore dies with pup (Request timed out), while ab-core verbs on the same box (list_windows, screenshots, record_window_*) answer fine. A whole-desktop recording with system audio is impossible there until Node, npm and Chrome for Testing all work. A desktop capability should not depend on a browser automation bridge being healthy.

Asks

  1. Move whole-desktop recording into ab core, native: Windows Graphics Capture on a monitor (GraphicsCaptureItem from an HMONITOR, the same stack wgc_record.rs already uses per HWND) plus WASAPI loopback for system audio, muxed to MP4 by Media Foundation. Same verb names and args as today (reason, confirmDesktopNotTabRecording, monitor, fps, audio, maxDurationMs), so callers do not change. No Chrome, no HUD process to spawn; the consent surface becomes ab's own overlay (like the demo panel), still always visible while a take runs.
  2. Stop pup from claiming desktop_* prefixes. ab should refuse a bridge manifest that claims a desktop_ prefix (that namespace is ab core's), and pup's bridge.json drops desktop_recorder_ / desktop_record_. If a transition period is needed, ab-core takes precedence when both claim a verb.
  3. Fix the docs now, independent of 1 and 2: VERBS.md and RECORDING.md must say who serves desktop_record_* today (pup) until ab-core does.

Related: #142 (pup never spawns on arav-rog), pup-bridge is the other half of this change.

4 Replies

John Lauer · 2mo ago

Confirmed from ab's code, and the history answers "stolen vs designed": it was DESIGNED this way, deliberately, and the design has aged badly.

What the wire does today (verified): the whole-desktop recorder - desktop_record_start/stop/status/list and desktop_recorder_open/close - is implemented in pup (HUD = a Chrome window in a recorder profile, capture = Chrome getDisplayMedia to .webm); pup's bridge.json claims the desktop_record/desktop_recorder_ prefixes and ab's CLI routes those verbs straight to the browser bridge. ab-core natively owns only desktop_record_window_* (wgc_record.rs: WGC + Media Foundation to MP4). Why: when the whole-desktop recorder was built, ab had no native monitor capture, and Chrome's screen-share gave a ready-made consent-gated capture WITH system audio for free - so ab named the verbs desktop_* (a desktop capability from the caller's view) and delegated the mechanism to pup. Native per-window capture came later and whole-monitor capture never followed it native. Your measurement on arav-rog is the exact cost: a desktop capability dying with a browser bridge.

Plan, in order: 3 (docs, DONE now, ships in the next pkg): VERBS.md and RECORDING.md say who serves desktop_record_* today (pup, via getDisplayMedia; only record_window_* is native) and point at record_window_* as the pup-down fallback. 2 (prefix ownership, shipping next release): ab refuses a bridge manifest that claims a desktop_ prefix - that namespace is ab-core's - with ab-core precedence on any conflict as the transition rule, so pup's current claim keeps working until native lands and then yields automatically. 1 (native whole-desktop recording in ab-core, the real work): WGC on a monitor (GraphicsCaptureItem from an HMONITOR - the same stack wgc_record.rs uses per HWND) + WASAPI loopback for system audio, muxed to MP4 by a two-stream Media Foundation sink writer; same verb names and args (reason, confirmDesktopNotTabRecording, monitor, fps, audio, maxDurationMs) so callers do not change; consent = ab's own always-visible overlay in the demo-panel family (no Chrome, no HUD process). Honest sizing: the video half is a small delta on existing code; the audio half (loopback capture + A/V sync in the sink writer) is the substantive piece - a solid few hours done carefully, queued as the next block. Once it lands, pup drops the prefixes (pup-bridge's half).

John Lauer · 2mo ago

Update after John's direction (2026-09-05): BOTH recorders stay, split by owner, instead of "same names, pup drops them".

Why the two coexist (this is the reason the original layering happened, re-derived): Chrome's getDisplayMedia recorder offers things a native Rust recorder does not - the OS share picker as the consent surface, system audio + mic mixing for free, hardware-accelerated VP9/AV1 MediaRecorder to WebM, tab-audio capture, cursor and surface-switching options, and it runs on macOS/Linux. ab-native (WGC + Media Foundation) offers what pup cannot - no browser/Node dependency, background capture of an occluded window, no modal picker (ab's own gate), H.264 MP4, and it survives a broken pup (arav-rog). Different tools; keep both.

The naming rule (John): everything pup implements is a pup_ verb; ab-core owns desktop_*.

  • pup: pup_screen_record_start/stop/status/list + pup_recorder_open/close (aliases first, old names kept until ab-native lands). Filed on pup-bridge.
  • ab-core: native desktop_record_start/stop/status/list (monitor WGC + WASAPI loopback -> MP4, same args) - ask 1, next block. When it ships, pup drops the desktop_ prefix claims and the old names route to ab automatically.

Shipped in ab 2.1.31 (insiders): ask 2 - a bridge manifest claiming a verb/prefix ab-core serves natively is refused at rescan (today desktop_record_window_); any other desktop_ claim is honored with a warning until ab-core serves it; resolve_verb and the direct API's infer_app give ab-core precedence, so pup's broad desktop_record_ prefix can no longer shadow desktop_record_window_* for ah/direct-API callers (it could before). Ask 3 - VERBS.md / RECORDING.md in the 2.1.31 pkg say who serves what today plus the split-namespace direction.

John Lauer · 2mo ago

pup-bridge's half is in 2.0.397 (insiders): the pup_describe catalog and every desktop_record* verb row now say "ab-core capability served by pup on its behalf until adom-bridge#143; moving out of pup", and pup's skills / README no longer present desktop recording as pup's feature (they point at adom-bridge-recording). The prefixes stay claimed until ab-core serves the verbs natively; with ab-core precedence as the transition rule, the drop is then a one-line manifest change on pup's side. Ping pup-bridge#93 when native lands and it ships the same day.

John Lauer · 2mo ago

Ask 1 shipped: ab 2.1.36 (insiders) has native whole-desktop recording.

  • desktop_record_start {monitor?: index|"primary"|device, fps?, codec?: h264|h265, audio?: true, maxDurationMs?: default 30 min, cap 2 h} -> WGC on the monitor (CreateForMonitor from the HMONITOR at the same index desktop_display_info reports) + WASAPI loopback of the system mix (f32/i16/i32 mix formats -> 16-bit stereo, silence padded to wall clock because loopback yields nothing while nothing plays) muxed as AAC 192 kbps into the same H.264 MP4; the encoder thread is the single writer, so audio and video share one timeline.
  • Consent surface = ab's own persistent caption at the top of the screen for the whole take ("Recording your screen and system audio (Adom Bridge) ..."), hidden on stop.
  • desktop_record_stop (id optional when one take runs), desktop_record_status (no id = every take; autoStopped:true when maxDurationMs hit), desktop_record_list (finished MP4s). Give hwnd/titleContains instead of monitor for single-window mode (record_window_* is the same thing by another name). desktop_recorder_open/close retired.
  • GPU (John's ask): the sink writer requests hardware transforms, and the response now reports encoder + hardwareEncoder, hardware detected from the live transform's MFT_ENUM_HARDWARE_URL_Attribute. Verified on AdomLapper: "NVIDIA H.264 Encoder MFT", hardwareEncoder:true, for both monitor and window takes. The per-frame GPU->CPU staging hop is the remaining cost; the zero-copy DXGI path is #149.
  • Verified live: 2560x1600@30 monitor take, 169 frames in 6 s, vide + soun tracks (avc1 + mp4a), moov finalized, 8.3 MB; caption visible in a screenshot during the take; window mode and list verified as well.

Ask 2 finished: desktop_record_ and desktop_recorder_ joined ab-core's owned prefix set, so pup's claim is refused at rescan. Per John today there are no aliases on pup's side - pup-bridge#93 now asks for the clean drop. Ask 3 (docs) updated again: RECORDING.md and VERBS.md describe the native recorder and point at pup_screen_record_* as the separate Chrome tool. Closing.

John Lauer closed this · 2mo ago

Log in to reply.