skill
DEPRECATED: Adom Bridge Control (moved into Adom Bridge)
Public Made by Adomby adom
Drive the user's real desktop from the cloud: their Chrome/Edge, headless pup, native UI, the Hydrogen editor, sibling Claude tabs, plus screen recording and video post.
← Commit history
Publish 1.1.0
5 files changed
+92−60
README.md+3−2@@ -48,8 +48,9 @@ adompkg install adom/adom-desktop-control - **hydrogen-web-control** - what's reachable in the Hydrogen editor and why its embedded Claude input is not. - **claude-control** - when to delegate to a sibling Claude vs do it yourself.-- **screen-recording** - `desktop_record` + `desktop_caption`, with the foreground-rebump,- recorder-reset, and caption-hygiene rules that make it actually work.+- **screen-recording** - WGC single-window capture (`desktop_record_window_start`):+ background, no picker, no banner, occlusion-safe, the right way to record a window+ (whole-desktop capture and its foreground theft is the wrong default). - **video-post-production** - crop, speed up, brand-font captions, Ken Burns on stills, TTS voiceover, concat, and wiki-ready keyframes.
SKILL.md+8−4@@ -28,7 +28,7 @@ pack of tested patterns, each one distilled from a real session, each labeling | A native app window (Fusion, dialogs, non-web UI) | `desktop_*` (UIA + SendInput) | [desktop-ui-control](skills/desktop-ui-control/SKILL.md) | | The Hydrogen web editor / its Claude panel | (mostly can't, read why first) | [hydrogen-web-control](skills/hydrogen-web-control/SKILL.md) | | A sibling Claude tab to run a task | depends on where it runs | [claude-control](skills/claude-control/SKILL.md) |-| Capture the screen to a video | `desktop_record_*` + `desktop_caption` | [screen-recording](skills/screen-recording/SKILL.md) |+| Capture a window to a video (the right way) | `desktop_record_window_start` (WGC, background) | [screen-recording](skills/screen-recording/SKILL.md) | | Turn raw clips into a finished demo | ffmpeg + `video-post` | [video-post-production](skills/video-post-production/SKILL.md) | ## The cross-cutting rules (true on every surface)@@ -44,11 +44,15 @@ pack of tested patterns, each one distilled from a real session, each labeling - **Upload files via DataTransfer, not the OS file picker.** Inject the bytes into the page and set the `<input type=file>`, works in pup and nbrowser. The OS picker is a last resort (needs foreground SendInput). See native-browser-control / pup-browser-control.-- **Recording captures whatever is topmost.** If you drive via background CDP while- recording, the recording shows the WRONG window unless you re-foreground the target- before every visible step (the "foreground re-bump"). See screen-recording.+- **To record, capture the WINDOW (WGC), not the whole desktop.** `desktop_record_window_start`+ records one window in the background even when it's occluded, so there is no foreground+ theft. Whole-desktop capture (`desktop_record_start`) records whatever is topmost, so a+ self-updating window (e.g. the Hydrogen panel) keeps stealing the take. See screen-recording. - **Verify by screenshot, don't assume.** After any click/type into a webview, screenshot and confirm the text/state actually landed, OS input silently no-ops on many webview editors.+- **Search the verb surface before you assume a feature is missing.** `adom-desktop help+ <namespace>` / `<verb>`. Several capabilities (WGC window recording, nbrowser on Edge,+ nbrowser recording) were missed by assuming they didn't exist. Look first. ## Install
package.json+1−1@@ -4,7 +4,7 @@ "type": "skill", "title": "Adom Desktop Control", "brief": "Drive the user's real desktop from the cloud: their Chrome/Edge, headless pup, native UI, the Hydrogen editor, sibling Claude tabs, plus screen recording and video post.",- "version": "1.0.1",+ "version": "1.1.0", "description": "A skill pack of hard-won, tested patterns for driving an Adom user's actual machine from the cloud AI through Adom Desktop: their real logged-in Chrome/Edge via the browser extension (CDP), headless pup browsers, native UI via UIA and SendInput, the Hydrogen web editor, sibling Claude tabs, and a full screen-recording + video post-production pipeline. Every skill is distilled from real sessions and labels what works, what fails, and why.", "org": "adom", "dependencies": {},
page.json+1−1@@ -4,7 +4,7 @@ "type": "skill", "title": "Adom Desktop Control", "brief": "Drive the user's real desktop from the cloud: their Chrome/Edge, headless pup, native UI, the Hydrogen editor, sibling Claude tabs, plus screen recording and video post.",- "version": "1.0.1",+ "version": "1.1.0", "description": "A skill pack of hard-won, tested patterns for driving an Adom user's actual machine from the cloud AI through Adom Desktop: their real logged-in Chrome/Edge via the browser extension (CDP), headless pup browsers, native UI via UIA and SendInput, the Hydrogen web editor, sibling Claude tabs, and a full screen-recording + video post-production pipeline. Every skill is distilled from real sessions and labels what works, what fails, and why.", "org": "adom", "dependencies": {},
skills/screen-recording/SKILL.md+79−52@@ -1,69 +1,96 @@ --- name: screen-recording description: >- Record the user's desktop to a video clip via Adom Desktop, with large on-screen- captions, and the gotchas that make or break it (foreground re-bump, recorder- reset, pulling the file). Trigger words: record the screen, screen recording,- desktop_record, record a demo, capture the screen, on-screen captions,- desktop_caption, record the browser flow.+ Record the user's screen to a video clip via Adom Desktop. THE right tool is+ WGC single-window capture (background, no picker, no banner, occlusion-safe), not+ whole-desktop capture. Covers the full recording verb surface, on-screen captions,+ and the gotchas that make or break it. Trigger words: record the screen, screen+ recording, desktop_record, record a window, WGC, record a demo, capture the screen,+ on-screen captions, desktop_caption, record the browser flow, record in the background. --- -# Screen recording (desktop capture + captions)+# Screen recording -Capture the whole desktop to a `.webm` while you drive an app, with big AD-rendered-captions burned into the capture. Hard-won, every rule below comes from a failure.+> **Search the verb surface before you assume.** This skill exists because an agent+> assumed whole-desktop capture was the only option and that nbrowser had no recording,+> both wrong. Run `adom-desktop help record` (the `desktop_recording` namespace, 10+> verbs) and `adom-desktop help <verb>` FIRST. The right tool was hiding in plain sight. -## The verbs+## Use WGC single-window capture (the right tool, almost always)++`desktop_record_window_start` records ONE window via **Windows Graphics Capture (WGC) ++Media Foundation**. This is what you want for recording a browser flow or an app:++- **No picker, no capture banner** (border suppressed on Win11).+- **Background-capturable**: the target window can be **occluded or off-screen** and it+ still records. So you do NOT need to foreground anything, the foreground-theft problem+ (another window stealing focus) simply does not apply.+- Resolve the window by `hwnd` (from `desktop_list_windows` / `desktop_find_window`) or+ `titleContains`, same as `desktop_screenshot_window`. ```bash-# whole-desktop capture (getDisplayMedia, 30fps). recordingId is at the TOP LEVEL of the response.-adom-desktop desktop_record_start '{"confirmDesktopNotTabRecording":true,"reason":"<why>","fps":30,"audio":false,"monitor":"primary"}'-adom-desktop desktop_record_stop '{"recordingId":"rec-N"}' # -> filePath, durationMs, sizeKB (top level)-adom-desktop desktop_recorder_close '{}' # reset stale recorder state (see below)--# large always-on-top click-through caption, captured by the recording:-adom-desktop desktop_caption '{"text":"...","size":"large","position":"bottom","id":"d","persist":true}'-adom-desktop desktop_caption '{"action":"force-clear"}' # nuke ALL captions+# start (recordingId is nested under output.data; path is top-level)+adom-desktop desktop_record_window_start '{"hwnd":17830478,"fps":30,"codec":"h264"}'+# ... drive the page via nbrowser_eval / browser_eval (also background) ...+adom-desktop desktop_record_window_stop '{"recordingId":"rec-<hex>"}' # -> .mp4 path+# the .mp4 lands under %TEMP%\adom-desktop-recordings\ ; pull_file it (lands in /tmp here) ``` -For a single browser tab you can also `browser_record_*` (pup), but it's low-fps CDP-screencast, prefer desktop capture for quality. See-[pup-browser-control](../pup-browser-control/SKILL.md).--## The five rules that make it work--1. **Parse `recordingId` from the TOP LEVEL** of `desktop_record_start`'s response, NOT- from a nested `output` field. Getting this wrong = empty RID = no recording.-2. **Reset the recorder first.** It goes stale (accumulated clips) and silently returns- no recordingId / a 439-byte empty file. Call `desktop_recorder_close` + `sleep 2`- before `desktop_record_start`. Run it inline / via the background tool, NOT detached- with `&` (detached runs produced empty files).-3. **Foreground re-bump before EVERY visible step.** The capture shows whatever is- topmost. If you drive via background CDP, the recording shows the WRONG window unless- you `desktop_bring_to_front` the target before each click/scroll. This was THE fix for- "my recording captured the wrong app." Re-bump, then act.-4. **Force-clear captions before each recording.** `persist:true` captions survive across- recordings and leak a stale caption into the next clip. `desktop_caption {action:"force-clear"}`- first. (Or caption in post instead, see below.)-5. **Live captions drift on autonomous flows.** If the page advances faster than your- sleeps, the live caption lands on the wrong screen. For anything whose timing you don't- control tightly, record CLEAN (no live captions) and add them in post, see- [video-post-production](../video-post-production/SKILL.md).--## Pull the file to post-produce+Parse: `recordingId = output.data.recordingId`, `path = top-level .path`.++### The one rule: WGC captures the window's ACTIVE tab++WGC records the window's pixels, i.e. its currently-visible tab. So **the tab you drive+via CDP must be the active tab in the recorded window.** nbrowser has no "activate tab"+verb, so: open a fresh tab with `nbrowser_open_tab` (it becomes active), confirm with+`desktop_find_window` that the window's title is your target, and drive THAT tab. Then+WGC + background CDP eval = a clean recording with zero foregrounding. Output is full-res+window-only (no desktop, no taskbar to crop).++## Whole-desktop capture (only when you truly need the whole screen)++`desktop_record_start` records EVERY pixel via Chrome getDisplayMedia. Avoid it for+single-window jobs, it has three problems WGC doesn't:++- **Foreground theft.** It captures whatever is topmost. If you drive via background CDP,+ the recording shows the WRONG window unless you `desktop_bring_to_front` the target+ before EVERY visible step (the "foreground re-bump"). A window that updates on its own+ (e.g. the Hydrogen panel showing your conversation) keeps stealing the foreground and+ ruins the take.+- **Picker + banner + wake-lock.**+- **Privacy.** It records the user's other tabs/apps.+- Gotchas if you must use it: `recordingId` is **top-level** in the response (not under+ `output`); reset stale state with `desktop_recorder_close` + `sleep 2` first (it+ silently returns no id / a ~439-byte empty file otherwise); run it via the background+ tool, NOT detached with `&` (detached runs produced empty files).++## nbrowser_record (getDisplayMedia, has a picker, inferior to WGC)++`nbrowser_record_start` exists but uses **getDisplayMedia**, so it parks in+`state:"awaiting_pick"` with a **share picker up on the user's screen**. You cannot stop+an `awaiting_pick` record via the API (`nbrowser_record_stop` errors "not recording"),+and the picker is browser chrome (not page DOM, not UIA-reachable). To clear a stuck+picker: click its **Cancel** button via a foregrounded `desktop_click` at the mapped+coordinate, or navigate the tab (cancels the pending request). **Just use WGC instead.**++## On-screen captions ```bash-adom-desktop pull_file '{"filePaths":["C:/Users/<u>/AppData/Local/Adom Desktop/plugins/puppeteer/recordings/rec-desktop-<ts>.webm"]}'-# NOTE: it lands in /tmp on the CLI host (your container), NOT your dest arg. Find it there.+adom-desktop desktop_caption '{"text":"Every part matched","size":"large","position":"bottom","id":"d","persist":true}'+adom-desktop desktop_caption '{"action":"force-clear"}' # nuke ALL captions ``` -## Privacy--Whole-desktop capture records EVERYTHING on the primary monitor (the user's other tabs/-apps). Maximize the target app so it dominates, and crop the taskbar / private tab strip-in post. The user must have authorized the recording (e.g. "I'm at the gym, go").+- A large always-on-top click-through overlay, captured by the recording. BUT WGC records+ a single WINDOW; a desktop caption overlay is a SEPARATE window, so WGC of the target+ window will NOT include it. For WGC clips, **caption in post** (see+ [video-post-production](../video-post-production/SKILL.md)). Use `desktop_caption` for+ whole-desktop recordings.+- `force-clear` before any recording: `persist:true` captions survive across clips and+ leak a stale caption into the next one.+- Live captions drift on autonomous flows (the page advances faster than your sleeps).+ Record CLEAN, caption in post, where you control the exact timing. -## Reliability note+## Always verify -The recorder is flaky. Verify after stop (`durationMs`, `sizeKB`), pull, and extract a-frame to CONFIRM the right window was captured before you build on it. Don't assume.+The recorder can be flaky. After stop, pull the file and **extract a frame to CONFIRM the+right window/tab was captured** before you build on it. Don't assume.