← Commit history

Initial commit: skill/adom-desktop-control

John Lauer ·c417ffea80 ·3mo ago
2 files changed +130
README.mdadded+61
@@ -0,0 +1,61 @@+# Adom Desktop Control++A skill pack of **tested patterns for driving an Adom user's real machine** from the+cloud AI through Adom Desktop: their logged-in Chrome/Edge, headless pup browsers,+native app windows, the Hydrogen editor, sibling Claude tabs, plus a full+screen-recording and video post-production pipeline.++Every skill here was distilled from a real session and labels **what works, what fails,+and why**, so the next agent doesn't relearn it the hard way.++```bash+adompkg install adom/adom-desktop-control+```++## Pick the right surface++| You want to drive... | Use | Skill |+|---|---|---|+| The user's REAL logged-in Chrome/Edge | `nbrowser_*` (extension, CDP) | **native-browser-control** |+| A throwaway / clean-to-record browser | `browser_*` (pup) | **pup-browser-control** |+| A native app window (Fusion, dialogs) | `desktop_*` (UIA + SendInput) | **desktop-ui-control** |+| The Hydrogen editor / its Claude panel | (mostly can't, read why) | **hydrogen-web-control** |+| A sibling Claude tab | depends where it runs | **claude-control** |+| Capture the screen to video | `desktop_record_*` + `desktop_caption` | **screen-recording** |+| Cut raw clips into a finished demo | ffmpeg + `video-post` | **video-post-production** |++## The rules that apply everywhere++- **Background by default, never steal focus.** CDP eval, UIA Invoke, and window+  screenshots all run on a background window. Only OS SendInput needs foreground, so+  avoid it when a CDP/UIA path exists.+- **Drive it yourself, including login.** You can take the user's real browser through an+  OAuth sign-in end to end. Never ask them to click or log in.+- **Upload via DataTransfer, not the OS file picker.** Inject the bytes, set the+  `<input type=file>`. Works in pup and nbrowser.+- **Recording captures whatever is topmost.** Re-foreground the target before every+  visible step or you record the wrong window.+- **Verify by screenshot.** OS input silently no-ops on many webview editors; confirm the+  text/state actually landed before building on it.++## What's inside++- **native-browser-control** — the user's real Chrome/Edge over CDP (incl. `browser:"edge"`,+  DataTransfer uploads, driving OAuth login, the cross-origin / no-frame-targeting limits).+- **pup-browser-control** — headless Puppeteer browsers for anonymous or clean-to-record work.+- **desktop-ui-control** — native windows via UIA (`find_control`/`ui_click`) and SendInput+  (`click`/`type`/`press_key`), with screenshot-to-coordinate mapping.+- **hydrogen-web-control** — what's reachable in the Hydrogen editor and why its embedded+  Claude input is not.+- **claude-control** — when to delegate to a sibling Claude vs do it yourself.+- **screen-recording** — `desktop_record` + `desktop_caption`, with the foreground-rebump,+  recorder-reset, and caption-hygiene rules that make it actually work.+- **video-post-production** — crop, speed up, brand-font captions, Ken Burns on stills,+  TTS voiceover, concat, and wiki-ready keyframes.++## Pairs with++[adom-desktop](https://wiki.adom.inc/adom/adom-desktop) (the app + relay),+[adom-browser-extension](https://wiki.adom.inc/adom/adom-browser-extension) (native CDP),+[adom-tts](https://wiki.adom.inc/adom/adom-tts) (voiceover),+[demo-recording](https://wiki.adom.inc/adom/demo-recording) (the broader demo workflow).
page.jsonadded+69
@@ -0,0 +1,69 @@+{+  "schema_version": 1,+  "slug": "adom-desktop-control",+  "type": "skill",+  "title": "Adom Desktop Control",+  "brief": "Drive the user's real desktop from the cloud: their Chrome/Edge, headless pup, native UI, the Hydrogen editor, sibling Claude tabs, plus screen recording and video post.",+  "version": "1.0.0",+  "description": "A skill pack of hard-won, tested patterns for driving an Adom user's actual machine from the cloud AI through Adom Desktop: their real logged-in Chrome/Edge via the browser extension (CDP), headless pup browsers, native UI via UIA and SendInput, the Hydrogen web editor, sibling Claude tabs, and a full screen-recording + video post-production pipeline. Every skill is distilled from real sessions and labels what works, what fails, and why.",+  "org": "adom",+  "dependencies": {},+  "tags": [+    "adom-desktop",+    "automation",+    "browser",+    "nbrowser",+    "pup",+    "uia",+    "recording",+    "video",+    "hydrogen",+    "claude"+  ],+  "discovery_triggers": [+    "drive the user's browser",+    "control chrome or edge",+    "native browser extension",+    "nbrowser",+    "drive a pup browser",+    "automate the desktop",+    "click a native window",+    "uia desktop control",+    "record the screen",+    "make a demo video",+    "screen recording",+    "desktop captions",+    "speed up a video",+    "drive the hydrogen editor",+    "type into the claude tab",+    "upload a file to a website via the browser",+    "drive an oauth login"+  ],+  "scripts": {+    "install": "./install.sh",+    "uninstall": "./uninstall.sh"+  },+  "files": [+    "SKILL.md",+    "README.md",+    "page.json",+    "install.sh",+    "uninstall.sh",+    "screenshots/hero.png",+    "skills/native-browser-control/SKILL.md",+    "skills/pup-browser-control/SKILL.md",+    "skills/desktop-ui-control/SKILL.md",+    "skills/hydrogen-web-control/SKILL.md",+    "skills/claude-control/SKILL.md",+    "skills/screen-recording/SKILL.md",+    "skills/video-post-production/SKILL.md"+  ],+  "hero": {+    "type": "image",+    "path": "screenshots/hero.png"+  },+  "author": {+    "name": "John Lauer",+    "email": "[email protected]"+  }+}\ No newline at end of file