← Commit history
v0.0.17: steer recording to WGC (desktop_record_window_start); gate getDisplayMedia fallback behind fallbackConfirmed; WGC-first skills
7 files changed
+78−18
bridge/BRIDGE_VERSION+1−1@@ -1 +1 @@-0.0.16+0.0.17
bridge/bridge.json+1−1@@ -2,7 +2,7 @@ "manifest_version": 1, "name": "native-browser", "displayName": "Adom Native Browser (Chrome + Edge)",- "version": "0.0.16",+ "version": "0.0.17", "description": "Drives the user's REAL signed-in Chrome/Edge via an installed extension (native-messaging transport). Mirrors the puppeteer bridge's verb surface as nbrowser_* but runs inside the actual logged-in profile, so cookies/SSO/saved-logins/captcha human-trust all work. The bridge binds an ephemeral loopback port (PORTS.md compliant) and writes a discovery file; the native-messaging host pumps stdio <-> this bridge; the extension never touches a TCP port.", "homepage": "https://wiki.adom.inc/adom/adom-browser-extension", "author": "Adom Inc.",
bridge/package.json+1−1@@ -1,6 +1,6 @@ { "name": "adom-native-browser-bridge",- "version": "0.0.16",+ "version": "0.0.17", "private": true, "description": "adom-desktop bridge for the Adom native-browser extension (Chrome + Edge)", "main": "server.js",
bridge/server.js+41−2@@ -245,7 +245,7 @@ const HELP = { '⭐ BACKGROUND IS THE WHOLE POINT — NEVER foreground the driven window to screenshot OR record. The user must keep working while a demo records behind their other windows; yanking Chrome forward makes the feature useless. nbrowser_screenshot (CDP) captures a background/occluded window unconditionally. WGC recording captures a background/occluded window TOO — but only when the recorded Chrome was launched with the occlusion-disable flags (see Recording); without them an occluded window paints blank. Foregrounding is the USER\'s choice, made by them when they want to watch — it is never something the AI should do on its own.', 'For demos: nbrowser_caption (in-page subtitle/alert) OR AD desktop_caption (fullscreen; use position:bottom — top covers the URL bar + Chrome warning).', 'Mouse verbs: pass cursor:true to show the gliding Adom cursor, and intent:"Clicking the Pay button" so the cursor LABEL narrates the action (builds user trust).',- 'Recording: method "wgc" (Windows, via adom-desktop 1.8.148+; no picker, no banner, mp4 on disk) is PREFERRED — it records in the BACKGROUND so the user keeps working. ⚠️ Background capture of an OCCLUDED Chrome window REQUIRES that Chrome be launched with `--disable-backgrounding-occluded-windows --disable-renderer-backgrounding --disable-background-timer-throttling --disable-features=CalculateNativeWinOcclusion` (pup\'s Chrome ships these; a default user Chrome does NOT — without them Chrome stops painting the occluded window and WGC records a blank surface). Stop with nbrowser_record_stop {method:"wgc",recordingId}. "getdisplaymedia" FORCES a share picker; "tabcapture" = the "Record this tab" toolbar button.',+ '⭐ Recording a browser window: the BEST tool is the CORE adom-desktop verb desktop_record_window_start (WGC) — or nbrowser_record_start {method:"wgc"}, which delegates to it. Background, up to 60fps, NO source picker, NO banner, captures OCCLUDED windows, never steals focus. This bridge\'s own getDisplayMedia/tabcapture recording is only a LAST-RESORT FALLBACK and is REFUSED unless you pass fallbackConfirmed:true, because a browser extension CANNOT silently screen-record (Chrome forces a manual share-source picker a human must click + a persistent banner, and won\'t reliably capture a background window). Prefer WGC every time; the extension is the fallback. ⚠️ For WGC background capture of an OCCLUDED Chrome window, Chrome must be launched with `--disable-backgrounding-occluded-windows --disable-renderer-backgrounding --disable-background-timer-throttling --disable-features=CalculateNativeWinOcclusion` (pup\'s Chrome ships these; a default user Chrome does NOT — without them Chrome stops painting the occluded window and WGC records a blank surface). Stop with nbrowser_record_stop {method:"wgc",recordingId}.', ], verbs: { 'open_window/open_tab/close_window/close_tab/switch_window/switch_tab/list_windows/list_tabs/rescan': 'Sessions = a real Chrome window (durable across reloads). tabId = real chrome tab id. close what you open.',@@ -287,6 +287,26 @@ function adCommand(app, command, args) { req.write(data); req.end(); }); }+// ⭐ The hint we hand back whenever an AI tries to record WITHOUT choosing WGC. The AI is not assumed+// to know the core adom-desktop window recorder exists, so we spell it out and explain — in detail —+// why the extension's own screen-capture is a heinous last resort. (Lesson from a real thread that+// blundered into the getDisplayMedia picker before discovering desktop_record_window_start on its own.)+const RECORD_WGC_HINT =+ '⭐ BEST PATH — record the window with WGC, NOT the extension. Use the CORE adom-desktop verb ' ++ 'desktop_record_window_start {"titleContains":"<window title>"} (or {"hwnd":<n>}) — or call THIS verb ' ++ 'with method:"wgc", which delegates straight to it. WGC (Windows Graphics Capture) records ONE specific ' ++ 'window in the BACKGROUND at HIGH frame rate (up to 60fps), with NO source picker, NO "sharing" banner, ' ++ 'captures the window even while it is OCCLUDED/behind other windows, and NEVER steals the user\'s focus. ' ++ 'Stop with desktop_record_window_stop (or nbrowser_record_stop {"method":"wgc","recordingId":...}); poll ' ++ 'desktop_record_window_status. This is the right tool for recording a browser window — reach for it first.\n' ++ '⛔ The extension\'s OWN recording (method:"getdisplaymedia"/"tabcapture") is a LAST-RESORT FALLBACK, used ' ++ 'ONLY if WGC is genuinely unavailable. By Chrome\'s security design a browser extension CANNOT silently ' ++ 'screen-record: getDisplayMedia FORCES a share-source PICKER that a HUMAN must click to choose the window/ ' ++ 'screen (yes — heinous: the whole point of a logged-in browser extension is that it should never need a ' ++ 'manual picker), it paints a persistent recording banner, and it does NOT reliably capture a background/ ' ++ 'occluded window. I am the fallback, not the default. If WGC truly will not work, re-call with ' ++ 'method:"getdisplaymedia" AND fallbackConfirmed:true to acknowledge the picker + banner tradeoff.';+ // AD's native per-window recorder shipped in adom-desktop 1.8.148 (standalone Rust WGC + Media // Foundation — no picker, no banner, background-capturable, real .mp4 under %TEMP%\adom-desktop- // recordings\, pullable with pull_file). We delegate method:"wgc" straight to it.@@ -423,7 +443,7 @@ function dispatch(verb, args) { d('nbrowser_fetch_url', "Fetch a URL using the profile's real cookies/session; optional save to desktop.", { url: 'required str', profile: P, desktopSaveTo: 'optional Windows path' }, { ok: 'bool', status: 'int', body: 'str?', savedTo: 'str?' }, 60, { longRunning: true, example: { url: 'https://example.com/api.json' } }), d('nbrowser_errors', 'Return buffered page/console errors for the tab.', { sessionId: S, tabId: 'optional str' }, { ok: 'bool', errors: '[...]' }, 20, { example: { sessionId: 'work' } }), d('nbrowser_close', 'Detach the debugger / close managed windows. No sessionId = all.', { sessionId: 'optional str' }, { ok: 'bool' }, 20, { example: {} }),- d('nbrowser_record_start', 'Start a background window recording (method:"wgc" = native WGC, no picker/banner, captures occluded).', { method: 'required "wgc"|"getdisplaymedia"|"tabcapture"', hwnd: 'int? (or titleContains)', titleContains: 'str?', fps: 'optional int (default 30)', codec: 'optional "h264"|"hevc"' }, { ok: 'bool', recordingId: 'str', path: 'str' }, 60, { statusVerb: 'nbrowser_record_status', longRunning: true, example: { method: 'wgc', titleContains: 'chip-fetcher' } }),+ d('nbrowser_record_start', '⭐ Record a browser window. STRONGLY prefer method:"wgc" (delegates to the core adom-desktop verb desktop_record_window_start: background, up to 60fps, NO picker, NO banner, captures occluded windows, no focus theft). method:"getdisplaymedia"/"tabcapture" is a LAST-RESORT fallback (Chrome forces a manual share picker + banner) and is REFUSED unless you also pass fallbackConfirmed:true. Calling with no method returns the WGC guidance hint.', { method: 'required "wgc" (recommended) | "getdisplaymedia" | "tabcapture"', hwnd: 'int? (or titleContains)', titleContains: 'str?', fps: 'optional int (default 30)', codec: 'optional "h264"|"hevc"', fallbackConfirmed: 'required true ONLY for getdisplaymedia/tabcapture, to acknowledge the picker/banner tradeoff' }, { ok: 'bool', recordingId: 'str', path: 'str' }, 60, { statusVerb: 'nbrowser_record_status', longRunning: true, example: { method: 'wgc', titleContains: 'chip-fetcher' } }), d('nbrowser_record_stop', 'Stop a recording and finalize the mp4.', { method: 'required "wgc"', recordingId: 'required str' }, { ok: 'bool', path: 'str', frames: 'int' }, 60, { example: { method: 'wgc', recordingId: 'rec-...' } }), d('nbrowser_record_status', 'Poll a recording.', { method: 'required "wgc"', recordingId: 'optional str' }, { ok: 'bool', status: 'str' }, 20, { example: { method: 'wgc' } }), d('nbrowser_record_list', 'List recent recordings.', {}, { ok: 'bool', recordings: '[...]' }, 20, { example: {} }),@@ -518,6 +538,25 @@ function dispatch(verb, args) { if (verb.startsWith('nbrowser_record_') && args && args.method === 'wgc') { return wgcRecord(verb, args).then(resolve); }+ // ⭐ Recording guidance gate. The AI is "dumb" about recording options, so STEER it: any record_start+ // that isn't WGC gets a great hint pointing at the core WGC recorder, and the extension's own+ // getDisplayMedia/tabCapture path is refused until the AI explicitly acknowledges it as a fallback+ // (fallbackConfirmed:true). Without this, a naive record_start silently lands on Chrome's share picker.+ if (verb === 'nbrowser_record_start' && (!args || args.method !== 'wgc')) {+ const m = args && args.method;+ const acknowledged = (m === 'getdisplaymedia' || m === 'tabcapture') && args && args.fallbackConfirmed === true;+ if (!acknowledged) {+ return resolve({+ success: false,+ error: m+ ? `method:"${m}" is the inferior fallback — WGC is strongly preferred for recording a browser window`+ : 'no recording method chosen — WGC is strongly recommended (this is a fallback recorder)',+ recommended: { method: 'wgc', coreVerb: 'desktop_record_window_start' },+ _hint: RECORD_WGC_HINT,+ });+ }+ // acknowledged fallback → fall through to the extension's getDisplayMedia/tabCapture path below.+ } // OS-level window raise/focus is owned by the CORE adom-desktop verb desktop_bring_to_front. The // extension deliberately does NOT implement a focus/raise-window verb — it must never steal the // user's focus. So we don't expose a phantom verb; but an AI coming from pup will guess
native-browser-bridge-manifest.json+5−5@@ -1,8 +1,8 @@ { "manifest_version": 1,- "version": "0.0.16",- "url": "native-browser-bridge-v0.0.16.bin",- "sha256": "befec031d289cfe3f5e3f8c081bf7ad4c174430d864a09d0cd65a1ef46de5fb2",- "size": 23008,- "released_at": "2026-06-27T17:18:55Z"+ "version": "0.0.17",+ "url": "native-browser-bridge-v0.0.17.bin",+ "sha256": "6fab19e58b2900cdea321af74346310743018edaad99a60e45d1f661f664f08e",+ "size": 24334,+ "released_at": "2026-06-27T19:47:28Z" }
skills/driving-the-extension/SKILL.md+1−1@@ -109,7 +109,7 @@ Mouser uses a **Username** field (not email) — check `input` by name, not just - **data (native advantage):** `fetch_url` (uses the user's REAL cookies — authed), `download` - **status/errors:** `errors`, `events` (window/tab closes incl. `byUser:true`) - **demo/UX:** `caption`, `focus`, **`flash`/`flash_stop`/`flash_mode`/`taskbar`**-- **recording:** `record_start`/`stop`/`status`/`list` — `method: wgc|getdisplaymedia|tabcapture`+- **recording:** `record_start`/`stop`/`status`/`list` — ⭐ use `method:"wgc"` (delegates to the core AD verb `desktop_record_window_start`: background, 60fps, no picker, no banner, captures occluded). `getdisplaymedia`/`tabcapture` are a refused-by-default fallback (Chrome forces a manual share picker) — need `fallbackConfirmed:true`. See **native-browser-recording**. ## Parallel multi-browser / multi-profile control Every **BROWSER × PROFILE is its own independently-addressable connection**, keyed `browser:email`
skills/native-browser-recording/SKILL.md+28−7@@ -1,6 +1,6 @@ --- name: native-browser-recording-description: How to record full-resolution, high-framerate demo videos of the user's REAL, logged-in Chrome/Edge through the Adom browser extension — in the BACKGROUND, so the user keeps working. Use this ONLY when a demo needs the user's real session/cookies/logins (a logged-in tool); when no login is needed, record via pup or Hydrogen (see the demo-recording skill) and never touch the user's real browser. Covers the paint-throttling problem, the occlusion/throttle-disable launch flags, the restart-native-Chrome-into-background-recording-mode flow (ask + warn + full process-tree kill + relaunch with profile + flags), the wgc/getdisplaymedia engines, and the offer-to-revert-for-battery step. Match on: record the native browser, record a logged-in demo, demo of the real browser, background recording of Chrome, nbrowser_record, full-framerate native recording. Part of the adom-browser-extension.+description: How to record full-resolution, high-framerate demo videos of a browser window in the BACKGROUND so the user keeps working. THE BEST PATH is the core adom-desktop verb desktop_record_window_start (WGC) — or nbrowser_record_start {method:"wgc"}, which delegates to it: no source picker, no banner, captures occluded windows, no focus theft. The extension's own getDisplayMedia/tabcapture recording is a LAST-RESORT FALLBACK (Chrome forces a manual share-source picker a human must click) and is refused unless you pass fallbackConfirmed:true. Use the native browser ONLY when a demo needs the user's real session/cookies/logins; otherwise record via pup or Hydrogen (see the demo-recording skill). Covers WGC vs the heinous getDisplayMedia picker, the paint-throttling problem, the occlusion/throttle-disable launch flags, the restart-native-Chrome-into-background-recording-mode flow (ask + warn + full process-tree kill + relaunch with profile + flags), and the offer-to-revert-for-battery step. Match on: record a browser window, record the native browser, record a logged-in demo, background recording of Chrome, nbrowser_record, desktop_record_window_start, WGC, source picker, share picker, full-framerate native recording. Part of the adom-browser-extension. --- Part of **adom-browser-extension** (see **driving-the-extension** for the verb surface). The broad demo pipeline (Hydrogen vs pup, scripting, TTS, storyboard, mux, wiki upload) lives in the **demo-recording**@@ -13,14 +13,35 @@ skill — this page is ONLY the part that's unique to recording the user's **rea (demo-recording skill) — those are throwaway/managed surfaces you can drive freely without disturbing the user's real browser. -## The goal forces everything: full-resolution, HIGH FRAMERATE (20–60 fps)+## ⭐ FIRST: record with WGC (the core adom-desktop window recorder), NOT the extension+**The single best way to record a browser window is the CORE adom-desktop verb+`desktop_record_window_start` (Windows Graphics Capture).** You can call it directly, or call+`nbrowser_record_start {method:"wgc", titleContains:"<title>"|hwnd:<n>}` which delegates straight to it.+WGC records ONE window in the **background, up to 60 fps, with NO source picker, NO "sharing" banner**,+captures the window **even when it is occluded**, and **never steals the user's focus**. Stop with+`desktop_record_window_stop` (or `nbrowser_record_stop {method:"wgc",recordingId}`); poll+`desktop_record_window_status`. **Reach for WGC first, every time.**++### ⛔ The extension's OWN recording is a LAST-RESORT FALLBACK, and the bridge enforces it+`method:"getdisplaymedia"` / `"tabcapture"` record through the browser itself, and `nbrowser_record_start`+**refuses them unless you also pass `fallbackConfirmed:true`** (a record_start with no method just returns+the WGC hint). Why they're bad, and why being a "full browser extension" does NOT save you:+- **Chrome forbids silent screen capture.** Even a full extension cannot record without `getDisplayMedia`+ popping a **share-source PICKER a human must click** to choose the window/screen. You do **not** get to+ override that — it is a hard browser-security gate. (This is the exact trap to avoid: an AI that blindly+ calls record_start lands on a picker that just sits there waiting for a human.)+- It paints a **persistent recording banner**, and it does **not** reliably capture a **background/occluded**+ window. WGC has none of these problems because it captures at the OS level, below the browser.+- Use `getdisplaymedia` ONLY if WGC is genuinely unavailable (e.g. non-Windows with no WGC). Even then the+ picker can only be skipped by **launching Chrome yourself** with+ `--auto-select-desktop-capture-source="<window title>"`.++## The goal forces everything: full-resolution, HIGH FRAMERATE (20-60 fps) Demos must look amazing. **The enemy is PAINT-THROTTLING, not the codec.** Chrome *correctly* stops painting occluded / background / static windows (it's the right battery-saving default), and capture is frame-driven,-so a throttled window silently drops to ~5 fps ("frozen" / sped-up video). CDP screencast is **not** the-problem — it's pup's good 20–30 fps engine. The native-browser engines are real high-fps too:-- **`wgc`** — Windows Graphics Capture (via adom-desktop), 60 fps, **picker-free**, mp4 on disk (preferred on Windows).-- **`getdisplaymedia`** — the WebRTC/Meet pipeline, 30–60 fps, cross-platform; the share picker is auto-skipped- with the Chrome flag `--auto-select-desktop-capture-source="<window title>"` **when we launch Chrome**.+so a throttled window silently drops to ~5 fps ("frozen" / sped-up video). This bites WGC too: WGC faithfully+records an occluded window, but if Chrome has stopped *painting* it, WGC captures a blank/frozen surface. The+fix is the occlusion/throttle-disable launch flags below, NOT foregrounding the window. ## Why the native path is different from pup: you must NOT foreground the user's real Chrome pup beats paint-throttling with the **foreground-rebump** trick (raise the throwaway window before each