← Commit history

Document the recording doctrine (full-framerate background; pup-vs-native-login; restart-into-mode + revert) in README + skill + record.js

John Lauer ·a295aba6db · ·parent 3bf2157
3 files changed +93−3
README.md+46−3
@@ -226,10 +226,53 @@ node bridge/server.js #    adom-desktop nbrowser_status   ->  {ok:true, ext:"adom-browser-extension", browser:"chrome"|"edge"} ``` -## Recording — two methods (`nbrowser_record_start {method}`)+## Recording — full-framerate, in the background (the doctrine) -The browser can be recorded for demos with **either** of two engines. Pick with `method`; on Windows-it defaults to `wgc` (via adom-desktop), otherwise `getdisplaymedia`.+> **Why this matters — and why we keep relearning it.** Demos must look **amazing**, which means+> **full-resolution, high-framerate (30–60 fps)** capture. That single requirement drives every choice+> here. ⛔ We do **NOT** record via CDP `Page.startScreencast` — it's low-fps JPEG and looks like garbage;+> CDP is for screenshots/driving, never demo video.++**Background recording needs TWO things, both required:**+1. **A high-fps engine** — `wgc` (60 fps, Windows, picker-free) or `getdisplaymedia` (the WebRTC/Meet+   pipeline, 30–60 fps, cross-platform).+2. **A browser that keeps *painting* the occluded window at full rate** — which requires the background+   launch flags `--disable-features=CalculateNativeWinOcclusion --disable-backgrounding-occluded-windows+   --disable-renderer-backgrounding --disable-background-timer-throttling`. **Without them Chrome stops+   painting an occluded window — which is its *correct, battery-saving default* — and the engine captures+   blank/frozen frames.** Those flags are an **exception we want only for the recorded window, only while+   recording**, then put back.++### The two scenarios+- **Scenario 1 — recording that does NOT need the user's login → use `pup`.** Pup is a browser *we*+  launch, already running with the background flags, so it captures full-framerate in the background out+  of the box; the user's real Chrome is untouched and they keep working. **Preferred whenever the app+  doesn't need the user's real session.**+- **Scenario 2 — recording that NEEDS the user's login → native Chrome (this extension).** Only the+  user's real Chrome has their cookies/logins. To record it at full framerate in the background:+  1. Check whether native Chrome already runs with the flags.+  2. If not → **ASK + WARN the user we must fully restart their Chrome.** Then **fully kill the entire+     `chrome.exe` process tree** — native Chrome spawns many helper/background processes (GPU, utility,+     crashpad, the background-mode launcher), so closing the window is NOT enough; the flags don't apply+     until *every* `chrome.exe` is gone — wait for full shutdown, and **relaunch with their real profile**+     (logins survive in the profile dir; tabs return via session-restore) **plus the flags**.+  3. Record the native window at full framerate **in the background** (`wgc`, or `getdisplaymedia` ++     `--auto-select-desktop-capture-source`) while they keep working.+  4. **Later, OFFER TO REVERT:** *"Chrome is in full-background recording mode, which uses more battery —+     want us to put you back to normal?"* → if yes, fully restart Chrome again **without** the flags.+     Best of all worlds: real session **and** amazing recordings **and** normal battery the rest of the time.++The hard requirement under both transitions is a **robust full-restart of native Chrome** (kill the whole+process tree, confirm it's down, relaunch with profile + flags — and again to revert).++**⛔ Never, no matter how tempting:** foreground the window to "fix" a blank recording (defeats the whole+point — the user must keep working) · leave the background flags on globally/permanently (drains laptop+battery) · use CDP screencast for demo video (low framerate, ugly).++### The two engines++The browser can be recorded with **either** of two engines. Pick with `method`; on Windows it defaults to+`wgc` (via adom-desktop), otherwise `getdisplaymedia`.  | | **`getdisplaymedia`** (a.k.a. desktopCapture) | **`wgc`** (Windows Graphics Capture, via adom-desktop 1.8.148+) | |---|---|---|
extension/src/record.js+12
@@ -1,5 +1,17 @@ // record.js — SW-side control for browser recording (nbrowser_record_*). //+// 🎥 DOCTRINE — why it's built this way (full detail: README "Recording" + driving-the-extension skill):+//   GOAL: demos must look AMAZING → full-res, HIGH FRAMERATE (30–60fps). That drives every choice.+//   ⛔ NEVER record demo video with CDP Page.startScreencast — low-fps jpeg, ugly. CDP = screenshots/driving only.+//   Background capture needs BOTH (1) a high-fps engine AND (2) a browser launched with the occlusion/throttle-+//   disable flags (--disable-features=CalculateNativeWinOcclusion --disable-backgrounding-occluded-windows+//   --disable-renderer-backgrounding --disable-background-timer-throttling) so it keeps PAINTING the occluded+//   window. Without them Chrome stops painting occluded windows (its CORRECT battery-saving default) → blank frames.+//     • No login needed  → record via PUP (we launch it already-flagged; full-fps background; real Chrome untouched).+//     • Login needed      → native Chrome: ASK+WARN, FULLY kill the whole chrome.exe process tree, relaunch with the+//       user's profile + the flags, record full-fps in the BACKGROUND, then OFFER TO REVERT later (battery saver).+//   ⛔ NEVER foreground the window to "fix" a blank frame; NEVER leave the flags on globally/forever.+// // TWO methods (caller picks via args.method; README has the full pros/cons): //   • 'getdisplaymedia' (desktopCapture) — CROSS-PLATFORM (Mac/Ubuntu/Windows). High-fps (the WebRTC //     pipeline Google Meet uses). Chrome REQUIRES a one-time "choose what to share" picker per
skills/driving-the-extension/SKILL.md+35
@@ -198,6 +198,41 @@ windows in the background. So: adom-desktop -t AdomLapper desktop_bring_to_front '{"titleContains":"<page title>","state":"restore"}'  # or {hwnd} ``` +## 🎥 Recording doctrine — full-framerate, in the background (READ THIS; we keep relearning it)++**The point:** demos must look **amazing** → **full-resolution, high-framerate (30–60 fps)**. That one+requirement drives everything. ⛔ **NEVER record demo video with CDP `Page.startScreencast`** — it's low-fps+JPEG, looks like garbage. CDP is for screenshots/driving only. The real engines are **`wgc`** (60 fps,+Windows, picker-free) and **`getdisplaymedia`** (WebRTC/Meet pipeline, 30–60 fps, cross-platform).++**Background capture needs BOTH:** (1) a high-fps engine, AND (2) a browser that keeps **painting the+occluded window** — which needs the launch flags `--disable-features=CalculateNativeWinOcclusion+--disable-backgrounding-occluded-windows --disable-renderer-backgrounding --disable-background-timer-throttling`.+Without them Chrome stops painting an occluded window (its **correct, battery-saving default**) and the+engine records blank/frozen frames. The flags are an exception we want **only for the recorded window,+only while recording**, then revert.++**Scenario 1 — no login needed → use `pup`.** Pup is launched by us, already flagged → full-framerate+background recording, the user's real Chrome untouched. **Preferred whenever the app needs no real session.**++**Scenario 2 — login/session needed → native Chrome (this extension).** Only the user's real Chrome has+their cookies/logins. To record it full-framerate in the background:+1. Check if native Chrome already has the flags.+2. If not → **ASK + WARN the user we must fully restart their Chrome**, then **kill the entire `chrome.exe`+   process tree** (it spawns many helper/background processes — closing the window is NOT enough; the flags+   don't apply until every `chrome.exe` is gone), wait for full shutdown, and **relaunch with their real+   profile** (logins survive; tabs restore) **plus the flags**.+3. Record full-framerate in the background (`wgc`, or `getdisplaymedia` + `--auto-select-desktop-capture-source`)+   while they keep working.+4. **Later OFFER TO REVERT:** "Chrome is in full-background mode (more battery) — back to normal?" → if yes,+   fully restart Chrome again **without** the flags. Real session + amazing recordings + normal battery.++**Hard requirement under both transitions:** a robust **full-restart of native Chrome** (kill the whole tree,+confirm down, relaunch with profile + flags — and again to revert).++**⛔ NEVER:** foreground the window to "fix" a blank recording (defeats the point — the user must keep+working) · leave the flags on globally/permanently (battery drain) · use CDP screencast for demo video.+ ## Captions (two systems) - **In-page** `nbrowser_caption {text, style, position}` — `style:"subtitle"` (calm white-on-dark, the   HD/web-hydrogen look) or `"alert"` (red **pulsing** banner). Tied to a page (cleared on navigation).