NB - Native Browser Bridge

@@ -1,12 +1,19 @@ --- name: driving-the-desktop-description: How to drive the OS desktop with adom-desktop (AD) `desktop_*` verbs when CDP/`nbrowser_*` CAN'T reach the target — the extension's OWN lifecycle (install via "Load unpacked", uninstall via "Remove", reload), `chrome://extensions` / `edge://extensions` and other `chrome://`/`edge://` pages, native file/permission dialogs, the omnibox/toolbar/tabs, and borderless popup bubbles. Match this when the user says "uninstall/reinstall/reload the extension", "install it for me", "the extension isn't loaded", "click the Remove/Load-unpacked button", "drive chrome://…", "click that OS dialog / popup / dropdown", or any browser-chrome UI outside the page. Covers the zero-math image-space click path (coordMap + space:"image"), background UI Automation (find_control → ui_click/ui_set), full-screen capture that SEES popups, modifier chords, and the hard-won traps. Needs AD ≥ 1.8.175 on the laptop + matching CLI; `--target` the laptop.+description: How to drive the OS desktop with adom-desktop (AD) `desktop_*` verbs when CDP/`nbrowser_*` CAN'T reach the target — the extension's OWN lifecycle (install via "Load unpacked", uninstall via "Remove", reload), `chrome://extensions` / `edge://extensions` and other `chrome://`/`edge://` pages, native file/permission dialogs, the omnibox/toolbar/tabs, and borderless popup bubbles. Match this when the user says "uninstall/reinstall/reload the extension", "install it for me", "the extension isn't loaded", "click the Remove/Load-unpacked button", "drive chrome://…", "click that OS dialog / popup / dropdown", or any browser-chrome UI outside the page. Covers the zero-math image-space click path (coordMap + space:"image"), background UI Automation (find_control → ui_click/ui_set), full-screen capture that SEES popups, modifier chords, and the hard-won traps. Needs AD ≥ 1.8.176 on the laptop (for `desktop_navigate` + `scope:"popups"` — the full background `chrome://` lifecycle) + matching CLI; `--target` the laptop. --- Parent skill: **driving-the-extension**  When CDP/`nbrowser_*` can't reach something, drive the **OS** instead. This skill is the playbook a previous AI spent an hour re-discovering — follow it and you won't have to. +> ✅ **The full `chrome://` lifecycle is background as of AD 1.8.176** (verified end-to-end on a real+> Edge uninstall→reinstall, 2026-06-23). `desktop_navigate` opens `chrome://`/`edge://` pages with no+> foreground, and `scope:"popups"` reaches the confirm bubble. **The ONE unavoidable foreground beat** is a+> brief (~1–2s) `desktop_bring_to_front` *while the confirm bubble is up* — the bubble is a Chromium popup+> that dies on focus loss, so the window must hold focus until you click its button. Everything else —+> navigate, find, the Remove/Load-unpacked clicks, even closing the window — is background.+ ## When CDP fails → switch to desktop control - **`chrome://` / `edge://` pages** (incl. `…/extensions`, settings) are **blocked to CDP** /   `nbrowser_navigate`. Only real OS input reaches them.@@ -72,44 +79,55 @@ Drive controls through the **accessibility tree** — the window is never raised it **auto-hides after 60s** (refreshed on each show/move). Extend with `ttlMs`, or `persist:true` then hide yourself. **No need to hide "to be safe"** — only hide early to clear it sooner. -## Recipe — open a `chrome://`/`edge://` page (CDP can't)-`Ctrl+N` (a NEW window so a background tab elsewhere can't steal the active tab) → `Ctrl+L` → type the URL-→ `Enter`, scoped to the new window's `hwnd`:+## Recipe — open a `chrome://`/`edge://` page (CDP can't) — BACKGROUND (AD 1.8.176)+Open a background window first (`nbrowser_open_window {profile, url:"https://example.com", focused:false}`,+then find its `hwnd`), then: ```-desktop_press_key {hwnd:<NW>, keys:["ctrl+n"]}      # then find the new window's hwnd-desktop_press_key {hwnd:<NW>, keys:["ctrl+l"]}-desktop_type      {hwnd:<NW>, text:"edge://extensions"}-desktop_press_key {hwnd:<NW>, keys:["enter"]}+desktop_navigate {hwnd:<W>, url:"edge://extensions"}   # SetValue omnibox + posted Enter — NO foreground ```+⚠️ **Scheme must match the browser:** `chrome://…` for Chrome, `edge://…` for Edge. Give Chrome an+`edge://` URL (or vice-versa) and the omnibox **searches** the text instead of navigating. `https://` works+on either. **Always verify** with `desktop_screenshot_window` (a posted Enter is app-specific). Fallback if+a target ignores it: `desktop_bring_to_front` + `desktop_press_key ["enter"]` (brief foreground).+(`desktop_ui_set` also takes `submit:true` to set+commit an arbitrary field.) -## Recipe — UNINSTALL the extension (proven; simplified for 1.8.175)-1. Open `edge://extensions` (recipe above).-2. `find_control {hwnd:<NW>, role:"button", contains:"Remove"}`. `invokable` → `ui_click` it (background,-   no bubble-focus dance). Page tree cold → retry once → else `desktop_click {x:rect.centerX,-   y:rect.centerY}`.-3. The **"Remove '<name>'?" confirm is a borderless popup bubble** anchored to the toolbar icon —-   invisible to `screenshot_window`, absent from `list_windows`. So: **`desktop_screenshot_screen`** → SEE-   the bubble → **`desktop_click {space:"image", shotId, x, y}`** on its "Remove".-   *(Pre-1.8.175 hack, no longer needed: open the bubble, then `desktop_press_key {keys:["enter"]}` with-   NO `hwnd` so focus isn't stolen — the bubble auto-dismisses on focus loss.)*-4. **Verify by state, not pixels:** poll **`nbrowser_profiles`** — the `edge:`/`chrome:` key **drops** when-   the SW dies. (The page render lagged/looked unchanged and burned an hour the first time.)+## Recipe — UNINSTALL the extension — BACKGROUND except one ~1s focus beat (verified AD 1.8.176)+1. `desktop_navigate {hwnd:<W>, url:"edge://extensions"}` (background). `find_control {hwnd:<W>,+   contains:"<ext name>"}` to confirm the card is present.+2. The confirm bubble is a Chromium **owned popup** that dies on focus loss, so it must hold focus while you+   click it. The working atomic sequence — **nothing between the trigger and the confirm**:+   ```+   desktop_bring_to_front {hwnd:<W>}                                  # the ONE ~1s foreground beat+   desktop_ui_click {hwnd:<W>, role:"button", contains:"Remove"}      # trigger → opens the bubble+   # wait ~1s+   desktop_ui_click {hwnd:<W>, scope:"popups", role:"button", name:"Remove"}   # confirm (background Invoke)+   ```+   `scope:"popups"` enumerates the window's **owned, borderless** popups (absent from `desktop_list_windows`)+   and UIA-searches them; the bubble's window title is `Remove "<ext>" from <browser>?`. **Don't** insert a+   `screenshot`/extra `find_control` between trigger and confirm — the bubble (~1–2s lifetime) closes and+   you'll get `scope:popups found no owned/popup windows`; just re-trigger and confirm immediately.+3. **Verify by state:** poll `nbrowser_profiles` — the `edge:`/`chrome:` key **drops** when the SW dies. -## Recipe — REINSTALL (Load unpacked)-1. Click **"Load unpacked"** (find_control → `ui_click`, or image-space click) → opens a Win32 **folder-   picker** (`#32770`; it DOES enumerate and has an a11y tree).-2. Put the extension dir in the **Folder:** field, then **Select Folder**:-   `ui_set` the Folder edit (or `desktop_type` the path, e.g.-   `C:\Users\<user>\adom-browser-extension\extension`) → `ui_click "Select Folder"`.-3. Load-unpacked reads the **on-disk** build (version may jump). Verify the `edge:`/`chrome:` key returns-   to `nbrowser_profiles` (reconnects in ~1–2s).+## Recipe — REINSTALL (Load unpacked) — BACKGROUND+1. `desktop_ui_click {hwnd:<W>, contains:"Load unpacked"}` → opens a Win32 **folder picker** (`#32770`;+   enumerates in `desktop_list_windows`, has an a11y tree).+2. `desktop_ui_click {hwnd:<picker>, role:"button", contains:"Select"}` to confirm the folder.+   ⚠️ `desktop_ui_set` on the picker's edit field can return *"operation was canceled by user"* — but the+   picker **remembers the last-used dir**, so if it's already inside the extension folder, Select Folder+   grabs the right one. If it might open elsewhere, navigate the picker / set the path some other way first+   (open AD ask: reliable picker path-fill).+3. Load-unpacked reads the **on-disk** build (version may jump). Verify the `edge:`/`chrome:` key returns to+   `nbrowser_profiles` (reconnects in ~1–2s).  ## Why these traps exist (so you don't re-discover them) - **"My click did nothing"** was an illusion — the click worked; the confirm bubble was just *invisible to-  window-capture*. Use `screenshot_screen`.-- **SendInput foregrounds** (it lands only on the foreground window) → interrupts the user *and* dismisses-  focus-sensitive bubbles. Prefer **UIA `ui_click`**; if you must SendInput to confirm a focus-sensitive-  popup, **don't pass an `hwnd`** (so AD doesn't raise/steal focus).+  window-capture* (it's an owned popup). Reach it with `find_control`/`ui_click` `scope:"popups"`; to merely+  see it, `screenshot_screen`.+- **The confirm bubble lives ~1–2s and dies on focus loss.** Hold focus with `desktop_bring_to_front`,+  then trigger + `scope:"popups"` confirm with NOTHING in between (no screenshot/extra find_control).+- **SendInput foregrounds** (lands only on the foreground window) → interrupts the user. Prefer UIA+  (`ui_click`/`ui_set`/`desktop_navigate`) which is background; reserve SendInput for controls UIA can't+  reach. - **Don't pixel-hunt.** Image-space click (`space:"image"`) or `find_control` rect. Don't fetch the window   rect to compute scale — `coordMap` already has it.