---
name: driving-the-extension
description: How to drive the Adom browser extension (nbrowser_* verbs) - control the user's REAL logged-in Chrome/Edge from the cloud. Covers the CDP-vs-native split (and the "debugging this browser" bar), the full verb surface, isTrusted input (click-by-text + xy-finding, type/press_key, hover, right-click), the gliding Adom cursor with intent narration, screenshots, the taskbar status feature (flash/progress/Adom-overlay), background-safe capture (never stealing the user's focus), captions, the chip-fetcher use case, and every pitfall learned building it.
---

# Driving the Adom Browser Extension (`nbrowser_*`)

This extension turns the user's **real, logged-in Chrome/Edge** into something the agent drives - the
native-browser counterpart to `pup`. Because it runs **inside the user's actual profile**, their
cookies, SSO, saved logins and human-trust signals all apply, so it sails through login walls and
captchas that block pup's empty controlled-Chromium. That is the whole point.

## ⛔ Golden rule: NEVER reuse a window the user is in - always open a fresh one

The user is working in their browser. **Never navigate, reload, or repurpose a window/tab they
already have open** - not with `nbrowser_navigate`, and **not with the OS-level `desktop_navigate`
either.** That yanks their live page out from under them and is rightly seen as hostile. This applies
to **everything you do for your own needs** - driving a workflow, reading a doc, and especially
**grabbing a screenshot** (e.g. of a `chrome://extensions` / `edge://extensions` page for a guide).

The rule, every time: **open a NEW window** (`nbrowser_open_window {sessionId, thread, purpose, url}`, or
a fresh OS window you then `desktop_navigate`), do your work and capture in THAT window, and **close it
when done** (`nbrowser_close_window {sessionId, includeSpawned:true}`). The
only thing you may do to one of their existing windows is `desktop_bring_to_front` to surface it for
them - never to drive or navigate it.

**As of extension v0.0.21 this rule is ENFORCED IN CODE, not just documented.** Every mutating verb
(`navigate, reload, back, forward, click, type, press_key, hover, input_dispatch, select, set_date,
eval, cursor, caption, focus, close_tab, switch_tab, wait_for, totp, autologin, login_form,
record_start`) refuses:
- a **bare call** (no `sessionId`/`tabId`) → `no_target` — the old active-tab fallback is gone; it is
  how an agent navigated a user's working tab and wiped their other AI threads;
- a **target outside a window the agent opened** → `not_owned` (`nbrowser_open_window`'s session map
  IS the ownership registry). `open_tab` likewise requires an owned session's `sessionId`.
There is **NO agent-passable override**. Refusals carry a teaching `_hint` with the corrected flow,
and the first response of each connection carries a one-time `_safety` banner stating this rule.
(`tools/check-guard-coverage.mjs` fails the build if a new verb ships unclassified or ungated.)

## ⭐ MANDATORY PREFLIGHT — profiles first, then your own window

Before driving ANYTHING, in this order:
1. `nbrowser_profiles` (read-only) → the connected `browser:email` keys (Chrome AND Edge). Each row
   also carries **`displayName`** — the human label the USER sees on the browser's own profile button
   ("John Personal", "adom.inc") — plus `profileDir` and `hasAvatar` (v0.1.20+). **Never assume which
   browser holds an account** — the currently-active browser is NOT "the" browser. A row with
   **`unresolved:true`** hasn't announced its email yet (fresh bridge restart): run
   `nbrowser_resolve_profiles` FIRST or you may drive the wrong account. A row with **`asleep:true`**
   EXISTS on the machine but has no open window (its extension service worker sleeps), so it can't be
   driven yet — **wake it yourself with `nbrowser_wake_profile {profile:"<key>"}`**: it launches the
   browser with `--no-startup-window`, so the profile + extension load in the BACKGROUND — **no window
   opens, nothing foregrounds, no focus is stolen** — and the profile flips to live in ~2s. NEVER ask
   the human to open a window for you, and NEVER launch a foreground browser to wake a profile — the
   wake verb exists precisely so waking honors background-by-default. (`woke:false` usually means the
   Adom extension isn't installed in THAT profile — extensions are per-profile; offer the install.)
2. Pick the profile whose **email matches the identity the task needs** — email DOMAIN + `displayName`
   tell you work vs personal (corporate domain = work; gmail.com etc. = personal). Setting up a tool for
   `person@gmail.com` → drive `chrome:person@gmail.com`, not whatever profile happens to be focused.
   If the needed profile isn't listed, ask the user to open a window in it — don't improvise in
   another one.
3. `nbrowser_open_window {sessionId:"<task id>", thread:"<which AI thread>", purpose:"<why, human-readable>", profile:"<key>", url:…}` → a BACKGROUND window auto-pinned to that profile. **All three of sessionId/thread/purpose are REQUIRED** (registration) or it refuses `session_unregistered`; `purpose` shows in the Session Monitor.
4. Drive ONLY via that `sessionId`. If a page you drive **spawns** a window (a Gmail-Contacts email click pops a compose; `target=_blank`/OAuth popups), your next verb is refused `unacknowledged_spawns` → `nbrowser_owned {sessionId}` to see it, `nbrowser_cleanup {sessionId}` to close the litter.
5. **End the task by closing what you own:** `nbrowser_close_window {sessionId, includeSpawned:true}`.

## ⛔ Background by default - NEVER foreground unless the user asks

**The user is almost always doing something else on their computer** - reading, typing, on a call,
in another app. So your work MUST **default to the background, especially when you open new browser
windows**: a window that jumps to the front interrupts whatever they were doing and (worse) their next
keystrokes land in the control you just surfaced. Background is not politeness, it is the **default
posture**; foreground is **opt-in, only when the user explicitly asks to watch.**
**Never raise a window to the foreground** for your own needs. Foregrounding STEALS the user's keyboard
focus - if they are typing, your window captures their keystrokes (it has typed a user's message into a
username field and killed Chrome's password dropdown). As of v0.1.0 a bare `userRequestedForeground:true`
is **not enough**: without a `foregroundReason` the window stays background. To genuinely foreground
(some auth pages need it): **first** `adom-desktop notify_user` to warn the user, **then** pass
`userRequestedForeground:true` + `foregroundReason:"<why>"`, do the one thing, and return to background.
The verbs are built for this: `nbrowser_open_window` opens **unfocused**, `open_tab` opens
`active:false`, `switch_window` does **not** raise - leave those defaults alone. Capture without
foregrounding: **`nbrowser_screenshot` (CDP) and `desktop_screenshot_window` (WGC) both capture a
background/occluded window**, and WGC records one at full fps (with the occlusion flags). So there is
never a reason to pull a window forward to grab a shot or a recording.

**Capturing a `chrome://` / `edge://` page** (e.g. `chrome://extensions` for a guide) is the classic
trap: do **NOT** launch the browser yourself - `msedge` / `chrome --new-window` (via `launch` /
`launch_process`) **raises the new window to the foreground.** Instead:

1. `nbrowser_open_window {sessionId:"guide", thread:"<you>", purpose:"screenshot chrome://extensions for a guide", url:"about:blank"}` → a fresh **background** window (note its `hwnd`).
2. `desktop_navigate {hwnd, url:"chrome://extensions"}` → loads the `chrome://` page **in the
   background**, no foreground.
3. `desktop_screenshot_window {hwnd}` → WGC captures it where it sits.
4. `nbrowser_close_window {sessionId}`.

Foregrounding is the **user's** choice - only `desktop_bring_to_front`, and only when they explicitly
ask to watch live.

⚠️ **Foreground can happen even when you NEVER call `bring_to_front`.** Opening a new window, and
especially **`desktop_navigate`** (its omnibox commit can surface/activate the window), may bring a
window to the FRONT despite `focused:false`. Judge by the **outcome, not the verb**: after an open or a
navigate, check `desktop_list_windows` (a window at `z:0` is topmost = you surfaced it) and treat that
as a disruption. Observed live: a `desktop_navigate` to `chrome://extensions` left the window at `z:0`
with no `bring_to_front` called.

**Path:** `cloud → relay → adom-desktop (laptop) → native-browser bridge → native-messaging host →
extension service worker → back`. You call `adom-desktop nbrowser_* '<json>'`.

## Targeting (READ FIRST)
If more than one Adom Desktop is on the relay (laptop **AdomLapper** + the **winvm**), every call must
name one or it errors `ambiguous_target`:
```bash
adom-desktop --target AdomLapper nbrowser_status
```
A "session" = a real Chrome window (`sessionId` you choose, or auto `s1`…). `tabId` = the real chrome
tab id. The session→window map persists to `chrome.storage`, so sessions survive reloads.

## ⭐ The #1 thing to understand: CDP vs native (and the "debugging this browser" bar)
The extension is a **CDP-primary hybrid** with two engines:
- **`chrome.debugger` (CDP)** - needed for *real* `isTrusted` input, full-page screenshots, and page
  `eval`. **Attaching it is what makes Chrome show the yellow `"Adom for Chrome & Edge" started
  debugging this browser` bar.** The bridge **lazy-attaches** only for CDP verbs and **idle-detaches
  ~8s** later, so the bar appears during active clicking/typing and clears when idle.
- **Native `chrome.*` APIs** - windows/tabs/navigation/downloads. **No debugger → no bar.**

| 🟡 Shows the bar (CDP) | ⚪ No bar (native) |
|---|---|
| `click`, `double_click`, `right_click`, `hover` | `open_window`, `close_window`, `switch_window`, `list_windows` |
| `type`, `press_key`, `input_dispatch`, `cursor` | `open_tab`, `close_tab`, `switch_tab`, `list_tabs`, `rescan` |
| `screenshot`, `screenshot_full_res` | `navigate`, `reload`, `back`, `forward`, `wait` |
| `eval` / `evaluate`, `errors` | `fetch_url`, `download`, `caption`, `focus`, `flash`/`taskbar` |

The bar is a Chrome security guarantee and **cannot be hidden on consumer Chrome** while driving. On
Adom-managed machines the `--silent-debugger-extension-api` launch flag removes it. If you only need to
read/navigate, prefer native verbs and the bar never shows.

## Verb surface
- **meta:** `status`, `ping`, `help` (full inline reference), `dev_reload` (self-completing 2-pass; throttled 1/90s)
- **windows/sessions:** `open_window` (REQUIRES sessionId+thread+purpose), `close_window`
  (`includeSpawned:true` also closes spawns), `switch_window`, `list_windows` (returns `bounds`+`state`+an
  `owner` label agent-opened|agent-spawned|user; `mine:true` filters to yours), `window_state`/`fullscreen`
  (maximize·fullscreen·normal·minimize, or `left/top/width/height` to move+resize - native, no bar;
  F11-via-key does NOT work), `rescan`, `resync_sessions`
- **ownership / cleanup:** `owned {sessionId}` (opened + SPAWNED windows/tabs a driven page opened;
  acknowledges spawns) · `cleanup {sessionId}` (close ONLY what this session spawned, never the user's) ·
  `force_close {windowId}` (two-step attested last resort for a TRUE orphan; never another thread's window)
- **account / credentials:** `login_state {url}` (signed-in NOW? via `chrome.cookies`) ·
  `credentials {match?, browser?, profile?}` - a **read-only QUERY of the browser's OWN saved-login
  store** (we don't store or replace anything; Chrome/Edge stay the source of truth). Returns
  `{browser, profile, host, username}` rows across **Chrome AND Edge**, all profiles - i.e. which
  vendors the user can auto-login to. Never passwords (the browser autofills those). The extension
  can't read this (sandbox); the bridge reads each browser's `Login Data` SQLite natively.
- **reliable automation:** `wait_for`, `keepalive`, robust `click` (returns `changed`), `set_date`,
  `download_wait`, `totp`/`register_totp` (2FA), `add_extension_origin` - see **Reliable automation** below.

## Reliable automation (waits, click-verify, 2FA, downloads, dates)
The primitives that make an unattended multi-vendor pull (Gusto/QuickBooks/Carta) actually hold up.
Prefer these over fixed sleeps + coordinate guessing.

- **Deterministic waits, not sleeps.** After a click/nav, `nbrowser_wait_for {sessionId, selector |
  urlIncludes | textIncludes, timeoutMs?}` blocks until the thing you're waiting for exists, then
  returns `{matched, by, url, waitedMs}`. `matched:false, timedOut:true` = it never appeared -
  screenshot to see why. Never race a fixed `nbrowser_wait {ms}` against a page load.
- **Click, then verify it landed.** `nbrowser_click` now scrolls the target into view and returns
  **`changed`** (did the URL/DOM actually change) + `urlBefore`/`urlAfter`. `changed:false` means the
  click dispatched but nothing happened (disabled/covered element, missed target) - re-resolve the
  selector or retry; don't assume a click worked.
- **Custom elements + shadow DOM just work.** The finder behind `nbrowser_click`/`eval`/`select`
  **pierces open shadow roots** (recursively), so a control whose real options live in a web-component's
  shadow tree (Schwab's account picker, QBO's report search) is findable by **visible text** or a
  **stable attribute** (`[data-testid=…]`) - no more "not found → fall back to fragile
  screenshot-coordinate clicks." (Only *open* shadow roots are reachable; closed ones are invisible to any
  script.)
- **Pick a dropdown option by its TEXT - never coordinate-click a list.** `nbrowser_select
  {label | near, optionText, timeoutMs?}` is the reliable primitive for custom AND native selects: it
  finds the trigger (`label` = the control's accessible name, or `near` = a visible label beside it),
  opens it with an isTrusted click, **waits** for the options to render, clicks the **exact** text match
  (shadow-piercing), and **returns the newly-selected value** (`selected` + `clickedText`) so you can
  confirm it. This is what stops coordinate clicks landing on the wrong row of a virtualized list
  (mis-selecting a bank account by one row). ALWAYS check `selected` before continuing.
- **Background clicks now land on occluded windows (auto).** When the user's own window is maximized
  over our background one, Chrome's occlusion tracking used to make isTrusted CDP clicks silently land
  NOWHERE (eval/screenshots still worked - they don't need a hit-testable frame). The extension now
  emulates focus on attach, so `nbrowser_click`/`type`/`select` land while the window stays in the
  background (no focus theft). If a click ever reports `changed:false` on a page you know is interactive,
  it is almost never occlusion anymore - re-resolve the target or `wait_for` the result.
- **Stay signed in.** `nbrowser_keepalive {sessionId, intervalSec?}` replays a benign mousemove/focus on
  an interval (default 4 min) to beat vendor idle sign-outs (Gusto/Intuit ~10-15 min). Set it at the
  START of a long automation; it auto-stops when the window closes; `{stop:true}` to cancel. (Client
  activity only - a vendor that expires the SERVER session regardless may still need periodic real actions.)
- **2FA / TOTP - the unattended-automation unblock.** One-time: `nbrowser_register_totp {host, seed}`
  with the vendor's base32 "manual entry / setup key" (shown by the 2FA QR). It's stored `0600` on the
  laptop and **NEVER returned**. Then, when a 2FA prompt appears, focus the code field (or pass
  `selector`) and `nbrowser_totp {host, submit?}` - the **bridge** computes the RFC-6238 code and the
  extension **types it**; the agent never sees the seed or the code. `errorCode:"no_totp_seed"` = register
  it first. Codes rotate every 30 s, so call it right when you need it.
- **Downloads by event, not by polling.** After you trigger an export, `nbrowser_download_wait
  {sessionId, timeoutMs?, sinceMs?}` returns the landed `files[]` via chrome.downloads events (no
  timestamp-guessing the Downloads folder). Pass `saveToSubfolder:"<name>"` to route subsequent
  downloads into a subfolder of Downloads.
- **Segmented date fields.** Finance apps use react-aria spinbutton date fields (month/day/year).
  `nbrowser_set_date {selector, date:"YYYY-MM-DD", order?}` focuses the field and types each digit as a
  real keydown so the segments advance. US MDY by default; pass `order:"DMY"` for day-first. Check
  `fieldValue` in the response.
- **Edge different-ID lockout.** If the extension is loaded in Edge but never shows in
  `nbrowser_profiles`, its loaded-unpacked ID may differ from the one in the native-host manifest's
  `allowed_origins`. Copy the ID off `edge://extensions` and `nbrowser_add_extension_origin {id}`, then
  **reload the extension** so the browser re-reads it. (Our manifest pins a `key`, so IDs should match -
  use this only if Edge diverged.)

## Logging into vendors (the key to fetching electronics data)
Every EE vendor (DigiKey, Mouser, SnapMagic, UltraLibrarian, ComponentSearchEngine, TI, NXP, Arrow…)
gates CAD/datasheet downloads behind a login. You can **automate all of it** without ever seeing a
password:
1. **Know what you can log into:** `nbrowser_credentials {match:"digikey"}` → if the vendor is in the
   library, the user has a saved login and Chrome will autofill it. (No match → ask the user to save it
   once, or pick another rung of the sourcing ladder.)
2. **Drive the login:** `nbrowser_navigate` to the sign-in page → `nbrowser_click` the email/username
   field (Chrome **autofills** username+password - you never see the secret) → `nbrowser_click` "Sign
   In"/"Log In" → `nbrowser_login_state` to confirm. Proven live on Mouser (`johnAdom`) + CSE.
3. **Signed in = no bot wall:** a real logged-in session also skips the anti-bot "security slider"
   captchas that block pup's empty profile. That's the whole point of driving the real browser.
Mouser uses a **Username** field (not email) - check `input` by name, not just `type=email`.
- **tabs:** `open_tab`, `close_tab`, `switch_tab`, `list_tabs`
- **navigation:** `navigate`, `reload`, `back`, `forward`, `wait{ms}`
- **input (isTrusted, CDP):** `click`, `double_click`, `right_click`, `hover`, `type`, `press_key`,
  `input_dispatch`, `cursor`
- **capture/eval:** `screenshot`, `screenshot_full_res`, `eval`/`evaluate`
- **data (native advantage):** `fetch_url` (uses the user's REAL cookies - authed), `download`
- **status/errors:** `errors`, `events` (window/tab closes incl. `byUser:true`)
- **demo/UX:** `caption`, `focus`, **`flash`/`flash_stop`/`flash_mode`/`taskbar`**
- **recording:** `record_start`/`stop`/`status`/`list` - ⭐ use `method:"wgc"` (delegates to the core AD verb `desktop_record_window_start`: background, 60fps, no picker, no banner, captures occluded). `getdisplaymedia`/`tabcapture` are a refused-by-default fallback (Chrome forces a manual share picker) - need `fallbackConfirmed:true`. See **native-browser-recording**.

## Parallel multi-browser / multi-profile control
Every **BROWSER × PROFILE is its own independently-addressable connection**, keyed `browser:email`
(e.g. `chrome:you@example.com`, `edge:you@example.com`, `chrome:work@example.com`). The same account signed into
Chrome AND Edge are DISTINCT connections that coexist - the host singleton AND the bridge both key by
`browser:profile`, so the 2nd browser's host no longer exits with *"Native host has exited."*

- **List them:** `nbrowser_profiles` → each entry's `profile` field is its addressable key, plus
  `browser`, `email`, `live`, `active`.
- **Target any of them, in PARALLEL:** pass `profile:"<key>"` on ANY verb -
  `nbrowser_navigate {profile:"edge:you@example.com", url:…}`. Also accepts `profile:"<email>"`+
  `browser:"edge"`, or `browser:"edge"` alone (that browser's active profile). A wrong/absent target
  no-routes cleanly (doesn't fall through to the wrong browser).
- **Multiple AI threads at once:** separate Claude/Codex threads can each drive their own
  browser/profile simultaneously - every verb carries its own explicit target, so there's no shared
  default to clobber. `nbrowser_use_profile` only sets the convenience default for verbs that omit a
  target; for parallel work, ALWAYS pass an explicit `profile`/`browser`.
- **Pin a session so you can't cross wires.** Driving several accounts and forgetting `profile:` on one
  call is how you act on the wrong profile. `nbrowser_open_window` **auto-pins** its `sessionId` to the
  profile it opened in, so every later verb carrying that sessionId auto-routes back to the SAME profile
  with NO `profile:` needed. Re-pin explicitly with `nbrowser_pin_session {sessionId, profile}`. An explicit
  `profile:` on a call still overrides the pin.
- **Resolve emails up front.** A freshly-connected profile can briefly key as `pending-N` with an empty
  email until its Google identity resolves (the extension now retries + re-announces automatically). To
  force it before you drive, call `nbrowser_resolve_profiles` (all connected profiles) or
  `nbrowser_whoami {profile}` (one) - keys become `chrome:<email>` / `edge:<email>`. A profile with no email
  isn't signed into Google; target it by its id-key or by `browser:`.
- **Exclude one:** `nbrowser_profile_block {profile}` (persisted) - a security whitelist that keeps a
  browser/profile from ever being driven.

For a browser to appear, that profile needs an open window AND the Adom extension loaded in THAT
browser (`chrome://extensions` / `edge://extensions` → Developer mode → Load unpacked → the extension
folder). The hello carries the browser (Chrome vs Edge via the UA), so each connection self-keys.

## Input - click, type, hover, right-click, and how xy is found
You don't guess pixels. The **smart finder** scores the DOM, takes the matched element's
`getBoundingClientRect`, and computes its center xy - then CDP dispatches the `isTrusted` event there.
Targeting is **DOM-truth**, not pixel-guessing.

### Why DPI / window-resize never breaks targeting (the question every AI asks)
This is **not** screenshot→eyeball-a-pixel→click automation, so it does **not** suffer that class of bug:
- **High-DPI / display scaling is handled by Chrome, not you.** `getBoundingClientRect` and CDP
  `Input.dispatchMouseEvent` are BOTH in **CSS pixels, viewport-relative** - the same space. The browser
  maps CSS→physical internally. A 4K@200% screen and a 1080p screen give the identical xy for the same
  button. There is **zero** DPI-translation code, hence zero DPI bugs. Never touch `devicePixelRatio`.
- **Screen resolution / window size / window position are irrelevant** - targeting is page-relative, not
  screen-relative. Drag Chrome to the other monitor mid-task; it doesn't matter.
- **No stale-coordinate / "dirty flag" problem.** Measure and act are **one atomic verb, sub-second,
  with no AI loop between them** - the finder re-measures `getBoundingClientRect` fresh on *every* click/
  type/hover. A user resizing the window 100ms before your click is fine: the next call reads the live
  rect. Nothing is cached across a gap. **So you can reassure the user that resizing/scrolling while Adom
  drives is SAFE** (a nice `nbrowser_caption`) - the opposite of the usual "don't touch anything" fear.
- **The only path that CAN go stale** is passing **raw `{x,y}`** to `input_dispatch` (a literal pixel,
  no re-resolve). Prefer `text`/`selector` - they re-resolve every call. AD is **not** involved in click
  xy at all (it only does OS-level foreground/taskbar/window-screenshots). **This is now ENFORCED
  (`stale_coordinates`, v0.1.0):** a raw `{x,y}` click is refused unless you pass `atScrollX`/`atScrollY`
  from your latest screenshot's coordMap AND the page hasn't scrolled since (the "Next" pixel that became
  "Back" after a re-render). Just click by `text`/`selector`.
- **Click-by-`text` that matches >1 visible element is refused (`ambiguous_target`, v0.1.0)** - two
  "Next"/"Back" buttons, three "Yes" radios. Pass a unique `selector`, or `{debug:true}` to see the
  numbered candidate overlay and pick one, or `acceptAmbiguous:true` to take the top-scored match. Verify
  `matchedText`/`changed` in the response after a text click.
```bash
# click by VISIBLE TEXT (finder computes xy) - returns {x,y,matchedText}
adom-desktop -t AdomLapper nbrowser_click '{"sessionId":"s1","text":"Pay","cursor":true,"intent":"Clicking the Pay button"}'
adom-desktop -t AdomLapper nbrowser_click '{"sessionId":"s1","selector":"#download"}'         # or by selector
adom-desktop -t AdomLapper nbrowser_type  '{"sessionId":"s1","selector":"#search","text":"STM32F103C8T6"}'
adom-desktop -t AdomLapper nbrowser_press_key '{"sessionId":"s1","key":"Enter"}'              # Enter/Tab/ArrowDown/…
adom-desktop -t AdomLapper nbrowser_hover '{"sessionId":"s1","text":"Hover me"}'              # fires :hover + tooltips
adom-desktop -t AdomLapper nbrowser_hover '{"sessionId":"s1","selector":".info-icon"}'        # or by selector (surest for hover)
adom-desktop -t AdomLapper nbrowser_right_click '{"sessionId":"s1","selector":".ctx"}'        # real context menu
```
**Hover targets are often NON-interactive** (info icons, cards, tooltip triggers, plain spans). The
finder first scans clickable elements (button/a/input/…); if `text` matches none, it **falls back to the
smallest visible element whose text matches**, so `hover {text:"…"}` works on a bare `<span>`. A CSS
`:hover`/JS `mouseenter` tooltip needs the pointer to *enter* and dwell, so hover does an approach-move →
settle → ~600ms dwell (tune with `hoverDwellMs`). When in doubt for hover, **a `selector` is surest.**

**The gliding Adom cursor:** pass `cursor:true` + `intent:"…"` on any input verb and the gorgeous Adom
cursor (teal arrow + pulsing ring + "Adom" label) glides to the target **while narrating its intention**
("Clicking the Pay button", "Hovering the card to reveal the tooltip"). Builds user trust - they always
see what the AI is doing and why. Mimics Hydrogen Desktop's cursor.

### Native JS dialogs (alert / confirm / prompt) - the invisible blocker
A native `alert()`/`confirm()`/`prompt()`/`beforeunload` is **not an OS window (no hwnd)** and does
**not appear in a screenshot** (Chrome renders it as browser chrome), but it **FREEZES the page's main
thread**. Symptom: your clicks/typing/eval suddenly do nothing and you "can't fill the form" (real
incident: an Intuit "questionnaire saved" alert sat over the form while the AI kept trying). ABE now
**auto-detects** it: the blocked verb returns `modal_open` (with the dialog text) and a screenshot carries
a `_modal` note. **Dismiss it:** `nbrowser_handle_dialog {sessionId, accept:true}` (OK/Yes/Leave),
`accept:false` (Cancel/Stay), or `text:"…"` to answer a prompt - then re-screenshot and continue. If an
action mysteriously stops working, suspect a dialog.

## Screenshots — pick the RIGHT one (page vs window vs NEVER fullscreen)
Three ways to "see", and the choice matters:
- **The PAGE → `nbrowser_screenshot {sessionId}`** (CDP). Your DEFAULT. Fast, background-safe, captures the
  page bitmap. But it shows **page content ONLY** - no browser chrome (URL/tabs) and **no native popups/
  dialogs** (Chrome's saved-password dropdown, print dialog, `alert`/`confirm`) which render OUTSIDE the page.
- **The WINDOW + chrome + native popups → `desktop_screenshot_window {hwnd | titleContains}`** (AD, WGC).
  Use this when you need the URL bar, a `chrome://` page, or a native popup/dialog. It captures the window
  **plus its OWNED CHILD windows and returns the full parent/child hwnd array** - the same technique that
  makes KiCad/Fusion error dialogs visible. Background-safe (no foregrounding). `nbrowser_open_window`
  returns the window's **`hwnd`** for exactly this; or find it via `desktop_list_windows` by title.
- **⛔ NEVER a fullscreen / whole-monitor screenshot to inspect a browser window.** It grabs the user's
  other work, is imprecise, and usually implies foregrounding. There is always a targeted option above.
  (`nbrowser_screenshot_full_res` is full-**PAGE**, not full-monitor - it still only sees the page.)
```bash
adom-desktop -t AdomLapper nbrowser_screenshot '{"sessionId":"s1","toFile":true}'   # PAGE bitmap → laptop PNG path
adom-desktop -t AdomLapper desktop_screenshot_window '{"hwnd":<from open_window>}'   # WINDOW + chrome + child popups
# then: adom-desktop -t AdomLapper pull_file '{"filePaths":["<path>"],"saveTo":"/tmp"}'
```
Screenshots are large base64 - over the relay they can time out, so **prefer `toFile:true` + `pull_file`**
(binary, chunked). `screenshot_full_res` is unresized; default resizes ≤1568 for the model.

Every screenshot response carries a **`viewport`** coordinate-map sampled at capture time, so you can turn
a screenshot pixel into a click coordinate EXACTLY without a separate `window.innerWidth`/`dpr` probe (and
detect scroll drift): `{innerWidth, innerHeight, devicePixelRatio, scrollX, scrollY, capturedWidth,
capturedHeight, imageWidth, imageHeight, resizeRatio, cssPerImageX, cssPerImageY, fullPage}`. To click a
feature you see at pixel `(px,py)` in the returned image: **`clickX = px * cssPerImageX`,
`clickY = py * cssPerImageY`** (viewport CSS px, what `nbrowser_click {x,y}` takes). Before clicking, re-read
`scrollX/scrollY` (or just re-screenshot) - if the page scrolled since the shot, a viewport pixel now points
at different content (that's the drift you were hitting). For a `fullPage:true` shot the map spans the whole
page, so those coords are page-absolute.

> **🚫 NEVER foreground the window to take a screenshot.** `nbrowser_screenshot` (CDP) captures the
> page bitmap with **zero OS-window interaction**, and AD `desktop_screenshot_window` (WGC) captures a
> **background / occluded** window fine. The user is usually working *in* their real browser - yanking
> it forward to grab a shot disrupts them. See [Do NOT foreground the window](#do-not-foreground-the-window-to-look-at-it--capture-in-place).

## Taskbar status - flash, progress, the "Adom is driving" overlay badge
Paint glanceable status on the **real browser window's Windows taskbar button** (the hwnd is captured at
`open_window`; an **Adom overlay badge** goes on automatically while driving - a trust tell). It lands on
**opened windows at open, and on adopted SPAWNED windows** when you call `nbrowser_owned` /
`nbrowser_list_windows` (that is when the bridge learns their hwnd). Driven via AD's `desktop_taskbar`.

**What the badge actually is (v0.1.12+):** Chrome/Edge use the taskbar overlay slot for the **profile
avatar**, so a bare Adom badge would evict it and the user loses "which profile is this?". The bridge
therefore builds a **composite**: the profile's own avatar **circle-cropped** (exactly how the browser
draws it) with the **Adom favicon in its lower-right corner** - one glance says both *whose profile* and
*Adom is driving it*. It works for Chrome AND Edge (each browser's own avatar file), is composited
natively on the user's machine, and is cached per profile.

**Badge lifecycle - you do NOT manage it:**
- Applied automatically at `open_window` and re-asserted by drive verbs - on EVERY addressing path:
  sessionId-keyed AND profile-keyed (`profile:` with no sessionId) both count as driving. It also
  survives bridge restarts (session-window mappings are persisted and restored).
- **Idle-expires after ~3 min** of no commands: the button repaints the **plain native avatar** (the
  overlay slot is NEVER blanked - blanking would erase the user's own avatar).
- `overlay:{badge:"none"}` = "restore native look" (safe); `overlay:{badge:"adom"}` = re-apply the
  composite (only needed if a badge went missing, e.g. immersive fullscreen hid the taskbar).
- On `close_window` / bridge restart, stale badges are swept by repainting native avatars - never by
  clearing slots.
- `nbrowser_flash {sessionId}` / `nbrowser_flash_stop` - orange "come look" cue (auto-clears on focus).
- `nbrowser_flash_mode {mode}` - **`quiet`** (default; auto-flash on come-look moments: page load,
  screenshot, an error/refusal, a human-gate, and freshly SPAWNED windows - flash MANUALLY with
  `nbrowser_flash` at task-done) · **`chatty`** (flash every action, for users who want to watch) ·
  **`off`** (NO auto-flash at all - the user opt-out; set this the moment the user says the flashing is
  annoying).
- `nbrowser_taskbar {progress:{state,value?}, overlay:{badge,tooltip?}}` - progress states:
  `normal`(green) · **`paused`(YELLOW - use for captcha/2FA/login human-gates)** · `error`(red) ·
  `indeterminate`(marquee) · `none`. Overlay `badge:"adom"|"none"` (see lifecycle above).

## No extension in the profile? You can still OPEN their real browser (`nbrowser_open_os_window`)
When a task only needs the user's real browser OPENED at a URL in the right profile - above all OAUTH
CONSENT (adom-google, Autodesk APS: the user is already signed in; they just click Approve and your
CLI's token poll picks up the result) - you do NOT need the extension installed:
`nbrowser_open_os_window {sessionId, thread, purpose, profile, url}` (registration required, like every
window). It opens **BACKGROUND by default** (v0.1.41+): it launches the window the same way Hydrogen
Desktop's browser picker does (a clean launch, no `--profile-directory=Default`), lets it fully build,
then minimizes it once with a gentle no-activate call - so it never steals focus and, importantly, never
crashes the browser (an earlier version minimized mid-construction in a tight loop and crashed the whole
Edge process - fixed). Its taskbar button flashes ORANGE so the user clicks it when THEY are ready. To
raise it later: `desktop_bring_to_front {hwnd}` (returned in the result) - only when the user asks to see
it. `foreground:true` needs `foregroundReason` + a `notify_user` first, same as everywhere else.
- You CANNOT read, drive, or screenshot that window (no extension there) - poll the outcome
  out-of-band or ask the user.
- Close it when the flow is done: `nbrowser_close_window {sessionId}` (handled bridge-side for these).
- Every response carries the honest limitation + the ABE upsell - relay it to the user at a natural
  moment ("with the extension installed I could have filled this for you"), never mid-auth.
- If the profile HAS the extension, prefer `nbrowser_wake_profile` + `nbrowser_open_window` - full
  control, guards, and cleanup beat a fire-and-forget open every time.

## File uploads (`nbrowser_set_file_input`) - never the OS picker
Upload into an `<input type=file>` over CDP - background, silent, no OS file dialog (which cannot be
driven with the debugger attached and would foreground). Three shapes:
- **Direct** (the default): `{sessionId, selector|text, filePaths:["C:/abs/laptop/path"]}` - the finder
  walks a Browse button/label to its associated input; HIDDEN inputs work (most real UIs hide them).
  `input` + `change` are dispatched for you so React/Angular notice.
- **Transient inputs** (created inside the Browse click handler - nothing exists to select):
  `{armForNextChooser:true, filePaths}` then `nbrowser_click` the Browse control - the page's file
  chooser is satisfied silently (30s auto-disarm). Confirm with `{chooserStatus:true}` or a screenshot.
- **Cloud-direct small files**: `files:[{name, contentBase64}]` - the bridge stages them in a janitored
  temp dir on the laptop and substitutes the paths. One call, no `send_files` round-trip. Use
  `adom-desktop send_files` + `filePaths` for big payloads.
Errors teach: `file_missing` (bridge stats every path first), `input_not_found` (with a page diagnosis),
`no_files`. **Hard limitation:** sites using the File System Access API (`showOpenFilePicker`) cannot be
driven over CDP at all - use the site's drag-drop zone manually. After injection, most UIs still need
their Submit/Upload button clicked.

## OS-level window inventory (`nbrowser_os_windows`) + extension-state diagnosis
`nbrowser_os_windows {}` returns every Chrome/Edge/pup window as the LAPTOP sees it (hwnd, title, rect,
z), annotated with `abeSession` (the live ABE session that owns it - NEVER force-close those) and
`kind` (`pup/chrome-for-testing` = browser_open/pup litter). It is measured bridge-side, so it works
when your own `desktop_list_windows` returns 0/null - usually AD mid-self-update-restart, or a wrong
`--target` when several ADs are connected. `ok:false` means "AD link down, data UNKNOWN", never "no
windows". Use it to find litter for the `nbrowser_force_close {hwnd}` gate, and to disambiguate
"separate windows vs tabs" without asking the user.

**No more not-connected mysteries (v0.1.26+):** when a verb is refused `extension not connected` for a
profile that exists locally, the refusal carries a DIAGNOSIS from the browser's own records: the Adom
extension is *not installed* in that profile (fix: installing-the-extension, Load unpacked into THAT
profile) vs *installed but disabled* (fix: enable the toggle) vs *enabled but not dialing* (fix: retry
`nbrowser_wake_profile`; if it never connects, `nbrowser_add_extension_origin {id}` for that browser's
extension id). `nbrowser_wake_profile`'s `woke:false` includes the same `extensionState`, and
`nbrowser_profiles` asleep rows carry `extensionInstalled` / `extensionEnabled` / `fix` - read them
BEFORE trying to wake, and never report "the profile doesn't exist" or "can't be fixed" when these
fields tell you exactly what to do.

## Activity log - "who drove what, when" (`nbrowser_activity`)
The bridge keeps a bounded in-memory trail (max 600 entries / ~1h TTL; resets on bridge restart) of every
forwarded verb: `{agoMs, verb, sessionId, thread, profile, ok, errorCode}` plus an `activeSessions`
summary (each with `lastAgoMs`). Use it to answer "why is that profile badged?", "what thread is driving
right now?", or to debug a stray window. Filter with `{sessionId, thread, profile, verb, errorsOnly,
sinceMs, limit}`. A session with a large `lastAgoMs` is idle - its badge has already idle-expired.

Play it like a status light: indeterminate green while working → yellow + flash on a human-gate → clear
+ one flash on done. **Respect the user's mode** (anxiety-driven users want `chatty`; trusting users want
`quiet`). Never leave it strobing.

## Do NOT foreground the window to look at it - capture in place
**The user is usually working IN their real browser. Stealing its focus disrupts them - this has been
called out directly ("that's disrupting my work massively").** The whole point of native-browser driving
is that the user keeps working in *their* foreground window while Adom operates one of their *other*
windows in the background. So:

- **Screenshots NEVER need foregrounding** - `nbrowser_screenshot` (CDP) captures a **background / occluded**
  window unconditionally (it reads the page bitmap, not the OS window). Capture where it sits; never raise it.
- **⭐ Recording in the BACKGROUND is the whole point - and it works, with one setup requirement.** The WGC
  recorder (and `desktop_screenshot_window`'s WGC path) capture the OS window's composited surface. Chrome,
  by default, **stops painting a fully-occluded window** (its occlusion detector), so WGC then records a
  **blank** surface. The fix is a one-time **launch-flag** requirement on the Chrome being recorded:
  `--disable-backgrounding-occluded-windows --disable-renderer-backgrounding --disable-background-timer-throttling --disable-features=CalculateNativeWinOcclusion`.
  **pup's Chrome already ships these** (which is why pup background-records reliably); a default user Chrome
  does **not** - so for the extension to record the user's real Chrome in the background, that Chrome must be
  launched with these flags (bake them into the Adom-managed Chrome launch / golden image). **Never** work
  around a blank background recording by foregrounding the window - that defeats the entire feature (the user
  must be able to keep working while a demo records). Foregrounding is the **user's** choice, when *they* want
  to watch - never the AI's.
- **`open_window` opens in the BACKGROUND** (focus is not stolen). Pass `focused:true` *only* when the
  user explicitly asked to watch it live.
- `nbrowser_focus` activates the right *tab* and flashes its taskbar button, but **by design does NOT
  raise the window** above other apps (Windows foreground-lock) - so it never yanks the user's focus.
  To say "come look" without forcing the window forward, use the taskbar **flash / overlay** above.
- **Only** raise a window when the user EXPLICITLY asks to see it live ("show me", "let me watch"):
```bash
adom-desktop -t AdomLapper desktop_bring_to_front '{"titleContains":"<page title>","state":"restore"}'  # or {hwnd}
```

## 🎥 Recording doctrine - full-framerate background recording of the NATIVE browser

**Goal:** demos must look **amazing** → full-resolution, **high framerate (20-60 fps)**. **The enemy is
PAINT-THROTTLING, not any codec.** Chrome correctly stops painting occluded / background / static windows
(its battery-saving default), and screencast-style capture is event-driven, so a throttled page silently
drops to ~5 fps ("frozen" / sped-up video). Every technique exists to keep frames flowing. *(The full
cross-surface reference - Hydrogen vs pup, the foreground-rebump trick, page-side animation, the keyframe
re-encode - lives in the **demo-recording** skill's `recording-mechanics.md`. CDP screencast is NOT garbage -
it's pup's `browser_record_start` engine and does 20-30 fps when the page paints.)*

**When to use the NATIVE browser (this extension) to record:** only when the demo needs the **user's real
session / cookies / logins** (a logged-in tool). If no login is needed, record via **Hydrogen or pup**
(see demo-recording) and don't touch the user's real Chrome.

**Native-browser recording = `nbrowser_record_start {method:"wgc"|"getdisplaymedia"}`** (the native path
does NOT use CDP screencast - that's pup's). Both are real high-fps engines: **`wgc`** (60 fps, Windows,
picker-free) and **`getdisplaymedia`** (WebRTC/Meet pipeline, 30-60 fps; picker auto-skipped with
`--auto-select-desktop-capture-source` when *we* launch Chrome).

**The key constraint that makes this different from pup:** you must **NOT foreground-rebump the user's real
working browser** (pup does that on a throwaway window, but on the user's real Chrome it wrecks their work).
Instead, make the real Chrome **paint while occluded** by launching it with the occlusion/throttle-disable
flags `--disable-features=CalculateNativeWinOcclusion --disable-backgrounding-occluded-windows
--disable-renderer-backgrounding --disable-background-timer-throttling`. Default Chrome lacks them (correctly
- battery). So for a logged-in background recording:
1. Check whether native Chrome already runs with the flags.
2. If not → **ASK + WARN the user we must fully restart their Chrome**, then **kill the entire `chrome.exe`
   process tree** (many helper/background processes - closing the window is NOT enough; the flags don't apply
   until every `chrome.exe` is gone), wait for full shutdown, and **relaunch with their real profile** (logins
   survive; tabs restore) **plus the flags**.
3. Record full-framerate **in the background** while they keep working.
4. **Later OFFER TO REVERT** (battery): "Chrome's in full-background recording mode - back to normal?" → if
   yes, fully restart Chrome again **without** the flags. Real session + amazing recordings + normal battery.

**Hard requirement under both transitions:** a robust **full-restart of native Chrome** (kill the whole tree,
confirm down, relaunch with profile + flags - and again to revert).

**⛔ NEVER:** foreground the user's *real* Chrome to fix framerate (use the flags) · leave the flags on
globally/permanently (battery drain) · upload a raw recording to the wiki without the `-g 30 -keyint_min 30`
keyframe re-encode (see recording-mechanics.md).

## Captions (two systems)
- **In-page** `nbrowser_caption {text, style, position}` - `style:"subtitle"` (calm white-on-dark, the
  HD/web-hydrogen look) or `"alert"` (red **pulsing** banner). Tied to a page (cleared on navigation).
- **Fullscreen** AD `desktop_caption {text, position:"bottom", id}` - Win32 always-on-top over every app,
  truly unmissable; auto-expires. (`position:"bottom"` - top covers the URL bar + the debugger banner.)

## ⭐ The killer use case: chip-fetcher through the native browser
EE CAD sources (**SnapEDA/SnapMagic, Ultralibrarian, Mouser, DigiKey**) gate the actual download behind
a **login** and often a **captcha**. `pup`'s empty controlled-Chromium hits the wall. The native browser
uses the engineer's **real logged-in session**, so the STEP / KiCad symbol+footprint / Altium / Fusion
`.lbr` bundles just download. Drive the chip-fetcher sourcing ladder (manufacturer-first → SnapMagic →
Mouser → DigiKey → … → LCSC last) with `nbrowser_*` instead of `browser_*` - same flow, real session, no
wall. This is what changes the EE workflow.

## Pitfalls + best practices
- **Confirm connected first** (`nbrowser_status`). After `dev_reload` OR an **AD restart**, the native
  host re-launches and re-dials the bridge - expect a reconnect delay (see RECONNECT below).
- **The extension dies if its profile has NO open windows** (MV3 SW sleeps) → `"not connected"`. Open a
  window in that profile; never close your LAST window there.
- **Always clean up windows you open** (`nbrowser_close_window`) - abandoned windows annoy the user, and
  closing clears that window's taskbar status.
- **Poll `nbrowser_events`** to learn if the USER closed your windows/tabs (`window_closed{byUser:true}`).
- **`nbrowser_eval` runs in the PAGE world** - `chrome.*` is NOT available there (only DOM/window).
- **`fetch_url` is the authed shortcut** - real cookies; grabs a file behind a login without clicking.
- **Run `nbrowser_help`** anytime - the bridge self-documents every verb + the flash/taskbar playbook.
- **Driving AD itself:** prefer the first-class file verbs (`read_file`/`write_file`/`list_dir`/
  `delete_file`, `process_list`, `net_stats`, `registry_*`) over `shell_execute` - cleaner, no quoting.
  (AD 1.8.165 fixed `shell_execute`'s quote-mangling, so quoted Windows paths with spaces now work too -
  the old `ADOMDE~1` 8.3-short-name workaround is no longer needed.)

## RECONNECT (the one operational gotcha)
Each bridge restart (deploying new bridge code) or AD restart drops the extension; the native host then
re-dials and reconnects (currently 30-60s - being fixed AD-side with a stable host port). If
`nbrowser_status` says "not connected" but you didn't change anything, AD probably restarted to deploy a
fix - wait, then retry. If it persists, check `process_list` for **stray bridge instances** (kill all but
one) and that `bridge_list` runtimePort matches the live bridge's port.

## Deploying a code change to the laptop
The `extension/` and `bridge/` here are **source** - they run from the *laptop*, so an edit takes effect
only after you push it there and reload:

- **Extension** (`src/*.js`, `manifest.json`) loads unpacked from
  `C:\Users\john\adom-browser-extension\extension`:
  1. `write_file` each changed file there (real quoted Windows paths work as of AD 1.8.165).
  2. `nbrowser_dev_reload` - **self-completing 2-pass**: it does a soft reload, then arms a one-shot flag
     so the SW does a second *hard* reload on restart that re-reads disk. A single call now reliably picks
     up your changes (older builds needed two manual calls because the soft reload served cached files).
     Throttled 1/90s. Verified: a fresh version loads in ~7s under clean conditions. **Gotcha:** if you
     ALSO restart the bridge in the same beat, the SW's reconnect churn can delay the 2nd pass - wait
     ~10s and don't read `nbrowser_status` until it settles (the version eventually lands; it's not stuck).
  3. Verify: `nbrowser_status` → `version` should reflect your `manifest.json` bump (bump it so you can tell).
- **Bridge** (`server.js`, `bridge.json`) runs from `…\bridges-cache\native-browser\` - the **single**
  runtime (`bridge_list` shows each bridge's `entrypoint`/`source`). `write_file` the changes there, then
  restart it; if it's a wiki-managed (`source:cache`) install, keep `BRIDGE_VERSION`/`.bridge-version` in
  sync with the deployed code so a future re-stream stays a no-op.

## Managing the extension's lifecycle / driving chrome:// pages → see **driving-the-desktop**
CDP/`nbrowser_*` **cannot** reach `chrome://`/`edge://` pages, native OS dialogs, browser chrome, or an
extension that **isn't loaded yet** - so the extension's own **install ("Load unpacked") / uninstall
("Remove") / reload**, plus anything on `edge://extensions`, is driven through AD's `desktop_*` verbs
instead. That whole playbook - the **zero-math image-space click** path (`coordMap` + `space:"image"`),
**background UI Automation** (`desktop_find_control` → `desktop_ui_click`/`desktop_ui_set`, no foreground),
**full-screen capture that SEES popup bubbles**, modifier chords, the ghost-cursor TTL, and the
proven uninstall→reinstall recipe (validated on John's Edge 2026-06-23, AD 1.8.175) - lives in the
**[driving-the-desktop](../driving-the-desktop/SKILL.md)** sub-skill. Reach for it whenever the user wants
to install/uninstall/reload the extension or you need to touch a surface CDP can't.

## Install / publish
Extension is loaded unpacked into the user's profile (Chrome 137+ removed `--load-extension`, so it's a
one-time `chrome://extensions → Developer mode → Load unpacked`; production = Chrome Web Store
force-install). Bridge: `bridge_install '{"manifestUrl":"…native-browser-bridge-manifest.json"}'` →
`bridges-cache`; native-messaging host registers in HKCU (Chrome + Edge); `persistent:true` keeps one
bridge alive on a stable port. See `install/`, `AD-REQUESTS.md`, `TASKBAR-FEATURE.md`.