NB (macOS) - Native Browser Bridge
Public Made by Adomby adom
Your real browser, driven by AI - Chrome, Edge and Safari on macOS.
name: installing-the-extension description: "How WE (native-browser-macos) install the Adom Native Browser Extension into the user's real Chrome/Edge on macOS in DEVELOPER MODE ("Load unpacked"), driving Bridge's GENERIC mac desktop verbs. This is OUR orchestration, not a Bridge feature — ab (bridge-macos) only exposes primitives (desktop_screenshot_window / desktop_list_windows / desktop_ocr_click / desktop_click / desktop_type / desktop_press_key / desktop_bring_to_front); the recipe, hardening, and recovery live here. This is the DAY-ONE install path: Chrome Web Store / Edge Add-ons review is months out, so until then dev-mode load-unpacked is how a real user gets the extension. macOS differs from the Windows sibling in EVERY UI primitive (OCR-click + CGEvent, not UIA + SendInput; an NSOpenPanel, not a #32770 dialog; ⌘⇧G, not Ctrl+L; a native-messaging host FILE, not an HKCU key). Use when the user says "install the extension", "install it for me", "load the extension into my browser", "set up the browser extension", "the extension isn't loaded yet". Part of the adom-native-browser (macOS)."
Part of adom-native-browser (macOS). Safety rules: native-browser-safety.
Installing the extension on macOS (dev-mode Load unpacked) — the orchestration WE own
Whose code is this? Ours. Bridge stays generic — on macOS it gives us OS primitives (
desktop_screenshot_window,desktop_list_windows,desktop_ocr_click,desktop_click,desktop_type,desktop_press_key,desktop_bring_to_front). The sequence that turns those into "the Adom extension is now loaded and connected" is this skill (plus the repo'sinstall/install.sh). ab never grows a "dev-mode install" verb. This is the day-one path until the Web Store / Add-ons listings clear review (budget months for the broad-permission set), so it has to be clean and repeatable.
⛔ macOS reality check — read this before you touch a UI verb
The Windows skill drives Win32 UIA (desktop_find_control/desktop_ui_click) and a #32770 common
dialog. None of that exists on macOS. What we actually have:
desktop_navigateis NOT implemented on macOS. You cannot drive a background window's address bar. Openchrome://extensionsby LAUNCHING the browser with the URL as an arg (below), not by navigating.- No UIA. To click a control you either OCR-click it by its visible text (
desktop_ocr_click {pid, want:["Load unpacked"]}) or compute its coordinates from a screenshot anddesktop_click {x,y}. - Synthetic input is CGEvent (
desktop_click/desktop_type/desktop_press_key), and it is silently ignored without the Accessibility grant — a call returns ok while nothing happens. Verify every synthetic action by screenshot, never by the call'sok:true. - Automation / System Events is DENIED on this machine and must stay denied (Kyle's standing rule —
see the
macos-no-system-eventsdoctrine). NEVERosascript -e 'tell application "System Events"…'and never reach for an AppleScript UI-scripting fallback. Screen Recording + Accessibility are the two grants this flow needs; both are drag-grantable. - There is no
desktop_captionon macOS. Narrate the user withnotify_usertoasts instead (see below), not an on-screen banner.
What makes this tractable: the extension ID is deterministic
The extension's manifest.json pins a key, so its ID is the same on every machine
(nncjgceobmlenfliamnibncjojpedbcc). The native-messaging host manifest's allowed_origins is known
ahead of time — install/install.sh already hardcodes it — so we do not have to capture the ID at
install time. The only genuinely hard part is getting "Load unpacked" to pick the right folder.
Preconditions (do these once, before driving any UI)
- The
extension/folder is on the laptop at a known absolute path — the repo already has it at~/adom/nb-bridge-macos/extension. Load-unpacked reads the on-disk build, so that path is what the NSOpenPanel must select. (Its Secure-Preferences path after loading is~/Library/Application Support/Google/Chrome/<Profile>/Secure Preferences→extensions.settings.<id>.path.) - The native-messaging host is registered. Run the repo installer — it writes the host manifest FILE
(macOS has no registry) and allow-lists the pinned ID:
This dropssh ~/adom/nb-bridge-macos/install/install.sh # Chrome sh ~/adom/nb-bridge-macos/install/install.sh --edge # also Edge~/Library/Application Support/<Google/Chrome | Microsoft Edge>/NativeMessagingHosts/inc.adom.native_browser.jsonpointing athost/native-host.sh. No admin prompt. - The bridge is running. The host talks to the bridge via
~/.adom/bridges/native-browser-macos/host.json, which only exists while the bridge runs. Confirmnbrowser_statusanswers (or ab shows the bridge green). - Grants: Screen Recording (for screenshots/OCR) and Accessibility (for CGEvent input) must be granted
to Adom Bridge.
desktop_screenshot_windowanddesktop_ocr_click/desktop_clickfail closed without them — a first run may need the user to grant both in System Settings → Privacy & Security. Do NOT try to script around a denial with System Events.
⚠ The host's Node — do NOT assume system Node exists (SDK runtime contract)
The bridge (server.js) is spawned by Bridge, so ab guarantees its Node. But the native-messaging host
is launched by CHROME, not ab, and a GUI-launched process on macOS gets launchd's THIN PATH
(/usr/bin:/bin:/usr/sbin:/sbin) — a Homebrew node at /opt/homebrew/bin/node is invisible to a bare
node. This is handled automatically:
- The bridge publishes the exact Node ab gave it (
process.execPath) to~/.adom/bridges/native-browser-macos/node-path.txtevery start. host/native-host.shreads that first, then probes absolute install dirs (Homebrew arm64 + Intel, nvm, MacPorts), and only falls back to PATHnodelast. So a machine with Homebrew node but a thin GUI PATH still starts the host. Never tell the user tobrew install node— reuse what ab has.
⭐ Verify by SCREENSHOT, not by control state (the rule for the whole flow)
OCR hints and ok:true from a CGEvent lie often enough to burn you: a toggle can read ambiguously, the
NSOpenPanel can stand in the wrong folder, and Load unpacked can "succeed" with a broken manifest (the
card shows a red Errors button). At every checkpoint below, take a real screenshot and READ it:
desktop_screenshot_window {hwnd:<W>} # BACKGROUND capture; works on the privileged chrome:// page
# (the extension's own CDP can't shoot it, and it isn't loaded yet)
Then look at the image: is Developer mode visibly ON? does the NSOpenPanel path field / breadcrumb read
…/extension? is the Adom card present, Enabled, with NO Errors button? Do not assert success off a
verb result alone. (Shooting a window also returns its owned sheets, so when the NSOpenPanel is open,
desktop_screenshot_window {hwnd:<browser>} hands back the page AND the panel in one call.)
⭐ Narrate + gate with notify_user (no desktop_caption on mac)
This runs on the USER's screen and the panel step needs them to not type for a second. Keep them oriented
with notify_user toasts, and use ONE as a start gate:
notify_user {id:"nbe-install", title:"Adom: install the browser extension?", body:"I'll load it into Chrome — ~30s, please don't type while the file panel is up.", buttons:[{label:"Start"}], scenario:"reminder"}
# poll notify_response {id:"nbe-install"} until they click Start, THEN drive.
Never ask them to type or paste the folder path — the agent types it.
⛔⛔ NEVER force the user's browser to the foreground for your own convenience
Foregrounding steals focus, so the user's next keystroke lands in whatever you just focused — e.g. the extensions Search box, which silently changes the filter and hides the card you were acting on. Drive a window you opened, and reach for the foreground only for the two steps that truly need it (the toggle click and the NSOpenPanel), briefly, after a "don't type" toast.
The flow
Step 0 — open chrome://extensions in a window YOU own (no desktop_navigate on mac)
nbrowser_open_os_window only accepts http(s), so launch the browser directly with the chrome:// URL as an
arg (Chrome opens it as a real page), then find that window by title:
# resolve the browser exe + the profile dir from nbrowser_profiles (key e.g. chrome:[email protected] → dir "Default")
ab desktop … launch_process {path:"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome",
args:["--profile-directory=Default","--new-window","chrome://extensions"]}
desktop_list_windows # find the new window: appBundleId com.google.Chrome, title "Extensions"; note its hwnd <W> and pid <PID>
desktop_screenshot_window {hwnd:<W>} # VERIFY the Extensions page rendered before doing anything else
(Edge: /Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge + edge://extensions; match the scheme.)
⚠ The just-launched window can briefly be a splash Chrome replaces — re-find the real "Extensions" window
via desktop_list_windows before driving, don't trust a stale hwnd.
Step A — ensure Developer mode is ON (verify VISUALLY)
On Chrome the Developer mode toggle is top-right; on Edge, bottom-left. It's web content (shadow DOM), so there's no UIA control — OCR-click it, or compute its knob coordinates from a screenshot:
desktop_screenshot_window {hwnd:<W>} # GROUND TRUTH: is the toggle ON or OFF?
# if OFF:
desktop_ocr_click {pid:<PID>, want:["Developer mode"]} # OCR finds the label; the toggle sits just right of it
desktop_screenshot_window {hwnd:<W>} # confirm it flipped AND "Load unpacked" appeared
If OCR clicks the label but the knob (a few px right of the text) didn't flip, compute the knob from the
screenshot: knobX_image * (windowRect.width / imageWidth) + windowRect.left, then
desktop_bring_to_front {hwnd:<W>} + desktop_click {x:<knobX>, y:<knobY>}. Do not proceed until the
screenshot shows Developer mode ON and Load unpacked visible.
Step B — Load unpacked → the NSOpenPanel (agent-driven, fast)
Run this as a TIGHT SCRIPT. ⛔ Never ask the user to type the path; ⛔ don't rely on the clipboard.
desktop_ocr_click {pid:<PID>, want:["Load unpacked"]} # opens the native NSOpenPanel (a sheet on the window)
desktop_screenshot_window {hwnd:<W>} # confirm the panel is up (it's an owned sheet in the shot)
desktop_bring_to_front {hwnd:<W>} # the panel is modal to the window; front it so CGEvent lands
desktop_press_key {keys:"cmd+shift+g"} # ⭐ THE KEY: open "Go to the folder" in the panel (mac's Ctrl+L)
desktop_type {text:"/Users/<user>/adom/nb-bridge-macos/extension"} # agent types the ABSOLUTE path
desktop_press_key {keys:"enter"} # 1st Enter: the Go-to-folder sheet navigates INTO the folder
desktop_screenshot_window {hwnd:<W>} # VERIFY the panel now stands in …/extension BEFORE committing
desktop_press_key {keys:"enter"} # 2nd Enter: the panel's default button (Open/Select) commits
desktop_screenshot_window {hwnd:<W>} # confirm the panel closed AND the page toasted "Extension loaded"
⚠ The subfolder trap. After ⌘⇧G → type → Enter you should be standing ON …/extension (the panel lists
its icons/, src/, manifest.json). Commit from HERE. If the path field reads …/extension/src, Open
loads src (no manifest → silent no-load or a manifest error). If you over-drilled, re-run ⌘⇧G + the full
path + Enter to reset, re-screenshot, then commit.
⚠ Modifier keys are CGEvent flags — desktop_press_key {keys:"cmd+shift+g"} (cmd = ⌘). If ⌘⇧G does
nothing in the panel, Accessibility is almost certainly not granted (CGEvents no-op silently): screenshot,
confirm nothing moved, and have the user grant Accessibility to Adom Bridge, then retry. Do NOT fall back to
System Events.
Step C — verify it loaded clean AND connected (screenshot FIRST, three checks)
- Screenshot the list and READ it — the check that matters: the Adom card is present, Enabled, and shows NO red Errors button. ⛔ A card WITH an Errors button loaded broken (bad manifest / SW threw) — it is NOT installed. OCR-click "Errors", screenshot, read it, fix, reload.
- Card present (secondary):
desktop_ocr_click-style read, or just confirm "Adom Native Browser" text in the screenshot. Optionally read the card's ID and confirm it equals the pinned ID. - Connected to the bridge: poll
nbrowser_status/nbrowser_profiles— thechrome:/edge:key appears (popup shows "Connected to Bridge") within ~1–3s once the service worker starts. Loaded-but-not- connected usually means the profile has no open window (SW asleep → open one, ornbrowser_wake_profile) or the host manifest is missing / has the wrong ID (re-runinstall.sh).
Step D — first useful action
Drive a real verb in a window YOU open (see native-browser-safety):
nbrowser_open_window {sessionId:"verify", thread:"<you>", purpose:"verify the extension drives", url:"https://example.com"} → nbrowser_screenshot.
Chrome AND Edge
Independent. Repeat per browser (scheme chrome:// vs edge://; toggle top-right on Chrome, bottom-left on
Edge). The host is registered per browser (install.sh --edge adds Edge), so each connects its own host.
Safari (nbe for Safari)
Safari cannot load the unpacked Chrome extension. It ships as a signed, notarized app that carries a Safari web extension. The user has to do the consent steps; you cannot click them for them (Safari requires the user's own decision, and never script Settings with System Events).
- Confirm the bridge side:
nbrowser_statuslistssafari:<profile>rows once Safari is connected. - If not: have the user install Adom NB for Safari.app (release asset on adom/nb-bridge-macos), open it once, then Safari > Settings > Extensions > tick Adom.
- Website access, PER SAFARI PROFILE: Edit Websites > Other websites: Allow (or Always Allow +
"Remember for other websites" on Safari's prompt). Until then Safari HIDES the extension's own windows,
tabs and cookies: a window you open looks closed a second later and fetches fail with "Load failed".
The replies say so (
SAFARI SITE ACCESShint). - Verify:
nbrowser_open_window {profile:"safari:<p>", sessionId, thread, purpose, url}thennbrowser_eval. What the Safari build can and cannot do is in the README ("Safari"): simulated input only, DOM-rendered screenshots while minimized (pixelExact:truefor exact), no saved-password login, no recording.
Updating the extension (the load-unpacked folder is what runs, not the page)
Load unpacked runs the FOLDER on disk. Updating = refreshing that folder and reloading:
- Read
nbrowser_readiness: noteextensionVersionsper profile andexpectedExtensionVersion. Two stale signals —extensionStale/staleProfiles(a CONNECTED profile runs an older build; the socket reports its version) andstaleOnDisk(the folder's manifest is old; this bridge computes it for connected profiles too). Match the extension againstexpectedExtensionVersion, not the pkg version. - Refresh the folder: overwrite
~/adom/nb-bridge-macos/extensionwith the current build (this repo IS the source of truth on this box; the drift watcher keeps it current with John's line). nbrowser_dev_reload {profile}PER PROFILE — a bare reload only touches the active profile.- Wait ~20s, re-read readiness: every profile in
extensionVersionsshows the new version. - A profile you WAKE later needs its own reload too, and readiness lies until then: a profile asleep
during the update starts its SW from a stale script cache — it announces the NEW manifest version while
still running OLD module code (no
windowEtiquetteon its replies).nbrowser_dev_reload {profile}on it clears that. A version number in readiness is not proof the code changed.
Recovery / common failures
- Load unpacked missing → Developer mode is off (Step A) or the Extensions page didn't render (re-launch the window with the chrome:// arg; verify by screenshot).
- ⌘⇧G / typing does nothing in the panel → Accessibility not granted (CGEvents no-op). Grant it; retry. Never substitute System Events.
- Panel selected the wrong folder → the silent killer; always screenshot the panel path before Open.
- Card present but never connects → SW asleep (open a window in that profile /
nbrowser_wake_profile), or host not registered / wrongallowed_originsID (re-runinstall.sh), orhost.jsonabsent because the bridge isn't running. - "refusing to drive a PWA / installed-app window" → you grabbed a
com.google.Chrome.app.*window; open a fresh normal window instead.
Honest status
The deterministic half (host registration via install.sh, folder staging) is solid. The GUI half is the
frontier on macOS: OCR-clicking web content in chrome://extensions and filling the NSOpenPanel via ⌘⇧G +
CGEvent are the parts to harden, and both depend on the Accessibility grant. Screenshot-verify every step,
narrate with notify_user, and if the automation stalls, fall back to guiding the USER through the exact
clicks (Developer mode → Load unpacked → ⌘⇧G, paste ~/adom/nb-bridge-macos/extension, Open) rather than
reaching for System Events. Report any panel/OCR flakiness so we can harden the helper.
---
name: installing-the-extension
description: "How WE (native-browser-macos) install the Adom Native Browser Extension into the user's real Chrome/Edge on macOS in DEVELOPER MODE (\"Load unpacked\"), driving Bridge's GENERIC mac desktop verbs. This is OUR orchestration, not a Bridge feature — ab (bridge-macos) only exposes primitives (desktop_screenshot_window / desktop_list_windows / desktop_ocr_click / desktop_click / desktop_type / desktop_press_key / desktop_bring_to_front); the recipe, hardening, and recovery live here. This is the DAY-ONE install path: Chrome Web Store / Edge Add-ons review is months out, so until then dev-mode load-unpacked is how a real user gets the extension. macOS differs from the Windows sibling in EVERY UI primitive (OCR-click + CGEvent, not UIA + SendInput; an NSOpenPanel, not a #32770 dialog; ⌘⇧G, not Ctrl+L; a native-messaging host FILE, not an HKCU key). Use when the user says \"install the extension\", \"install it for me\", \"load the extension into my browser\", \"set up the browser extension\", \"the extension isn't loaded yet\". Part of the adom-native-browser (macOS)."
---
Part of **adom-native-browser (macOS)**. Safety rules: **native-browser-safety**.
# Installing the extension on macOS (dev-mode Load unpacked) — the orchestration WE own
> **Whose code is this?** Ours. Bridge stays **generic** — on macOS it gives us OS primitives
> (`desktop_screenshot_window`, `desktop_list_windows`, `desktop_ocr_click`, `desktop_click`,
> `desktop_type`, `desktop_press_key`, `desktop_bring_to_front`). The sequence that turns those into
> "the Adom extension is now loaded and connected" is **this skill** (plus the repo's `install/install.sh`).
> ab never grows a "dev-mode install" verb. **This is the day-one path** until the Web Store / Add-ons
> listings clear review (budget months for the broad-permission set), so it has to be clean and repeatable.
## ⛔ macOS reality check — read this before you touch a UI verb
The Windows skill drives Win32 UIA (`desktop_find_control`/`desktop_ui_click`) and a `#32770` common
dialog. **None of that exists on macOS.** What we actually have:
- **`desktop_navigate` is NOT implemented on macOS.** You cannot drive a background window's address bar.
Open `chrome://extensions` by LAUNCHING the browser with the URL as an arg (below), not by navigating.
- **No UIA.** To click a control you either **OCR-click it by its visible text** (`desktop_ocr_click
{pid, want:["Load unpacked"]}`) or **compute its coordinates from a screenshot** and `desktop_click {x,y}`.
- **Synthetic input is CGEvent** (`desktop_click` / `desktop_type` / `desktop_press_key`), and it is
**silently ignored without the Accessibility grant** — a call returns ok while nothing happens. Verify
every synthetic action by screenshot, never by the call's `ok:true`.
- **Automation / System Events is DENIED on this machine and must stay denied** (Kyle's standing rule —
see the `macos-no-system-events` doctrine). NEVER `osascript -e 'tell application "System Events"…'`
and never reach for an AppleScript UI-scripting fallback. Screen Recording + Accessibility are the two
grants this flow needs; both are drag-grantable.
- **There is no `desktop_caption` on macOS.** Narrate the user with `notify_user` toasts instead
(see below), not an on-screen banner.
## What makes this tractable: the extension ID is deterministic
The extension's `manifest.json` pins a `key`, so its ID is the **same on every machine**
(`nncjgceobmlenfliamnibncjojpedbcc`). The native-messaging **host manifest's `allowed_origins` is known
ahead of time** — `install/install.sh` already hardcodes it — so we do **not** have to capture the ID at
install time. The only genuinely hard part is getting "Load unpacked" to pick the right folder.
## Preconditions (do these once, before driving any UI)
1. **The `extension/` folder is on the laptop** at a known absolute path — the repo already has it at
`~/adom/nb-bridge-macos/extension`. Load-unpacked reads the **on-disk** build, so that path is what the
NSOpenPanel must select. (Its Secure-Preferences path after loading is
`~/Library/Application Support/Google/Chrome/<Profile>/Secure Preferences` → `extensions.settings.<id>.path`.)
2. **The native-messaging host is registered.** Run the repo installer — it writes the host manifest FILE
(macOS has no registry) and allow-lists the pinned ID:
```
sh ~/adom/nb-bridge-macos/install/install.sh # Chrome
sh ~/adom/nb-bridge-macos/install/install.sh --edge # also Edge
```
This drops `~/Library/Application Support/<Google/Chrome | Microsoft Edge>/NativeMessagingHosts/inc.adom.native_browser.json`
pointing at `host/native-host.sh`. No admin prompt.
3. **The bridge is running.** The host talks to the bridge via `~/.adom/bridges/native-browser-macos/host.json`,
which only exists while the bridge runs. Confirm `nbrowser_status` answers (or ab shows the bridge green).
4. **Grants:** Screen Recording (for screenshots/OCR) and Accessibility (for CGEvent input) must be granted
to Adom Bridge. `desktop_screenshot_window` and `desktop_ocr_click`/`desktop_click` fail closed without
them — a first run may need the user to grant both in System Settings → Privacy & Security. Do NOT try to
script around a denial with System Events.
### ⚠ The host's Node — do NOT assume system Node exists (SDK runtime contract)
The bridge (`server.js`) is spawned by Bridge, so ab guarantees its Node. **But the native-messaging host
is launched by CHROME, not ab**, and a GUI-launched process on macOS gets launchd's THIN PATH
(`/usr/bin:/bin:/usr/sbin:/sbin`) — a Homebrew node at `/opt/homebrew/bin/node` is invisible to a bare
`node`. This is handled automatically:
- The bridge publishes the exact Node ab gave it (`process.execPath`) to
`~/.adom/bridges/native-browser-macos/node-path.txt` every start.
- `host/native-host.sh` reads that first, then probes absolute install dirs (Homebrew arm64 + Intel, nvm,
MacPorts), and only falls back to PATH `node` last. So a machine with Homebrew node but a thin GUI PATH
still starts the host. Never tell the user to `brew install node` — reuse what ab has.
## ⭐ Verify by SCREENSHOT, not by control state (the rule for the whole flow)
OCR hints and `ok:true` from a CGEvent lie often enough to burn you: a toggle can read ambiguously, the
NSOpenPanel can stand in the wrong folder, and **Load unpacked can "succeed" with a broken manifest** (the
card shows a red **Errors** button). At **every** checkpoint below, take a real screenshot and READ it:
```
desktop_screenshot_window {hwnd:<W>} # BACKGROUND capture; works on the privileged chrome:// page
# (the extension's own CDP can't shoot it, and it isn't loaded yet)
```
Then look at the image: is Developer mode visibly ON? does the NSOpenPanel path field / breadcrumb read
`…/extension`? is the Adom card present, **Enabled, with NO Errors button**? Do not assert success off a
verb result alone. (Shooting a window also returns its owned sheets, so when the NSOpenPanel is open,
`desktop_screenshot_window {hwnd:<browser>}` hands back the page AND the panel in one call.)
## ⭐ Narrate + gate with notify_user (no desktop_caption on mac)
This runs on the USER's screen and the panel step needs them to not type for a second. Keep them oriented
with `notify_user` toasts, and use ONE as a start gate:
```
notify_user {id:"nbe-install", title:"Adom: install the browser extension?", body:"I'll load it into Chrome — ~30s, please don't type while the file panel is up.", buttons:[{label:"Start"}], scenario:"reminder"}
# poll notify_response {id:"nbe-install"} until they click Start, THEN drive.
```
Never ask them to type or paste the folder path — the agent types it.
## ⛔⛔ NEVER force the user's browser to the foreground for your own convenience
Foregrounding steals focus, so **the user's next keystroke lands in whatever you just focused** — e.g. the
extensions **Search box**, which silently changes the filter and hides the card you were acting on. Drive a
window **you** opened, and reach for the foreground only for the two steps that truly need it (the toggle
click and the NSOpenPanel), briefly, after a "don't type" toast.
## The flow
### Step 0 — open chrome://extensions in a window YOU own (no desktop_navigate on mac)
`nbrowser_open_os_window` only accepts http(s), so launch the browser directly with the chrome:// URL as an
arg (Chrome opens it as a real page), then find that window by title:
```
# resolve the browser exe + the profile dir from nbrowser_profiles (key e.g. chrome:[email protected] → dir "Default")
ab desktop … launch_process {path:"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome",
args:["--profile-directory=Default","--new-window","chrome://extensions"]}
desktop_list_windows # find the new window: appBundleId com.google.Chrome, title "Extensions"; note its hwnd <W> and pid <PID>
desktop_screenshot_window {hwnd:<W>} # VERIFY the Extensions page rendered before doing anything else
```
(Edge: `/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge` + `edge://extensions`; match the scheme.)
⚠ The just-launched window can briefly be a splash Chrome replaces — re-find the real "Extensions" window
via `desktop_list_windows` before driving, don't trust a stale hwnd.
### Step A — ensure Developer mode is ON (verify VISUALLY)
On Chrome the **Developer mode** toggle is top-right; on Edge, bottom-left. It's web content (shadow DOM),
so there's no UIA control — OCR-click it, or compute its knob coordinates from a screenshot:
```
desktop_screenshot_window {hwnd:<W>} # GROUND TRUTH: is the toggle ON or OFF?
# if OFF:
desktop_ocr_click {pid:<PID>, want:["Developer mode"]} # OCR finds the label; the toggle sits just right of it
desktop_screenshot_window {hwnd:<W>} # confirm it flipped AND "Load unpacked" appeared
```
If OCR clicks the label but the knob (a few px right of the text) didn't flip, compute the knob from the
screenshot: `knobX_image * (windowRect.width / imageWidth) + windowRect.left`, then
`desktop_bring_to_front {hwnd:<W>}` + `desktop_click {x:<knobX>, y:<knobY>}`. Do not proceed until the
screenshot shows Developer mode ON and **Load unpacked** visible.
### Step B — Load unpacked → the NSOpenPanel (agent-driven, fast)
Run this as a TIGHT SCRIPT. ⛔ Never ask the user to type the path; ⛔ don't rely on the clipboard.
```
desktop_ocr_click {pid:<PID>, want:["Load unpacked"]} # opens the native NSOpenPanel (a sheet on the window)
desktop_screenshot_window {hwnd:<W>} # confirm the panel is up (it's an owned sheet in the shot)
desktop_bring_to_front {hwnd:<W>} # the panel is modal to the window; front it so CGEvent lands
desktop_press_key {keys:"cmd+shift+g"} # ⭐ THE KEY: open "Go to the folder" in the panel (mac's Ctrl+L)
desktop_type {text:"/Users/<user>/adom/nb-bridge-macos/extension"} # agent types the ABSOLUTE path
desktop_press_key {keys:"enter"} # 1st Enter: the Go-to-folder sheet navigates INTO the folder
desktop_screenshot_window {hwnd:<W>} # VERIFY the panel now stands in …/extension BEFORE committing
desktop_press_key {keys:"enter"} # 2nd Enter: the panel's default button (Open/Select) commits
desktop_screenshot_window {hwnd:<W>} # confirm the panel closed AND the page toasted "Extension loaded"
```
⚠ **The subfolder trap.** After ⌘⇧G → type → Enter you should be standing ON `…/extension` (the panel lists
its `icons/`, `src/`, `manifest.json`). Commit from HERE. If the path field reads `…/extension/src`, Open
loads `src` (no manifest → silent no-load or a manifest error). If you over-drilled, re-run ⌘⇧G + the full
path + Enter to reset, re-screenshot, then commit.
⚠ **Modifier keys are CGEvent flags** — `desktop_press_key {keys:"cmd+shift+g"}` (cmd = ⌘). If ⌘⇧G does
nothing in the panel, Accessibility is almost certainly not granted (CGEvents no-op silently): screenshot,
confirm nothing moved, and have the user grant Accessibility to Adom Bridge, then retry. Do NOT fall back to
System Events.
### Step C — verify it loaded clean AND connected (screenshot FIRST, three checks)
1. **Screenshot the list and READ it** — the check that matters: the **Adom** card is present, **Enabled**,
and shows **NO red Errors button**. ⛔ A card WITH an Errors button loaded broken (bad manifest / SW
threw) — it is NOT installed. OCR-click "Errors", screenshot, read it, fix, reload.
2. **Card present (secondary):** `desktop_ocr_click`-style read, or just confirm "Adom Native Browser" text
in the screenshot. Optionally read the card's ID and confirm it equals the pinned ID.
3. **Connected to the bridge:** poll `nbrowser_status` / `nbrowser_profiles` — the `chrome:`/`edge:` key
appears (popup shows "Connected to Bridge") within ~1–3s once the service worker starts. Loaded-but-not-
connected usually means the profile has no open window (SW asleep → open one, or `nbrowser_wake_profile`)
or the host manifest is missing / has the wrong ID (re-run `install.sh`).
### Step D — first useful action
Drive a real verb in a window YOU open (see `native-browser-safety`):
`nbrowser_open_window {sessionId:"verify", thread:"<you>", purpose:"verify the extension drives", url:"https://example.com"}` → `nbrowser_screenshot`.
## Chrome AND Edge
Independent. Repeat per browser (scheme `chrome://` vs `edge://`; toggle top-right on Chrome, bottom-left on
Edge). The host is registered per browser (`install.sh --edge` adds Edge), so each connects its own host.
## Safari (nbe for Safari)
Safari cannot load the unpacked Chrome extension. It ships as a signed, notarized app that carries a Safari
web extension. The user has to do the consent steps; you cannot click them for them (Safari requires the
user's own decision, and never script Settings with System Events).
1. Confirm the bridge side: `nbrowser_status` lists `safari:<profile>` rows once Safari is connected.
2. If not: have the user install **Adom NB for Safari.app** (release asset on adom/nb-bridge-macos), open it
once, then Safari > Settings > Extensions > tick **Adom**.
3. Website access, PER SAFARI PROFILE: Edit Websites > Other websites: **Allow** (or **Always Allow** +
"Remember for other websites" on Safari's prompt). Until then Safari HIDES the extension's own windows,
tabs and cookies: a window you open looks closed a second later and fetches fail with "Load failed".
The replies say so (`SAFARI SITE ACCESS` hint).
4. Verify: `nbrowser_open_window {profile:"safari:<p>", sessionId, thread, purpose, url}` then `nbrowser_eval`.
What the Safari build can and cannot do is in the README ("Safari"): simulated input only, DOM-rendered
screenshots while minimized (`pixelExact:true` for exact), no saved-password login, no recording.
## Updating the extension (the load-unpacked folder is what runs, not the page)
Load unpacked runs the FOLDER on disk. Updating = refreshing that folder and reloading:
1. Read `nbrowser_readiness`: note `extensionVersions` per profile and `expectedExtensionVersion`. Two stale
signals — `extensionStale`/`staleProfiles` (a CONNECTED profile runs an older build; the socket reports
its version) and `staleOnDisk` (the folder's manifest is old; this bridge computes it for connected
profiles too). Match the extension against `expectedExtensionVersion`, not the pkg version.
2. Refresh the folder: overwrite `~/adom/nb-bridge-macos/extension` with the current build (this repo IS the
source of truth on this box; the drift watcher keeps it current with John's line).
3. `nbrowser_dev_reload {profile}` PER PROFILE — a bare reload only touches the active profile.
4. Wait ~20s, re-read readiness: every profile in `extensionVersions` shows the new version.
5. **A profile you WAKE later needs its own reload too, and readiness lies until then**: a profile asleep
during the update starts its SW from a stale script cache — it announces the NEW manifest version while
still running OLD module code (no `windowEtiquette` on its replies). `nbrowser_dev_reload {profile}` on
it clears that. A version number in readiness is not proof the code changed.
## Recovery / common failures
- **Load unpacked missing** → Developer mode is off (Step A) or the Extensions page didn't render (re-launch
the window with the chrome:// arg; verify by screenshot).
- **⌘⇧G / typing does nothing in the panel** → Accessibility not granted (CGEvents no-op). Grant it; retry.
Never substitute System Events.
- **Panel selected the wrong folder** → the silent killer; always screenshot the panel path before Open.
- **Card present but never connects** → SW asleep (open a window in that profile / `nbrowser_wake_profile`),
or host not registered / wrong `allowed_origins` ID (re-run `install.sh`), or `host.json` absent because
the bridge isn't running.
- **"refusing to drive a PWA / installed-app window"** → you grabbed a `com.google.Chrome.app.*` window;
open a fresh normal window instead.
## Honest status
The deterministic half (host registration via `install.sh`, folder staging) is solid. The **GUI half is the
frontier on macOS**: OCR-clicking web content in `chrome://extensions` and filling the NSOpenPanel via ⌘⇧G +
CGEvent are the parts to harden, and both depend on the Accessibility grant. Screenshot-verify every step,
narrate with `notify_user`, and if the automation stalls, fall back to guiding the USER through the exact
clicks (Developer mode → Load unpacked → ⌘⇧G, paste `~/adom/nb-bridge-macos/extension`, Open) rather than
reaching for System Events. Report any panel/OCR flakiness so we can harden the helper.