main
Kyle Bergstedt Publish 0.16.42 eef4c9f 1d ago

name: installing-the-extension description: "How WE (native-browser-macos) install the Adom Native Browser Extension into the user's real Chrome/Edge on macOS in DEVELOPER MODE ("Load unpacked"), driving Bridge's GENERIC mac desktop verbs. This is OUR orchestration, not a Bridge feature — ab (bridge-macos) only exposes primitives (desktop_screenshot_window / desktop_list_windows / desktop_ocr_click / desktop_click / desktop_type / desktop_press_key / desktop_bring_to_front); the recipe, hardening, and recovery live here. This is the DAY-ONE install path: Chrome Web Store / Edge Add-ons review is months out, so until then dev-mode load-unpacked is how a real user gets the extension. macOS differs from the Windows sibling in EVERY UI primitive (OCR-click + CGEvent, not UIA + SendInput; an NSOpenPanel, not a #32770 dialog; ⌘⇧G, not Ctrl+L; a native-messaging host FILE, not an HKCU key). Use when the user says "install the extension", "install it for me", "load the extension into my browser", "set up the browser extension", "the extension isn't loaded yet". Part of the adom-native-browser (macOS)."

Part of adom-native-browser (macOS). Safety rules: native-browser-safety.

Installing the extension on macOS (dev-mode Load unpacked) — the orchestration WE own

Whose code is this? Ours. Bridge stays generic — on macOS it gives us OS primitives (desktop_screenshot_window, desktop_list_windows, desktop_ocr_click, desktop_click, desktop_type, desktop_press_key, desktop_bring_to_front). The sequence that turns those into "the Adom extension is now loaded and connected" is this skill (plus the repo's install/install.sh). ab never grows a "dev-mode install" verb. This is the day-one path until the Web Store / Add-ons listings clear review (budget months for the broad-permission set), so it has to be clean and repeatable.

⛔ macOS reality check — read this before you touch a UI verb

The Windows skill drives Win32 UIA (desktop_find_control/desktop_ui_click) and a #32770 common dialog. None of that exists on macOS. What we actually have:

  • desktop_navigate is NOT implemented on macOS. You cannot drive a background window's address bar. Open chrome://extensions by LAUNCHING the browser with the URL as an arg (below), not by navigating.
  • No UIA. To click a control you either OCR-click it by its visible text (desktop_ocr_click {pid, want:["Load unpacked"]}) or compute its coordinates from a screenshot and desktop_click {x,y}.
  • Synthetic input is CGEvent (desktop_click / desktop_type / desktop_press_key), and it is silently ignored without the Accessibility grant — a call returns ok while nothing happens. Verify every synthetic action by screenshot, never by the call's ok:true.
  • Automation / System Events is DENIED on this machine and must stay denied (Kyle's standing rule — see the macos-no-system-events doctrine). NEVER osascript -e 'tell application "System Events"…' and never reach for an AppleScript UI-scripting fallback. Screen Recording + Accessibility are the two grants this flow needs; both are drag-grantable.
  • There is no desktop_caption on macOS. Narrate the user with notify_user toasts instead (see below), not an on-screen banner.

What makes this tractable: the extension ID is deterministic

The extension's manifest.json pins a key, so its ID is the same on every machine (nncjgceobmlenfliamnibncjojpedbcc). The native-messaging host manifest's allowed_origins is known ahead of time — install/install.sh already hardcodes it — so we do not have to capture the ID at install time. The only genuinely hard part is getting "Load unpacked" to pick the right folder.

Preconditions (do these once, before driving any UI)

  1. The extension/ folder is on the laptop at a known absolute path — the repo already has it at ~/adom/nb-bridge-macos/extension. Load-unpacked reads the on-disk build, so that path is what the NSOpenPanel must select. (Its Secure-Preferences path after loading is ~/Library/Application Support/Google/Chrome/<Profile>/Secure Preferences → extensions.settings.<id>.path.)
  2. The native-messaging host is registered. Run the repo installer — it writes the host manifest FILE (macOS has no registry) and allow-lists the pinned ID:
    sh ~/adom/nb-bridge-macos/install/install.sh          # Chrome
    sh ~/adom/nb-bridge-macos/install/install.sh --edge    # also Edge
    
    This drops ~/Library/Application Support/<Google/Chrome | Microsoft Edge>/NativeMessagingHosts/inc.adom.native_browser.json pointing at host/native-host.sh. No admin prompt.
  3. The bridge is running. The host talks to the bridge via ~/.adom/bridges/native-browser-macos/host.json, which only exists while the bridge runs. Confirm nbrowser_status answers (or ab shows the bridge green).
  4. Grants: Screen Recording (for screenshots/OCR) and Accessibility (for CGEvent input) must be granted to Adom Bridge. desktop_screenshot_window and desktop_ocr_click/desktop_click fail closed without them — a first run may need the user to grant both in System Settings → Privacy & Security. Do NOT try to script around a denial with System Events.

⚠ The host's Node — do NOT assume system Node exists (SDK runtime contract)

The bridge (server.js) is spawned by Bridge, so ab guarantees its Node. But the native-messaging host is launched by CHROME, not ab, and a GUI-launched process on macOS gets launchd's THIN PATH (/usr/bin:/bin:/usr/sbin:/sbin) — a Homebrew node at /opt/homebrew/bin/node is invisible to a bare node. This is handled automatically:

  • The bridge publishes the exact Node ab gave it (process.execPath) to ~/.adom/bridges/native-browser-macos/node-path.txt every start.
  • host/native-host.sh reads that first, then probes absolute install dirs (Homebrew arm64 + Intel, nvm, MacPorts), and only falls back to PATH node last. So a machine with Homebrew node but a thin GUI PATH still starts the host. Never tell the user to brew install node — reuse what ab has.

⭐ Verify by SCREENSHOT, not by control state (the rule for the whole flow)

OCR hints and ok:true from a CGEvent lie often enough to burn you: a toggle can read ambiguously, the NSOpenPanel can stand in the wrong folder, and Load unpacked can "succeed" with a broken manifest (the card shows a red Errors button). At every checkpoint below, take a real screenshot and READ it:

desktop_screenshot_window {hwnd:<W>}     # BACKGROUND capture; works on the privileged chrome:// page
                                         # (the extension's own CDP can't shoot it, and it isn't loaded yet)

Then look at the image: is Developer mode visibly ON? does the NSOpenPanel path field / breadcrumb read …/extension? is the Adom card present, Enabled, with NO Errors button? Do not assert success off a verb result alone. (Shooting a window also returns its owned sheets, so when the NSOpenPanel is open, desktop_screenshot_window {hwnd:<browser>} hands back the page AND the panel in one call.)

⭐ Narrate + gate with notify_user (no desktop_caption on mac)

This runs on the USER's screen and the panel step needs them to not type for a second. Keep them oriented with notify_user toasts, and use ONE as a start gate:

notify_user {id:"nbe-install", title:"Adom: install the browser extension?", body:"I'll load it into Chrome — ~30s, please don't type while the file panel is up.", buttons:[{label:"Start"}], scenario:"reminder"}
# poll notify_response {id:"nbe-install"} until they click Start, THEN drive.

Never ask them to type or paste the folder path — the agent types it.

⛔⛔ NEVER force the user's browser to the foreground for your own convenience

Foregrounding steals focus, so the user's next keystroke lands in whatever you just focused — e.g. the extensions Search box, which silently changes the filter and hides the card you were acting on. Drive a window you opened, and reach for the foreground only for the two steps that truly need it (the toggle click and the NSOpenPanel), briefly, after a "don't type" toast.

The flow

Step 0 — open chrome://extensions in a window YOU own (no desktop_navigate on mac)

nbrowser_open_os_window only accepts http(s), so launch the browser directly with the chrome:// URL as an arg (Chrome opens it as a real page), then find that window by title:

# resolve the browser exe + the profile dir from nbrowser_profiles (key e.g. chrome:[email protected] → dir "Default")
ab desktop … launch_process {path:"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome",
                             args:["--profile-directory=Default","--new-window","chrome://extensions"]}
desktop_list_windows                      # find the new window: appBundleId com.google.Chrome, title "Extensions"; note its hwnd <W> and pid <PID>
desktop_screenshot_window {hwnd:<W>}      # VERIFY the Extensions page rendered before doing anything else

(Edge: /Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge + edge://extensions; match the scheme.) ⚠ The just-launched window can briefly be a splash Chrome replaces — re-find the real "Extensions" window via desktop_list_windows before driving, don't trust a stale hwnd.

Step A — ensure Developer mode is ON (verify VISUALLY)

On Chrome the Developer mode toggle is top-right; on Edge, bottom-left. It's web content (shadow DOM), so there's no UIA control — OCR-click it, or compute its knob coordinates from a screenshot:

desktop_screenshot_window {hwnd:<W>}                         # GROUND TRUTH: is the toggle ON or OFF?
# if OFF:
desktop_ocr_click {pid:<PID>, want:["Developer mode"]}       # OCR finds the label; the toggle sits just right of it
desktop_screenshot_window {hwnd:<W>}                         # confirm it flipped AND "Load unpacked" appeared

If OCR clicks the label but the knob (a few px right of the text) didn't flip, compute the knob from the screenshot: knobX_image * (windowRect.width / imageWidth) + windowRect.left, then desktop_bring_to_front {hwnd:<W>} + desktop_click {x:<knobX>, y:<knobY>}. Do not proceed until the screenshot shows Developer mode ON and Load unpacked visible.

Step B — Load unpacked → the NSOpenPanel (agent-driven, fast)

Run this as a TIGHT SCRIPT. ⛔ Never ask the user to type the path; ⛔ don't rely on the clipboard.

desktop_ocr_click {pid:<PID>, want:["Load unpacked"]}        # opens the native NSOpenPanel (a sheet on the window)
desktop_screenshot_window {hwnd:<W>}                         # confirm the panel is up (it's an owned sheet in the shot)
desktop_bring_to_front {hwnd:<W>}                            # the panel is modal to the window; front it so CGEvent lands
desktop_press_key {keys:"cmd+shift+g"}                       # ⭐ THE KEY: open "Go to the folder" in the panel (mac's Ctrl+L)
desktop_type      {text:"/Users/<user>/adom/nb-bridge-macos/extension"}   # agent types the ABSOLUTE path
desktop_press_key {keys:"enter"}                             # 1st Enter: the Go-to-folder sheet navigates INTO the folder
desktop_screenshot_window {hwnd:<W>}                         # VERIFY the panel now stands in …/extension BEFORE committing
desktop_press_key {keys:"enter"}                             # 2nd Enter: the panel's default button (Open/Select) commits
desktop_screenshot_window {hwnd:<W>}                         # confirm the panel closed AND the page toasted "Extension loaded"

⚠ The subfolder trap. After ⌘⇧G → type → Enter you should be standing ON …/extension (the panel lists its icons/, src/, manifest.json). Commit from HERE. If the path field reads …/extension/src, Open loads src (no manifest → silent no-load or a manifest error). If you over-drilled, re-run ⌘⇧G + the full path + Enter to reset, re-screenshot, then commit. ⚠ Modifier keys are CGEvent flags — desktop_press_key {keys:"cmd+shift+g"} (cmd = ⌘). If ⌘⇧G does nothing in the panel, Accessibility is almost certainly not granted (CGEvents no-op silently): screenshot, confirm nothing moved, and have the user grant Accessibility to Adom Bridge, then retry. Do NOT fall back to System Events.

Step C — verify it loaded clean AND connected (screenshot FIRST, three checks)

  1. Screenshot the list and READ it — the check that matters: the Adom card is present, Enabled, and shows NO red Errors button. ⛔ A card WITH an Errors button loaded broken (bad manifest / SW threw) — it is NOT installed. OCR-click "Errors", screenshot, read it, fix, reload.
  2. Card present (secondary): desktop_ocr_click-style read, or just confirm "Adom Native Browser" text in the screenshot. Optionally read the card's ID and confirm it equals the pinned ID.
  3. Connected to the bridge: poll nbrowser_status / nbrowser_profiles — the chrome:/edge: key appears (popup shows "Connected to Bridge") within ~1–3s once the service worker starts. Loaded-but-not- connected usually means the profile has no open window (SW asleep → open one, or nbrowser_wake_profile) or the host manifest is missing / has the wrong ID (re-run install.sh).

Step D — first useful action

Drive a real verb in a window YOU open (see native-browser-safety): nbrowser_open_window {sessionId:"verify", thread:"<you>", purpose:"verify the extension drives", url:"https://example.com"} → nbrowser_screenshot.

Chrome AND Edge

Independent. Repeat per browser (scheme chrome:// vs edge://; toggle top-right on Chrome, bottom-left on Edge). The host is registered per browser (install.sh --edge adds Edge), so each connects its own host.

Safari (nbe for Safari)

Safari cannot load the unpacked Chrome extension. It ships as a signed, notarized app that carries a Safari web extension. The user has to do the consent steps; you cannot click them for them (Safari requires the user's own decision, and never script Settings with System Events).

  1. Confirm the bridge side: nbrowser_status lists safari:<profile> rows once Safari is connected.
  2. If not: have the user install Adom NB for Safari.app (release asset on adom/nb-bridge-macos), open it once, then Safari > Settings > Extensions > tick Adom.
  3. Website access, PER SAFARI PROFILE: Edit Websites > Other websites: Allow (or Always Allow + "Remember for other websites" on Safari's prompt). Until then Safari HIDES the extension's own windows, tabs and cookies: a window you open looks closed a second later and fetches fail with "Load failed". The replies say so (SAFARI SITE ACCESS hint).
  4. Verify: nbrowser_open_window {profile:"safari:<p>", sessionId, thread, purpose, url} then nbrowser_eval. What the Safari build can and cannot do is in the README ("Safari"): simulated input only, DOM-rendered screenshots while minimized (pixelExact:true for exact), no saved-password login, no recording.

Updating the extension (the load-unpacked folder is what runs, not the page)

Load unpacked runs the FOLDER on disk. Updating = refreshing that folder and reloading:

  1. Read nbrowser_readiness: note extensionVersions per profile and expectedExtensionVersion. Two stale signals — extensionStale/staleProfiles (a CONNECTED profile runs an older build; the socket reports its version) and staleOnDisk (the folder's manifest is old; this bridge computes it for connected profiles too). Match the extension against expectedExtensionVersion, not the pkg version.
  2. Refresh the folder: overwrite ~/adom/nb-bridge-macos/extension with the current build (this repo IS the source of truth on this box; the drift watcher keeps it current with John's line).
  3. nbrowser_dev_reload {profile} PER PROFILE — a bare reload only touches the active profile.
  4. Wait ~20s, re-read readiness: every profile in extensionVersions shows the new version.
  5. A profile you WAKE later needs its own reload too, and readiness lies until then: a profile asleep during the update starts its SW from a stale script cache — it announces the NEW manifest version while still running OLD module code (no windowEtiquette on its replies). nbrowser_dev_reload {profile} on it clears that. A version number in readiness is not proof the code changed.

Recovery / common failures

  • Load unpacked missing → Developer mode is off (Step A) or the Extensions page didn't render (re-launch the window with the chrome:// arg; verify by screenshot).
  • ⌘⇧G / typing does nothing in the panel → Accessibility not granted (CGEvents no-op). Grant it; retry. Never substitute System Events.
  • Panel selected the wrong folder → the silent killer; always screenshot the panel path before Open.
  • Card present but never connects → SW asleep (open a window in that profile / nbrowser_wake_profile), or host not registered / wrong allowed_origins ID (re-run install.sh), or host.json absent because the bridge isn't running.
  • "refusing to drive a PWA / installed-app window" → you grabbed a com.google.Chrome.app.* window; open a fresh normal window instead.

Honest status

The deterministic half (host registration via install.sh, folder staging) is solid. The GUI half is the frontier on macOS: OCR-clicking web content in chrome://extensions and filling the NSOpenPanel via ⌘⇧G + CGEvent are the parts to harden, and both depend on the Accessibility grant. Screenshot-verify every step, narrate with notify_user, and if the automation stalls, fall back to guiding the USER through the exact clicks (Developer mode → Load unpacked → ⌘⇧G, paste ~/adom/nb-bridge-macos/extension, Open) rather than reaching for System Events. Report any panel/OCR flakiness so we can harden the helper.