← Commit history

Update 3 file(s)

John Lauer ·123682c7f6 ·3mo ago ·parent 79b1441
2 files changed +59−31
skills/driving-the-extension/SKILL.md+42−31
@@ -362,6 +362,14 @@ Paint glanceable status on the **real browser window's Windows taskbar button** **opened windows at open, and on adopted SPAWNED windows** when you call `nbrowser_owned` / `nbrowser_list_windows` (that is when the bridge learns their hwnd). Driven via AD's `desktop_taskbar`. +**TWO badges, two meanings (v0.1.45+).** The user must be able to tell "an AI is reading/clicking this+window" apart from "Adom merely opened it":+- **SOLID** Adom mark on their avatar = **layer 2: an AI has live control** (extension driving it).+- **HOLLOW** (outline) Adom mark = **layer 1: Adom OPENED this window but cannot see or drive it** - the+  USER is in control (an extension-free `open_os_window`, e.g. an OAuth consent page).+Both self-clear back to the plain avatar after ~3 min idle, on close, and when the USER closes the+window themselves (reaped within ~30s). You never manage this - do not hand-paint badges.+ **What the badge actually is (v0.1.12+):** Chrome/Edge use the taskbar overlay slot for the **profile avatar**, so a bare Adom badge would evict it and the user loses "which profile is this?". The bridge therefore builds a **composite**: the profile's own avatar **circle-cropped** (exactly how the browser@@ -392,39 +400,36 @@ natively on the user's machine, and is cached per profile. ## No extension in the profile? You can still OPEN their real browser (`nbrowser_open_os_window`) When a task only needs the user's real browser OPENED at a URL in the right profile - above all OAUTH CONSENT (adom-google, Autodesk APS: the user is already signed in; they just click Approve and your-CLI's token poll picks up the result) - you do NOT need the extension installed:+CLI's token poll picks up the result) - you do NOT need the extension: `nbrowser_open_os_window {sessionId, thread, purpose, profile, url}` (registration required, like every-window). It opens **BACKGROUND by default** (v0.1.41+): it launches the window the same way Hydrogen-Desktop's browser picker does (a clean launch, no `--profile-directory=Default`), lets it fully build,-then minimizes it once with a gentle no-activate call - so it never steals focus and, importantly, never-crashes the browser (an earlier version minimized mid-construction in a tight loop and crashed the whole-Edge process - fixed). Its taskbar button flashes ORANGE so the user clicks it when THEY are ready. To-raise it later: `desktop_bring_to_front {hwnd}` (returned in the result) - only when the user asks to see-it. `foreground:true` needs `foregroundReason` + a `notify_user` first, same as everywhere else.-- You CANNOT read, drive, or screenshot that window (no extension there) - poll the outcome-  out-of-band or ask the user.-- Close it when the flow is done: `nbrowser_close_window {sessionId}` (handled bridge-side for these).-- Every response carries the honest limitation + the ABE upsell - relay it to the user at a natural-  moment ("with the extension installed I could have filled this for you"), never mid-auth.-- If the profile HAS the extension, prefer `nbrowser_wake_profile` + `nbrowser_open_window` - full-  control, guards, and cleanup beat a fire-and-forget open every time.+window).++**Read the result fields - NEVER parse `_hint` prose:**+- `ok` / `launched` - the browser launch fired. For a consent flow this IS success: the page is now in+  front of the user.+- `hwndResolved` + `hwndReason` - whether we got a window handle you can address later+  (minimize/close/foreground). `hwndResolved:false` + `hwndReason:"grouped_into_existing_window"` means+  the browser put the page in an EXISTING window, so there is no new window to address. **That is not a+  failure.**+- `extensionInstalled` / `extensionLive` - whether that profile could be driven at layer 2 instead.++**Background is the default and it is enforced.** The window opens minimized with its taskbar button+flashing orange; the user clicks it when ready. `foreground:true` **without** `foregroundReason` is+REFUSED (`foreground_unjustified`) - no window opens at all. -## File uploads (`nbrowser_set_file_input`) - never the OS picker-Upload into an `<input type=file>` over CDP - background, silent, no OS file dialog (which cannot be-driven with the debugger attached and would foreground). Three shapes:-- **Direct** (the default): `{sessionId, selector|text, filePaths:["C:/abs/laptop/path"]}` - the finder-  walks a Browse button/label to its associated input; HIDDEN inputs work (most real UIs hide them).-  `input` + `change` are dispatched for you so React/Angular notice.-- **Transient inputs** (created inside the Browse click handler - nothing exists to select):-  `{armForNextChooser:true, filePaths}` then `nbrowser_click` the Browse control - the page's file-  chooser is satisfied silently (30s auto-disarm). Confirm with `{chooserStatus:true}` or a screenshot.-- **Cloud-direct small files**: `files:[{name, contentBase64}]` - the bridge stages them in a janitored-  temp dir on the laptop and substitutes the paths. One call, no `send_files` round-trip. Use-  `adom-desktop send_files` + `filePaths` for big payloads.-Errors teach: `file_missing` (bridge stats every path first), `input_not_found` (with a page diagnosis),-`no_files`. **Hard limitation:** sites using the File System Access API (`showOpenFilePicker`) cannot be-driven over CDP at all - use the site's drag-drop zone manually. After injection, most UIs still need-their Submit/Upload button clicked.+**What the USER sees (and why your strings matter):** any open the user will actually see gets a+**3-second caption at the bottom of their screen naming your reason** - an explicit foreground grant, a+page the browser grouped into an existing window, or `open_tab {active:true}`. The text comes from your+`foregroundReason`, falling back to your session `purpose`.+⭐ **So `purpose` and `foregroundReason` are USER-FACING COPY, not log shorthand.** Write a sentence a+human reads while it flashes by:+- bad: `purpose:"oauth flow"` / `purpose:"step 3"`+- good: `purpose:"signing you in to Autodesk so Fusion can reach your cloud files"`+An empty reason renders as "no reason given", which reads to the user as an AI that will not say what+it is doing.++**Afterwards:** close it with `nbrowser_close_window {sessionId}` (handled bridge-side for these). If the+USER closes it themselves, the bridge notices within ~30s, drops the window and removes its badge.  ## OS-level window inventory (`nbrowser_os_windows`) + extension-state diagnosis `nbrowser_os_windows {}` returns every Chrome/Edge/pup window as the LAPTOP sees it (hwnd, title, rect,@@ -446,6 +451,12 @@ BEFORE trying to wake, and never report "the profile doesn't exist" or "can't be fields tell you exactly what to do.  ## Activity log - "who drove what, when" (`nbrowser_activity`)+Internal trails you can query when the trust signals misbehave: `{verb:"_taskbar"}` = every badge paint+and flash (hwnd, which op, ok, HTTP status on failure); `{verb:"_open_announce"}` = every caption shown+to the user and its text; `{verb:"_ad_link"}` = AD-link outages/recoveries (if these show failures, the+badge/flash are degraded and it is NOT your call's fault); `{verb:"_window_reaped"}` = windows the USER+closed that we cleaned up.+ The bridge keeps a bounded in-memory trail (max 600 entries / ~1h TTL; resets on bridge restart) of every forwarded verb: `{agoMs, verb, sessionId, thread, profile, ok, errorCode}` plus an `activeSessions` summary (each with `lastAgoMs`). Use it to answer "why is that profile badged?", "what thread is driving
skills/native-browser-safety/SKILL.md+17
@@ -175,6 +175,23 @@ glance, that an AI is in that browser and *which profile* it is. Three rules: The bounded activity trail (`nbrowser_activity`) records who drove what and when - if the user asks "why is my browser badged?", query it instead of guessing. +## The user SEES your reason - write `purpose` for a human+Anything you open that the user will actually see (an approved foreground open, a page the browser+grouped into an existing window, `open_tab {active:true}`) puts a **3-second caption at the bottom of+their screen naming your reason** - taken from `foregroundReason`, falling back to your session+`purpose`. Those strings are **user-facing copy**, not log shorthand:+- bad: `"oauth flow"`, `"step 3"`, `"automation"` - reads as an AI that will not say what it is doing+- good: `"signing you in to Autodesk so Fusion can reach your cloud files"`+Say what you are accomplishing FOR THEM, not what the command does. This is the same standard as a+`reason` on any desktop action: written for the human in the moment, not for a log.++## Two badges: know which one you are earning+- **SOLID** Adom mark on the profile avatar = an AI has **live control** of that window.+- **HOLLOW** outline = Adom **opened** the window but cannot drive it; the user is in control.+Both fade to the plain avatar after ~3 min idle, on close, or when the user closes the window. Never+hand-paint or fake a badge - it is the user's only at-a-glance answer to "is something driving my+browser right now?", and a badge that lies is worse than no badge.+ ## Every verb tells you how to behave - read the hints Every `nbrowser_*` response carries a `_hint` (what you got, what to do next, the trap to avoid), and `nbrowser_describe` lists each verb's `hint` / `related` / `pitfalls`. When unsure, call