← Commit history

Record across ALL skills: foreground happens even without bring_to_front (opening a window + desktop_navigate can surface it to z:0 despite focused:false); judge by outcome (z-order); always screenshot-verify the page actually LOADED, not just the omnibox URL

John Lauer ·a1e7d692a4 ·4mo ago ·parent 8fc9fbf
5 files changed +41−7
skills/driving-the-desktop/SKILL.md+9−2
@@ -87,9 +87,16 @@ desktop_navigate {hwnd:<W>, url:"edge://extensions"}   # SetValue omnibox + post ``` ⚠️ **Scheme must match the browser:** `chrome://…` for Chrome, `edge://…` for Edge. Give Chrome an `edge://` URL (or vice-versa) and the omnibox **searches** the text instead of navigating. `https://` works-on either. **Always verify** with `desktop_screenshot_window` (a posted Enter is app-specific). Fallback if-a target ignores it: `desktop_bring_to_front` + `desktop_press_key ["enter"]` (brief foreground).+on either. **Always verify** with `desktop_screenshot_window` that the PAGE actually loaded (the target's+tab title changes, the page renders) - NOT just that the omnibox shows the URL; the posted Enter sometimes+does not commit and you must re-issue `desktop_navigate`. Fallback if a target ignores it:+`desktop_bring_to_front` + `desktop_press_key ["enter"]` (brief foreground). (`desktop_ui_set` also takes `submit:true` to set+commit an arbitrary field.)+⚠️ **`desktop_navigate` is NOT always foreground-free.** Despite the "NO foreground" note above, its+omnibox commit can **surface/activate the window** (observed: a `chrome://extensions` navigate left the+window at `z:0` with no `bring_to_front` and `focused:false`). Foreground is about the OUTCOME, not the+verb: after a navigate, check `desktop_list_windows` (`z:0` = topmost = you surfaced it), and avoid+repeated navigations while the user is mid-task.  ## Recipe — UNINSTALL the extension — BACKGROUND except one ~1s focus beat (verified AD 1.8.176) 1. `desktop_navigate {hwnd:<W>, url:"edge://extensions"}` (background). `find_control {hwnd:<W>,
skills/driving-the-extension/SKILL.md+7
@@ -45,6 +45,13 @@ trap: do **NOT** launch the browser yourself - `msedge` / `chrome --new-window` Foregrounding is the **user's** choice - only `desktop_bring_to_front`, and only when they explicitly ask to watch live. +⚠️ **Foreground can happen even when you NEVER call `bring_to_front`.** Opening a new window, and+especially **`desktop_navigate`** (its omnibox commit can surface/activate the window), may bring a+window to the FRONT despite `focused:false`. Judge by the **outcome, not the verb**: after an open or a+navigate, check `desktop_list_windows` (a window at `z:0` is topmost = you surfaced it) and treat that+as a disruption. Observed live: a `desktop_navigate` to `chrome://extensions` left the window at `z:0`+with no `bring_to_front` called.+ **Path:** `cloud → relay → adom-desktop (laptop) → native-browser bridge → native-messaging host → extension service worker → back`. You call `adom-desktop nbrowser_* '<json>'`. 
skills/installing-the-extension/SKILL.md+7
@@ -70,6 +70,13 @@ Rules, in order:   the user to pause typing for a moment, do the single action, and return immediately. Never hold the   foreground, never do it silently, and never do it while the user is actively typing if there is any   other way.+- ⚠️ **Foreground can happen even when you NEVER call `bring_to_front`.** Opening a new window, and+  especially **`desktop_navigate`** (its omnibox commit can surface/activate the window), may bring a+  window to the FRONT despite `focused:false`. Judge by the **outcome**: after an open or a navigate,+  check `desktop_list_windows` - a window at `z:0` is topmost = you surfaced it. (Observed live here: a+  `desktop_navigate` to `chrome://extensions` left the window at `z:0` with no `bring_to_front` called.)+  Also screenshot-verify the page **actually loaded** (tab title becomes "Extensions"), not just that the+  omnibox shows the URL - the posted-Enter sometimes does not commit, and the navigate may need a retry.  ## The flow (background; never foreground the user's work) Open a fresh background window you own, get its `hwnd`, and drive **that** (never the user's tabs):
skills/native-browser-safety/SKILL.md+12−3
@@ -41,9 +41,18 @@ user is typing, **their next keystrokes land in whatever control you just focuse text field), silently changing your state and derailing the task (real case: a foregrounded `chrome://extensions` search turned into `Adoms` mid-flow and the target extension vanished). Note this applies to **SendInput verbs too** - `desktop_type` / `desktop_press_key` / `desktop_click` foreground-the target before sending. Prefer background UIA (`desktop_ui_click` Invoke, `desktop_ui_set` SetValue)-and `desktop_navigate`; if a step genuinely needs foreground keyboard input, caption the user to pause,-do the one action, and return focus immediately.+the target before sending. Prefer background UIA (`desktop_ui_click` Invoke, `desktop_ui_set` SetValue);+if a step genuinely needs foreground keyboard input, caption the user to pause, do the one action, and+return focus immediately.++⚠️ **Foreground can happen even when you NEVER call `bring_to_front`.** Opening a new window, and+especially **`desktop_navigate`** (its omnibox SetValue + posted-Enter commit can surface/activate the+window to land the Enter), may bring a browser window to the FRONT despite `focused:false`. The rule is+about the **outcome, not the verb**: after any open or navigate, check `desktop_list_windows` (a window+at `z:0` is topmost = you surfaced it) and treat a window that jumped forward as a disruption. Minimize+opens and navigations, drive `chrome://` pages only when the user is not mid-task, and never repeatedly+surface a window. (Observed live: a `desktop_navigate` to `chrome://extensions` left the window at `z:0`+even though no `bring_to_front` was called and the window was opened `focused:false`.)  ## The "debugging this browser" bar is expected, not a bug CDP verbs (`click`, `type`, `screenshot`, `eval`, ...) attach `chrome.debugger`, which makes Chrome
skills/uninstalling-the-extension/SKILL.md+6−2
@@ -15,8 +15,12 @@ Like install, this is **our** recipe driving adom-desktop's **generic** OS verbs    user keeps typing, their keystrokes land in the control you focused (the Search box), silently    changing your filter and **hiding the card you were about to remove** (real case: search became    `Adoms`, the Adom card vanished). Use background UIA (`desktop_ui_click` Invoke, `desktop_ui_set`-   SetValue), `desktop_navigate`, `desktop_screenshot_window` - none steal focus. There is exactly ONE-   unavoidable foreground beat (the confirm bubble, Step 3); narrate it and keep it to ~1s.+   SetValue), `desktop_screenshot_window` - none steal focus. There is exactly ONE unavoidable foreground+   beat (the confirm bubble, Step 3); narrate it and keep it to ~1s.+   - ⚠️ Foreground can happen even with NO `bring_to_front`: opening a window and **`desktop_navigate`**+     (omnibox commit) can surface the window despite `focused:false`. Judge by the **outcome** - check+     `desktop_list_windows` (`z:0` = topmost = you surfaced it). And screenshot-verify the page actually+     loaded (tab title "Extensions"), not just the omnibox URL; the posted-Enter sometimes needs a retry. 2. **Isolate the target or you may remove the WRONG extension.** On `chrome://extensions` every card's    Remove button has the **same** accessible name "Remove" and `automationId:"removeButton"` - a blind    `contains:"Remove"` is ambiguous. **Filter the list to just the Adom card first** (Step 2). Verify by