← Commit history

Install VERIFIED working end-to-end: bring picker to FRONT then agent-types path (the focus beat is the fix); run as a tight script; never ask user to type/paste the path; clipboard_set doesnt stick; notify_user+notify_response for consent. Extension loaded successfully.

John Lauer ·6f02564036 ·4mo ago ·parent f5a6d6e
2 files changed +47−39
docs/INSTALL-FLOW.md+9−5
@@ -117,10 +117,14 @@ loaded: > just focused (e.g. the extensions search box), silently changing your filter and hiding the card you > were acting on. We learned this live - a foregrounded search became `Adoms` and the extension > "vanished." The orchestration uses background UIA (`desktop_ui_click` / `desktop_ui_set` /-> `desktop_navigate` / `desktop_screenshot_window`) throughout; the one keyboard-only step (typing the-> path into the folder picker) is done via **clipboard-assist** (we put the path on the clipboard and-> the user pastes) rather than by stealing focus to type. Foreground is a narrated, last-resort-> exception, never the default.+> `desktop_navigate` / `desktop_screenshot_window`) throughout. The **one** unavoidable foreground is the+> folder-picker fill: the picker opens *behind* the user's windows, so the agent **brings it to front**+> (`desktop_bring_to_front`) and then types the path itself (`ctrl+l` -> `desktop_type` -> Enter ->+> Select Folder) - a brief (~2s) burst with a "please don't type" caption. Verified live: that focus+> beat is required (SendInput sent blind goes to the user's foreground window, not the picker). Do NOT+> ask the user to type/paste the ~50-char path, and do NOT rely on the clipboard (`desktop_clipboard_set`+> did not stick on a real machine). For consent, use `notify_user` (a Start button) + `notify_response`+> to wait for the click - then run the fast script.  ### What each step looks like (real screenshots) @@ -138,7 +142,7 @@ an empty Folder field. Never trust "it remembers the folder" - you must set the  ![The folder picker opened in the wrong directory (Downloads), Folder field empty](/blob/app/adom-browser-extension/screenshots/install-picker-wrong-dir.png) -**c) After setting the path** (clipboard-assist preferred, or the address bar) the picker is in the+**c) After the agent fronts the picker and types the path** (`bring_to_front` -> `ctrl+l` -> type -> Enter) the picker is in the `extension/` folder (`icons`, `src`, Folder = "extension"). **Screenshot-verify this path before** you click **Select Folder** - selecting the wrong folder loads a broken/old build silently: 
skills/installing-the-extension/SKILL.md+38−34
@@ -100,46 +100,50 @@ desktop_screenshot_window {hwnd:<W>}                              # confirm it f Do not proceed until the screenshot shows Developer mode ON and the **Load unpacked** button visible. If the row didn't appear, the toggle didn't flip - re-read and retry, don't proceed blind. -### Step B - Load unpacked -> the native folder picker (VERIFIED method, 2026-06-27)+### Step B - Load unpacked -> the native folder picker (WORKING method, verified live 2026-06-28)+**Run this as a TIGHT SCRIPT** (back-to-back calls, minimal sleeps) - a slow step-by-step with a caption+"sitting there" while you idle is infuriating to the user. The picker fill is **agent-driven and fast**.+⛔ **Never ask the user to type or paste the path** (it is ~50 chars - awful UX), and **do not rely on+the clipboard** (see note). The whole trick is one line: **bring the picker to the front before typing.**+ ```-desktop_caption {text:"Adom: selecting the extension folder..."}  # narrate to the user-desktop_ui_click {hwnd:<W>, automationId:"loadUnpacked"}          # VERIFIED stable id + invokable; background -> opens the picker+desktop_caption  {text:"Adom: installing - 2 seconds, please dont type"}    # warn ONCE, briefly+desktop_ui_click {hwnd:<W>, automationId:"loadUnpacked"}                    # opens the picker (#32770) - it appears BEHIND the user's windows+# ensure exactly one picker; grab its hwnd <PK> via desktop_list_windows (cancel extras)+desktop_bring_to_front {hwnd:<PK>}                                          # ⭐ THE KEY: picker opens at z:1/z:2; SendInput only lands once it is z:0+desktop_press_key {hwnd:<PK>, keys:"ctrl+l"}                                # focus the picker address bar+desktop_type      {hwnd:<PK>, text:"C:\\Users\\<user>\\adom-browser-extension\\extension"}   # agent types the path - fast+desktop_press_key {hwnd:<PK>, keys:"enter"}                                 # navigates INTO the folder+desktop_screenshot_window {hwnd:<PK>}                                       # VERIFY breadcrumb shows ...\extension BEFORE committing+desktop_ui_click  {hwnd:<PK>, contains:"Select Folder"}                     # commit -> the page toasts "Extension loaded"+desktop_caption  {text:""}                                                  # clear the caption immediately ```-The picker is a Win32 dialog (`#32770`, title "Select the extension directory."). ⚠️ **It opens in an-arbitrary last-used directory (observed: Downloads) - NEVER trust "it remembers the folder."** You MUST-set the path. -> ⛔⛔ **HARD-LEARNED (live, 2026-06-28): do NOT auto-fill the picker with SendInput while the user is at-> the machine.** The picker opens **BEHIND the user's active windows** (observed `z:1`/`z:2`, never `z:0`;-> the foreground was the user's VS Code / app). `desktop_press_key`/`desktop_type` into it did **not**-> fill the picker and the keystrokes most likely went to the **user's foreground window instead** (risking-> pasting your path / a stray Enter into their editor). UIA `desktop_ui_set` also can't fill it (the Folder-> combobox has **no ValuePattern**). So there is **no safe, reliable background way for the agent to type-> the path.** Use clipboard-assist:+**Why `bring_to_front` is the whole game:** the picker opens BEHIND the user's active windows (observed+`z:1`/`z:2`). SendInput without fronting it first sends the keys to the USER's foreground window (their+editor) - they never reach the picker (learned the hard way: a stray paste/Enter into the user's VS+Code). Front the picker (it goes `z:0`), and `ctrl+l` -> `type` -> `enter` -> `Select Folder` works+every time. This is a brief (~2s) **necessary** foreground - caption a "don't type" warning, do it fast,+clear the caption. (Step 1's `desktop_navigate` is focus-neutral on AD 1.9.34+, so this picker fill is+the ONLY foreground in the whole install.) -**Clipboard-assist (the method - the USER does the one keyboard action):**-1. Stage the path on the clipboard (native verb, background, no focus steal):-   `desktop_clipboard_set {text:"<abs path to extension>", returnPrior:true}` (AD v1.9.35+). Keep the-   returned `prior` and **restore it afterward** with another `desktop_clipboard_set`. ⚠️ The clipboard is-   **volatile** (other apps/the user overwrite it) - set it **immediately** before asking the user, and-   `desktop_clipboard_get` to confirm it still holds the path.-2. `desktop_caption {text:"Adom: the folder picker is open - paste the path (Ctrl+V) into the Folder box, then click Select Folder"}`.-3. The user pastes + clicks Select Folder. Then `desktop_screenshot_window` to confirm the card loaded.+⛔ **Do NOT** use the clipboard for this: `desktop_clipboard_set` did **not** stick on a real machine+(`set` -> `get` returned `""`), so a "paste the path" handoff leaves the user with an empty clipboard.+Agent-types-it-after-fronting is the method. (Clipboard verbs exist - AD 1.9.35 - but are unreliable for+this; flagged.) -**Agent-driven fill is ONLY acceptable when the user is truly away** (and even then it is finicky): first-`desktop_bring_to_front {picker hwnd}` so the picker is genuinely `z:0` (verify with `desktop_list_windows`),-caption a "don't type" warning, then `ctrl+l` -> `desktop_type <path>` -> `enter` -> screenshot-verify the-breadcrumb shows `...\extension` -> click Select Folder. Never do this blind while they are working - the-keys will not reliably reach the picker.+**Consent / handoff with notify (use it for the ONE gate, not the path):** before driving,+`notify_user {id, title, body, buttons:[{label:"Start"}], scenario:"reminder"}` (sticky toast + action+button) and poll `notify_response {id}` until they click Start. Verified round-trip. Use it to get a go+-ahead, then run the fast script above - never to ask them to type a path. -**Gotchas observed live:**-- **Load unpacked can open MULTIPLE pickers** (saw 2 from one flow). Before filling, `desktop_list_windows`-  and ensure exactly one `#32770` "Select the extension directory." is open; cancel extras.-- **Cancel the picker with `desktop_ui_click {hwnd:<picker>, contains:"Cancel"}`** - `automationId:"2"`-  returned success but did **not** close it; the name-based click did. ("Select Folder" likewise: prefer-  `contains:"Select Folder"` over `automationId:"1"`.)-- **ALWAYS screenshot + READ the path BEFORE committing** (`desktop_screenshot_window {hwnd:<browser>}`-  returns the picker in `screenshots[]`); the wrong folder loads a broken/old build silently.+**Gotchas (live):**+- **Load unpacked can open MULTIPLE pickers** (saw 2). `desktop_list_windows`, ensure exactly one `#32770`+  "Select the extension directory.", cancel extras.+- **Cancel = `desktop_ui_click {hwnd:<PK>, contains:"Cancel"}`** (the name; `automationId:"2"` reported+  success but did NOT close it). Likewise prefer `contains:"Select Folder"` over `automationId:"1"`.+- **ALWAYS screenshot + READ the path BEFORE committing** - the wrong folder loads a broken/old build+  silently.  ### Step C - verify it loaded clean AND connected (screenshot FIRST, three checks) 1. **Screenshot the extensions list and READ it** - this is the check that actually matters: