app
Adom Bridge
Public Made by Adomby adom
Adom Bridge unleashes your AI onto your full PC: full power, full safety. The AI breaks out of the container it runs in and onto your real machine, managing and running your entire laptop as you: launch apps, move files, drive any window, control KiCad, Fusion 360 or your real signed-in browser. Works with any AI, cloud or local (Claude Desktop, Claude Code, Codex), no Hydrogen required. Bridge and its bridges are an operating system for AI, with managed Node and Python runtimes, and a human-onl
← Commit history
Publish 1.9.174
7 files changed
+115−1187
README.md+28@@ -46,12 +46,30 @@ That installs the **container-side** pieces: the `adom-desktop` CLI (SHA256-veri `adom-wiki pkg update` keeps it current. The **KiCad** and **Fusion 360** bridge skills install from their own cloud-owned bridge pages. +## Drive it from a local AI on the same PC + +You do not need the cloud. The Windows installer puts `adom-desktop-cli.exe` on PATH and links the Adom Desktop skills into `~/.claude/skills` and `~/.codex/skills` automatically, so Claude Desktop, Claude Code, or Codex on the same machine can drive your desktop immediately with no relay and no setup. Deep dive: [using Adom Desktop from a local AI](docs/local-ai.md). + +## Sign in once, connect everywhere + +Sign in to your Adom account inside the app (Connections panel) and your cloud containers appear in the connection list and connect automatically: no JSON, no copy-paste. An AI in a container registers itself with one command. Deep dive: [sign-in and cloud configuration](docs/auth-and-cloud-config.md). + +## Many machines, many containers + +One desktop can connect to many containers, many desktops can share one container, and both at once. Address any desktop by name with `--target`. Deep dive with a topology diagram: [connection topology](docs/topology.md). + +## Running under Hydrogen Desktop + +When Hydrogen Desktop is present it runs Adom Desktop embedded: hidden window, HD-owned permission prompts, and a distinct taskbar tile. If HD ever goes away, Adom Desktop comes back standalone so your machine stays reachable. Deep dive: [how AD interacts with Hydrogen Desktop](docs/hydrogen-desktop.md). + ## The laptop GUI The **Windows GUI** installs separately: a signed NSIS installer served from this page (*Download for your machine* above; Hydrogen Desktop installs it automatically). It installs to `%LOCALAPPDATA%\Adom Desktop\`, adds a tray icon, starts on login, and **auto-updates itself** from this page's `version.json` (prompt by default; every 4h and on launch). The main window (top) puts everything on one surface: **Connections** (each relay plus the other Adom Desktops on it as peers), the **Activity Log** (every command, badged by source and type, filterable), the **Bridges sidebar** (installed bridges as mini-app cards), the provisioned **Runtimes**, and the footer approval bar. +Deep dives: [presence, tray and icon states](docs/presence.md) and [the managed Node and Python runtimes](docs/runtimes.md). + ## Downloads Everything a real user installs comes from this page. @@ -88,6 +106,8 @@ Anything AD can see, it can screenshot, losslessly, with **no focus steal** (`Pr AD records the desktop or a browser session to video. The recorder HUD (above) **requires a `reason`** shown to the user up front, so a recording never starts silently, with live Screen/Microphone toggles and a Start/Stop control. `desktop_recorder_open {reason}` and `desktop_recorder_close` for the desktop; `browser_record_start` and `browser_record_stop` for the pup browser. Built for smooth, high-frame-rate video, not a choppy screenshot loop. +Deep dive: [recording](docs/recording.md). + ## Bridges Each bridge is a small, independently-versioned process the app spawns, supervises, health-checks, and auto-updates from its own wiki page. Every install ships with the six below, and each shows up in the Bridges sidebar as a card: its billboard, a health LED, and language/source badges. @@ -130,6 +150,8 @@ The local dev-environment app AD can embed into; AD relays HD's runtime state an Install a third-party bridge by pasting its manifest URL, or `bridge_install {"manifestUrl":"..."}`. Build one with the [Bridge SDK](https://wiki.adom.inc/adom/adom-desktop-bridges). +Deep dive: [bridges from the user's side](docs/bridges.md) (cards, LEDs, console, installing and updating). + ## The taskbar icon tells you the mode AD's window carries a mode-specific app identity, so the taskbar button swaps its whole tile AND its name the moment the mode changes: no restart, no flicker. @@ -166,6 +188,8 @@ Auto-connect behavior, the update channel (**prompt** / auto / off) and *restart On top of that, the **remote cloud-AI path is approval-gated.** Verbs that run code or change the machine (`shell_execute`, `run_script`, `launch_process`, `write_file`, `registry_*`, `process_kill`, and so on) require a human grant when they arrive over the *relay*: **+1hr / +24hr** (time-boxed), **Bypass** (indefinite), or **Revoke**. The grant is human-only; local callers (your CLI, HD, bridges) are trusted by transport and never prompt; read-only verbs are never gated. +Deep dive: [permissions](docs/permissions.md) (what each Approve choice means, and how to revoke). + ## Notifications AD speaks to you through native Windows toasts via `notify_user`, under Adom Desktop's own identity (the Adom logo, and Action Center history like any first-class app). A cloud AI or a bridge, even one on a different machine addressed with `target`, fires them to reach you wherever you are. Every image below is the real toast, captured live from the exact call shown. @@ -228,6 +252,8 @@ Clicking the toast body brings the window named by `focus` to the front, or open The AI does not have to read any of this first. When a call leaves something on the table, such as no `focus`, or buttons on a toast that will vanish before you see it, the response says so and lists what the verb can do. That coaching stops once the AI gets it right. Full reference lives in the `adom-desktop-notify` skill. +Deep dive: [notifications](docs/notifications.md). + ## Verbs  @@ -243,6 +269,8 @@ curl -s -X POST 127.0.0.1:47200/command \ `adom-desktop help` and `GET /commands` are the live discovery surfaces for the full verb set. +Deep dive on the CLI itself: [the CLI guide](docs/cli.md). + ## Versioning This package version tracks the Adom Desktop release. The scaffold is small; the multi-MB binaries are fetched at install time with SHA256 verification (the manifest's `assets` map carries server-computed checksums per platform).
SKILL.md+39−589@@ -33,115 +33,6 @@ The `adom-desktop` binary is a single Rust CLI that does everything: The relay server runs in the Docker container. The Adom Desktop app runs on the user's PC and connects via WebSocket. -## Recent additions (v1.8.103 → v1.8.125) - -For readers of older copies of this skill: a lot has landed in the last -~14 releases. Quick summary so you don't try to call out to features that -don't exist or miss ones that do. - -- **Embedded `--start-hidden` honored + window preference** (v1.8.119+) — - when HD spawns AD with `--embedded --start-hidden`, AD now stays hidden - (v1.8.102–118 force-showed it). The release default respects the launch - flags; the machine owner can opt into always-show via the Settings - checkbox (config `always_show_window_on_launch`). Same preference pattern - as shell auto-approve. - -- **Popup-risk pre-flight on shell commands** (v1.8.117+) — `shell_execute` - and cmd-interpreter `run_script` are scanned BEFORE approval/execution - for `start <target>` invocations whose target doesn't resolve. Those used - to flash a modal "Windows cannot find '<x>'" dialog on the user's desktop - (no silent mode exists — the box is cmd/ShellExecute's own UI). Now AD - refuses with `errorCode: "popup_risk"` + a reprimand `_hint` pointing at - `find_exe` / `launch` / `desktop_open_url`. If your command creates its - own target mid-command and the scan misjudged it, re-send with - `{"allowPopupRisk": true}`. http/https URL targets pass; cmd builtins - pass; `copy x y && start y` patterns pass automatically. - -- **Shell auto-approve is a user preference** (v1.8.116+) — per-command - approval is the default everywhere now. Durable opt-in via AD's Settings - checkbox or `shell_auto_approve '{"permanent": true}'`; HD passes - `--shell-approval <prompt|1h|24h|permanent>` when spawning AD embedded - (session-only) and owns the setting while embedded. The old v1.8.44 - embedded-permanent default (and its leak into standalone sessions) is - gone — see "Shell auto-approve is a user preference" below. - -- **Registry-driven install** (v1.8.115+) — `adompkg install adom-desktop` - (with `?org=adom` on raw packages-API calls) now serves the current - release: the V2 wiki page registers a real adompkg version per release - and its manifest carries a server-derived `assets` map (url + sha256 per - platform) for the installer binaries. The kicad v0.8.6 verbs - (`kicad_export_*`, modifier-combo `kicad_send_key`, `kicad_open_3d_viewer` - editor arg), puppeteer v1.0.6 native-drive changes - (`native_relaunch_required` flow), and the embedded-mode fixes all reach - end-user installers starting with this release. - -- **`desktop_relaunch_self` + `desktop_relauncher_status`** (v1.8.103+) — - visible self-restart helper. AD spawns a separate signed Rust GUI - (`ad-relauncher.exe`) in HOST context that owns the entire swap-binary- - and-restart sequence: preflight → stop AD → atomic deploy → CreateProcessW - launch → verify reachable + PE FileVersion + window visible. Has its - own HTTP API on `:47210`. Don't run shell glue for AD relaunches; use - this. Full recipe: [`skills/AD_RELAUNCH.md`](AD_RELAUNCH.md). - -- **`restart`** (v1.9.90) — restart AD **in place** on a deployed/production - box: stop → relaunch the SAME installed exe → verify, with **no redeploy**. - Use this — not `desktop_relaunch_self`, which redeploys from a dev build tree - and silently no-ops on a normal install — when a headless / RDP / fleet box - needs AD cycled from the relay. Poll `desktop_relauncher_status` until - `overall` is `done_ok`. Pair with `--target <box>` for a specific machine. - The self-updater is relay-driveable too (`update_status`, `update_check`, - `apply_update`, `update_set_mode {auto|prompt|off}`) so a fleet box a version - behind can be updated without a human clicking the footer banner (v1.9.87+). - -- **Auto-update via wiki manifest** (v1.3.39+; single canonical wiki.adom.inc). - AD polls a `version.json` manifest every 4h + on launch from - `https://wiki.adom.inc/api/v1/pages/adom-desktop/files/version.json`. - Dead-end guard probes the `installer_url` before flipping - `update_available` (HEAD + Range-GET fallback for the wiki's HEAD-404 bug). Footer banner Install Now / Later; - user picks `prompt` / `auto` / `off` in Settings. SHA256-verified - silent NSIS install + autostart relaunch. - -- **`updater_get_status` payload extension** (v1.8.112+). `UpdaterStatus` - now includes `manifest_url` (the URL that served the cached manifest) - and `manifest_urls` (full priority chain). Sibling apps that read this - endpoint can show users which wiki AD is checking. The Build-details - tooltip on the footer version chip surfaces all of it + a clickable - link to the wiki page. - -- **Asset naming convention** (v1.8.113+). The full AD GUI installer and - the CLI companion binary now disambiguate cleanly in the wiki: - - **Full AD GUI:** `Adom.Desktop_<ver>_<os>_<arch>.<ext>` - Example: `Adom.Desktop_1.8.114_windows_x64-setup.exe` - - **CLI companion:** `adom-desktop-cli_<ver>_<os>_<arch>[.exe]` - Example: `adom-desktop-cli_1.8.114_linux_x86_64` - The pre-v1.8.113 Linux name `adom-desktop-linux` ships as a byte- - identical alias through v1.8.116. After install, the binary on PATH is - still `adom-desktop` — the wiki name disambiguates fetch time only. - -- **`version.json` `.cli` slot** (v1.8.113+). Additive schema — pre-v1.8.113 - readers ignore. Each populated platform has `binary_url` / `binary_sha256` - / `binary_size`. Today the manifest carries `cli.linux_x86_64` and - `cli.windows_x64`. HD / sibling apps should prefer this typed lookup - over hard-coding filenames. - -- **Windows CLI bundled with AD** (v1.8.114+). Installing AD drops - `adom-desktop-cli.exe` in `%LOCALAPPDATA%\Adom Desktop\` and adds the - install dir to user PATH (HKCU, no UAC) via the NSIS POSTINSTALL hook. - After install, `adom-desktop-cli screenshot_window …` works from any - fresh PowerShell / cmd / Claude Desktop terminal. The CLI auto-finds - the local AD via direct-API probe at `127.0.0.1:47200` — no relay - required for verbs that don't need streaming. - -- **CLI direct-API probe + relay self-detect** (v1.8.114+). The CLI's - probe drops the dead `host.docker.internal` candidate (we don't use - Docker Desktop anymore), adds WSL2 `/mnt/c/Users/*/.adom/direct-api-port` - DrvFs glob for AD's discovery file, and adds a NAT-mode WSL2 fallback - that resolves the Windows host IP from `/etc/resolv.conf`. The CLI's - `serve` subcommand probes `127.0.0.1:8766/health` before binding and - skips with a friendly message if another relay is already answering - (override with `--force-bind`). Both fixes matter for the mirrored- - networking WSL2 setup HD uses. - ## Starting the Relay Server The relay must be running before the desktop app can connect. **In a container, start it SUPERVISED** (v1.9.168+): @@ -598,188 +489,37 @@ Reference implementations: [`scripts/sample-bridges/hello-python/server.py`](../ - Full guide + copy-paste examples per type: the **adom-desktop-notify** skill ([`skills/NOTIFY.md`](NOTIFY.md)). - `notify_response` -- Poll which toast button the user clicked, by correlation `id`. `{pending:true}` until they click; `{pending:false, action}` once they do (consumed on read). -### KiCad Tools - -KiCad supports **multi-version side-by-side installs** — KiCad 9 and KiCad 10 (and older versions) can coexist on the same machine. Every `kicad_open_*` (and `install_*`, `run_drc`) command accepts an optional `kicadVersion` arg (e.g. `"9.0"` or `"10.0"`). Omit it to use the **newest installed** version. The success response includes `kicadVersionUsed` so you can confirm which install actually ran. - -- `kicad_list_versions` -- List every installed KiCad version with their paths and which is the default (newest). Run this first if you're unsure what's available. - - Example: `adom-desktop kicad_list_versions` - - Returns: `{versions: [{version, base_dir, kicad_exe, default}], default: "10.0", count: 2}` -- `kicad_install_symbol` -- Send a .kicad_sym file and install it as a library. Args include optional `kicadVersion`. -- `kicad_install_library` -- Install a symbol, footprint, or 3D model library. Each KiCad version has its own sym-lib-table / fp-lib-table — pass `kicadVersion` to target a specific install (otherwise installs into the newest version's tables). -- `kicad_open_board` -- Open a .kicad_pcb file. Args: `filePath`, optional `kicadVersion`. Example: `adom-desktop kicad_open_board '{"filePath":"C:/foo.kicad_pcb","kicadVersion":"9.0"}'` -- `kicad_open_schematic` -- Open a .kicad_sch file. Args: `filePath`, optional `kicadVersion`. -- `kicad_open_symbol_editor` -- Open the Symbol Editor. Optional `kicadVersion`. -- `kicad_open_footprint_editor` -- Open the Footprint Editor. Optional `kicadVersion`. -- `kicad_open_3d_viewer` -- Open the 3D Viewer (works from Footprint Editor or PCB Editor). Optional `kicadVersion`. -- `kicad_run_drc` -- Run Design Rule Check on a board via kicad-cli (headless, ~1-3s, structured JSON). v1.7.12+: prefer `kicad_lint_board` which wraps DRC + file-format sanity + schematic-parity in one structured response. -- `kicad_run_erc` -- **v1.7.12+** Run Electrical Rule Check on a schematic via kicad-cli. Mirrors `kicad_run_drc` for `.kicad_sch`. Catches unconnected pins, duplicate references, hierarchical-label mismatches, no-driver nets. Headless, ~1-3s. -- `kicad_lint_board` -- **v1.7.12+** Comprehensive headless pre-flight for a `.kicad_pcb` via kicad-cli: file existence + magic-byte sniff + DRC with `--severity-all` + `--all-track-errors` + `--schematic-parity` (default on). Returns structured `{data:{violations[], summary, fileFormat, schematicParityChecked}, _hint}`. Args: `filePath`, optional `schematicParity` (default true). **USE THIS BEFORE kicad_open_board** for any file you didn't just generate — catches DRC errors, parse issues, format-upgrade-needed BEFORE the GUI opens and possibly hangs on a modal dialog. -- `kicad_lint_schematic` -- **v1.7.12+** Comprehensive headless pre-flight for a `.kicad_sch` via kicad-cli: file existence + magic-byte sniff + ERC with `--severity-all`. Same structured shape as `kicad_lint_board`. **USE BEFORE kicad_open_schematic** for unfamiliar files. -- `kicad_lint_library` -- **v1.7.13+** Validate a library file or directory before install. Accepts a single `.kicad_sym`, `.kicad_mod`, `.pretty` directory, or directory containing `.kicad_sym` files. Parses S-expression structure (no kicad-cli call — `sym/fp upgrade` has no `--dry-run` and would rewrite). Returns symbol/footprint inventory + per-file `fileVersion` + tiered `_hint`. **USE BEFORE kicad_install_library / kicad_install_symbol / kicad_install_footprint** to catch malformed files, outdated formats, or wrong-filetype-renamed-with-.kicad_sym surprises in ~100ms. -- `kicad_format_upgrade` -- **v1.7.13+** Upgrade a KiCad file to the current format via `kicad-cli {pcb,sch,sym,fp} upgrade`. Auto-detects `kind` from extension if omitted. MUTATES the file in place (kicad-cli has no `--dry-run`). Common flow: lint_library reports outdated `fileFormatVersions` → surface to user → ask permission → call this. After upgrade, ANY OPEN KiCad windows holding this file must be closed + reopened. - -#### KiCad bridge — Phase 2.ac additions (v1.7.14+, via `kicad_bridge_call`) - -Three additions completing the Phase 2.ab loose ends: - -- **`verify_loaded {kind, expectedName?}`** — reads the Symbol/Footprint Editor frame title to confirm whether a load succeeded. `kind` is `"symbol"` or `"footprint"`. Optional `expectedName` adds a `matchesExpected` field comparing the loaded item against your target. Use AFTER `select_and_load_*` to confirm the symbol/footprint actually loaded (the load itself is "unverified" until you check). +### KiCad (`kicad_*`) — bridge-owned; light pointer -- **`get_net_topology {netName}`** — net-graph traversal: returns every footprint pinning the named net, plus the OTHER nets each of those footprints bridges. `{padCount, componentCount, components:[{ref, value, padsOnNet, otherPads:[{padName, netName}]}], bridgedNets}`. Use for "trace VCC outward from its source through the components it touches" analyses. +The KiCad bridge is **cloud-owned** and ships its OWN full skill (wiki pkg `adom/adom-desktop-kicad-bridge`). This page keeps only the pointer: run `adom-desktop sync_skills` after connecting and the full KiCad guide (project workflows, three-tier `kicad_open_*` routing, DRC recipes, library install) lands in `~/.claude/skills/`. -- **`select_and_load_{symbol,footprint} {name, useSendInput?: false}`** — v0.7.0 adds optional `useSendInput:true`. When `true`, escalates from `wx.PostEvent` (default; safe but may not propagate to KiCad's TOOL_DISPATCHER) to real Win32 `SendInput` with the proven `Enter` → `Ctrl+Shift+E` sequence. SendInput briefly STEALS foreground focus AND requires the Symbol Editor to be the foreground window before keystrokes dispatch — both default and SendInput paths are still "best-effort", so ALWAYS pair with `verify_loaded` to confirm. If verify_loaded reports `hasLoaded:false`, fall back to the legacy `kicad_open_symbol_editor '{"symbolName":"..."}'` path which uses out-of-process PowerShell SendKeys + mouse-click on the search box (proven but heavyweight). -- `kicad_close` -- Close all KiCad windows. Args: `force` (optional bool — skip graceful close, just taskkill) +Most-used verbs to get moving: -**When to specify `kicadVersion`:** -- The user asks to open something "in KiCad 9" / "in KiCad 10" / "in the older version" — pass that explicitly. -- The user is regression-testing across versions (open in 9, screenshot, close, open in 10, screenshot, compare). -- A library was installed under a specific version's tables and the user wants to use it. - -If you pass an invalid `kicadVersion`, the bridge returns `available_versions` and a `_hint` listing what IS installed — surface that to the user. - -#### KiCad reverse bridge (v1.7.5+) — direct in-process control - -KiCad v2 Phase 2 introduces a **reverse bridge**: an HTTP/JSON-RPC server runs *inside* every KiCad GUI process (kicad.exe project manager, pcbnew.exe PCB+Footprint editors, eeschema.exe Schematic+Symbol editors) once they restart after install. This lets adom-desktop introspect frames + post menu commands directly, bypassing UI screen scraping for everything the bridge supports. - -**Auto-installation is transparent.** The first `kicad_*` command of every bridge boot auto-deploys two files (`adom_bridge.py` + `usercustomize.py`) into `%USERPROFILE%\Documents\KiCad\<ver>\3rdparty\Python311\site-packages\` for every installed KiCad version. The response includes a `pluginAutoInstall` field with full details: - -```json -{ - "pluginAutoInstall": { - "triggered": true, - "summary": "installed" | "updated" | "already-current" | "partial-failure" | "skipped", - "payloadVersion": "0.1.0", - "perVersion": [{"kicadVersion":"10.0", "action":"installed", "userSitePath":"...", "filesWritten":[...], "_hint":"..."}], - "runningKiCad": [{"exeName":"eeschema.exe","pid":12345}], - "_hint": "Plugin successfully INSTALLED. KiCad currently running... ask user to restart KiCad..." - } -} +```bash +adom-desktop kicad_open_board '{"path": "C:/projects/board/board.kicad_pcb"}' +adom-desktop kicad_screenshot_all '{}' # every KiCad window incl. error dialogs, as an array +adom-desktop kicad_run_drc '{}' +adom-desktop kicad_window_info '{}' +adom-desktop kicad_open_symbol_editor '{"library": "Device"}' ``` -If installation fails, the `_hint` field tells you exactly what to report and how to recover. Failure modes: `payload_missing` (bundle regression), `user_site_unavailable` (no USERPROFILE), `permission_denied` (Documents dir uncreatable — likely OneDrive sync), `copy_failed` (KiCad has the .py locked — ask user to close KiCad), `no_kicad_installed` (offer `desktop_install_kicad`), `disabled_via_env`. - -**Currently-running KiCad processes don't pick up the bridge until restarted** (Python init runs once per process). If `pluginAutoInstall.runningKiCad` is non-empty AND summary is `installed`, surface the `_hint` to the user — they need to close + reopen KiCad to activate the bridge on those instances. - -- `kicad_bridge_status` -- Enumerate every running KiCad GUI process with the adom_bridge plugin loaded. Reads `%TEMP%\adom-kicad-bridge-<exe>.json` discovery files + probes each `/status`. - - Example: `adom-desktop kicad_bridge_status` - - Returns: `{plugins: [{exeName, pid, port, version, alive, uptimeMs, requestCount}], summary: {total, alive}, pluginAutoInstall: {...}}` - - If `summary.alive=0` after a fresh KiCad launch, check `pluginAutoInstall._hint` — the install probably failed and you have a specific recovery to offer. - -- `kicad_bridge_call` -- Generic passthrough: call any RPC method on the bridge for a specific KiCad exe. v1.7.6+ supports multi-instance disambiguation via `pid` arg. - - Args: `exeName` (required: `kicad` / `pcbnew` / `eeschema` / `kicad-cli`), `method` (required), `params` (optional dict), `timeout` (optional float seconds), `pid` (optional int — pin to a specific process; otherwise the newest alive instance is picked) - - Example — open Symbol Editor from running Schematic Editor: - ```bash - adom-desktop kicad_bridge_call '{"exeName":"eeschema","method":"wm_command","params":{"frame_index":0,"menu_id":20390}}' - ``` - - Methods (v1.7.6): - - `ping` — `{version, exeName, pid, uptimeMs, requestCount}`. Health check. - - `list_frames` — `[{frameClass, frameTitle, hwnd, menubarTopCount, menubarTotalItems, isShown}, ...]`. Replaces screen scraping. - - `get_menu_ids` — full menu walk per frame: `[{frameClass, frameTitle, menuItems: [{topMenu, label, id, kind, accelerator, helpString}, ...]}, ...]`. - - `wm_command` — posts a Win32 `WM_COMMAND` to the target frame via `PostMessageW(hwnd, WM_COMMAND, menu_id, 0)`. Goes through KiCad's TOOL_DISPATCHER. Args: `{frame_index, menu_id}`. - - `get_board_info` — **pcbnew only**. Returns `{hasBoard, fileName, trackCount, footprintCount, drawingsCount, netCount, copperLayerCount, boundingBoxMm:{widthMm,heightMm,centerXMm,centerYMm}}`. Replaces screenshot-poll for state queries. - - `get_schematic_info` — **eeschema only**. Returns `{available, frames:[{frameClass, frameTitle, hwnd, editorKind, fileName, isUnsaved, isUntitled}], frameCount}`. Title-based until KiCad 11 ships kipy. - - `get_open_editors` — per-process inventory: `{available, exeName, pid, editors:[{kind, frameClass, frameTitle, hwnd, isShown}], editorCount}`. For cross-process aggregator use the `kicad_open_editors` verb instead. - - **`get_pcb_footprints`** (v1.7.7+, pcbnew only) — full footprint list with `{ref, value, libNickname, itemName, fpid, x_mm, y_mm, rotationDeg, layer, isFlipped, isLocked, padCount, boundingBoxMm}` per footprint. Replaces screen-scrape "what's placed where". - - **`get_pcb_layers`** (v1.7.7+, pcbnew only) — layer stackup: `{copperLayerCount, totalEnabledLayers, layers:[{id, name, kind, isCopper}]}` where kind is one of: copper, mask, silk, paste, courtyard, adhesive, edge, fab, comments, user, margin, other. - - **`get_pcb_nets`** (v1.7.7+, pcbnew only) — net list with usage stats: `{netCount, totalTrackLengthMm, nets:[{code, name, padCount, trackLengthMm}]}`. - - **`get_pcb_design_rules`** (v1.7.7+, pcbnew only) — DR summary: `{trackWidthsMm, viaDimensions:[{diameterMm,drillMm}], minClearanceMm, minThroughHoleMm, minViaDiameterMm}`. - - **`navigate_symbol`** (v1.7.7+, eeschema only) — `params:{symbolName}` or `params:{query}`. Sets the wx.SearchCtrl value in the Symbol Editor's library panel + posts EVT_TEXT to activate the live filter. NON-destructive — does NOT load the symbol, only filters the visible library tree. **Replaces the PowerShell SendKeys path used by `kicad_open_symbol_editor` when `symbolName` is passed** — substantially more reliable + doesn't steal foreground focus. - - **`navigate_footprint`** (v1.7.7+, pcbnew only) — `params:{footprintName}` or `params:{query}`. Same shape as navigate_symbol but for the Footprint Editor. - - **`get_pcb_tracks`** (v1.7.8+, pcbnew only) — every track (segments, arcs, vias) with `{type, layer, layerName, widthMm, netCode, netName, lengthMm, startMm, endMm}`. Vias show up here too (with no length); call `get_pcb_vias` for via-specific fields. - - **`get_pcb_pads`** (v1.7.8+, pcbnew only) — every pad across all footprints: `{footprintRef, padName, positionMm, sizeMm, drillMm, netCode, netName, attribute}`. Position is absolute (post-rotation of parent footprint). - - **`get_pcb_vias`** (v1.7.8+, pcbnew only) — every via: `{positionMm, diameterMm, drillMm, viaType, topLayer, topLayerName, bottomLayer, bottomLayerName, netCode, netName}`. viaType is "through" / "blind_buried" / "microvia". - - **`get_pcb_drawings`** (v1.7.8+, pcbnew only) — silk text, edge cuts, dimensions, user-drawn shapes: `{type, layer, layerName, text?, positionMm?, startMm?, endMm?, centerMm?, radiusMm?}`. - - **`get_pcb_zones`** (v1.7.9+, pcbnew only) — copper pour inventory: `{zones:[{netCode, netName, layers, layerNames, priority, isFilled, isKeepout, clearanceMm, minWidthMm, polygonVertexCount, boundingBoxMm}]}`. - - **`get_footprint_connections`** (v1.7.9+, pcbnew only) — `params:{ref}`. Net-graph traversal: for a given footprint reference, returns per-pad connectivity. `{found, ref, value, padCount, pads:[{padName, netCode, netName, connectedTo:[{ref, padName}]}]}`. Each `connectedTo` list excludes the pad itself. - - **`get_drc_markers`** (v1.7.9+, pcbnew only) — read existing DRC markers from the board: `{markerCount, bySeverityCounts, markers:[{severity, severityCode, message, errorText, layer, layerName, positionMm, markerClass}]}`. Does NOT trigger DRC — for that use `kicad_run_drc`; this reads results from the LAST run. - - **`export_svg`** (v1.7.8+, pcbnew only) — multi-layer SVG export via `pcbnew.PLOT_CONTROLLER` (in-process, NO `kicad-cli` subprocess). `params:{outputDir?, layers?, mirror?, plotFrameRef?, useAuxOrigin?, subtractMaskFromSilk?, drillMarks?, filenamePrefix?}`. Default `layers` is every enabled copper layer plus F.Silkscreen + B.Silkscreen + Edge.Cuts. Output files: `<outputDir>/<boardBasename>-<layer>.svg` (or `<boardBasename>-<prefix>-<layer>.svg` if `filenamePrefix` is passed). Returns `{plottedLayerCount, outputDir, files:[{layerId, layerName, filePath, fileSize}]}`. **Use timeout:30+ on the bridge_call** because plotting can take 5-15s. - - **`export_pdf`** (v1.7.8+, pcbnew only) — same shape as export_svg, format=PDF. - - **`select_and_load_symbol`** / **`select_and_load_footprint`** (v1.7.9+, EXPERIMENTAL) — filters the library tree (via `navigate_*`) then attempts to activate the first match. **Known to segfault eeschema on large symbol libraries — only safe on small/test libs.** Phase 2.ab will fix via keystroke simulation. Until then, the default `kicad_open_symbol_editor` / `kicad_open_footprint_editor` paths continue to use the proven PowerShell SendKeys load — they're NOT routed through this method. - -- `kicad_install_plugin` -- **v1.7.6+** Explicitly redeploy the Phase 2 reverse-bridge plugin to every detected KiCad version's USER_SITE. Bypasses the auto-install cache. - - Args: `force` (optional bool, default true) - - When to use: after updating the plugin payload between releases, or to recover from a partial-failure auto-install (read perVersion errorCodes from `pluginAutoInstall` first). - -- `kicad_open_editors` -- **v1.7.6+** Cross-process editor inventory sourced via the bridge (not screen scraping). Hits each alive plugin's `get_open_editors` and concatenates. - - Returns: `{editors:[{kind, exeName, pid, frameTitle, hwnd, isShown}], byKind:{schematic_editor:[…], pcb_editor:[…], symbol_editor:[…], footprint_editor:[…], project_manager:[…]}, summary:{totalEditors, probedPlugins, failedPlugins, byKindCounts}}`. - - When to use INSTEAD of `kicad_window_info`: when you have the bridge available (the data is more structured + cheaper than the screenshot-fallback path). - -### Phase 3 — three-tier routing for `kicad_open_*` verbs (v1.7.6+) - -The legacy `kicad_open_symbol_editor`, `kicad_open_footprint_editor`, and `kicad_open_3d_viewer` verbs now try the bridge first before falling back to legacy WM_COMMAND-from-outside / icon-click / Alt+keystroke. Their responses include: -- `pathway` — which tier won: `"plugin"` (bridge), `"wm_command"` (legacy external post), `"icon_click"` (footprint editor launcher icon), `"alt_keystroke"` (3D Viewer from PCB Editor). -- `pathwaysTried` — ordered list of every tier attempted with success/failure annotations. -- `bridgePid` — if pathway was `"plugin"`, which KiCad process handled it. - -For Symbol Editor specifically, tier-1 has two sub-paths because Symbol Editor is hosted inside eeschema.exe (not as a standalone process): -- `plugin-eeschema-20390` — preferred when eeschema is already alive; sends menu_id 20390 to its Schematic Editor frame (opens Symbol Editor as a sub-window in the existing process). -- `plugin-kicad-20011` — cold-start path; sends menu_id 20011 to kicad.exe Project Manager (spawns a new eeschema.exe with Symbol Editor as top-level). - -**Bug fixed in v1.7.6** (caught via Phase 4 menu catalog audit): `kicad_open_symbol_editor` was sending menu_id 20012 to the project manager, but 20012 is actually **PCB Editor** in KiCad 10. The correct ID is 20011. The legacy WM_COMMAND-from-outside path silently failed for KiCad 10 users (Symbol Editor never opened, sometimes a PCB Editor window flashed instead). v1.7.6 fixes the constant + adds the bridge tier-1 path that uses the proven eeschema-20390 mechanism whenever possible. - -**Key menu IDs catalogued** (see `phase4-results/*.json` in the plan repo for the full ~870-item dump): - -| From frame | Action | menu_id | -|---|---|---| -| Project Manager | Open Schematic Editor | 20010 | -| Project Manager | Open Symbol Editor | 20011 | -| Project Manager | Open PCB Editor | 20012 | -| Project Manager | Open Footprint Editor | 20013 | -| Schematic Editor | Switch to PCB Editor | 20150 | -| Schematic Editor | Symbol Editor | 20390 | -| Schematic Editor | ERC | 20001 | -| Schematic Editor | Annotate Schematic | 20139 | -| PCB Editor | DRC | 20088 | -| PCB Editor | 3D Viewer | 20563 | -| PCB Editor | Footprint Editor | 20567 | -| PCB Editor | Switch to Schematic | 20234 | - -The bridge is a **Phase 2 MVP**. The legacy `kicad_open_*` / `kicad_window_info` / `kicad_screenshot_all` commands still work and remain the primary path for most flows; the bridge is additive. Phase 2.x will add `get_board_info`, `run_drc`, `navigate_symbol/footprint`, `export_svg`, and friends — when those land, this section will list them. - -#### KiCad UI Interaction - -Commands for detecting KiCad windows/dialogs, taking screenshots, clicking, and sending keyboard input. Essential for handling blocking dialogs and automating multi-window KiCad workflows. - -- `kicad_window_info` -- Returns all KiCad windows grouped as `projectManager`, `editors[]`, `modalDialogs[]` with HWNDs and titles. Uses **process-based enumeration** (finds all PIDs running from KiCad's bin dir, then enumerates their windows) — catches every KiCad window regardless of title. Check `hasModalDialogs` to detect blocking save/error prompts. - - Example: `adom-desktop kicad_window_info` - - Returns: `{projectManager: {hwnd, title}, editors: [{hwnd, title}], modalDialogs: [{hwnd, title, ownerHwnd}], hasModalDialogs: bool}` +If KiCad is not installed, the bridge knows how to install it for the user — offer, never point them at a download page. Full details: the bridge's own skill. -- `kicad_screenshot_all` -- Screenshots every KiCad window in one call (editors first, then project manager, then dialogs). Uses same process-based enumeration as `kicad_window_info`. Images downscaled to ≤1568px — use relative coords (0.0-1.0) for clicking, they're scale-independent. READ each screenshot to see what's on screen. - - Example: `adom-desktop kicad_screenshot_all` - - Returns: `{screenshots: [{type: "main"|"editor"|"dialog", title, hwnd, savedTo, sizeKB}]}` +### Fusion 360 (`fusion_*`) — bridge-owned; light pointer -- `kicad_send_key` -- Send a keystroke to a KiCad window. **Preferred way to dismiss dialogs**: enter to confirm, escape to cancel, tab to cycle buttons. Use click only if tab order doesn't reach the right button. - - Args: `key` (required: "enter", "escape", "tab", "space", "f1"-"f12", or single char), `hwnd` (optional — if omitted, sends to foreground KiCad window) - - Example: `adom-desktop kicad_send_key '{"key": "enter"}'` (dismiss dialog) - - Example: `adom-desktop kicad_send_key '{"key": "escape", "hwnd": 12345}'` (target specific window) +The Fusion 360 bridge is **cloud-owned** and ships its OWN full skill (wiki pkg `adom/adom-desktop-fusion-bridge`). Run `adom-desktop sync_skills` and the full guide (cloud files, import/export, add-in troubleshooting, APS search) lands in `~/.claude/skills/`. -- `kicad_click` -- Click at coordinates within a KiCad window. Relative coords (0.0-1.0) are scale-independent — estimate position as percentage from screenshot. To click a button at pixel (px,py) in an image of size (W,H), use `x=px/W, y=py/H`. - - Args: `hwnd` (required), `x` (required), `y` (required), `relative` (optional bool, default true — 0.0-1.0 range) - - Example: `adom-desktop kicad_click '{"hwnd": 12345, "x": 0.5, "y": 0.8}'` +Most-used verbs: -#### KiCad Dialog Detection Workflow - -KiCad has multiple independent windows (project manager, eeschema, pcbnew, footprint editor, symbol editor, 3D viewer). Unlike Fusion which has one main window, KiCad dialogs are owned by specific parent windows. - -**After any open/install command**, check for blocking dialogs: -1. `kicad_window_info` → check `hasModalDialogs` -2. If true: `kicad_screenshot_all` → READ each dialog screenshot -3. Dismiss with `kicad_send_key {"key":"enter"}` (OK) or `{"key":"escape"}` (cancel) -4. For specific button clicks: identify position in screenshot → `kicad_click {"hwnd":<dialog_hwnd>, "x":0.5, "y":0.8}` - -### Fusion 360 Tools - -Fusion 360 uses a two-tier architecture: -- **External bridge** (port 8773, auto-started): handles launch, detection, file opening -- **AdomBridge add-in** (port 8774, runs inside Fusion): required for export, design queries, Electronics/EAGLE commands - -- `fusion_start` -- **PREFERRED.** First-class Fusion 360 startup. Idempotent — safe to call whether Fusion is running or not. Discovers `Fusion360.exe` via webdeploy glob (no hardcoded hashes, no "Windows cannot find" GUI dialogs), verifies path exists before spawn, polls the AdomBridge add-in until it's online, and calls `fusion_dismiss_blocking_dialogs` to clear the startup picker and any other blocking modals. **Auto-dismisses startup dialogs during add-in wait** — the "What do you want to design?" picker and "Recovered Documents" dialog are dismissed every 5 seconds while polling, so the add-in becomes responsive within ~10-15s instead of timing out at 60s. Returns `{addinReady, mainThreadResponsive, dialogsDismissed[], dialogsRemaining[], dismissHint, pid, resolvedPath, elapsedMs, alreadyRunning}`. **Has a hard process-detection gate** — will never spawn a second Fusion process, so the "Multiple instances are not supported" GUI dialog cannot reach an end user. Use this as the very first call before any other Fusion command. -- `fusion_dismiss_blocking_dialogs` -- **First-class blocking-dialog killer.** Whenever any fusion_* command returns `errorCode: fusion_addin_not_responding` (or you see `_hint` mentioning a blocked add-in), call this FIRST before retrying. Two-layer design: (1) deterministic pattern match against known blockers — "What do you want to design?" picker, error toasts ("New design cannot be created..."), "Multiple instances" warning, crash recovery prompts, save prompts — dismisses each with `fusion_send_key escape` targeted at the exact hwnd. (2) AI-in-the-loop fallback — for anything the patterns didn't match, auto-screenshots every remaining Fusion dialog and returns the image paths inline in `remaining[].screenshotPath`, plus an explicit per-hwnd recovery script in `_hint` that you follow mechanically: Read the screenshot with the Read tool, understand what the dialog says, then `fusion_send_key '{"key":"escape","hwnd":<hwnd>}'` (or `"enter"` for OK-only dialogs). Re-call `fusion_dismiss_blocking_dialogs` to verify `mainThreadResponsive=true`, then retry your original command. **Fusion updates break pattern matches constantly, so the AI fallback is the critical path, not the deterministic side.** If you identify a new recurring pattern, add its title substring to `BLOCKING_DIALOG_PATTERNS` in `cli/src/commands.rs` so future sessions handle it automatically. +```bash +adom-desktop fusion_start '{}' # launches Fusion; slow on cold start (minutes) +adom-desktop fusion_get_app_state '{}' # the status/poll verb +adom-desktop fusion_addin_status '{}' +adom-desktop fusion_dismiss_blocking_dialogs '{}' +adom-desktop fusion_send_key '{"key": "Escape"}' +``` -- `fusion_addin_status` -- **Non-blocking** check of the AdomBridge add-in's busy state. Does NOT touch the Fusion main thread — queries the add-in's `/status` HTTP endpoint directly (~5ms). Returns `{busy, busyCommand, elapsedSeconds, requestId, walkProgress}`. Use this to check if a long-running command (walk, search) is in progress **from any bridge/session** before sending new commands. The CLI's auto-recovery path calls this before attempting dialog dismissal — if the add-in is just busy (not dialog-blocked), it returns `main_thread_busy` immediately instead of mashing Escape into a working Fusion. - - Example: `adom-desktop fusion_addin_status '{}'` - - When idle: `{"busy": false}` - - During a walk: `{"busy": true, "busyCommand": "walk_cloud_tree", "elapsedSeconds": 42.3, "walkProgress": {"foldersVisited": 15, "filesFound": 87, "currentFolder": "Molecules/XRP", "queueSize": 8}}` +Fusion launches take 2-4 minutes cold; respect `timeoutSeconds` + `statusVerb` on responses instead of re-issuing. Full details: the bridge's own skill. ### Launching apps (generic) @@ -1698,326 +1438,36 @@ AD auto-updates itself from the wiki manifest, but historically only a HUMAN cli - `update_set_mode {mode: auto|prompt|off}` -- the auto-apply POLICY. `auto` = unattended stage+apply on the 4h/on-launch poll (set this ONCE on headless fleet boxes and they self-update every release with no relay call); `prompt` = show the banner (default, attended); `off` = no checks. - Fleet flow: `update_status --target <box>` → if `updateAvailable` and unattended, `apply_update --target <box>` (or set `update_set_mode {"mode":"auto"}` once). A box must already be on ≥1.9.87 to have these verbs; the one-time bootstrap of an older box still needs a human click OR `auto` mode. -### Browser Automation (Puppeteer) - -Multi-session Puppeteer -- each session gets its own Chrome window. Auto-starts on first command. **Always pass `sessionId` on every command.** - -Two levels of granularity: **window** (one Chromium window per session, own taskbar icon) and **tab** (many tabs per session, one taskbar icon). Use windows when isolation matters, tabs when showing many related views (10 alignment previews, etc.) to avoid flooding the taskbar. - -**Window-level commands** (one session = one Chrome window): -- `browser_rescan` -- (v1.5.1+) Recover orphaned Chrome windows whose CDP socket dropped. Walks every known profile (in-memory + on-disk session files), reconnects via the persisted DevToolsActivePort, then walks every page and rebuilds session entries by parsing the `(session: X)` tag injected into each page's title at launch. Idempotent. Args: `adoptOrphans?` (default false; when true, pages without a tag get attached under a generated sessionId — useful when Chrome has tabs the bridge has never seen). - - **When you need this**: any time `browser_navigate` / `browser_eval` / `browser_screenshot` errors with `errorCode: "session_disconnected"` (the bridge knows about your sessionId but its CDP socket dropped) or `errorCode: "session_not_found"` (after a bridge restart that rebuilt from disk but couldn't reach the live Chrome). The 30s health check now auto-attempts reconnect, so most transient blips recover before you notice — but `browser_rescan` is the explicit recovery primitive. - - Returns: `{ok, rescanned, profilesReconnected, profilesUnreachable, reattached, orphansAdopted, liveSessions, disconnectedSessions, _hint}`. -- `browser_open_window` -- Open a Chrome window. **Default mode (CfT — Chrome for Testing):** args `sessionId` (required), `profile` (required), `url` (required), `freshProfile` (optional), `strictPermissions` (optional, default `false`), `downloadPath` (optional, default `%USERPROFILE%\Downloads`). **Native mode (v1.0.6+, opt-in — drives the user's REAL Chrome/Edge/Brave, actual signed-in profile):** instead of `profile`, pass both `nativeBrowser` (`chrome`/`edge`/`brave` from `desktop_list_browsers`) and `nativeProfile` (profile id: `Default`/`Profile 1`/`Profile 2`). The bridge smart-launches: fresh-launch if the browser is closed (no prompt, returns `launchedFresh:true`), reconnect if it already exposes a debug port, or returns `errorCode:"native_relaunch_required"` (ask the user, re-call with `nativeRelaunch:true`) if it's open without one. If the debug port is enterprise-policy-blocked it returns `errorCode:"native_cdp_blocked"` with a `diagnosis` + `alternatives` (other installed browsers — Edge often works when managed Chrome can't). See the "When pup means 'the user's real Chrome/Edge'" section above for the full flow + routing rules. - - **Full-capability default (v1.6.3+):** every pup session opens with the capabilities chip-fetcher-style flows actually need: - - **Scripted downloads enabled.** `Page.setDownloadBehavior` is called with `behavior: 'allow'` and `downloadPath: %USERPROFILE%\Downloads` (override via the `downloadPath` arg). Re-applied on every main-frame navigation as a safety belt. This unblocks the silent-download-failure scenario: before v1.6.3, JS-triggered downloads (clicks on `<a download>`, `fetch().then(blob → save)`, vendor "Download Symbol" buttons that do `window.location = url`) could be silently dropped by Chrome — no error, no UI, no file. chip-fetcher debugged this for hours before the fix. - - **Clipboard read/write granted.** Paste-into-vendor-form flows work without permission prompts. - - **Notifications + geolocation + MIDI denied.** Chrome doesn't pop up permission dialogs under automation. - - The response includes `defaultPermissionsApplied: true`, `defaultPermissions: ["clipboard-read=granted", "clipboard-write=granted", "notifications=denied", "geolocation=denied", "midi=denied"]`, `downloadsEnabled: true`, `downloadPath: <resolved abs path>`. The path is what `desktop_watch_files` polls by default — the two ends meet without configuration. - - **Opt-out:** pass `strictPermissions: true` to keep Chromium's defaults in place. Rarely needed — only when driving an untrusted site you're auditing. With strict mode, scripted downloads may be silently dropped, clipboard prompts fire, etc. - - **History note:** v1.4.12 used `Browser.grantPermissions(['automaticDownloads', 'notifications', 'clipboardReadWrite'])` — but Chrome 146 rejects `automaticDownloads` as an unknown permission name, taking the whole grant down. v1.6.3 fixes by using `Browser.setPermission` per-permission (per-permission failure tolerance) AND by realizing `Page.setDownloadBehavior` is the actual mechanism for unblocking scripted downloads, not a permission grant. -- `browser_close_window` -- Close a session's Chrome window (closes ALL tabs in that session). Args: `sessionId` -- `browser_list_windows` -- List all open sessions with URL, title, tabCount, active status - -**Tab-level commands** (manage tabs within a session's window): -- `browser_open_tab` -- Add a tab to an existing session. Returns `{tabId, url, title, active}`. Args: `sessionId`, `url` -- `browser_switch_tab` -- Make a specific tab the active one (calls `bringToFront`). Args: `sessionId`, `tabId` -- `browser_close_tab` -- Close a specific tab. Leaves other tabs + session intact. Args: `sessionId`, `tabId` -- `browser_list_tabs` -- List all tabs in a session: `{tabs:[{tabId,url,title,active,errorCount,opener,openerTabId}], activeTabId, count}`. Args: `sessionId`. **Auto-tracks popup tabs since v1.4.7** — tabs the page spawned via `window.open()` / `target="_blank"` / form-submit-with-target=_blank appear here automatically with `opener:"popup"` and `openerTabId` pointing at the tab that triggered them. Tabs Claude opened via `browser_open_window`/`browser_open_tab` have `opener:"user"`. - -**Tab-aware operations** (all accept optional `tabId`; omit = use active tab): -- `browser_navigate` -- Navigate to a new URL. Args: `sessionId`, `url`, `tabId?` -- `browser_screenshot` -- Capture screenshot, auto-resized to <=1568px. Args: `sessionId`, `maxWidth`, `fullPage`, `tabId?` -- `browser_eval` -- Evaluate JS in the page context. Args: `sessionId`, `expr`, `tabId?` -- `browser_input_dispatch` -- **Trusted** click / type / key / move via Chromium's CDP-backed `Input.dispatchMouseEvent` / `Input.dispatchKeyEvent`. Events have `isTrusted: true`, so they pass framework click-handler gates (React/Vue/Svelte) and most vendor anti-automation checks that `browser_eval`-side `dispatchEvent(new MouseEvent(...))` silently fails on. Args: `sessionId?`, `tabId?`, `type` (`click`|`move`|`type`|`key`), plus per-type fields: - - `click`: either `selector` (uses bounding rect — preferred when DOM is stable) OR `{x, y}` viewport coords. Optional `button` (`left`/`right`/`middle`), `clickCount` (1 or 2 for double-click), `delay`, `firstMatch` (default false; see below). - - `move`: `{x, y}` plus optional `steps` for human-like trajectory. - - `type`: `text` to type; pass optional `selector` to focus that element first; optional `delay` between keystrokes. - - `key`: a Puppeteer KeyInput name (`Enter`, `Escape`, `Tab`, `ArrowUp`, `F1`, etc). - - **Smart selector pick (v1.4.8+, modal-scoped in v1.4.9+, plausibility-filtered in v1.4.10+):** when a `selector` matches multiple elements (a class like `button.wg-button--primary` often does — cookie X buttons + duplicate hidden copies + the actual submit), the bridge picks intelligently: - 1. **If a plausible modal/dialog is open** — `<dialog open>`, `[aria-modal="true"]`, `[role="dialog"]`, OR a fixed/absolute element with z-index ≥100 covering >25% of the viewport — AND it contains at least one visible interactive element with text — restrict the candidate set to elements **inside** that modal first. Tiebreak by largest visual area inside it. The page behind a modal is visually inert; clicking elements behind it is almost never what the caller intended. The "plausibility filter" added in v1.4.10 prevents empty overlay containers (Vue-Toastification toast wrappers, notification mount points, full-screen ad scrims) from being mistaken for the real modal — a common false-positive on Vue/React apps that mount toast portals at body level. - 2. Otherwise filter to **visible elements with non-empty text** and tiebreak by **largest visual area**. - 3. Fall back to visible-only, then document-order first as last resort. - - Response includes `matchedCount`, `chosenIndex`, `clickedRect:{x,y,w,h}`, `clickedText`, `insideModal`, `modalDetected`, `modalRoot:{tag,id,cls,z,role,ariaModal}|null` (v1.4.10+ — which DOM node won the modal-detection heuristic, for diagnosing false-positives), and `pickStrategy` (`only-match` | `first-match` | `modal-scoped-largest` | `visible-text-largest` | `visible-text-largest-no-modal-match` | `visible-largest` | `fallback-first-document-order`). **Always sanity-check the response** — if `clickedText` doesn't look right, inspect `modalRoot` to see whether modal detection latched onto the intended overlay or got fooled by a sibling, then refine the selector OR pass `firstMatch:true` to skip the smart pick OR fall back to `{x, y}` coords. - - **Use this whenever a click "lands but does nothing"** — most vendor "Generate Datasheet" / "Download" / "Submit" buttons gate on `event.isTrusted`. After clicking, follow up with `browser_list_tabs` (popups auto-track) to grab any new tab the click spawned. - -- `browser_fetch_url` -- Fetch an arbitrary URL with the session's cookies and return raw bytes. Bypasses Chrome's PDF Viewer wrapper — critical for grabbing PDF binaries from popup tabs where in-page `fetch(location.href)` returns the viewer HTML wrapper (~200 KB stub) instead of the actual PDF (multi-MB). Also handles ZIPs, CAD bundles, anything served as a binary content-type. - - Args: `sessionId?`, `tabId?` (picks cookie context — defaults to active tab), `url` (required), `method?` (default GET), `headers?` (object; Cookie auto-included from session), `body?` (raw string). - - **`saveTo` writes on the DOCKER container's filesystem** (the CLI handles the write after receiving bytes from the bridge). Pass an absolute path. Parent dirs are created. Returns `savedTo` with the canonical absolute path (verified to exist on disk) plus `savedToFilesystem:"container"`. - - **`desktopSaveTo` writes on the WINDOWS desktop's filesystem** (the bridge writes directly via `fs.writeFileSync`). Pass an absolute Windows path (e.g. `C:\\Users\\you\\Downloads\\foo.pdf`). Returns `desktopSavedTo` with the resolved absolute path. Use this when the user wants the file local on the desktop (e.g. dropping a CAD bundle into Downloads). - - **Without saveTo or desktopSaveTo**, returns `bodyBase64` in the response — caller decides what to do. - - Returns: `{ok, bytes, contentType, status, sessionId, tabId, bodyBase64?, savedTo?, savedToFilesystem?, desktopSavedTo?, desktopSaveError?}`. - - **CRITICAL READ-THE-RESPONSE rule:** when you pass `saveTo`, the response's `savedTo` is the actual filesystem path the file was written to. v1.4.8 had a bug where the bridge would respond `savedTo:"/tmp/foo.pdf"` but the file existed nowhere; v1.4.9 fixes this — `savedTo` always reflects a real on-disk path. - - **Recipe — popup PDF capture (the right pattern):** - ```bash - # After browser_input_dispatch click that spawns a PDF popup, READ THE URL - # FROM browser_list_tabs (don't eval against the popup's tabId — popups - # auto-close fast and eval-after-close errors with a structured hint). - LIST=$(adom-desktop browser_list_tabs '{"sessionId":"chip-fetcher"}') - POPUP_URL=$(printf '%s' "$LIST" | jq -r '.tabs[] | select(.opener=="popup") | .url' | head -1) - POPUP_TAB=$(printf '%s' "$LIST" | jq -r '.tabs[] | select(.opener=="popup") | .tabId' | head -1) - - adom-desktop browser_fetch_url "{ - \"sessionId\":\"chip-fetcher\", - \"tabId\":\"$POPUP_TAB\", - \"url\":\"$POPUP_URL\", - \"saveTo\":\"/tmp/datasheet.pdf\" - }" - # → {ok:true, savedTo:"/tmp/datasheet.pdf" (exists), savedToFilesystem:"container", - # bytes:3798336, contentType:"application/pdf", status:200} - ``` -- **Per-call auto-recovery (v1.6.1+):** every tab-aware verb (`browser_navigate / eval / screenshot / input_dispatch / errors / reload / fetch_url`) now **silently reconnects or relaunches** before surfacing any session error. The Docker container should NEVER see "Session closed" / "detached Frame" / "session_disconnected" again unless recovery is genuinely impossible. The recovery ladder: - 1. Live session → return immediately. - 2. Session is `_lostBrowser` (CDP socket dropped after sleep/wake / network blip / puppeteer hiccup): try `attemptReconnectProfile()` which checks (a) the on-disk session file's recorded `cdpPort`, (b) the profile dir's `DevToolsActivePort`, and (c) **scans running Chrome processes for a `--remote-debugging-port=N` matching this profile dir** (this catches the canonical sleep/wake symptom — Chrome alive, CDP serving, but the bridge has no on-disk record). If reconnect succeeds, walk pages and re-attach by parsing the `(session: X)` tag from titles. - 3. Reconnect failed: kill any orphan Chrome processes for this profile, clean the profile lock files (`lockfile` on Windows, `SingletonLock` / `SingletonSocket` / `SingletonCookie` on POSIX, plus stale `DevToolsActivePort`), then relaunch via `launchSession` with the last-known URL from the session file. The sessionId is preserved — the caller gets a transparent recovery. - 4. Total failure (no profile name, no last-known URL, can't launch) → only THEN does the structured `session_not_found` / `session_disconnected` error fire. - - When recovery happens, the response includes `_recovered: true` and `_recoveryReason: "reconnect-no-page" | "relaunch"` so callers can log / metric the recovery rate. - -- **Stale sessionId detection (v1.5.1+):** when auto-recovery (above) genuinely cannot fix the problem (no profile known, no last-known URL, Chrome won't relaunch — rare), a structured error fires. Two error codes you might see: - - **`session_not_found`** — the bridge has no record of this sessionId. Recipe: check `liveSessions` for the right name, or `browser_open_window` if you want to start fresh. - - **`session_disconnected`** — the bridge has the session in memory (and on disk) but auto-recovery couldn't reach its Chrome. Recipe: call **`browser_rescan`** to force a manual recovery pass, or `browser_open_window` with explicit URL to relaunch. - -- **`browser_rescan`** (v1.5.1+, enhanced v1.6.1+) — explicit recovery primitive. v1.6.1 adds a running-process scan: walks all Chrome processes whose `--user-data-dir` is under the puppeteer profiles dir, even if the bridge has no in-memory or on-disk record of them. This is what fixed the chipsmith-after-sleep scenario where the bridge had been restarted while Chrome was alive — neither in-memory state nor session files referenced the live Chrome, but the process list did. -- **Stale tabId detection (v1.4.10):** every tab-aware verb (`browser_navigate`, `browser_screenshot`, `browser_eval`, `browser_input_dispatch`, `browser_errors`, `browser_reload`, `browser_close_tab`, `browser_fetch_url`) returns a structured error when you pass a `tabId` that no longer exists: - ```json - { - "ok": false, - "errorCode": "tab_not_found", - "error": "Tab \"tab-3\" not found in session \"chip-fetcher\".", - "currentTabs": ["tab-1", "tab-2"], - "lastKnownUrl": "https://wago.priintcloud.com/datasheets/2601-3105/en/...", - "opener": "popup", - "openerTabId": "tab-1", - "closedMsAgo": 1842, - "_hint": "This tab closed 2s ago at URL https://... Call browser_fetch_url with that URL (and the opener tabId \"tab-1\" for cookies) to get its bytes — do NOT try to eval against the closed tabId." - } - ``` - - `lastKnownUrl` + `opener` + `openerTabId` + `closedMsAgo` are populated when the bogus tabId matches a tab that closed in the last few seconds (kept in a 20-entry per-session ring buffer). Critical for popup PDF workflows where the viewer auto-closes after rendering — you can `browser_fetch_url` directly against `lastKnownUrl` instead of having to re-trigger the spawning click. - - **Important regression note:** v1.4.9 documented this error but the CLI was silently stripping `tabId` from `browser_navigate` / `browser_screenshot` / `browser_eval` / `browser_errors` / `browser_reload` calls before forwarding to the bridge — so the bridge always saw "no tabId" and fell back to the active tab. v1.4.10 fixes the CLI to forward `tabId` so the structured error actually fires. If you're still seeing silent fallback, your CLI is < v1.4.10 (`adom-desktop --version` to check). -- `browser_errors` -- Collected console/page errors. Args: `sessionId`, `clear` (default true), `tabId?` -- `browser_reload` -- Reload the page. Args: `sessionId`, `tabId?` - -**Other**: -- `browser_status` -- All sessions with URLs, tab counts, and error counts -- `browser_close` -- Close ALL sessions -- `browser_wait` -- Wait for content to settle. Args: `ms` - -#### Credential vault — silent HTTP Basic Auth (v1.4.4+) - -When pup navigates to a host that issues an HTTP Basic Auth challenge (every `*.componentsearchengine.com` subdomain that wraps NXP / Mouser / TI / etc CAD bundles, several vendor design + doc portals), Chrome shows a NATIVE auth dialog that lives outside the page DOM. `browser_eval` can't dismiss it. Without the credential vault, the user has to type the same email + password every navigation — chip-fetcher hits this 5–10 times per chip. - -The vault stores credentials in the OS keychain (Windows DPAPI / macOS Keychain / Linux libsecret) and applies them via `page.authenticate()` BEFORE the navigation, so the dialog never appears. **Passwords never leave the keychain** — `credential_list` returns host + username only. - -- `credential_set` -- Store creds for a host pattern. Encrypted at rest by the user's session key. - - Args: `host` (glob: `"*.componentsearchengine.com"` or exact `"nxp.componentsearchengine.com"`), `username`, `password` - - Returns: `{ok, host, username}` (no password) -- `credential_list` -- List stored entries. - - Returns: `{credentials: [{host, username, addedAt, updatedAt}]}` - - Passwords are **never** returned by this verb. -- `credential_delete` -- Remove a credential entry. Drops the password from the keychain and the entry from the index. - - Args: `host` (must match the pattern used at `credential_set` time, exactly) - -**Host matching:** simple glob. `*.example.com` matches any subdomain; exact strings match exactly. Best-match wins (longest non-glob suffix beats shorter glob), so a `nxp.componentsearchengine.com` entry would override a `*.componentsearchengine.com` entry for that exact host. - -**Recipe — chip-fetcher / CSE flow:** - -```bash -# 1. One-time setup (ask user for their CSE creds, then store) -adom-desktop credential_set '{ - "host":"*.componentsearchengine.com", - "username":"[email protected]", - "password":"<paste>" -}' -# → {ok:true, host:"*.componentsearchengine.com", username:"[email protected]"} - -# 2. Navigate freely — no popup -adom-desktop browser_open_window '{ - "sessionId":"chip-fetcher", - "profile":"chip-fetcher", - "url":"https://nxp.componentsearchengine.com/preview_newDesign.php?..." -}' -adom-desktop browser_eval '{"sessionId":"chip-fetcher","expr":"document.title"}' -# → "SamacSys Part Preview" (NOT "Sign in") - -# 3. Audit — passwords are never returned -adom-desktop credential_list -# → {credentials:[{host:"*.componentsearchengine.com", username:"[email protected]", ...}]} - -# 4. Remove -adom-desktop credential_delete '{"host":"*.componentsearchengine.com"}' -``` - -Hooks fire BEFORE every `page.goto` in three places: `browser_open_window`'s first nav, `browser_open_tab`, `browser_navigate`. The credential lookup is `O(N)` over stored entries which is fine for typical N (single digits). - -If a host has no stored creds, navigation proceeds normally — the user will see Chrome's native dialog as before. There's no fallback prompt; the next step is for you to ask the user for the credential, then `credential_set` it. - -Example — show 10 alignment previews in ONE Chrome window: -```bash -adom-desktop browser_open_window '{"sessionId":"align","profile":"align","url":"http://127.0.0.1:8901/"}' -for port in 8902 8903 8904 8905 8906 8907 8908 8909 8910; do - adom-desktop browser_open_tab "{\"sessionId\":\"align\",\"url\":\"http://127.0.0.1:$port/\"}" -done -adom-desktop browser_list_tabs '{"sessionId":"align"}' # see all 10 -adom-desktop browser_screenshot '{"sessionId":"align","tabId":"tab-3"}' -adom-desktop browser_eval '{"sessionId":"align","tabId":"tab-5","expr":"document.getElementById(\"meta-seat-z\").textContent"}' -``` -- `browser_alert_window` -- Flash the Windows taskbar orange. **Always call after updating a pup window.** Args: `sessionId` -- `browser_focus_window` -- **Tab-only.** Calls Puppeteer's `page.bringToFront()` — activates the session's tab WITHIN its Chrome window, but does NOT raise the OS window above other apps on the desktop. Args: `sessionId`. **For OS-level foreground use `browser_raise_os_window` (below).** -- `browser_raise_os_window` -- **Real OS foreground raise.** Brings the Chrome window hosting this pup session above all other desktop apps. Use this BEFORE recording, screenshots, animations, or any flow where Chrome being occluded would matter — when Chrome is in the background, `document.hidden=true` and Chrome throttles `requestAnimationFrame` to ~1 Hz, breaking cinematic camera orbits, CSS animations, video element auto-play, fps counters, etc. Internally: focuses the tab inside Chrome, then EnumWindows-finds the OS window by title containing `(session: <sessionId>)`, restores it from minimized if needed, and runs the foreground-lock bypass (`AttachThreadInput` + `keybd_event` phantom-key trick). Args: `sessionId`. Returns: `{sessionId, hwnd, title, raised, error?}`. - ```bash - # Typical recording prep - adom-desktop browser_open_window '{"sessionId":"demo","profile":"demo","url":"…"}' - adom-desktop browser_raise_os_window '{"sessionId":"demo"}' # ← page now actually visible - adom-desktop desktop_record_start '{"reason":"…"}' - ``` -- `browser_lower_os_window` -- **OS-level minimize.** Sends the Chrome window for this session to the back / minimized so the user gets their desktop back after Claude finishes a long pup-driven flow. Args: `sessionId`. Returns: `{sessionId, hwnd, lowered, error?}`. - -### Screen Recording - -> **CRITICAL — pick the right verb.** Claude frequently mistakes "record a pup" / "record this Chrome window" / "record the page" for desktop recording. They are NOT the same. Picking wrong gives a black/wrong-content `.webm` because the pup window may not be in the OS foreground when the desktop recorder snaps frames. - -#### Decision tree (read this BEFORE you reach for any record command) - -| User said... | Use | -|---|---| -| "record a pup" / "record this pup window" / "record the tab" / "record the page" / "record this Chrome window I just opened" | **`browser_record_start`** — tab-scoped via CDP, captures the pup tab regardless of foreground state, ~50 fps actual at 30 fps target, single `.webm` | -| "record my screen" / "record what I'm doing" (and Hydrogen is available, which is most of the time) | **Hydrogen's built-in:** `recording start --share screen` — same `getDisplayMedia` pipeline but already wired into Hydrogen with native UI. Prefer this for casual "record my screen" asks. | -| "record everything on screen including KiCad / Fusion / OS dialogs / multiple apps switching" | **`desktop_record_start`** with `confirmDesktopNotTabRecording: true` (required arg — see below) | -| "record a pup tab AND give me a parallel desktop video at the same time" (e.g. ralph-loop screenshot tests with simultaneous narration) | **Both at once:** `browser_record_start` on the tab AND `desktop_record_start` on the desktop. Adom-desktop's desktop recorder exists for this case — Hydrogen's recorder can't run while it's busy doing tab capture, so adom-desktop fills that gap. | - -#### Why this matters - -- **Wrong verb → wrong content.** desktop_record_* captures whatever is on the user's monitor at frame time. If the pup window is occluded behind your IDE / Slack / their email client, the resulting clip shows YOUR app, not the pup tab. Even with `browser_raise_os_window` first, the user's natural alt-tab activity covers the pup window within seconds. -- **Hydrogen already has desktop recording.** Most "record my screen" asks should just be Hydrogen's recorder. Adom-desktop's `desktop_record_*` is mostly redundant — it shines exactly when Hydrogen is busy capturing a tab and you need a parallel desktop angle. - -#### The guardrail - -`desktop_record_start` REQUIRES `confirmDesktopNotTabRecording: true`. Without it, the call returns `errorCode: desktop_record_needs_confirmation` with a long error string explaining the alternatives (browser_record_start, Hydrogen, or pass the confirm). This is intentional friction — re-read the error and confirm you really want full-desktop, not pup-tab. - -#### Output summary - -| Mode | Commands | HUD on screen | Concurrent? | Output | -|---|---|---|---|---| -| **Tab** (one pup tab's viewport) | `browser_record_*` | NO (the user already sees the tab) | YES (one per tab) | Single finished `.webm` (real-time playback, ~50fps actual) | -| **Desktop** (full screen) | `desktop_record_*` | YES — always-on-top control panel with reason + manual stop | No (one at a time) | Single finished `.webm` | - -#### Desktop recording (HUD) - -The recorder is a **visible Chrome window** in the bottom-right corner showing recording status, the stated reason, and a manual Stop button. The HUD persists across multiple short clips (a "session"); only `desktop_recorder_close` (or 5 min idle) dismisses it. +### Browser automation (`browser_*`, Puppeteer/pup) — bridge-owned; light pointer -**Two REQUIRED args on `desktop_record_start`:** -- `reason` — string, why you're recording (renders in HUD title + sidecar `.json`) -- `confirmDesktopNotTabRecording: true` — boolean acknowledgement that you understand this captures the WHOLE desktop, NOT a pup tab. The bridge rejects calls without this with `errorCode: desktop_record_needs_confirmation` and a hint pointing at `browser_record_start` (for pup tab) or Hydrogen (for casual screen recording). +The pup bridge is **cloud-owned** and ships its OWN full skill (its wiki page / skill pkg — `bridge_list` names it; `adom-desktop sync_skills` installs it). This page keeps only the pointer plus first-touch examples; sessions/profiles, credential vault, downloads, recording, CDP details are the bridge's skill. -```bash -# Start a desktop clip (HUD opens with reason; recording begins). -# Note: confirmDesktopNotTabRecording: true is REQUIRED — without it the -# bridge rejects with errorCode: desktop_record_needs_confirmation. -adom-desktop desktop_record_start '{ - "reason":"Parallel desktop angle alongside Hydrogen ralph-loop tab capture", - "confirmDesktopNotTabRecording": true, - "monitor":"primary", - "fps":30, - "audio":false -}' -# → { "ok":true, "recordingId":"rec-1", "filePath":"…\\rec-desktop-….webm", … } - -# … drive KiCad/Fusion/etc … - -# Stop this clip — HUD stays open, ready for next clip -adom-desktop desktop_record_stop '{"recordingId":"rec-1"}' -# → { "ok":true, "filePath":"…\\rec-desktop-….webm", "sizeKB":4231, "durationMs":18432 } - -# (Optional) Record more clips in the same session — reason persists, but -# confirmDesktopNotTabRecording must be passed every time. -adom-desktop desktop_record_start '{ - "reason":"Parallel desktop angle alongside Hydrogen ralph-loop tab capture", - "confirmDesktopNotTabRecording": true -}' -# … etc … - -# When done: close the HUD -adom-desktop desktop_recorder_close -# → { "ok":true, "sessionSummary":{"clipCount":3, "totalSizeKB":12390, "reason":"…", "clips":[…]} } - -# Pull each WebM to Docker (already a finished video — no muxing needed) -adom-desktop pull_file '{"filePaths":["C:\\…\\rec-desktop-….webm"], "saveTo":"/tmp"}' -ffprobe -v error -show_entries stream=codec_name,duration -of default=nw=1 /tmp/rec-desktop-….webm -# → codec_name=vp9, duration=18.432 -``` - -Other commands: -- `desktop_recorder_open '{"reason":"…"}'` — open the HUD without starting a clip (so the user knows ahead of time you're about to record) -- `desktop_record_status` — `{hudOpen, reason, currentRecording, clipsThisSession}` -- `desktop_record_list` — completed `.webm` files with sidecar metadata (incl. reason) -- `desktop_list_monitors` — primary monitor info (v1.1: full multi-monitor enumeration) - -The user can click Stop in the HUD at any time — that converges through the same code path as your `desktop_record_stop` call. Don't fight it; if the user stops manually, treat the clip as complete and react accordingly. - -#### Tab recording (real-time video, no HUD, concurrent) - -**For smooth video — narrated demos, motion-heavy scenes, animations, anything the user will watch end-to-end — use `browser_record_start/stop` NOT a `browser_screenshot` loop.** The screenshot loop maxes out around 3 fps and reads as choppy. - -**Mechanism:** CDP `Page.startScreencast` (Chrome's native compositor-driven JPEG-frame stream) with `Page.screencastFrameAck` per-frame backpressure. Each frame is written to disk with a wall-clock timestamp, then ffmpeg muxes into a single VP9 `.webm` using the concat demuxer with **per-frame durations** so playback matches real wall time. Tab-scoped at the protocol level (the CDP session is bound to a specific Page target — no cross-tab/cross-Chrome leakage). - -**FPS expectations.** Chrome's compositor pushes a frame each time the page paints. `everyNthFrame` is computed from the requested fps (e.g. 30 fps → every 2nd paint frame). On rAF-animated pages with the tab foregrounded, expect actual fps to **meet or exceed the target** — measured 49.7 fps actual at 30 fps target on a 2.5K viewport. On occluded windows Chrome paint-throttles, so always raise the OS window first. - -**Why not MediaRecorder + getDisplayMedia?** Tested — Chrome 146 crashes when `getDisplayMedia({preferCurrentTab:true})` is called via Puppeteer (known upstream bug, [puppeteer #13478](https://github.com/puppeteer/puppeteer/issues/13478)). `Page.startScreencast` is Chrome's other native video pipeline; it's stable and produces equivalent output (compositor frames as JPEGs). - -Output is a single finished **`.webm`** file — no tar, no concat staging exposed to callers, no Docker-side mux. Just `pull_file`. - -**Raise the OS window first.** Chrome paint-throttles occluded windows, so calling `browser_record_start` on a backgrounded tab produces stutters. Always: +Most-used verbs: ```bash -adom-desktop browser_raise_os_window '{"sessionId":"align"}' # surface pup tab to OS foreground -adom-desktop browser_record_start '{"sessionId":"align","fps":30}' +adom-desktop browser_open_window '{"sessionId": "s1", "profile": "p1", "url": "https://example.com"}' +adom-desktop browser_navigate '{"sessionId": "s1", "url": "https://example.com/page2"}' +adom-desktop browser_screenshot '{"sessionId": "s1"}' +adom-desktop browser_eval '{"sessionId": "s1", "expression": "document.title"}' +adom-desktop browser_fetch_url '{"url": "https://example.com/data.json"}' ``` -**Anti-throttle launch flags (already baked into pup).** Pup spawns Chrome with four flags that suppress most of Chrome's background throttling so recordings don't collapse to ~1 fps when the user alt-tabs to another app: +`browser_readiness` is the read-only probe (never spawns or downloads). Full details: the bridge's own skill via `sync_skills`. -- `--disable-renderer-backgrounding` -- `--disable-backgrounding-occluded-windows` -- `--disable-background-timer-throttling` -- `--disable-features=CalculateNativeWinOcclusion` +### Screen recording — AD's desktop recorder (tab recording is pup's) -Verified: with all four flags, `browser_record_start` on a window completely covered by Notepad still produces **29.65 fps actual at 30 fps target**. - -> **Caveat: pup sessions launched in a bridge version older than v1.3.32 don't have these flags.** Chrome launch flags apply only at process spawn — they cannot be added retroactively. If `fpsActual` reports ≤1 even with `browser_raise_os_window` first, the session's Chrome was launched with an old flag set. Fix: close + reopen the session (`browser_close '{"sessionId":"<id>"}'` then `browser_open_window`) so it spawns a fresh Chrome with the current flags. The `kicadVersionUsed`-style proof: there's no per-session "flags" reporter yet, so the reliable test is the recording fps itself. - -Multiple recordings on different tabs run in parallel — each tab has its own capture loop. +AD owns DESKTOP recording: a visible HUD opens with the REASON you supply, so the user always sees that recording is on and why. Full guide: the `adom-desktop-recording` skill. ```bash -# Record three tabs in parallel -for TAB in tab-1 tab-2 tab-3; do - adom-desktop browser_record_start "{\"sessionId\":\"align\",\"tabId\":\"$TAB\",\"fps\":30}" -done - -# … drive each tab through its scenario … - -# Stop and pull each (already a finished webm — no muxing step on Docker) -for REC_ID in rec-tab-1 rec-tab-2 rec-tab-3; do - STOP=$(adom-desktop browser_record_stop "{\"sessionId\":\"align\",\"recordingId\":\"$REC_ID\"}") - WEBM=$(printf '%s' "$STOP" | jq -r '.output | fromjson | .filePath') - adom-desktop pull_file "{\"filePaths\":[\"$WEBM\"],\"saveTo\":\"/tmp\"}" -done +adom-desktop desktop_recorder_open '{"reason": "record the DRC walkthrough for review"}' +adom-desktop desktop_record_start '{"confirmDesktopNotTabRecording": true}' +# ... drive the desktop ... +adom-desktop desktop_record_stop '{}' +adom-desktop desktop_record_list '{}' +adom-desktop desktop_recorder_close '{}' ``` -Args: `sessionId` (required), `tabId` (optional — defaults to active tab), `fps` (default 30; note the actual ceiling is ~10-15 fps), `quality` (default 85, 1-100 JPEG quality), `maxDurationMs` (default 600000). - -Returns from `browser_record_stop`: `{recordingId, sessionId, tabId, filePath, sizeKB, durationMs, frameCount, fpsTarget, fpsActual, stopReason}`. - -Other commands: -- `browser_record_status '{"sessionId":"align"}'` — list active tab recordings with live `frameCount`, `fpsTarget`, `fpsActual`, `durationMs`, `sizeBytesApprox` -- `browser_record_list` — completed `.webm` files on disk with sidecar metadata -- Tab close auto-stops any recording for that tab (mux fires, file finalizes, you don't have to call `browser_record_stop` first) - -**Notes & gotchas:** -- The desktop HUD itself is captured by the desktop recording (it's a visible window). Same as Hydrogen — accepted cost for the visible-status UX. The window is small + bottom-right. -- ffmpeg is **required** on Windows for tab recording (the mux step). Install via `winget install Gyan.FFmpeg`. Bridge logs the detection result at startup. -- `maxDurationMs` defaults to 600000 (10 min) for safety; raise it for longer recordings. -- On bridge shutdown (graceful), active recordings are drained — the mux runs and you get a valid `.webm`. -- The mux file (`<rec>.webm`) is produced by ffmpeg and has a valid `format=duration`. The frames staging dir (`<rec>.frames/`) is deleted after a successful mux. +`desktop_record_start` without `confirmDesktopNotTabRecording: true` refuses with `errorCode: desktop_record_needs_confirmation` — that guard exists because recording a BROWSER TAB is the pup bridge's separate feature (`browser_record_start` etc., see pup's own skill). ### Filesystem primitives — `desktop_list_files` / `desktop_watch_files` / `desktop_pull_glob` (v1.5.0+)
dist/linux/adom-desktop⋯ 1 unchanged line ⋯
package.json+1−1@@ -1,6 +1,6 @@ { "slug": "adom-desktop", - "version": "1.9.173", + "version": "1.9.174", "type": "app", "brief": "Adom Desktop is a small operating system for AI on your Windows PC: it gives an AI real hands on your native desktop, including your signed-in browser, KiCad, Fusion 360, the shell, files, windows and screenshots. It provisions its own Node and Python so bridges run without touching your setup, and it talks straight to your machine. It ships with Hydrogen Desktop and installs standalone too, so if you drive your PC with Claude Desktop or another AI, this is what gives that AI hands.", "description": "Install this package in a cloud or Hydrogen Desktop container, not on the PC. It delivers two things: the Linux adom-desktop CLI (linked onto PATH) and the Claude skill set that teaches an AI every desktop verb (shell, files, screenshots, windows, notifications, KiCad, Fusion 360, browser automation). The CLI drives the Adom Desktop app running on your Windows PC over the relay; the Windows app itself is the separate installer in the Download box, not part of this package. Run adom-wiki pkg update to keep the CLI and skills current.",
page.json+1−1@@ -1,6 +1,6 @@ { "slug": "adom-desktop",- "version": "1.9.173",+ "version": "1.9.174", "type": "app", "description": "Install this package in a cloud or Hydrogen Desktop container, not on the PC. It delivers two things: the Linux adom-desktop CLI (linked onto PATH) and the Claude skill set that teaches an AI every desktop verb (shell, files, screenshots, windows, notifications, KiCad, Fusion 360, browser automation). The CLI drives the Adom Desktop app running on your Windows PC over the relay; the Windows app itself is the separate installer in the Download box, not part of this package. Run adom-wiki pkg update to keep the CLI and skills current.", "keywords": [
skills/adom-desktop/SKILL.md+39−589@@ -33,115 +33,6 @@ The `adom-desktop` binary is a single Rust CLI that does everything: The relay server runs in the Docker container. The Adom Desktop app runs on the user's PC and connects via WebSocket. -## Recent additions (v1.8.103 → v1.8.125) - -For readers of older copies of this skill: a lot has landed in the last -~14 releases. Quick summary so you don't try to call out to features that -don't exist or miss ones that do. - -- **Embedded `--start-hidden` honored + window preference** (v1.8.119+) — - when HD spawns AD with `--embedded --start-hidden`, AD now stays hidden - (v1.8.102–118 force-showed it). The release default respects the launch - flags; the machine owner can opt into always-show via the Settings - checkbox (config `always_show_window_on_launch`). Same preference pattern - as shell auto-approve. - -- **Popup-risk pre-flight on shell commands** (v1.8.117+) — `shell_execute` - and cmd-interpreter `run_script` are scanned BEFORE approval/execution - for `start <target>` invocations whose target doesn't resolve. Those used - to flash a modal "Windows cannot find '<x>'" dialog on the user's desktop - (no silent mode exists — the box is cmd/ShellExecute's own UI). Now AD - refuses with `errorCode: "popup_risk"` + a reprimand `_hint` pointing at - `find_exe` / `launch` / `desktop_open_url`. If your command creates its - own target mid-command and the scan misjudged it, re-send with - `{"allowPopupRisk": true}`. http/https URL targets pass; cmd builtins - pass; `copy x y && start y` patterns pass automatically. - -- **Shell auto-approve is a user preference** (v1.8.116+) — per-command - approval is the default everywhere now. Durable opt-in via AD's Settings - checkbox or `shell_auto_approve '{"permanent": true}'`; HD passes - `--shell-approval <prompt|1h|24h|permanent>` when spawning AD embedded - (session-only) and owns the setting while embedded. The old v1.8.44 - embedded-permanent default (and its leak into standalone sessions) is - gone — see "Shell auto-approve is a user preference" below. - -- **Registry-driven install** (v1.8.115+) — `adompkg install adom-desktop` - (with `?org=adom` on raw packages-API calls) now serves the current - release: the V2 wiki page registers a real adompkg version per release - and its manifest carries a server-derived `assets` map (url + sha256 per - platform) for the installer binaries. The kicad v0.8.6 verbs - (`kicad_export_*`, modifier-combo `kicad_send_key`, `kicad_open_3d_viewer` - editor arg), puppeteer v1.0.6 native-drive changes - (`native_relaunch_required` flow), and the embedded-mode fixes all reach - end-user installers starting with this release. - -- **`desktop_relaunch_self` + `desktop_relauncher_status`** (v1.8.103+) — - visible self-restart helper. AD spawns a separate signed Rust GUI - (`ad-relauncher.exe`) in HOST context that owns the entire swap-binary- - and-restart sequence: preflight → stop AD → atomic deploy → CreateProcessW - launch → verify reachable + PE FileVersion + window visible. Has its - own HTTP API on `:47210`. Don't run shell glue for AD relaunches; use - this. Full recipe: [`skills/AD_RELAUNCH.md`](AD_RELAUNCH.md). - -- **`restart`** (v1.9.90) — restart AD **in place** on a deployed/production - box: stop → relaunch the SAME installed exe → verify, with **no redeploy**. - Use this — not `desktop_relaunch_self`, which redeploys from a dev build tree - and silently no-ops on a normal install — when a headless / RDP / fleet box - needs AD cycled from the relay. Poll `desktop_relauncher_status` until - `overall` is `done_ok`. Pair with `--target <box>` for a specific machine. - The self-updater is relay-driveable too (`update_status`, `update_check`, - `apply_update`, `update_set_mode {auto|prompt|off}`) so a fleet box a version - behind can be updated without a human clicking the footer banner (v1.9.87+). - -- **Auto-update via wiki manifest** (v1.3.39+; single canonical wiki.adom.inc). - AD polls a `version.json` manifest every 4h + on launch from - `https://wiki.adom.inc/api/v1/pages/adom-desktop/files/version.json`. - Dead-end guard probes the `installer_url` before flipping - `update_available` (HEAD + Range-GET fallback for the wiki's HEAD-404 bug). Footer banner Install Now / Later; - user picks `prompt` / `auto` / `off` in Settings. SHA256-verified - silent NSIS install + autostart relaunch. - -- **`updater_get_status` payload extension** (v1.8.112+). `UpdaterStatus` - now includes `manifest_url` (the URL that served the cached manifest) - and `manifest_urls` (full priority chain). Sibling apps that read this - endpoint can show users which wiki AD is checking. The Build-details - tooltip on the footer version chip surfaces all of it + a clickable - link to the wiki page. - -- **Asset naming convention** (v1.8.113+). The full AD GUI installer and - the CLI companion binary now disambiguate cleanly in the wiki: - - **Full AD GUI:** `Adom.Desktop_<ver>_<os>_<arch>.<ext>` - Example: `Adom.Desktop_1.8.114_windows_x64-setup.exe` - - **CLI companion:** `adom-desktop-cli_<ver>_<os>_<arch>[.exe]` - Example: `adom-desktop-cli_1.8.114_linux_x86_64` - The pre-v1.8.113 Linux name `adom-desktop-linux` ships as a byte- - identical alias through v1.8.116. After install, the binary on PATH is - still `adom-desktop` — the wiki name disambiguates fetch time only. - -- **`version.json` `.cli` slot** (v1.8.113+). Additive schema — pre-v1.8.113 - readers ignore. Each populated platform has `binary_url` / `binary_sha256` - / `binary_size`. Today the manifest carries `cli.linux_x86_64` and - `cli.windows_x64`. HD / sibling apps should prefer this typed lookup - over hard-coding filenames. - -- **Windows CLI bundled with AD** (v1.8.114+). Installing AD drops - `adom-desktop-cli.exe` in `%LOCALAPPDATA%\Adom Desktop\` and adds the - install dir to user PATH (HKCU, no UAC) via the NSIS POSTINSTALL hook. - After install, `adom-desktop-cli screenshot_window …` works from any - fresh PowerShell / cmd / Claude Desktop terminal. The CLI auto-finds - the local AD via direct-API probe at `127.0.0.1:47200` — no relay - required for verbs that don't need streaming. - -- **CLI direct-API probe + relay self-detect** (v1.8.114+). The CLI's - probe drops the dead `host.docker.internal` candidate (we don't use - Docker Desktop anymore), adds WSL2 `/mnt/c/Users/*/.adom/direct-api-port` - DrvFs glob for AD's discovery file, and adds a NAT-mode WSL2 fallback - that resolves the Windows host IP from `/etc/resolv.conf`. The CLI's - `serve` subcommand probes `127.0.0.1:8766/health` before binding and - skips with a friendly message if another relay is already answering - (override with `--force-bind`). Both fixes matter for the mirrored- - networking WSL2 setup HD uses. - ## Starting the Relay Server The relay must be running before the desktop app can connect. **In a container, start it SUPERVISED** (v1.9.168+): @@ -598,188 +489,37 @@ Reference implementations: [`scripts/sample-bridges/hello-python/server.py`](../ - Full guide + copy-paste examples per type: the **adom-desktop-notify** skill ([`skills/NOTIFY.md`](NOTIFY.md)). - `notify_response` -- Poll which toast button the user clicked, by correlation `id`. `{pending:true}` until they click; `{pending:false, action}` once they do (consumed on read). -### KiCad Tools - -KiCad supports **multi-version side-by-side installs** — KiCad 9 and KiCad 10 (and older versions) can coexist on the same machine. Every `kicad_open_*` (and `install_*`, `run_drc`) command accepts an optional `kicadVersion` arg (e.g. `"9.0"` or `"10.0"`). Omit it to use the **newest installed** version. The success response includes `kicadVersionUsed` so you can confirm which install actually ran. - -- `kicad_list_versions` -- List every installed KiCad version with their paths and which is the default (newest). Run this first if you're unsure what's available. - - Example: `adom-desktop kicad_list_versions` - - Returns: `{versions: [{version, base_dir, kicad_exe, default}], default: "10.0", count: 2}` -- `kicad_install_symbol` -- Send a .kicad_sym file and install it as a library. Args include optional `kicadVersion`. -- `kicad_install_library` -- Install a symbol, footprint, or 3D model library. Each KiCad version has its own sym-lib-table / fp-lib-table — pass `kicadVersion` to target a specific install (otherwise installs into the newest version's tables). -- `kicad_open_board` -- Open a .kicad_pcb file. Args: `filePath`, optional `kicadVersion`. Example: `adom-desktop kicad_open_board '{"filePath":"C:/foo.kicad_pcb","kicadVersion":"9.0"}'` -- `kicad_open_schematic` -- Open a .kicad_sch file. Args: `filePath`, optional `kicadVersion`. -- `kicad_open_symbol_editor` -- Open the Symbol Editor. Optional `kicadVersion`. -- `kicad_open_footprint_editor` -- Open the Footprint Editor. Optional `kicadVersion`. -- `kicad_open_3d_viewer` -- Open the 3D Viewer (works from Footprint Editor or PCB Editor). Optional `kicadVersion`. -- `kicad_run_drc` -- Run Design Rule Check on a board via kicad-cli (headless, ~1-3s, structured JSON). v1.7.12+: prefer `kicad_lint_board` which wraps DRC + file-format sanity + schematic-parity in one structured response. -- `kicad_run_erc` -- **v1.7.12+** Run Electrical Rule Check on a schematic via kicad-cli. Mirrors `kicad_run_drc` for `.kicad_sch`. Catches unconnected pins, duplicate references, hierarchical-label mismatches, no-driver nets. Headless, ~1-3s. -- `kicad_lint_board` -- **v1.7.12+** Comprehensive headless pre-flight for a `.kicad_pcb` via kicad-cli: file existence + magic-byte sniff + DRC with `--severity-all` + `--all-track-errors` + `--schematic-parity` (default on). Returns structured `{data:{violations[], summary, fileFormat, schematicParityChecked}, _hint}`. Args: `filePath`, optional `schematicParity` (default true). **USE THIS BEFORE kicad_open_board** for any file you didn't just generate — catches DRC errors, parse issues, format-upgrade-needed BEFORE the GUI opens and possibly hangs on a modal dialog. -- `kicad_lint_schematic` -- **v1.7.12+** Comprehensive headless pre-flight for a `.kicad_sch` via kicad-cli: file existence + magic-byte sniff + ERC with `--severity-all`. Same structured shape as `kicad_lint_board`. **USE BEFORE kicad_open_schematic** for unfamiliar files. -- `kicad_lint_library` -- **v1.7.13+** Validate a library file or directory before install. Accepts a single `.kicad_sym`, `.kicad_mod`, `.pretty` directory, or directory containing `.kicad_sym` files. Parses S-expression structure (no kicad-cli call — `sym/fp upgrade` has no `--dry-run` and would rewrite). Returns symbol/footprint inventory + per-file `fileVersion` + tiered `_hint`. **USE BEFORE kicad_install_library / kicad_install_symbol / kicad_install_footprint** to catch malformed files, outdated formats, or wrong-filetype-renamed-with-.kicad_sym surprises in ~100ms. -- `kicad_format_upgrade` -- **v1.7.13+** Upgrade a KiCad file to the current format via `kicad-cli {pcb,sch,sym,fp} upgrade`. Auto-detects `kind` from extension if omitted. MUTATES the file in place (kicad-cli has no `--dry-run`). Common flow: lint_library reports outdated `fileFormatVersions` → surface to user → ask permission → call this. After upgrade, ANY OPEN KiCad windows holding this file must be closed + reopened. - -#### KiCad bridge — Phase 2.ac additions (v1.7.14+, via `kicad_bridge_call`) - -Three additions completing the Phase 2.ab loose ends: - -- **`verify_loaded {kind, expectedName?}`** — reads the Symbol/Footprint Editor frame title to confirm whether a load succeeded. `kind` is `"symbol"` or `"footprint"`. Optional `expectedName` adds a `matchesExpected` field comparing the loaded item against your target. Use AFTER `select_and_load_*` to confirm the symbol/footprint actually loaded (the load itself is "unverified" until you check). +### KiCad (`kicad_*`) — bridge-owned; light pointer -- **`get_net_topology {netName}`** — net-graph traversal: returns every footprint pinning the named net, plus the OTHER nets each of those footprints bridges. `{padCount, componentCount, components:[{ref, value, padsOnNet, otherPads:[{padName, netName}]}], bridgedNets}`. Use for "trace VCC outward from its source through the components it touches" analyses. +The KiCad bridge is **cloud-owned** and ships its OWN full skill (wiki pkg `adom/adom-desktop-kicad-bridge`). This page keeps only the pointer: run `adom-desktop sync_skills` after connecting and the full KiCad guide (project workflows, three-tier `kicad_open_*` routing, DRC recipes, library install) lands in `~/.claude/skills/`. -- **`select_and_load_{symbol,footprint} {name, useSendInput?: false}`** — v0.7.0 adds optional `useSendInput:true`. When `true`, escalates from `wx.PostEvent` (default; safe but may not propagate to KiCad's TOOL_DISPATCHER) to real Win32 `SendInput` with the proven `Enter` → `Ctrl+Shift+E` sequence. SendInput briefly STEALS foreground focus AND requires the Symbol Editor to be the foreground window before keystrokes dispatch — both default and SendInput paths are still "best-effort", so ALWAYS pair with `verify_loaded` to confirm. If verify_loaded reports `hasLoaded:false`, fall back to the legacy `kicad_open_symbol_editor '{"symbolName":"..."}'` path which uses out-of-process PowerShell SendKeys + mouse-click on the search box (proven but heavyweight). -- `kicad_close` -- Close all KiCad windows. Args: `force` (optional bool — skip graceful close, just taskkill) +Most-used verbs to get moving: -**When to specify `kicadVersion`:** -- The user asks to open something "in KiCad 9" / "in KiCad 10" / "in the older version" — pass that explicitly. -- The user is regression-testing across versions (open in 9, screenshot, close, open in 10, screenshot, compare). -- A library was installed under a specific version's tables and the user wants to use it. - -If you pass an invalid `kicadVersion`, the bridge returns `available_versions` and a `_hint` listing what IS installed — surface that to the user. - -#### KiCad reverse bridge (v1.7.5+) — direct in-process control - -KiCad v2 Phase 2 introduces a **reverse bridge**: an HTTP/JSON-RPC server runs *inside* every KiCad GUI process (kicad.exe project manager, pcbnew.exe PCB+Footprint editors, eeschema.exe Schematic+Symbol editors) once they restart after install. This lets adom-desktop introspect frames + post menu commands directly, bypassing UI screen scraping for everything the bridge supports. - -**Auto-installation is transparent.** The first `kicad_*` command of every bridge boot auto-deploys two files (`adom_bridge.py` + `usercustomize.py`) into `%USERPROFILE%\Documents\KiCad\<ver>\3rdparty\Python311\site-packages\` for every installed KiCad version. The response includes a `pluginAutoInstall` field with full details: - -```json -{ - "pluginAutoInstall": { - "triggered": true, - "summary": "installed" | "updated" | "already-current" | "partial-failure" | "skipped", - "payloadVersion": "0.1.0", - "perVersion": [{"kicadVersion":"10.0", "action":"installed", "userSitePath":"...", "filesWritten":[...], "_hint":"..."}], - "runningKiCad": [{"exeName":"eeschema.exe","pid":12345}], - "_hint": "Plugin successfully INSTALLED. KiCad currently running... ask user to restart KiCad..." - } -} +```bash +adom-desktop kicad_open_board '{"path": "C:/projects/board/board.kicad_pcb"}' +adom-desktop kicad_screenshot_all '{}' # every KiCad window incl. error dialogs, as an array +adom-desktop kicad_run_drc '{}' +adom-desktop kicad_window_info '{}' +adom-desktop kicad_open_symbol_editor '{"library": "Device"}' ``` -If installation fails, the `_hint` field tells you exactly what to report and how to recover. Failure modes: `payload_missing` (bundle regression), `user_site_unavailable` (no USERPROFILE), `permission_denied` (Documents dir uncreatable — likely OneDrive sync), `copy_failed` (KiCad has the .py locked — ask user to close KiCad), `no_kicad_installed` (offer `desktop_install_kicad`), `disabled_via_env`. - -**Currently-running KiCad processes don't pick up the bridge until restarted** (Python init runs once per process). If `pluginAutoInstall.runningKiCad` is non-empty AND summary is `installed`, surface the `_hint` to the user — they need to close + reopen KiCad to activate the bridge on those instances. - -- `kicad_bridge_status` -- Enumerate every running KiCad GUI process with the adom_bridge plugin loaded. Reads `%TEMP%\adom-kicad-bridge-<exe>.json` discovery files + probes each `/status`. - - Example: `adom-desktop kicad_bridge_status` - - Returns: `{plugins: [{exeName, pid, port, version, alive, uptimeMs, requestCount}], summary: {total, alive}, pluginAutoInstall: {...}}` - - If `summary.alive=0` after a fresh KiCad launch, check `pluginAutoInstall._hint` — the install probably failed and you have a specific recovery to offer. - -- `kicad_bridge_call` -- Generic passthrough: call any RPC method on the bridge for a specific KiCad exe. v1.7.6+ supports multi-instance disambiguation via `pid` arg. - - Args: `exeName` (required: `kicad` / `pcbnew` / `eeschema` / `kicad-cli`), `method` (required), `params` (optional dict), `timeout` (optional float seconds), `pid` (optional int — pin to a specific process; otherwise the newest alive instance is picked) - - Example — open Symbol Editor from running Schematic Editor: - ```bash - adom-desktop kicad_bridge_call '{"exeName":"eeschema","method":"wm_command","params":{"frame_index":0,"menu_id":20390}}' - ``` - - Methods (v1.7.6): - - `ping` — `{version, exeName, pid, uptimeMs, requestCount}`. Health check. - - `list_frames` — `[{frameClass, frameTitle, hwnd, menubarTopCount, menubarTotalItems, isShown}, ...]`. Replaces screen scraping. - - `get_menu_ids` — full menu walk per frame: `[{frameClass, frameTitle, menuItems: [{topMenu, label, id, kind, accelerator, helpString}, ...]}, ...]`. - - `wm_command` — posts a Win32 `WM_COMMAND` to the target frame via `PostMessageW(hwnd, WM_COMMAND, menu_id, 0)`. Goes through KiCad's TOOL_DISPATCHER. Args: `{frame_index, menu_id}`. - - `get_board_info` — **pcbnew only**. Returns `{hasBoard, fileName, trackCount, footprintCount, drawingsCount, netCount, copperLayerCount, boundingBoxMm:{widthMm,heightMm,centerXMm,centerYMm}}`. Replaces screenshot-poll for state queries. - - `get_schematic_info` — **eeschema only**. Returns `{available, frames:[{frameClass, frameTitle, hwnd, editorKind, fileName, isUnsaved, isUntitled}], frameCount}`. Title-based until KiCad 11 ships kipy. - - `get_open_editors` — per-process inventory: `{available, exeName, pid, editors:[{kind, frameClass, frameTitle, hwnd, isShown}], editorCount}`. For cross-process aggregator use the `kicad_open_editors` verb instead. - - **`get_pcb_footprints`** (v1.7.7+, pcbnew only) — full footprint list with `{ref, value, libNickname, itemName, fpid, x_mm, y_mm, rotationDeg, layer, isFlipped, isLocked, padCount, boundingBoxMm}` per footprint. Replaces screen-scrape "what's placed where". - - **`get_pcb_layers`** (v1.7.7+, pcbnew only) — layer stackup: `{copperLayerCount, totalEnabledLayers, layers:[{id, name, kind, isCopper}]}` where kind is one of: copper, mask, silk, paste, courtyard, adhesive, edge, fab, comments, user, margin, other. - - **`get_pcb_nets`** (v1.7.7+, pcbnew only) — net list with usage stats: `{netCount, totalTrackLengthMm, nets:[{code, name, padCount, trackLengthMm}]}`. - - **`get_pcb_design_rules`** (v1.7.7+, pcbnew only) — DR summary: `{trackWidthsMm, viaDimensions:[{diameterMm,drillMm}], minClearanceMm, minThroughHoleMm, minViaDiameterMm}`. - - **`navigate_symbol`** (v1.7.7+, eeschema only) — `params:{symbolName}` or `params:{query}`. Sets the wx.SearchCtrl value in the Symbol Editor's library panel + posts EVT_TEXT to activate the live filter. NON-destructive — does NOT load the symbol, only filters the visible library tree. **Replaces the PowerShell SendKeys path used by `kicad_open_symbol_editor` when `symbolName` is passed** — substantially more reliable + doesn't steal foreground focus. - - **`navigate_footprint`** (v1.7.7+, pcbnew only) — `params:{footprintName}` or `params:{query}`. Same shape as navigate_symbol but for the Footprint Editor. - - **`get_pcb_tracks`** (v1.7.8+, pcbnew only) — every track (segments, arcs, vias) with `{type, layer, layerName, widthMm, netCode, netName, lengthMm, startMm, endMm}`. Vias show up here too (with no length); call `get_pcb_vias` for via-specific fields. - - **`get_pcb_pads`** (v1.7.8+, pcbnew only) — every pad across all footprints: `{footprintRef, padName, positionMm, sizeMm, drillMm, netCode, netName, attribute}`. Position is absolute (post-rotation of parent footprint). - - **`get_pcb_vias`** (v1.7.8+, pcbnew only) — every via: `{positionMm, diameterMm, drillMm, viaType, topLayer, topLayerName, bottomLayer, bottomLayerName, netCode, netName}`. viaType is "through" / "blind_buried" / "microvia". - - **`get_pcb_drawings`** (v1.7.8+, pcbnew only) — silk text, edge cuts, dimensions, user-drawn shapes: `{type, layer, layerName, text?, positionMm?, startMm?, endMm?, centerMm?, radiusMm?}`. - - **`get_pcb_zones`** (v1.7.9+, pcbnew only) — copper pour inventory: `{zones:[{netCode, netName, layers, layerNames, priority, isFilled, isKeepout, clearanceMm, minWidthMm, polygonVertexCount, boundingBoxMm}]}`. - - **`get_footprint_connections`** (v1.7.9+, pcbnew only) — `params:{ref}`. Net-graph traversal: for a given footprint reference, returns per-pad connectivity. `{found, ref, value, padCount, pads:[{padName, netCode, netName, connectedTo:[{ref, padName}]}]}`. Each `connectedTo` list excludes the pad itself. - - **`get_drc_markers`** (v1.7.9+, pcbnew only) — read existing DRC markers from the board: `{markerCount, bySeverityCounts, markers:[{severity, severityCode, message, errorText, layer, layerName, positionMm, markerClass}]}`. Does NOT trigger DRC — for that use `kicad_run_drc`; this reads results from the LAST run. - - **`export_svg`** (v1.7.8+, pcbnew only) — multi-layer SVG export via `pcbnew.PLOT_CONTROLLER` (in-process, NO `kicad-cli` subprocess). `params:{outputDir?, layers?, mirror?, plotFrameRef?, useAuxOrigin?, subtractMaskFromSilk?, drillMarks?, filenamePrefix?}`. Default `layers` is every enabled copper layer plus F.Silkscreen + B.Silkscreen + Edge.Cuts. Output files: `<outputDir>/<boardBasename>-<layer>.svg` (or `<boardBasename>-<prefix>-<layer>.svg` if `filenamePrefix` is passed). Returns `{plottedLayerCount, outputDir, files:[{layerId, layerName, filePath, fileSize}]}`. **Use timeout:30+ on the bridge_call** because plotting can take 5-15s. - - **`export_pdf`** (v1.7.8+, pcbnew only) — same shape as export_svg, format=PDF. - - **`select_and_load_symbol`** / **`select_and_load_footprint`** (v1.7.9+, EXPERIMENTAL) — filters the library tree (via `navigate_*`) then attempts to activate the first match. **Known to segfault eeschema on large symbol libraries — only safe on small/test libs.** Phase 2.ab will fix via keystroke simulation. Until then, the default `kicad_open_symbol_editor` / `kicad_open_footprint_editor` paths continue to use the proven PowerShell SendKeys load — they're NOT routed through this method. - -- `kicad_install_plugin` -- **v1.7.6+** Explicitly redeploy the Phase 2 reverse-bridge plugin to every detected KiCad version's USER_SITE. Bypasses the auto-install cache. - - Args: `force` (optional bool, default true) - - When to use: after updating the plugin payload between releases, or to recover from a partial-failure auto-install (read perVersion errorCodes from `pluginAutoInstall` first). - -- `kicad_open_editors` -- **v1.7.6+** Cross-process editor inventory sourced via the bridge (not screen scraping). Hits each alive plugin's `get_open_editors` and concatenates. - - Returns: `{editors:[{kind, exeName, pid, frameTitle, hwnd, isShown}], byKind:{schematic_editor:[…], pcb_editor:[…], symbol_editor:[…], footprint_editor:[…], project_manager:[…]}, summary:{totalEditors, probedPlugins, failedPlugins, byKindCounts}}`. - - When to use INSTEAD of `kicad_window_info`: when you have the bridge available (the data is more structured + cheaper than the screenshot-fallback path). - -### Phase 3 — three-tier routing for `kicad_open_*` verbs (v1.7.6+) - -The legacy `kicad_open_symbol_editor`, `kicad_open_footprint_editor`, and `kicad_open_3d_viewer` verbs now try the bridge first before falling back to legacy WM_COMMAND-from-outside / icon-click / Alt+keystroke. Their responses include: -- `pathway` — which tier won: `"plugin"` (bridge), `"wm_command"` (legacy external post), `"icon_click"` (footprint editor launcher icon), `"alt_keystroke"` (3D Viewer from PCB Editor). -- `pathwaysTried` — ordered list of every tier attempted with success/failure annotations. -- `bridgePid` — if pathway was `"plugin"`, which KiCad process handled it. - -For Symbol Editor specifically, tier-1 has two sub-paths because Symbol Editor is hosted inside eeschema.exe (not as a standalone process): -- `plugin-eeschema-20390` — preferred when eeschema is already alive; sends menu_id 20390 to its Schematic Editor frame (opens Symbol Editor as a sub-window in the existing process). -- `plugin-kicad-20011` — cold-start path; sends menu_id 20011 to kicad.exe Project Manager (spawns a new eeschema.exe with Symbol Editor as top-level). - -**Bug fixed in v1.7.6** (caught via Phase 4 menu catalog audit): `kicad_open_symbol_editor` was sending menu_id 20012 to the project manager, but 20012 is actually **PCB Editor** in KiCad 10. The correct ID is 20011. The legacy WM_COMMAND-from-outside path silently failed for KiCad 10 users (Symbol Editor never opened, sometimes a PCB Editor window flashed instead). v1.7.6 fixes the constant + adds the bridge tier-1 path that uses the proven eeschema-20390 mechanism whenever possible. - -**Key menu IDs catalogued** (see `phase4-results/*.json` in the plan repo for the full ~870-item dump): - -| From frame | Action | menu_id | -|---|---|---| -| Project Manager | Open Schematic Editor | 20010 | -| Project Manager | Open Symbol Editor | 20011 | -| Project Manager | Open PCB Editor | 20012 | -| Project Manager | Open Footprint Editor | 20013 | -| Schematic Editor | Switch to PCB Editor | 20150 | -| Schematic Editor | Symbol Editor | 20390 | -| Schematic Editor | ERC | 20001 | -| Schematic Editor | Annotate Schematic | 20139 | -| PCB Editor | DRC | 20088 | -| PCB Editor | 3D Viewer | 20563 | -| PCB Editor | Footprint Editor | 20567 | -| PCB Editor | Switch to Schematic | 20234 | - -The bridge is a **Phase 2 MVP**. The legacy `kicad_open_*` / `kicad_window_info` / `kicad_screenshot_all` commands still work and remain the primary path for most flows; the bridge is additive. Phase 2.x will add `get_board_info`, `run_drc`, `navigate_symbol/footprint`, `export_svg`, and friends — when those land, this section will list them. - -#### KiCad UI Interaction - -Commands for detecting KiCad windows/dialogs, taking screenshots, clicking, and sending keyboard input. Essential for handling blocking dialogs and automating multi-window KiCad workflows. - -- `kicad_window_info` -- Returns all KiCad windows grouped as `projectManager`, `editors[]`, `modalDialogs[]` with HWNDs and titles. Uses **process-based enumeration** (finds all PIDs running from KiCad's bin dir, then enumerates their windows) — catches every KiCad window regardless of title. Check `hasModalDialogs` to detect blocking save/error prompts. - - Example: `adom-desktop kicad_window_info` - - Returns: `{projectManager: {hwnd, title}, editors: [{hwnd, title}], modalDialogs: [{hwnd, title, ownerHwnd}], hasModalDialogs: bool}` +If KiCad is not installed, the bridge knows how to install it for the user — offer, never point them at a download page. Full details: the bridge's own skill. -- `kicad_screenshot_all` -- Screenshots every KiCad window in one call (editors first, then project manager, then dialogs). Uses same process-based enumeration as `kicad_window_info`. Images downscaled to ≤1568px — use relative coords (0.0-1.0) for clicking, they're scale-independent. READ each screenshot to see what's on screen. - - Example: `adom-desktop kicad_screenshot_all` - - Returns: `{screenshots: [{type: "main"|"editor"|"dialog", title, hwnd, savedTo, sizeKB}]}` +### Fusion 360 (`fusion_*`) — bridge-owned; light pointer -- `kicad_send_key` -- Send a keystroke to a KiCad window. **Preferred way to dismiss dialogs**: enter to confirm, escape to cancel, tab to cycle buttons. Use click only if tab order doesn't reach the right button. - - Args: `key` (required: "enter", "escape", "tab", "space", "f1"-"f12", or single char), `hwnd` (optional — if omitted, sends to foreground KiCad window) - - Example: `adom-desktop kicad_send_key '{"key": "enter"}'` (dismiss dialog) - - Example: `adom-desktop kicad_send_key '{"key": "escape", "hwnd": 12345}'` (target specific window) +The Fusion 360 bridge is **cloud-owned** and ships its OWN full skill (wiki pkg `adom/adom-desktop-fusion-bridge`). Run `adom-desktop sync_skills` and the full guide (cloud files, import/export, add-in troubleshooting, APS search) lands in `~/.claude/skills/`. -- `kicad_click` -- Click at coordinates within a KiCad window. Relative coords (0.0-1.0) are scale-independent — estimate position as percentage from screenshot. To click a button at pixel (px,py) in an image of size (W,H), use `x=px/W, y=py/H`. - - Args: `hwnd` (required), `x` (required), `y` (required), `relative` (optional bool, default true — 0.0-1.0 range) - - Example: `adom-desktop kicad_click '{"hwnd": 12345, "x": 0.5, "y": 0.8}'` +Most-used verbs: -#### KiCad Dialog Detection Workflow - -KiCad has multiple independent windows (project manager, eeschema, pcbnew, footprint editor, symbol editor, 3D viewer). Unlike Fusion which has one main window, KiCad dialogs are owned by specific parent windows. - -**After any open/install command**, check for blocking dialogs: -1. `kicad_window_info` → check `hasModalDialogs` -2. If true: `kicad_screenshot_all` → READ each dialog screenshot -3. Dismiss with `kicad_send_key {"key":"enter"}` (OK) or `{"key":"escape"}` (cancel) -4. For specific button clicks: identify position in screenshot → `kicad_click {"hwnd":<dialog_hwnd>, "x":0.5, "y":0.8}` - -### Fusion 360 Tools - -Fusion 360 uses a two-tier architecture: -- **External bridge** (port 8773, auto-started): handles launch, detection, file opening -- **AdomBridge add-in** (port 8774, runs inside Fusion): required for export, design queries, Electronics/EAGLE commands - -- `fusion_start` -- **PREFERRED.** First-class Fusion 360 startup. Idempotent — safe to call whether Fusion is running or not. Discovers `Fusion360.exe` via webdeploy glob (no hardcoded hashes, no "Windows cannot find" GUI dialogs), verifies path exists before spawn, polls the AdomBridge add-in until it's online, and calls `fusion_dismiss_blocking_dialogs` to clear the startup picker and any other blocking modals. **Auto-dismisses startup dialogs during add-in wait** — the "What do you want to design?" picker and "Recovered Documents" dialog are dismissed every 5 seconds while polling, so the add-in becomes responsive within ~10-15s instead of timing out at 60s. Returns `{addinReady, mainThreadResponsive, dialogsDismissed[], dialogsRemaining[], dismissHint, pid, resolvedPath, elapsedMs, alreadyRunning}`. **Has a hard process-detection gate** — will never spawn a second Fusion process, so the "Multiple instances are not supported" GUI dialog cannot reach an end user. Use this as the very first call before any other Fusion command. -- `fusion_dismiss_blocking_dialogs` -- **First-class blocking-dialog killer.** Whenever any fusion_* command returns `errorCode: fusion_addin_not_responding` (or you see `_hint` mentioning a blocked add-in), call this FIRST before retrying. Two-layer design: (1) deterministic pattern match against known blockers — "What do you want to design?" picker, error toasts ("New design cannot be created..."), "Multiple instances" warning, crash recovery prompts, save prompts — dismisses each with `fusion_send_key escape` targeted at the exact hwnd. (2) AI-in-the-loop fallback — for anything the patterns didn't match, auto-screenshots every remaining Fusion dialog and returns the image paths inline in `remaining[].screenshotPath`, plus an explicit per-hwnd recovery script in `_hint` that you follow mechanically: Read the screenshot with the Read tool, understand what the dialog says, then `fusion_send_key '{"key":"escape","hwnd":<hwnd>}'` (or `"enter"` for OK-only dialogs). Re-call `fusion_dismiss_blocking_dialogs` to verify `mainThreadResponsive=true`, then retry your original command. **Fusion updates break pattern matches constantly, so the AI fallback is the critical path, not the deterministic side.** If you identify a new recurring pattern, add its title substring to `BLOCKING_DIALOG_PATTERNS` in `cli/src/commands.rs` so future sessions handle it automatically. +```bash +adom-desktop fusion_start '{}' # launches Fusion; slow on cold start (minutes) +adom-desktop fusion_get_app_state '{}' # the status/poll verb +adom-desktop fusion_addin_status '{}' +adom-desktop fusion_dismiss_blocking_dialogs '{}' +adom-desktop fusion_send_key '{"key": "Escape"}' +``` -- `fusion_addin_status` -- **Non-blocking** check of the AdomBridge add-in's busy state. Does NOT touch the Fusion main thread — queries the add-in's `/status` HTTP endpoint directly (~5ms). Returns `{busy, busyCommand, elapsedSeconds, requestId, walkProgress}`. Use this to check if a long-running command (walk, search) is in progress **from any bridge/session** before sending new commands. The CLI's auto-recovery path calls this before attempting dialog dismissal — if the add-in is just busy (not dialog-blocked), it returns `main_thread_busy` immediately instead of mashing Escape into a working Fusion. - - Example: `adom-desktop fusion_addin_status '{}'` - - When idle: `{"busy": false}` - - During a walk: `{"busy": true, "busyCommand": "walk_cloud_tree", "elapsedSeconds": 42.3, "walkProgress": {"foldersVisited": 15, "filesFound": 87, "currentFolder": "Molecules/XRP", "queueSize": 8}}` +Fusion launches take 2-4 minutes cold; respect `timeoutSeconds` + `statusVerb` on responses instead of re-issuing. Full details: the bridge's own skill. ### Launching apps (generic) @@ -1698,326 +1438,36 @@ AD auto-updates itself from the wiki manifest, but historically only a HUMAN cli - `update_set_mode {mode: auto|prompt|off}` -- the auto-apply POLICY. `auto` = unattended stage+apply on the 4h/on-launch poll (set this ONCE on headless fleet boxes and they self-update every release with no relay call); `prompt` = show the banner (default, attended); `off` = no checks. - Fleet flow: `update_status --target <box>` → if `updateAvailable` and unattended, `apply_update --target <box>` (or set `update_set_mode {"mode":"auto"}` once). A box must already be on ≥1.9.87 to have these verbs; the one-time bootstrap of an older box still needs a human click OR `auto` mode. -### Browser Automation (Puppeteer) - -Multi-session Puppeteer -- each session gets its own Chrome window. Auto-starts on first command. **Always pass `sessionId` on every command.** - -Two levels of granularity: **window** (one Chromium window per session, own taskbar icon) and **tab** (many tabs per session, one taskbar icon). Use windows when isolation matters, tabs when showing many related views (10 alignment previews, etc.) to avoid flooding the taskbar. - -**Window-level commands** (one session = one Chrome window): -- `browser_rescan` -- (v1.5.1+) Recover orphaned Chrome windows whose CDP socket dropped. Walks every known profile (in-memory + on-disk session files), reconnects via the persisted DevToolsActivePort, then walks every page and rebuilds session entries by parsing the `(session: X)` tag injected into each page's title at launch. Idempotent. Args: `adoptOrphans?` (default false; when true, pages without a tag get attached under a generated sessionId — useful when Chrome has tabs the bridge has never seen). - - **When you need this**: any time `browser_navigate` / `browser_eval` / `browser_screenshot` errors with `errorCode: "session_disconnected"` (the bridge knows about your sessionId but its CDP socket dropped) or `errorCode: "session_not_found"` (after a bridge restart that rebuilt from disk but couldn't reach the live Chrome). The 30s health check now auto-attempts reconnect, so most transient blips recover before you notice — but `browser_rescan` is the explicit recovery primitive. - - Returns: `{ok, rescanned, profilesReconnected, profilesUnreachable, reattached, orphansAdopted, liveSessions, disconnectedSessions, _hint}`. -- `browser_open_window` -- Open a Chrome window. **Default mode (CfT — Chrome for Testing):** args `sessionId` (required), `profile` (required), `url` (required), `freshProfile` (optional), `strictPermissions` (optional, default `false`), `downloadPath` (optional, default `%USERPROFILE%\Downloads`). **Native mode (v1.0.6+, opt-in — drives the user's REAL Chrome/Edge/Brave, actual signed-in profile):** instead of `profile`, pass both `nativeBrowser` (`chrome`/`edge`/`brave` from `desktop_list_browsers`) and `nativeProfile` (profile id: `Default`/`Profile 1`/`Profile 2`). The bridge smart-launches: fresh-launch if the browser is closed (no prompt, returns `launchedFresh:true`), reconnect if it already exposes a debug port, or returns `errorCode:"native_relaunch_required"` (ask the user, re-call with `nativeRelaunch:true`) if it's open without one. If the debug port is enterprise-policy-blocked it returns `errorCode:"native_cdp_blocked"` with a `diagnosis` + `alternatives` (other installed browsers — Edge often works when managed Chrome can't). See the "When pup means 'the user's real Chrome/Edge'" section above for the full flow + routing rules. - - **Full-capability default (v1.6.3+):** every pup session opens with the capabilities chip-fetcher-style flows actually need: - - **Scripted downloads enabled.** `Page.setDownloadBehavior` is called with `behavior: 'allow'` and `downloadPath: %USERPROFILE%\Downloads` (override via the `downloadPath` arg). Re-applied on every main-frame navigation as a safety belt. This unblocks the silent-download-failure scenario: before v1.6.3, JS-triggered downloads (clicks on `<a download>`, `fetch().then(blob → save)`, vendor "Download Symbol" buttons that do `window.location = url`) could be silently dropped by Chrome — no error, no UI, no file. chip-fetcher debugged this for hours before the fix. - - **Clipboard read/write granted.** Paste-into-vendor-form flows work without permission prompts. - - **Notifications + geolocation + MIDI denied.** Chrome doesn't pop up permission dialogs under automation. - - The response includes `defaultPermissionsApplied: true`, `defaultPermissions: ["clipboard-read=granted", "clipboard-write=granted", "notifications=denied", "geolocation=denied", "midi=denied"]`, `downloadsEnabled: true`, `downloadPath: <resolved abs path>`. The path is what `desktop_watch_files` polls by default — the two ends meet without configuration. - - **Opt-out:** pass `strictPermissions: true` to keep Chromium's defaults in place. Rarely needed — only when driving an untrusted site you're auditing. With strict mode, scripted downloads may be silently dropped, clipboard prompts fire, etc. - - **History note:** v1.4.12 used `Browser.grantPermissions(['automaticDownloads', 'notifications', 'clipboardReadWrite'])` — but Chrome 146 rejects `automaticDownloads` as an unknown permission name, taking the whole grant down. v1.6.3 fixes by using `Browser.setPermission` per-permission (per-permission failure tolerance) AND by realizing `Page.setDownloadBehavior` is the actual mechanism for unblocking scripted downloads, not a permission grant. -- `browser_close_window` -- Close a session's Chrome window (closes ALL tabs in that session). Args: `sessionId` -- `browser_list_windows` -- List all open sessions with URL, title, tabCount, active status - -**Tab-level commands** (manage tabs within a session's window): -- `browser_open_tab` -- Add a tab to an existing session. Returns `{tabId, url, title, active}`. Args: `sessionId`, `url` -- `browser_switch_tab` -- Make a specific tab the active one (calls `bringToFront`). Args: `sessionId`, `tabId` -- `browser_close_tab` -- Close a specific tab. Leaves other tabs + session intact. Args: `sessionId`, `tabId` -- `browser_list_tabs` -- List all tabs in a session: `{tabs:[{tabId,url,title,active,errorCount,opener,openerTabId}], activeTabId, count}`. Args: `sessionId`. **Auto-tracks popup tabs since v1.4.7** — tabs the page spawned via `window.open()` / `target="_blank"` / form-submit-with-target=_blank appear here automatically with `opener:"popup"` and `openerTabId` pointing at the tab that triggered them. Tabs Claude opened via `browser_open_window`/`browser_open_tab` have `opener:"user"`. - -**Tab-aware operations** (all accept optional `tabId`; omit = use active tab): -- `browser_navigate` -- Navigate to a new URL. Args: `sessionId`, `url`, `tabId?` -- `browser_screenshot` -- Capture screenshot, auto-resized to <=1568px. Args: `sessionId`, `maxWidth`, `fullPage`, `tabId?` -- `browser_eval` -- Evaluate JS in the page context. Args: `sessionId`, `expr`, `tabId?` -- `browser_input_dispatch` -- **Trusted** click / type / key / move via Chromium's CDP-backed `Input.dispatchMouseEvent` / `Input.dispatchKeyEvent`. Events have `isTrusted: true`, so they pass framework click-handler gates (React/Vue/Svelte) and most vendor anti-automation checks that `browser_eval`-side `dispatchEvent(new MouseEvent(...))` silently fails on. Args: `sessionId?`, `tabId?`, `type` (`click`|`move`|`type`|`key`), plus per-type fields: - - `click`: either `selector` (uses bounding rect — preferred when DOM is stable) OR `{x, y}` viewport coords. Optional `button` (`left`/`right`/`middle`), `clickCount` (1 or 2 for double-click), `delay`, `firstMatch` (default false; see below). - - `move`: `{x, y}` plus optional `steps` for human-like trajectory. - - `type`: `text` to type; pass optional `selector` to focus that element first; optional `delay` between keystrokes. - - `key`: a Puppeteer KeyInput name (`Enter`, `Escape`, `Tab`, `ArrowUp`, `F1`, etc). - - **Smart selector pick (v1.4.8+, modal-scoped in v1.4.9+, plausibility-filtered in v1.4.10+):** when a `selector` matches multiple elements (a class like `button.wg-button--primary` often does — cookie X buttons + duplicate hidden copies + the actual submit), the bridge picks intelligently: - 1. **If a plausible modal/dialog is open** — `<dialog open>`, `[aria-modal="true"]`, `[role="dialog"]`, OR a fixed/absolute element with z-index ≥100 covering >25% of the viewport — AND it contains at least one visible interactive element with text — restrict the candidate set to elements **inside** that modal first. Tiebreak by largest visual area inside it. The page behind a modal is visually inert; clicking elements behind it is almost never what the caller intended. The "plausibility filter" added in v1.4.10 prevents empty overlay containers (Vue-Toastification toast wrappers, notification mount points, full-screen ad scrims) from being mistaken for the real modal — a common false-positive on Vue/React apps that mount toast portals at body level. - 2. Otherwise filter to **visible elements with non-empty text** and tiebreak by **largest visual area**. - 3. Fall back to visible-only, then document-order first as last resort. - - Response includes `matchedCount`, `chosenIndex`, `clickedRect:{x,y,w,h}`, `clickedText`, `insideModal`, `modalDetected`, `modalRoot:{tag,id,cls,z,role,ariaModal}|null` (v1.4.10+ — which DOM node won the modal-detection heuristic, for diagnosing false-positives), and `pickStrategy` (`only-match` | `first-match` | `modal-scoped-largest` | `visible-text-largest` | `visible-text-largest-no-modal-match` | `visible-largest` | `fallback-first-document-order`). **Always sanity-check the response** — if `clickedText` doesn't look right, inspect `modalRoot` to see whether modal detection latched onto the intended overlay or got fooled by a sibling, then refine the selector OR pass `firstMatch:true` to skip the smart pick OR fall back to `{x, y}` coords. - - **Use this whenever a click "lands but does nothing"** — most vendor "Generate Datasheet" / "Download" / "Submit" buttons gate on `event.isTrusted`. After clicking, follow up with `browser_list_tabs` (popups auto-track) to grab any new tab the click spawned. - -- `browser_fetch_url` -- Fetch an arbitrary URL with the session's cookies and return raw bytes. Bypasses Chrome's PDF Viewer wrapper — critical for grabbing PDF binaries from popup tabs where in-page `fetch(location.href)` returns the viewer HTML wrapper (~200 KB stub) instead of the actual PDF (multi-MB). Also handles ZIPs, CAD bundles, anything served as a binary content-type. - - Args: `sessionId?`, `tabId?` (picks cookie context — defaults to active tab), `url` (required), `method?` (default GET), `headers?` (object; Cookie auto-included from session), `body?` (raw string). - - **`saveTo` writes on the DOCKER container's filesystem** (the CLI handles the write after receiving bytes from the bridge). Pass an absolute path. Parent dirs are created. Returns `savedTo` with the canonical absolute path (verified to exist on disk) plus `savedToFilesystem:"container"`. - - **`desktopSaveTo` writes on the WINDOWS desktop's filesystem** (the bridge writes directly via `fs.writeFileSync`). Pass an absolute Windows path (e.g. `C:\\Users\\you\\Downloads\\foo.pdf`). Returns `desktopSavedTo` with the resolved absolute path. Use this when the user wants the file local on the desktop (e.g. dropping a CAD bundle into Downloads). - - **Without saveTo or desktopSaveTo**, returns `bodyBase64` in the response — caller decides what to do. - - Returns: `{ok, bytes, contentType, status, sessionId, tabId, bodyBase64?, savedTo?, savedToFilesystem?, desktopSavedTo?, desktopSaveError?}`. - - **CRITICAL READ-THE-RESPONSE rule:** when you pass `saveTo`, the response's `savedTo` is the actual filesystem path the file was written to. v1.4.8 had a bug where the bridge would respond `savedTo:"/tmp/foo.pdf"` but the file existed nowhere; v1.4.9 fixes this — `savedTo` always reflects a real on-disk path. - - **Recipe — popup PDF capture (the right pattern):** - ```bash - # After browser_input_dispatch click that spawns a PDF popup, READ THE URL - # FROM browser_list_tabs (don't eval against the popup's tabId — popups - # auto-close fast and eval-after-close errors with a structured hint). - LIST=$(adom-desktop browser_list_tabs '{"sessionId":"chip-fetcher"}') - POPUP_URL=$(printf '%s' "$LIST" | jq -r '.tabs[] | select(.opener=="popup") | .url' | head -1) - POPUP_TAB=$(printf '%s' "$LIST" | jq -r '.tabs[] | select(.opener=="popup") | .tabId' | head -1) - - adom-desktop browser_fetch_url "{ - \"sessionId\":\"chip-fetcher\", - \"tabId\":\"$POPUP_TAB\", - \"url\":\"$POPUP_URL\", - \"saveTo\":\"/tmp/datasheet.pdf\" - }" - # → {ok:true, savedTo:"/tmp/datasheet.pdf" (exists), savedToFilesystem:"container", - # bytes:3798336, contentType:"application/pdf", status:200} - ``` -- **Per-call auto-recovery (v1.6.1+):** every tab-aware verb (`browser_navigate / eval / screenshot / input_dispatch / errors / reload / fetch_url`) now **silently reconnects or relaunches** before surfacing any session error. The Docker container should NEVER see "Session closed" / "detached Frame" / "session_disconnected" again unless recovery is genuinely impossible. The recovery ladder: - 1. Live session → return immediately. - 2. Session is `_lostBrowser` (CDP socket dropped after sleep/wake / network blip / puppeteer hiccup): try `attemptReconnectProfile()` which checks (a) the on-disk session file's recorded `cdpPort`, (b) the profile dir's `DevToolsActivePort`, and (c) **scans running Chrome processes for a `--remote-debugging-port=N` matching this profile dir** (this catches the canonical sleep/wake symptom — Chrome alive, CDP serving, but the bridge has no on-disk record). If reconnect succeeds, walk pages and re-attach by parsing the `(session: X)` tag from titles. - 3. Reconnect failed: kill any orphan Chrome processes for this profile, clean the profile lock files (`lockfile` on Windows, `SingletonLock` / `SingletonSocket` / `SingletonCookie` on POSIX, plus stale `DevToolsActivePort`), then relaunch via `launchSession` with the last-known URL from the session file. The sessionId is preserved — the caller gets a transparent recovery. - 4. Total failure (no profile name, no last-known URL, can't launch) → only THEN does the structured `session_not_found` / `session_disconnected` error fire. - - When recovery happens, the response includes `_recovered: true` and `_recoveryReason: "reconnect-no-page" | "relaunch"` so callers can log / metric the recovery rate. - -- **Stale sessionId detection (v1.5.1+):** when auto-recovery (above) genuinely cannot fix the problem (no profile known, no last-known URL, Chrome won't relaunch — rare), a structured error fires. Two error codes you might see: - - **`session_not_found`** — the bridge has no record of this sessionId. Recipe: check `liveSessions` for the right name, or `browser_open_window` if you want to start fresh. - - **`session_disconnected`** — the bridge has the session in memory (and on disk) but auto-recovery couldn't reach its Chrome. Recipe: call **`browser_rescan`** to force a manual recovery pass, or `browser_open_window` with explicit URL to relaunch. - -- **`browser_rescan`** (v1.5.1+, enhanced v1.6.1+) — explicit recovery primitive. v1.6.1 adds a running-process scan: walks all Chrome processes whose `--user-data-dir` is under the puppeteer profiles dir, even if the bridge has no in-memory or on-disk record of them. This is what fixed the chipsmith-after-sleep scenario where the bridge had been restarted while Chrome was alive — neither in-memory state nor session files referenced the live Chrome, but the process list did. -- **Stale tabId detection (v1.4.10):** every tab-aware verb (`browser_navigate`, `browser_screenshot`, `browser_eval`, `browser_input_dispatch`, `browser_errors`, `browser_reload`, `browser_close_tab`, `browser_fetch_url`) returns a structured error when you pass a `tabId` that no longer exists: - ```json - { - "ok": false, - "errorCode": "tab_not_found", - "error": "Tab \"tab-3\" not found in session \"chip-fetcher\".", - "currentTabs": ["tab-1", "tab-2"], - "lastKnownUrl": "https://wago.priintcloud.com/datasheets/2601-3105/en/...", - "opener": "popup", - "openerTabId": "tab-1", - "closedMsAgo": 1842, - "_hint": "This tab closed 2s ago at URL https://... Call browser_fetch_url with that URL (and the opener tabId \"tab-1\" for cookies) to get its bytes — do NOT try to eval against the closed tabId." - } - ``` - - `lastKnownUrl` + `opener` + `openerTabId` + `closedMsAgo` are populated when the bogus tabId matches a tab that closed in the last few seconds (kept in a 20-entry per-session ring buffer). Critical for popup PDF workflows where the viewer auto-closes after rendering — you can `browser_fetch_url` directly against `lastKnownUrl` instead of having to re-trigger the spawning click. - - **Important regression note:** v1.4.9 documented this error but the CLI was silently stripping `tabId` from `browser_navigate` / `browser_screenshot` / `browser_eval` / `browser_errors` / `browser_reload` calls before forwarding to the bridge — so the bridge always saw "no tabId" and fell back to the active tab. v1.4.10 fixes the CLI to forward `tabId` so the structured error actually fires. If you're still seeing silent fallback, your CLI is < v1.4.10 (`adom-desktop --version` to check). -- `browser_errors` -- Collected console/page errors. Args: `sessionId`, `clear` (default true), `tabId?` -- `browser_reload` -- Reload the page. Args: `sessionId`, `tabId?` - -**Other**: -- `browser_status` -- All sessions with URLs, tab counts, and error counts -- `browser_close` -- Close ALL sessions -- `browser_wait` -- Wait for content to settle. Args: `ms` - -#### Credential vault — silent HTTP Basic Auth (v1.4.4+) - -When pup navigates to a host that issues an HTTP Basic Auth challenge (every `*.componentsearchengine.com` subdomain that wraps NXP / Mouser / TI / etc CAD bundles, several vendor design + doc portals), Chrome shows a NATIVE auth dialog that lives outside the page DOM. `browser_eval` can't dismiss it. Without the credential vault, the user has to type the same email + password every navigation — chip-fetcher hits this 5–10 times per chip. - -The vault stores credentials in the OS keychain (Windows DPAPI / macOS Keychain / Linux libsecret) and applies them via `page.authenticate()` BEFORE the navigation, so the dialog never appears. **Passwords never leave the keychain** — `credential_list` returns host + username only. - -- `credential_set` -- Store creds for a host pattern. Encrypted at rest by the user's session key. - - Args: `host` (glob: `"*.componentsearchengine.com"` or exact `"nxp.componentsearchengine.com"`), `username`, `password` - - Returns: `{ok, host, username}` (no password) -- `credential_list` -- List stored entries. - - Returns: `{credentials: [{host, username, addedAt, updatedAt}]}` - - Passwords are **never** returned by this verb. -- `credential_delete` -- Remove a credential entry. Drops the password from the keychain and the entry from the index. - - Args: `host` (must match the pattern used at `credential_set` time, exactly) - -**Host matching:** simple glob. `*.example.com` matches any subdomain; exact strings match exactly. Best-match wins (longest non-glob suffix beats shorter glob), so a `nxp.componentsearchengine.com` entry would override a `*.componentsearchengine.com` entry for that exact host. - -**Recipe — chip-fetcher / CSE flow:** - -```bash -# 1. One-time setup (ask user for their CSE creds, then store) -adom-desktop credential_set '{ - "host":"*.componentsearchengine.com", - "username":"[email protected]", - "password":"<paste>" -}' -# → {ok:true, host:"*.componentsearchengine.com", username:"[email protected]"} - -# 2. Navigate freely — no popup -adom-desktop browser_open_window '{ - "sessionId":"chip-fetcher", - "profile":"chip-fetcher", - "url":"https://nxp.componentsearchengine.com/preview_newDesign.php?..." -}' -adom-desktop browser_eval '{"sessionId":"chip-fetcher","expr":"document.title"}' -# → "SamacSys Part Preview" (NOT "Sign in") - -# 3. Audit — passwords are never returned -adom-desktop credential_list -# → {credentials:[{host:"*.componentsearchengine.com", username:"[email protected]", ...}]} - -# 4. Remove -adom-desktop credential_delete '{"host":"*.componentsearchengine.com"}' -``` - -Hooks fire BEFORE every `page.goto` in three places: `browser_open_window`'s first nav, `browser_open_tab`, `browser_navigate`. The credential lookup is `O(N)` over stored entries which is fine for typical N (single digits). - -If a host has no stored creds, navigation proceeds normally — the user will see Chrome's native dialog as before. There's no fallback prompt; the next step is for you to ask the user for the credential, then `credential_set` it. - -Example — show 10 alignment previews in ONE Chrome window: -```bash -adom-desktop browser_open_window '{"sessionId":"align","profile":"align","url":"http://127.0.0.1:8901/"}' -for port in 8902 8903 8904 8905 8906 8907 8908 8909 8910; do - adom-desktop browser_open_tab "{\"sessionId\":\"align\",\"url\":\"http://127.0.0.1:$port/\"}" -done -adom-desktop browser_list_tabs '{"sessionId":"align"}' # see all 10 -adom-desktop browser_screenshot '{"sessionId":"align","tabId":"tab-3"}' -adom-desktop browser_eval '{"sessionId":"align","tabId":"tab-5","expr":"document.getElementById(\"meta-seat-z\").textContent"}' -``` -- `browser_alert_window` -- Flash the Windows taskbar orange. **Always call after updating a pup window.** Args: `sessionId` -- `browser_focus_window` -- **Tab-only.** Calls Puppeteer's `page.bringToFront()` — activates the session's tab WITHIN its Chrome window, but does NOT raise the OS window above other apps on the desktop. Args: `sessionId`. **For OS-level foreground use `browser_raise_os_window` (below).** -- `browser_raise_os_window` -- **Real OS foreground raise.** Brings the Chrome window hosting this pup session above all other desktop apps. Use this BEFORE recording, screenshots, animations, or any flow where Chrome being occluded would matter — when Chrome is in the background, `document.hidden=true` and Chrome throttles `requestAnimationFrame` to ~1 Hz, breaking cinematic camera orbits, CSS animations, video element auto-play, fps counters, etc. Internally: focuses the tab inside Chrome, then EnumWindows-finds the OS window by title containing `(session: <sessionId>)`, restores it from minimized if needed, and runs the foreground-lock bypass (`AttachThreadInput` + `keybd_event` phantom-key trick). Args: `sessionId`. Returns: `{sessionId, hwnd, title, raised, error?}`. - ```bash - # Typical recording prep - adom-desktop browser_open_window '{"sessionId":"demo","profile":"demo","url":"…"}' - adom-desktop browser_raise_os_window '{"sessionId":"demo"}' # ← page now actually visible - adom-desktop desktop_record_start '{"reason":"…"}' - ``` -- `browser_lower_os_window` -- **OS-level minimize.** Sends the Chrome window for this session to the back / minimized so the user gets their desktop back after Claude finishes a long pup-driven flow. Args: `sessionId`. Returns: `{sessionId, hwnd, lowered, error?}`. - -### Screen Recording - -> **CRITICAL — pick the right verb.** Claude frequently mistakes "record a pup" / "record this Chrome window" / "record the page" for desktop recording. They are NOT the same. Picking wrong gives a black/wrong-content `.webm` because the pup window may not be in the OS foreground when the desktop recorder snaps frames. - -#### Decision tree (read this BEFORE you reach for any record command) - -| User said... | Use | -|---|---| -| "record a pup" / "record this pup window" / "record the tab" / "record the page" / "record this Chrome window I just opened" | **`browser_record_start`** — tab-scoped via CDP, captures the pup tab regardless of foreground state, ~50 fps actual at 30 fps target, single `.webm` | -| "record my screen" / "record what I'm doing" (and Hydrogen is available, which is most of the time) | **Hydrogen's built-in:** `recording start --share screen` — same `getDisplayMedia` pipeline but already wired into Hydrogen with native UI. Prefer this for casual "record my screen" asks. | -| "record everything on screen including KiCad / Fusion / OS dialogs / multiple apps switching" | **`desktop_record_start`** with `confirmDesktopNotTabRecording: true` (required arg — see below) | -| "record a pup tab AND give me a parallel desktop video at the same time" (e.g. ralph-loop screenshot tests with simultaneous narration) | **Both at once:** `browser_record_start` on the tab AND `desktop_record_start` on the desktop. Adom-desktop's desktop recorder exists for this case — Hydrogen's recorder can't run while it's busy doing tab capture, so adom-desktop fills that gap. | - -#### Why this matters - -- **Wrong verb → wrong content.** desktop_record_* captures whatever is on the user's monitor at frame time. If the pup window is occluded behind your IDE / Slack / their email client, the resulting clip shows YOUR app, not the pup tab. Even with `browser_raise_os_window` first, the user's natural alt-tab activity covers the pup window within seconds. -- **Hydrogen already has desktop recording.** Most "record my screen" asks should just be Hydrogen's recorder. Adom-desktop's `desktop_record_*` is mostly redundant — it shines exactly when Hydrogen is busy capturing a tab and you need a parallel desktop angle. - -#### The guardrail - -`desktop_record_start` REQUIRES `confirmDesktopNotTabRecording: true`. Without it, the call returns `errorCode: desktop_record_needs_confirmation` with a long error string explaining the alternatives (browser_record_start, Hydrogen, or pass the confirm). This is intentional friction — re-read the error and confirm you really want full-desktop, not pup-tab. - -#### Output summary - -| Mode | Commands | HUD on screen | Concurrent? | Output | -|---|---|---|---|---| -| **Tab** (one pup tab's viewport) | `browser_record_*` | NO (the user already sees the tab) | YES (one per tab) | Single finished `.webm` (real-time playback, ~50fps actual) | -| **Desktop** (full screen) | `desktop_record_*` | YES — always-on-top control panel with reason + manual stop | No (one at a time) | Single finished `.webm` | - -#### Desktop recording (HUD) - -The recorder is a **visible Chrome window** in the bottom-right corner showing recording status, the stated reason, and a manual Stop button. The HUD persists across multiple short clips (a "session"); only `desktop_recorder_close` (or 5 min idle) dismisses it. +### Browser automation (`browser_*`, Puppeteer/pup) — bridge-owned; light pointer -**Two REQUIRED args on `desktop_record_start`:** -- `reason` — string, why you're recording (renders in HUD title + sidecar `.json`) -- `confirmDesktopNotTabRecording: true` — boolean acknowledgement that you understand this captures the WHOLE desktop, NOT a pup tab. The bridge rejects calls without this with `errorCode: desktop_record_needs_confirmation` and a hint pointing at `browser_record_start` (for pup tab) or Hydrogen (for casual screen recording). +The pup bridge is **cloud-owned** and ships its OWN full skill (its wiki page / skill pkg — `bridge_list` names it; `adom-desktop sync_skills` installs it). This page keeps only the pointer plus first-touch examples; sessions/profiles, credential vault, downloads, recording, CDP details are the bridge's skill. -```bash -# Start a desktop clip (HUD opens with reason; recording begins). -# Note: confirmDesktopNotTabRecording: true is REQUIRED — without it the -# bridge rejects with errorCode: desktop_record_needs_confirmation. -adom-desktop desktop_record_start '{ - "reason":"Parallel desktop angle alongside Hydrogen ralph-loop tab capture", - "confirmDesktopNotTabRecording": true, - "monitor":"primary", - "fps":30, - "audio":false -}' -# → { "ok":true, "recordingId":"rec-1", "filePath":"…\\rec-desktop-….webm", … } - -# … drive KiCad/Fusion/etc … - -# Stop this clip — HUD stays open, ready for next clip -adom-desktop desktop_record_stop '{"recordingId":"rec-1"}' -# → { "ok":true, "filePath":"…\\rec-desktop-….webm", "sizeKB":4231, "durationMs":18432 } - -# (Optional) Record more clips in the same session — reason persists, but -# confirmDesktopNotTabRecording must be passed every time. -adom-desktop desktop_record_start '{ - "reason":"Parallel desktop angle alongside Hydrogen ralph-loop tab capture", - "confirmDesktopNotTabRecording": true -}' -# … etc … - -# When done: close the HUD -adom-desktop desktop_recorder_close -# → { "ok":true, "sessionSummary":{"clipCount":3, "totalSizeKB":12390, "reason":"…", "clips":[…]} } - -# Pull each WebM to Docker (already a finished video — no muxing needed) -adom-desktop pull_file '{"filePaths":["C:\\…\\rec-desktop-….webm"], "saveTo":"/tmp"}' -ffprobe -v error -show_entries stream=codec_name,duration -of default=nw=1 /tmp/rec-desktop-….webm -# → codec_name=vp9, duration=18.432 -``` - -Other commands: -- `desktop_recorder_open '{"reason":"…"}'` — open the HUD without starting a clip (so the user knows ahead of time you're about to record) -- `desktop_record_status` — `{hudOpen, reason, currentRecording, clipsThisSession}` -- `desktop_record_list` — completed `.webm` files with sidecar metadata (incl. reason) -- `desktop_list_monitors` — primary monitor info (v1.1: full multi-monitor enumeration) - -The user can click Stop in the HUD at any time — that converges through the same code path as your `desktop_record_stop` call. Don't fight it; if the user stops manually, treat the clip as complete and react accordingly. - -#### Tab recording (real-time video, no HUD, concurrent) - -**For smooth video — narrated demos, motion-heavy scenes, animations, anything the user will watch end-to-end — use `browser_record_start/stop` NOT a `browser_screenshot` loop.** The screenshot loop maxes out around 3 fps and reads as choppy. - -**Mechanism:** CDP `Page.startScreencast` (Chrome's native compositor-driven JPEG-frame stream) with `Page.screencastFrameAck` per-frame backpressure. Each frame is written to disk with a wall-clock timestamp, then ffmpeg muxes into a single VP9 `.webm` using the concat demuxer with **per-frame durations** so playback matches real wall time. Tab-scoped at the protocol level (the CDP session is bound to a specific Page target — no cross-tab/cross-Chrome leakage). - -**FPS expectations.** Chrome's compositor pushes a frame each time the page paints. `everyNthFrame` is computed from the requested fps (e.g. 30 fps → every 2nd paint frame). On rAF-animated pages with the tab foregrounded, expect actual fps to **meet or exceed the target** — measured 49.7 fps actual at 30 fps target on a 2.5K viewport. On occluded windows Chrome paint-throttles, so always raise the OS window first. - -**Why not MediaRecorder + getDisplayMedia?** Tested — Chrome 146 crashes when `getDisplayMedia({preferCurrentTab:true})` is called via Puppeteer (known upstream bug, [puppeteer #13478](https://github.com/puppeteer/puppeteer/issues/13478)). `Page.startScreencast` is Chrome's other native video pipeline; it's stable and produces equivalent output (compositor frames as JPEGs). - -Output is a single finished **`.webm`** file — no tar, no concat staging exposed to callers, no Docker-side mux. Just `pull_file`. - -**Raise the OS window first.** Chrome paint-throttles occluded windows, so calling `browser_record_start` on a backgrounded tab produces stutters. Always: +Most-used verbs: ```bash -adom-desktop browser_raise_os_window '{"sessionId":"align"}' # surface pup tab to OS foreground -adom-desktop browser_record_start '{"sessionId":"align","fps":30}' +adom-desktop browser_open_window '{"sessionId": "s1", "profile": "p1", "url": "https://example.com"}' +adom-desktop browser_navigate '{"sessionId": "s1", "url": "https://example.com/page2"}' +adom-desktop browser_screenshot '{"sessionId": "s1"}' +adom-desktop browser_eval '{"sessionId": "s1", "expression": "document.title"}' +adom-desktop browser_fetch_url '{"url": "https://example.com/data.json"}' ``` -**Anti-throttle launch flags (already baked into pup).** Pup spawns Chrome with four flags that suppress most of Chrome's background throttling so recordings don't collapse to ~1 fps when the user alt-tabs to another app: +`browser_readiness` is the read-only probe (never spawns or downloads). Full details: the bridge's own skill via `sync_skills`. -- `--disable-renderer-backgrounding` -- `--disable-backgrounding-occluded-windows` -- `--disable-background-timer-throttling` -- `--disable-features=CalculateNativeWinOcclusion` +### Screen recording — AD's desktop recorder (tab recording is pup's) -Verified: with all four flags, `browser_record_start` on a window completely covered by Notepad still produces **29.65 fps actual at 30 fps target**. - -> **Caveat: pup sessions launched in a bridge version older than v1.3.32 don't have these flags.** Chrome launch flags apply only at process spawn — they cannot be added retroactively. If `fpsActual` reports ≤1 even with `browser_raise_os_window` first, the session's Chrome was launched with an old flag set. Fix: close + reopen the session (`browser_close '{"sessionId":"<id>"}'` then `browser_open_window`) so it spawns a fresh Chrome with the current flags. The `kicadVersionUsed`-style proof: there's no per-session "flags" reporter yet, so the reliable test is the recording fps itself. - -Multiple recordings on different tabs run in parallel — each tab has its own capture loop. +AD owns DESKTOP recording: a visible HUD opens with the REASON you supply, so the user always sees that recording is on and why. Full guide: the `adom-desktop-recording` skill. ```bash -# Record three tabs in parallel -for TAB in tab-1 tab-2 tab-3; do - adom-desktop browser_record_start "{\"sessionId\":\"align\",\"tabId\":\"$TAB\",\"fps\":30}" -done - -# … drive each tab through its scenario … - -# Stop and pull each (already a finished webm — no muxing step on Docker) -for REC_ID in rec-tab-1 rec-tab-2 rec-tab-3; do - STOP=$(adom-desktop browser_record_stop "{\"sessionId\":\"align\",\"recordingId\":\"$REC_ID\"}") - WEBM=$(printf '%s' "$STOP" | jq -r '.output | fromjson | .filePath') - adom-desktop pull_file "{\"filePaths\":[\"$WEBM\"],\"saveTo\":\"/tmp\"}" -done +adom-desktop desktop_recorder_open '{"reason": "record the DRC walkthrough for review"}' +adom-desktop desktop_record_start '{"confirmDesktopNotTabRecording": true}' +# ... drive the desktop ... +adom-desktop desktop_record_stop '{}' +adom-desktop desktop_record_list '{}' +adom-desktop desktop_recorder_close '{}' ``` -Args: `sessionId` (required), `tabId` (optional — defaults to active tab), `fps` (default 30; note the actual ceiling is ~10-15 fps), `quality` (default 85, 1-100 JPEG quality), `maxDurationMs` (default 600000). - -Returns from `browser_record_stop`: `{recordingId, sessionId, tabId, filePath, sizeKB, durationMs, frameCount, fpsTarget, fpsActual, stopReason}`. - -Other commands: -- `browser_record_status '{"sessionId":"align"}'` — list active tab recordings with live `frameCount`, `fpsTarget`, `fpsActual`, `durationMs`, `sizeBytesApprox` -- `browser_record_list` — completed `.webm` files on disk with sidecar metadata -- Tab close auto-stops any recording for that tab (mux fires, file finalizes, you don't have to call `browser_record_stop` first) - -**Notes & gotchas:** -- The desktop HUD itself is captured by the desktop recording (it's a visible window). Same as Hydrogen — accepted cost for the visible-status UX. The window is small + bottom-right. -- ffmpeg is **required** on Windows for tab recording (the mux step). Install via `winget install Gyan.FFmpeg`. Bridge logs the detection result at startup. -- `maxDurationMs` defaults to 600000 (10 min) for safety; raise it for longer recordings. -- On bridge shutdown (graceful), active recordings are drained — the mux runs and you get a valid `.webm`. -- The mux file (`<rec>.webm`) is produced by ffmpeg and has a valid `format=duration`. The frames staging dir (`<rec>.frames/`) is deleted after a successful mux. +`desktop_record_start` without `confirmDesktopNotTabRecording: true` refuses with `errorCode: desktop_record_needs_confirmation` — that guard exists because recording a BROWSER TAB is the pup bridge's separate feature (`browser_record_start` etc., see pup's own skill). ### Filesystem primitives — `desktop_list_files` / `desktop_watch_files` / `desktop_pull_glob` (v1.5.0+)
version.json+7−7@@ -1,16 +1,16 @@ { "manifest_version": 1, - "version": "1.9.171", - "released_at": "2026-07-24T04:05:00Z", + "version": "1.9.174", + "released_at": "2026-07-24T11:15:00Z", "windows": { - "installer_url": "https://wiki.adom.inc/download/adom/adom-desktop/1.9.171/Adom.Desktop_1.9.171_windows_x64-setup.exe", - "installer_filename": "Adom.Desktop_1.9.171_windows_x64-setup.exe", - "installer_sha256": "510b2e48d1952f974e93499ad5579d7b80eb080658e0307caf98b17ce2e102d3", - "installer_size": 13512088 + "installer_url": "https://wiki.adom.inc/download/adom/adom-desktop/1.9.174/Adom.Desktop_1.9.174_windows_x64-setup.exe", + "installer_filename": "Adom.Desktop_1.9.174_windows_x64-setup.exe", + "installer_sha256": "3b925b3335bed5c75c4179b9026f605ee6099e80b10e9d137f99079e3e02d4cf", + "installer_size": 13557288 }, "changelog_url": "https://wiki.adom.inc/adom/adom-desktop", "minimum_required": "1.3.30", - "note": "Connection setup is one command now: `adom-desktop setup_desktop` (or `connect`) returns a ready-to-run register_container command that injects the connection into your Adom account, so it auto-appears when you sign in to Adom Desktop. No JSON paste on the common path.", + "note": "Big documentation release: nine new skills (auth, CLI, runtimes, permissions, embedded, presence, bridges, local AI, recording) install with the app and the pkg, plus deep-dive sub-readmes on the wiki page with a connection-topology diagram. Bridge deep-dives now live with each bridge; run sync_skills for the full per-bridge guides.", "macos": { "installer_url": "https://wiki.adom.inc/download/adom/adom-desktop-macos/1.9.162/Adom.Desktop_1.9.162_macos_aarch64.dmg", "installer_filename": "Adom.Desktop_1.9.162_macos_aarch64.dmg",