--- name: adom-desktop description: Use when the user wants to send files to their desktop, control KiCad or Fusion 360, send desktop notifications, or troubleshoot the desktop connection. Provides CLI tools for bridging the Docker container to the user's local machine. --- # Adom Desktop Bridge between Claude Code (running in an Adom Docker container) and the user's desktop applications via WebSocket. **Install surface:** the canonical install page is `apps/adom-desktop` on the Adom wiki — it hosts the Linux CLI (`docker_binary`), the Windows installer (`.exe`), and the four sibling skills (pup, kicad-bridge, fusion-bridge, + this one). The gallia `adom-desktop-discovery` skill surfaces this page on any related user query. `adom-desktop setup_desktop` also returns a dynamic installer URL pulled from the latest GitHub release. **First-time setup?** If the user hasn't installed the desktop app yet, see the Setup section below to walk them through downloading, installing, and connecting the app. Quick check if desktop is connected: ```bash adom-desktop ping ``` **Companion skills** (installed alongside this one from the wiki): - `adom-desktop-kicad` — KiCad bridge: launch editors, open designs, install libraries, run DRC, window capture + keyboard/click automation (`plugins/kicad/SKILL.md`) - `adom-desktop-fusion` — Fusion 360 bridge: launch, open designs, STEP/GLB/.lbr import-export, BOM + API queries, Fusion screenshots (`plugins/fusion360/SKILL.md`) - `pup` — browser automation (Puppeteer-style): open URLs, screenshot, eval JS, multi-session Chrome (`gallia/skills/pup/SKILL.md`) ## How It Works ``` Claude Code -> adom-desktop -> Relay Server (HTTP :8766) -> WebSocket :8765 -> Adom Desktop App -> KiCad / Fusion 360 / Browser / Shell ``` The `adom-desktop` binary is a single Rust CLI that does everything: - `adom-desktop serve` -- Start the relay server (WebSocket :8765 + HTTP API :8766) - `adom-desktop ''` -- Send commands to the desktop app via the running relay The relay server runs in the Docker container. The Adom Desktop app runs on the user's PC and connects via WebSocket. ## Recent additions (v1.8.103 → v1.8.125) For readers of older copies of this skill: a lot has landed in the last ~14 releases. Quick summary so you don't try to call out to features that don't exist or miss ones that do. - **Embedded `--start-hidden` honored + window preference** (v1.8.119+) — when HD spawns AD with `--embedded --start-hidden`, AD now stays hidden (v1.8.102–118 force-showed it). The release default respects the launch flags; the machine owner can opt into always-show via the Settings checkbox (config `always_show_window_on_launch`). Same preference pattern as shell auto-approve. - **Popup-risk pre-flight on shell commands** (v1.8.117+) — `shell_execute` and cmd-interpreter `run_script` are scanned BEFORE approval/execution for `start ` invocations whose target doesn't resolve. Those used to flash a modal "Windows cannot find ''" dialog on the user's desktop (no silent mode exists — the box is cmd/ShellExecute's own UI). Now AD refuses with `errorCode: "popup_risk"` + a reprimand `_hint` pointing at `find_exe` / `launch` / `desktop_open_url`. If your command creates its own target mid-command and the scan misjudged it, re-send with `{"allowPopupRisk": true}`. http/https URL targets pass; cmd builtins pass; `copy x y && start y` patterns pass automatically. - **Shell auto-approve is a user preference** (v1.8.116+) — per-command approval is the default everywhere now. Durable opt-in via AD's Settings checkbox or `shell_auto_approve '{"permanent": true}'`; HD passes `--shell-approval ` when spawning AD embedded (session-only) and owns the setting while embedded. The old v1.8.44 embedded-permanent default (and its leak into standalone sessions) is gone — see "Shell auto-approve is a user preference" below. - **Registry-driven install** (v1.8.115+) — `adompkg install adom-desktop` (with `?org=adom` on raw packages-API calls) now serves the current release: the V2 wiki page registers a real adompkg version per release and its manifest carries a server-derived `assets` map (url + sha256 per platform) for the installer binaries. The kicad v0.8.6 verbs (`kicad_export_*`, modifier-combo `kicad_send_key`, `kicad_open_3d_viewer` editor arg), puppeteer v1.0.6 native-drive changes (`native_relaunch_required` flow), and the embedded-mode fixes all reach end-user installers starting with this release. - **`desktop_relaunch_self` + `desktop_relauncher_status`** (v1.8.103+) — visible self-restart helper. AD spawns a separate signed Rust GUI (`ad-relauncher.exe`) in HOST context that owns the entire swap-binary- and-restart sequence: preflight → stop AD → atomic deploy → CreateProcessW launch → verify reachable + PE FileVersion + window visible. Has its own HTTP API on `:47210`. Don't run shell glue for AD relaunches; use this. Full recipe: [`skills/AD_RELAUNCH.md`](AD_RELAUNCH.md). - **Auto-update via wiki manifest** (v1.3.39+; single canonical wiki.adom.inc). AD polls a `version.json` manifest every 4h + on launch from `https://wiki.adom.inc/api/v1/pages/adom-desktop/files/version.json`. Dead-end guard probes the `installer_url` before flipping `update_available` (HEAD + Range-GET fallback for the wiki's HEAD-404 bug). Footer banner Install Now / Later; user picks `prompt` / `auto` / `off` in Settings. SHA256-verified silent NSIS install + autostart relaunch. - **`updater_get_status` payload extension** (v1.8.112+). `UpdaterStatus` now includes `manifest_url` (the URL that served the cached manifest) and `manifest_urls` (full priority chain). Sibling apps that read this endpoint can show users which wiki AD is checking. The Build-details tooltip on the footer version chip surfaces all of it + a clickable link to the wiki page. - **Asset naming convention** (v1.8.113+). The full AD GUI installer and the CLI companion binary now disambiguate cleanly in the wiki: - **Full AD GUI:** `Adom.Desktop___.` Example: `Adom.Desktop_1.8.114_windows_x64-setup.exe` - **CLI companion:** `adom-desktop-cli___[.exe]` Example: `adom-desktop-cli_1.8.114_linux_x86_64` The pre-v1.8.113 Linux name `adom-desktop-linux` ships as a byte- identical alias through v1.8.116. After install, the binary on PATH is still `adom-desktop` — the wiki name disambiguates fetch time only. - **`version.json` `.cli` slot** (v1.8.113+). Additive schema — pre-v1.8.113 readers ignore. Each populated platform has `binary_url` / `binary_sha256` / `binary_size`. Today the manifest carries `cli.linux_x86_64` and `cli.windows_x64`. HD / sibling apps should prefer this typed lookup over hard-coding filenames. - **Windows CLI bundled with AD** (v1.8.114+). Installing AD drops `adom-desktop-cli.exe` in `%LOCALAPPDATA%\Adom Desktop\` and adds the install dir to user PATH (HKCU, no UAC) via the NSIS POSTINSTALL hook. After install, `adom-desktop-cli screenshot_window …` works from any fresh PowerShell / cmd / Claude Desktop terminal. The CLI auto-finds the local AD via direct-API probe at `127.0.0.1:47200` — no relay required for verbs that don't need streaming. - **CLI direct-API probe + relay self-detect** (v1.8.114+). The CLI's probe drops the dead `host.docker.internal` candidate (we don't use Docker Desktop anymore), adds WSL2 `/mnt/c/Users/*/.adom/direct-api-port` DrvFs glob for AD's discovery file, and adds a NAT-mode WSL2 fallback that resolves the Windows host IP from `/etc/resolv.conf`. The CLI's `serve` subcommand probes `127.0.0.1:8766/health` before binding and skips with a friendly message if another relay is already answering (override with `--force-bind`). Both fixes matter for the mirrored- networking WSL2 setup HD uses. ## Starting the Relay Server The relay must be running before the desktop app can connect: ```bash adom-desktop serve & ``` This starts: - WebSocket server on `0.0.0.0:8765` (desktop app connects here) - HTTP API on `127.0.0.1:8766` (CLI commands go here) Check if it's running: ```bash curl -sf http://127.0.0.1:8766/health ``` ## First-Time Setup (Install & Connect) **You are running on a Docker container. You have NO access to the user's desktop.** Guide them step-by-step, ask questions, wait for answers, and verify each step. ### Step 1: Check if already connected ```bash adom-desktop ping ``` If this returns `{ "status": "connected" }`, the desktop is already set up -- skip to "Verify the connection" below. If it errors, continue. ### Step 2: Ensure the relay is running ```bash curl -sf http://127.0.0.1:8766/health ``` If not running: `adom-desktop serve &` ### Step 3: Ask what OS they use Ask the user: **"What operating system is your desktop/laptop? (Windows, Mac, or Linux)"** - **Windows 10/11** -- proceed - **macOS / Linux** -- "Support is coming soon. The desktop app currently only runs on Windows." ### Step 4: Install & connect Run `adom-desktop setup_desktop` to get the `installer_url` and `server_config` fields. Then present **both options** to the user — the Claude Code prompt (fastest) and the manual steps (if they don't have Claude Code/Desktop): --- **Option A: Automatic setup via Claude Code (recommended)** If you have **Claude Code** or **Claude Desktop** on your laptop, paste this prompt: > Install Adom Desktop and connect it to my cloud container. Download the installer from ``, run it silently, then write this JSON to `%USERPROFILE%\.adom\config.json`: > ```json > {"servers":[]} > ``` > Then launch "Adom Desktop" from the Start Menu. *(Replace `` and `` with the actual values from `setup_desktop`.)* **Option B: Manual setup** 1. Download the installer: **[]()** 2. Run it and follow the prompts — the app installs to the Start Menu as "Adom Desktop" 3. Open **Adom Desktop** from the Start Menu 4. Paste this JSON into the text field that says **"Paste server JSON to add ..."** and press **Enter**: ```json ``` 5. The server will appear and auto-connect --- Wait for the user to confirm they see a green dot next to the server name before proceeding. **Important:** Each container has its own relay server. New containers need a new entry -- old entries from previous containers won't work. ### Step 6: Verify the connection ```bash adom-desktop ping # Expected: { "echo": "pong", "roundTripMs": ..., "status": "connected" } adom-desktop status # Expected: one client with the user's hostname and capabilities adom-desktop notify_user '{"title":"Hello from Docker!","body":"Your desktop is connected."}' ``` Tell the user what you see. If `ping` succeeds: > "Your desktop is connected! I can now send files to your machine, open browser windows for visual debugging, control KiCad and Fusion 360, take screenshots of your desktop, and send you notifications." ### Step 7: Optional -- Node.js for browser features Ask: **"Would you like to use the visual browser debugging feature? This opens Chrome windows on your desktop that I can control."** - **Yes + Node.js installed** -- "Great, the browser bridge auto-installs deps on first use." - **Yes + no Node.js** -- "Download Node.js from https://nodejs.org (LTS), install it, then restart Adom Desktop." - **No** -- skip, they can enable it later ### Common connection issues | Symptom | Fix | |---------|-----| | `ping` returns `"No desktop client connected"` | User hasn't added this container in the desktop app yet | | Desktop app shows "disconnected" | Check the URL uses `wss://` (not `ws://` or `https://`), port is `8765` | | Relay server not running | `adom-desktop serve &` | | Multiple stale connections | `adom-desktop kick_all` -- app auto-reconnects within seconds | | Desktop app not installed | Download from [github.com/adom-inc/adom-desktop/releases](https://github.com/adom-inc/adom-desktop/releases) | ## CLI Tool ```bash adom-desktop '' ``` **Examples:** ```bash adom-desktop ping adom-desktop status adom-desktop browser_open_window '{"sessionId":"dart2","url":"https://example.com"}' adom-desktop browser_eval '{"sessionId":"dart2","expr":"document.title"}' adom-desktop browser_screenshot '{"sessionId":"dart2"}' adom-desktop browser_list_windows adom-desktop browser_close_window '{"sessionId":"dart2"}' adom-desktop notify_user '{"title":"Hello","body":"From Docker"}' adom-desktop shell_execute '{"command":"echo hello"}' adom-desktop pull_file '{"filePaths":["C:\\Users\\john\\Downloads\\image.png"],"saveTo":"/tmp"}' ``` **Output:** JSON to stdout. Screenshots save to `/tmp/adom-desktop-screenshots/` and return the file path. **Config location:** The desktop app stores server config at `~/.adom/config.json` (`%USERPROFILE%\.adom\config.json`). This is separate from the binary — config survives updates and reinstalls. ## Available Commands **Get full structured command list with descriptions, args, and prerequisites:** ```bash adom-desktop list_commands ``` Returns categorized JSON with every command, its required/optional args, return values, prerequisites, and workflow notes. Use this when you need to discover available commands or understand what a command expects. ### Connection Management - `ping` -- 5s round-trip test. Use BEFORE browser/shell commands to verify the desktop connection is alive. - `status` -- Check who's connected, their capabilities, desktop paths, and **app installation status**. The `desktop.apps` object shows: - `kicad.installed` / `kicad.version` / `kicad.bridgeRunning` - `fusion360.installed` / `fusion360.running` / `fusion360.bridgeRunning` / `fusion360.addinInstalled` / `fusion360.addinConnected` - `browser.bridgeRunning` - `kick_all` -- Force-disconnect all WebSocket clients. Active Adom Desktop apps auto-reconnect within seconds. #### Programmatic server registration (v1.8.22+) Three verbs let external apps (Hydrogen Desktop's Docker container, in particular) register their relay server with adom-desktop without the user pasting JSON into the GUI by hand. Mirrors the GUI's Quick Add bar + connect/disconnect buttons. - `server_add` -- Upsert a relay server by `name`. If a server with that name already exists, the URL (and optionally authToken) is updated rather than creating a duplicate. ```bash adom-desktop server_add '{"name":"hydrogen-workspace","url":"ws://localhost:8765","autoConnect":true}' # → {ok:true, name, url, id, connected:bool, created:bool, _hint} ``` - `name` (required) — dedup key. Repeat calls with the same name are idempotent. - `url` (required) — relay WebSocket URL. - `authToken` (optional) — defaults to `adom-dev-token-2025`. - `autoConnect` (optional, default `true`) — connect right after upsert. Set `false` to add the entry without dialling out. - Behavior: same URL + already connected → no churn, returns ok. URL changed + autoConnect=true → disconnect old loop + spawn new one. URL changed + autoConnect=false → disconnect old, leave entry registered but not connected. - `server_remove` -- Disconnect (if connected) and delete an entry by name. Returns `{ok, removed:bool, wasConnected:bool}`. Idempotent (removing a non-existent entry returns ok with removed:false). ```bash adom-desktop server_remove '{"name":"hydrogen-workspace"}' ``` - `server_list` -- Persisted server list with live connection status: `{servers:[{name, url, id, autoConnect, enabled, connected:bool, clientCount:0|1, status}]}`. `status` is the fine-grained `connected | reconnecting | disconnected` state machine value. Persists to the same `~/.adom/config.json` the GUI uses, so entries survive a GUI restart. The `ws_client` supervisor (runs every 30s inside the GUI) auto-reconnects entries with `autoConnect:true` on next launch. HD-style usage: ```bash # At HD container startup adom-desktop server_add '{"name":"hydrogen-workspace","url":"ws://localhost:8765","autoConnect":true}' # At HD container shutdown adom-desktop server_remove '{"name":"hydrogen-workspace"}' ``` #### Addressing multiple Adom Desktops on one relay (v1.8.133+) One relay can have **several** Adom Desktops connected at once — e.g. a laptop and an Azure VM, eventually many. Each AD has a **name** (its `--client-name` / Settings value, defaulting to the OS hostname). The cloud CLI addresses a specific one, or fans out to all: - `adom-desktop targets` — list connected ADs: `{count, targets:[{name, hostname, version, os, connectedAt}]}`. Discover names here first. - `adom-desktop --target ` — route the verb to that AD only. `--target` is a **global** flag (goes before the verb; case-insensitive; matches `clientName`, else `hostname`). `ADOM_TARGET=` sets a default (the flag wins). - `adom-desktop --target all ` — fan out to **every** connected AD; returns `{broadcast:true, targets:[...], results:{:}, summary:{total,ok,failed}}`. A per-AD timeout/failure is isolated to that AD's entry. ```bash adom-desktop targets adom-desktop --target azurevm desktop_screenshot_screen adom-desktop --target all hd_status # every box at once, results keyed by name ``` **No-target behavior:** with exactly ONE AD connected, no `--target` is needed (back-compat). With MULTIPLE connected and no `--target`, the call FAILS with `{error:"ambiguous_target", connected:[...], _hint}` (exit 1) — it won't guess which box. A bad name → `{error:"target_not_found", requested, connected:[...]}` (exit 1). Set each AD's name with the `--client-name ` launch flag (HD passes this when spawning the embedded AD), the Settings field, or `config.json` `client_name`. Full design + the relay-routing internals: [`skills/MULTI_AD_ADDRESSING.md`](MULTI_AD_ADDRESSING.md). #### Direct HTTP API for sibling apps (v1.8.25+) If you're authoring a sibling Tauri app on the same machine (Hydrogen Desktop, or any future "Adom-family" app), you can skip the CLI binary entirely and POST commands straight into the running adom-desktop GUI on `127.0.0.1:47200` (was `127.0.0.1:8770` through v1.8.32 — moved to be a better neighbor to HD on 47080+ and avoid the 8000-range collision risk). The endpoint runs inside the GUI process (loopback-only bind), uses the same dispatcher the WS path uses, and returns the same JSON shape as `adom-desktop ` — including every `_hint` field. | Method | Path | Body | Returns | |---|---|---|---| | `GET` | `/health` | — | `{"ok":true,"service":"adom-desktop"}` cheap probe | | `GET` | `/status` | — | service banner + version + schema + `directApi.cliRequired` list | | `POST` | `/command` | `{"app":"","command":"","args":{...}}` | The verb's normal payload (200 OK), or `{error, errorCode, _hint}` (4xx/5xx) | **v1.8.33+ port discovery (sibling apps READ THIS):** Don't hardcode 47200. The GUI may have bound to 47200-47209 instead if the default port was taken (zombie socket, dev instance, third-party collision). Discovery protocol: 1. Read `~/.adom/direct-api-port` (Windows: `%USERPROFILE%\.adom\direct-api-port`) — single-line `host:port`, written by the GUI at bind time, removed at graceful shutdown 2. If file missing or its port doesn't `/health`, scan `127.0.0.1:47200..=47209` for any port answering with `{"ok":true,"service":"adom-desktop"}` 3. Validate `service == "adom-desktop"` to disambiguate from other apps that might bind a port in our range The CLI does this automatically (see `cli/src/direct_probe.rs`). For sibling-app Rust code, see [`skills/DIRECT_API.md`](DIRECT_API.md) for a copy-paste-ready helper. **Per-verb timeouts (v1.8.31+):** the direct API's command-timeout is no longer a hardcoded 120s — it mirrors the CLI dispatcher's per-verb table (`walk_cloud_tree`/`search_cloud_files`=620s; heavy exports=320s; `bridge_install`/`fusion_start`=300s; default=120s). Caller can override via `args.timeout` (seconds, clamped to 1800). Example — programmatic server registration without the CLI: ```bash curl -X POST http://127.0.0.1:47200/command \ -H 'Content-Type: application/json' \ -d '{"app":"desktop","command":"server_add","args":{"name":"hydrogen-workspace","url":"ws://localhost:8765","autoConnect":true}}' # → identical JSON to `adom-desktop server_add '{...}'`, including the _hint ``` **What's safe to send directly:** essentially everything sync (`server_*`, `bridge_list`, `hd_status`, `hd_build_status`, `desktop_list_windows`, `desktop_screenshot_*`, `kicad_*`, `fusion_*`, `browser_screenshot`, `notify_user`, ...) plus async-dispatching verbs that return a job id in <500 ms (`bridge_install`, `hd_build`, `desktop_install_kicad`, ...). **What requires the CLI fallback:** verbs returning a structured `errorCode:"cli_required"` from this endpoint — currently `pull_file`, `send_files`, `shell_execute`. These use binary streaming or multi-minute approval polling that doesn't fit a single synchronous HTTP request. `GET /status` returns the full list at runtime so callers can branch defensively. Full integration guide (loopback trust model, recipes for HD startup/shutdown, retry/backoff patterns, port-discovery code in Rust + TypeScript) lives in [`skills/DIRECT_API.md`](DIRECT_API.md) in this repo, and as a wiki asset attached to `apps/adom-desktop`. #### Embedded mode — AD bundled inside Hydrogen Desktop (v1.8.42+) When Hydrogen Desktop (HD) bundles AD, AD enters "embedded mode" at boot. HD owns the system-tray icon (AD's is suppressed), HD owns the Adom Cloud login (HD's session token is handed to AD via Phase 4's existing channels), HD owns auto-updates (AD's wiki-poll is disabled — HD bundles a fresh AD on every HD release), and HD spawns AD hidden (its tray "Open Adom Desktop" surfaces the window on demand). Standalone AD users see zero change — none of the signals fire. **Detection** — three-signal cascade (any one triggers embedded mode): 1. `--embedded` CLI flag (what HD passes on every spawn) 2. `ADOM_EMBEDDED=1` env var (backup channel) 3. `%LOCALAPPDATA%\Adom Desktop\embedded.json` marker file (survives AD restarts; written by HD's installer; deleted by HD's uninstaller) **CLI flags HD passes when spawning AD**: ```bash adom-desktop.exe --embedded --start-hidden \ --relay-url ws://127.0.0.1:8765 --relay-name hydrogen-desktop \ --session-token ``` - `--embedded` → enter embedded mode (also writes the marker) - `--start-hidden` → boot with main window invisible. **v1.8.119+: honored in embedded mode** (HD owns the surface; AD stays in the tray and opens via `desktop_window_show`). v1.8.102–118 had a bug where embedded mode force-showed the window even with `--start-hidden`; that's fixed. The machine owner can still flip AD's "Always show the window at launch" Settings checkbox (config `always_show_window_on_launch`, default OFF) to override and always show — a personal-dev-box preference, not the product default. - `--shell-approval ` → HD's passthrough of its user's shell-approval choice (v1.8.116+, session-only) - `--relay-url ` → upsert this relay via existing `server_add` dedup; default name `hydrogen-desktop` - `--relay-name ` → override the default name - `--session-token ` → already covered by Phase 4 handoff (env / CLI arg / file) **Docker introspection** — `adom-desktop desktop_embedded_status` returns `{embedded, owner, source, enteredVia, pendingRelayUrl, pendingRelayName, startHidden, markerPath, markerExists}` so cloud-side callers can branch on whether AD is standalone or HD-managed. **v1.8.73+:** `enteredVia` reports `"launch-flag"` (HD spawned AD as a child via `--embedded`) or `"runtime-adopt"` (AD was running standalone; HD adopted it at runtime) — see "Runtime adopt + release" below. **New verbs HD uses to drive AD** (all also usable standalone): | Verb | Purpose | |---|---| | `desktop_window_show` | Bring AD's main window to foreground | | `desktop_window_hide` | Hide AD's main window (app keeps running) | | `desktop_connect_all` | Spawn ws_loop for every enabled server | | `desktop_disconnect_all` | Drop every live WS connection (entries stay in config) | | `desktop_shutdown` | Explicit AD termination (stop bridges, then exit) | | `desktop_embedded_status` | Introspect current embedded state (incl. `enteredVia`) | | `desktop_embedded_enter` | **v1.8.73+** runtime adopt: standalone → embedded as `runtime-adopt` | | `desktop_embedded_release` | **v1.8.73+** revert to standalone. **v1.8.77+: always reverts, never exits AD** (regardless of `entered_via`) — use `desktop_shutdown` for explicit exit | | `desktop_logout` | Clear AD's `~/.adom/session.json` (HD's sign-out propagation) | #### Runtime adopt + release (v1.8.73+) — the LaunchFlag-vs-RuntimeAdopt distinction HD has two ways to put AD into embedded mode: 1. **Spawn AD with `--embedded`** (LaunchFlag) — HD launches a fresh AD process as its child. AD enters embedded with `entered_via=launch-flag`. When HD subsequently dies ungracefully (without calling `/embedded/release` or `desktop_shutdown` first), AD's 60s `embedded_monitor` tick notices HD's control port is dead and **exits AD** (the "no hanging chad" rule — no tray-less orphan). 2. **Adopt at runtime via `POST /embedded/enter`** (RuntimeAdopt) — AD was already running standalone; HD calls the direct-API endpoint to flip AD into embedded mode without a respawn. AD enters embedded with `entered_via=runtime-adopt`, preserving its open state, current PID, all in-flight connections. When HD subsequently dies ungracefully, AD's monitor **reverts AD to standalone** — tray restored, footer pill hidden, marker deleted. AD process stays alive. The rule is simple: **whoever started AD owns AD's death**. If AD started itself (standalone → runtime adopt), HD's death doesn't take AD down with it. **Two new direct-API endpoints** (v1.8.73+, sugar over `POST /command`): ```bash # Adopt — runs the same handler as desktop_embedded_enter: curl -s -X POST http://127.0.0.1:47200/embedded/enter # Release — runs the same handler as desktop_embedded_release. # v1.8.77+: ALWAYS reverts to standalone. AD never exits via this call. curl -s -X POST http://127.0.0.1:47200/embedded/release ``` **Probe target (v1.8.75+):** the 60s `embedded_monitor` checks HD's *control port* (default `47084`, discovered from `%APPDATA%\hydrogen-desktop\ports.json` key `control`), not HD's legacy `:9001` webview-eval port. Same discovery `hd_api` uses. Pre-v1.8.75 this was hardcoded to `:9001` — HD doesn't bind that port anymore, so the probe always reported HD dead and every embedded AD reverted/exited within 60s of HD starting. **Live frontend update (v1.8.76+):** the "Embedded · HD" footer pill responds to Tauri `mode-changed` events emitted on every transition. So if AD's window is open when HD adopts it (or HD releases it), the pill flips immediately — no page reload, no manual refresh. **Stale-marker safety (v1.8.74+):** if the marker file says `entered_via=launch-flag` AND HD's control port isn't responding at boot, AD deletes the marker BEFORE exiting. Without the delete, every subsequent manual launch (no `--embedded` flag) would hit the same exit branch and AD would be unlaunchable until the user manually rm'd the marker. The delete-before-exit makes the next launch start standalone normally. **Shell auto-approve is a user preference** (v1.8.116+ — replaces the v1.8.44 embedded-permanent default). The safe default everywhere is per-command approval. Resolution order at AD boot: 1. `--shell-approval ` launch flag — HD's passthrough of its user's choice when spawning AD embedded. **Session-only**: never persisted by AD. 2. The durable Settings checkbox ("Auto-approve shell commands permanently", config field `shell_auto_approve_permanent`, default OFF). Also set/cleared via `shell_auto_approve '{"permanent": true|false}'`. 3. A restored timed grant from a previous session (+1h/+24h buttons, `duration_secs` calls). 4. Otherwise: prompt per command. In embedded mode **HD owns the preference** (HD is the parent, AD the child): AD's Settings checkbox goes read-only with "Managed by Hydrogen Desktop", and the footer shows "**Permanent** *(managed by Hydrogen Desktop)*" when armed. HD changes it at runtime with the same verb: ```bash # revoke everything (timed + permanent + the saved checkbox): adom-desktop shell_auto_approve '{"duration_secs": 0}' # set the durable preference (equivalent to ticking the Settings checkbox): adom-desktop shell_auto_approve '{"permanent": true}' ``` Migration note: pre-1.8.116 embedded boots persisted a permanent grant that leaked into standalone sessions. On first v1.8.116 boot, a permanent-class grant on disk WITHOUT the config preference is cleared — re-opt-in via the checkbox or the verb if you actually wanted it. #### Bridge ports are dynamic — you never need to know one (v1.8.31+) KiCad, Fusion, Browser/Puppeteer, and all third-party bridges now bind OS-assigned ephemeral ports (not the legacy 8772/8773/8851 you may remember). The runtime port changes every spawn. **Callers never need to know it.** - The CLI verb namespace (`kicad_*`, `fusion_*`, `browser_*`, plus any third-party `_*`) is the contract. Always go through that. - adom-desktop's direct API forwards to whatever port each bridge is on at the moment of the call. - `bridge_list` reports `spawn.runtimePort` per bridge for debugging — do NOT hardcode it anywhere. - Why this changed: HD's bridges also wanted 8772/8773/8851; we used to collide silently. Dynamic ports = clean coexistence. ### Managing bridges (the AI fully controls bridges on its own) Bridges are AD's plugins (kicad, puppeteer, fusion360 ship bundled; others install from the wiki). You — the AI — can see, update, install, remove, pause, and resume them entirely through verbs. No human step is required. | Verb | What it does | |---|---| | `bridge_list` | Every installed bridge: `name`, `displayName`, `version` (what AD runs now), `source` (bundled / cache / third_party), `paused`, spawn kind, verbPrefixes. Start here. | | `bridge_check_updates` | **READ-ONLY.** Per bridge: `current` vs `latest` (wiki) + `updateAvailable`. Downloads nothing. The "what's behind?" probe. | | `refresh_bridges` | Apply updates. No args = sync ALL bridges from the wiki. `{"name":"kicad"}` = update just one. | | `bridge_install` | Install a new third-party bridge from a wiki manifest URL (`{"manifestUrl":"…/-bridge-manifest.json"}`). On `success:true` the **capsule appears in the AD GUI immediately** (no verb call needed; it's gray/idle until first use, then green). On `success:false` **nothing is registered and no capsule appears** — read `error` (a `read zip body`/`builder error` means the manifest's zip URL couldn't be fetched). Don't treat a parsed manifest as a completed install; confirm with `bridge_list`. The manifest `url` may be relative (resolved against the manifest's dir, v1.8.169+). | | `bridge_uninstall` | Remove a bridge (`{"name":"…"}`). Bundled ones revert to the bundled copy; third-party ones are fully removed. | | `bridge_pause` / `bridge_resume` | Flip a routing flag so a bridge's verbs fail-fast with `bridge_paused` (or resume). Does NOT kill the process. | **Naming convention for published bridges (3rd-party authors, read this):** when you publish a bridge to the Adom wiki, its **page title / catalog descriptive name MUST start with `Adom Desktop - `** (spaced hyphen) — e.g. `Adom Desktop - Altium Bridge`, matching the bundled bridges (`Adom Desktop - Puppeteer Bridge`, `Adom Desktop - KiCad Bridge`). This keeps every AD bridge grouped with consistent names in app-index pages instead of scattering as bare slugs. It applies to the **wiki/catalog title only** — NOT `bridge.json` `displayName` (the in-app chip label, which stays tool-descriptive like `Altium Designer`) or `name` (the lowercase slug). Full authoring walkthrough: the `adom-desktop-bridge-author` skill. **Typical "am I on the latest bridges?" flow:** ```bash adom-desktop bridge_check_updates # read-only: see current vs wiki per bridge # → {bridges:[{name,current,latest,updateAvailable,source,wikiReachable}], updatesAvailable:N} adom-desktop refresh_bridges # update ALL that are behind adom-desktop refresh_bridges '{"name":"kicad"}' # or just one ``` **Version semantics (so the numbers make sense):** - `current` = what AD actually runs = the newer of the writable cache and the bundled copy (version-aware, numeric semver — `1.0.10 > 1.0.9`). - A bundled bridge that ships AHEAD of the wiki shows `updateAvailable:false` — AD correctly keeps the newer bundled copy. **puppeteer is bundled-only** (its node_modules deps exceed the wiki file limit), so it never shows a wiki update; it updates with the AD installer. - After an `updated` action, the bridge SERVER PROCESS still holds the old code in memory until it respawns. `is_bridge_running` is version-aware (v1.8.142+) and forces a respawn on the next verb call, so you normally don't need to kill anything — but if you want it instant: `taskkill /f /im python.exe` (kicad/fusion) or `node.exe` (puppeteer). **Auto-update cadence:** a running AD also auto-syncs bridges on launch + every 4 hours. `bridge_check_updates` / `refresh_bridges` (or the GUI's **"Check for updates"** button next to "+ Bridge") are the on-demand "I just published, pick it up now" path. #### Bridge lifecycle — one instance, stable port (v1.8.153+) A dynamic bridge gets **one stable port** that AD reuses across every spawn and across AD restarts (persisted in `~/.adom/bridge-ports.json`). A long-lived external client (e.g. a browser extension that dials the bridge) keeps reaching the same port — no chasing a fresh ephemeral port each cycle, no "the client is on bridge A but AD routes to bridge B" split. AD enforces a **single instance**: it tracks every spawned bridge PID, reaps a stale/wedged one before respawning on the same port, and `bridge_stop_all` reaps orphans too. `spawn.persistent` (in `bridge.json`) is now surfaced in `bridge_list` (`spawn.persistent`) and means: **AD auto-respawns this bridge on death**, on its stable port (a supervisor runs each ~15 s). So "kill the bridge to load new code" → AD brings it right back at the same port the client keeps dialing. `bridge_install` / `refresh_bridges` also no longer fail with the cache-dir lock (AD reaps the process holding the dir as its cwd, then swaps; in-place merge fallback if a handle lingers). **v1.8.156 — strays + self-documenting hints.** AD now tracks every bridge PID it spawns (persisted across AD restarts) and **reaps all stray instances** before respawning — so you don't end up with 4 copies of one bridge fighting over the routing. `bridge_list` surfaces `spawn.instanceCount` per dynamic bridge (normally 0 or 1; **>1 = strays piled up** — visible at a glance). If a bridge verb fails to reach a healthy bridge, the error `_hint` tells you to retry / check `instanceCount` / `process_list`; and a verb response right after a (re)spawn carries an `_ad_note` saying the external host reconnects within a few seconds — **retry** instead of treating a transient "not connected" as terminal. For verb discovery from a bridge, `GET /commands` lists every reachable verb (see [`skills/DIRECT_API.md`](DIRECT_API.md)). #### Bridge status chip — self-reported LED + tooltip (v1.8.152+) Each bridge has a status dot in the AD GUI sidebar. **AD is a pure renderer of bridge health** — a live bridge decides its own LED color and hover text; AD just paints what the bridge reports. This replaced the old scheme where AD inferred the LED from whether it was holding a spawn handle, which left fire-and-forget (dynamic/installed) bridges stuck RED even while healthy and routing. **The contract (all three fields OPTIONAL).** On the endpoint AD already polls — your `bridge.json` `healthEndpoint` (default `/health`; the sample bridges use `/status`) — return a 2xx JSON body that MAY include: | Field | Type | Meaning | |---|---|---| | `led` | `"green"` \| `"yellow"` \| `"red"` | The dot color. Green = healthy; yellow = degraded/warning; red = the bridge itself knows it's broken (e.g. lost its upstream). | | `summary` | string | Short label shown next to the dot. Keep it tiny (≤ ~24 chars) — e.g. `"2 windows"`, `"ext disconnected"`. | | `tooltip` | string | Full hover text. Newlines allowed; no hard length limit (keep it readable). | **Rules AD enforces:** - Endpoint returns 2xx **with** `led`/`summary`/`tooltip` → rendered **verbatim**, no parsing or logic. - Endpoint returns 2xx **without** them → AD defaults to **green** (you're reachable) + a generic tooltip. Existing bridges work unchanged — this is opt-in. - Endpoint **unreachable** → AD owns this ONE state: a **gray/offline** dot + AD's own tooltip ("Not running" / "Bridge not reachable"). A dead bridge can't self-report, so AD never lets a bridge claim "offline" — it detects it. **Cadence:** AD re-polls every ~15 s, and a dynamic bridge flips green the instant it spawns. So a bridge that surfaces live counters (open windows, queue depth, ext-connection state) in `summary`/`tooltip` updates in the GUI within a tick. Reference implementations: [`scripts/sample-bridges/hello-python/server.py`](../scripts/sample-bridges/hello-python/server.py) and [`hello-rust/src/main.rs`](../scripts/sample-bridges/hello-rust/src/main.rs) both self-report `led`/`summary`/`tooltip` on `/status` — fork either. ### File Transfer - `send_files` -- Send files from the Docker container to the desktop. Files are base64-encoded in transit. - `filePaths`: array of absolute paths on the server - `targetApp`: "kicad", "fusion360", or "general" - `dest`: **absolute** Windows path (under a safe root) to land the files exactly where you want (v1.8.148+). Prefer this when you know the target dir. - `destinationFolder`: **relative subfolder only** (e.g. `"kicad/symbols"`, `"fusion"`). The desktop app controls the base directory. **Absolute paths are rejected.** With neither `dest` nor `destinationFolder`, files land in the laptop's **Downloads** folder. - Returns `destinationPaths[]` with the absolute path of every saved file, plus (v1.8.168) `savedToDir` (the folder) and a rich `_hint`. **The files land on the LAPTOP (Windows host), not in your container** — verify with `list_dir`, pull one back with `pull_file`. - **`send_files` only COPIES bytes — it does NOT install or register a bridge.** Dropping bridge files with `send_files` will NOT make a capsule appear or verbs route. To add a bridge use `bridge_install '{"manifestUrl":"…/-bridge-manifest.json"}'`, then confirm with `bridge_list`. - `pull_file` -- Pull files from the desktop to the container. - `filePaths`: array of absolute Windows paths on the desktop - `saveTo`: directory on the container to save files (default: `/tmp`) - **Streaming since v1.4.3.** Each file is transferred as 1 MiB binary WS frames straight to disk on the Docker side, with incremental SHA256 verification. The legacy 30s base64-JSON path is gone — large files (50 MB+ datasheets, 75 MB reference manuals) no longer time out. Per-file timeout is 600s. - Returns `files: [{name, path, size, sha256, chunks}]`. Use `sha256` to verify the transfer (the desktop side computes it during streaming and the container side verifies on completion; mismatch deletes the partial file and reports failure). `chunks` is the count of 1 MiB binary frames received. - When at least one but not all files succeed: `success` is `true`, `errors[]` lists the failures alongside `files[]`. When ALL fail: `success` is `false`. ### Desktop Notifications - `notify_user` -- Send a desktop notification with optional action buttons. - `title`, `body`, `level` (info/warning/error/emergency), `actions` (array of button labels) ### KiCad Tools KiCad supports **multi-version side-by-side installs** — KiCad 9 and KiCad 10 (and older versions) can coexist on the same machine. Every `kicad_open_*` (and `install_*`, `run_drc`) command accepts an optional `kicadVersion` arg (e.g. `"9.0"` or `"10.0"`). Omit it to use the **newest installed** version. The success response includes `kicadVersionUsed` so you can confirm which install actually ran. - `kicad_list_versions` -- List every installed KiCad version with their paths and which is the default (newest). Run this first if you're unsure what's available. - Example: `adom-desktop kicad_list_versions` - Returns: `{versions: [{version, base_dir, kicad_exe, default}], default: "10.0", count: 2}` - `kicad_install_symbol` -- Send a .kicad_sym file and install it as a library. Args include optional `kicadVersion`. - `kicad_install_library` -- Install a symbol, footprint, or 3D model library. Each KiCad version has its own sym-lib-table / fp-lib-table — pass `kicadVersion` to target a specific install (otherwise installs into the newest version's tables). - `kicad_open_board` -- Open a .kicad_pcb file. Args: `filePath`, optional `kicadVersion`. Example: `adom-desktop kicad_open_board '{"filePath":"C:/foo.kicad_pcb","kicadVersion":"9.0"}'` - `kicad_open_schematic` -- Open a .kicad_sch file. Args: `filePath`, optional `kicadVersion`. - `kicad_open_symbol_editor` -- Open the Symbol Editor. Optional `kicadVersion`. - `kicad_open_footprint_editor` -- Open the Footprint Editor. Optional `kicadVersion`. - `kicad_open_3d_viewer` -- Open the 3D Viewer (works from Footprint Editor or PCB Editor). Optional `kicadVersion`. - `kicad_run_drc` -- Run Design Rule Check on a board via kicad-cli (headless, ~1-3s, structured JSON). v1.7.12+: prefer `kicad_lint_board` which wraps DRC + file-format sanity + schematic-parity in one structured response. - `kicad_run_erc` -- **v1.7.12+** Run Electrical Rule Check on a schematic via kicad-cli. Mirrors `kicad_run_drc` for `.kicad_sch`. Catches unconnected pins, duplicate references, hierarchical-label mismatches, no-driver nets. Headless, ~1-3s. - `kicad_lint_board` -- **v1.7.12+** Comprehensive headless pre-flight for a `.kicad_pcb` via kicad-cli: file existence + magic-byte sniff + DRC with `--severity-all` + `--all-track-errors` + `--schematic-parity` (default on). Returns structured `{data:{violations[], summary, fileFormat, schematicParityChecked}, _hint}`. Args: `filePath`, optional `schematicParity` (default true). **USE THIS BEFORE kicad_open_board** for any file you didn't just generate — catches DRC errors, parse issues, format-upgrade-needed BEFORE the GUI opens and possibly hangs on a modal dialog. - `kicad_lint_schematic` -- **v1.7.12+** Comprehensive headless pre-flight for a `.kicad_sch` via kicad-cli: file existence + magic-byte sniff + ERC with `--severity-all`. Same structured shape as `kicad_lint_board`. **USE BEFORE kicad_open_schematic** for unfamiliar files. - `kicad_lint_library` -- **v1.7.13+** Validate a library file or directory before install. Accepts a single `.kicad_sym`, `.kicad_mod`, `.pretty` directory, or directory containing `.kicad_sym` files. Parses S-expression structure (no kicad-cli call — `sym/fp upgrade` has no `--dry-run` and would rewrite). Returns symbol/footprint inventory + per-file `fileVersion` + tiered `_hint`. **USE BEFORE kicad_install_library / kicad_install_symbol / kicad_install_footprint** to catch malformed files, outdated formats, or wrong-filetype-renamed-with-.kicad_sym surprises in ~100ms. - `kicad_format_upgrade` -- **v1.7.13+** Upgrade a KiCad file to the current format via `kicad-cli {pcb,sch,sym,fp} upgrade`. Auto-detects `kind` from extension if omitted. MUTATES the file in place (kicad-cli has no `--dry-run`). Common flow: lint_library reports outdated `fileFormatVersions` → surface to user → ask permission → call this. After upgrade, ANY OPEN KiCad windows holding this file must be closed + reopened. #### KiCad bridge — Phase 2.ac additions (v1.7.14+, via `kicad_bridge_call`) Three additions completing the Phase 2.ab loose ends: - **`verify_loaded {kind, expectedName?}`** — reads the Symbol/Footprint Editor frame title to confirm whether a load succeeded. `kind` is `"symbol"` or `"footprint"`. Optional `expectedName` adds a `matchesExpected` field comparing the loaded item against your target. Use AFTER `select_and_load_*` to confirm the symbol/footprint actually loaded (the load itself is "unverified" until you check). - **`get_net_topology {netName}`** — net-graph traversal: returns every footprint pinning the named net, plus the OTHER nets each of those footprints bridges. `{padCount, componentCount, components:[{ref, value, padsOnNet, otherPads:[{padName, netName}]}], bridgedNets}`. Use for "trace VCC outward from its source through the components it touches" analyses. - **`select_and_load_{symbol,footprint} {name, useSendInput?: false}`** — v0.7.0 adds optional `useSendInput:true`. When `true`, escalates from `wx.PostEvent` (default; safe but may not propagate to KiCad's TOOL_DISPATCHER) to real Win32 `SendInput` with the proven `Enter` → `Ctrl+Shift+E` sequence. SendInput briefly STEALS foreground focus AND requires the Symbol Editor to be the foreground window before keystrokes dispatch — both default and SendInput paths are still "best-effort", so ALWAYS pair with `verify_loaded` to confirm. If verify_loaded reports `hasLoaded:false`, fall back to the legacy `kicad_open_symbol_editor '{"symbolName":"..."}'` path which uses out-of-process PowerShell SendKeys + mouse-click on the search box (proven but heavyweight). - `kicad_close` -- Close all KiCad windows. Args: `force` (optional bool — skip graceful close, just taskkill) **When to specify `kicadVersion`:** - The user asks to open something "in KiCad 9" / "in KiCad 10" / "in the older version" — pass that explicitly. - The user is regression-testing across versions (open in 9, screenshot, close, open in 10, screenshot, compare). - A library was installed under a specific version's tables and the user wants to use it. If you pass an invalid `kicadVersion`, the bridge returns `available_versions` and a `_hint` listing what IS installed — surface that to the user. #### KiCad reverse bridge (v1.7.5+) — direct in-process control KiCad v2 Phase 2 introduces a **reverse bridge**: an HTTP/JSON-RPC server runs *inside* every KiCad GUI process (kicad.exe project manager, pcbnew.exe PCB+Footprint editors, eeschema.exe Schematic+Symbol editors) once they restart after install. This lets adom-desktop introspect frames + post menu commands directly, bypassing UI screen scraping for everything the bridge supports. **Auto-installation is transparent.** The first `kicad_*` command of every bridge boot auto-deploys two files (`adom_bridge.py` + `usercustomize.py`) into `%USERPROFILE%\Documents\KiCad\\3rdparty\Python311\site-packages\` for every installed KiCad version. The response includes a `pluginAutoInstall` field with full details: ```json { "pluginAutoInstall": { "triggered": true, "summary": "installed" | "updated" | "already-current" | "partial-failure" | "skipped", "payloadVersion": "0.1.0", "perVersion": [{"kicadVersion":"10.0", "action":"installed", "userSitePath":"...", "filesWritten":[...], "_hint":"..."}], "runningKiCad": [{"exeName":"eeschema.exe","pid":12345}], "_hint": "Plugin successfully INSTALLED. KiCad currently running... ask user to restart KiCad..." } } ``` If installation fails, the `_hint` field tells you exactly what to report and how to recover. Failure modes: `payload_missing` (bundle regression), `user_site_unavailable` (no USERPROFILE), `permission_denied` (Documents dir uncreatable — likely OneDrive sync), `copy_failed` (KiCad has the .py locked — ask user to close KiCad), `no_kicad_installed` (offer `desktop_install_kicad`), `disabled_via_env`. **Currently-running KiCad processes don't pick up the bridge until restarted** (Python init runs once per process). If `pluginAutoInstall.runningKiCad` is non-empty AND summary is `installed`, surface the `_hint` to the user — they need to close + reopen KiCad to activate the bridge on those instances. - `kicad_bridge_status` -- Enumerate every running KiCad GUI process with the adom_bridge plugin loaded. Reads `%TEMP%\adom-kicad-bridge-.json` discovery files + probes each `/status`. - Example: `adom-desktop kicad_bridge_status` - Returns: `{plugins: [{exeName, pid, port, version, alive, uptimeMs, requestCount}], summary: {total, alive}, pluginAutoInstall: {...}}` - If `summary.alive=0` after a fresh KiCad launch, check `pluginAutoInstall._hint` — the install probably failed and you have a specific recovery to offer. - `kicad_bridge_call` -- Generic passthrough: call any RPC method on the bridge for a specific KiCad exe. v1.7.6+ supports multi-instance disambiguation via `pid` arg. - Args: `exeName` (required: `kicad` / `pcbnew` / `eeschema` / `kicad-cli`), `method` (required), `params` (optional dict), `timeout` (optional float seconds), `pid` (optional int — pin to a specific process; otherwise the newest alive instance is picked) - Example — open Symbol Editor from running Schematic Editor: ```bash adom-desktop kicad_bridge_call '{"exeName":"eeschema","method":"wm_command","params":{"frame_index":0,"menu_id":20390}}' ``` - Methods (v1.7.6): - `ping` — `{version, exeName, pid, uptimeMs, requestCount}`. Health check. - `list_frames` — `[{frameClass, frameTitle, hwnd, menubarTopCount, menubarTotalItems, isShown}, ...]`. Replaces screen scraping. - `get_menu_ids` — full menu walk per frame: `[{frameClass, frameTitle, menuItems: [{topMenu, label, id, kind, accelerator, helpString}, ...]}, ...]`. - `wm_command` — posts a Win32 `WM_COMMAND` to the target frame via `PostMessageW(hwnd, WM_COMMAND, menu_id, 0)`. Goes through KiCad's TOOL_DISPATCHER. Args: `{frame_index, menu_id}`. - `get_board_info` — **pcbnew only**. Returns `{hasBoard, fileName, trackCount, footprintCount, drawingsCount, netCount, copperLayerCount, boundingBoxMm:{widthMm,heightMm,centerXMm,centerYMm}}`. Replaces screenshot-poll for state queries. - `get_schematic_info` — **eeschema only**. Returns `{available, frames:[{frameClass, frameTitle, hwnd, editorKind, fileName, isUnsaved, isUntitled}], frameCount}`. Title-based until KiCad 11 ships kipy. - `get_open_editors` — per-process inventory: `{available, exeName, pid, editors:[{kind, frameClass, frameTitle, hwnd, isShown}], editorCount}`. For cross-process aggregator use the `kicad_open_editors` verb instead. - **`get_pcb_footprints`** (v1.7.7+, pcbnew only) — full footprint list with `{ref, value, libNickname, itemName, fpid, x_mm, y_mm, rotationDeg, layer, isFlipped, isLocked, padCount, boundingBoxMm}` per footprint. Replaces screen-scrape "what's placed where". - **`get_pcb_layers`** (v1.7.7+, pcbnew only) — layer stackup: `{copperLayerCount, totalEnabledLayers, layers:[{id, name, kind, isCopper}]}` where kind is one of: copper, mask, silk, paste, courtyard, adhesive, edge, fab, comments, user, margin, other. - **`get_pcb_nets`** (v1.7.7+, pcbnew only) — net list with usage stats: `{netCount, totalTrackLengthMm, nets:[{code, name, padCount, trackLengthMm}]}`. - **`get_pcb_design_rules`** (v1.7.7+, pcbnew only) — DR summary: `{trackWidthsMm, viaDimensions:[{diameterMm,drillMm}], minClearanceMm, minThroughHoleMm, minViaDiameterMm}`. - **`navigate_symbol`** (v1.7.7+, eeschema only) — `params:{symbolName}` or `params:{query}`. Sets the wx.SearchCtrl value in the Symbol Editor's library panel + posts EVT_TEXT to activate the live filter. NON-destructive — does NOT load the symbol, only filters the visible library tree. **Replaces the PowerShell SendKeys path used by `kicad_open_symbol_editor` when `symbolName` is passed** — substantially more reliable + doesn't steal foreground focus. - **`navigate_footprint`** (v1.7.7+, pcbnew only) — `params:{footprintName}` or `params:{query}`. Same shape as navigate_symbol but for the Footprint Editor. - **`get_pcb_tracks`** (v1.7.8+, pcbnew only) — every track (segments, arcs, vias) with `{type, layer, layerName, widthMm, netCode, netName, lengthMm, startMm, endMm}`. Vias show up here too (with no length); call `get_pcb_vias` for via-specific fields. - **`get_pcb_pads`** (v1.7.8+, pcbnew only) — every pad across all footprints: `{footprintRef, padName, positionMm, sizeMm, drillMm, netCode, netName, attribute}`. Position is absolute (post-rotation of parent footprint). - **`get_pcb_vias`** (v1.7.8+, pcbnew only) — every via: `{positionMm, diameterMm, drillMm, viaType, topLayer, topLayerName, bottomLayer, bottomLayerName, netCode, netName}`. viaType is "through" / "blind_buried" / "microvia". - **`get_pcb_drawings`** (v1.7.8+, pcbnew only) — silk text, edge cuts, dimensions, user-drawn shapes: `{type, layer, layerName, text?, positionMm?, startMm?, endMm?, centerMm?, radiusMm?}`. - **`get_pcb_zones`** (v1.7.9+, pcbnew only) — copper pour inventory: `{zones:[{netCode, netName, layers, layerNames, priority, isFilled, isKeepout, clearanceMm, minWidthMm, polygonVertexCount, boundingBoxMm}]}`. - **`get_footprint_connections`** (v1.7.9+, pcbnew only) — `params:{ref}`. Net-graph traversal: for a given footprint reference, returns per-pad connectivity. `{found, ref, value, padCount, pads:[{padName, netCode, netName, connectedTo:[{ref, padName}]}]}`. Each `connectedTo` list excludes the pad itself. - **`get_drc_markers`** (v1.7.9+, pcbnew only) — read existing DRC markers from the board: `{markerCount, bySeverityCounts, markers:[{severity, severityCode, message, errorText, layer, layerName, positionMm, markerClass}]}`. Does NOT trigger DRC — for that use `kicad_run_drc`; this reads results from the LAST run. - **`export_svg`** (v1.7.8+, pcbnew only) — multi-layer SVG export via `pcbnew.PLOT_CONTROLLER` (in-process, NO `kicad-cli` subprocess). `params:{outputDir?, layers?, mirror?, plotFrameRef?, useAuxOrigin?, subtractMaskFromSilk?, drillMarks?, filenamePrefix?}`. Default `layers` is every enabled copper layer plus F.Silkscreen + B.Silkscreen + Edge.Cuts. Output files: `/-.svg` (or `--.svg` if `filenamePrefix` is passed). Returns `{plottedLayerCount, outputDir, files:[{layerId, layerName, filePath, fileSize}]}`. **Use timeout:30+ on the bridge_call** because plotting can take 5-15s. - **`export_pdf`** (v1.7.8+, pcbnew only) — same shape as export_svg, format=PDF. - **`select_and_load_symbol`** / **`select_and_load_footprint`** (v1.7.9+, EXPERIMENTAL) — filters the library tree (via `navigate_*`) then attempts to activate the first match. **Known to segfault eeschema on large symbol libraries — only safe on small/test libs.** Phase 2.ab will fix via keystroke simulation. Until then, the default `kicad_open_symbol_editor` / `kicad_open_footprint_editor` paths continue to use the proven PowerShell SendKeys load — they're NOT routed through this method. - `kicad_install_plugin` -- **v1.7.6+** Explicitly redeploy the Phase 2 reverse-bridge plugin to every detected KiCad version's USER_SITE. Bypasses the auto-install cache. - Args: `force` (optional bool, default true) - When to use: after updating the plugin payload between releases, or to recover from a partial-failure auto-install (read perVersion errorCodes from `pluginAutoInstall` first). - `kicad_open_editors` -- **v1.7.6+** Cross-process editor inventory sourced via the bridge (not screen scraping). Hits each alive plugin's `get_open_editors` and concatenates. - Returns: `{editors:[{kind, exeName, pid, frameTitle, hwnd, isShown}], byKind:{schematic_editor:[…], pcb_editor:[…], symbol_editor:[…], footprint_editor:[…], project_manager:[…]}, summary:{totalEditors, probedPlugins, failedPlugins, byKindCounts}}`. - When to use INSTEAD of `kicad_window_info`: when you have the bridge available (the data is more structured + cheaper than the screenshot-fallback path). ### Phase 3 — three-tier routing for `kicad_open_*` verbs (v1.7.6+) The legacy `kicad_open_symbol_editor`, `kicad_open_footprint_editor`, and `kicad_open_3d_viewer` verbs now try the bridge first before falling back to legacy WM_COMMAND-from-outside / icon-click / Alt+keystroke. Their responses include: - `pathway` — which tier won: `"plugin"` (bridge), `"wm_command"` (legacy external post), `"icon_click"` (footprint editor launcher icon), `"alt_keystroke"` (3D Viewer from PCB Editor). - `pathwaysTried` — ordered list of every tier attempted with success/failure annotations. - `bridgePid` — if pathway was `"plugin"`, which KiCad process handled it. For Symbol Editor specifically, tier-1 has two sub-paths because Symbol Editor is hosted inside eeschema.exe (not as a standalone process): - `plugin-eeschema-20390` — preferred when eeschema is already alive; sends menu_id 20390 to its Schematic Editor frame (opens Symbol Editor as a sub-window in the existing process). - `plugin-kicad-20011` — cold-start path; sends menu_id 20011 to kicad.exe Project Manager (spawns a new eeschema.exe with Symbol Editor as top-level). **Bug fixed in v1.7.6** (caught via Phase 4 menu catalog audit): `kicad_open_symbol_editor` was sending menu_id 20012 to the project manager, but 20012 is actually **PCB Editor** in KiCad 10. The correct ID is 20011. The legacy WM_COMMAND-from-outside path silently failed for KiCad 10 users (Symbol Editor never opened, sometimes a PCB Editor window flashed instead). v1.7.6 fixes the constant + adds the bridge tier-1 path that uses the proven eeschema-20390 mechanism whenever possible. **Key menu IDs catalogued** (see `phase4-results/*.json` in the plan repo for the full ~870-item dump): | From frame | Action | menu_id | |---|---|---| | Project Manager | Open Schematic Editor | 20010 | | Project Manager | Open Symbol Editor | 20011 | | Project Manager | Open PCB Editor | 20012 | | Project Manager | Open Footprint Editor | 20013 | | Schematic Editor | Switch to PCB Editor | 20150 | | Schematic Editor | Symbol Editor | 20390 | | Schematic Editor | ERC | 20001 | | Schematic Editor | Annotate Schematic | 20139 | | PCB Editor | DRC | 20088 | | PCB Editor | 3D Viewer | 20563 | | PCB Editor | Footprint Editor | 20567 | | PCB Editor | Switch to Schematic | 20234 | The bridge is a **Phase 2 MVP**. The legacy `kicad_open_*` / `kicad_window_info` / `kicad_screenshot_all` commands still work and remain the primary path for most flows; the bridge is additive. Phase 2.x will add `get_board_info`, `run_drc`, `navigate_symbol/footprint`, `export_svg`, and friends — when those land, this section will list them. #### KiCad UI Interaction Commands for detecting KiCad windows/dialogs, taking screenshots, clicking, and sending keyboard input. Essential for handling blocking dialogs and automating multi-window KiCad workflows. - `kicad_window_info` -- Returns all KiCad windows grouped as `projectManager`, `editors[]`, `modalDialogs[]` with HWNDs and titles. Uses **process-based enumeration** (finds all PIDs running from KiCad's bin dir, then enumerates their windows) — catches every KiCad window regardless of title. Check `hasModalDialogs` to detect blocking save/error prompts. - Example: `adom-desktop kicad_window_info` - Returns: `{projectManager: {hwnd, title}, editors: [{hwnd, title}], modalDialogs: [{hwnd, title, ownerHwnd}], hasModalDialogs: bool}` - `kicad_screenshot_all` -- Screenshots every KiCad window in one call (editors first, then project manager, then dialogs). Uses same process-based enumeration as `kicad_window_info`. Images downscaled to ≤1568px — use relative coords (0.0-1.0) for clicking, they're scale-independent. READ each screenshot to see what's on screen. - Example: `adom-desktop kicad_screenshot_all` - Returns: `{screenshots: [{type: "main"|"editor"|"dialog", title, hwnd, savedTo, sizeKB}]}` - `kicad_send_key` -- Send a keystroke to a KiCad window. **Preferred way to dismiss dialogs**: enter to confirm, escape to cancel, tab to cycle buttons. Use click only if tab order doesn't reach the right button. - Args: `key` (required: "enter", "escape", "tab", "space", "f1"-"f12", or single char), `hwnd` (optional — if omitted, sends to foreground KiCad window) - Example: `adom-desktop kicad_send_key '{"key": "enter"}'` (dismiss dialog) - Example: `adom-desktop kicad_send_key '{"key": "escape", "hwnd": 12345}'` (target specific window) - `kicad_click` -- Click at coordinates within a KiCad window. Relative coords (0.0-1.0) are scale-independent — estimate position as percentage from screenshot. To click a button at pixel (px,py) in an image of size (W,H), use `x=px/W, y=py/H`. - Args: `hwnd` (required), `x` (required), `y` (required), `relative` (optional bool, default true — 0.0-1.0 range) - Example: `adom-desktop kicad_click '{"hwnd": 12345, "x": 0.5, "y": 0.8}'` #### KiCad Dialog Detection Workflow KiCad has multiple independent windows (project manager, eeschema, pcbnew, footprint editor, symbol editor, 3D viewer). Unlike Fusion which has one main window, KiCad dialogs are owned by specific parent windows. **After any open/install command**, check for blocking dialogs: 1. `kicad_window_info` → check `hasModalDialogs` 2. If true: `kicad_screenshot_all` → READ each dialog screenshot 3. Dismiss with `kicad_send_key {"key":"enter"}` (OK) or `{"key":"escape"}` (cancel) 4. For specific button clicks: identify position in screenshot → `kicad_click {"hwnd":, "x":0.5, "y":0.8}` ### Fusion 360 Tools Fusion 360 uses a two-tier architecture: - **External bridge** (port 8773, auto-started): handles launch, detection, file opening - **AdomBridge add-in** (port 8774, runs inside Fusion): required for export, design queries, Electronics/EAGLE commands - `fusion_start` -- **PREFERRED.** First-class Fusion 360 startup. Idempotent — safe to call whether Fusion is running or not. Discovers `Fusion360.exe` via webdeploy glob (no hardcoded hashes, no "Windows cannot find" GUI dialogs), verifies path exists before spawn, polls the AdomBridge add-in until it's online, and calls `fusion_dismiss_blocking_dialogs` to clear the startup picker and any other blocking modals. **Auto-dismisses startup dialogs during add-in wait** — the "What do you want to design?" picker and "Recovered Documents" dialog are dismissed every 5 seconds while polling, so the add-in becomes responsive within ~10-15s instead of timing out at 60s. Returns `{addinReady, mainThreadResponsive, dialogsDismissed[], dialogsRemaining[], dismissHint, pid, resolvedPath, elapsedMs, alreadyRunning}`. **Has a hard process-detection gate** — will never spawn a second Fusion process, so the "Multiple instances are not supported" GUI dialog cannot reach an end user. Use this as the very first call before any other Fusion command. - `fusion_dismiss_blocking_dialogs` -- **First-class blocking-dialog killer.** Whenever any fusion_* command returns `errorCode: fusion_addin_not_responding` (or you see `_hint` mentioning a blocked add-in), call this FIRST before retrying. Two-layer design: (1) deterministic pattern match against known blockers — "What do you want to design?" picker, error toasts ("New design cannot be created..."), "Multiple instances" warning, crash recovery prompts, save prompts — dismisses each with `fusion_send_key escape` targeted at the exact hwnd. (2) AI-in-the-loop fallback — for anything the patterns didn't match, auto-screenshots every remaining Fusion dialog and returns the image paths inline in `remaining[].screenshotPath`, plus an explicit per-hwnd recovery script in `_hint` that you follow mechanically: Read the screenshot with the Read tool, understand what the dialog says, then `fusion_send_key '{"key":"escape","hwnd":}'` (or `"enter"` for OK-only dialogs). Re-call `fusion_dismiss_blocking_dialogs` to verify `mainThreadResponsive=true`, then retry your original command. **Fusion updates break pattern matches constantly, so the AI fallback is the critical path, not the deterministic side.** If you identify a new recurring pattern, add its title substring to `BLOCKING_DIALOG_PATTERNS` in `cli/src/commands.rs` so future sessions handle it automatically. - `fusion_addin_status` -- **Non-blocking** check of the AdomBridge add-in's busy state. Does NOT touch the Fusion main thread — queries the add-in's `/status` HTTP endpoint directly (~5ms). Returns `{busy, busyCommand, elapsedSeconds, requestId, walkProgress}`. Use this to check if a long-running command (walk, search) is in progress **from any bridge/session** before sending new commands. The CLI's auto-recovery path calls this before attempting dialog dismissal — if the add-in is just busy (not dialog-blocked), it returns `main_thread_busy` immediately instead of mashing Escape into a working Fusion. - Example: `adom-desktop fusion_addin_status '{}'` - When idle: `{"busy": false}` - During a walk: `{"busy": true, "busyCommand": "walk_cloud_tree", "elapsedSeconds": 42.3, "walkProgress": {"foldersVisited": 15, "filesFound": 87, "currentFolder": "Molecules/XRP", "queueSize": 8}}` ### Launching apps (generic) Two generic CLI commands exist for launching any Windows executable safely — they verify the target exists BEFORE handing it to the OS, so you never trigger a "Windows cannot find" GUI dialog: - `adom-desktop find_exe '{"name":"..."}'` -- Resolve an exe by absolute path, glob (e.g. `C:/.../webdeploy/production/*/Fusion360.exe` returns newest), bare name (searches PATH + Start Menu `.lnk` targets). Returns `{path, source}`. Does NOT launch. - `adom-desktop launch '{"path":"...", "args":[...], "cwd":"...", "detached":true}'` -- Same resolution rules as `find_exe`, then spawns. Fails in terminal (exit 1) with a clear error if the path doesn't exist. **Always prefer this over raw `start`.** For Fusion specifically, use `fusion_start` — it wraps `launch` plus the full startup-picker / add-in-readiness dance. - `fusion_import_step` -- Import a STEP/STL/IGES file into Fusion 360 - `fusion_open_lbr` -- Open an EAGLE .lbr library file - `fusion_open_electronics` -- Check if the Electronics workspace is active - `fusion_electron_run` -- Execute any EAGLE command via Electron.run. Returns rich state: `activeWorkspace`, `activeDocument`, `commandType`, `_hint`, and `fusionOperations` (diff of Fusion's internal operation log showing what actually fired). **Avoid blocking commands** — see list below. **Full EAGLE command reference:** [`skills/eagle-commands.md`](./eagle-commands.md) — popular/safe commands split from blocking/modal ones, with layer reference and chaining syntax. - `fusion_execute_text_command` -- Low-level app.executeTextCommand() access. Returns the command result plus workspace context. - `fusion_board_info` -- Get structured board data from the open PCB layout. Returns: component placements (name, package, x, y, rotation), net names, copper traces, layer setup, board thickness, DRC violations. Much richer than a screenshot — gives exact coordinates and connectivity. Requires a .brd board open in PCB Editor. #### `fusion_electron_run` — EAGLE Command Execution Executes EAGLE commands inside Fusion 360's Electronics workspace. Works in **Schematic Editor**, **PCB Editor (Board Layout)**, and **Electronics Library** contexts. **How it works:** Sends the command string via Fusion's `Electron.run` text command. EAGLE's `Electron.run` is fire-and-forget — it never returns output or throws on invalid commands. To compensate, the handler snapshots Fusion's internal operation log (`Diagnostics.RecentOperations`) before and after execution, returning a diff showing what actually fired. **Usage:** ```bash # Basic command adom-desktop fusion_electron_run '{"command": "WINDOW FIT"}' # Multiple commands in sequence (use semicolons) adom-desktop fusion_electron_run '{"command": "DISPLAY NONE; DISPLAY 1 16 17 18 20 21"}' # Navigate in library editor adom-desktop fusion_electron_run '{"command": "EDIT SOIC8.pac"}' adom-desktop fusion_electron_run '{"command": "EDIT RESISTOR.sym"}' adom-desktop fusion_electron_run '{"command": "EDIT MYDEVICE.dev"}' ``` **Response fields:** - `activeWorkspace` — Current workspace (Schematic Editor, PCB Editor, Electronics Library) - `activeDocument` — Name of the open document - `commandType` — Detected type: `view_control`, `layer_control`, `edit`, `design_rule`, etc. - `editorType` — For EDIT commands: `package`, `symbol`, `device` - `fusionOperations` — Array of Fusion operations that fired (diff of internal log) - `hint` — Human-readable description of what happened - `rawResult` — Raw return from Electron.run (usually empty) **EAGLE Command Reference — Safe for automation:** | Command | Context | Description | |---------|---------|-------------| | **View / Navigation** | | | | `WINDOW FIT` | Any | Zoom to fit all content | | `WINDOW (x1 y1 x2 y2)` | Any | Zoom to specific area (coordinates in current units) | | `DISPLAY ALL` | Any | Show all layers | | `DISPLAY NONE` | Any | Hide all layers | | `DISPLAY 1 16 17 18 20 21` | Board | Show specific layers by number | | **Grid** | | | | `GRID MM 0.1` | Any | Set grid to 0.1mm | | `GRID MIL 25` | Any | Set grid to 25mil | | `GRID INCH 0.05` | Any | Set grid to 0.05 inch | | **Board Layout** | | | | `RATSNEST` | Board | Recalculate airwires (unrouted connections) | | `RIPUP` | Board | Remove all routed traces | | `RIPUP *` | Board | Remove all traces (same as RIPUP with no selection) | | `ROUTE` | Board | Start auto-router | | `DRC` | Board | Run design rule check | | `BOARD` | Schematic | Switch to paired board layout | | `SCHEMATIC` | Board | Switch to paired schematic | | **Schematic** | | | | `VALUE value` | Schematic | Set component value | | `NAME name` | Any | Rename selected element | | `SMASH` | Any | Detach name/value labels from components | | **Library Editor** | | | | `EDIT name.pac` | Library | Open a package (footprint) for editing | | `EDIT name.sym` | Library | Open a symbol for editing | | `EDIT name.dev` | Library | Open a deviceset for editing | | `EXPORT SCRIPT 'path.scr'` | Library | Export entire library as EAGLE script | | **Scripting / Settings** | | | | `SET CONFIRM YES` | Any | Suppress confirmation dialogs | | `SET CONFIRM OFF` | Any | Re-enable confirmation dialogs | | `SCRIPT 'path.scr'` | Any | Run batch commands from a .scr script file | **EAGLE Layer Numbers (commonly used):** | Layer | Name | What it shows | |-------|------|--------------| | 1 | Top | Top copper | | 16 | Bottom | Bottom copper | | 17 | Pads | Through-hole pads | | 18 | Vias | Via holes | | 19 | Unrouted | Airwires (ratsnest) | | 20 | Dimension | Board outline (required for 3D) | | 21 | tPlace | Top silkscreen | | 22 | bPlace | Bottom silkscreen | | 25 | tNames | Top component names | | 27 | tValues | Top component values | | 29 | tStop | Top solder mask | | 31 | tCream | Top stencil/paste | | 51 | tDocu | Top documentation | **Blocking commands — AVOID from automation:** | Command | Why it blocks | |---------|---------------| | `WRITE` | Opens Save As dialog — use `fusion_save_lbr` or `fusion_close_document` instead | | `ADD` | Opens component picker dialog | | `SHOW name` | Opens interactive highlight mode | | `SET` (no params) | Opens settings dialog | | `GRID` (no params) | Opens grid settings dialog | | `EDIT new.sym` | Opens "Create new?" confirmation if symbol doesn't exist | | `CHANGE` | Opens interactive change mode | | `MOVE` | Opens interactive move mode | **Tips:** - Always run `WINDOW FIT` after opening a file or switching views - Use `DISPLAY NONE` then `DISPLAY ` to show only specific layers - Combine commands with `;` — e.g., `SET CONFIRM YES; RIPUP *; RATSNEST` - For board screenshots: `DISPLAY NONE; DISPLAY 1 16 17 18 20 21; WINDOW FIT` - Use `fusion_board_info` instead of EAGLE commands when you need structured data - `fusion_export_lbr` -- Export the open Electronics library as an EAGLE .scr script (note: does NOT include 3D package references — those are cloud-linked only) - `fusion_save_lbr` -- Save the open Electronics library as a .flbr file #### EAGLE Libraries with 3D Packages Fusion 360's .lbr format supports `package3d` elements that link footprints to 3D models. Key facts: - **3D models are cloud-hosted** — each `package3d` has a `wip_urn` (e.g., `urn:adsk.wipprod:fs.file:vf.xxxxx`) pointing to a Fusion cloud document. You cannot embed STEP files directly in .lbr XML. - **Creating 3D packages requires the Fusion UI** — use `Package3DCreateCmd` in Electronics Library Editor, which opens a new Design workspace where you model/import the 3D shape, then save to link it. - **`EXPORT SCRIPT` strips 3D references** — the .scr export only contains 2D data (symbols, footprints, devicesets). To preserve 3D links, keep the .lbr XML format. - **Fusion's built-in examples** have 3D packages — 34 of 37 libraries in the EAGLE examples directory (e.g., `Connector_USB.lbr`, `Resistor.lbr`, `Capacitor.lbr`) include `package3d` references. - **Example libraries location**: `%LOCALAPPDATA%/Autodesk/webdeploy/production//Applications/Electron/LibEagle/examples/libraries/examples/` To open a built-in example library for reference: ```bash # Find the examples directory first adom-desktop fusion_execute_text_command '{"command": "Python.RunScript C:/tmp/find_eagle_libs.py"}' # Then open one adom-desktop fusion_open_lbr '{"filePath": "/Connector_USB.lbr"}' ``` - `fusion_open_schematic` -- Open a .sch schematic in Fusion's Schematic Editor. Args: `filePath`. - `fusion_open_board` -- Open a .brd board layout in Fusion's Board Layout editor. Args: `filePath`. - `fusion_show_3d_board` -- Switch to 3D PCB board view (must have a .brd open). Board MUST have an outline on layer 20 (Dimension) or 3D generation fails. Auto-zooms to fit after switching. - `fusion_show_2d_board` -- Switch back to 2D board layout from 3D PCB view. EAGLE commands via `fusion_electron_run` only work in 2D. - `fusion_close_document` -- Close a document without save dialog. Args: `name` (optional, defaults to active doc), `save` (optional, default false). Essential for automation — avoids modal save dialog that blocks Fusion. - `fusion_document_info` -- List ALL open documents/tabs with name, type, and active status, plus detailed cloud info for the active document. Returns `openDocuments` array (every tab) and active doc details (cloud project, folder, file ID, version, save status). Lightweight — uses only in-memory data, no cloud API calls. Use this instead of `fusion_walk_cloud_tree` when you just need to know what's open. - `fusion_activate_document` -- Switch to a specific open document tab. Args: `name` (substring match, case-insensitive), `documentType` ("Electronics", "PCB", "FusionDesign", "Drawing"). Essential for automation — switch between schematic, board, and library tabs without user interaction. If no match found, returns the list of open documents so you can refine. - `fusion_close` -- Close Fusion 360. **Always call this when done with Fusion 360 commands to clean up.** - `fusion_dismiss_recovery` -- Dismiss recovery document dialogs (both "Recovered Documents" list and "Open recovery document instead?" prompts). Also relocates recovery files to `~/.adom/recovery/fusion/` for safekeeping. - `fusion_relocate_recovery` -- Proactively move Fusion crash recovery files to `~/.adom/recovery/fusion//` without dismissing any dialogs. Call this **before** launching Fusion to prevent recovery dialogs from appearing. Files are preserved (not deleted) so the user can manually restore them if needed. - `fusion_close_all_documents` -- Close all open documents. Args: `saveChanges` (default: false). Use before force-killing Fusion to prevent recovery files. #### Fusion 360 UI Interaction Commands for interacting with Fusion 360's UI — detecting dialogs, taking screenshots, clicking, and sending keyboard input. Essential for handling blocking dialogs and automating CEF-based UI elements. - `fusion_window_info` -- Returns the Fusion main window HWND, title, rect, and a list of all Qt dialog windows (recovery dialogs, wizards, file pickers). Essential for detecting blocking dialogs before/after operations. - Example: `adom-desktop fusion_window_info` - Returns: `{hwnd, title, rect, dialogs: [{hwnd, title, className, rect}]}` - `fusion_screenshot_fusion` -- Captures the Fusion main window or a specific dialog by HWND. Uses PrintWindow (works without bringing to foreground). Saves WebP to `C:/tmp/adom-desktop-screenshots/` (falls back to PNG if WebP unavailable). Downscaled to ≤1568px — use relative coords (0.0-1.0) for clicking, they're scale-independent. DPI-aware. - Args: `hwnd` (optional — dialog HWND to screenshot instead of main window) - Example: `adom-desktop fusion_screenshot_fusion` (main window) - Example: `adom-desktop fusion_screenshot_fusion '{"hwnd": 12345}'` (specific dialog) - `fusion_screenshot_all` -- Screenshots the main Fusion window and lists all dialog windows with their HWNDs. Use `fusion_screenshot_fusion {"hwnd": ...}` to capture each dialog. - Example: `adom-desktop fusion_screenshot_all` - `fusion_click_fusion` -- Click at coordinates within the Fusion window or a specific dialog. x/y are relative (0.0-1.0) by default. Set `"relative": false` for pixel offsets. Uses SendInput for CEF dialog compatibility. **Prefer `fusion_send_key` for dialogs** — enter/escape/tab covers most cases. - Args: `x` (required), `y` (required), `relative` (optional, default true), `hwnd` (optional — target a specific dialog HWND instead of main window) - Example: `adom-desktop fusion_click_fusion '{"x": 0.5, "y": 0.7}'` (main window) - Example: `adom-desktop fusion_click_fusion '{"hwnd": 12345, "x": 0.75, "y": 0.85}'` (dialog button) - `fusion_send_key` -- Send keyboard input to Fusion or a specific dialog via SendInput. **Preferred way to dismiss dialogs**: enter to confirm, escape to cancel, tab to cycle buttons. Use click only if tab order doesn't reach the right button. - Args: `key` (required), `hwnd` (optional — target a specific dialog HWND) - Example: `adom-desktop fusion_send_key '{"key": "escape"}'` (dismiss CEF dialog) - Example: `adom-desktop fusion_send_key '{"key": "enter", "hwnd": 12345}'` (confirm dialog) #### Fusion 360 Workflow Guide ##### Opening Electronics Projects 1. Open the `.fprj` file: `adom-desktop fusion_open_cloud_file '{"projectName":"Main","fileName":"DRV8411A","fileExtension":"fprj","folderPath":"Molecules/XRP/DRV8411A"}'` 2. Do NOT try to open `.fbrd` or `.fsch` directly — they fail or show a "Select Electronics Design File" dialog 3. Enter the board editor: `adom-desktop fusion_show_2d_board` 4. Enter the schematic editor: After step 3, use `adom-desktop fusion_electron_run '{"command":"EDIT .sch"}'` 5. Switch back to board: `adom-desktop fusion_electron_run '{"command":"EDIT .brd"}'` ##### Auto-Screenshot on Open Commands **Open commands automatically screenshot Fusion and return the images in the response.** The following commands include `postOpenScreenshot` in their `data` field: - `fusion_open_cloud_file`, `fusion_open_schematic`, `fusion_open_board`, `fusion_show_3d_board`, `fusion_show_2d_board` The response `data.postOpenScreenshot` contains: - `screenshots[]` — array of `{type, savedTo, sizeKB, title?, hwnd?}`. Type is `"main_window"` or `"dialog"`. - `message` — human-readable instruction to READ each screenshot and check for blocking dialogs - `dialogBlocking` — `true` if a modal dialog was detected **After receiving the response, you MUST:** 1. **READ each screenshot file** using the Read tool to visually inspect what Fusion shows 2. Look for blocking dialogs in the screenshots: - "What to design?" wizard → dismiss with `fusion_send_key {"key": "escape"}` - "PCB out of date" banner → click Update or X - "Recovered Documents" → `fusion_dismiss_recovery` - "Save changes?" → `fusion_send_key {"key": "escape"}` - Any other modal → identify and dismiss 3. **Screenshot again** after dismissing to confirm it's clear Screenshots are saved as WebP (lossless, ~20-40KB each, downscaled to ≤1568px) for token efficiency. Both the main Fusion window AND all Qt dialog windows are captured separately. **Why this matters**: Fusion shows Qt dialogs and CEF overlays that are invisible to the API. The `success: true` response from an open command does NOT mean the UI is ready — a dialog may be blocking all further operations. The API cannot detect these. Only a screenshot can. ##### Detecting and Handling Blocking Dialogs - The auto-screenshot captures dialog windows separately (look for `type: "dialog"` entries in `postOpenScreenshot.screenshots[]`) - Common blocking dialogs: "What do you want to design?", "Recovered Documents", "Open recovery document instead?", "Select Electronics Design File" - Dismiss recovery dialogs: `adom-desktop fusion_dismiss_recovery` - Dismiss CEF dialogs (inside main window): `adom-desktop fusion_send_key '{"key": "escape"}'` or `fusion_click_fusion` - Dismiss "What do you want to design?" wizard: `adom-desktop fusion_send_key '{"key": "escape"}'` - For manual screenshot of specific dialogs: `adom-desktop desktop_screenshot_window '{"hwnd": }'` ##### Preventing Recovery Documents - Recovery files are at: `%LOCALAPPDATA%\Autodesk\Autodesk Fusion 360\\CrashRecovery\` - Before force-killing Fusion, close all documents: `adom-desktop fusion_close_all_documents '{"saveChanges": false}'` - **Best practice**: Call `adom-desktop fusion_relocate_recovery` before launching Fusion. This moves recovery files to `~/.adom/recovery/fusion//` — preserving them for the user while preventing modal dialogs. - The `fusion_start` command also auto-relocates recovery files before starting Fusion. - The `fusion_dismiss_recovery` command handles both "Recovered Documents" list and "Open recovery document instead?" prompts, and also relocates files. ##### EAGLE Export Limitations - **Supported**: `EXPORT IMAGE`, `EXPORT NETLIST`, `EXPORT PARTLIST` - **NOT supported** (fail silently): `EXPORT DXF`, `EXPORT SVG`, `EXPORT DRILL` - Always verify export output: the updated `fusion_electron_run` now checks if the output file was created - Use `fusion_export_bom` and `fusion_export_cpl` for manufacturing data (these use the add-in's XML parser, not EAGLE export) - Use `fusion_export_gerbers` for Gerber files — produces a ZIP with all gerber layers (GTL, GBL, GTS, GBS, GTP, GBP, GTO, GBO, GKO, XLN). Auto-detects 2-layer vs 4-layer boards. ##### 3D Model Exports - From the 3D view (activate the .f3d document): `fusion_export_step`, `fusion_export_stl`, `fusion_export_3mf`, `fusion_export_f3d`, `fusion_export_usdz`, `fusion_export_iges`, `fusion_export_sat` - Switch to 3D view: `fusion_show_3d_board` or activate the .f3d document - **STEP** — Industry-standard CAD interchange (SolidWorks, CATIA, Creo). Highest geometric fidelity. - **IGES** — Legacy CAD interchange. Use STEP for modern workflows. - **SAT** — ACIS solid model format. Used by SolidWorks, SpaceClaim. - **STL** — Mesh format for 3D printing and visualization. Options: refinement "low"/"medium"/"high". - **3MF** — Modern 3D printing with color/material support and multi-body. - **F3D** — Native Fusion 360 archive. Preserves parametric features, sketches, timeline, component refs. Best for archival. - **USDZ** — **Best for digital twins and GLB conversion.** Preserves full component hierarchy (Board, copper layers, soldermask, Packages), PBR materials, and named nodes. Each component becomes a toggleable node in GLB viewers. Also viewable directly on iOS/macOS (Apple Quick Look / AR). - Digital twin pipeline: `fusion_export_usdz` → pull_file → `blender --background --python-expr "import bpy; bpy.ops.wm.usd_import(filepath='board.usdz'); bpy.ops.export_scene.gltf(filepath='board.glb')"` - **FBX** — Not available in current Fusion builds via the API. The command exists but fails clearly. Use `fusion_export_usdz` instead. - **DXF/DWG/OBJ/SKP** — Not available via the Fusion API (dialog-only). Commands exist but return clear errors with alternatives. ##### 3D Viewport Captures - `fusion_take_screenshot` — Capture the Fusion viewport at any resolution without opening a dialog. Uses Fusion's render API (`saveAsImageFile`). - Args: `outputPath` (required), `width` (default 1920), `height` (default 1080), `orientation` (optional) - Orientations: `home`, `front`, `back`, `top`, `bottom`, `left`, `right` - Example: Capture all 6 standard views for use as product images/icons: ```bash for orient in home front back top bottom left right; do adom-desktop fusion_take_screenshot "{\"outputPath\": \"C:/tmp/3d-${orient}.png\", \"width\": 1920, \"height\": 1080, \"orientation\": \"${orient}\"}" done ``` ##### Electronics Source File Export (`fusion_export_source`) - `fusion_export_source` — Export the active electronics document as `.fsch`, `.fbrd`, or `.flbr` source file. - Args: `outputPath` (required — full path with extension) - The extension determines the format: `.fsch` (schematic), `.fbrd` (board), `.flbr` (library) - Validates extension, creates output directory, verifies file was created, returns file size - **Full workflow to export both board and schematic from a cloud project:** ```bash # Step 1: Open the .fprj (NOT .fbrd/.fsch directly — those fail) adom-desktop fusion_open_cloud_file '{"projectName":"Main", "fileName":"MyDesign", "fileExtension":"fprj", "folderPath":"Molecules/MyDesign"}' # Step 2: Screenshot to check for blocking dialogs (always do this after open) adom-desktop fusion_screenshot_fusion # Step 3: Enter board view (MUST be Board Layout workspace, not 3D) adom-desktop fusion_show_2d_board # Step 4: Export .fbrd (board first — it's already in board view) adom-desktop fusion_export_source '{"outputPath": "C:/tmp/exports/MyDesign.fbrd"}' # Step 5: Switch to schematic (EDIT .s1 = first schematic sheet) adom-desktop fusion_electron_run '{"command": "EDIT .s1"}' # Step 6: Export .fsch adom-desktop fusion_export_source '{"outputPath": "C:/tmp/exports/MyDesign.fsch"}' # Step 7: Pull files to Docker adom-desktop pull_file '{"filePaths":["C:/tmp/exports/MyDesign.fbrd","C:/tmp/exports/MyDesign.fsch"], "saveTo":"/tmp/exports"}' ``` - **Critical gotchas:** - You MUST open the `.fprj` first — opening `.fbrd`/`.fsch` directly fails or triggers blocking dialogs - You MUST call `fusion_show_2d_board` before exporting `.fbrd` (the Electronics Design overview won't work) - Always export `.fbrd` FIRST (from board view), then switch to schematic for `.fsch` - If export fails with "file not found", the wrong workspace is active — screenshot to verify - After `fusion_open_cloud_file`, always screenshot to check for "Select Electronics Design File" dialog - `WRITE` command is blocked — it opens a blocking save dialog. Use `fusion_export_source` instead - Note: `WRITE` is blocked — it opens a "Version Description" dialog on cloud docs even with a path argument (tested 2026-04-10). Use `fusion_save_to_cloud` to save to cloud, `fusion_export_source` for Fusion-format, or `fusion_export_eagle_source` for plain EAGLE format. ##### Plain EAGLE Source Export (`fusion_export_eagle_source`) - `fusion_export_eagle_source` — Export the active electronics document as plain EAGLE XML `.sch` or `.brd`. - Args: `outputPath` (required — full path ending in `.sch` or `.brd`) - Internally: exports `.fsch`/`.fbrd` via `Document.CopyToDesktop`, then extracts the EAGLE XML from the ZIP container, cleans up the temp file. - The output is valid EAGLE XML (`...`) parseable by standalone EAGLE, KiCad import, or any XML tool. - Same workflow/prerequisites as `fusion_export_source` — just use `.sch`/`.brd` extensions instead of `.fsch`/`.fbrd`. ```bash # Board (.brd) — must be in PCB Editor / Board Layout adom-desktop fusion_show_2d_board adom-desktop fusion_export_eagle_source '{"outputPath": "C:/tmp/exports/MyDesign.brd"}' # Schematic (.sch) — must be in Schematic Editor adom-desktop fusion_electron_run '{"command": "EDIT .s1"}' adom-desktop fusion_export_eagle_source '{"outputPath": "C:/tmp/exports/MyDesign.sch"}' ``` ##### Dialog Dismissal (`fusion_close_window`) - `fusion_close_window` — Close a specific Fusion dialog by sending WM_CLOSE (equivalent to clicking X). - Args: `hwnd` (required — from `fusion_window_info` or `fusion_dismiss_blocking_dialogs` remaining[]) - Works on dialogs that Escape doesn't close — e.g. "Recovered Documents" - Does NOT force-kill — the dialog can still intercept WM_CLOSE ##### Electronics Import (Round-Trip) - `fusion_import_electronics` — Import Fusion-native `.fsch`, `.fbrd`, or `.flbr` files as new local documents - Uses `Document.newDesignFromLocal` under the hood - Auto-screenshots after import (catches blocking dialogs) - Args: `filePath` (required Windows path to .fsch, .fbrd, or .flbr) - After import, use `fusion_save_to_cloud` to persist to Fusion cloud - **`.fsch` import works standalone** — creates a new schematic project - **`.fbrd` import may fail** — boards have external references to schematics/libraries. Error: "New design cannot be created from a local file containing external references" - **`.flbr` import works standalone** — creates a new library project - For legacy EAGLE files: use `fusion_open_schematic` (.sch), `fusion_open_board` (.brd), `fusion_open_lbr` (.lbr) ##### Library Round-Trip Workflow Export and re-import Fusion electronics libraries: ```bash # 1. Open library from cloud adom-desktop fusion_open_cloud_file '{"projectName":"Main","fileName":"Adom Common Components","folderPath":"Molecules/Libraries"}' # 2. Export as .flbr (Fusion-native binary) and .scr (EAGLE script text) adom-desktop fusion_save_lbr '{"outputPath":"C:/tmp/library.flbr"}' adom-desktop fusion_export_lbr '{"outputPath":"C:/tmp/library.scr"}' adom-desktop fusion_close_document # 3. Re-import the .flbr adom-desktop fusion_import_electronics '{"filePath":"C:/tmp/library.flbr"}' # 4. Verify symbols survived round-trip adom-desktop fusion_export_lbr '{"outputPath":"C:/tmp/library-verify.scr"}' # 5. Save to cloud with new name adom-desktop fusion_save_to_cloud '{"name":"Library-copy"}' ``` ##### Demo Projects 171+ electronics molecules are exported in the `adom-desktop-demo` repo (separate from adom-desktop). Three reference boards with full export formats: | Board | Cloud Path | Layers | Components | Files | |-------|-----------|--------|------------|-------| | DRV8411A | Main / Molecules / XRP / DRV8411A | 4 | 29 | 27 | | DRV8323SR | Main / Molecules / Experiments / MotorControl / DRV8323SR | 2 | 43 | 27 | | VL53L8BreakoutMolecule | Main / Molecules / XRP / TimeOfFlightVL53L8 / VL53L8BreakoutMolecule | 2 | 30 | 27 | Each folder contains: .fsch, .fbrd, bom.csv, cpl.csv, gerbers.zip, 6 board images, 7 3D renders (home/front/back/top/bottom/left/right), STEP, IGES, SAT, STL, 3MF, F3D, USDZ, board screenshot, 3D board screenshot. #### Cloud Document Management Manage Fusion 360 cloud documents (hub projects, files, versions). Required for 3D package workflows since EAGLE library 3D models are stored as cloud documents. - `fusion_save_to_cloud` -- Save the active Fusion document to the cloud. Args: `name` (required), `projectName` (optional, defaults to active project), `folderPath` (optional), `description` (optional). Returns: `fileId`, `versionNumber`, `wipUrn` (if cloud-hosted). - `fusion_list_cloud_projects` -- List all cloud projects in the user's hub. Returns array of `{name, id}` per hub. - `fusion_list_cloud_files` -- List files in a cloud project/folder. Args: `projectName` (optional), `folderPath` (optional). Returns: `files` array with `{name, id, versionNumber, fileExtension, dateModified}` and `subfolders` array. - `fusion_create_cloud_folder` -- Create a folder in a cloud project. Args: `folderName` (required), `projectName` (optional), `parentPath` (optional). Returns `folderId`. Idempotent — returns existing folder if it already exists. - `fusion_check_recovery` -- Check if a cloud file has a recovery document from a previous crash. Args: `fileName` (required), `projectName` (optional), `folderPath` (optional). Returns `hasRecovery: true/false`. Use this BEFORE opening files to avoid the blocking "Open recovery document instead?" dialog. If recovery exists, call `fusion_open_cloud_file` with `recovery: "open"` (restore unsaved work) or `recovery: "discard"` (delete recovery, open cloud version). - `fusion_open_cloud_file` -- Open a cloud file in Fusion by name. **If a recovery document exists and no `recovery` arg is given, the command STOPS and reports the recovery instead of opening** — you must decide whether to preserve or discard unsaved work. Args: `fileName` (required), `projectName` (optional), `folderPath` (optional), `recovery` ("open" = restore unsaved work, "discard" = delete recovery and open cloud version). - `fusion_export_cloud_file` -- Export the active Fusion document to a local file for transfer back to Docker. Args: `outputPath` (required), `format` (optional, default "step"). The exported file can then be pulled back to Docker via `pull_file`. - `fusion_delete_cloud_file` -- Delete a cloud file by name. Args: `fileName` (required), `projectName` (optional), `folderPath` (optional). File must not be open in Fusion — close it first with `fusion_close_document`. - `fusion_walk_cloud_tree` -- **Long-running.** BFS walk of a cloud folder tree. Returns a flat list of all files and folders. Runs entirely on the Fusion main thread — **blocks all other add-in commands** until done (check progress with `fusion_addin_status`). - Args: `projectName` (optional), `folderPath` (optional, starting folder), `maxDepth` (default 10), `maxFolders` (default 500), `extensions` (optional list, e.g. `["f3d","fprj"]`), `nameContains` (optional substring filter), `includeFiles` (default true) - Returns: `{project, rootFolder, folders[], files[], stats: {foldersVisited, foldersSkipped, filesFound, maxDepthReached, truncated}}` - **Per-folder timeout (30s):** If a single folder's cloud API calls take >30s (common for large projects), the folder is skipped and counted in `foldersSkipped`. This prevents indefinite hangs. - **Progress tracking:** While running, `fusion_addin_status` returns `walkProgress` with `{foldersVisited, filesFound, currentFolder, queueSize}` — poll this every 1–10s to monitor progress (see "Live folder progress streaming" below). - **Non-blocking alternative:** Use `fusion_search_cloud_files` for targeted searches, or `fusion_list_cloud_files` for single-folder listings (these are faster but don't recurse). - Example: `adom-desktop fusion_walk_cloud_tree '{"projectName":"Main","folderPath":"Molecules","nameContains":"DRV","extensions":["fprj"]}'` #### Live folder progress streaming with `watch` For long walks, use the built-in `watch` wrapper. It spawns the inner search command on a worker thread, polls `fusion_addin_status` internally, and emits one JSON event per line to stdout as the walker visits each folder. **No manual polling loop needed.** ```bash adom-desktop watch '{"command":"fusion_walk_cloud_tree","args":{"projectName":"Main","folderPath":"Molecules","nameContains":"BQ25792"}}' ``` Output is one JSON object per line, three event types: ```jsonl {"event":"started","command":"fusion_walk_cloud_tree","args":{...},"interval":2,"_hint":"streaming progress events follow, one per line, ending with 'complete' or 'error'"} {"event":"progress","busyCommand":"walk_cloud_tree","elapsedSeconds":15.0,"foldersVisited":14,"queueSize":50,"filesFound":0,"currentFolder":"Molecules/Sensing"} {"event":"progress","busyCommand":"walk_cloud_tree","elapsedSeconds":23.4,"foldersVisited":22,"queueSize":108,"filesFound":0,"currentFolder":"Molecules/Examples"} {"event":"progress","busyCommand":"walk_cloud_tree","elapsedSeconds":31.8,"foldersVisited":30,"queueSize":100,"filesFound":0,"currentFolder":"Molecules/RAPID NAME TAGS/Connor Wood"} ... (one per real change in walkProgress, deduped) ... {"event":"complete","result":{"folders":[...],"files":[...],"stats":{"foldersVisited":169,"filesFound":12,"truncated":false}}} ``` Read stdout line-by-line. Stop when you see `event:complete` or `event:error`. The full final result is in the `complete` event's `result` field. **Optional `interval` arg** (default 2s, clamped to [1, 30]): ```bash adom-desktop watch '{"command":"fusion_walk_cloud_tree","args":{...},"interval":1}' ``` **For Claude Code consumers**: pipe `watch` directly into the `Monitor` tool. Each JSON line becomes a real-time event notification in the chat — you (and the user) see folder names appear one-by-one as the walker visits them. No bash loop, no `disown`, no subprocess gymnastics. Verified live in 1.3.16 on a 169-folder walk that returned all 12 BQ25792 cloud files cleanly. **Watchable commands** (whitelist): `fusion_walk_cloud_tree`, `fusion_search_cloud_files`. Other commands return `success:false` with a `_hint` listing the watchable set. **When NOT to use watch**: short single-folder operations (`fusion_list_cloud_files`, `fusion_open_cloud_file`) — those return in <2s and don't need streaming. The `watch` wrapper is purely for the long BFS commands. - `fusion_search_cloud_files` -- **Long-running.** Recursive substring search on file names across cloud folders. v1.0.2+ of the add-in adds: per-folder timeout, doEvents() every 50 files, per-file try/except, iterative BFS, and a stale-lock watchdog — Fusion stays responsive throughout (no "Not Responding" freeze). - **Args (no hard upper caps in v1.0.2+):** `query` (required, substring, case-insensitive), `projectName` (optional, defaults to active), `folderPath` (optional starting subfolder — **narrow with this**), `recursive` (default false), `maxDepth` (default 2), `maxFolders` (default 10), `maxResults` (default 20), `folderTimeout` (default 30s), `searchTimeout` (default 120s), `timeout` (HTTP envelope, default 620s). - **Before calling:** if the user knows roughly where the file is (e.g. customer-named folder), **ask them**. Naming a subfolder cuts 5-40 min searches down to ~10s. - **Why slow:** Autodesk's free Fusion 360 Python API has no indexed file-search endpoint. We walk the Data API one folder at a time — each = one HTTP round-trip to Autodesk. **This is an Autodesk API limitation, not adom-desktop's.** When telling the user the search is taking a while, attribute it to Autodesk / Fusion 360, not to adom-desktop's bridge. **There is now a FAST native alternative — `aps_search` (see "APS — fast indexed cloud search" below).** It hits Autodesk Platform Services' indexed Data Management API and returns ~instantly, even when Fusion is CLOSED. Call `aps_status` first; if `configured:true`, prefer `aps_search` and only fall back to this folder-walking verb when APS isn't set up. - **Interpreting results (critical for no-false-negatives):** the response contains `searchComplete` (bool), `foldersSkipped`, `filesSkipped`, `truncated`, `folderLimitReached`, `searchTimedOut`. `searchComplete:true` ONLY when **all five** are clean — that's the confidence flag. If `searchComplete:false`, the search hit a cap before exhausting the scope; **don't report "file not found"** — re-run with broader caps OR narrower `folderPath`. - **Case sensitivity:** fully case-insensitive both directions (query and file names lowercased). Substring match, not whole-word — `"cosm"` matches `COSMIIC`, `COSMOCOIL`, etc. - **Cost feedback in response:** `costAnalysis: {elapsedSeconds, foldersPerSecond, estimatedSecondsPer100Folders}` so the AI can budget the next search realistically. - **Real numbers from a live test:** searching `Main/Molecules` recursively (173 folders deep, max 8 depth) for "cosmiic" returned in 168 s with `searchComplete:true, totalFound:3`. Fusion stayed `main_thread:responsive` throughout. - **Pair with `watch`** for streaming progress updates: see below. #### APS — fast indexed cloud search (v1.8.182+) `aps_search` is a **native, ~instant** alternative to the folder-walking `fusion_search_cloud_files`. It queries **Autodesk Platform Services** (APS, formerly Forge) **Data Management API**, which has a real server-side index — so a search that takes minutes by folder-walking returns in well under a second, and it works **even when Fusion 360 is closed** (it's pure HTTPS, no bridge required). adom-desktop runs its own 3-legged **PKCE** OAuth, so the user's APS **client secret never leaves their machine**. **One-time setup (user side):** 1. Register a free app at → enable the **Data Management API**. 2. Set the app's **Callback URL** to exactly `http://localhost:8910/aps/callback`. 3. Put the **Client ID** in AD config key `aps_client_id` (in `~/.adom/config.json`) or via Settings. 4. Run `aps_auth` once — it opens the system browser for Autodesk sign-in. Tokens cache in `~/.adom/aps_tokens.json` and auto-refresh; you won't be prompted again. **Verbs:** - `aps_status` — **call this first.** Non-prompting. Returns `{configured, signedIn}`. `configured:false` → no client id; use `fusion_search_cloud_files`. `configured:true, signedIn:false` → run `aps_auth`. Both true → `aps_search` is ready. - `aps_auth` — interactive sign-in (opens browser). One-time per device. - `aps_hubs` — `{hubs[]{id,name}}` (your Autodesk teams). - `aps_projects {hubId?}` — `{hubId, projects[]{id,name}}` (default: first hub). - `aps_search {query, projectName?, projectId?, hubId?}` — `{query, hubId, count, results[]{name,id,type,lastModified,project}}`. `query` is a case-insensitive substring of the file's displayName. Without `projectName`/`projectId` it searches up to 8 projects in the hub. Open a hit with `fusion_open_cloud_file {fileName, projectName}`. **Decision flow for an AI doing a cloud file search:** `aps_status` → if `configured` (sign in with `aps_auth` if needed) use `aps_search`; else fall back to `fusion_search_cloud_files` and tell the user that registering a free APS app (above) would make future searches instant. **Export formats for `fusion_export_cloud_file`:** | Format | Extension | Use case | |--------|-----------|----------| | `step` | .step | Industry-standard CAD interchange (default) | | `stl` | .stl | 3D printing, mesh-based | | `f3d` | .f3d | Fusion 360 native archive (preserves all features) | | `iges` | .iges | Legacy CAD interchange | | `sat` | .sat | ACIS solid modeling kernel format | | `smt` | .smt | Parasolid format | Also supported for import via `fusion_import_step`: STEP (.step/.stp), STL (.stl), IGES (.iges/.igs), SAT (.sat), SMT (.smt), OBJ (.obj), F3D (.f3d). **Round-trip: Docker → Fusion Cloud → Docker** ```bash # === Docker to Fusion Cloud === # 1. Send file from Docker to Windows desktop adom-desktop send_files '{"files": [{"path": "/home/user/component.step"}]}' # 2. Import into Fusion adom-desktop fusion_import_step '{"filePath": "C:/Users/john/Downloads/component.step"}' # 3. Save to cloud (gets a wip_urn for 3D library linking) adom-desktop fusion_save_to_cloud '{"name": "my-component", "projectName": "Personal"}' # === Fusion Cloud to Docker === # 1. Find and open the cloud file adom-desktop fusion_walk_cloud_tree '{"projectName": "Personal", "nameContains": "my-component"}' adom-desktop fusion_open_cloud_file '{"fileName": "my-component", "projectName": "Personal"}' # 2. Export to local filesystem adom-desktop fusion_export_cloud_file '{"outputPath": "C:/tmp/export/my-component.step", "format": "step"}' # 3. Pull back to Docker adom-desktop pull_file '{"path": "C:/tmp/export/my-component.step"}' # === Cloud management === adom-desktop fusion_list_cloud_projects '{}' adom-desktop fusion_create_cloud_folder '{"folderName": "Electronics", "projectName": "Main"}' adom-desktop fusion_list_cloud_files '{"projectName": "Main", "folderPath": "Electronics"}' adom-desktop fusion_delete_cloud_file '{"fileName": "old-file", "projectName": "Main"}' ``` #### Manufacturing Exports (Gerbers, BOM, CPL) Export manufacturing files from an open PCB board — everything needed to fabricate boards and assemble components via Adom's PCBA service. **All manufacturing commands require a .brd board open in PCB Editor** (use `fusion_open_board` or `fusion_show_2d_board`). **Recommended workflow order:** `detect_layers` → `set_design_rules` → DRC → `export_gerbers` → `export_bom` → `export_cpl` → `pull_file` all back to Docker. Every manufacturing command returns structured JSON with: - `message` — AI-oriented summary explaining what was produced and why it matters - `nextSteps[]` — ordered list of what to run next in the manufacturing pipeline - `hint` — tips for interpreting results or recovering from issues - `data` — structured metadata (paths, counts, layer info) for programmatic use **Decision guide — which command to run:** - **Don't know the layer count?** → Run `fusion_detect_layers` first - **Need to check if the board meets fab specs?** → Run `fusion_set_design_rules` then DRC - **Ready to generate fab files?** → Run `fusion_export_gerbers`, then `fusion_export_bom`, then `fusion_export_cpl` - **Need visual review before export?** → Run `fusion_export_board_image` with preset `assembly_top` or `fabrication` - **Something failed?** → Check `data.hint` and `data.recoverySteps` in the error response **Commands:** - `fusion_detect_layers` -- Detect if the open board is 2-layer or 4-layer. Uses ULP script (primary) and CAM comparison (fallback). Returns `layerCount`, `copperLayers[]`, `method` (how it was detected), and `nextSteps[]`. **Run this first** — all other manufacturing commands auto-detect layers too, but running this explicitly gives you the data before committing to exports. - `fusion_set_design_rules` -- Apply Adom's JLCPCB-derived design rules (.edru XML files) to the open board. Auto-detects 2-layer vs 4-layer and loads the appropriate rule set. Returns `description` (human-readable rule summary), `edruFile`, and `nextSteps[]`. - `action`: `"apply"` (default) loads rules into the board; `"export"` saves current board rules to a file; `"show"` displays rule capabilities without modifying anything - `layers`: `"auto"` (default), `"2"`, or `"4"` — force a specific rule set - After applying: run `fusion_electron_run '{"command": "DRC"}'` to check for violations. DRC markers appear in the board editor. - `fusion_apply_instapcb_rules` -- Convenience wrapper that applies the bundled Adom InstaPCB design rule sets without needing a local `.edru` file. Args: `layers` (`"2"` or `"4"`). Sends the bundled `.edru` from Docker to Windows automatically and loads it via the EAGLE `drc load` command. Use this instead of `fusion_set_design_rules` when you just want Adom's standard 2-layer or 4-layer InstaPCB rules. - `fusion_load_design_rules` -- Generic loader for ANY custom `.edru` file by path. Args: `filePath`. Use this for third-party fab vendor rules (JLCPCB, PCBWay, OSHPark, etc.) that the user has on disk. - `fusion_export_gerbers` -- Export Gerber (RS-274X) + Excellon drill files as a ZIP. Auto-detects 2-layer vs 4-layer and selects the correct JLCPCB-compatible CAM job. Returns `zipPath`, `zipSizeKB`, `files[]` (list of gerber files in the ZIP with sizes), `layerCount`, and `nextSteps[]`. - `outputDir`: directory for the output ZIP (default: `C:/tmp/adom-gerbers/`) - `boardName`: prefix for the ZIP filename (default: from active document name) - `layers`: `"auto"` (default), `"2"`, or `"4"` — force CAM job selection - Output ZIP contains: GTL, GBL (copper), GTS, GBS (solder mask), GTP, GBP (paste), GTO, GBO (silkscreen), GKO (outline), XLN (drill). 4-layer boards also get G1, G2 (inner copper). - `fusion_export_bom` -- Export Bill of Materials as CSV. Groups identical parts by value+package with quantity counts. Returns `componentCount`, `uniquePartCount`, and `nextSteps[]`. - `outputPath`: (default: `C:/tmp/adom-bom.csv`) - `grouped`: `true` (default) groups by value+package; `false` lists every component individually - Output columns: Comment, Designator, Footprint, Quantity, Library — compatible with JLCPCB, PCBWay, Mouser, Digi-Key. - `fusion_export_cpl` -- Export Component Placement List (pick-and-place) as CSV. Returns `totalPlacements`, `topCount`, `bottomCount`, and `nextSteps[]`. - `outputPath`: (default: `C:/tmp/adom-cpl.csv`) - `side`: `"all"` (default), `"top"`, or `"bottom"` — filter for single-sided assembly - Output columns: Designator, Mid X, Mid Y, Layer, Rotation — coordinates in mm. - `fusion_export_board_image` -- Export PNG image of the board with layer presets. Returns `fileSize`, `preset`, and `nextSteps[]`. - `outputPath`: (default: `C:/tmp/adom-board.png`) - `dpi`: resolution (default: 300, max: 600) - `preset`: layer preset name (see table below) - `layers`: custom layer numbers as int array — overrides preset - `monochrome`: `true` for black & white - `listPresets`: set `true` to get available presets instead of exporting **Layer presets for `fusion_export_board_image`:** | Preset | Layers | Description | |--------|--------|-------------| | `all` | All | Every layer visible | | `top_copper` | 1, 17, 18 | Top copper + pads + vias | | `bottom_copper` | 16, 17, 18 | Bottom copper + pads + vias | | `top_silkscreen` | 21, 25 | Top silkscreen + component names | | `bottom_silkscreen` | 22, 26 | Bottom silkscreen + component names | | `top_soldermask` | 29 | Top solder mask openings | | `bottom_soldermask` | 30 | Bottom solder mask openings | | `top_paste` | 31 | Top paste/stencil openings | | `bottom_paste` | 32 | Bottom paste/stencil openings | | `board_outline` | 20 | Board outline (dimension layer) | | `drill` | 44, 45, 17, 18 | Drill holes + vias | | `assembly_top` | 1, 17, 18, 20, 21, 25, 51 | Top assembly — copper + silk + outline | | `assembly_bottom` | 16, 17, 18, 20, 22, 26, 52 | Bottom assembly — copper + silk + outline | | `fabrication` | 1, 16, 17–20, 21, 22, 25, 26, 29, 30, 51 | Full fabrication view | **Adom InstapcbPCB manufacturing capabilities (used by `fusion_set_design_rules`):** | Parameter | Metric | Imperial | |-----------|--------|----------| | Layers | 1, 2, 4, 6 | — | | Min trace width | 0.08mm | 3 mil | | Min trace spacing | 0.08mm | 3 mil | | Min via drill | 0.2mm | 8 mil | | Min silkscreen text | 0.1mm | 4 mil | | Board edge clearance | 0.05mm | 2 mil | | Board thickness | 1.6mm | 63 mil | | Copper weight | 0.5 oz (17.5 μm) | — | | Solder mask | Green | — | | Surface finish | HASL / ENIG | — | | Turnaround | 4 hours (fab + assembly) | — | **Manufacturing workflow — Fusion to Adom PCBA:** ```bash # 1. Open the board adom-desktop fusion_open_board '{"filePath": "C:/projects/myboard.brd"}' # 2. Detect layer count (auto-selects 2-layer or 4-layer rules/CAM) adom-desktop fusion_detect_layers # 3. Apply Adom design rules for the detected layer count + run DRC adom-desktop fusion_set_design_rules '{"action": "apply"}' adom-desktop fusion_electron_run '{"command": "DRC"}' # 4. Export manufacturing files (gerbers auto-select correct CAM job) adom-desktop fusion_export_gerbers '{"outputDir": "C:/tmp/mfg"}' adom-desktop fusion_export_bom '{"outputPath": "C:/tmp/mfg/bom.csv"}' adom-desktop fusion_export_cpl '{"outputPath": "C:/tmp/mfg/cpl.csv"}' # 5. Export board images for review adom-desktop fusion_export_board_image '{"outputPath": "C:/tmp/mfg/top-copper.png", "preset": "top_copper"}' adom-desktop fusion_export_board_image '{"outputPath": "C:/tmp/mfg/assembly-top.png", "preset": "assembly_top"}' adom-desktop fusion_export_board_image '{"outputPath": "C:/tmp/mfg/board-outline.png", "preset": "board_outline"}' # 6. Pull files back to Docker for Adom PCBA ordering adom-desktop pull_file '{"path": "C:/tmp/mfg/bom.csv"}' adom-desktop pull_file '{"path": "C:/tmp/mfg/cpl.csv"}' ``` ### Desktop Tools - `desktop_open_folder` -- Open a file or folder in Windows Explorer. If given a file path, Explorer opens with the file highlighted (selected). Args: `path` (file or folder path). - `desktop_open_url` -- Open a URL in the user's **NATIVE OS BROWSER** — i.e. the real Edge / Chrome / Firefox / Brave they use every day, with their saved logins, history, bookmarks, and extensions. This is NOT pup, NOT Chrome for Testing — it's the browser the human actually uses. Hand it off when the user needs to interact with a logged-in account themselves. - Args: `url` (required string), `browser` (optional: `"default"` (Windows-registered default — could be Edge, Chrome, Firefox, Brave; whatever they set), `"chrome"`, `"edge"`, `"firefox"`, `"brave"`). - Examples: - `adom-desktop desktop_open_url '{"url":"https://claude.ai/"}'` -- opens in user's default native browser, already signed in - `adom-desktop desktop_open_url '{"url":"https://docs.example.com","browser":"edge"}'` -- force native Edge - Returns `{ok, browser, url, exePath?}`. Allowed schemes: http, https, mailto, ftp, ftps. Other schemes refused. #### `desktop_open_url` vs `browser_open_window` — two completely different browsers | | `desktop_open_url` | `browser_open_window` | |---|---|---| | **What browser** | Native OS browser the user uses daily — Microsoft Edge / Google Chrome / Mozilla Firefox / Brave (whichever they've set as Windows default, or the one you name explicitly) | Puppeteer-controlled **Chrome for Testing** — a separate Chromium build that pup launches | | **Profile / data** | The user's real profile: saved logins, history, bookmarks, extensions, autofill | Isolated profile under `plugins/puppeteer/profiles//` — empty, no saved logins | | **Who interacts with the page** | The HUMAN. Claude hands the URL off and is done. | CLAUDE drives it programmatically — screenshots, clicks, eval, recording. The human just watches. | | **Use when** | A login is required (claude.ai, GitHub, internal tools, banking) — the human's existing session must be reused | Automation, scraping, screenshot capture, video recording, headful UI testing | | **Can Claude control it after launch?** | No — once handed off, Claude can't see or drive it | Yes — every `browser_*` verb (screenshot, eval, navigate, record, etc.) | **Decision rule:** human-in-the-loop login flow → `desktop_open_url`. AI-driven automation → `browser_open_window` (defaults to Chrome for Testing). AI-driven automation that **must** be the user's real, signed-in profile (GCP Console with org/projects, internal SSO behind a managed-Chrome policy, anything Context-Aware Access blocks for non-enrolled browsers) → `browser_open_window` with the new `nativeBrowser` + `nativeProfile` args (v1.0.5+, see below). #### When pup means "the user's real Chrome/Edge" — `browser_open_window` native mode (v1.0.6+, smart flow v1.0.11) Default `browser_open_window` (no native args) launches Chrome for Testing — same as day one. **Never auto-switch a plain pup request to native.** "Open this URL in pup", "screenshot this", "click that button" all stay on CfT. Native mode drives the user's **REAL browser** — their actual signed-in profile, with live cookies, org policies, and device enrollment intact (NOT a clone — v1.0.6 replaced the old clone approach). Opt-in via TWO additive args from `desktop_list_browsers`: ```bash adom-desktop desktop_list_browsers # installed browsers + profiles (+ the signed-in account per profile) adom-desktop browser_open_window '{ "sessionId":"gcp", "nativeBrowser":"chrome", # chrome / edge / brave "nativeProfile":"Default", # profile id: Default / Profile 1 / Profile 2 "url":"https://console.cloud.google.com" }' ``` **The smart flow.** CDP (the protocol pup drives) needs the browser running with `--remote-debugging-port`, which can only be set at launch. So the bridge handles each situation differently and tells you which happened: | Situation | What AD does | Response | |---|---|---| | Already driving that browser+profile | Reuse the live connection | `ok` + `narrative` | | Running **with** a debug port (power-user launched it that way, or AD did earlier) | Connect, no relaunch | `ok` + `narrative` | | **Not running at all** | **Launch it fresh with the debug port — NO prompt** (nothing to disrupt) | `ok`, `launchedFresh:true` + `narrative` — just proceed | | **Open without** a debug port | **Ask the user** — AD must close+reopen it | `errorCode:"native_relaunch_required"` + `confirmArgs` + `alternatives` | | Debug port **refuses to bind** (enterprise policy) | Won't close it again | `errorCode:"native_cdp_blocked"` + `diagnosis` + `alternatives` | **Handling the two consent/blocked codes:** - **`native_relaunch_required`** — the browser is open without the port, so driving it means closing + reopening it (with `--restore-last-session`, so all tabs/windows come back). **Ask the user's permission**, then re-call with `"nativeRelaunch":true`. The response's `alternatives` lists other installed browsers you could offer instead. - **`native_cdp_blocked`** — the debug port won't open, almost always an **enterprise policy** (`RemoteDebuggingAllowed` disabled via Google Workspace / Chrome cloud management — affects **Chrome browser-wide, every profile**). `diagnosis` says whether it's a confirmed local-registry policy or a likely cloud-managed one. AD will **not** close that browser again. `alternatives` names other installed browsers — **Edge usually escapes Google management**, so it often drives natively when Chrome can't. Offer the user the alternative, or fall back to generic CfT. **Always available — generic CfT.** `browser_open_window` with NO `nativeBrowser`/`nativeProfile` drives a clean Chrome-for-Testing instance: no user session, no managed policy, fully controllable. If a task needs the user's logged-in session there, have them sign in once. This is the universal fallback whenever native is blocked. **Power-user shortcut:** if the user launches their browser at startup with `--remote-debugging-port=`, AD reconnects every session with no prompt at all (a productized opt-in "keep native browsers debug-ready at boot" is on the roadmap, default OFF for security). **Routing rule — DON'T auto-escalate.** Treat native as opt-in: | User says | What to do | |---|---| | "pup" / "open in pup" / screenshot / eval / debug | DEFAULT pup (CfT). Never mention profiles. | | "native chrome" / "my real browser" / "my work profile" / "my `` profile" | Native. Call `desktop_list_browsers`. If exactly one profile clearly fits the request, use it and say which one ("opening in your `alice@acme.com` profile on Chrome"). If multiple profiles could fit (e.g. a `google.com` URL with both `alice@acme.com` and `alice@gmail.com` profiles signed in), **ASK** ("Which profile — work `alice@acme.com` or personal `alice@gmail.com`?") and then drive that pair. | | anything else | Default pup. | **Anti-fingerprint:** native sessions launch without `--enable-automation` and with `--disable-blink-features=AutomationControlled`. `navigator.webdriver === false`. No "controlled by automated software" banner. Sites that probe for automation see a normal browser. **Firefox** appears in `desktop_list_browsers` but is rejected for native drive with `errorCode:"firefox_no_cdp"` — Firefox doesn't speak the Chrome DevTools Protocol, so puppeteer can't drive it. If the user wants Firefox, hand off via `desktop_open_url` instead. **Native-mode error codes:** - `native_relaunch_required` — browser open without a port; ask the user, re-call with `nativeRelaunch:true` (see the smart-flow table above). - `native_cdp_blocked` — debug port policy-blocked; read `diagnosis`, offer `alternatives` or generic CfT (see above). - `firefox_no_cdp` — caller passed `nativeBrowser:"firefox"`. Firefox doesn't speak CDP; hand off via `desktop_open_url`. - `native_browser_not_installed` — that browser isn't installed on this machine. - `native_user_data_missing` — installed but never launched, so the User Data dir doesn't exist yet. Ask the user to open it once. - `native_profile_not_found` — `nativeProfile` isn't a real profile id under this browser's User Data. Call `desktop_list_browsers` to see what's valid. #### ⚠️ Common footgun — `localhost` / `file://` URLs from Docker AIs If you're running this CLI from a remote Docker container (galliaApril, dartv4, etc.), `browser_open_window` / `browser_open_tab` / `browser_navigate` will load the URL inside the USER's local pup — NOT inside your container. So a URL like `http://localhost:8080/foo.html` resolves to the user's machine, not your container's service. **v1.7.10+ detects this.** When you pass a URL whose host is `localhost`, `127.0.0.1`, `0.0.0.0`, `::1`, or scheme is `file://`, the response includes a `_hint` field explaining the issue and suggesting the public-slug URL pattern (`https://--.adom.cloud/proxy//`). **The call still proceeds** — rare legitimate cases exist where the user actually does have something running on their localhost. **Recipe — load something from inside your container into pup:** ```bash # 1. Start your service inside the container (e.g. a local HTML viewer): python3 -m http.server 8786 --bind 0.0.0.0 & # 2. Ask your USER for the container's adom.cloud slug suffix # (look at their browser URL when they open hydrogen or claude.ai/code). # Slug looks like: john-galliaapril-8v0y8o3547h2 # 3. Open the public URL — NOT localhost: adom-desktop browser_open_window '{ "sessionId":"demo", "url":"https://john-galliaapril-8v0y8o3547h2.adom.cloud/proxy/8786/foo.html" }' ``` If you don't have the suffix handy, just ask the user. Or if the file you wanted to show pup is one your container already wrote to disk, push it to a public location (wiki asset, S3) and load from there. - `desktop_bring_to_front` -- Bring a window to the foreground by HWND or title substring. Uses keybd_event + AttachThreadInput to bypass Windows foreground lock. Preserves maximized state. - Args: `hwnd` (int) OR `titleContains` (string, case-insensitive). One required. - Example: `adom-desktop desktop_bring_to_front '{"titleContains": "Fusion"}'` - `desktop_set_window_state` -- Change a window's show state without bringing it to the foreground (or both, if combined with `desktop_bring_to_front`). Args: `hwnd` (int) OR `titleContains` (string), plus `state` (one of `"maximize"`, `"minimize"`, `"restore"`, `"show"`, `"hide"`). Use to ensure Fusion or KiCad windows are maximized for screenshots, or to hide noisy background windows during demos. - Example: `adom-desktop desktop_set_window_state '{"hwnd":133560,"state":"maximize"}'` #### Real OS input — mouse + keyboard (v1.8.170+) `desktop_click` / `desktop_double_click` / `desktop_right_click` / `desktop_hover` / `desktop_type` / `desktop_press_key` synthesize **real OS input via `SendInput`** — so they land on **`chrome://` pages and native dialogs that browser CDP (`nbrowser_*` / `browser_*`) cannot reach**. The verb names mirror the browser extension's `nbrowser_*` so the model transfers between in-page and whole-desktop driving. - **Gated behind shell-auto-approve** (same risk class as `shell_execute` — keystrokes/clicks hit whatever is focused on the user's live machine). If it's off you get `errorCode:"input_not_approved"`; enable once with `adom-desktop shell_auto_approve '{"permanent":true}'`. Every action is written to the activity log. - **Focus a window first** by passing `{window:""}` (alias `titleContains`) or `{hwnd:<int>}` — reuses `desktop_find_window`'s resolver, so input lands on the intended window, not whatever happened to be focused. Get the hwnd from `desktop_list_windows` / `desktop_find_window`. - **Coords are physical screen pixels** — the same space as `desktop_list_windows` rects. Verbs: - `desktop_click {x, y, button?:"left"|"right"|"middle", double?, window?|hwnd?}` — left-click by default. `desktop_double_click` / `desktop_right_click` are wrappers; `desktop_hover` moves the cursor without clicking (reveals tooltips). - `desktop_type {text, window?|hwnd?}` — types a literal Unicode string (`KEYEVENTF_UNICODE`, any keyboard layout). - `desktop_press_key {keys:[...], repeat?, window?|hwnd?}` (alias `desktop_send_keys`) — chords/shortcuts. Modifiers `ctrl/alt/shift/win`; named keys `enter/tab/esc/space/backspace/delete/insert/home/end/pgup/pgdn/up/down/left/right/f1-f12`. Canonical "navigate a browser to a URL" flow (works on `chrome://` where CDP can't): ```bash adom-desktop desktop_press_key '{"keys":["ctrl+l"],"window":"Edge"}' # focus Edge + select the address bar adom-desktop desktop_type '{"text":"edge://extensions","window":"Edge"}' adom-desktop desktop_press_key '{"keys":["enter"],"window":"Edge"}' ``` For a control that isn't reachable by CDP/coords (e.g. the Reload button on `edge://extensions`, which lives in a shadow DOM), Tab-navigate to it then Enter: `desktop_press_key {"keys":["tab","tab","enter"]}`. **Visual "Adom cursor" (v1.8.171+)** — a click-through teal cursor + a label that narrates what you're driving (trust cue; byte-faithful to the browser extension's cursor). Pure visual, *not* gated: - `desktop_cursor {action:"show"|"move"|"hide", x, y, label}` — drive/narrate the cursor on its own ("watch me"). Returns the overlay `hwnd`. - Or pass `cursor:true` + `label:"Clicking Reload"` on `desktop_click`/`desktop_double_click`/`desktop_right_click`/`desktop_hover` — AD glides the teal cursor to the target (narrating the label) *then* does the real click. - **Auto-hide (v1.8.174+):** the cursor hides itself after `ttlMs` (default **60s**) so it never lingers on the user's screen if you forget to hide it. Each `show`/`move` refreshes the timer. Pass `ttlMs:<ms>` (≤600000) for longer, or `persist:true` to keep it until you explicitly `desktop_cursor {action:"hide"}`. If a cursor you showed was auto-hidden, the **next** cursor/click response prefixes its `_hint` with a NOTE so you learn (no chatty unsolicited message). #### Background input via UI Automation (v1.8.174+) — no foreground, no focus steal The mouse/keyboard verbs above use `SendInput` = real OS input, which only lands on the **foreground** window — so AD foregrounds the target first, yanking it in front of the user's work. When you must NOT steal focus, use **UI Automation** instead: `InvokePattern.Invoke()` / `ValuePattern.SetValue()` are **programmatic** — the app services them via its accessibility tree with **no cursor move, no focus change, no foregrounding**. Chromium/Edge expose their a11y tree to UIA, so `chrome://` page buttons/links are reachable **by accessible name** (not coordinates). - `desktop_find_control {window|hwnd, name|contains, role?}` — **read-only** discovery (ungated). Returns `best:{name, automationId, role, rect, invokable, settable}`. Call this first to learn whether a control is UIA-reachable. - `desktop_ui_click {window|hwnd, name|contains, role?}` — UIA **Invoke** by accessible name, in the background. Gated behind shell-auto-approve. - `desktop_ui_set {window|hwnd, name|contains, text}` — UIA **SetValue** on a text field, in the background. Gated. `role` ∈ `button|edit|link|checkbox|menuitem|tab|listitem|combobox|radiobutton|document|group|spinner`. **Caveat (surfaced in `_hint`, not hidden):** some shadow-DOM controls aren't in the UIA tree — if `find_control` reports `invokable:false`/not-found, the control genuinely needs a **foregrounded** `desktop_click`. So: prefer `desktop_ui_click` (background) → fall back to `desktop_click` (foreground) only when UIA can't reach it. #### Zero-math coordinate clicks — `coordMap` + image-space (v1.8.175+) You no longer recompute scale/offset/DPI to click what you see in a screenshot. Every screenshot verb returns a **`coordMap`**: ```jsonc "coordMap": { "shotId": "shot-w5050328-1", "image": { "w": 1372, "h": 1031 }, // the DELIVERED (safe) PNG's pixels "screenRect": { "x": 594, "y": 254, "w": 1372, "h": 1031 }, // physical px = input-verb space "dpiScale": 1.5, "imageToScreen": { "sx": 1.0, "sy": 1.0, "ox": 594, "oy": 254 } // screen = img*s + o } ``` Two ways to use it: - **Let AD convert (preferred, zero math):** `desktop_click {space:"image", shotId, x, y}` where `x,y` are pixels in that screenshot. Also works on `desktop_double_click`/`right_click`/`hover`. The `shotId` comes from the capture's `coordMap.shotId` (last ~64 shots retained; re-capture if it expires). - **Compute yourself:** `screenX = x*sx + ox`, `screenY = y*sy + oy`. `screenRect` is in the SAME physical-px space as `desktop_list_windows` rects and the `desktop_click` default (`space:"screen"`). **Popups (owned):** As of **v1.8.177**, `desktop_screenshot_window` **also captures the window's OWNED popups** — borderless confirm bubbles (Chromium's "Remove '\<ext\>'?"), owned dropdowns, owned dialogs — that don't enumerate in `desktop_list_windows` and aren't visible in a PrintWindow of the main window. They ride back as a top-level **`screenshots[]`** array (the same generic shape `kicad_screenshot_all` uses; the CLI auto-pulls each entry's `fullPath`→`localPath` and `safePath`→`localSafePath`). `ownedPopupCount` is always present (0 = none). Each entry carries `hwnd`, `title`, `kind:"owned_popup"`, and its own `coordMap`+`shotId`, so you can **click the bubble's buttons** with `{space:"image", shotId, x, y}` or drive them in the background by name via `desktop_ui_click {scope:"popups"}`. This is the generic mechanism — there's no app-specific "Chrome bubble" verb; every window screenshot surfaces any owned popups so you get the data and decide. **Use `desktop_screenshot_screen`** (full virtual-screen grab) only for popups *not owned* by your target window, or to see everything at once. **Screen geometry on every shot (v1.8.178):** every `desktop_screenshot_window` / `_screen` response now carries: - **`windowRect: {x,y,w,h,monitor}`** — the window's position + size on the overall (possibly multi-monitor) desktop, in physical screen px. Each owned popup entry in `screenshots[]` gets its own **`rect: {x,y,w,h,monitor}`** too — so you know where every captured surface sits, not just its pixels. - **`display: {virtualScreen, primary{w,h,dpi,scale}, monitors[], monitorOfWindow}`** — the full monitor topology: total virtual bounds, and per-monitor `bounds`/`dpi`/`scale`/`primary`. `monitorOfWindow` indexes into `monitors[]`. (Same data as the standalone `desktop_display` verb, folded in.) - **`drive: "background"`** — screenshots never foreground the window (PrintWindow/WGC). The companion `_related` field lists sibling screenshot/window verbs. **Background vs foreground driving (v1.8.178):** every desktop driving verb carries a structured **`drive`** field (`"background"` | `"foreground"` | `"n/a"`) — in `GET /commands`, in `adom-desktop help <verb>`, and echoed in the verb's own response. **Background** (no foreground / focus steal / cursor move — UIA + capture): `desktop_ui_click`, `desktop_ui_set`, `desktop_navigate`, `desktop_find_control`, `desktop_screenshot_window/_screen`, `desktop_record_window_start`. **Foreground** (SendInput → goes to the focused window, so AD foregrounds first): `desktop_click`/`_double_click`/`_right_click`/`_hover`, `desktop_type`, `desktop_press_key`, `desktop_bring_to_front`. Run **`adom-desktop help background`** for the full grouping. Rule: to act without disturbing the user's window, drive by NAME (`desktop_find_control` → `desktop_ui_click`/`ui_set`); fall back to coordinate clicks/keys only when the control isn't in the UIA tree. **Click a control by name → coords:** `desktop_find_control` returns `best.rect{x,y,w,h,centerX,centerY}` in screen px — `desktop_click {x:centerX, y:centerY}` to SendInput-click it (handy when `invokable`/`settable` are both false). Target by `name`, `contains`, `role`, or `automationId` (stable, locale-independent). **Keyboard chords:** `desktop_press_key` accepts a chord as one string `["ctrl+w"]` OR separate tokens `["ctrl","w"]` — both mean Ctrl held + W. A *sequence* of distinct chords uses `+` per element: `["ctrl+l","enter"]`. The reply echoes `chords` showing how AD interpreted your `keys`. - `desktop_flash_window` (v1.8.152+) -- Flash an **arbitrary** window's taskbar button orange (Win32 `FlashWindowEx`) as an "attention / ongoing activity" cue. Resolve the HWND with `desktop_list_windows` first. Args: `hwnd` (int) OR `titleContains` (string); `mode` (`"until_focused"` default — flashes until the user clicks the window; or `"count"`); `count` (int, `mode:"count"` only, default 3); `stop` (bool — clear an active flash). Returns `{ok, hwnd, title, mode}`. **Lightweight by design** (direct FlashWindowEx, no process spawn) so a bridge can call it after every verb. Distinct from the puppeteer bridge's `browser_alert_window`, which only flashes pup's OWN tracked windows — this flashes any window on the host. For progress + overlay-icon status, use `desktop_taskbar`. - Example: `adom-desktop desktop_flash_window '{"hwnd":133560}'` then `adom-desktop desktop_flash_window '{"hwnd":133560,"stop":true}'` to clear. - `desktop_taskbar` (v1.8.152+) -- Paint **glanceable status** on any window's taskbar button via `ITaskbarList3` (progress bar + overlay icon) + `FlashWindowEx`, so a bridge can show what the AI is doing on the **real app window it drives** (e.g. native-browser putting an "Adom is driving" badge + task progress on the Chrome/Edge button). Works on windows owned by **other processes**. Persistent COM (init once, reuse) — safe to tick progress frequently; no process spawn. Resolve the HWND via `desktop_list_windows`. Args (all channels optional; provide ≥1): - `hwnd` (int) OR `titleContains` (string) - `flash`: `"until_focused"` | `"stop"` - `progress`: `{state: "normal"|"paused"|"error"|"indeterminate"|"none", value?: 0..100}` — **state→color**: normal=green, **paused=yellow** (use for human-in-the-loop gates: captcha / 2FA / login), error=red, indeterminate=marquee, none=clear. - `overlay`: `{badge: "adom"|"none", icon?: "<.ico path or base64>", tooltip?: "<hover>"}` — built-in badges are `adom` (the Adom logo — a "driven by Adom" trust tell) and `none` (clear); 3rd-party bridges pass their own `icon` (`.ico` path or base64; PNG not yet supported). `tooltip` is the overlay's accessible description. - Returns `{ok, hwnd, title, applied:[...]}`. Clears: `flash:"stop"`, `progress.state:"none"`, `overlay.badge:"none"` — call these on task-end / window-close to clean up. - Example (long task, human gate, then done): ```bash adom-desktop desktop_taskbar '{"hwnd":133560,"overlay":{"badge":"adom","tooltip":"Adom is driving this browser"},"progress":{"state":"indeterminate"}}' adom-desktop desktop_taskbar '{"hwnd":133560,"progress":{"state":"paused"},"flash":"until_focused"}' # captcha — come look adom-desktop desktop_taskbar '{"hwnd":133560,"progress":{"state":"none"},"overlay":{"badge":"none"},"flash":"stop"}' # done — clear all ``` - `desktop_revoke_approvals` -- Revoke all shell auto-approve permissions and deny pending approvals. The next shell command will show the approval dialog. - Example: `adom-desktop desktop_revoke_approvals` - `desktop_caption` -- Show a global caption overlay that stays visible above ALL windows (KiCad, Fusion, Chrome). Native Win32 overlay — not a browser DOM element. Click-through: mouse events pass to windows below. Captured by screen recordings (browser `getDisplayMedia`, OBS, Game Bar). - Show: `adom-desktop desktop_caption '{"text":"Step 1: Opening the board","position":"top","size":"large","duration":3000}'` - Custom position: `adom-desktop desktop_caption '{"text":"Look here","x":0.3,"y":0.2,"size":"large","persist":true}'` - Hide: `adom-desktop desktop_caption '{"action":"hide"}'` - Multiple simultaneous captions (use `id` to keep them independent): ``` adom-desktop desktop_caption '{"text":"Step 1: Opening board","id":"step","position":"center","size":"large","persist":true}' adom-desktop desktop_caption '{"text":"REC ●","id":"status","position":"bottom-left","size":"medium","persist":true}' ``` - Replace only the step caption (status stays): ``` adom-desktop desktop_caption '{"text":"Step 2: Routing","id":"step","position":"center","size":"large","persist":true}' ``` - Hide just the step caption: `adom-desktop desktop_caption '{"action":"hide","id":"step"}'` - Args: - `text` (string) — caption text - `id` (string, optional) — identifies this caption. Captions with the same id replace each other; different ids coexist simultaneously. Default `"_default"` — so bare calls without id still replace each other for backward compat. - `position` — preset: `"top"`, `"bottom"` (default), `"center"`, `"top-left"`, `"top-right"`, `"bottom-left"`, `"bottom-right"` - `x` (float 0.0–1.0) — normalized screen X, overrides position horizontal. 0.0 = left edge, 1.0 = right edge. Centers the caption box on this point. - `y` (float 0.0–1.0) — normalized screen Y, overrides position vertical. 0.0 = top edge, 1.0 = bottom edge. - `size` — preset: `"large"` (72px), `"medium"` (32px, default), `"small"` (20px) - `fontSize` (int) — custom font size in px (8–400). Overrides `size` preset when provided. - `duration` (ms) — auto-dismiss timer. **Omitted or `0` → `30000` (30s); clamped to max `600000` (10 min).** Captions ALWAYS auto-expire (v1.8.138+) so a forgotten one can't be stranded on the user's desktop. To keep one up longer, **re-issue the same `id`** (resets the timer) or pass a bigger `duration`. - `persist` (bool, optional) — `true` = sticky, NO auto-dismiss. Supervised use only (e.g. demo recordings); you MUST then dismiss it yourself with `action:"hide"`. This is the only way to a never-expiring caption (`duration:0` no longer does that). - `action` — `"hide"` to dismiss a caption (with `id`: hides only that caption; without `id`: hides all captions), `"force-clear"` to EnumWindows and destroy ALL caption windows regardless of id (nuclear option — use at top of demo scripts for guaranteed clean slate) - Captions with the same `id` replace each other instantly (previous destroyed, no fade overlap). Captions with different ids coexist — multiple captions can be visible simultaneously. ### Desktop Screenshots Take screenshots of the user's desktop or individual windows. All screenshots use lossless PNG. **Always save screenshots** to `project-content/screenshots/`. **Naming convention:** `desktop-<descriptive-name>-YYYY-MM-DD-HhMMam/pm.png` **Workflow:** Call `desktop_list_windows` first to get HWNDs, then `desktop_screenshot_window` with the HWND. - `desktop_list_windows` -- List all visible windows (returns HWND, title, class name, position/size, and `z` = Z-order index, 0 = topmost) - `desktop_find_window` (v1.8.153+) -- Resolve a window's HWND by `titleContains` and/or `className` (case-insensitive substrings), returned in Z-order with `best` = the topmost match. Use this instead of diffing `desktop_list_windows` before/after launching something — it's race-free for "the window I just opened." Returns `{ok, count, best:{hwnd,title,className,z,rect}|null, windows[]}` (`ok` true only when ≥1 matched). Pass `best.hwnd` to `desktop_taskbar` / `desktop_flash_window` / `desktop_screenshot_window`. - Example: `adom-desktop desktop_find_window '{"className":"Chrome_WidgetWin_1","titleContains":"GitHub"}'` - `desktop_activity_filter` (v1.8.155+) -- Scope the **user's GUI activity log** so they watch the slice you care about while you work (e.g. focus it on one bridge before a debugging session). Args: `bridge` (a bridge registry name from `bridge_list` | `"none"` = only global/non-bridge AD commands | `"all"`/omit = everything), `text` (free-text substring, ANDs with bridge), `clear` (bool — reset). **Drives the human's GUI view only — it does NOT change which verbs you can call or what they return.** The GUI shows a status bar stating exactly what's shown + what's hidden. - Example: `adom-desktop desktop_activity_filter '{"bridge":"kicad"}'` then `adom-desktop desktop_activity_filter '{"clear":true}'` - `desktop_screenshot_window` -- Capture a specific window by HWND - `desktop_screenshot_screen` -- Capture the entire desktop (all monitors) ### Browser Automation (Puppeteer) Multi-session Puppeteer -- each session gets its own Chrome window. Auto-starts on first command. **Always pass `sessionId` on every command.** Two levels of granularity: **window** (one Chromium window per session, own taskbar icon) and **tab** (many tabs per session, one taskbar icon). Use windows when isolation matters, tabs when showing many related views (10 alignment previews, etc.) to avoid flooding the taskbar. **Window-level commands** (one session = one Chrome window): - `browser_rescan` -- (v1.5.1+) Recover orphaned Chrome windows whose CDP socket dropped. Walks every known profile (in-memory + on-disk session files), reconnects via the persisted DevToolsActivePort, then walks every page and rebuilds session entries by parsing the `(session: X)` tag injected into each page's title at launch. Idempotent. Args: `adoptOrphans?` (default false; when true, pages without a tag get attached under a generated sessionId — useful when Chrome has tabs the bridge has never seen). - **When you need this**: any time `browser_navigate` / `browser_eval` / `browser_screenshot` errors with `errorCode: "session_disconnected"` (the bridge knows about your sessionId but its CDP socket dropped) or `errorCode: "session_not_found"` (after a bridge restart that rebuilt from disk but couldn't reach the live Chrome). The 30s health check now auto-attempts reconnect, so most transient blips recover before you notice — but `browser_rescan` is the explicit recovery primitive. - Returns: `{ok, rescanned, profilesReconnected, profilesUnreachable, reattached, orphansAdopted, liveSessions, disconnectedSessions, _hint}`. - `browser_open_window` -- Open a Chrome window. **Default mode (CfT — Chrome for Testing):** args `sessionId` (required), `profile` (required), `url` (required), `freshProfile` (optional), `strictPermissions` (optional, default `false`), `downloadPath` (optional, default `%USERPROFILE%\Downloads`). **Native mode (v1.0.6+, opt-in — drives the user's REAL Chrome/Edge/Brave, actual signed-in profile):** instead of `profile`, pass both `nativeBrowser` (`chrome`/`edge`/`brave` from `desktop_list_browsers`) and `nativeProfile` (profile id: `Default`/`Profile 1`/`Profile 2`). The bridge smart-launches: fresh-launch if the browser is closed (no prompt, returns `launchedFresh:true`), reconnect if it already exposes a debug port, or returns `errorCode:"native_relaunch_required"` (ask the user, re-call with `nativeRelaunch:true`) if it's open without one. If the debug port is enterprise-policy-blocked it returns `errorCode:"native_cdp_blocked"` with a `diagnosis` + `alternatives` (other installed browsers — Edge often works when managed Chrome can't). See the "When pup means 'the user's real Chrome/Edge'" section above for the full flow + routing rules. - **Full-capability default (v1.6.3+):** every pup session opens with the capabilities chip-fetcher-style flows actually need: - **Scripted downloads enabled.** `Page.setDownloadBehavior` is called with `behavior: 'allow'` and `downloadPath: %USERPROFILE%\Downloads` (override via the `downloadPath` arg). Re-applied on every main-frame navigation as a safety belt. This unblocks the silent-download-failure scenario: before v1.6.3, JS-triggered downloads (clicks on `<a download>`, `fetch().then(blob → save)`, vendor "Download Symbol" buttons that do `window.location = url`) could be silently dropped by Chrome — no error, no UI, no file. chip-fetcher debugged this for hours before the fix. - **Clipboard read/write granted.** Paste-into-vendor-form flows work without permission prompts. - **Notifications + geolocation + MIDI denied.** Chrome doesn't pop up permission dialogs under automation. - The response includes `defaultPermissionsApplied: true`, `defaultPermissions: ["clipboard-read=granted", "clipboard-write=granted", "notifications=denied", "geolocation=denied", "midi=denied"]`, `downloadsEnabled: true`, `downloadPath: <resolved abs path>`. The path is what `desktop_watch_files` polls by default — the two ends meet without configuration. - **Opt-out:** pass `strictPermissions: true` to keep Chromium's defaults in place. Rarely needed — only when driving an untrusted site you're auditing. With strict mode, scripted downloads may be silently dropped, clipboard prompts fire, etc. - **History note:** v1.4.12 used `Browser.grantPermissions(['automaticDownloads', 'notifications', 'clipboardReadWrite'])` — but Chrome 146 rejects `automaticDownloads` as an unknown permission name, taking the whole grant down. v1.6.3 fixes by using `Browser.setPermission` per-permission (per-permission failure tolerance) AND by realizing `Page.setDownloadBehavior` is the actual mechanism for unblocking scripted downloads, not a permission grant. - `browser_close_window` -- Close a session's Chrome window (closes ALL tabs in that session). Args: `sessionId` - `browser_list_windows` -- List all open sessions with URL, title, tabCount, active status **Tab-level commands** (manage tabs within a session's window): - `browser_open_tab` -- Add a tab to an existing session. Returns `{tabId, url, title, active}`. Args: `sessionId`, `url` - `browser_switch_tab` -- Make a specific tab the active one (calls `bringToFront`). Args: `sessionId`, `tabId` - `browser_close_tab` -- Close a specific tab. Leaves other tabs + session intact. Args: `sessionId`, `tabId` - `browser_list_tabs` -- List all tabs in a session: `{tabs:[{tabId,url,title,active,errorCount,opener,openerTabId}], activeTabId, count}`. Args: `sessionId`. **Auto-tracks popup tabs since v1.4.7** — tabs the page spawned via `window.open()` / `target="_blank"` / form-submit-with-target=_blank appear here automatically with `opener:"popup"` and `openerTabId` pointing at the tab that triggered them. Tabs Claude opened via `browser_open_window`/`browser_open_tab` have `opener:"user"`. **Tab-aware operations** (all accept optional `tabId`; omit = use active tab): - `browser_navigate` -- Navigate to a new URL. Args: `sessionId`, `url`, `tabId?` - `browser_screenshot` -- Capture screenshot, auto-resized to <=1568px. Args: `sessionId`, `maxWidth`, `fullPage`, `tabId?` - `browser_eval` -- Evaluate JS in the page context. Args: `sessionId`, `expr`, `tabId?` - `browser_input_dispatch` -- **Trusted** click / type / key / move via Chromium's CDP-backed `Input.dispatchMouseEvent` / `Input.dispatchKeyEvent`. Events have `isTrusted: true`, so they pass framework click-handler gates (React/Vue/Svelte) and most vendor anti-automation checks that `browser_eval`-side `dispatchEvent(new MouseEvent(...))` silently fails on. Args: `sessionId?`, `tabId?`, `type` (`click`|`move`|`type`|`key`), plus per-type fields: - `click`: either `selector` (uses bounding rect — preferred when DOM is stable) OR `{x, y}` viewport coords. Optional `button` (`left`/`right`/`middle`), `clickCount` (1 or 2 for double-click), `delay`, `firstMatch` (default false; see below). - `move`: `{x, y}` plus optional `steps` for human-like trajectory. - `type`: `text` to type; pass optional `selector` to focus that element first; optional `delay` between keystrokes. - `key`: a Puppeteer KeyInput name (`Enter`, `Escape`, `Tab`, `ArrowUp`, `F1`, etc). - **Smart selector pick (v1.4.8+, modal-scoped in v1.4.9+, plausibility-filtered in v1.4.10+):** when a `selector` matches multiple elements (a class like `button.wg-button--primary` often does — cookie X buttons + duplicate hidden copies + the actual submit), the bridge picks intelligently: 1. **If a plausible modal/dialog is open** — `<dialog open>`, `[aria-modal="true"]`, `[role="dialog"]`, OR a fixed/absolute element with z-index ≥100 covering >25% of the viewport — AND it contains at least one visible interactive element with text — restrict the candidate set to elements **inside** that modal first. Tiebreak by largest visual area inside it. The page behind a modal is visually inert; clicking elements behind it is almost never what the caller intended. The "plausibility filter" added in v1.4.10 prevents empty overlay containers (Vue-Toastification toast wrappers, notification mount points, full-screen ad scrims) from being mistaken for the real modal — a common false-positive on Vue/React apps that mount toast portals at body level. 2. Otherwise filter to **visible elements with non-empty text** and tiebreak by **largest visual area**. 3. Fall back to visible-only, then document-order first as last resort. - Response includes `matchedCount`, `chosenIndex`, `clickedRect:{x,y,w,h}`, `clickedText`, `insideModal`, `modalDetected`, `modalRoot:{tag,id,cls,z,role,ariaModal}|null` (v1.4.10+ — which DOM node won the modal-detection heuristic, for diagnosing false-positives), and `pickStrategy` (`only-match` | `first-match` | `modal-scoped-largest` | `visible-text-largest` | `visible-text-largest-no-modal-match` | `visible-largest` | `fallback-first-document-order`). **Always sanity-check the response** — if `clickedText` doesn't look right, inspect `modalRoot` to see whether modal detection latched onto the intended overlay or got fooled by a sibling, then refine the selector OR pass `firstMatch:true` to skip the smart pick OR fall back to `{x, y}` coords. - **Use this whenever a click "lands but does nothing"** — most vendor "Generate Datasheet" / "Download" / "Submit" buttons gate on `event.isTrusted`. After clicking, follow up with `browser_list_tabs` (popups auto-track) to grab any new tab the click spawned. - `browser_fetch_url` -- Fetch an arbitrary URL with the session's cookies and return raw bytes. Bypasses Chrome's PDF Viewer wrapper — critical for grabbing PDF binaries from popup tabs where in-page `fetch(location.href)` returns the viewer HTML wrapper (~200 KB stub) instead of the actual PDF (multi-MB). Also handles ZIPs, CAD bundles, anything served as a binary content-type. - Args: `sessionId?`, `tabId?` (picks cookie context — defaults to active tab), `url` (required), `method?` (default GET), `headers?` (object; Cookie auto-included from session), `body?` (raw string). - **`saveTo` writes on the DOCKER container's filesystem** (the CLI handles the write after receiving bytes from the bridge). Pass an absolute path. Parent dirs are created. Returns `savedTo` with the canonical absolute path (verified to exist on disk) plus `savedToFilesystem:"container"`. - **`desktopSaveTo` writes on the WINDOWS desktop's filesystem** (the bridge writes directly via `fs.writeFileSync`). Pass an absolute Windows path (e.g. `C:\\Users\\you\\Downloads\\foo.pdf`). Returns `desktopSavedTo` with the resolved absolute path. Use this when the user wants the file local on the desktop (e.g. dropping a CAD bundle into Downloads). - **Without saveTo or desktopSaveTo**, returns `bodyBase64` in the response — caller decides what to do. - Returns: `{ok, bytes, contentType, status, sessionId, tabId, bodyBase64?, savedTo?, savedToFilesystem?, desktopSavedTo?, desktopSaveError?}`. - **CRITICAL READ-THE-RESPONSE rule:** when you pass `saveTo`, the response's `savedTo` is the actual filesystem path the file was written to. v1.4.8 had a bug where the bridge would respond `savedTo:"/tmp/foo.pdf"` but the file existed nowhere; v1.4.9 fixes this — `savedTo` always reflects a real on-disk path. - **Recipe — popup PDF capture (the right pattern):** ```bash # After browser_input_dispatch click that spawns a PDF popup, READ THE URL # FROM browser_list_tabs (don't eval against the popup's tabId — popups # auto-close fast and eval-after-close errors with a structured hint). LIST=$(adom-desktop browser_list_tabs '{"sessionId":"chip-fetcher"}') POPUP_URL=$(printf '%s' "$LIST" | jq -r '.tabs[] | select(.opener=="popup") | .url' | head -1) POPUP_TAB=$(printf '%s' "$LIST" | jq -r '.tabs[] | select(.opener=="popup") | .tabId' | head -1) adom-desktop browser_fetch_url "{ \"sessionId\":\"chip-fetcher\", \"tabId\":\"$POPUP_TAB\", \"url\":\"$POPUP_URL\", \"saveTo\":\"/tmp/datasheet.pdf\" }" # → {ok:true, savedTo:"/tmp/datasheet.pdf" (exists), savedToFilesystem:"container", # bytes:3798336, contentType:"application/pdf", status:200} ``` - **Per-call auto-recovery (v1.6.1+):** every tab-aware verb (`browser_navigate / eval / screenshot / input_dispatch / errors / reload / fetch_url`) now **silently reconnects or relaunches** before surfacing any session error. The Docker container should NEVER see "Session closed" / "detached Frame" / "session_disconnected" again unless recovery is genuinely impossible. The recovery ladder: 1. Live session → return immediately. 2. Session is `_lostBrowser` (CDP socket dropped after sleep/wake / network blip / puppeteer hiccup): try `attemptReconnectProfile()` which checks (a) the on-disk session file's recorded `cdpPort`, (b) the profile dir's `DevToolsActivePort`, and (c) **scans running Chrome processes for a `--remote-debugging-port=N` matching this profile dir** (this catches the canonical sleep/wake symptom — Chrome alive, CDP serving, but the bridge has no on-disk record). If reconnect succeeds, walk pages and re-attach by parsing the `(session: X)` tag from titles. 3. Reconnect failed: kill any orphan Chrome processes for this profile, clean the profile lock files (`lockfile` on Windows, `SingletonLock` / `SingletonSocket` / `SingletonCookie` on POSIX, plus stale `DevToolsActivePort`), then relaunch via `launchSession` with the last-known URL from the session file. The sessionId is preserved — the caller gets a transparent recovery. 4. Total failure (no profile name, no last-known URL, can't launch) → only THEN does the structured `session_not_found` / `session_disconnected` error fire. When recovery happens, the response includes `_recovered: true` and `_recoveryReason: "reconnect-no-page" | "relaunch"` so callers can log / metric the recovery rate. - **Stale sessionId detection (v1.5.1+):** when auto-recovery (above) genuinely cannot fix the problem (no profile known, no last-known URL, Chrome won't relaunch — rare), a structured error fires. Two error codes you might see: - **`session_not_found`** — the bridge has no record of this sessionId. Recipe: check `liveSessions` for the right name, or `browser_open_window` if you want to start fresh. - **`session_disconnected`** — the bridge has the session in memory (and on disk) but auto-recovery couldn't reach its Chrome. Recipe: call **`browser_rescan`** to force a manual recovery pass, or `browser_open_window` with explicit URL to relaunch. - **`browser_rescan`** (v1.5.1+, enhanced v1.6.1+) — explicit recovery primitive. v1.6.1 adds a running-process scan: walks all Chrome processes whose `--user-data-dir` is under the puppeteer profiles dir, even if the bridge has no in-memory or on-disk record of them. This is what fixed the chipsmith-after-sleep scenario where the bridge had been restarted while Chrome was alive — neither in-memory state nor session files referenced the live Chrome, but the process list did. - **Stale tabId detection (v1.4.10):** every tab-aware verb (`browser_navigate`, `browser_screenshot`, `browser_eval`, `browser_input_dispatch`, `browser_errors`, `browser_reload`, `browser_close_tab`, `browser_fetch_url`) returns a structured error when you pass a `tabId` that no longer exists: ```json { "ok": false, "errorCode": "tab_not_found", "error": "Tab \"tab-3\" not found in session \"chip-fetcher\".", "currentTabs": ["tab-1", "tab-2"], "lastKnownUrl": "https://wago.priintcloud.com/datasheets/2601-3105/en/...", "opener": "popup", "openerTabId": "tab-1", "closedMsAgo": 1842, "_hint": "This tab closed 2s ago at URL https://... Call browser_fetch_url with that URL (and the opener tabId \"tab-1\" for cookies) to get its bytes — do NOT try to eval against the closed tabId." } ``` - `lastKnownUrl` + `opener` + `openerTabId` + `closedMsAgo` are populated when the bogus tabId matches a tab that closed in the last few seconds (kept in a 20-entry per-session ring buffer). Critical for popup PDF workflows where the viewer auto-closes after rendering — you can `browser_fetch_url` directly against `lastKnownUrl` instead of having to re-trigger the spawning click. - **Important regression note:** v1.4.9 documented this error but the CLI was silently stripping `tabId` from `browser_navigate` / `browser_screenshot` / `browser_eval` / `browser_errors` / `browser_reload` calls before forwarding to the bridge — so the bridge always saw "no tabId" and fell back to the active tab. v1.4.10 fixes the CLI to forward `tabId` so the structured error actually fires. If you're still seeing silent fallback, your CLI is < v1.4.10 (`adom-desktop --version` to check). - `browser_errors` -- Collected console/page errors. Args: `sessionId`, `clear` (default true), `tabId?` - `browser_reload` -- Reload the page. Args: `sessionId`, `tabId?` **Other**: - `browser_status` -- All sessions with URLs, tab counts, and error counts - `browser_close` -- Close ALL sessions - `browser_wait` -- Wait for content to settle. Args: `ms` #### Credential vault — silent HTTP Basic Auth (v1.4.4+) When pup navigates to a host that issues an HTTP Basic Auth challenge (every `*.componentsearchengine.com` subdomain that wraps NXP / Mouser / TI / etc CAD bundles, several vendor design + doc portals), Chrome shows a NATIVE auth dialog that lives outside the page DOM. `browser_eval` can't dismiss it. Without the credential vault, the user has to type the same email + password every navigation — chip-fetcher hits this 5–10 times per chip. The vault stores credentials in the OS keychain (Windows DPAPI / macOS Keychain / Linux libsecret) and applies them via `page.authenticate()` BEFORE the navigation, so the dialog never appears. **Passwords never leave the keychain** — `credential_list` returns host + username only. - `credential_set` -- Store creds for a host pattern. Encrypted at rest by the user's session key. - Args: `host` (glob: `"*.componentsearchengine.com"` or exact `"nxp.componentsearchengine.com"`), `username`, `password` - Returns: `{ok, host, username}` (no password) - `credential_list` -- List stored entries. - Returns: `{credentials: [{host, username, addedAt, updatedAt}]}` - Passwords are **never** returned by this verb. - `credential_delete` -- Remove a credential entry. Drops the password from the keychain and the entry from the index. - Args: `host` (must match the pattern used at `credential_set` time, exactly) **Host matching:** simple glob. `*.example.com` matches any subdomain; exact strings match exactly. Best-match wins (longest non-glob suffix beats shorter glob), so a `nxp.componentsearchengine.com` entry would override a `*.componentsearchengine.com` entry for that exact host. **Recipe — chip-fetcher / CSE flow:** ```bash # 1. One-time setup (ask user for their CSE creds, then store) adom-desktop credential_set '{ "host":"*.componentsearchengine.com", "username":"alice@gmail.com", "password":"<paste>" }' # → {ok:true, host:"*.componentsearchengine.com", username:"alice@gmail.com"} # 2. Navigate freely — no popup adom-desktop browser_open_window '{ "sessionId":"chip-fetcher", "profile":"chip-fetcher", "url":"https://nxp.componentsearchengine.com/preview_newDesign.php?..." }' adom-desktop browser_eval '{"sessionId":"chip-fetcher","expr":"document.title"}' # → "SamacSys Part Preview" (NOT "Sign in") # 3. Audit — passwords are never returned adom-desktop credential_list # → {credentials:[{host:"*.componentsearchengine.com", username:"alice@gmail.com", ...}]} # 4. Remove adom-desktop credential_delete '{"host":"*.componentsearchengine.com"}' ``` Hooks fire BEFORE every `page.goto` in three places: `browser_open_window`'s first nav, `browser_open_tab`, `browser_navigate`. The credential lookup is `O(N)` over stored entries which is fine for typical N (single digits). If a host has no stored creds, navigation proceeds normally — the user will see Chrome's native dialog as before. There's no fallback prompt; the next step is for you to ask the user for the credential, then `credential_set` it. Example — show 10 alignment previews in ONE Chrome window: ```bash adom-desktop browser_open_window '{"sessionId":"align","profile":"align","url":"http://127.0.0.1:8901/"}' for port in 8902 8903 8904 8905 8906 8907 8908 8909 8910; do adom-desktop browser_open_tab "{\"sessionId\":\"align\",\"url\":\"http://127.0.0.1:$port/\"}" done adom-desktop browser_list_tabs '{"sessionId":"align"}' # see all 10 adom-desktop browser_screenshot '{"sessionId":"align","tabId":"tab-3"}' adom-desktop browser_eval '{"sessionId":"align","tabId":"tab-5","expr":"document.getElementById(\"meta-seat-z\").textContent"}' ``` - `browser_alert_window` -- Flash the Windows taskbar orange. **Always call after updating a pup window.** Args: `sessionId` - `browser_focus_window` -- **Tab-only.** Calls Puppeteer's `page.bringToFront()` — activates the session's tab WITHIN its Chrome window, but does NOT raise the OS window above other apps on the desktop. Args: `sessionId`. **For OS-level foreground use `browser_raise_os_window` (below).** - `browser_raise_os_window` -- **Real OS foreground raise.** Brings the Chrome window hosting this pup session above all other desktop apps. Use this BEFORE recording, screenshots, animations, or any flow where Chrome being occluded would matter — when Chrome is in the background, `document.hidden=true` and Chrome throttles `requestAnimationFrame` to ~1 Hz, breaking cinematic camera orbits, CSS animations, video element auto-play, fps counters, etc. Internally: focuses the tab inside Chrome, then EnumWindows-finds the OS window by title containing `(session: <sessionId>)`, restores it from minimized if needed, and runs the foreground-lock bypass (`AttachThreadInput` + `keybd_event` phantom-key trick). Args: `sessionId`. Returns: `{sessionId, hwnd, title, raised, error?}`. ```bash # Typical recording prep adom-desktop browser_open_window '{"sessionId":"demo","profile":"demo","url":"…"}' adom-desktop browser_raise_os_window '{"sessionId":"demo"}' # ← page now actually visible adom-desktop desktop_record_start '{"reason":"…"}' ``` - `browser_lower_os_window` -- **OS-level minimize.** Sends the Chrome window for this session to the back / minimized so the user gets their desktop back after Claude finishes a long pup-driven flow. Args: `sessionId`. Returns: `{sessionId, hwnd, lowered, error?}`. ### Screen Recording > **CRITICAL — pick the right verb.** Claude frequently mistakes "record a pup" / "record this Chrome window" / "record the page" for desktop recording. They are NOT the same. Picking wrong gives a black/wrong-content `.webm` because the pup window may not be in the OS foreground when the desktop recorder snaps frames. #### Decision tree (read this BEFORE you reach for any record command) | User said... | Use | |---|---| | "record a pup" / "record this pup window" / "record the tab" / "record the page" / "record this Chrome window I just opened" | **`browser_record_start`** — tab-scoped via CDP, captures the pup tab regardless of foreground state, ~50 fps actual at 30 fps target, single `.webm` | | "record my screen" / "record what I'm doing" (and Hydrogen is available, which is most of the time) | **Hydrogen's built-in:** `recording start --share screen` — same `getDisplayMedia` pipeline but already wired into Hydrogen with native UI. Prefer this for casual "record my screen" asks. | | "record everything on screen including KiCad / Fusion / OS dialogs / multiple apps switching" | **`desktop_record_start`** with `confirmDesktopNotTabRecording: true` (required arg — see below) | | "record a pup tab AND give me a parallel desktop video at the same time" (e.g. ralph-loop screenshot tests with simultaneous narration) | **Both at once:** `browser_record_start` on the tab AND `desktop_record_start` on the desktop. Adom-desktop's desktop recorder exists for this case — Hydrogen's recorder can't run while it's busy doing tab capture, so adom-desktop fills that gap. | #### Why this matters - **Wrong verb → wrong content.** desktop_record_* captures whatever is on the user's monitor at frame time. If the pup window is occluded behind your IDE / Slack / their email client, the resulting clip shows YOUR app, not the pup tab. Even with `browser_raise_os_window` first, the user's natural alt-tab activity covers the pup window within seconds. - **Hydrogen already has desktop recording.** Most "record my screen" asks should just be Hydrogen's recorder. Adom-desktop's `desktop_record_*` is mostly redundant — it shines exactly when Hydrogen is busy capturing a tab and you need a parallel desktop angle. #### The guardrail `desktop_record_start` REQUIRES `confirmDesktopNotTabRecording: true`. Without it, the call returns `errorCode: desktop_record_needs_confirmation` with a long error string explaining the alternatives (browser_record_start, Hydrogen, or pass the confirm). This is intentional friction — re-read the error and confirm you really want full-desktop, not pup-tab. #### Output summary | Mode | Commands | HUD on screen | Concurrent? | Output | |---|---|---|---|---| | **Tab** (one pup tab's viewport) | `browser_record_*` | NO (the user already sees the tab) | YES (one per tab) | Single finished `.webm` (real-time playback, ~50fps actual) | | **Desktop** (full screen) | `desktop_record_*` | YES — always-on-top control panel with reason + manual stop | No (one at a time) | Single finished `.webm` | #### Desktop recording (HUD) The recorder is a **visible Chrome window** in the bottom-right corner showing recording status, the stated reason, and a manual Stop button. The HUD persists across multiple short clips (a "session"); only `desktop_recorder_close` (or 5 min idle) dismisses it. **Two REQUIRED args on `desktop_record_start`:** - `reason` — string, why you're recording (renders in HUD title + sidecar `.json`) - `confirmDesktopNotTabRecording: true` — boolean acknowledgement that you understand this captures the WHOLE desktop, NOT a pup tab. The bridge rejects calls without this with `errorCode: desktop_record_needs_confirmation` and a hint pointing at `browser_record_start` (for pup tab) or Hydrogen (for casual screen recording). ```bash # Start a desktop clip (HUD opens with reason; recording begins). # Note: confirmDesktopNotTabRecording: true is REQUIRED — without it the # bridge rejects with errorCode: desktop_record_needs_confirmation. adom-desktop desktop_record_start '{ "reason":"Parallel desktop angle alongside Hydrogen ralph-loop tab capture", "confirmDesktopNotTabRecording": true, "monitor":"primary", "fps":30, "audio":false }' # → { "ok":true, "recordingId":"rec-1", "filePath":"…\\rec-desktop-….webm", … } # … drive KiCad/Fusion/etc … # Stop this clip — HUD stays open, ready for next clip adom-desktop desktop_record_stop '{"recordingId":"rec-1"}' # → { "ok":true, "filePath":"…\\rec-desktop-….webm", "sizeKB":4231, "durationMs":18432 } # (Optional) Record more clips in the same session — reason persists, but # confirmDesktopNotTabRecording must be passed every time. adom-desktop desktop_record_start '{ "reason":"Parallel desktop angle alongside Hydrogen ralph-loop tab capture", "confirmDesktopNotTabRecording": true }' # … etc … # When done: close the HUD adom-desktop desktop_recorder_close # → { "ok":true, "sessionSummary":{"clipCount":3, "totalSizeKB":12390, "reason":"…", "clips":[…]} } # Pull each WebM to Docker (already a finished video — no muxing needed) adom-desktop pull_file '{"filePaths":["C:\\…\\rec-desktop-….webm"], "saveTo":"/tmp"}' ffprobe -v error -show_entries stream=codec_name,duration -of default=nw=1 /tmp/rec-desktop-….webm # → codec_name=vp9, duration=18.432 ``` Other commands: - `desktop_recorder_open '{"reason":"…"}'` — open the HUD without starting a clip (so the user knows ahead of time you're about to record) - `desktop_record_status` — `{hudOpen, reason, currentRecording, clipsThisSession}` - `desktop_record_list` — completed `.webm` files with sidecar metadata (incl. reason) - `desktop_list_monitors` — primary monitor info (v1.1: full multi-monitor enumeration) The user can click Stop in the HUD at any time — that converges through the same code path as your `desktop_record_stop` call. Don't fight it; if the user stops manually, treat the clip as complete and react accordingly. #### Tab recording (real-time video, no HUD, concurrent) **For smooth video — narrated demos, motion-heavy scenes, animations, anything the user will watch end-to-end — use `browser_record_start/stop` NOT a `browser_screenshot` loop.** The screenshot loop maxes out around 3 fps and reads as choppy. **Mechanism:** CDP `Page.startScreencast` (Chrome's native compositor-driven JPEG-frame stream) with `Page.screencastFrameAck` per-frame backpressure. Each frame is written to disk with a wall-clock timestamp, then ffmpeg muxes into a single VP9 `.webm` using the concat demuxer with **per-frame durations** so playback matches real wall time. Tab-scoped at the protocol level (the CDP session is bound to a specific Page target — no cross-tab/cross-Chrome leakage). **FPS expectations.** Chrome's compositor pushes a frame each time the page paints. `everyNthFrame` is computed from the requested fps (e.g. 30 fps → every 2nd paint frame). On rAF-animated pages with the tab foregrounded, expect actual fps to **meet or exceed the target** — measured 49.7 fps actual at 30 fps target on a 2.5K viewport. On occluded windows Chrome paint-throttles, so always raise the OS window first. **Why not MediaRecorder + getDisplayMedia?** Tested — Chrome 146 crashes when `getDisplayMedia({preferCurrentTab:true})` is called via Puppeteer (known upstream bug, [puppeteer #13478](https://github.com/puppeteer/puppeteer/issues/13478)). `Page.startScreencast` is Chrome's other native video pipeline; it's stable and produces equivalent output (compositor frames as JPEGs). Output is a single finished **`.webm`** file — no tar, no concat staging exposed to callers, no Docker-side mux. Just `pull_file`. **Raise the OS window first.** Chrome paint-throttles occluded windows, so calling `browser_record_start` on a backgrounded tab produces stutters. Always: ```bash adom-desktop browser_raise_os_window '{"sessionId":"align"}' # surface pup tab to OS foreground adom-desktop browser_record_start '{"sessionId":"align","fps":30}' ``` **Anti-throttle launch flags (already baked into pup).** Pup spawns Chrome with four flags that suppress most of Chrome's background throttling so recordings don't collapse to ~1 fps when the user alt-tabs to another app: - `--disable-renderer-backgrounding` - `--disable-backgrounding-occluded-windows` - `--disable-background-timer-throttling` - `--disable-features=CalculateNativeWinOcclusion` Verified: with all four flags, `browser_record_start` on a window completely covered by Notepad still produces **29.65 fps actual at 30 fps target**. > **Caveat: pup sessions launched in a bridge version older than v1.3.32 don't have these flags.** Chrome launch flags apply only at process spawn — they cannot be added retroactively. If `fpsActual` reports ≤1 even with `browser_raise_os_window` first, the session's Chrome was launched with an old flag set. Fix: close + reopen the session (`browser_close '{"sessionId":"<id>"}'` then `browser_open_window`) so it spawns a fresh Chrome with the current flags. The `kicadVersionUsed`-style proof: there's no per-session "flags" reporter yet, so the reliable test is the recording fps itself. Multiple recordings on different tabs run in parallel — each tab has its own capture loop. ```bash # Record three tabs in parallel for TAB in tab-1 tab-2 tab-3; do adom-desktop browser_record_start "{\"sessionId\":\"align\",\"tabId\":\"$TAB\",\"fps\":30}" done # … drive each tab through its scenario … # Stop and pull each (already a finished webm — no muxing step on Docker) for REC_ID in rec-tab-1 rec-tab-2 rec-tab-3; do STOP=$(adom-desktop browser_record_stop "{\"sessionId\":\"align\",\"recordingId\":\"$REC_ID\"}") WEBM=$(printf '%s' "$STOP" | jq -r '.output | fromjson | .filePath') adom-desktop pull_file "{\"filePaths\":[\"$WEBM\"],\"saveTo\":\"/tmp\"}" done ``` Args: `sessionId` (required), `tabId` (optional — defaults to active tab), `fps` (default 30; note the actual ceiling is ~10-15 fps), `quality` (default 85, 1-100 JPEG quality), `maxDurationMs` (default 600000). Returns from `browser_record_stop`: `{recordingId, sessionId, tabId, filePath, sizeKB, durationMs, frameCount, fpsTarget, fpsActual, stopReason}`. Other commands: - `browser_record_status '{"sessionId":"align"}'` — list active tab recordings with live `frameCount`, `fpsTarget`, `fpsActual`, `durationMs`, `sizeBytesApprox` - `browser_record_list` — completed `.webm` files on disk with sidecar metadata - Tab close auto-stops any recording for that tab (mux fires, file finalizes, you don't have to call `browser_record_stop` first) **Notes & gotchas:** - The desktop HUD itself is captured by the desktop recording (it's a visible window). Same as Hydrogen — accepted cost for the visible-status UX. The window is small + bottom-right. - ffmpeg is **required** on Windows for tab recording (the mux step). Install via `winget install Gyan.FFmpeg`. Bridge logs the detection result at startup. - `maxDurationMs` defaults to 600000 (10 min) for safety; raise it for longer recordings. - On bridge shutdown (graceful), active recordings are drained — the mux runs and you get a valid `.webm`. - The mux file (`<rec>.webm`) is produced by ffmpeg and has a valid `format=duration`. The frames staging dir (`<rec>.frames/`) is deleted after a successful mux. ### Filesystem primitives — `desktop_list_files` / `desktop_watch_files` / `desktop_pull_glob` (v1.5.0+) **These are the canonical "wait for a download" primitives.** They replace the `shell_execute` + PowerShell-poll-Downloads pattern that earlier callers (chip-fetcher, similar tools) had to use. No shell, no user approval prompt, no parsing of `dir` output. They run pure Rust on the desktop side, so they're fast, predictable, and never trip over PowerShell quoting. The pattern they replace looks like this: ```powershell # OLD — DON'T do this for new code: $before = (Get-Date).ToFileTime() # … click download … while (-not (Get-ChildItem ~/Downloads -Filter ul_*.zip | Where { $_.LastWriteTime.ToFileTime() -gt $before })) { Start-Sleep -Seconds 1 } $file = Get-ChildItem ~/Downloads -Filter ul_*.zip | Sort LastWriteTime -Desc | Select -First 1 ``` becomes one call: ```bash # NEW — single primitive, no shell: adom-desktop desktop_watch_files '{ "path": "%USERPROFILE%\\Downloads", "glob": "ul_*.zip", "timeoutMs": 60000 }' # → {ok:true, file:{path,name,size,mtime}, elapsedMs} on success # → {ok:false, error:"timeout", elapsedMs, _hint, ...} on timeout ``` #### `desktop_list_files` — one-shot directory listing Lists files in a directory (non-recursive) matching a shell-glob, optionally filtered to those modified after a given timestamp. - Args: `path` (Windows abs path; `~` / `%USERPROFILE%` / `%APPDATA%` / `%LOCALAPPDATA%` / `%TEMP%` are expanded), `glob` (default `*`; shell-style with `*` and `?`; case-insensitive on Windows), `modifiedSince` (optional unix-seconds OR ISO-8601 string). - Returns: `{ok, path, glob, files: [{path, name, size, mtime}, ...] sorted newest-first, count}`. - Does NOT recurse. Lists one directory only. #### `desktop_watch_files` — block until a match arrives Polls the directory every `pollMs` (default 1000) until at least one file matches the glob with `mtime > since`, or `timeoutMs` (default 60000, max 600000 = 10 min) elapses. - Args: same as `desktop_list_files` plus `since` (defaults to **now** — without an explicit value, only NEW files report), `timeoutMs`, `pollMs`. - Returns on match: `{ok:true, file:{path,name,size,mtime}, elapsedMs}`. - Returns on timeout: `{ok:false, error:"timeout", elapsedMs, _hint, path, glob}`. - The `since`-defaults-to-now behavior matches the canonical "click then watch" flow. If you might miss the file by starting the watch slightly late (e.g. fast small downloads), record `$(date +%s)` BEFORE the click and pass it as `since`. #### `desktop_pull_glob` — one-shot orchestrator (recommended for download flows) Composes `desktop_list_files` (or `desktop_watch_files` if `wait:true`) with the existing streaming `pull_file` mechanism. The whole "wait for a download then pull it" flow in one call: ```bash BEFORE=$(date +%s) # … trigger the download click via browser_input_dispatch … adom-desktop desktop_pull_glob "{ \"path\": \"%USERPROFILE%\\\\Downloads\", \"glob\": \"ul_*.zip\", \"since\": $BEFORE, \"wait\": true, \"timeoutMs\": 60000, \"saveTo\": \"/tmp/cse-out\" }" # → {ok:true, files:[{name,path,size,sha256,chunks}, ...], errors:[], matchedCount} ``` - When `wait:true`: blocks via `desktop_watch_files` until at least one match appears, then re-lists the directory and pulls **everything matching** (so a `.crdownload` + final `.zip` written in the same poll tick both come along). - When `wait:false` (default): just lists what's there now and pulls those. - Pulls use the existing `pull_file` streaming pipeline — sha256-verified, ~1MB chunks, resumes the same way `pull_file` does. #### Glob semantics - `*` matches any run of non-separator chars (within a single filename — these primitives don't recurse). - `?` matches a single non-separator char. - Case-insensitive on Windows (matches the filesystem); case-sensitive on macOS/Linux. - Examples: `ul_*.zip`, `LIB_*.zip`, `*.pdf`, `datasheet_*.pdf`, `*.step`, `*.STEP` (same on Windows). #### Path expansion - `~` and `~/foo` → home dir (Linux/macOS convention; works on Windows too). - `%USERPROFILE%` → home dir (Windows; also works cross-platform). - `%APPDATA%` → Windows roaming app data (`dirs::config_dir()` on others). - `%LOCALAPPDATA%` → Windows local app data (`dirs::data_local_dir()` on others). - `%TEMP%` → Windows TEMP env var; `/tmp` on Linux/macOS. - Expansion is case-insensitive: `%userprofile%` works the same as `%USERPROFILE%`. - Forward slashes (`/`) and back slashes (`\`) both work in the path string. #### Time format for `since` / `modifiedSince` - Number: unix seconds (e.g. `1746483600`). Floats accepted. - String of digits: same (e.g. `"1746483600"`). - ISO-8601 / RFC-3339 string: e.g. `"2026-05-04T22:30:00Z"` or with offset `"2026-05-04T15:30:00-07:00"`. - Files with `mtime <= since` are filtered out (strict `>`). ### Hydrogen Desktop (`hd_*`) — proxy + build/lifecycle for the sibling Tauri app When the user has **Hydrogen Desktop** running (a sibling Tauri v2 app at `C:\Github\hydrogen-desktop`), built-in `hd_*` verbs reach into it. NOT a separate bridge process — adom-desktop proxies HTTP calls directly to HD's control API. **Port discovery (v1.8.72+):** HD's control API lives on **port `47084`** (default; the actual port is read from `%APPDATA%\hydrogen-desktop\ports.json` key `control` on every `hd_api` call, fallback 47084). Pre-v1.8.72 the verbs were hardcoded to HD's legacy `:9001` webview-eval port, which HD doesn't bind anymore — that's why `hd_status` reporting "HD not running" while HD's window is visible is the upgrade signal. #### The canonical generic verb — `hd_api` (v1.8.72+) ```bash adom-desktop hd_api '{"method":"GET","path":"/health"}' adom-desktop hd_api '{"method":"POST","path":"/setup/run-step","body":{"id":"welcome"}}' adom-desktop hd_api '{"method":"GET","path":"/wsl/status"}' ``` Generic HTTP pass-through to any HD endpoint. Raw JSON in `body` — no curl-through-the-relay quote mangling. Returns `{success, status, body, port, method, path}` mirroring HD's response. If HD isn't running, returns `{ok:false, error:"HD not reachable on :47084", _hint:"..."}` instead of timing out. **Public denylist** — these paths are refused with `{ok:false, error:"not_allowed_in_public", denylist_match:"..."}` on public AD builds (use the internal `hd-dev` cargo feature to bypass): ``` /wsl/unregister, /setup/virgin-reset, /container/destroy, /container/reset, /workspace/reset, /workspace/destroy ``` #### Inspect + drive a running HD (v1.8.15+, post-v1.8.72 rework) | Verb | What | Status | |---|---|---| | `hd_status` | Composes AD-side facts (running, repo SHA) + HD's `/workspace/health` + `/buildinfo` relayed | ✓ shipped | | `hd_api` | Generic HTTP pass-through (above) | ✓ v1.8.72+ | | `hd_log '{"tail":30}'` | Tail `%APPDATA%\hydrogen-desktop\hydrogen-desktop.log` (reads disk — works EVEN when HD is down) | ✓ shipped | | `hd_open_url '{"url":"...","browser":"chrome","profileDir":"Default"}'` | POST `/open-in-profile` — open URL in a specific browser profile | ✓ shipped | | `hd_browser_profiles` | GET `/browser-profiles` — enumerate browsers + profiles | ✓ shipped | | `hd_screenshot` / `hd_shot` | Find HD window, capture lossless PNG via PrintWindow; `hd_shot` adds region-scoped variant | ✓ shipped | | `hd_eval '{"js":"..."}'` | POST `/eval` — run JS in HD's main webview | ⚠ **Moved to `hd-dev` in v1.8.72.** Public returns `verb_moved_to_hd_dev`. Use `hd_api '{"method":"POST","path":"/eval","body":{"js":"..."}}'` | | `hd_iframe_eval '{"js":"...","contextIndex":0}'` | POST `/iframe-eval` — JS in code-server iframe via CDP | ⚠ Moved to `hd-dev`. Use `hd_api` equivalent | | `hd_reload_vscode` | POST `/reload-vscode` | ⚠ Moved to `hd-dev`. Use `hd_api` equivalent | | `hd_container_exec '{"command":"..."}'` | POST `/container-exec` | ⚠ Moved to `hd-dev`. Use `hd_api` equivalent | | `hd_launch` / `hd_stop` / `hd_restart` | HD process lifecycle | ✓ shipped | | `hd_ship` | Atomic build+relaunch+verify-SHA (see below) | ✓ shipped | | `hd_build*` (v1.8.16+) | Build pipeline orchestration | ⚠ Moved to `hd-dev` in v1.8.72 (HD developers only) | #### Atomic ship — `hd_ship` (v1.8.49+) — START HERE for post-push rebuilds **One verb, one call, one structured pass/fail.** Replaces the 4-step chain `hd_stop` → `hd_build_rust` → poll `hd_build_status` → `hd_launch` → `curl :47084/buildinfo`. Eliminates the "is the new code actually running?" guesswork after every push. Sequence inside one call: 1. taskkill HD + poll until zero instances (frees the exe lock) 2. `git fetch && git reset --hard origin/main && rev-parse HEAD` — asserts `HEAD == expectSha` (fails fast if GitHub hasn't replicated the push yet) 3. if `clean:true`, `cargo clean -p hydrogen-desktop` (defeats stale-relink) 4. `cargo build` and parse stderr for `Compiling hydrogen-desktop` to know whether anything actually compiled 5. launch + poll `http://127.0.0.1:47084/buildinfo` until it responds + assert `build_sha == expectSha` ```bash # End-to-end after a push: git push && SHA=$(git rev-parse HEAD) && \ adom-desktop hd_ship "{\"expectSha\":\"$SHA\"}" ``` Args: - `expectSha` — **REQUIRED** string. Full or prefix SHA of the commit you want running. Bidirectional-prefix match (`abc123` matches the full SHA starting with `abc123` and vice versa). - `clean` — optional bool, default `false`. `true` forces `cargo clean -p hydrogen-desktop` before build. Use when a prior `hd_ship` succeeded but `runningSha != expectSha` (the stale-relink trap — cargo said `Finished` but reused the cached binary). Returns on success: `{ok:true, builtSha (== expectSha), runningSha (== expectSha), compiled:bool, elapsedSecs, _hint}`. `compiled:false` on success means cargo found nothing to recompile (cached relink or HEAD didn't actually change — usually fine but surface to caller in case the push was a no-op). Returns on failure: `{ok:false, failedAt:'args'|'kill'|'git'|'clean'|'build'|'launch'|'verify', error, builtSha (set after git step), runningSha (set if launch worked but verify saw the wrong SHA), compiled (set after build step), elapsedSecs, _hint}`. The partials let you reason about WHERE it broke: | `failedAt` | What it means | |---|---| | `args` | `expectSha` missing / malformed | | `kill` | Couldn't taskkill HD (rare — usually permissions) | | `git` | `git fetch / reset` failed, OR `HEAD != expectSha` after reset (push hasn't replicated to origin/main yet — retry in a few seconds) | | `clean` | `cargo clean` failed (rare) | | `build` | `cargo build` failed — read `_hint` for next step (usually `hd_build_log` to see the compile error) | | `launch` | Cargo finished but `hydrogen-desktop.exe` wouldn't start — `binary_missing` or process spawn error | | `verify` | HD launched but `/buildinfo` either never responded or reported a different SHA than `expectSha` (the stale-relink trap — retry with `clean:true`) | Long-running: cold cargo build can take 2-3 min. CLI WS timeout is 12 min — call as a single sync invocation and wait. Don't poll inside `hd_ship`; it does its own internal polling. #### Lower-level build + lifecycle suite (v1.8.16+, enriched in v1.8.52+) Use these directly only when you need finer control than `hd_ship` provides (e.g. frontend-only build, watching log stream live, manual launch sequencing). For the common "I pushed code, get it running" path use `hd_ship`. The relay's `shell_execute` has a 30s timeout; HD's `pnpm build` + `cargo build` exceed that, so the build is split into async-spawn verbs + sync poll/tail verbs: ```bash # Frontend-only rebuild, watch progress, relaunch (no hd_ship since hd_ship is Rust-build-only) adom-desktop hd_stop adom-desktop hd_build_frontend '{"show":false}' # returns {ok, pid, logPath} immediately OFFSET=0 while true; do RESP=$(adom-desktop hd_build_tail "{\"offset\":$OFFSET}") echo "$RESP" | jq -r '.lines[]' DONE=$(echo "$RESP" | jq -r '.done') OFFSET=$(echo "$RESP" | jq -r '.newOffset') [[ "$DONE" == "true" ]] && break sleep 3 done adom-desktop hd_build_status # confirm succeeded:true && relinkOnly:false adom-desktop hd_launch # start the new debug binary ``` ##### `hd_build` / `hd_build_frontend` / `hd_build_rust` — async-spawn build verbs | Verb | What | |---|---| | `hd_build '{"show":false}'` | Full async build: `git pull` + `pnpm build` + `cargo build`. Returns instantly with `{pid, logPath, flavor:'full'}`. Final log line is `BUILD_OK` or `BUILD_FAILED: <step>`. | | `hd_build_frontend` | Just `pnpm build` (root). Returns `{pid, logPath, flavor:'frontend'}`. | | `hd_build_rust` | Just `cargo build` (src-tauri/). **v1.8.52+:** dispatch response also carries `builtSha` + `exeMtimeBefore` + `exeSizeBefore` (captured at spawn time into `%TEMP%\hd-build-meta.json`) so the subsequent `hd_build_status` / `hd_launch` don't need a separate `git rev-parse` + `stat` call. Returns `{ok, pid, logPath, flavor:'rust', show, lingerSecs, builtSha, exeMtimeBefore, exeSizeBefore, _hint}`. | ##### `hd_build_status` — sync state probe (v1.8.52+ enriched) Returns: ``` { ok, building, succeeded, failed, lastLines:[string], # last ~40 lines of stderr logPath, pid, pidAlive, failureReason, # 'compile_error' | 'exe_locked' | 'orphaned_no_sentinel' compiledCrates:[string], # names parsed from 'Compiling X v...' lines relinkOnly:bool, # true iff succeeded && compiledCrates is empty exeChanged:bool, # mtime OR size after differs from before exeMtimeBefore:int, exeMtimeAfter:int, # unix seconds exeSizeBefore:int, exeSizeAfter:int, # bytes exeLocked:bool, # true when build failed because hydrogen-desktop.exe was held holderPids:[int], # PIDs holding the exe — taskkill these builtSha:string|null, # SHA hd_build_rust/hd_build was dispatched at elapsedSecs:int, flavor:string, # 'rust' | 'frontend' | 'full' _hint } ``` **`relinkOnly:true` on success is the silent-fail trap to look for** — your push didn't actually rebuild because cargo found nothing to recompile. Usually means your source changes weren't pulled, or you touched a file the workspace doesn't track. Use `hd_ship` with `clean:true` to force a fresh build. **`exeLocked:true` on failure** + `holderPids` tells you exactly which PIDs to taskkill before retry; or just use `hd_ship` which kills HD before building. ##### `hd_launch` — start the debug binary (v1.8.52+ enriched) Args: `killExisting` — optional bool, default `false`. When `false` and HD is already running, refuses with `reason:'already_running'` so you don't accidentally attach to a stale process. When `true`, taskkill the existing instance with a 600 ms grace period before launching fresh. Returns on launch: `{ok:true, launched:true, wasAlreadyRunning:bool, killedFirst:bool, pid:int (HD's actual PID, not the cmd-wrapper), exePath, builtSha:string|null, _hint}`. Returns on refusal: `{ok:false, launched:false, wasAlreadyRunning:bool, pid:int|null, killedFirst:false, reason, _hint}`. `reason` ∈: - `build_in_progress` — wait for `hd_build_status.building:false` - `build_failed` — fix and retry `hd_build` (or use `hd_ship`) - `already_running` — pass `killExisting:true` to force-kill+relaunch, or use `hd_restart` - `binary_missing` — `target/debug/hydrogen-desktop.exe` not on disk; run `hd_build_rust` (or `hd_ship`) first **`launched:true, wasAlreadyRunning:false, killedFirst:false` is the proof you need that truly fresh code is running.** Any other combo means you should compare `builtSha` to HD's actual `/buildinfo.build_sha`. ##### `hd_stop` / `hd_restart` / `hd_build_log` / `hd_build_tail` | Verb | What | |---|---| | `hd_stop` | `taskkill /F /IM hydrogen-desktop.exe`. `{wasRunning}` distinguishes killed vs no-op. | | `hd_restart` | Stop + launch in one call. Same guards as `hd_launch` (skip `already_running`). | | `hd_build_log` | Full log dump from `%TEMP%\hd-build.log`. | | `hd_build_tail '{"offset":N}'` | Incremental stream — `{lines, newOffset, totalBytes, done, succeeded, _hint}`. Pass `newOffset` back for the next chunk. | The `show: true` arg on any build verb opens a visible PowerShell console window so the user can watch the build scroll. Default `show: false` runs hidden. **Auto-close (v1.8.23+).** Visible build windows auto-close 30 seconds after the final `BUILD_OK` / `BUILD_FAILED` line — long enough for the user to read the last error, short enough that build-after-build sessions don't accumulate orphan windows. The window prints a clear green/red "auto-closes in 30 seconds" banner before sleeping. Override with `lingerSecs`: - `'{"show":true, "lingerSecs":60}'` — give yourself 60s instead of 30 - `'{"show":true, "lingerSecs":0}'` — close immediately, no grace period - `'{"show":true, "lingerSecs":3600}'` — keep open for an hour (cap) The response includes the applied `lingerSecs` so you can confirm what landed. Hidden builds (`show:false`) ignore `lingerSecs` entirely — no window to linger. **Don't call `hd_launch` until `hd_build_status` shows `succeeded: true`** — the guard refuses with `reason: "build_failed"` if you do, and the `_hint` tells the AI exactly what to do (`hd_build_log` to see errors, fix, rebuild). Same for `hd_restart`. #### Named-region screenshots — `hd_shot` (v1.8.54+) **One verb replaces the `desktop_list_windows | grep 'Hydrogen' → desktop_screenshot_window <hwnd>` chain** (3 calls + fragile HWND parsing). Captures a named region of HD, produces BOTH a full-res PNG and a Claude-safe-resized copy, and returns rich metadata so the AI never has to chain screenshot → pull → convert → crop → resize → Read manually. Each shot is **self-identifying**: it includes the running build's SHA + local build time read from HD's `/buildinfo` so you can confirm you're looking at the build you expect before trusting what you see (no more "this screenshot was from the previous binary, not my fix" confusion). Regions are deterministic — they resolve via HD window bounds + CDP element rects queried through HD's `/eval` endpoint. No AI-supplied pixel coords. Args: - `region` — **REQUIRED**. One of: - `full` — whole HD window - `titlebar` — top ~40 px strip (HD's build SHA / timestamp readout lives here) - `vscode` — VS Code iframe pane via CDP element rect - `setup-panel` — Setup Steps panel (`.setup-panel` / `.setup-panel-output`) - `wiki` — wiki / browser tab pane - `claude` — Claude Code panel - `resizeMax` — optional int, default `1400`. Longest side of the safe-resized copy. Use ≈800 to fit more shots in one Claude turn; 1568 for max fidelity. Returns: ``` { ok, region, fullPath, # lossless raw PNG — for shotlog/archival, NOT Read safePath, # ≤resizeMax PNG — pull_file this, Read this fullSize:[w,h], safeSize:[w,h], clipRect:[x,y,w,h], # captured-image pixels (NOT screen pixels) capturedAtLocal, runningBuildSha, # from HD's /buildinfo — null if HD doesn't expose it yet runningBuildLocal, hdWindowFound:bool, regionFound:bool, windowOuter:[w,h], capturedSize:[w,h], _hints: { pull, # exact pull_file command for safePath fullForArchive, # note that fullPath is for archive, not Read staleWindow, # set when runningBuildSha looks wrong regionMiss # set when regionFound:false } } ``` Recipe — verify-which-build-is-running + confirm-config-change-took-effect + full dump: ```bash # After hd_ship, confirm the titlebar shows the SHA you expect: adom-desktop hd_shot '{"region":"titlebar"}' # → runningBuildSha lets you assert this before trusting any other shot # After flipping a Setup Steps toggle, screenshot the panel to confirm: adom-desktop hd_shot '{"region":"setup-panel","resizeMax":1568}' # Full-window dump for general debugging: adom-desktop hd_shot '{"region":"full"}' ``` Edge cases: - **Unknown region** — refusal: `{ok:false, error, knownRegions:[...]}`. No fallback — fix the call. - **`regionFound:false`** — the element rect wasn't in HD's DOM. The panel/iframe may not be mounted yet, OR HD's markup changed and `region_selectors()` in `hd_bridge.rs` is out of date. The shot is **still produced** (falls back to the full window) so you have something to look at; check `_hints.regionMiss` for the reason. - **`runningBuildSha:null`** — HD's `/buildinfo` isn't exposed yet on the running version. The shot is still produced; you just can't self-verify which build it represents. **Always pull_file the `safePath` and Read THAT** — it's already sized for Claude's image analysis. The `fullPath` is for shotlog / disk archival, NOT direct Read (may exceed Claude's per-image budget). The `_hints.pull` field gives you the exact `pull_file` command to copy-paste. **Old workaround to drop:** Do NOT use `desktop_list_windows | filter 'Hydrogen' | desktop_screenshot_window <hwnd>` anymore for HD — `hd_shot` replaces that entire pattern. Generic `desktop_screenshot_window` still applies to non-HD windows (other apps, dialogs, etc.). ### TCP Port-forward Tunnels — `tunnel open|close|list|check|rdp` + `display` (v1.8.85+, reworked v1.8.147) Reach a TCP service addressable only from the **cloud side** through the **laptop side**: AD binds a `127.0.0.1:<port>` listener on the laptop and pumps bytes over the existing AD↔relay WebSocket; the relay dials the upstream `to`. First use case: RDP into a Windows VM behind an NSG only the cloud container reaches — mstsc on the laptop connects to a local port and is talking through the cloud. Same path works for SSH, VNC, HTTP, postgres — any TCP. **`tunnel` is a grouped verb** (`rdp` / `tunnel_close` / `tunnel_list` remain back-compat aliases): ```bash # Open a VERIFIED tunnel (both legs probed before ok). Auto-picks a free local port. adom-desktop --target <name> tunnel open '{"to":"127.0.0.1:13389"}' # RDP: open + verify + launch mstsc, sized to the laptop's real screen. adom-desktop --target <name> tunnel rdp '{"to":"127.0.0.1:13389","size":"90%"}' # size: "90%" (default) | "100%" | "fullscreen" | "maximized" # OR monitor:<index> (0=primary) OR explicit width+height adom-desktop --target <name> tunnel list # live tunnels + leak detection adom-desktop --target <name> tunnel check '{"tunnelId":<id>}' # per-leg health adom-desktop --target <name> tunnel close '{"tunnelId":<id>}' # or {"all":true} adom-desktop --target <name> display # monitor geometry (for sizing) ``` **`--target` is mandatory when >1 AD is connected.** v1.8.147 fixed a false-success bug: the bind, the upstream probe, and the mstsc launch now ALL resolve to the SAME machine. (Before, the bind could land on one AD and mstsc launch on another → "port bound" on the VM, "nothing listening" on the laptop, a bogus `ok:true`.) No target + multiple ADs → `ambiguous_target` listing the names; `--target all` is rejected (a tunnel binds one machine). `adom-desktop targets` lists connected ADs. **Active health verification — every `open`/`rdp`/`check` PROVES the path before reporting `ok`:** - **local-bind** — AD self-probes `127.0.0.1:<boundPort>` right after binding. If it can't connect, AD tears the tunnel down and returns `ok:false` (this alone kills the false-success class). - **upstream-refused** — the relay dials `to` (retry/backoff ~4×/2.5s). If unreachable, the tunnel is torn down (no leaked row) and `ok:false`. - A failure names the leg: `failedLeg: "relay-to-ad" | "local-bind" | "upstream-refused"`. Health fields surfaced: `listening`, `relayConnected`, `upstreamReachable`, `connCount`, `mstscAlive`. **Progressive hints.** Every response carries `_hint` (what's true now) + `_next` (the exact next verb with real values, copy-pasteable). e.g. a successful open's `_next` hands you the `tunnel rdp` / `tunnel check` / `tunnel close` commands with the real `to`/`target`/`tunnelId` filled in. A missing `to` returns a structured error with an example. Walk the workflow from the responses alone. **Display-aware RDP sizing.** `tunnel rdp` resolves `size`/`monitor`/`width`/`height` against the laptop's real screen (queried via `display` — `EnumDisplayMonitors` + per-monitor DPI), writes an `.rdp` file with the geometry, and launches mstsc. e.g. `size:"90%"` on a 2560×1600 primary → a 2304×1440 window. Default 90% of primary. You never hardcode a resolution — call `display` if you want the layout first. **`tunnel list` reconciles + reaps leaks.** It cross-checks the relay registry against every connected AD's own registry and reports `leaks:[...]` for rows only one side knows about; `tunnel close '{"all":true}'` reaps them. The relay stores the actually-bound `localPort` (so `tunnel close '{"local":<port>}'` works). Architectural notes for Claude: - Bytes-only — no payload parsing, so any TCP service works. TLS/cert-pinning still pins to whatever the cloud-side dial resolves (for RDP the upstream cert CN must match what mstsc expects). - Each fresh laptop-side TCP connection gets its own connId multiplexed over one tunnelId; concurrent mstsc reconnects / parallel sessions work without re-opening. Backpressure bounded (256-slot mpsc/conn, 64 KiB buf). - Tunnels don't survive AD/relay restarts. If mstsc disconnects, `tunnel check` (names the dead leg), then re-open. - The classic RDP collision (Windows' own RDP owns `0.0.0.0:3389`) is dodged by `rdp`'s default `local:13389` + `SO_EXCLUSIVEADDRUSE` + ephemeral fallback (`fellBack:true` in the response; read `localPort` for the real port). **Old workarounds to drop**: (1) NSG WAN-IP whitelisting for roaming-laptop RDP — the tunnel rides existing auth, reachable regardless of WAN IP. (2) Trusting a bare `ok:true` from the old `rdp` — it could be a dead port; now `ok` means both legs were probed. (3) Hardcoding mstsc resolution — pass `size`/`monitor`; AD resolves against the real screen. ### Shell — `shell_execute` (escape hatch only) - `shell_execute` -- Run a shell command on the desktop. The CLI handles approval polling internally — it returns the final `{success, output, error, exitCode}` once the user clicks Allow on the desktop dialog. **You do not need to poll, retry, or call `get_deferred_result` yourself.** The CLI emits `HINT:` lines to stderr while waiting (every 15s) so an AI Monitor sees progress. - `shell_kill_all` -- Kill all running shell commands and deny pending approvals. **v1.7.16 fix.** Earlier versions (v1.7.15 specifically) had a regression where the CLI's approval-wait loop re-sent the command every 1s instead of polling for the deferred result; that produced an infinite "Another shell command is already waiting for approval" loop because each retry created a fresh approval the user could never out-click. v1.7.16 fixes it. If you see that error string with a `>= 1.7.16` CLI, the bug is back — file an issue. If you see it with `< 1.7.16`, upgrade the CLI: `adom-wiki asset get apps/adom-desktop docker_binary -o /usr/local/bin/adom-desktop`. **Deprecated for download polling.** As of v1.5.0, use `desktop_watch_files` / `desktop_pull_glob` for waiting on download arrivals — those don't require user approval, don't go through PowerShell quoting, and are O(directory entries) rather than spawning a process every second. `shell_execute` itself stays as an escape hatch for genuinely shell-only operations (multi-step ad-hoc admin tasks, chained pipelines, etc.). Use `where python` / `C:\Python3xx\python.exe` for Python -- avoid `python` which may not be on PATH. #### `run_script` + `wsl_exec` — base64-payload script runners (v1.8.46+) When you want to send a multi-line script to the desktop without the shell-quoting hell that `shell_execute` requires, use these two verbs. The script crosses the wire as base64 so NO escaping survives to be mangled by cmd.exe, PowerShell parser quirks, MSYS path conversion, or wsl.exe's `--` / nested-quote / `$_` rules. The caller never hand-escapes anything. Both verbs require shell auto-approve to be active — same trust level as `shell_execute`. They refuse cleanly with an actionable error if not approved; no per-call modal (these are programmatic verbs). **Reading the result — `status` + `durationMs` (v1.8.162+).** EVERY CLI result now carries a single authoritative **`status`** field so you never have to guess whether an empty `stdout` was a real empty-output success or a timeout: | `status` | meaning | what to do | |---|---|---| | `ok` | completed (stdout may legitimately be empty) | use `exitCode`/`stdout`/`stderr` | | `timeout` | did NOT finish in time (NOT an empty success) | raise `timeoutSeconds`, or use a detached job (below) | | `busy` | desktop couldn't service it (WSL wedged / no client connected) | check `adom-desktop status`; `wsl_recover` for WSL; retry | | `error` | the command/verb failed | read `error` / `stderr` / `_hint` | `durationMs` is the wall-clock of the call. `exitCode`/`stdout`/`stderr` pass through unchanged; on `timeout` the verb's `partialStdout`/`partialStderr` hold what was written before the kill. A `timeout`/`busy` result also carries a recovery `_hint`. **For multi-minute work don't fight the timeout** — use a detached job that returns instantly + polls: `wsl_exec_async` / `wsl_import` / `wsl_export` → poll `wsl_job_status` (or `launch_process` for a general long-running command). This is the cure for the "1.4 GB import EOFs the sync call" problem. **`run_script`** — base64 script + interpreter (cmd / powershell / bash): ```bash SCRIPT_B64=$(python3 -c "import base64; print(base64.b64encode(b''' echo \"multi-line script\" VAR=\"value with spaces and \$dollars\" echo \"VAR=\$VAR\" exit 0 ''').decode())") adom-desktop run_script "{\"interpreter\":\"bash\",\"scriptB64\":\"$SCRIPT_B64\"}" # → {success, ok, stdout, stderr, exitCode} ``` Args: `interpreter` (`cmd` | `powershell` | `bash`), `scriptB64` (required), `timeoutSeconds` (default 60). PowerShell prefers `pwsh` (PS 7+) if on PATH, falls back to `powershell.exe` (5.1). - **`bash` runs via WSL2, NOT Git Bash.** The Windows host's first `bash` on PATH is `wsl.exe -d <default-distro> bash` (verified at `C:\Windows\System32\bash.exe`). The script is piped to bash's stdin (no temp file, no path-form mismatch — no MSYS `/S` → `S:/` corruption, no nested-quote loss). The decoded script bytes are written to bash's stdin and stdin is closed; bash reads + executes without ever opening a file. For explicit distro / user control, use `wsl_exec`. **`wsl_exec`** — base64 script + distro + user, executed via the canonical `wsl -d <distro> -u <user> -- bash -lc <decoded>` argv array: ```bash SCRIPT_B64=$(python3 -c "import base64; print(base64.b64encode(b''' cd /mnt/c/Github/adom-desktop git status --short | head -10 exit 0 ''').decode())") adom-desktop wsl_exec "{\"distro\":\"Ubuntu\",\"user\":\"root\",\"scriptB64\":\"$SCRIPT_B64\"}" # → {success, ok, stdout, stderr, exitCode} ``` Args: `distro` (e.g. `Ubuntu`, `Adom-Workspace`), `user` (e.g. `adom`, `root`), `scriptB64`, `timeoutSeconds` (default 60). Why argv (not shell string): Rust's `std::process::Command` on Windows applies proper `CommandLineToArgvW` quoting per argv element. `wsl.exe` sees `bash -lc <decoded>` as three distinct argv items even when the decoded script contains quotes, `$`, backslashes, newlines, pipes — anything. `bash`'s `-c` then sees the entire script as ONE string with no outer-shell interpretation. Output decoding: `wsl.exe` defaults to UTF-16LE when invoked from the Windows console subsystem. Both verbs auto-detect (BOM check + odd-position-NULL heuristic) and decode to UTF-8. Output is never wrapped in another shell that might re-interpret it. Pitfalls these verbs eliminate: - `cmd.exe`'s `&`, `|`, `^`, `%var%` interpretation - PowerShell's parser quirks around nested quotes + `$` interpolation - Git Bash MSYS converting `/S` → `S:/` etc. - WSL's `--` parser limits + nested-quote loss - UTF-16LE corruption on wsl.exe output #### WSL wedge defense — `wsl_wedged` + `wsl_recover` (v1.8.157+) Every `wsl_exec` / `run_script bash` spawns a `wsl.exe` client. If WSL's service (`LxssManager`) wedges — common after a laptop runs for a day, or under heavy parallel cloud use — those clients block in a wait that `taskkill /F` can't always finish, and retries keep spawning more. Left unchecked they pile up into the hundreds and wedge the whole machine. Three defenses: 1. **Refuse-cap.** When there are already too many `wsl.exe` (`WSL_REFUSE_AT=80`), AD refuses NEW wsl spawns with `errorCode:"wsl_wedged"` instead of adding another doomed client. If you get `wsl_wedged`, don't retry the same command — run `wsl_recover` first. 2. **`wsl_recover`** — reap orphaned/hung `wsl.exe`, **DRY-RUN by default** (nothing is killed unless you pass `confirm:true`): ```bash adom-desktop wsl_recover # dry-run: lists `candidates` (would-reap) + `spared` adom-desktop wsl_recover '{"confirm":true}' # reap the candidates adom-desktop wsl_recover '{"confirm":true,"sparePids":[1234]}' # also protect pid 1234 ``` It classifies each `wsl.exe`: childless + (0-CPU & old | runaway-CPU) → **candidate**; everything else → **spared**. It **spares any `wsl.exe` with child processes** (that's almost certainly the live distro, e.g. the HD `Adom-Workspace`), honors `sparePids`, and never touches `wslservice`/`wslhost`/`vmmem`. Review each candidate's `cmd` in the dry-run to confirm the live workspace isn't among them, then confirm. **Never** `wsl --shutdown` (it kills the live workspace too). Reaping childless orphans often cascades — their now-parentless children exit on their own (observed 114 → 18 from reaping 54). 3. **status wsl block** — `{wslProcessCount, wedged, _hint}` lets you see a pile-up *before* it blocks work. Path differs by transport: **`status.desktop.wsl`** via the CLI/relay (`adom-desktop status` — nested like `desktop.apps`), or top-level **`status.wsl`** on the local direct API (`GET /status`). `wedged:true` at/above `WSL_WARN_AT=50`. #### Long-running in-distro work + distro lifecycle — golden-image bakes (v1.8.161+) `wsl_exec` is **synchronous** — it waits for the command and is bounded by the relay's HTTP timeout (~60-120s), so a multi-minute bake (apt, a big download) EOFs. Backgrounding inside the distro (`nohup … &`) doesn't help either: the `wsl.exe` client returns, WSL sees no active session, shuts the distro down, and kills the bg process (wiping `/tmp`). For anything long, use the **async held-session** verbs — AD holds one `wsl.exe` in the foreground for the command's whole lifetime, so the distro stays up: - **`wsl_exec_async`** → returns a `jobId` + `outputPath` (a live UTF-8 log) immediately; the command runs detached for as long as it takes. - **`wsl_job_status {jobId}`** → `{running, exitCode, elapsedMs, outputTail, outputPath}`. Poll until `running:false`; `pull_file` the `outputPath` for the full log. (`wsl_job_list`, `wsl_job_kill {jobId}` round it out.) ```bash # golden-image bake: import → run (async) → poll → export → unregister, all via AD adom-desktop wsl_import '{"distro":"bake","installDir":"C:/Users/<you>/.adom/wsl/bake","tarball":"C:/.../base.tar"}' # → {jobId}; poll wsl_job_status until done (exitCode 0) B64=$(printf 'set -euo pipefail\napt-get update && apt-get install -y build-essential\n' | base64 -w0) adom-desktop wsl_exec_async "{\"distro\":\"bake\",\"user\":\"root\",\"scriptB64\":\"$B64\"}" adom-desktop wsl_job_status '{"jobId":"wsljob-xxxxxxxx"}' # repeat until running:false; outputTail = progress adom-desktop wsl_export '{"distro":"bake","tarball":"C:/.../golden.tar"}' # poll, then pull_file the tarball adom-desktop wsl_unregister '{"distro":"bake"}' # cleanup (REFUSES Adom-Workspace) adom-desktop wsl_list # {distros:[{name,state,version,default}]} ``` **systemd user session — `viaRoot`.** On a `systemd=true` distro, `-u <user>` can fail "Failed to start the systemd user session for '<user>'". Pass `"viaRoot": true` to `wsl_exec_async` — AD runs `-u root -- runuser -u <user> -- bash -l`, dropping to the user WITHOUT depending on `user@UID.service`. `wsl_import` / `wsl_export` / `wsl_unregister` are async (jobId + poll) too, so big imports/exports never EOF. All are **strictly scoped to the named distro** — there is no global `wsl --shutdown` path (that would kill the live HD workspace), and `wsl_unregister` refuses `Adom-Workspace`. Job ids live in AD's memory only (not persisted across an AD restart; the log file survives). The wedge defense above still applies — an over-cap spawn returns `wsl_wedged`. ## Detecting App Installation After connecting, always run `adom-desktop status` to check what's installed. The `desktop.apps` object tells you exactly what the user has: ```bash adom-desktop status # Look at the desktop.apps field in the response ``` ### Handling "not installed" errors When a command returns `errorCode: "node_not_found"`, `errorCode: "kicad_not_installed"`, or `errorCode: "fusion_not_installed"`, guide the user through installation: **Picking the right native browser + profile (v1.7.1+):** When you need to open a URL in the user's NATIVE browser AND it matters which account is signed in (work Google Workspace vs personal Gmail vs media YouTube channel etc.), `desktop_open_url` alone isn't enough — you need to target a specific profile. The flow: ```bash # 1. Discover what's installed + which profiles are configured. adom-desktop desktop_list_browsers '{}' # → { # "default": "chrome", # "browsers": [ # { "name": "chrome", "displayName": "Google Chrome", "version": "146...", # "exePath": "C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe", # "profiles": [ # { "id": "Default", "name": "Alice (work)", "gaia": "alice@acme.com", "isDefault": true }, # { "id": "Profile 1", "name": "Personal", "gaia": "alice@gmail.com" }, # { "id": "Profile 2", "name": "Adom Media", "gaia": "media@adom.inc" } # ] }, # { "name": "edge", "profiles": [...] }, # { "name": "firefox", "profiles": [{"id":"default-release","name":"default","isDefault":true}] } # ] # } # 2. Match the URL's context to the right profile, then open it there. # E.g. opening a Google Doc shared by your work team → use the work profile: adom-desktop desktop_open_url '{ "url":"https://docs.google.com/document/d/...", "browser":"chrome", "profile":"Default" }' # Or YouTube channel management for the media account: adom-desktop desktop_open_url '{ "url":"https://studio.youtube.com", "browser":"chrome", "profile":"Profile 2" }' ``` Profile-picking heuristics for Docker Claude: - **Workspace / @adom.inc URLs** → match `profile.gaia.endsWith("@adom.inc")` and not the `media@` one → typically `Default`. - **Personal Gmail / Drive / etc.** → match `profile.gaia === "alice@gmail.com"` (or whatever the user's personal address resolves to). - **YouTube Studio / channel-specific work** → match the media account profile. - **Random scratch / experiments** → use the user's Edge profile, where they keep miscellaneous accounts (per the user's stated preference). - **No clear match** → fall back to `browser:"default"` (no profile flag) and let the user pick. `profile` is optional. Omit it to open in whichever profile the browser was last using (legacy v1.6.x behavior). Profile flag is silently ignored when `browser:"default"` (no clean way to inject through the OS URL handler — name the browser explicitly to use it). **Node.js not installed (the puppeteer bridge can't auto-spawn):** v1.7.0+ — any `browser_*` command returning `errorCode:"node_not_found"` means Node isn't on the desktop machine. Trigger the unattended winget install: ```bash adom-desktop desktop_install_node '{}' # → runs `winget install --id OpenJS.NodeJS.LTS --silent ...` (Windows only). # → ~1-2 min; downloads + installs the Node.js LTS line (well-tested with puppeteer). # → returns {ok:true, exitCode:0, _hint:"Retry the original browser_* command"} on success. # → on failure: {ok:false, errorCode:"winget_unavailable" | "winget_install_failed", _hint}. ``` After install, **just retry the original `browser_*` command** — the bridge picks up the new node.exe via the registry App Paths lookup (which winget writes), no Adom Desktop restart needed. v1.7.0+ also pre-fetches Node + Chrome for Testing on first launch after install (background task on AUTOSTART_VERSION 11), so most users will never see `node_not_found` to begin with. If `desktop_install_node` returns `errorCode:"winget_unavailable"` (older Windows or managed corporate device), fall back to manual: > Node.js LTS is required for browser automation. winget isn't available on this machine — download Node.js LTS from https://nodejs.org/ and run the installer. Let me know when it's done and I'll retry. **v1.7.0+ Node detection.** The bridge looks for `node.exe` in: - System PATH (via `where node` on Windows, `which node` elsewhere) - Windows registry App Paths (catches winget, MSIX, custom-dir installs) - Standard install dirs: `C:\Program Files\nodejs\`, `C:\Program Files (x86)\nodejs\`, `%LOCALAPPDATA%\Programs\nodejs\` - Per-user version managers: nvm-windows (`%APPDATA%\nvm\`), volta (`%LOCALAPPDATA%\Volta\bin\`), fnm (`%FNM_DIR%`) - macOS Homebrew (`/opt/homebrew/bin`, `/usr/local/bin`) If detection still misses an install on a user's machine, get the actual `node.exe` path from the user — that's a real bug to file. **KiCad not installed:** **KiCad not installed:** v1.6.0+ — try the unattended winget install FIRST before asking the user to download manually: ```bash adom-desktop desktop_install_kicad '{}' # → runs `winget install --id KiCad.KiCad --silent ...` (Windows only). # → 2-5 minutes; downloads ~700 MB then installs. # → returns {ok:true, exitCode:0, _hint:"Run kicad_list_versions to verify"} on success. # → on failure (no winget / network failure / corporate-managed device): # {ok:false, errorCode:"winget_unavailable" | "winget_install_failed", _hint, ...} ``` If `desktop_install_kicad` fails with `errorCode:"winget_unavailable"`, fall back to manual: > KiCad isn't installed and winget isn't available. Download from https://www.kicad.org/download/ — it's free and open source, install the latest stable (9.x or 10.x). > Let me know when the install is done and I'll verify the connection. After EITHER path, have them restart Adom Desktop OR call `kicad_list_versions` again — the kicad bridge caches detection results, so a fresh scan may be needed to pick up freshly-installed binaries. **v1.6.0+ — improved KiCad detection.** The bridge now scans for KiCad in: - `C:\Program Files\KiCad\<version>\` (the standard location) - `C:\Program Files (x86)\KiCad\<version>\` (32-bit edge cases) - `%LOCALAPPDATA%\Programs\KiCad\<version>\` (winget per-user installs) - `%LOCALAPPDATA%\KiCad\<version>\` (rare manual installs) - Windows registry: `HKLM\SOFTWARE\KiCad\<version>\InstallationPath` and the `WOW6432Node` mirror - `HKCU` mirrors of the above for per-user installs - macOS: `/Applications/KiCad/` and `/Applications/` directly If detection still misses an install on a user's machine, that's a real bug to file — get the actual install path from the user and we'll add it to the scan list. **Fusion 360 not installed:** > Fusion 360 isn't installed on your desktop. Would you like to install it? > Download from: https://www.autodesk.com/products/fusion-360 > It's free for personal/hobby use (requires an Autodesk account). > Let me know when the install is done and I'll verify the connection. After install, have them restart Adom Desktop, then run `adom-desktop status` to verify. ### Handling "not running" errors When `errorCode: "fusion_not_running"`, launch it programmatically with the first-class startup command: ```bash adom-desktop fusion_start # On Windows: discovers exe via %LOCALAPPDATA% env var webdeploy glob. # On Docker: delegates to the bridge on the connected desktop via relay. # ~15-30s typical. Auto-dismisses startup dialogs. # Returns {"addinReady": true, "pickerDismissed": true|false, ...} ``` KiCad doesn't need to be running for most commands (the bridge launches it on demand). ### Handling "add-in not installed" errors When Fusion is running but `addinInstalled: false` or `addinConnected: false`: > The AdomBridge add-in needs to be installed in Fusion 360. I can install it for you — this lets me control Fusion remotely. The add-in auto-installs when the Fusion bridge starts and Fusion is detected. ### Handling `fusion_addin_not_responding` errors When `errorCode: "fusion_addin_not_responding"`, Fusion is running but the AdomBridge add-in isn't answering. Call `fusion_dismiss_blocking_dialogs` FIRST — a modal dialog is the most common cause. If that doesn't help, try `fusion_start` to restart Fusion cleanly. Last resort: user enables add-in manually via UTILITIES > ADD-INS > AdomBridge > Run on Startup + Run. ### Handling `main_thread_busy` errors When `errorCode: "main_thread_busy"`, the Fusion add-in's main thread is occupied by a long-running command (typically `walk_cloud_tree` or `search_cloud_files`). This applies **across all bridges/sessions** — even if you didn't start the walk, another session might have. **Do NOT:** - Retry the failed command — it will block behind the same lock - Call `fusion_dismiss_blocking_dialogs` — there's no dialog to dismiss, and sending Escape will interrupt the active walk - Force-kill Fusion — the walk will complete on its own **Do:** - Wait for the walk/search to finish. If you started it, you should be using `adom-desktop watch` (see "Live folder progress streaming with `watch`" above) which streams live progress automatically. If another session started it, poll `fusion_addin_status` every 2–5s to check progress. - Use commands that **don't need the main thread** while waiting: - `fusion_addin_status` — check busy state and walk progress - `fusion_window_info` — get window HWND, title, dialogs - `fusion_screenshot_fusion` — capture what Fusion looks like - `fusion_click_fusion` — click in the Fusion window - `fusion_send_key` — send keyboard input - `fusion_close_window` — close a specific dialog by HWND The response includes progress info: ```json { "errorCode": "main_thread_busy", "busyCommand": "walk_cloud_tree", "elapsedSeconds": 42.3, "walkProgress": { "foldersVisited": 15, "filesFound": 87, "currentFolder": "Molecules/XRP", "queueSize": 8 }, "_hint": "Add-in is busy with a long-running command. Do NOT retry..." } ``` **Stalled walk detection**: If `fusion_addin_status` returns `busy: true` but `walkProgress` is missing and `mainThreadStalled: true`, a modal dialog is blocking the event loop — the walk was dispatched but never started. Call `fusion_dismiss_blocking_dialogs`, then the walk auto-resumes. ### Auto-recovery (`_autoRecovery` field) When a fusion_* command fails with "not responding" / "not connected", the CLI automatically: 1. Calls `fusion_dismiss_blocking_dialogs` to clear any modal 2. If successful, retries the original command 3. Attaches `_autoRecovery: {action, dismissed[]}` to the retry result If you see `_autoRecovery` in a response, the command already succeeded after auto-dismissal — no manual intervention needed. The field is informational. ## Troubleshooting ### Check connection status Use `status` to see connected clients. A healthy connection shows one client from the user's hostname with a recent `lastPong` timestamp. ### Stale connections causing timeouts Use `kick_all` to reset -- active Adom Desktop apps reconnect within seconds. ### No desktop client connected 1. Confirm the Adom Desktop app is running on the user's PC 2. Confirm it's pointed at the correct WebSocket URL 3. Check if port 8765 is exposed and reachable ### Relay not running ```bash curl -sf http://127.0.0.1:8766/health # If fails: adom-desktop serve & ``` ### Shell commands on Windows `shell_execute` runs in `cmd.exe`. Prefer writing scripts via `send_files` then executing them. Use the full Python path. ## Building from Source ```bash cd cli && cargo build --release # Binary at: cli/target/release/adom-desktop ``` ## Repo [github.com/adom-inc/adom-desktop](https://github.com/adom-inc/adom-desktop)