← Commit history

Fold 'Driving HD remotely' (adom-desktop verbs guide) into the HD page as a doc + README link, instead of a separate wiki page

John Lauer ·f81d9da29d ·4mo ago ·parent be9c3d6
2 files changed +139
README.md+10
@@ -103,6 +103,16 @@ This is for humans and for the AI in your workspace alike (see the bundled  ![Multiple Claude conversations as tabs](screenshots/claude-conversations.png) +### Driving HD from outside (developers / cloud agents)++HD is also fully drivable from an **external** AI or script — e.g. one in an Adom cloud+container — through the `adom-desktop` (Adom Desktop) verbs, the same way these conversation+features were built and tested. If you're automating HD from outside (calling its control API+with `hd_api`, screenshotting windows, driving the real Chrome over the extension bridge),+read **[Driving Hydrogen Desktop remotely →](driving-hd-remotely.md)** — it covers the verbs+and the mistakes to avoid (use `hd_api` not curl; prefer background UIA/CDP over blind+synthetic keystrokes; the control port is dynamic and sometimes needs a restart).+ ---  ## What's new in v15
driving-hd-remotely.mdadded+129
@@ -0,0 +1,129 @@+# Driving Hydrogen Desktop remotely (via Adom Desktop verbs)++You are (often) an AI in an **Adom cloud container**. The user's **Hydrogen Desktop (HD)** runs+on **their Windows laptop**, with a **WSL2 workspace** inside it. You reach all of it through the+**`adom-desktop` CLI** — the Adom Desktop (AD) relay. Don't SSH, don't hand-roll transports; use+the verbs.++```+  cloud container (you)  ──adom-desktop──▶  AD on the laptop  ──▶  HD app  ──▶  WSL2 workspace+        Tier 1                               Tier 2                 │            (code-server,+                                                                    └─ control API   claude, tools)+                                                                       (dynamic port)+```++Discover everything: `adom-desktop help`, `adom-desktop help <namespace>`,+`adom-desktop help <verb>`, `adom-desktop commands` (full JSON catalog).++## Call HD's control API with `hd_api` — NOT curl++HD exposes a large control API on a **dynamic localhost port**. To call ANY endpoint:++```bash+adom-desktop hd_api '{"path":"/claude/conversations"}'+adom-desktop hd_api '{"path":"/claude/inject","method":"POST","body":{"text":"hi","context_index":7,"submit":true}}'+```+Returns `{ok, status, body, port, ...}` with `body` already parsed as JSON.++- **`hd_api` auto-discovers the port** (from `ports.json`, falls back to 47084) and takes a real+  JSON `body` — no escaping.+- ❌ **MISTAKE TO AVOID:** calling the control API via `shell_execute "curl …"`. The relay mangles+  nested quotes, you hardcode a port that changes between launches, and you burn time on escaping.+  `hd_api` exists precisely so you never do this.++## The HD lifecycle / dev verbs (`hd` namespace)++| Verb | Use |+|---|---|+| `hd_status` | Composed snapshot of HD state |+| `hd_log` | Tail HD's log (`%APPDATA%/hydrogen-desktop/hydrogen-desktop.log`) |+| `hd_shot` `{region}` | Screenshot an HD region (full/vscode/wiki/claude…) → returns local PNG path |+| `hd_launch` / `hd_stop` / `hd_restart` | Start (detached) / kill / both |+| `hd_build_rust` `{show}` + `hd_build_status` | Rust build (async) + poll until `building:false`, assert `succeeded` |+| `hd_build_frontend` | Frontend-only build |+| `hd_ship` | Atomic rebuild-from-pushed-SHA + relaunch + verify |+| `hd_send_files` | Push files straight into the WSL2 project |+| `hd_open_url` | Open a URL in a specific browser profile via HD |++- ⚠️ **Control port intermittently fails to bind on a launch.** If `hd_api` says "HD not+  reachable", check `netstat` for the control port on HD's PID; a `hd_restart` usually fixes it.+- ⚠️ Don't trust a build's SHA stamp to confirm "my code shipped" — verify by behavior (call the+  new endpoint) or a code-marker, not the stamp.++## Run things on the laptop + in the workspace++```bash+# Windows (cmd, NOT PowerShell) on the laptop:+adom-desktop shell_execute '{"command":"tasklist | findstr hydrogen","timeoutSeconds":15}'+# Read .stdout / .stderr from the result — NOT .output (parsing .output comes back empty).++# Multi-line / quote-heavy scripts — base64 them, no quoting survives mangling otherwise:+adom-desktop run_script '{"interpreter":"powershell","scriptB64":"<b64>"}'++# Inside the WSL2 workspace (the Linux side: claude logs, ~/.claude, project files):+adom-desktop wsl_exec '{"distro":"Adom-Workspace","user":"adom","scriptB64":"<b64>"}'+```+- `shell_execute` runs in **cmd.exe** — `Start-Process` etc. need `powershell -NoProfile -Command "…"`.+- Long jobs (builds) must be launched **detached** and polled; the relay has a ~30–120s timeout.++## Move files++```bash+adom-desktop send_files '{"filePaths":["/tmp/x.json"],"dest":"C:/Users/<you>/Downloads"}'+adom-desktop pull_file  '{"filePaths":["C:/path/to/file.png"],"saveTo":"/tmp"}'+```+- ⚠️ `dest` is a **folder** (the file keeps its basename). Files whose basename starts with `_`+  are silently dropped — use plain names.++## Drive the real OS / windows (`desktop` namespace, 80+ verbs)++Two modes — **always prefer background**:++- **BACKGROUND (no focus steal):** `desktop_screenshot_window {hwnd}`, `desktop_find_window+  {titleContains}`, `desktop_find_control {hwnd,name|contains|role}` (reports rect + invokable ++  settable), `desktop_ui_click` (UIA Invoke), `desktop_ui_set` (UIA SetValue), `desktop_ui_focus`,+  `desktop_navigate` (set a browser omnibox without foreground).+- **FOREGROUND (SendInput — steals focus, risky):** `desktop_bring_to_front {hwnd}`,+  `desktop_click {x,y}`, `desktop_type {text}`, `desktop_press_key {keys}`.++### ❌ The biggest mistake: blind synthetic typing++`desktop_type` / `desktop_press_key` send OS-level keystrokes to whatever window currently has+focus. If your target window isn't reliably foregrounded, **your keystrokes land in the WRONG+window** — including the user's own chat with you. (This actually happened: a test marker typed+"into a web page" instead got submitted into the operator's chat.) Rules:++1. Prefer **background** verbs: `desktop_ui_set` / `desktop_ui_click` by accessible name — no+   focus, no cursor move. Chromium/Edge expose their a11y tree, so most page buttons/inputs are+   reachable this way.+2. If a field is **shadow-DOM** (`desktop_find_control` returns `settable:false, invokable:false`+   — common for modern web apps like the claude.ai composer), UIA can't set it. Use the+   **native-browser CDP bridge** (`nbrowser_*`, install via `bridge_install`) to type into the+   page over CDP. Do NOT fall back to blind SendInput.+3. If you MUST use SendInput: `desktop_bring_to_front {hwnd}`, then **verify** the target is+   actually foreground before typing, and confirm the result landed where you intended.+4. `desktop_find_control` gives `rect.centerX/centerY` (screen px) for a precise `desktop_click`+   — but the window must be on top at that point or you click whatever is there.++## Worked example: open a Claude conversation on claude.ai from the cloud++```bash+# 1. find a conversation + its cloud URL (Remote Control)+adom-desktop hd_api '{"path":"/claude/conversations"}'+adom-desktop hd_api '{"path":"/claude/remote-control","method":"POST","body":{"context_index":7}}'+#    → body.remote_control_url = https://claude.ai/code/session_…+# 2. open it in the user's Chrome (new window)+adom-desktop shell_execute '{"command":"powershell -NoProfile -Command \"Start-Process chrome.exe -ArgumentList '"'"'--new-window'"'"','"'"'<url>'"'"'\""}'+# 3. screenshot to confirm — background, no focus steal+adom-desktop desktop_find_window '{"titleContains":"Claude Code - Google Chrome"}'   # → best.hwnd+adom-desktop desktop_screenshot_window '{"hwnd":<hwnd>}'                              # → local PNG path+```++## Checklist before you act++- Reaching HD's API? → `hd_api`, not curl.+- Reading a command's output? → `.stdout`/`.stderr`, not `.output`.+- Typing into a window? → background UIA / CDP first; only SendInput after foreground+verify.+- Long build? → launch detached, poll `hd_build_status`, verify by behavior.+- Port "not reachable"? → it's dynamic and sometimes doesn't bind; `hd_restart`.+- Moving files? → `dest` is a folder; no leading-underscore basenames.