app
Grok Bot
Public Made by Adomby adom
Grok Bridge lets Adom talk to Grok Bot. Adom's AI environment is for electrical engineers: any provider, any desktop power tool, one place.
← Commit history
Add grok_chat_status and grok_chat_read plus hwnd find so the Grok Bot window is visible.
14 files changed
+596−60
BRIDGE_VERSION+1−1@@ -1 +1 @@-1.1.0+1.2.0
CLAUDE.md+5−5@@ -30,9 +30,9 @@ will seed/stream the runtime from the release zip via `updateManifestUrl` in `br only when Grok Bot.exe is missing (`hostAppFallback:"Cursor"`). - `caller_identity.py` — copy of kicad's; `DELEGATE = "grok"`. All ab callbacks go through `ad_callback`. Never `shell_execute` / `pull_file` / `send_files` from the HTTP path.-- `handlers/*.py` — readiness, describe, windows (open/screenshot/list), send, app- (close/resize/record/login/logout/plugin), webhook, bots.-- `bridge.json` + `BRIDGE_VERSION` — KEEP IN LOCKSTEP (currently 1.1.0).+- `handlers/*.py` — readiness, describe, windows (open/screenshot/list), chat (status/read),+ send, app (close/resize/record/login/logout/plugin), webhook, bots.+- `bridge.json` + `BRIDGE_VERSION` — KEEP IN LOCKSTEP (currently 1.2.0). - `adom-bridge-grok-manifest.json` — streaming manifest ab's `updateManifestUrl` points at; its `url` is the RELEASE download URL of the zip, with matching `sha256`/`size`. - `SKILL.md` (root) — the main user skill, installed as **`grok`** (not `grok-bridge`).@@ -40,10 +40,10 @@ will seed/stream the runtime from the release zip via `updateManifestUrl` in `br - `package.json` — wiki skills pkg. Description is the skills-not-runtime sentence. Runtime files and `*.zip` do **not** belong in `files[]`. -## Verbs (v1.1.0)+## Verbs (v1.2.0) Existing: `grok_readiness` `grok_prewarm` `grok_describe` `grok_status` `grok_open`-`grok_screenshot` `grok_list_windows` `grok_send` `grok_uninstall`.+`grok_screenshot` `grok_list_windows` `grok_chat_status` `grok_chat_read` `grok_send` `grok_uninstall`. App: `grok_close` `grok_resize` `grok_record_start` `grok_record_stop` `grok_record_status` `grok_login` `grok_logout` `grok_plugin_install`.
README.md+4−2@@ -2,7 +2,7 @@ Two-way street between Grok Bot and Adom. -- **Out:** Claude, Codex, and Adom Hydrogen talk to the Grok Bot app on your Windows or Mac so they work as one cohesive unit. `grok_*` can **open / close / resize / screenshot / record** the Grok Bot window and **create bots / channels** via the Grok Bridge webhook.+- **Out:** Claude, Codex, and Adom Hydrogen talk to the Grok Bot app on your Windows or Mac so they work as one cohesive unit. `grok_*` can **open / close / resize / screenshot / record** the Grok Bot window, **report thinking vs idle** (`grok_chat_status`) and **read the visible transcript** (`grok_chat_read`), and **create bots / channels** via the Grok Bridge webhook. - **Back:** Grok Bot gets Adom Anywhere (all of Adom's skills) so it knows how to use Adom. Two products, one page (same shape as pup):@@ -28,12 +28,14 @@ Wiki: https://wiki.adom.inc/adom/grok-bridge ## Verbs -`grok_readiness` · `grok_prewarm` · `grok_describe` · `grok_status` · `grok_open` · `grok_screenshot` · `grok_list_windows` · `grok_send` · `grok_uninstall` · `grok_close` · `grok_resize` · `grok_record_start` · `grok_record_stop` · `grok_record_status` · `grok_login` · `grok_logout` · `grok_plugin_install` · `grok_webhook_status` · `grok_webhook_bind` · `grok_webhook_ensure` · `grok_webhook_fire` · `grok_bot_create` · `grok_channel_create` · `grok_bot_list`+`grok_readiness` · `grok_prewarm` · `grok_describe` · `grok_status` · `grok_open` · `grok_screenshot` · `grok_list_windows` · `grok_chat_status` · `grok_chat_read` · `grok_send` · `grok_uninstall` · `grok_close` · `grok_resize` · `grok_record_start` · `grok_record_stop` · `grok_record_status` · `grok_login` · `grok_logout` · `grok_plugin_install` · `grok_webhook_status` · `grok_webhook_bind` · `grok_webhook_ensure` · `grok_webhook_fire` · `grok_bot_create` · `grok_channel_create` · `grok_bot_list` CLI: `adom-bridge grok_*`. Always `--ai-thread`. Background by default. `grok_send {message}` toasts `Grok Bot: {message}` and appends `~/.adom/grok-inbox.jsonl`. Desktop typing is opt-in — don't steal the keyboard. +`grok_chat_status` reports `thinking` vs `idle` from the UIA working chip (`"<agent> is working"`). `grok_chat_read` reads the visible Document transcript (`desktop_ui_text` role Document range all). Optional `limit` truncates from the end. Never OCR-first, never CDP 9222, never webhook keys in the payload.+ ## Webhook Prefer `grok_webhook_ensure {}` — it reads UIA Edit controls **Webhook URL** / **Webhook key** from the Grok Bot routine pane (no OCR) and writes `~/.adom/grok-webhook.json` (Windows: `%USERPROFILE%\.adom\grok-webhook.json`) as `{url, key, header, boundAt}`. Manual `grok_webhook_bind {url, key}` still works. The key is never logged or returned. POSTs use `Authorization: Bearer`. Then `grok_bot_create` / `grok_channel_create` / `grok_bot_list`.
SKILL.md+6−4@@ -1,7 +1,7 @@ --- name: grok user-invocable: true-description: "Drive Grok Bot on the user's desktop from the cloud via the grok_* Adom Bridge verbs. Host app is Grok Bot.exe (standalone Electron; Cursor is last-resort fallback only if Grok Bot.exe is missing). Open/close/resize/screenshot/record the app, queue a message (toast + inbox), bind the Grok Bridge webhook and create bots/channels. No public Grok HTTP API — the spawned Python runtime talks back to ab over desktop_* callbacks. Trigger words: grok, grok bot, grok_open, grok_close, grok_webhook_ensure, grok_webhook_bind, talk to grok, send to grok bot, grok screenshot."+description: "Drive Grok Bot on the user's desktop from the cloud via the grok_* Adom Bridge verbs. Host app is Grok Bot.exe (standalone Electron; Cursor is last-resort fallback only if Grok Bot.exe is missing). Open/close/resize/screenshot/record the app, report thinking vs idle (grok_chat_status), read the visible transcript (grok_chat_read), queue a message (toast + inbox), bind the Grok Bridge webhook and create bots/channels. No public Grok HTTP API — the spawned Python runtime talks back to ab over desktop_* callbacks. Trigger words: grok, grok bot, grok_open, grok_close, grok_chat_status, grok_chat_read, grok_webhook_ensure, grok_webhook_bind, talk to grok, send to grok bot, grok screenshot." --- # grok — talk to Grok Bot from Adom Bridge@@ -29,7 +29,7 @@ There is no public Grok HTTP API. `ab` spawns a small Python server from the **r | `grok_readiness` `hostAppInstalled:false` / `errorCode: host_app_not_installed` | Grok Bot.exe is not on this PC | **Do not send the user to a download page.** Offer `grok_open`. That verb detects, then asks ab to `launch` / `desktop_open`. | | `hostAppFallback:"Cursor"` | Grok Bot.exe missing; Cursor is last-resort host | Prefer Grok Bot. Drive Cursor only as fallback. **Never kill Cursor.** | | `hostAppInstalled:true`, `hostAppRunning:false` | Grok Bot is on disk, not running | `grok_open {}` — launches in the **background**. |-| `ready:true` / host running | Grok Bot is up | `grok_list_windows` / `grok_screenshot` / `grok_send` / `grok_close` / `grok_resize`. |+| `ready:true` / host running | Grok Bot is up | `grok_list_windows` / `grok_chat_status` / `grok_chat_read` / `grok_screenshot` / `grok_send` / `grok_close` / `grok_resize`. | | `errorCode: webhook_not_bound` | `~/.adom/grok-webhook.json` has no url | `grok_webhook_ensure {}` — scrapes the **Grok Bridge webhook** routine pane (no human copy-paste). Manual fallback: `grok_webhook_bind {url, key}`. Never put the key in a hint or log. | | unknown verb / bridge not spawned | runtime zip not installed on ab yet | `adom-bridge bridge_install` with this page's `updateManifestUrl`. Skills ≠ runtime. | | yellow `/status` chip | bridge is up; Grok Bot isn't (or isn't running) | Not a crash. Read `tooltip` / `_hint`. |@@ -50,8 +50,10 @@ adom-bridge --ai-thread "<your thread>" grok_readiness '{}' | `grok_describe` | Catalog with hint/related/pitfalls. | — | | `grok_status` | Detect + last send + inbox + webhook bound (host only) | — | | `grok_open` | Launch Grok Bot in the background; if running, list windows. | optional `foreground` + **required** `foregroundReason` |-| `grok_list_windows` | `desktop_list_windows` titleContains Grok Bot then Grok + `process_list` nameFilter Grok Bot | optional `titleContains` |-| `grok_screenshot` | `desktop_screenshot_window` titleContains Grok Bot or Grok | optional `titleContains` |+| `grok_list_windows` | `desktop_find_window` titleContains exact Grok Bot (preferred; `best.hwnd`) then `desktop_list_windows` + `process_list`. Not bare Grok (Chrome `ai-grok-bot`). | optional `titleContains` |+| `grok_chat_status` | Thinking vs idle. `thinking=true` when UIA exposes `"<agent> is working"` (proven: `"New Bot is working"`). Never steals focus. | — |+| `grok_chat_read` | Visible transcript via `desktop_ui_text {hwnd, role:"Document", range:"all"}` (Document named `"Grok Bot"`). Optional `limit` truncates from the **end** (latest). Never returns webhook keys. | optional `limit` |+| `grok_screenshot` | `desktop_screenshot_window` titleContains Grok Bot | optional `titleContains` | | `grok_close` | Hide the Grok Bot window (`desktop_set_window_state` hide). Prefer hide over kill. Never kill Cursor. | — | | `grok_resize` | `desktop_set_window_bounds` on the Grok Bot hwnd. Restores first if maximized. Physical px (same as ab). | `w`, `h`, optional `x`, `y` (omit to keep current origin) | | `grok_record_start` | `desktop_record_window_start` on the Grok Bot hwnd. | **`reason` required** |
adom-bridge-grok-manifest.json+4−4@@ -1,10 +1,10 @@ { "manifest_version": 1, "name": "grok",- "version": "1.1.0",- "url": "https://wiki.adom.inc/download/adom/grok-bridge/1.1.0/adom-bridge-grok-v1.1.0.zip",- "sha256": "ff0eea91ce1de059f65bb25e8b5feae86d3a472331beb9b05972a93a94763e6c",- "size": 32340,+ "version": "1.2.0",+ "url": "https://wiki.adom.inc/download/adom/grok-bridge/1.2.0/adom-bridge-grok-v1.2.0.zip",+ "sha256": "ef48c6f625d85f2785c58d9ea0142b8afe52b8013cbfaebb632490564bda2ba4",+ "size": 37346, "verbPrefixes": [ "grok_" ],
bridge.json+4−2@@ -2,8 +2,8 @@ "manifest_version": 1, "name": "grok", "displayName": "Grok Bot",- "version": "1.1.0",- "description": "Drive Grok Bot (standalone Electron app, Grok Bot.exe) from Adom Bridge: open/close/resize/screenshot/record the app, queue messages, bind the Grok Bridge webhook and create bots/channels. No public Grok HTTP API \u2014 the runtime talks back to ab over desktop_* callbacks. Cursor is last-resort fallback only if Grok Bot.exe is missing.",+ "version": "1.2.0",+ "description": "Drive Grok Bot (standalone Electron app, Grok Bot.exe) from Adom Bridge: open/close/resize/screenshot/record the app, read thinking vs idle and the visible transcript of the focused thread, queue messages, bind the Grok Bridge webhook and create bots/channels. No public Grok HTTP API \u2014 the runtime talks back to ab over desktop_* callbacks. Cursor is last-resort fallback only if Grok Bot.exe is missing.", "homepage": "https://wiki.adom.inc/adom/grok-bridge", "author": "Adom Inc.", "license": "MIT",@@ -27,6 +27,8 @@ "grok_open", "grok_screenshot", "grok_list_windows",+ "grok_chat_status",+ "grok_chat_read", "grok_send", "grok_uninstall", "grok_close",
grok_detect.py+2−2@@ -26,9 +26,9 @@ CREATE_NO_WINDOW = getattr(subprocess, "CREATE_NO_WINDOW", 0) HOST_APP = "Grok Bot" APP_PATHS_EXE = "Grok Bot.exe"-WINDOWS_HINT = 'titleContains "Grok Bot" first, then "Grok"'+WINDOWS_HINT = 'titleContains "Grok Bot" (exact; not bare Grok — that matches Chrome ai-grok-bot)' PROCESS_FILTER = "Grok Bot"-TITLE_NEEDLES = ["Grok Bot", "Grok"]+TITLE_NEEDLES = ["Grok Bot"] FALLBACK_HOST_APP = "Cursor" FALLBACK_APP_PATHS_EXE = "Cursor.exe"
handlers/chat.pyadded+354@@ -0,0 +1,354 @@+"""Chat verbs: grok_chat_status / grok_chat_read via UIA on the Grok Bot hwnd.++Never steal focus. Never return webhook keys. UIA first, not OCR. Never CDP 9222.+Uses grok_hwnd / _cb / _callback_ok from handlers.windows after the hwnd unwrap fix.+"""++from __future__ import annotations++import re++from grok_detect import HOST_APP, detect_host+from handlers.windows import (+ _cb,+ _callback_ok,+ _title_of,+ flatten_callback,+ grok_hwnd,+)++# Proven UIA on Grok Bot.exe hwnd (AdomLapper): Document named "Grok Bot",+# Edit "Prompt", heading / sand-conversation-heading ("New Bot"),+# Text/button "<agent> is working" while thinking.+DOCUMENT_NAME = "Grok Bot"+PROMPT_NAME = "Prompt"+HEADING_AID = "sand-conversation-heading"+VIEW_DETAILS = "View conversation details"++SIDEBAR_NOISE = {+ "bot list",+ "new bot or channel",+ "search",+ "prompt",+ "view conversation details",+ "conversation",+ "grok bot",+}++_WORKING_SUFFIXES = (" is working", " is thinking")++# Never return webhook keys / bearer tokens in transcript.+_SECRET_RE = re.compile(+ r"(?:crsr_[A-Za-z0-9_-]+)"+ r"|(?:Bearer\s+[A-Za-z0-9._\-+=/]+)"+ r"|(?:Webhook\s+key\s*[:=]\s*\S+)",+ re.IGNORECASE,+)+++def _as_controls(blob) -> list[dict]:+ merged = flatten_callback(blob)+ if isinstance(merged, list):+ return [c for c in merged if isinstance(c, dict)]+ if not isinstance(merged, dict):+ return []+ out: list[dict] = []+ for key in ("controls", "matches", "items", "result", "nodes", "windows"):+ val = merged.get(key)+ if isinstance(val, list):+ out.extend(c for c in val if isinstance(c, dict))+ data = merged.get("data")+ if isinstance(data, list):+ out.extend(c for c in data if isinstance(c, dict))+ best = merged.get("best")+ if isinstance(best, dict):+ out.append(best)+ if not out and (merged.get("name") or merged.get("hwnd") or merged.get("role")):+ out.append(merged)+ return out+++def _ctrl_name(c: dict) -> str:+ return str(c.get("name") or c.get("Name") or c.get("title") or "")+++def _ctrl_value(c: dict) -> str:+ for k in ("value", "Value", "currentValue", "text", "content"):+ v = c.get(k)+ if v is not None and str(v).strip():+ return str(v).strip()+ return ""+++def _extract_text(blob) -> str:+ merged = flatten_callback(blob)+ if isinstance(merged, str):+ return merged+ if isinstance(merged, list):+ parts = []+ for item in merged:+ if isinstance(item, str) and item.strip():+ parts.append(item)+ elif isinstance(item, dict):+ t = _extract_text(item)+ if t:+ parts.append(t)+ return "\n".join(parts)+ if not isinstance(merged, dict):+ return ""+ for k in ("text", "value", "content", "all", "visible"):+ v = merged.get(k)+ if isinstance(v, str) and v.strip():+ return v+ data = merged.get("data")+ if isinstance(data, str) and data.strip():+ return data+ if isinstance(data, dict):+ for k in ("text", "value", "content"):+ v = data.get(k)+ if isinstance(v, str) and v.strip():+ return v+ best = merged.get("best")+ if isinstance(best, dict):+ t = _extract_text(best)+ if t:+ return t+ return ""+++def _looks_working(name: str) -> bool:+ nl = (name or "").strip().lower()+ if not nl:+ return False+ if nl.endswith(" is working") or nl.endswith(" is thinking"):+ return True+ if nl in ("working", "thinking"):+ return True+ return False+++def _strip_working(name: str) -> str:+ n = (name or "").strip()+ nl = n.lower()+ for suf in _WORKING_SUFFIXES:+ if nl.endswith(suf):+ return n[: len(n) - len(suf)].strip()+ return ""+++def _redact_secrets(text: str) -> str:+ if not text:+ return text+ return _SECRET_RE.sub("[redacted]", text)+++def _window_missing(verb: str) -> dict:+ info = detect_host()+ thinking = False+ return {+ "success": False,+ "errorCode": "window_not_found",+ "error": "No Grok Bot window.",+ "thinking": thinking,+ "idle": False,+ "agentName": None,+ "hwnd": None,+ "title": None,+ "hostApp": HOST_APP,+ "hostAppFallback": info.get("hostAppFallback"),+ "windowEtiquette": "background",+ "_hint": (+ "No Grok Bot hwnd (desktop_find_window titleContains exact \"Grok Bot\"). "+ f"Call grok_open {{}} then grok_{verb} {{}}. Never steal focus. Never kill Cursor. "+ "Do not treat Google Chrome / ai-grok-bot as the host."+ ),+ "_next": f"grok_open {{}} then grok_{verb} {{}}",+ }+++def _find_working(hwnd: int) -> tuple[bool, str]:+ """Return (thinking, working_control_name). Read-only; never focuses."""+ queries = (+ {"hwnd": hwnd, "contains": "is working"},+ {"hwnd": hwnd, "nameContains": "is working"},+ {"hwnd": hwnd, "contains": "working"},+ )+ for args in queries:+ blob = _cb("desktop_find_controls", args)+ ctrls = _as_controls(blob)+ if not ctrls:+ ctrls = _as_controls(_cb("desktop_find_control", args))+ for c in ctrls:+ n = _ctrl_name(c) or _ctrl_value(c)+ if _looks_working(n):+ return True, n+ return False, ""+++def _read_agent_name(hwnd: int, working_name: str = "") -> str:+ stripped = _strip_working(working_name)+ if stripped:+ return stripped+ queries = (+ {"hwnd": hwnd, "automationId": HEADING_AID},+ {"hwnd": hwnd, "name": HEADING_AID},+ {"hwnd": hwnd, "role": "Heading"},+ {"hwnd": hwnd, "name": VIEW_DETAILS},+ )+ for args in queries:+ blob = _cb("desktop_find_control", args)+ for c in _as_controls(blob):+ n = (_ctrl_name(c) or _ctrl_value(c)).strip()+ if n and n.lower() not in SIDEBAR_NOISE:+ return n+ blob2 = _cb("desktop_find_controls", args)+ for c in _as_controls(blob2):+ n = (_ctrl_name(c) or _ctrl_value(c)).strip()+ if n and n.lower() not in SIDEBAR_NOISE:+ return n+ return HOST_APP+++def _prompt_value(hwnd: int) -> str:+ blob = _cb("desktop_ui_get", {"hwnd": hwnd, "name": PROMPT_NAME, "role": "Edit"})+ text = _extract_text(blob)+ if text:+ return text.strip()+ blob = _cb("desktop_find_control", {"hwnd": hwnd, "name": PROMPT_NAME, "role": "Edit"})+ for c in _as_controls(blob):+ v = _ctrl_value(c)+ if v:+ return v+ return ""+++def _strip_prompt_garbage(text: str, prompt_val: str) -> str:+ """Drop the Prompt compose box if it is tacked onto the Document transcript."""+ out = text or ""+ pv = (prompt_val or "").strip()+ if pv and out.rstrip().endswith(pv):+ out = out.rstrip()[: -len(pv)].rstrip()+ # Trailing isolated "Prompt" label (compose chrome, not a message).+ lines = out.splitlines()+ while lines and lines[-1].strip().lower() in ("prompt", ""):+ if lines[-1].strip() == "":+ lines.pop()+ continue+ if lines[-1].strip().lower() == "prompt":+ lines.pop()+ continue+ break+ return "\n".join(lines).rstrip()+++def _read_document_text(hwnd: int) -> tuple[str, dict | None]:+ """desktop_ui_text {hwnd, role:Document, range:all} — proven Document 'Grok Bot'."""+ last = None+ for args in (+ {"hwnd": hwnd, "role": "Document", "range": "all"},+ {"hwnd": hwnd, "name": DOCUMENT_NAME, "role": "Document", "range": "all"},+ {"hwnd": hwnd, "name": DOCUMENT_NAME, "range": "all"},+ ):+ last = _cb("desktop_ui_text", args)+ if not _callback_ok(last):+ continue+ text = _extract_text(last)+ if text.strip():+ return text, last+ return "", last+++def handle_chat_status(_args: dict) -> dict:+ """thinking=true when UIA exposes '<agent> is working' / working chip. Never steals focus."""+ hwnd, win = grok_hwnd(allow_cursor=False)+ if not hwnd:+ hwnd, win = grok_hwnd(allow_cursor=True)+ if not hwnd:+ return _window_missing("chat_status")++ thinking, working_name = _find_working(hwnd)+ agent = _read_agent_name(hwnd, working_name)+ title = _title_of(win) or HOST_APP+ return {+ "success": True,+ "thinking": bool(thinking),+ "idle": (not thinking),+ "agentName": agent,+ "hwnd": hwnd,+ "title": title,+ "windowEtiquette": "background",+ "_hint": (+ f"Grok Bot hwnd={hwnd} agent={agent!r} "+ + ("thinking (UIA name like '<agent> is working')." if thinking else "idle (no working chip).")+ + " Read-only UIA — focus not stolen. grok_chat_read for the visible transcript."+ ),+ }+++def handle_chat_read(args: dict) -> dict:+ """Visible transcript via desktop_ui_text Document range=all. limit truncates from the END."""+ hwnd, win = grok_hwnd(allow_cursor=False)+ if not hwnd:+ hwnd, win = grok_hwnd(allow_cursor=True)+ if not hwnd:+ miss = _window_missing("chat_read")+ miss["text"] = ""+ miss["length"] = 0+ return miss++ thinking, working_name = _find_working(hwnd)+ agent = _read_agent_name(hwnd, working_name)+ raw, last = _read_document_text(hwnd)+ prompt_val = _prompt_value(hwnd)+ text = _strip_prompt_garbage(raw, prompt_val)+ text = _redact_secrets(text)++ limit = args.get("limit")+ try:+ limit_n = int(limit) if limit is not None and str(limit).strip() != "" else None+ except (TypeError, ValueError):+ limit_n = None+ if limit_n is not None and limit_n < 0:+ limit_n = None+ if limit_n:+ text = text[-limit_n:]++ ok = bool(text) or _callback_ok(last)+ if not ok and not text:+ return {+ "success": False,+ "errorCode": "transcript_unavailable",+ "agentName": agent,+ "thinking": bool(thinking),+ "idle": (not thinking),+ "text": "",+ "length": 0,+ "hwnd": hwnd,+ "title": _title_of(win),+ "windowEtiquette": "background",+ "_hint": (+ "desktop_ui_text {hwnd, role:Document, range:all} returned no text. "+ "Needs a live Grok Bot window with the chat Document (name \"Grok Bot\"). "+ "Never OCR-first. Never CDP 9222. grok_chat_status {} then retry grok_chat_read."+ ),+ "_next": "grok_chat_status {} then grok_chat_read {}",+ }++ return {+ "success": True,+ "agentName": agent,+ "thinking": bool(thinking),+ "idle": (not thinking),+ "text": text,+ "length": len(text),+ "hwnd": hwnd,+ "title": _title_of(win),+ "windowEtiquette": "background",+ "_hint": (+ f"Read UIA Document range=all on Grok Bot hwnd={hwnd} "+ f"({len(text)} chars, agent={agent!r}, "+ + ("thinking" if thinking else "idle")+ + "). Latest messages are at the end. Webhook keys redacted. "+ "Prompt compose box stripped when distinguishable. Never stole focus."+ ),+ }
handlers/describe.py+32−5@@ -60,11 +60,37 @@ VERB_CATALOG = { ], }, "list_windows": {- "summary": "List Grok Bot windows (desktop_list_windows + process_list nameFilter Grok Bot).",+ "summary": "List Grok Bot windows (desktop_find_window preferred, then list_windows + process_list).", "long": False,- "hint": "titleContains Grok Bot first, then Grok. Read-only; does not focus or click.",- "related": ["grok_screenshot", "grok_open", "grok_close", "grok_resize"],- "pitfalls": ["empty list usually means Grok Bot isn't running — grok_open, don't panic"],+ "hint": "titleContains exact Grok Bot (not bare Grok / Chrome ai-grok-bot). Read-only; does not focus or click.",+ "related": ["grok_screenshot", "grok_open", "grok_chat_status", "grok_close", "grok_resize"],+ "pitfalls": [+ "empty list usually means Grok Bot isn't running — grok_open, don't panic",+ "Do not treat Google Chrome / ai-grok-bot as the host.",+ ],+ },+ "chat_status": {+ "summary": "Read-only: is the focused Grok Bot thread thinking or idle? Never steals focus.",+ "long": False,+ "hint": "thinking=true when UIA exposes a name like \"<agent> is working\" (proven: \"New Bot is working\"). idle when not thinking.",+ "related": ["grok_chat_read", "grok_list_windows", "grok_screenshot"],+ "pitfalls": [+ "Needs a live Grok Bot window — grok_open first.",+ "Never steal focus. UIA first, not OCR, never CDP 9222.",+ "Do not treat Google Chrome / ai-grok-bot as the host.",+ ],+ },+ "chat_read": {+ "summary": "Read the visible transcript of the focused Grok Bot thread (UIA Document TextPattern).",+ "long": False,+ "hint": "desktop_ui_text {hwnd, role:Document, range:all}. Optional limit truncates from the END (latest messages). Never returns webhook keys.",+ "related": ["grok_chat_status", "grok_list_windows", "grok_send"],+ "pitfalls": [+ "Visible transcript only, not a full history API.",+ "Optional limit is from the end (latest), not the start.",+ "Do not treat Prompt compose-box garbage as a message.",+ "Never echo webhook keys. Never OCR-first. Never CDP 9222.",+ ], }, "send": { "summary": "Queue a message to Grok Bot: toast + durable inbox. Typing is opt-in.",@@ -135,7 +161,7 @@ VERB_CATALOG = { "related": ["grok_logout", "grok_open"], "pitfalls": [ "Do not type passwords or 2FA.",- "Only grokbot:// settings?id=* and plugin/add?id= routes exist in v1.1.",+ "Only grokbot:// settings?id=* and plugin/add?id= routes exist in v1.2.", ], }, "logout": {@@ -254,6 +280,7 @@ def handle_describe(_args: dict) -> dict: "Every verb ALSO returns its own rich _hint when called; that's what to read in practice. " "Host app is Grok Bot.exe (Cursor last-resort fallback). " "Drive the app: grok_open / grok_close / grok_resize / grok_screenshot / grok_record_start. "+ "Chat: grok_chat_status / grok_chat_read (UIA, no focus steal). " "Bots/channels: grok_webhook_ensure then grok_bot_create / grok_channel_create / grok_bot_list. " "CLI: adom-bridge grok_* . Always --ai-thread. Background by default." ),
handlers/webhook.py+1−1@@ -239,7 +239,7 @@ def handle_webhook_fire(args: dict) -> dict: headers = { "Content-Type": "application/json", "Accept": "application/json",- "User-Agent": f"{SOURCE}/1.1.0",+ "User-Agent": f"{SOURCE}/1.2.0", } key = cfg.get("key") or "" if key:
handlers/windows.py+164−29@@ -6,6 +6,8 @@ HTTP callback. All driving goes through caller_identity.ad_callback. from __future__ import annotations +import json+ import caller_identity from grok_detect import ( HOST_APP,@@ -31,36 +33,144 @@ def _callback_ok(res: dict | None) -> bool: return True +def _parse_output_field(blob: dict):+ """ad_callback may wrap the real payload in blob['output'] (JSON object/array)."""+ raw = blob.get("output")+ if isinstance(raw, (dict, list)):+ return raw+ if not isinstance(raw, str):+ return None+ s = raw.strip()+ if not s or s[0] not in "{[":+ return None+ try:+ parsed = json.loads(s)+ except json.JSONDecodeError:+ return None+ if isinstance(parsed, (dict, list)):+ return parsed+ return None+++def flatten_callback(blob):+ """Unwrap ad_callback wrappers: output JSON string, then data dict.++ 1. json.loads(blob['output']) when it is a JSON object/array and merge+ 2. unwrap blob['data'] if dict+ """+ if isinstance(blob, list):+ return blob+ if not isinstance(blob, dict):+ return blob+ merged = dict(blob)+ parsed = _parse_output_field(blob)+ if isinstance(parsed, list):+ return parsed+ if isinstance(parsed, dict):+ merged.update(parsed)+ data = merged.get("data")+ if isinstance(data, dict):+ inner = dict(merged)+ inner.update(data)+ merged = inner+ elif isinstance(data, list):+ return data+ return merged+++def _add_unique_dicts(out: list[dict], seen: set, items) -> None:+ for item in items or []:+ if not isinstance(item, dict):+ continue+ h = item.get("hwnd")+ if h is None:+ h = item.get("handle")+ pid = item.get("pid") if item.get("pid") is not None else item.get("id")+ try:+ if h is not None:+ key = ("hwnd", int(h))+ elif pid is not None:+ key = ("pid", int(pid) if str(pid).lstrip("-").isdigit() else pid)+ else:+ key = ("obj", id(item), str(item.get("title") or item.get("name") or ""))+ except (TypeError, ValueError):+ key = ("obj", id(item), str(item.get("title") or item.get("name") or ""))+ if key in seen:+ continue+ seen.add(key)+ out.append(item)++ def _as_windows(blob) -> list[dict]:+ """Collect window dicts from a desktop_list_windows / desktop_find_window blob.++ Unwraps output JSON + data dict. Collects windows/items/result lists AND+ blob['best'] when it has hwnd/title (find_window shape).+ """+ if isinstance(blob, list):+ return [w for w in blob if isinstance(w, dict)] if not isinstance(blob, dict):- if isinstance(blob, list):- return [w for w in blob if isinstance(w, dict)] return []+ parsed = _parse_output_field(blob)+ if isinstance(parsed, list):+ windows = [w for w in parsed if isinstance(w, dict)]+ best = blob.get("best")+ if isinstance(best, dict) and (best.get("hwnd") or best.get("title") or best.get("handle")):+ windows.append(best)+ return windows++ merged = flatten_callback(blob)+ if isinstance(merged, list):+ return [w for w in merged if isinstance(w, dict)]+ if not isinstance(merged, dict):+ return []++ out: list[dict] = []+ seen: set = set() for key in ("windows", "items", "result"):- val = blob.get(key)+ val = merged.get(key) if isinstance(val, list):- return [w for w in val if isinstance(w, dict)]- data = blob.get("data")- if isinstance(data, dict):- return _as_windows(data)+ _add_unique_dicts(out, seen, val)+ data = merged.get("data") if isinstance(data, list):- return [w for w in data if isinstance(w, dict)]- if blob.get("hwnd") or blob.get("title") or blob.get("handle"):- return [blob]- return []+ _add_unique_dicts(out, seen, data)+ best = merged.get("best")+ if isinstance(best, dict) and (best.get("hwnd") or best.get("title") or best.get("handle")):+ _add_unique_dicts(out, seen, [best])+ if not out and (merged.get("hwnd") or merged.get("title") or merged.get("handle")):+ _add_unique_dicts(out, seen, [merged])+ return out def _as_processes(blob) -> list[dict]:+ """Same unwrap as _as_windows: output JSON, data dict, processes/items/result, best."""+ if isinstance(blob, list):+ return [p for p in blob if isinstance(p, dict)] if not isinstance(blob, dict): return []+ parsed = _parse_output_field(blob)+ if isinstance(parsed, list):+ return [p for p in parsed if isinstance(p, dict)]++ merged = flatten_callback(blob)+ if isinstance(merged, list):+ return [p for p in merged if isinstance(p, dict)]+ if not isinstance(merged, dict):+ return []++ out: list[dict] = []+ seen: set = set() for key in ("processes", "items", "result"):- val = blob.get(key)+ val = merged.get(key) if isinstance(val, list):- return [p for p in val if isinstance(p, dict)]- data = blob.get("data")- if isinstance(data, dict):- return _as_processes(data)- return []+ _add_unique_dicts(out, seen, val)+ data = merged.get("data")+ if isinstance(data, list):+ _add_unique_dicts(out, seen, data)+ best = merged.get("best")+ if isinstance(best, dict) and (best.get("pid") or best.get("name") or best.get("imageName")):+ _add_unique_dicts(out, seen, [best])+ return out def _hwnd_of(win: dict | None):@@ -81,12 +191,18 @@ def _title_of(win: dict | None) -> str: return str(win.get("title") or win.get("name") or "") +def _is_host_impostor(title: str) -> bool:+ """Google Chrome / ai-grok-bot are NOT the Grok Bot host."""+ tl = (title or "").lower()+ return "google chrome" in tl or "ai-grok-bot" in tl++ def _title_rank(title: str) -> int: tl = (title or "").lower()+ if _is_host_impostor(title):+ return 9 if "grok bot" in tl: return 0- if "grok" in tl:- return 1 if "cursor" in tl: return 2 return 9@@ -106,23 +222,41 @@ def _dedupe_windows(windows: list[dict]) -> list[dict]: def collect_grok_windows(title_contains: str | None = None) -> tuple[list[dict], bool]:- """List Grok Bot windows. titleContains: Grok Bot first, then Grok (then Cursor if fallback)."""+ """List Grok Bot windows. Prefer desktop_find_window (best.hwnd) over list_windows.++ titleContains: exact "Grok Bot" first. Bare "Grok" is too greedy (Chrome "ai-grok-bot").+ """ info = detect_host() needles = [title_contains] if (title_contains and str(title_contains).strip()) else window_title_needles(info) windows: list[dict] = [] ab_up = False++ def _ingest(blob) -> None:+ nonlocal ab_up+ ab_up = ab_up or _callback_ok(blob)+ windows.extend(_as_windows(blob))++ def _hosts() -> list[dict]:+ return [w for w in windows if not _is_host_impostor(_title_of(w))]+ for needle in needles: needle = (needle or "").strip() if not needle: continue- blob = _cb("desktop_list_windows", {"titleContains": needle})- ab_up = ab_up or _callback_ok(blob)- windows.extend(_as_windows(blob))- found = _cb("desktop_find_window", {"titleContains": needle})- ab_up = ab_up or _callback_ok(found)- windows.extend(_as_windows(found))- windows.sort(key=lambda w: _title_rank(_title_of(w)))- return _dedupe_windows(windows), ab_up+ _ingest(_cb("desktop_find_window", {"titleContains": needle}))+ if _hosts():+ break+ if not _hosts():+ for needle in needles:+ needle = (needle or "").strip()+ if not needle:+ continue+ _ingest(_cb("desktop_list_windows", {"titleContains": needle}))+ if _hosts():+ break+ filtered = _hosts()+ filtered.sort(key=lambda w: _title_rank(_title_of(w)))+ return _dedupe_windows(filtered), ab_up def pick_grok_window(windows: list[dict] | None = None, *, allow_cursor: bool = True) -> dict | None:@@ -248,7 +382,8 @@ def handle_list_windows(args: dict) -> dict: "abCallbackReached": ab_up, "windowEtiquette": "background", "_hint": (- "Listed via ab desktop_list_windows (titleContains Grok Bot first, then Grok) and "+ "Listed via ab desktop_find_window (preferred; best.hwnd) then desktop_list_windows, "+ "titleContains exact Grok Bot (not bare Grok / Chrome ai-grok-bot), and " f"process_list nameFilter {filt!r}. Read-only — no focus steal. " + empty_hint + fallback_hint
package.json+13−4@@ -1,8 +1,8 @@ { "slug": "grok-bridge",- "version": "1.1.0",+ "version": "1.2.0", "type": "app",- "description": "Claude, Codex, and Hydrogen talk to Grok Bot on Windows or Mac. Grok Bot gets Adom Anywhere so they work as one. grok_* can open/close/resize/screenshot/record the Grok Bot app and create bots/channels via webhook.",+ "description": "Claude, Codex, and Hydrogen talk to Grok Bot on Windows or Mac. grok_* can open/close/resize/screenshot/record the app, report thinking vs idle, read the visible transcript, and create bots/channels via webhook. Grok Bot gets Adom Anywhere so they work as one.", "keywords": [ "grok", "grok-bot",@@ -39,7 +39,7 @@ "adom/adom-bridge": "^2.0.0" }, "author": "John Lauer <[email protected]>",- "discovery_pitch": "Two-way street: cloud AIs (Claude, Codex, Hydrogen) drive the Grok Bot app on your desktop via grok_* (open/close/resize/screenshot/record, webhook bots/channels); Grok Bot gets Adom Anywhere so it knows how to use Adom.",+ "discovery_pitch": "Two-way street: cloud AIs (Claude, Codex, Hydrogen) drive the Grok Bot app on your desktop via grok_* (open/close/resize/screenshot/record, chat_status/chat_read, webhook bots/channels); Grok Bot gets Adom Anywhere so it knows how to use Adom.", "discovery_triggers": [ "grok", "grok bot",@@ -58,7 +58,12 @@ "create grok bot", "grok_webhook_bind", "sign in grok bot",- "grok_webhook_ensure"+ "grok_webhook_ensure",+ "grok_chat_status",+ "grok_chat_read",+ "is grok thinking",+ "read grok chat",+ "grok transcript" ], "sample_prompts": [ {@@ -84,6 +89,10 @@ { "label": "Record Grok Bot", "prompt": "Start a background recording of the Grok Bot window for a demo, then stop it"+ },+ {+ "label": "Read Grok chat",+ "prompt": "Call grok_chat_status then grok_chat_read and tell me if Grok Bot is thinking and what the latest visible messages say" } ], "hero": {
server.py+5@@ -38,6 +38,7 @@ from handlers.describe import handle_describe from handlers.readiness import handle_prewarm, handle_readiness, handle_status, handle_uninstall from handlers.send import handle_send from handlers.webhook import handle_webhook_bind, handle_webhook_ensure, handle_webhook_fire, handle_webhook_status+from handlers.chat import handle_chat_read, handle_chat_status from handlers.windows import handle_list_windows, handle_open, handle_screenshot DEFAULT_PORT = 8769@@ -58,6 +59,8 @@ KNOWN_VERBS = [ "grok_open", "grok_screenshot", "grok_list_windows",+ "grok_chat_status",+ "grok_chat_read", "grok_send", "grok_uninstall", "grok_close",@@ -96,6 +99,8 @@ COMMAND_HANDLERS = { "open": handle_open, "screenshot": handle_screenshot, "list_windows": handle_list_windows,+ "chat_status": handle_chat_status,+ "chat_read": handle_chat_read, "send": handle_send, "uninstall": handle_uninstall, "close": handle_close,
skills/grok-anywhere/SKILL.md+1−1@@ -52,7 +52,7 @@ For the full desktop map, load **adom-bridge-anywhere**. For loopback-vs-relay / If a *cloud* AI needs to reach **this** Grok session from a Windows ab, that is the **grok** skill — the spawned runtime on the PC: -- App: `grok_open` / `grok_close` / `grok_resize` / `grok_screenshot` / `grok_record_start` / `grok_send`+- App: `grok_open` / `grok_close` / `grok_resize` / `grok_screenshot` / `grok_record_start` / `grok_send` / `grok_chat_status` / `grok_chat_read` - Auth (human clicks, no passwords): `grok_login` / `grok_logout` (Sign In with Cursor / Sign Out) - Plugins: `grok_plugin_install {id}` - Webhook (routine **Grok Bridge webhook**): `grok_webhook_ensure {}` scrapes the pane (Edit **Webhook URL** / **Webhook key**) and stores `~/.adom/grok-webhook.json` (never log the key). Manual fallback `grok_webhook_bind {url, key}`. Then `grok_bot_create` / `grok_channel_create` / `grok_bot_list`.