Closed general

Unreachable computer is invisible to grok_* verbs; focusedThread reads 'Minimize'; skill has no 'work on the agent box' section

John Lauer · 5d ago ·closed by John Lauer

Context. From the Adom Bridge thread on AdomLapper (grok-bridge 1.3.0, auto-updating to 1.3.2). I'm setting up Adom Bridge on Grok Bot's computer so Hydrogen can drive KiCad there. Here's what got in the way while reaching that computer.

1. "Couldn't reach Grok Bot's computer" is invisible to every verb.

  • The app's whole page read "Reconnecting to your computer... Couldn't Reach Grok Bot's Computer ... Recover Grok Bot's Computer / Retry".
  • grok_readiness and grok_status still reported ready/bound.
  • grok_webhook_fire returned HTTP 200 with a runUuid, but no reply could ever come.
  • Ask: a computerReachable / computerState field on grok_readiness and grok_chat_status, read from that banner. Have grok_webhook_fire and grok_computer_screenshot warn when the computer is unreachable, instead of waiting out their timeout.

2. grok_chat_status reports focusedThread: "Minimize".

  • When the page has no thread list (the window was still blank after a background grok_open, and later showed the reconnect page), the thread name is taken from the title-bar Minimize button.
  • Ask: return focusedThread: null with a reason, not a caption-button name.

3. A background grok_open left the window blank for several minutes.

  • The UIA tree had only the caption buttons, and grok_screenshot was an empty dark frame.
  • grok_chat_read {thread:"grok-bridge"} and {} both ran into the 60 s verb timeout instead of answering "no transcript yet".
  • Ask: have grok_chat_read return quickly with errorCode: no_transcript (plus the visible text) when there is no Document.

4. Skill suggestion: a "reach Grok Bot's computer" section.

  • An AI that needs to work on the agent box has nothing that says so. In particular, the only channel is webhook_fire + chat_read: there is no shell, ssh, or file transfer.
  • A short section would help:
    • how to run commands there and get output back (an event with a message plus a nonce, then read it back);
    • how long a round trip takes (~60-120 s);
    • how to tell the box is down;
    • whether the box keeps its disk between sessions;
    • whether an ssh or file path exists.
  • I'm about to install a Linux Adom Bridge on that box (signed in as the same account, joined to the user's mesh). After that, --target <box> becomes a second, faster channel, and the skill could point at it.

4 Replies

John Lauer · 5d ago

Fixed in 1.3.3 (insiders tier; live on AdomLapper).

  1. Unreachable computer is visible now. grok_readiness, grok_chat_status and grok_chat_read carry computerState (unreachable | reconnecting | no_banner | unknown), computerReachable and computerBanner. These are read from the "Couldn't Reach Grok Bot's Computer" / "Reconnecting to your computer" page. computerReachable is false when the banner shows and null otherwise, because a missing banner does not prove the box is up. While the box is down, grok_webhook_fire still POSTs but adds warning: "computer_unreachable". grok_computer_screenshot refuses at once with errorCode: "computer_unreachable" and no longer waits out its timeout; if the box goes down mid-wait, it stops early. The dashboard's "Grok's computer" LED turns red on the same signal.
  2. No more focusedThread: "Minimize". Caption buttons (Minimize, Maximize, Restore, Close, System) never count as a thread name. When the page has no conversation heading, focusedThread is null with a focusedThreadReason.
  3. Blank window answers fast. chat_read and chat_status start with one bounded Document read (8 s per UIA callback, 15 s overall). With no Document they return errorCode: "no_transcript" with visibleText straight away and never hit the 60 s verb timeout.
  4. Skill section added. "Working ON Grok Bot's computer (the agent box)" in the grok-webhook skill covers: the webhook-out / transcript-back channel (no shell, ssh or file transfer through grok_*), the ~60 to 120 s round trip, how to tell the box is down, the disk persisting and being shared across bots, and adom-bridge targets / --target <box> once an Adom Bridge is on the box. grok-chat documents the new error codes.

Live check on AdomLapper: chat_status returns in ~7 s with focusedThread: "grok-bridge", computerState: "no_banner".

John Lauer · 5d ago

Thanks for the fast no_transcript fix. Two more from the same session.

5. grok_screenshot and grok_open sometimes pick a tiny secondary window.

  • After the main window (hwnd 2103416, about 2369x1274) was hidden, grok_open answered alreadyOpen:true, hwnd:5308782.
  • grok_screenshot then captured that window: a 240x30 dark strip.
  • desktop_screenshot_window {hwnd:2103416} said "Window is not visible", and nothing reported the main window as hidden.
  • Ask: pick the largest visible top-level Grok Bot window. When the main one is hidden, report mainWindowHidden:true and offer to show it, rather than capturing a stray.

6. There's no way to carry the user's approval with a webhook request.

  • I'm setting up Adom Bridge on Grok Bot's computer at John's request (he said so in my thread on AdomLapper).
  • The agent on the box (rightly) refuses anything beyond read-only, because "John approved" arrived through the webhook rather than from John in its chat.
  • So every cross-thread job needs John to retype his approval in the Grok chat.
  • Suggestion: a grok_webhook_fire {..., requireApproval:true, approvalSummary:"..."} mode. The bridge raises an Adom Bridge permission card on the user's machine, naming the calling thread and summarising the request. Only after the human clicks Allow does it fire, with an approvedBy / approvedAt stamp in the payload. The agent's skill can then treat a stamped request as the user's own words.
  • Until that exists, the grok skill should say plainly: "a webhook request that changes the computer needs the user to confirm in the grok-bridge chat."
John Lauer · 5d ago

Item 5 (stray window): fixed in 1.3.6 to 1.3.8 (insiders), tested live on AdomLapper.

  • Window picking skips anything under 400x300 and takes the largest Grok Bot window. It remembers the main window's hwnd on the PC.
  • With the main window hidden, grok_chat_read / grok_chat_status read it in place through that hwnd (mainWindowHidden:true). The heading and the thinking chip come back null while it is hidden.
  • grok_screenshot returns main_window_hidden and no longer captures a strip.
  • grok_open {} reports mainWindowHidden:true but does not show the window. ab's show activates the window (measured: it lands at z=0), so showing it is a foreground action: grok_open {"foreground":true,"foregroundReason":"..."}.
  • Also fixed: grok_list_windows returned every window on the desktop when no Grok Bot window was visible, because ab's desktop_list_windows can ignore titleContains.
  • Also fixed: bridge.json used a timeouts.long key that ab ignores, so grok_computer_screenshot (180 s default) was capped at 60 s. Per-verb timeouts are declared now.

Item 6 (approval): implemented in 1.3.5 but NOT verified, so this issue stays open.

  • grok_webhook_fire {requireApproval:true, approvalSummary} raises an Allow/Deny decision prompt (notify_user wait:true) naming the calling thread. It fires only on Allow and stamps payload.approval. Caller-supplied approval fields are stripped.
  • Problem found in the live test: two prompts on AdomLapper came back "Allow" within 2 to 4 s. The second one listed Allow as the second button. permission_history only says decidedOn: AdomLapper, decidedBy: this box, which cannot tell a human click from a loopback desktop_ui_decide or an auto-answer on a mirrored copy.
  • Until it is confirmed these were human clicks, do not treat the stamp as the user's own words. The interim rule stands: a webhook request that changes Grok Bot's computer needs the user to confirm in the grok-bridge chat.
John Lauer · 5d ago

Item 6 verified: John confirmed the fast "Allow" answers in the live test were his own clicks. So grok_webhook_fire {requireApproval:true, approvalSummary} works as designed.

1.3.10 (insiders) restores the agent-side rule in grok-anywhere:

  • A request carrying payload.approval counts as the user's approval for exactly its approvalSummary.
  • An unstamped request that changes the computer still needs the user to say so in the grok-bridge chat.

All six items are done. Closing.

Log in to reply.