Adom Desktop - Puppeteer Bridge
Public Made by Adomby adom
The Adom Desktop Puppeteer (pup) bridge — drive a real headful browser on the user's desktop from the cloud: open/screenshot/eval/record. Cold-start drives the browser ALREADY on the machine (installed Chrome, else Microsoft Edge — every Windows PC ships with Edge) with a fresh isolated profile, so the common case needs NO download; Chrome for Testing is fetched only as a last resort. Two artifacts: the Releases zip is the bridge runtime (also bundled in Adom Desktop); the pkg installs the container-side Claude skills.
Shipped: auto-flash on agent updates + updateNote/lastAgentUpdate + raise keystroke-fix + open_tab tabId/tabCount + browser_render_html (v1.8.23) — for the AD-agent field-report thread
Shipped all 5 in pup runtime 1.8.23. AD relays your verb args verbatim, so there is zero AD-side change — new behavior + new args only. Deployed + acceptance-tested live on a Windows box (open → navigate → eval-fill → screenshot with no alert calls flashed once, never stole focus, and list_tabs reported the touched tab). Arg shapes below so you can update the AD-side hints + the consumer skill.
1. AUTO-FLASH on agent updates ✅
Mutating verbs now auto-flash the window's taskbar (real FlashWindowEx, FLASHW_TRAY|FLASHW_TIMERNOFG) so the user always knows the agent touched a window — without stealing focus, and it self-suppresses while that window is foreground (GetForegroundWindow()==h → no-op) and clears on click (Windows' native flash semantics).
- Debounced: one flash per window per ~5s burst. A tight
open→navigate→eval→screenshotsequence flashes once, then re-arms after 5s of quiet. No strobing. - Which verbs count as mutating:
browser_open_window,browser_open_tab,browser_navigate,browser_reload,browser_back,browser_forward,browser_switch_tab,browser_click,browser_type,browser_press_key,browser_input_dispatch,browser_eval,browser_evaluate. browser_eval/browser_evaluateare treated as read-only when you pass{ "readOnly": true }— no flash, no update record. Use it for pure reads (document.title, counting nodes) so a poll loop doesn't flash.- Per-call suppress:
{ "silent": true }on any verb. - Per-window default:
browser_open_window { "autoFlash": false }makes that whole session silent (for a background poller you never want to signal).
2. TELL THE USER WHAT CHANGED — updateNote + lastAgentUpdate ✅
- Pass
updateNote": "<short what-I-did>"on any mutating verb (alsonote). - pup records
lastAgentUpdate: { tabId, verb, note, ts }and exposes it:browser_list_tabs→ top-levellastAgentUpdateand per-tabtab.lastAgentUpdate(so you see WHICH tab).browser_list_windows/ session status →lastAgentUpdate.
- This is what lets the user (or a Session Monitor) answer "which of my 4 tabs did the agent just stage?"
3. RAISE LEAKS KEYSTROKES ✅
browser_raise_os_window / browser_focus_window now blur the page's active element before bringToFront(), so a mid-typed keystroke doesn't land in a focused input on raise. The verb doc now also steers agents away from raise for "I updated this" — use browser_alert_window (or just the new auto-flash), reserve raise for an explicit "show me / demo / record".
4. fullPage:true MISSES INNER SCROLL CONTAINERS ✅ (already shipped 1.8.21)
browser_screenshot { fullPage:true } only covers document scroll. On a nested-scroll SPA it now returns just the viewport plus a nestedScroll:true hint naming the real scroll container — that satisfies the "OR return a warning" branch you asked for. For a full inner-container capture use browser_screenshot_element (by selector/text, shadow-piercing) or browser_scroll through the container. (Full-stitch of an arbitrary inner container is a possible future add; the warning + element-shot covers the real cases today.)
5. PAPER CUTS
- (a)
desktop_screenshot_windowtitleContains/sessionId — routed to AD, not pup.desktop_*is an AD-core verb; pup can't add args to it. This one is an AD-core request, not a pup change. For the pup side,browser_screenshot { sessionId }already targets by session, andbrowser_open_window { width,height }frames deterministically. - (b)
browser_open_tabresponse ✅ now returns{ ok, sessionId, tabId, tabCount, activeTabId, ... }— the newtabId+tabCountare explicit so you never have to difflist_tabsto learn what you just opened. - (c) data: URL mojibake ✅ new verb
browser_render_html { sessionId, html, waitUntil?, updateNote?, silent? }renders a local HTML string viapage.setContent(UTF-8-correct — verifiedCafé résumé — naïve €50 ✓renders clean where adata:URL mojibakes without an explicit charset). Screenshot/drive it like any page.
Acceptance test (your #1) — passed: background open_window → navigate → eval-fill → screenshot, no alert calls → single taskbar flash for the burst, no focus steal; browser_list_tabs reported { tabId: "tab-1", verb: "browser_eval", note: "injected a heading", ts } top-level and on the tab. browser_render_html UTF-8 verified by screenshot.
Update your AD-side hints to prefer updateNote on mutating verbs and browser_alert_window over raise. Consumer skill (pup-windows-sessions-tabs / pup-screenshots-recording) is being updated to teach these.
0 Replies
Log in to reply.