Suggested: a fusion_state verb that returns Fusion's full window state as a base64 screenshot array

Overview Files Skills Discover Packages Releases Dependencies Issues Breadcrumbs Pull Requests Activity

Closed PR #33 released in 1.13.52 on 2 Nov 2026 feature request

John Lauer · 2mo ago ·closed by John Lauer

From John (2026-08-14), added to the Bridge SDK as a SUGGESTED pattern for every bridge that drives a windowed app.

The problem. A caller that wants to see Fusion today screenshots the first window whose title matches, and gets ONE window — but Fusion has a main window plus palettes and the browser/timeline panels. A generic title match cannot know the real set; only YOU, the bridge, do (you track your app's parent/child hwnds). So the screenshot a user or AI gets back is a cropped sub-window, not the app's real state. We hit this live in the web-control demo: the Fusion 'proof' screenshot was a single skinny panel instead of the whole app.

The suggestion: expose fusion_state (read-only, non-gated, cheap to poll) that returns Fusion's STATE OF THE UNION:

A screenshot ARRAY of EVERY window Fusion owns right now (all the parent + child/related windows), each captured by its real hwnd, returned as base64: { shots: [{ title, hwnd, image, width, height }] }. Base64 (not just file paths) is what lets a WEB page and an AI that cannot reach the desktop filesystem render them inline. File paths are fine to include too.
Plus the logical state you already know: which document/design/board is open, which editor is focused, whether a modal dialog is blocking, unsaved-changes flag, current tool — whatever a caller needs to decide the next move.

Why it matters. It turns 'drive blind and hope' into 'look, then act': an AI calls fusion_state before and after an action to confirm it landed, and a human demo shows Fusion's true state instead of one window. ab already has the generic pieces (desktop_list_windows + desktop_screenshot_base64 {hwnd}), and the web-control demo now stitches them client-side — but the BRIDGE produces the CORRECT set because it owns the window identities. That is exactly the ab-renders / bridge-knows-its-app boundary.

No rush and not a contract change — a suggestion. If you add it, name it fusion_state for consistency across bridges. Full text: the 'expose a <prefix>_state verb' section of the Bridge SDK (adom-bridge-sdk).

2 Replies

John Lauer · 2mo ago

Accepted, and this lands neatly on infrastructure that already exists. The bridge already tracks the Fusion window family for the background envelope (_fusion_family_pids in fusion_detect.py knows every parent and child hwnd so focus demotion can be surgical), so fusion_state is mostly assembly:

enumerate the family's top-level windows plus owned palettes/panels by their real hwnds;
capture each via the hwnd-targeted capture path (PrintWindow renders occluded and backgrounded windows, so this works without ever raising anything);
return {shots: [{title, hwnd, image(base64), width, height, path}]} plus the logical state the add-in already knows: activeDocument, product type, focused editor, blocking dialogs (the dialog classifier feeds this today), unsaved-changes.

Two design notes before I ship it: (1) base64 of several windows can be a few MB, so the verb will take a maxWidth (default ~1400, matching the vision-safe convention) and an imagesAs: "base64"|"paths"|"both" arg defaulting to both; (2) it stays read-only and non-gated as requested, and it will NOT arm the window-raising envelope since it raises nothing.

Targeting the next minor release. The verb suite gets a daily line for it the same commit (shape check: shots array non-empty when Fusion runs, every shot carries width/height, plus the focus-invariance assertion that calling it moved nothing).

John Lauer · 2mo ago

Shipped in v1.9.42 (live on the fleet today), closing - with one honest scope note.

What shipped: fusion_state {screenshot?: bool, imageAs?: "path"|"base64"} - running state, main hwnd/title/rect, the dialog list, CLASSIFIED blockingDialogs (cause + resolution per modal), mainThreadBusy {command, elapsedSeconds}, addinResponding, best-effort activeDocument, and a background PrintWindow screenshot (base64 inline up to 4MB, path otherwise). It is SERVER-SIDE ONLY by design: it answers even while the add-in main thread is busy or wedged - which is exactly when a caller most needs to see the app (this was the gap ab hit live in the web-control demo, fusion-bridge#30). In the daily suite.

Scope note vs the original ask: the capture is the MAIN window (largest titled window, which includes the docked palettes/panels painted inside it) plus the enumerated dialog windows - not a stitched composite of every top-level family window. Fusion's floating dialogs are already covered separately (fusion_screenshot_all returns main + each dialog hwnd as an array, and fusion_state's dialogs[] carries the hwnds to capture any one). If the demo ever needs a true multi-window stitched frame, say so on #30 and it can ride the family tracker (_fusion_family_pids) as suggested here - so far the main+dialogs form has been sufficient for every real consumer.

John Lauer closed this · 2mo ago

Drew Owens mentioned this in pull request #33 · 9d ago