app

Pup - Puppeteer Bridge

Public Made by Adom

by adom

pup is the AI's own browser: a real, full Chrome on the user's desktop that the AI fully controls (a sandbox, not the user's signed-in browser). Rides Bridge; pup_* verbs open windows and tabs, navigate, screenshot, and eval JS.

skillspuppuppeteerbrowseradom-bridgebridgepup-bridge
Overview Files Skills Discover Packages Releases Dependencies Issues4 Breadcrumbs Pull Requests Activity
pup-bridge / dev-skills / pup-taskbar-identity / SKILL.md
Download

name: pup-taskbar-identity description: DEV skill (source-only) - how pup maps each window to its OS taskbar button (AUMID tile + overlay badge), the hard-won architecture, and the RALPH TEST you must run before claiming any taskbar-identity fix works. Read before touching birth-hwnd capture, resolveSessionHwndByBounds, hwndBelongsToPup, stampPupIdentity, or applyAppOverlay.

pup taskbar identity: birth-hwnd capture + the ralph gate

Every pup window must present as its own teal "Adom Pup" taskbar button (per-session AUMID) with a favicon overlay badge. All pup windows share ONE Chrome process (the adom-you profile), so the only way to address a specific window for the AUMID stamp and overlay is its OS window handle (hwnd). Getting that handle reliably is the entire problem, and it has broken ~a dozen times.

THE RALPH RULE (John, 2026-08-09: "you need to do your own ralph test")

A SINGLE open looking right proves nothing — every taskbar-identity regression here passed a one-window spot check and failed under repetition/concurrency. Before claiming ANY fix works, run ralph_icons.sh (in this folder): 9 rounds cycling single / concurrent-3 / multi-tab-pair opens, asserting each window gets a DISTINCT handle + a successful stamped appId + an overlay PAINT/ ENFORCED, and that no two sessions ever share a handle (COLLISION). Warm the bridge first (pup_readiness until ok) — the first 1-2 rounds after a bridge_install respawn fail on ab cold-start flakiness, NOT icon logic, and will mislead you. A clean run is 18/18 windows, 0 failures, 18 distinct handles. Baseline proof (2.0.39): two consecutive warm runs, 0 failures each.

bash dev-skills/pup-taskbar-identity/ralph_icons.sh 9

How handle capture actually works (2.0.38, the version that finally held)

  1. Every pup window on Windows is created via createPageInNewWindow (CDP Target.createTarget {newWindow:true}) — including the FIRST window on a profile. (Pre-2.0.37 the first window used browser.newPage(), which captures no handle; that window was permanently unresolvable and showed a raw Chrome button. THE bug John kept catching.)
  2. Capture is a serialized set-diff, NOT position matching:
    • A global _birthLock serializes creation so only one window is born at a time.
    • Snapshot all Chrome Chrome_WidgetWin_1 handles BEFORE createTarget; the ONE new handle after (not in the snapshot, not owned by another live session) is this window's, wherever Chrome placed it. Do not identify the window by position — Chrome clamps the -32000 birth coordinate on-screen on some boxes, so position matching finds nothing or cross-wires (the fusion+nb-shared-handle collision).
  3. hwndBelongsToPup trusts a birth-captured handle as proof by construction (no title tag, no pid membership). hwndOwnedByOtherSession guards every resolve path so two sessions can never share one handle.
  4. Only then does stampPupIdentity set the per-session AUMID (creates the teal button) and applyAppOverlay paint the favicon badge — deferred until the handle exists, stamp-once then overlay-once (see the AUMID/overlay ordering in server.js).

Bans / traps (each shipped and hurt)

  • Never identify a window by screen position — Chrome clamps off-screen births; positions collide under concurrency. Set-diff by handle only.
  • Never remove the title tag without replacing its handle-resolution role — 2.0.24 killed the title tag purely as user-facing cosmetics, silently removing the shared-profile fallback that the stamp AND hwndBelongsToPup relied on. That single change caused weeks of raw-Chrome icons.
  • Never trust a "stamped ✓" log as proof the tile is right — the log says the AUMID call succeeded; it does NOT prove Windows repainted the button or that the handle was the RIGHT window. Verify with the ralph handle-distinctness + a real taskbar screenshot.
  • Overlay state is WRITE-ONLY: an audit that greps paint logs is not an audit (2.0.47, John: "that was a shit audit"). Windows cannot be asked whether a badge is still on the button, and shell-side events (the taskbar progress cycle among them) clear overlays WITHOUT pup knowing — paint memory said painted, the screen showed a bare tile, bad-lines=0. Auditing icons means SCREENSHOTTING THE TASKBAR and checking EVERY tile for its badge, one by one. Code-side defense: every progress change voids _lastPaintSig/_appOverlay (enforcer repaints in 4s), and a 60s per-window ground-truth re-assert repaints unconditionally so memory can never drift from the screen for more than a minute.
  • Warm the bridge before asserting — cold-start (post-install/idle respawn) flakiness is an ab lifecycle issue (ab#61), not icon logic; it produces false failures in rounds 1-2.

Jump-list tasks: pup calls ITSELF (the direct-callback model, v2.0.41)

A jump-list task is a Windows ShellLink {title, target?, args?, iconPath?, description?} that desktop_set_window_jumplist writes onto the taskbar button's right-click menu. The trap: if you OMIT target, ab supplies the program (its own bundled CLI) — an exe path pup neither controls nor can verify. That path broke when the CLI RENAMED (Adom Desktop\adom-desktop-cli.exe → Adom Bridge\adom-bridge-cli.exe), so every task (incl. "Close ALL") silently no-opped: the click launched a missing exe. John: "be way smarter about how to call back into ab for the jump lists ... should pup be able to talk back to itself when it's its own bridge?"

Answer: yes, for pup_ verbs.* Every task now sets target: 'powershell.exe' and args that run a pup-OWNED launcher script (%LOCALAPPDATA%\Adom Bridge\pup-jumplist-call.ps1, written on every bind). The launcher:

  1. reads pup's OWN loopback port from a discovery file pup rewrites on bind (%LOCALAPPDATA%\Adom Bridge\pup-bridge-port.json), then
  2. POSTs the verb straight to http://127.0.0.1:<port>/command — pup talking to itself: no ab relay hop, no CLI exe, no stale path (the two things that broke it), and
  3. ONLY if that direct hit throws (pup idled/died — the Chrome windows OUTLIVE the node process, and pup→pup can't wake a dead pup) falls back to the resolved adom-bridge-cli.exe, which goes through ab and RESPAWNS pup. Fast+accurate when up; ab-wake when asleep.

No ab feature was needed — the per-task target field already existed. Relevant code: pupTask(), ensurePupJumplistLauncher(), writePupPortFile(), and pupCliExe() (disk-resolves the renamed CLI, Bridge name first, legacy fallback) in server.js.

Traps (each cost a debug cycle)

  • JSON args ride as BASE64, not escaped JSON. The ShellLink→Explorer→powershell argv re-parse ate raw escaped JSON for years (the v1.9.9 quoting saga). Base64 is argv-safe (no spaces/quotes); the launcher decodes it. pup_close's {} → e30=.
  • The launcher path has a SPACE ("Adom Bridge"), so pupTask wraps it in quotes: -File "${launcher}". When you TEST via shell_execute→cmd, you must double the backslashes in the JSON (C:\\Users\\...) AND keep the quotes, or cmd/JSON mangles the path and powershell gets -File C:\...\Adom — the exact false-negative that made me think the launcher was broken when it wasn't. The real ShellLink (ab-authored) has no such re-escape; it works on click.
  • user-taskbar-menu is the caller identity (header x-adom-caller-thread + args.caller). The ownership gate in /command EXEMPTS it, so a human clicking Close ALL is never refused for "not owning" the window. Don't route jump-list clicks through a normal thread name.

THE JUMP-LIST RALPH RULE (John: "have you ralph tested those?")

A single click looking right proves nothing here either. ralph_jumplist.sh (this folder) runs N rounds: open wiki windows → fire owner-info + view-toggle + Close ALL through the real deployed launcher (the actual taskbar-click path, powershell -File launcher verb b64) → assert the observable effect (toggle/info don't crash the window; Close ALL empties the count to 0). Clean run is 0 failures. Baseline proof (2.0.41): 5/5 rounds, 0 failures.

bash dev-skills/pup-taskbar-identity/ralph_jumplist.sh 5

Recovered windows: how a respawned pup re-finds handles (2.0.43–2.0.45)

A bridge respawn (every bridge_install) orphans birth capture — the new process created none of the existing windows. The mimic ralph (ralph_mimic.sh, which replays John's real 3-thread workload INCLUDING a respawn leg) proved every recovered window went bare, then caught the fix STEALING handles. The resolution stack for a recovered window, in order:

  1. Continuity proof (2.0.43): the session file persists _hwnd AND Chrome's pid; Chrome outlives pup respawns, so "persisted hwnd whose current owner pid == persisted pid" is proof. Recovery records session._recoveredHwndPid; hwndBelongsToPup accepts the pair.
  2. Natural-title match (2.0.44): the OS window title mirrors the tab CHROME displays. Ask Chrome which tab is document.visibilityState === 'visible' (NOT pup's activeTabId — pup's bookkeeping says first-tab while Chrome shows last-opened; asking the wrong title made a session uniquely match SOMEONE ELSE'S window — the 2.0.45 theft). PID-FENCED to pup-owned pids so the user's own Chrome can never match. A title carried by 2+ pup rows (counted BEFORE ownership exclusions) is refused — excluding the true owner's row is exactly how a theft looks "unique".
  3. Off-screen re-capture (2.0.44): for a bare BACKGROUND window, move it via CDP and identify by bounds CHANGE (set-diff), never absolute position — this box CLAMPS off-screen coordinates, so the old -33000 position match failed silently forever. Restore the EXACT original rect (dumping windows at 0,0 was itself user-visible wonk).
  4. Persisted-claims map (2.0.45): loaded from EVERY session file at recovery START, before any resolve can race — an early-resolving session can otherwise steal a handle whose true owner has not yet restored its in-memory claim (the recovery-storm race the theft rode in on).

Ghost windows: close by WINDOW, never by bound tabs (2.0.46)

Recovery binds ONE tab per session (the sibling-tab rescan runs later), so a pup_close_window issued soon after a respawn closed 1 of N tabs, declared the session closed, and left the OS window ALIVE with the unbound tabs — an orphan pup later adopted as a strange auto-named adom-you-win**** session. The ghost/mystery windows John kept seeing. Fix: the close collects the CDP windowId of every bound tab and closes EVERY page living in those windows, bound or not (with a guard that never touches a page tracked by another session). If a close "succeeds" but the window count on the desktop doesn't drop, suspect this class first: compare desktop_enum_windows (pup pids) count against pup_list_windows session count — they must match.

Capturing a FOREGROUND window: screencast rail, never captureScreenshot (2.0.83)

CDP captureScreenshot compositor-swaps the surface, which visibly BLIPS on a foreground, GPU-accelerated window. Every classic pup screenshot shoots BACKGROUND windows, so the flash was invisible for years; annotate was the first feature photographing the window the user is actively watching, and John saw the blip instantly. Neither captureBeyondViewport:false nor fromSurface tweaks remove it. The artifact-free primitive is a SINGLE SCREENCAST FRAME (the recorder's rail: startScreencast, take one frame, ack, stop), DPR-scale-cropped server-side with sharp (flickerFreeShot in server.js). Rule: any capture of a window the user is looking at rides the screencast rail. And the meta-lesson, John's words after three shipped guesses: trace the WHOLE path in the code you own before shipping a fix; the trace found in minutes what the guesses circled for an hour.

Harness traps (each produced a FALSE verdict)

  • Anchor log analysis to the LAST boot marker (grep -n "Puppeteer Bridge running" | tail -1). The pulled log spans respawns; grepping head-of-file "proves" bugs from a prior boot. Also: a bridge_install takes 40s+ (zip + npm reconstruction) and the OLD process keeps serving during adopt-before-reap — poll /health for the NEW version before asserting anything.
  • Close test windows AS their owner --ai-thread. The ownership gate rightly refuses third-party closes; a harness closing as its own thread gets a silent no-op and leaves ghost windows that recover on every subsequent respawn.
  • Never reset the collision map between phases, and assert NON-test windows too. Handles survive respawns, so a recovered window claiming a different session's known handle is THEFT — a reset map (or asserting only your own windows) grades it OK. That's how the 2.0.45 theft self-graded 0 failures on its first run.

Known remaining gap

Cold-start: the first 1-2 windows opened immediately after a bridge respawn can miss their handle (ab wedge / slow spawn). refresh_bridges + a warm pup_readiness clears it. Tracked as the ab lifecycle work; pup's own capture is clean once warm (see the ralph baseline).