app

Pup - Puppeteer Bridge

Public Made by Adom

by adom

pup is the AI's own browser: a real, full Chrome on the user's desktop that the AI fully controls (a sandbox, not the user's signed-in browser). Rides Bridge; pup_* verbs open windows and tabs, navigate, screenshot, and eval JS.

skillspuppuppeteerbrowseradom-bridgebridgepup-bridge
Overview Files Skills Discover Packages Releases Dependencies Issues5 Breadcrumbs Pull Requests Activity

Commit History

latest 1000 ← Back to Activity
Issue #28 (Drew Owens) — browser_set_viewport no longer pins a window for life. All four of his requested fixes: (1) RESET FORM — {reset:true} (or omitting width/height) calls setViewport(null), clearing the Emulation device-metrics override so the window tracks the real OS window again; previously width/height were required ints with no null, no reset, and the only escape was closing the window, which on a shared Chrome can destroy other threads' windows. (2) STATE SURFACED — emulatedViewport {width,height,deviceScaleFactor}|null now appears in browser_status and browser_list_windows, so a window that ignores OS resizing is visibly in that state instead of being indistinguishable from an app with broken layout code (which cost Drew a real debugging session hunting his own resize handlers). (3) HONEST HINT — the response now states the LIFECYCLE up front: the pin survives navigation AND hard reload, the window stops tracking OS resizing while set, and it names the exact reset call; the old hint sold the feature's benefit and never mentioned the effect was permanent. (4) DOCUMENTED in browser_describe with the same warning, so an agent reading the catalog sees the trap before walking into it.
John Lauer · dea06cc · 11d ago
Publish 1.9.218
John Lauer · 5288152 · 11d ago
Issue #15 — pup now SELF-HEALS its dependencies instead of dying on every respawn. pup ships a source-only zip (node_modules is 88MB of native builds and cannot go in a wiki release) and the design assumes AD runs npm install in the bridge dir on spawn. In practice that does not reliably happen: AD re-provisions the cache dir on respawn, wiping any manual install, and the bridge then dies at require('puppeteer') with MODULE_NOT_FOUND. Reporter's words: 'pup keeps breaking and I always have to reinstall it' — and a human had to npm install by hand again TODAY to bring it back. A bridge that cannot start without a step someone else may not perform is not robust, so it now detects missing deps BEFORE the require that would crash and installs them itself: idempotent, only runs when deps are actually absent, and turns a hard crash-loop into a slow first start. This does not replace AD's install, it makes pup survive when that install did not happen. Also adds a browser_deps verb reporting depsPresent / selfInstalledThisBoot / lastError so 'pup is broken' can be diagnosed without reading the spawn log.
John Lauer · 4e0dd0e · 11d ago
Publish 1.9.217
John Lauer · b2b5e43 · 11d ago
Fetch favicons IN-PAGE (with cookies) before falling back to the cookie-less bridge fetch. John: 'the website being loaded has its favicon?' — dashboard-v2.adom.inc was showing the GENERIC icon despite serving a real one. Cause: since 1.9.130 pup fetches favicons from the BRIDGE process to dodge CORS, and that fetch carries NO cookies, so any favicon behind auth returns 401/403 and pup falls back to a generic icon. The code comment called that tradeoff 'rare' — it is not, it hits EVERY authed Adom dashboard and app. Now pup asks the PAGE to fetch its own favicon first (credentials included, so same-origin authed icons resolve), and only falls back to the bridge fetch for cross-origin icons the page itself cannot read. Both paths retained; neither regresses the other.
John Lauer · 410e53f · 11d ago
Publish 1.9.216
John Lauer · b784c45 · 12d ago
Resolve a pup window's OS handle WITHOUT its title, fixing the taskbar button John has asked about repeatedly. Measured first this time: the composed badge PNG was dumped and inspected and it is CORRECT (PDF sheet + Adom mark inset), so the compositor was never the problem — ten revisions of the image were wasted. The failure is DELIVERY: every paint/park/lookup found windows via the '(session: <id>)' title tag, and Chrome's PDF viewer cannot carry that tag, so the paint had no target and vanished while reporting success. v1.9.212 tried the handle but only stored it when park SUCCEEDED, and park fails on exactly these windows — circular. New resolveSessionHwndByBounds breaks the circle using facts pup already has: the profile's Chrome PID plus the window's exact bounds from CDP Browser.getWindowBounds. It enumerates visible Chrome_WidgetWin_1 top-level windows, matches on geometry across common DPI scales, and caches the handle on the session. The overlay paint and sessionHwnd both resolve through it before ever falling back to a title lookup, so park, badge paint and ownership all work on PDF/untagged windows.
John Lauer · aa18fac · 12d ago
Diagnostic: dump the exact composed badge PNG to disk plus the target (hwnd vs titleContains) before handing it to AD, so the composite can be INSPECTED rather than inferred from a screenshot. Four successive theories about the unpainted first taskbar button were wrong (SVG rasterisation, code-not-running, duplicate sessions, the composite cache); this measurement splits the problem cleanly — if the PNG contains the generic icon then the compositor is exonerated and the fault is delivery to that specific window.
John Lauer · f8cdc90 · 12d ago
Publish 1.9.214
John Lauer · 4b30a5f · 12d ago
THE reason ten badge revisions never appeared: the composite was memo-cached on a key that did NOT include the bridge build. The first image built for a given (session, favicon-href) pair was cached forever, so every later run hit the cache, logged success with no errors, returned the OLD bytes, and the taskbar never changed regardless of what the compositor code did. That is precisely the symptom John reported over and over — 'the code runs, no errors, and icon 1 still shows nothing' — and it silently invalidated EVERY badge fix from 1.9.203 onward, including the Chrome-style generic globe and the generic PDF datasheet icon, which were being built correctly and then discarded in favour of a stale cached image. The cache key now includes BRIDGE_VERSION, so any new build rebuilds its badges. This also means my intermediate diagnoses (SVG cannot rasterise, the code is not running, duplicate sessions are the blocker) were wrong: the compositor was fine, the delivery was fine, the CACHE was serving history.
John Lauer · 2be3865 · 12d ago
Publish 1.9.213
John Lauer · 771a2bd · 12d ago
Remove the deliberate throw my own 1.9.210 revert left in the FIRST no-favicon path, which ran before the 1.9.211 generic-icon code and killed every faviconless badge with 'glyph render failed'. There were TWO competing no-favicon paths and I only converted the second one — so the Chrome-style globe and the PDF datasheet icon never had a chance to render. Both paths now build the generic icon (PDF sheet for a .pdf tab, grey globe otherwise) with the Adom mark inset.
John Lauer · 12103f3 · 12d ago
Publish 1.9.212
John Lauer · 7a5fca6 · 12d ago
ACTUALLY FIX the root cause I kept describing instead of doing (John: 'and yet its still not fixed?'). Everything — taskbar paint, park, session resolution — located windows by their '(session: <id>)' TITLE TAG, and Chrome's PDF viewer cannot carry that tag. So on a PDF window every lookup failed: no badge was ever painted no matter what the compositor produced, park returned park-failed (which is also why those PDFs jumped to the foreground unasked), and the reclaim janitor kept re-adopting the same unresolvable window as a NEW session every 90s — five of them by the end, all fighting over one taskbar button, each seeing only its own slice of the window's tabs (which is why one painter reported 'no favicon anywhere' while a sibling session held the real hanover.com favicon). Four changes: (1) park STORES the resolved window handle on the session (title-independent identity); (2) the taskbar overlay is addressed BY HWND when known, falling back to the title tag only when it is not; (3) sessionHwnd prefers the stored handle over a title lookup; (4) reclaim REFUSES to adopt a window whose OS window id already belongs to a live session, so duplicates stop being minted. Nine badge revisions could not fix this because the compositor was never the problem — the paint had no target.
John Lauer · a441bd1 · 12d ago
Publish 1.9.211
John Lauer · 4a97b89 · 12d ago
Generic badge icons, per John: mirror what Chrome does. A faviconless page now gets a GENERIC WEBPAGE icon (grey globe on a light plate, matching Chrome's own generic favicon) and a PDF gets a distinct GENERIC PDF icon (document sheet with folded corner + red PDF wordmark) — 'since that's common in our world cuz of pdf datasheets'. Both keep the design's structure: generic icon as the main image with the mini Adom mark inset in the lower-right, so every pup taskbar button looks alike. Replaces the rejected placeholders (a flat teal tile, then a full-size Adom logo). Also correcting my own earlier diagnosis recorded in 1.9.208: sharp DOES rasterise SVG in this build — the wiki's purple badge is itself an SVG favicon — so the 'SVG silently fails without librsvg' claim was wrong and these icons are drawn as SVG.
John Lauer · f382753 · 12d ago
Revert the invented badge placeholder (John: the teal tile was 'hideous', and a full-size Adom logo — 'no it shouldn't'). The badge design is the TAB'S OWN FAVICON with the mini Adom mark inset, full stop. When there is no usable favicon pup now paints NO overlay and logs why, instead of substituting a tile or a blown-up logo. Recorded in the code where it matters: the window that keeps hitting this branch DOES have tabs with real favicons (registration.hanover.com) — it reports 'no favicon anywhere in this window' only because duplicate sessions have split one window's tabs across several session objects, so the session painting that taskbar button can only see the PDF tabs. Fixing identity resolution (HWND instead of the title tag Chrome's PDF viewer cannot carry) removes the cause; inventing placeholder art was papering over it.
John Lauer · e0a4a52 · 12d ago
Publish 1.9.208
John Lauer · 4da8727 · 12d ago
Stop depending on SVG in the badge compositor — this is why John kept seeing 'only the Adom mark'. Every fallback I added was SVG (the plate, the glyph), and sharp requires librsvg to rasterise SVG and silently THROWS without it, so each one produced nothing while the Adom mark — a PNG — rendered fine. That also explains the earlier 'invisible plate'. Both fallbacks are now built from a raw pixel buffer via sharp create() plus PNG assets, with no SVG anywhere in the path, so a window with no usable favicon gets a solid Adom-teal tile as its MAIN image and the badge can never render as a lone corner mark. Also added explicit logging when the fallback itself fails, so a silent compositor failure can no longer hide.
John Lauer · e1cb20e · 12d ago
Publish 1.9.207
John Lauer · 8ec4373 · 12d ago
Find and fix the ACTUAL hole behind the bare first taskbar button, by tracing the decode path instead of guessing at pixels for a sixth time. The 1.9.205 glyph fallback only covered 'no favicon FOUND at all'. But when a favicon IS found and sharp cannot DECODE it — a BMP-encoded .ico (which icoLargestPng deliberately returns null for), or a malformed/exotic SVG — iconPng stayed null while src was truthy, so that fallback never ran, no favicon layer was pushed, and the composite came out as the Adom mark ALONE. That is exactly the 'no main overlay icon' John pointed at through five revisions, and no amount of colour or size tuning could have fixed it. Now ANY undecodable favicon falls back to the generic glyph, so a badge can never be mark-only. Also fixes the non-ico decode to resize 48/cover to match the 48px composite (it was still 22/inside from the old 32px layout, which left transparent margins and made badges look inconsistently sized).
John Lauer · efec3a9 · 12d ago
Publish 1.9.206
John Lauer · ed148a2 · 12d ago
Make the generic no-favicon glyph actually visible. My 1.9.205 glyph used a #123033 plate with 2.2px strokes — nearly the same colour as the dark taskbar and far too fine to survive the ~16px overlay slot — so the glyph vanished and only the teal corner mark read, which looked identical to the bare badge John was complaining about. This is the same dark-on-dark error made earlier today with the badge plate. Now a solid bright-teal plate with heavy dark strokes, so a faviconless window's button clearly shows a main icon plus the Adom mark.
John Lauer · af08478 · 12d ago
Publish 1.9.205
John Lauer · 1793ac0 · 12d ago
A badge must ALWAYS have a main icon (John: 'why does the first taskbar icon still not have a main overlay icon?'). The no-favicon path bailed out and left the button showing the bare Adom mark or nothing at all — which is what he kept seeing on the local-PDF/globe-page window. Chrome itself never shows nothing there; a faviconless page gets its generic globe. Now: (a) if the visible tab has no favicon, try the SIBLING tabs in that same window, since a multi-tab window usually has at least one real icon and that is far more informative than a placeholder; (b) if no tab has one, render a generic teal-on-dark globe glyph. Either way the badge keeps its structure — main image with the Adom mark inset flush in the lower-right — so every pup taskbar button looks alike instead of some being bare.
John Lauer · 0bb579c · 12d ago
Publish 1.9.204
John Lauer · 7c70526 · 12d ago
Fix the overlay badges properly — three separate root causes, all mine. (1) WRONG TAB: pup picked the favicon source from its OWN session.activeTabId bookkeeping, which drifts out of sync the moment the user switches tabs (the same drift that made the unstick misfire). John asked for 'the favicon from the chrome tab that's in the foreground' — new resolveVisibleTab() ASKS CHROME (exactly one tab per window reports visibilityState 'visible') and re-syncs the bookkeeping. (2) .ICO NEVER DECODED: sharp cannot read an .ico container, and the compositor only built a favicon layer when the source was NOT .ico — so every site serving favicon.ico rendered as the Adom mark ALONE with no site favicon (John's 3rd taskbar button). New icoLargestPng() parses the ICO directory, takes the largest entry and uses its embedded PNG, and the favicon layer now composites the DECODED bytes instead of re-decoding the raw container. (3) file:// FAVICONS UNFETCHABLE: the bridge fetches favicons over HTTP, but a local-file page's favicon href is itself file://, which Node's fetch cannot retrieve — so local-file windows lost their icon entirely and rendered NO overlay (John's 1st taskbar button, whose tab plainly had a favicon). The bridge runs on the same machine, so it now reads those off disk. Also composites at 48px like the working adom-browser-extension badge builder (compositing at 32 and letting Windows shrink to the ~16px slot turned both images to mush) with the Adom mark at a fixed 3/8 proportion so it is identical on every window.
John Lauer · a40f510 · 12d ago
Publish 1.9.202
John Lauer · b953914 · 12d ago
Record the AUDIT YOURSELF procedure (icons are MANDATORY) + fix the three regressions that audit found. John, twice: 'when i say audit your windows, i mean check your icons too'. SKILL.md and pup-bridge-test now carry a seven-section self-audit: sessions, ICONS/TASKBAR (per-window overlay + AUMID key count + an ON-GLASS taskbar screenshot, because the API reports a badge applied while it is invisible), OS ground truth, health, log (explicitly surfacing park-failed and reclaim-adopted lines), Chrome footprint split pup-vs-user, and cleanup state — with the rule to state what is WRONG first and name your own regressions. Fixes from running it: (1) recovered windows came back with NO badge — under minimalTouch the overlay applies only on open/navigate and the re-stamp sweeps are gated off, so every bridge restart left surviving windows bare; recovery now re-wears it. (2) RECLAIMED/adopted windows were never badged either; they are now. (3) park-failed was logged and then ignored, leaving a window wherever Chrome put it (possibly on the user's screen, the one thing park exists to prevent) — it now retries before falling back to the AD verb.
John Lauer · a7f0fa3 · 12d ago
Publish 1.9.201
John Lauer · 3100a22 · 12d ago
Show the foreground REASON on screen, and fix why a PDF window came to the front unasked. (1) John: 'every time you open a window in the foreground and you force the AI to give you a reason you should flash that on screen as an AD caption for 1 second' — every GRANTED foreground now captions the caller's justification on screen for ~1s, so the gate's audit trail is visible in real time instead of only in lastForeground. Deliberately not gated by minimalTouch: it is a user-protection signal, not cosmetic. (2) Root-caused his 5-PDF window arriving in the foreground although NO ai requested it and pup's own log said 'backgrounded': Chrome's PDF viewer activates the window on load, and BOTH of pup's safety nets were dead — my 1.9.186 minimal-touch change had dropped the park re-assert as 'redundant' (it is the net that catches exactly this), and reassertBottomAfterNav relied solely on AD's desktop_set_window_state, a verb absent on this desktop, so it silently no-opped. The re-assert is restored in minimal-touch, reassertBottomAfterNav now bottoms the window in-bridge via SetWindowPos with the AD verb only as fallback, and navigate re-asserts too. All paths still respect userIsUsingWindow so a window the user chose to have up is never yanked.
John Lauer · aaed8eb · 12d ago
Publish 1.9.200
John Lauer · 6336c51 · 13d ago
Record the native-input hard rule in the SUB-SKILL and BOTH maintainer skills, not just the main SKILL.md (John: 'have you recorded this in skills and dev skills?' — I had only done the main one). pup-vendor-login now opens its real-cursor/FedCM section with the warn-and-wait protocol, since that skill is what teaches the native-click path and is therefore the highest-risk place to omit it. pup-bridge-dev records three more lessons paid for today: autonomous machinery needs direct evidence + two consecutive observations + a hard cap or it becomes the bug (the tab reconciler destroyed live windows; the unstick misfired and clicked the user's screen every 30s); never invent UI Adom already has (the canonical AI cursor is the extension's cursorScript, a byte-faithful copy of HD's AI_CURSOR_JS); and prove taskbar/visual work ON GLASS because the badge API reported success on an invisible badge through three revisions. pup-bridge-test records that a TEST may never fire native OS input without warning, since every scenario runs on the user's real machine while he works, and that a green API result is not a passing test.
John Lauer · 4711dd1 · 13d ago
Publish 1.9.199
John Lauer · e55040e · 13d ago
HARD RULE recorded + enforced: never send native OS input without warning the user first. What happened: an agent (me) fired AD desktop_type at a pup window without any warning; the user — who runs many AI threads and alt-tabs away from a foregrounded window almost immediately, every time — had already returned to their own work, so the text landed in THEIR window. I then misdiagnosed it as 'Windows failed to foreground the window', blaming the OS for a human simply taking their focus back because nobody asked them not to. Native input (desktop_click/type/press_key) does NOT target a window; it goes to whatever has OS focus at that instant. Added: (1) a prominent hard-rule section in the pup SKILL.md so EVERY user's agent learns it — toast + caption + an explicit 'don't type or click until I say done' + wait-for-foreground + re-verify before EACH event + caption when done, and prefer CDP input which needs no foreground at all; (2) ensureSafeForNativeInput() in the bridge, which warns, waits for the window to genuinely be foreground, and re-verifies at dispatch time, refusing rather than dispatching blind; (3) a browser_describe entry so the warning appears in the verb catalog, not only in the skill file.
John Lauer · 34f0b59 · 13d ago
Publish 1.9.198
John Lauer · 76e4bc7 · 13d ago
Fix the Bills thread's second-pass findings + the root cause of the input-blocking freeze. ORPHANED CURSOR: pre-1.9.196 pup drew '#__pupCursor' while the HD port draws '#__adom_cursor__', and nothing removed the legacy element — plus the cursor is injected per-DOCUMENT, so interacting with a field inside an iframe (Chase's iframe#logonbox) left that frame's instance stranded forever. showPupCursor now sweeps EVERY frame and removes BOTH ids before drawing. STUCK OCCLUSION (the big one): Chrome's Win32 native occlusion detector can latch a window as occluded and never re-evaluate; that window reports visibilityState hidden while FRONTMOST and Chrome withholds both paint AND input — measured directly, browser_eval DOM writes land while CDP key events vanish, which is why typing silently failed and scrolling did nothing. pup parks windows at z-bottom by design so it hits this constantly. --disable-features=CalculateNativeWinOcclusion is now a DEFAULT launch flag; it disables only the occlusion calculation, so ordinary background throttling (and the issue-#14 battery saving) is untouched. Also: new browser_hover verb (pointer-over without clicking, reveals tooltips), and browser_input_dispatch {type:'move'} now glides the drawn cursor instead of only clicks moving it.
John Lauer · 8e39742 · 13d ago
Publish 1.9.197
John Lauer · 58bbf63 · 13d ago
Fix 6 real-session bugs reported by the Bills thread driving Chase Connect's login. BUG 1: browser_press_key ignored modifiers entirely (keyboard.press only), so Ctrl+A typed a literal 'a' and every chord was unusable — modifiers are now held down around the press and released in reverse (aliases ctrl/cmd/command/option accepted; unknown ones rejected instead of silently dropped). BUG 2: browser_click accepted clickCount and silently ignored it — clickCount/button/delay are now passed through, so triple-click selects. BUG 3 (top priority): browser_type always APPENDED with no way to clear, which with bugs 1+2 left NO way to replace a field value (caller got 'adominc1adominc1' and fell back to 20 Backspaces) — new clear:true empties first using real select-all+Delete key events, which React-controlled inputs honour unlike setting .value. BUG 4: owner was never written to the session file, so every bridge restart erased it and list_windows reported owner:null for everything, silently disabling session_owned_by_another_thread — owner now persists and is restored on recovery. GAP 5: browser_eval only ran in the top frame, making iframe login forms reachable only via same-origin contentDocument hacks and cross-origin ones unreachable — new frame:'<name|url-substring|index>' selects the context, and a miss returns frame_not_found WITH the tab's frame list. GAP 6: new browser_maximize verb (CDP Browser.setWindowBounds, no AD verb dependency, does not foreground) replaces the window.resizeTo workaround. All documented in browser_describe.
John Lauer · 8698de3 · 13d ago
Publish 1.9.196
John Lauer · 93cdf18 · 13d ago
Cursor: port HD's AI cursor byte-faithful + add the Adom mark (John: 'we should have a consistent cursor in all of our apps, hd's looks amazing' + 'i can't even read the indicator text ... and i don't see the adom logo'). pup's cursor was a hand-drawn fourth variant with NO intent label at all (reasoning went only to a bottom-of-screen caption) and no brand mark, just a 7px letter A. It is now HD's cursor ported verbatim from the canonical source (adom-browser-extension/extension/src/cdp.js cursorScript, itself an exact match of HD's hd-control AI_CURSOR_JS): teal arrow with glow, PULSING teal ring at the tip, and the readable monospace intent label that flips sides near the screen edge. Added for pup per John: the real Adom mark (icons/pup-overlay-32.png inlined) inside the label chip so the user can tell ADOM is driving and not another AI. browser_click now composes HD-style narration from intent/reason/target so the label never reads a bare Adom.
John Lauer · b6ba9d7 · 13d ago
Publish 1.9.195
John Lauer · f3e6ae2 · 13d ago
Pre-emptively refuse guaranteed-broken file:// URLs (John: a thread opened file:///home/adom/project/.../helium-lab.html and pup said NOTHING, just rendered Chrome's ERR_FILE_NOT_FOUND). urlLint now handles file:// — a CONTAINER-shaped path (/home,/root,/workspace,/tmp,/usr,/var,/opt,/mnt) can never exist on the user's desktop, so it is refused with a reprimand telling the caller to serve it via the container proxy URL or push_file it to the desktop first; any other file:// path gets an EXISTENCE PRE-FLIGHT (the bridge runs on the desktop, so it just checks) and is refused with the real reason if missing. browser_open_window and browser_navigate both refuse BEFORE spending a window, instead of opening one that renders an error page the calling AI has to guess at.
John Lauer · edd776d · 13d ago
Publish 1.9.194
John Lauer · 3111689 · 13d ago
Badge: the Adom favicon sits WHOLE and FLUSH in the lower-right corner (John's correction). Dropped the dark disc/container that made the mark look inset-inside-a-thing; the Adom favicon itself now occupies the lower-right quadrant, flush to both edges, directly over the page's own full-bleed favicon.
John Lauer · 63b5490 · 13d ago
Publish 1.9.193
John Lauer · 36de8f5 · 13d ago
Badge design per John: no plate. The badge IS the page's own favicon at full bleed with the Adom mark inset into its lower-right corner — the favicon keeps the exact look it was designed with instead of sitting on a tinted card (the light plate from 1.9.192 read as ugly, and the dark plate before it hid dark favicons like GitHub's). The corner mark keeps a small dark disc purely for legibility over busy or light favicon artwork at taskbar size.
John Lauer · 5afef89 · 13d ago
Publish 1.9.192
John Lauer · 9ac029d · 13d ago
Composite badge: light plate so dark favicons are visible. Proven on glass in 1.9.191 that the wiki badge rendered but the GitHub badge did not, despite applying successfully — GitHub's favicon is a near-black mark and it composited dark-on-dark against the dark plate. Favicons are authored for the browser TAB STRIP, which is light, so a large share of them are dark on transparency. The plate is now light (#eef4f5), which renders both dark and light favicons; the Adom mark keeps its dark disc in the lower-right so brand contrast holds either way.
John Lauer · c20acfb · 13d ago
Publish 1.9.191
John Lauer · a117d62 · 13d ago