app
Pup - the AI's own browser
Public Made by Adomby adom
pup gives your AI a real browser on your desktop that it drives completely, in the background, signed in as you: windows that never jump in front of your work, one shared profile that learns your logins, and every window labeled on the taskbar with its AI thread's name.
← Commit history
skills: pup-vendor-login gains the BACKGROUND Google-SSO click recipe (classic full-page accountchooser is fully background-drivable via elementFromPoint+full-event-sequence; only the FedCM One-Tap bubble needs a real cursor - proven signing John into Autodesk 2026-08-22); pup-taskbar-states records the owner-loss->category downgrade CLOSED (v2.0.277 tileOwnerName) and the remaining crash-on-shared-profile bare-icon root
2 files changed
+57−13
skills/pup-taskbar-states/SKILL.md+21−2@@ -80,5 +80,24 @@ it has not applied. - Structural since: the thread AUMID folds in a hash of the tile's CONTENT (v2.0.271 — Windows caches a button's icon PER AUMID, so only a virgin AUMID can change art), and the button rebuild waits for the icon registration to land first (v2.0.275).-- Genuinely OPEN: recovered windows that lose their `owner` silently downgrade to a category- button — issue #70.+- Owner-loss downgrade to a category glyph (part of issue #70): **CLOSED v2.0.277.** A window that+ lost its owning ai-thread (recovered / reclaimed popup) used to fall through to a category AUMID+ (`Adom.Pup.<sid>.<cat>`) and render the pup category icon next to real thread tiles — John's+ "inconsistent taskbar icons" (2026-08-22). `tileOwnerName()` now DERIVES a stable readable tile+ name from the sessionId (cleaned) when no owner is present and freezes it on the session, so in+ tiles mode EVERY window wears a name tile, never a category glyph. Verified: an owner-less Fusion+ window flipped from `Adom.Pup.ai-fusion-dashboard.app` to `Adom.Pup.thread.ai-fusion-dashboard-…`+ and rendered an "ai fusion dashboard" tile.+- Still GENUINELY OPEN (the harder half of #70): a window that **crashes on the shared `adom-you`+ profile** loses its birth-hwnd, and because EVERY pup window opens at the identical rect (work+ area, 10px inset), bounds-resolution finds N same-geometry windows and cannot disambiguate it+ (`[hwnd] "<sid>" NOT resolved — N windows on this profile, N wiggle-matches; leaving bare ++ BACKING OFF`). With `titleTag` off the window carries no `(ai-thread: <sid>)` suffix either, so the+ stamp's `titleContains` fallback also fails and the button sits BARE (Chrome's default icon) until+ the window is closed+reopened (a fresh off-screen birth captures an unambiguous hwnd). The three+ candidate cures each have a real cost — transiently inject the title tag via `setTitleDecor` (it+ is exactly what `titleTag:on` does, but it mangles the title John wants clean), give each window a+ 1-2px unique birth offset so bounds are always distinct (changes placement, touches the+ just-rewritten off-screen-birth path), or an OS wiggle (foreground/CPU risk) — so it needs a+ deliberate decision, not a rushed patch. Backoff makes it self-limiting (5 misses → 30-min+ retries), and a reopen fixes it immediately.
skills/pup-vendor-login/SKILL.md+36−11@@ -40,23 +40,48 @@ persists on disk. Email+password works too but needs a credential you do not hav and the reclaim sweep adopts it as its **own session** named like `adom-you-win<rand>` at `signin.autodesk.com`. Run `pup_list_windows` and look for the new session; drive THAT one. -4. **Click the account — Google's "Sign in with Google" chooser (FedCM) REJECTS every in-page and- in-app click.** Proven 2026-07-28 signing John into Autodesk: DOM `.click()`, a full synthetic- pointer-event sequence, `page.mouse.click` (pup_click x/y), AND UIA-by-name- (`desktop_ui_click`) ALL failed — the account lives in a cross-origin FedCM iframe hardened- against automation. **The ONLY thing that works is a REAL OS cursor HOVER-then-CLICK** via AD's- desktop verbs (this is the breakthrough):+4. **Click the account — FIRST figure out WHICH chooser you got, because they are two different+ things with two different answers (corrected 2026-08-22, this is the important update):**++ **(A) The CLASSIC FULL-PAGE OAuth chooser — the common case, and it is FULLY BACKGROUND-DRIVABLE,+ NO foreground, NO real cursor.** After "Continue with Google" the WINDOW NAVIGATES (a real+ top-level navigation) to `accounts.google.com/v3/signin/accountchooser?...flowName=GeneralOAuthFlow...`+ — an ordinary Google web page, same-origin to Google, NOT an iframe. This accepts a properly+ dispatched synthetic click. **Proven 2026-08-22 signing John into Autodesk entirely in the+ background** (window never left z-bottom). The recipe that works — and the two traps that make a+ naive click fail:+ ```+ # find the account ROW (a <li>) whose text is JUST the account, not the whole list, get its center px:+ pup_eval {sessionId, expr:"(function(){var a=[...document.querySelectorAll('*')].filter(e=>/YOUR_EMAIL/i.test(e.innerText||'')&&!/another account/i.test(e.innerText||''));var t=a.sort((x,y)=>x.innerText.length-y.innerText.length)[0];var c=t.closest('li')||t;var r=c.getBoundingClientRect();return JSON.stringify({x:Math.round(r.x+r.width/2),y:Math.round(r.y+r.height/2)});})()"}+ # then dispatch a FULL native event sequence on the element AT THAT POINT (elementFromPoint), not on the div you matched:+ pup_eval {sessionId, expr:"(function(){var x=X,y=Y;var el=document.elementFromPoint(x,y);var t=el.closest('[jsaction],li')||el;var o={bubbles:true,cancelable:true,view:window,clientX:x,clientY:y,button:0};['pointerover','pointerenter','pointerdown','mousedown','focus','pointerup','mouseup','click'].forEach(function(n){try{t.dispatchEvent(new (n.slice(0,7)==='pointer'?PointerEvent:n==='focus'?FocusEvent:MouseEvent)(n,o));}catch(e){t.dispatchEvent(new MouseEvent(n==='pointerdown'?'mousedown':n,o));}});return 'clicked';})()"}+ ```+ - **Trap 1: a bare `element.click()` or `pup_click {x,y}` (CDP mouse) does NOTHING here** — Google's+ rows are wired to a jsaction handler that wants the full pointer+mouse sequence, not a lone click.+ Measured 2026-08-22: `.click()` on the matched wrapper div = no-op; `pup_click` at the coords =+ no-op; the FULL dispatched sequence above = navigated straight to `profile.autodesk.com`.+ - **Trap 2: match the tight ROW, dispatch on `elementFromPoint`.** The element whose `innerText`+ contains the email is often a big container that ALSO contains "Use another account"; clicking it+ misses. Take the shortest-text match, resolve its `<li>`, dispatch on whatever is actually topmost+ at its center pixel.++ **(B) The FedCM One-Tap BUBBLE — the hardened case, needs a REAL OS cursor.** If "Continue with+ Google" shows the account WITHOUT a full navigation (a small "Sign in with Google" bubble/One-Tap,+ the URL stays on the vendor page, the account sits in a cross-origin FedCM iframe), then every+ programmatic click is rejected — DOM `.click()`, the synthetic sequence, `pup_click`, AND+ `desktop_ui_click` all fail (proven 2026-07-28). Only a real OS cursor HOVER-then-CLICK advances it: ``` pup_raise_os_window {sessionId, foregroundReason:"<the user's words>"} # FedCM needs the window focused desktop_screenshot_window {hwnd} # get shotId + coordMap desktop_hover {space:"image", shotId, x:<account px>, y:<account px>} # MOVE the real cursor FIRST — this is the key desktop_click {space:"image", shotId, x:<same>, y:<same>} # then the real SendInput click ```- The HOVER before the click is what satisfies FedCM's genuine-user-activation check — a bare- desktop_click with NO preceding hover did NOT advance; hover-then-click did. Aim by the- SCREENSHOT pixel (the account is in an iframe, so top-frame getBoundingClientRect can't locate- it). If even this is refused, the window is foregrounded with the account visible — ask the USER- to tap it once.+ The HOVER before the click satisfies FedCM's genuine-user-activation check. Aim by the SCREENSHOT+ pixel (iframe, so top-frame getBoundingClientRect can't locate it). Prefer `pup_assisted_click`+ (below) over hand-rolling this. If even that is refused, ask the USER to tap the visible account once.++ **So: try path (A) first — it is the common shape and needs nobody's screen.** Only fall to the+ real-cursor path (B) when there was no navigation and the account is in a FedCM bubble. 5. **Confirm the redirect landed** by SCREENSHOT, not eval — the eval often reads one beat before the redirect completes and reports the stale chooser URL while the screenshot already shows the