Pup - the AI's own browser

@@ -1,10 +1,10 @@ {   "manifest_version": 1,   "name": "puppeteer",-  "version": "1.9.198",-  "url": "https://wiki.adom.inc/download/adom/adom-desktop-puppeteer-bridge/1.9.198/adom-bridge-puppeteer-v1.9.198.zip",-  "sha256": "2d894a7dee8ddf3cee2bcd935d94e73a66440e434b5c782a7afd55739e97a61b",-  "size": 431523,+  "version": "1.9.199",+  "url": "https://wiki.adom.inc/download/adom/adom-desktop-puppeteer-bridge/1.9.199/adom-bridge-puppeteer-v1.9.199.zip",+  "sha256": "3021bdf03073ae6c5328017ebd098f08c7a5ab3e142383b9af15e1975d56c9e9",+  "size": 432928,   "verbPrefixes": [     "browser_",     "desktop_recorder_",

@@ -1 +1 @@-1.9.198+1.9.199

@@ -2,7 +2,7 @@   "manifest_version": 1,   "name": "puppeteer",   "displayName": "Puppeteer (Chrome for Testing + Native)",-  "version": "1.9.198",+  "version": "1.9.199",   "description": "Drives a browser ALREADY on the machine \u2014 installed Chrome, else Microsoft Edge (every Windows PC ships with Edge) \u2014 launched with a FRESH isolated profile, so the common case has NO ~150 MB Chrome-for-Testing download and it never touches the user's real tabs/logins. Binds its local server to LOOPBACK (127.0.0.1) only, so no Windows Firewall 'allow Node through the firewall' prompt ever pops. Chrome for Testing is fetched only as a LAST RESORT if the machine has no Chromium browser at all. SPAWN-VERIFIED fallback: pup tries each detected browser (Chrome \u2192 Edge \u2192 CfT) and actually launches it, falling through to the next if one won't start (and handling browsers like Edge that relaunch onto another PID), so a corrupt/partial install never hard-fails an open. The browser that launches is cached as the default (persisted across restarts) so it isn't re-probed each time. On spawn the bridge SELF-WARMS: detects browsers + disk, and only if the box has no Chrome/Edge and no cached CfT does it background-fetch CfT so the first open is instant. browser_use pins a specific browser (chrome | edge | cft | auto); {browser:'chrome',install:true} downloads & installs REAL Google Chrome (user-scoped, no admin) on an Edge-only box. browser_readiness (READ-ONLY) reports which browser will be used + the full candidate list + the default + free disk. Every failure returns a specific errorCode + _hint (chrome_install_no_disk, chrome_download_network, chrome_install_permission, chrome_stable_not_detected, \u2026). Launches with FULL browser permissions by default for full debug (strictPermissions:true opts out). Headful automation: open/close windows+tabs, navigate, screenshot, eval JS, record. pup is a GENERIC, non-profile browser only \u2014 every session is a fresh isolated Chrome/Edge process, never the user's real signed-in browser; profile defaults to sessionId. To drive the user's REAL Chrome/Edge (their profile, cookies, SSO), use the Adom browser EXTENSION's nbrowser_* verbs \u2014 a separate bridge, not pup's job. macOS (v1.9.74+): full parity with mac-native mechanics \u2014 background-by-default is the app-level hide (windows stay drivable/screenshottable while hidden), raise/lower map to unhide+activate/hide, browser_alert_window posts a native toast whose click focuses the window, browser_use {browser:'chrome',install:true} installs Chrome from the official dmg, and taskbar identity/badging remains Windows-only.",   "homepage": "https://wiki.adom.inc/adom/adom-desktop",   "author": "Adom Inc.",

@@ -1837,6 +1837,54 @@ async function showPupCursor(page, x, y, { click = true, label = null } = {}) {   } catch (e) {} } // v1.9.167 (John's assisted-foreground protocol) — a general OS caption (what/why narration).+// v1.9.199 (John, 2026-07-29, after I typed into HIS window): NATIVE OS INPUT GUARD.+//+// desktop_click / desktop_type / desktop_press_key do NOT target a window — they go to WHATEVER+// WINDOW HAS OS FOCUS at that instant. This user runs many AI threads and alt-tabs away from a+// foregrounded window almost immediately, every time. An agent (me) fired desktop_type without ever+// warning him, he had already alt-tabbed back to his own work, and the text landed in HIS window.+// Then I misdiagnosed it as "Windows failed to foreground the window" — blaming the OS for a human+// simply taking their focus back, because nobody had asked them not to.+//+// This guard makes that impossible by construction. Every native-input path MUST call it immediately+// before EACH event (not once per sequence — focus can change between two keystrokes, and half a+// password in the wrong window is real harm). It: warns via AD toast + on-screen caption, tells the+// user explicitly to stop typing, waits until the target window is genuinely the OS foreground, and+// re-verifies at dispatch time. Returns false if we never got a safe window — callers MUST NOT+// dispatch on false.+async function ensureSafeForNativeInput(sessionId, { reason = 'drive the window directly', warn = true } = {}) {+  if (process.platform !== 'win32') return true;+  const hwnd = await sessionHwnd(sessionId);+  if (!hwnd) {+    console.log(`[native-guard] "${sessionId}": no hwnd — REFUSING native input (cannot prove where it would land)`);+    return false;+  }+  const fg = osGetForegroundWindow();+  if (String(fg) === String(hwnd)) return true;   // already ours; safe to dispatch++  if (!warn) {+    console.log(`[native-guard] "${sessionId}": window is NOT foreground — REFUSING native input`);+    return false;+  }+  // Warn BEFORE asking for the screen, and say the quiet part out loud: hands off the keyboard.+  const msg = `Adom needs your "${sessionId}" window in front to ${reason}. Please DON'T type or click until I say done.`;+  try { await chrome.adCommand('notify_user', { title: 'Adom needs the keyboard', body: msg }, { timeoutMs: 5000 }); } catch (e) {}+  try { await pupCaption(msg, 15000, 'pup-native-guard'); } catch (e) {}+  const ready = await waitForUserReady(sessionId, hwnd, { maxMs: 60000 });+  if (!ready) {+    try { await pupCaption('Adom skipped that step — the window never came to the front, so nothing was typed.', 6000, 'pup-native-guard'); } catch (e) {}+    console.log(`[native-guard] "${sessionId}": user never brought the window forward — REFUSED native input`);+    return false;+  }+  // Re-verify at the last possible moment.+  const fg2 = osGetForegroundWindow();+  if (String(fg2) !== String(hwnd)) {+    console.log(`[native-guard] "${sessionId}": focus moved again at dispatch time — REFUSED native input`);+    return false;+  }+  return true;+}+ async function pupCaption(text, expiresInMs = 8000, id = 'pup-assist') {   try { await chrome.adCommand('desktop_caption', { text, expiresInMs, id, position: 'bottom', size: 'medium' }, { timeoutMs: 4000 }); } catch (e) {} }@@ -7371,6 +7419,7 @@ const server = http.createServer(async (req, res) => {             V('browser_assisted_click', 'RESPECTFUL real-cursor click for a surface pup CDP cannot drive (a NATIVE OS dialog, or a cross-origin FedCM "Sign in with Google" account chooser that rejects every in-page click). Do NOT hand-roll raise+desktop_click for these — use THIS. It: captions the reason + toasts BEFORE touching the screen; foregrounds; then WAITS, detecting if the user alt-tabbed away and holding (captioning "Adom is waiting — click back when ready") until the user is actually on the window; then runs the TIGHT real-cursor hover->click that FedCM accepts (interleaving any screenshot/eval between raise and click makes the click silently no-op); captions each step. Times out as user_not_ready clicking NOTHING if they never return. First take a desktop_screenshot_window of the session to read the target\'s WINDOW-pixel x,y.', { sessionId: 'required str', x: 'required int — WINDOW-image pixel of the target (from desktop_screenshot_window)', y: 'required int', reason: 'required str — WHY you must click (shown to the user as a caption)', label: 'required str — WHAT you are clicking (e.g. "your Google account")', maxWaitMs: 'optional int — how long to wait for the user (default 120000)' }, 130, { sessionId: 'adsk', x: 900, y: 379, reason: 'clicking your Google account to finish signing into Autodesk', label: 'your Google account' }),             V('browser_type', 'Type text into a field. Pass clear:true to REPLACE the existing value instead of appending (this is what you want on any prefilled/React form — it selects-all then deletes with real key events, which React-controlled inputs honour, unlike setting .value).', { sessionId: 'required str', text: 'required str', selector: 'optional str', clear: 'optional bool — empty the field first (replace, not append)' }, 20, { sessionId: 'work', selector: '#user', text: 'adominc1', clear: true }),             V('browser_press_key', 'Press a key, optionally with modifier chords held (Ctrl+A, Ctrl+V, Shift+Tab).', { sessionId: 'required str', key: 'required str', modifiers: 'optional array — any of ["Control","Shift","Alt","Meta"] (aliases ctrl/cmd/command/option accepted); held down around the key press' }, 15, { sessionId: 'work', key: 'a', modifiers: ['Control'] }),+            V('browser_NATIVE_INPUT_WARNING', '⛔ READ BEFORE USING AD\'s desktop_click / desktop_type / desktop_press_key ON A PUP WINDOW. Native OS input does NOT target a window — it goes to WHATEVER WINDOW HAS OS FOCUS at that instant. This user runs many AI threads and alt-tabs away from a foregrounded window almost immediately. If you fire native input without warning them, your keystrokes land in THEIR document (this happened: a username was typed into the user\'s own window). REQUIRED: notify_user toast + desktop_caption FIRST, tell them explicitly "don\'t type or click until I say done", wait until the window is genuinely foreground, re-verify immediately before EACH event, and caption when done. browser_assisted_click already does all of this — use it. And ALWAYS prefer CDP input (browser_click/browser_type/browser_hover): it needs no foreground and disturbs nobody. A native verb returning ok:true means the event was dispatched to the OS, NOT that it reached your window.', {}, 0, {}),             V('browser_hover', 'Move the pointer over a target WITHOUT clicking (reveals tooltips / hover menus). Glides the visible Adom cursor there too.', { sessionId: 'required str', selector: 'optional str', x: 'optional int', y: 'optional int', intent: 'optional str — narrated on the cursor label', steps: 'optional int (default 8)' }, 15, { sessionId: 'work', selector: '#menu' }),             V('browser_maximize', 'Maximize (or restore/fullscreen/minimize) a pup window via CDP — no AD desktop verb required. Does NOT foreground it.', { sessionId: 'required str', state: 'optional "maximized"|"normal"|"minimized"|"fullscreen" (default maximized)' }, 15, { sessionId: 'work' }),             V('browser_screenshot', 'Screenshot the page (PNG; fullPage optional). NOTE: fullPage only covers DOCUMENT-level scroll — on SPAs that scroll inside a nested container it returns just the viewport + a nestedScroll hint; use browser_screenshot_element or browser_scroll for those.', { sessionId: 'required str', fullPage: 'optional bool' }, 30, { sessionId: 'work' }),