app
Pup - the AI's own browser
Public Made by Adomby adom
pup gives your AI a real browser on your desktop that it drives completely, in the background, signed in as you: windows that never jump in front of your work, one shared profile that learns your logins, and every window labeled on the taskbar with its AI thread's name.
← Commit history
v2.0.397 (insiders): First-run fix (arav-rog): npm install can no longer fail on puppeteer's postinstall Chrome download (.puppeteerrc.cjs skipDownload; pup provisions CfT itself), and every cache folder without an executable is pruned at warmup. pup-bridge#92: waits on a human answer blocked_on_user (consent registry, 45s in-verb / 120s toast, re-attach on re-call; assisted click; login 2FA/captcha walls). pup-bridge#93: desktop_record_* labelled as ab-core's, served by pup until adom-bridge#143.
7 files changed
+263−61
SKILL.md+6@@ -38,6 +38,12 @@ dropdown. Backed by the OS keychain (encrypted at rest); passwords are never log bulk, returned ONCE, and it must never be logged or echoed into another verb. A relayed (cloud container) call is refused with `relay_forbidden`: from a cloud thread you do not need the secret, call `pup_login {submit:true, submitReason}` and pup signs the site in inside its own window.+ **Waiting on the click is never a timeout** (v2.0.397, ab SDK rule): if the user has not clicked+ within 45s the verb answers `errorCode: blocked_on_user, stillRunning: true, consentId` while the+ toast stays up (120s total). Call the SAME verb with the SAME args to re-attach and collect the+ decision (no second toast), or watch `pup_readiness {}` `active[]`. `pup_fill_card` behaves the+ same; `pup_assisted_click` and a `pup_login` submit that lands on a 2FA/captcha wall answer+ `blocked_on_user` with `stillRunning: false` (the human must act, then you re-issue the verb). (v2.0.374: the gate rides ab's forge-proof `X-Adom-Verified-Origin` header — only a VERIFIED local transport passes; a missing/bridge origin is refused with `origin_unverified`.)
package.json+1−1@@ -1,6 +1,6 @@ { "slug": "pup-bridge",- "version": "2.0.396",+ "version": "2.0.397", "type": "app", "description": "The pup SKILL PACK \u2014 the AI skills that teach a cloud/container AI how to drive the Adom Desktop Puppeteer (pup) bridge (browser_* verbs): the everyday `pup` skill plus focused sub-skills for windows/sessions/tabs, screenshots+recording, browsers+Chrome install, and driving the Adom wiki. Installs into ~/.claude/skills + ~/.codex/skills. DOCS ONLY \u2014 this is NOT the bridge runtime. The bridge itself (the Node program that drives the browser) runs on the DESKTOP: Adom Desktop bundles it and auto-installs/updates it, and it's published separately on the Releases tab as adom-bridge-puppeteer-v<ver>.zip. The pkg version is kept in lockstep with the Release version so the page's max version row always carries BOTH a pkg tarball (skills) and a runtime zip (bridge) \u2014 a skills-only or runtime-only bump on the top row breaks constraint installs. TL;DR: pkg install = the skills for your AI; the bridge = the Release zip, which Adom Desktop manages for you.", "keywords": [
src/BRIDGE_VERSION+1−1@@ -1 +1 @@-2.0.396+2.0.397
src/bridge.json+1−1@@ -2,7 +2,7 @@ "manifest_version": 1, "name": "puppeteer", "displayName": "Puppeteer (Chrome for Testing + Native)",- "version": "2.0.396",+ "version": "2.0.397", "description": "Drives a browser ALREADY on the machine \u2014 installed Chrome, else Microsoft Edge (every Windows PC ships with Edge) \u2014 launched with a FRESH isolated profile, so the common case has NO ~150 MB Chrome-for-Testing download and it never touches the user's real tabs/logins. Binds its local server to LOOPBACK (127.0.0.1) only, so no Windows Firewall 'allow Node through the firewall' prompt ever pops. Chrome for Testing is fetched only as a LAST RESORT if the machine has no Chromium browser at all. SPAWN-VERIFIED fallback: pup tries each detected browser (Chrome \u2192 Edge \u2192 CfT) and actually launches it, falling through to the next if one won't start (and handling browsers like Edge that relaunch onto another PID), so a corrupt/partial install never hard-fails an open. The browser that launches is cached as the default (persisted across restarts) so it isn't re-probed each time. On spawn the bridge SELF-WARMS: detects browsers + disk, and only if the box has no Chrome/Edge and no cached CfT does it background-fetch CfT so the first open is instant. pup_use pins a specific browser (chrome | edge | cft | auto); {browser:'chrome',install:true} downloads & installs REAL Google Chrome (user-scoped, no admin) on an Edge-only box. pup_readiness (READ-ONLY) reports which browser will be used + the full candidate list + the default + free disk. Every failure returns a specific errorCode + _hint (chrome_install_no_disk, chrome_download_network, chrome_install_permission, chrome_stable_not_detected, \u2026). Launches with FULL browser permissions by default for full debug (strictPermissions:true opts out). Headful automation: open/close windows+tabs, navigate, screenshot, eval JS, record. pup is a GENERIC, non-profile browser only \u2014 every session is a fresh isolated Chrome/Edge process, never the user's real signed-in browser; profile defaults to sessionId. To drive the user's REAL Chrome/Edge (their profile, cookies, SSO), use the Adom browser EXTENSION's nbrowser_* verbs \u2014 a separate bridge, not pup's job. macOS (v1.9.74+): full parity with mac-native mechanics \u2014 background-by-default is the app-level hide (windows stay drivable/screenshottable while hidden), raise/lower map to unhide+activate/hide, pup_alert_window posts a native toast whose click focuses the window, pup_use {browser:'chrome',install:true} installs Chrome from the official dmg, and taskbar identity/badging remains Windows-only.", "homepage": "https://wiki.adom.inc/adom/adom-bridge", "author": "Adom Inc.",
src/chrome.js+44@@ -380,6 +380,7 @@ function installChrome({ onProgress } = {}) { // user's only escape is knowing to pin a system browser. detectChrome() already reports // installed:false here (it validates the real exe), so the dir is known junk. Delete it // and let the install proceed instead of failing in a loop.+ try { pruneBrokenCacheDirs({ keepBuildId: buildId }); } catch (e) {} try { const { computeExecutablePath } = require('@puppeteer/browsers'); const expectedExe = computeExecutablePath({ browser: Browser.CHROME, buildId, cacheDir: cacheDir() });@@ -1017,6 +1018,47 @@ function chromeStableStatus() { return { ...chromeStableState, inFlight: !!_chro // disk logging so the first open is instant. If the box has NO Chrome/Edge AND no // cached CfT — the only case that would otherwise force a mid-open download — start // fetching CfT NOW in the background (guarded on disk) so the user doesn't wait later.+// v2.0.397 (arav-rog first-run failure): PRUNE every broken Chrome-for-Testing folder in the+// cache, not only the one build pup is about to install. A folder under <cache>/chrome/ with no+// executable inside is junk left by an interrupted download or a killed unzip (arav-rog held a+// 3-file "win64-148.0.7778.97" from July and a 196 MB folder literally named "chrome"), and+// @puppeteer/browsers refuses to install into an existing folder, so junk like this turns EVERY+// later install into a deterministic failure. Runs at warmup (every bridge spawn) and before an+// install; never touches a folder a download is currently writing.+function hasCftExecutable(dir) {+ try {+ for (const sub of fs.readdirSync(dir)) {+ if (!sub.startsWith('chrome-')) continue;+ const base = path.join(dir, sub);+ if (fileExists(path.join(base, 'chrome.exe'))) return true; // win+ if (fileExists(path.join(base, 'chrome'))) return true; // linux+ if (fs.existsSync(path.join(base, 'Google Chrome for Testing.app'))) return true; // mac+ }+ } catch (e) {}+ return false;+}+function pruneBrokenCacheDirs({ keepBuildId = null } = {}) {+ const root = path.join(cacheDir(), 'chrome');+ const removed = [];+ try {+ if (!fs.existsSync(root)) return removed;+ for (const name of fs.readdirSync(root)) {+ const dir = path.join(root, name);+ let st = null; try { st = fs.statSync(dir); } catch (e) { continue; }+ if (!st.isDirectory()) continue;+ if (keepBuildId && name.includes(keepBuildId)) continue; // the install path handles its own build+ if (state.installing && state.buildId && name.includes(state.buildId)) continue; // being written right now+ if (hasCftExecutable(dir)) continue;+ try {+ fs.rmSync(dir, { recursive: true, force: true });+ removed.push(name);+ console.log(`[chrome] pruned a Chrome-for-Testing cache folder with no executable (interrupted download or unzip): ${dir}`);+ } catch (e) { console.log(`[chrome] could not prune broken cache folder ${dir}: ${e.message}`); }+ }+ } catch (e) { console.log(`[chrome] cache prune skipped: ${e.message}`); }+ return removed;+}+ function warmup() { // v1.8.77: pup OWNS its browser provisioning (John: "isn't that your decision? why would // you need ad to change anything?"). CfT is pup's browser, PERIOD (demarcation doctrine),@@ -1027,6 +1069,7 @@ function warmup() { // first "open in pup" finds it cached — no first-use wait, no AD policy involved. // AD's job ends at "pup is installed"; the rest is ours. (Its v1.9.76 prewarmer policy is // now redundant for pup either way — the install singleton dedupes if both fire.)+ try { pruneBrokenCacheDirs(); } catch (e) {} try { const cands = launchCandidates(); const free = diskFreeMb();@@ -1063,5 +1106,6 @@ module.exports = { installChrome, ensureChromeReady, readiness, expectedBuildId, cacheDir, diskFreeMb, setDefaultBrowser, readDefault, clearDefault, markVerified, pickSummary, installChromeStable, chromeStableStatus, warmup, setPreferCft, expireTemporaryDefault,+ pruneBrokenCacheDirs, hasCftExecutable, abCommand, // v1.8.40: reused by server.js to call AD verbs (e.g. desktop_taskbar) via the direct-API };
src/consent.jsadded+111@@ -0,0 +1,111 @@+// consent.js — the human-consent registry behind pup's approval toasts (v2.0.397, pup-bridge#92).+//+// ab's BRIDGE_SDK hard rule (ab 2.1.30, adom-bridge#142/#137): a verb parked on a HUMAN decision+// must never end in a bare transport timeout. It answers with a structured, non-terminal+// { success:false, errorCode:"blocked_on_user", stillRunning:true, blockedBy, _hint }+// so the calling AI can tell "waiting on the user" from "failed".+//+// pup's approval toasts (pup_credential_reveal, pup_fill_card) used to block the verb for the+// toast's full 120 s. That is longer than some ab per-verb budgets (a 60 s cap was measured on+// ab 2.0.97), so a slow human produced exactly the bare timeout the rule forbids, while the toast+// stayed up and a late click authorised something nobody was listening for.+//+// The registry fixes that in one place:+// - the verb waits IN-VERB for at most inVerbMs (45 s, under every ab budget), then returns+// blocked_on_user with stillRunning:true; the toast stays up and the decision keeps being+// collected in the background for totalSec (120 s);+// - a re-call of the SAME verb with the SAME args (same key) RE-ATTACHES to the pending toast:+// no second toast, and a decision that arrived in between is handed over exactly ONCE;+// - on the toast's own timeout the sticky toast is retracted (notify_dismiss) and the recorded+// outcome is consent_timeout, picked up by the next re-call or dropped after pickupMs.+//+// Pure module: everything OS-facing (notify_user / notify_dismiss) comes in through `abCommand`.++'use strict';++function createRegistry({ abCommand, abPayload, inVerbMs = 45000, totalSec = 120, pickupMs = 120000, log = () => {}, now = Date.now, setTimer = setTimeout } = {}) {+ if (typeof abCommand !== 'function') throw new Error('consent registry needs abCommand(verb, args, opts)');+ const payloadOf = typeof abPayload === 'function' ? abPayload : (r) => r;+ const map = new Map(); // consentId -> record { id, key, verb, what, since, decided, decision, pickedUp, attached, promise }++ function findPending(key) {+ for (const rec of map.values()) if (rec.key === key && !rec.pickedUp) return rec;+ return null;+ }++ // Resolve to the decision string (the button label) or null on timeout / transport error.+ function collect(rec, toast, callerThread, callerReason) {+ return (async () => {+ let decision = null;+ try {+ const r = await abCommand('notify_user', { id: rec.id, ...toast, wait: true, timeoutSec: totalSec }, { timeoutMs: (totalSec + 10) * 1000, callerThread, callerReason });+ const p = payloadOf(r) || {};+ decision = p.action || (r && r.action) || null;+ } catch (e) { decision = null; }+ rec.decision = decision; rec.decided = now();+ let retract = 'n/a';+ if (!decision) {+ // A reminder toast is STICKY: ab never auto-dismisses it, timeoutSec only bounds the wait.+ try { const rr = await abCommand('notify_dismiss', { id: rec.id }, { timeoutMs: 8000 }); const pp = payloadOf(rr) || {}; retract = String(pp.dismissed !== undefined ? pp.dismissed : (rr && rr.dismissed)); }+ catch (e) { retract = 'error: ' + e.message; }+ }+ log(rec, decision, retract);+ const t = setTimer(() => { if (map.get(rec.id) === rec) map.delete(rec.id); }, pickupMs);+ if (t && typeof t.unref === 'function') t.unref();+ return decision;+ })();+ }++ // opts: { key, verb, what, toast:{ idPrefix, ...notify_user args (title, body, buttons, level, scenario) }, callerThread, callerReason }+ // -> { pending:true, consentId, since, attached, remainingSec }+ // -> { pending:false, consentId, decision:string|null, attached }+ async function ask(opts) {+ const { key, verb, what, toast, callerThread, callerReason } = opts || {};+ if (!key || !verb || !toast) throw new Error('consent.ask needs key, verb, toast');+ let rec = findPending(key);+ if (!rec) {+ const { idPrefix, ...toastArgs } = toast;+ rec = { id: `${idPrefix || 'consent'}-${now().toString(36)}-${Math.random().toString(36).slice(2, 6)}`, key, verb, what: what || `Waiting for the user to answer "${toastArgs.title || verb}"`, since: now(), decided: null, decision: undefined, pickedUp: false, attached: 0, promise: null };+ rec.promise = collect(rec, toastArgs, callerThread, callerReason);+ map.set(rec.id, rec);+ } else {+ rec.attached++;+ }+ let capTimer = null;+ const cap = new Promise((resolve) => { capTimer = setTimer(() => resolve('__pending__'), inVerbMs); }); // NOT unref'd: the verb is awaiting it+ const outcome = await Promise.race([rec.promise, cap]);+ if (capTimer) clearTimeout(capTimer);+ if (outcome === '__pending__') {+ const remainingSec = Math.max(0, Math.round(totalSec - (now() - rec.since) / 1000));+ return { pending: true, consentId: rec.id, since: rec.since, attached: rec.attached, remainingSec };+ }+ rec.pickedUp = true; map.delete(rec.id);+ return { pending: false, consentId: rec.id, decision: rec.decision, attached: rec.attached };+ }++ // Rows for pup_readiness.active[]: still-open toasts are blockedOnUser; a decided-but-unpicked+ // record shows as consent_decided so a poller knows to re-call the verb and collect it.+ function rows(at) {+ const t = at || now(); const out = [];+ for (const [id, c] of map) {+ const decided = !!c.decided;+ out.push({+ id, verb: c.verb, phase: decided ? 'consent_decided' : 'consent', stepLabel: decided ? `The user answered; re-call ${c.verb} with the same args to collect the decision` : c.what,+ percent: null, elapsedSec: Math.round((t - c.since) / 1000), estimatedSec: null, etaSec: null, confidence: 'measured',+ blockedOnUser: !decided, blockedBy: decided ? null : 'your Approve/Deny click on the desktop toast', reattach: `${c.verb} again with the SAME args`,+ });+ }+ return out;+ }++ return { ask, rows, map, inVerbMs, totalSec };+}++// The response shape every blocked-on-a-human path returns (SDK 2.1.30). stillRunning:true = the+// wait continues in the background (re-call the same verb to re-attach); false = the human must+// act first, then the caller re-issues the verb.+function blockedOnUser(extra) {+ return Object.assign({ success: false, status: 'error', errorCode: 'blocked_on_user', stillRunning: false }, extra || {});+}++module.exports = { createRegistry, blockedOnUser };
src/server.js+99−58@@ -937,7 +937,16 @@ function pupTimingEstimate(kind) { function pupTimingsSummary() { const t = _readTimings(); const o = {}; for (const k of Object.keys(t)) o[k] = pupTimingEstimate(k); return o; } let _lastLaunchAt = 0, _lastLaunchMs = 0; // set where the browser process is actually spawned const _inflightOpens = new Map(); // id -> { t0, sessionId, thread }-const _pendingConsents = new Map(); // toastId -> { since, what }+// v2.0.397 (#92): consent toasts live in a registry (src/consent.js) so a verb parked on the+// human's click answers blocked_on_user (stillRunning:true) inside every ab budget instead of+// timing out, and a re-call re-attaches to the same toast. _pendingConsents stays the readiness map.+const _consentMod = require('./consent.js');+const consent = _consentMod.createRegistry({+ abCommand: (v, a, o) => chrome.abCommand(v, a, o), abPayload: (r) => abPayload(r),+ log: (rec, decision, retract) => console.log(`[consent] ${rec.verb} ${decision ? 'ANSWERED "' + decision + '"' : 'TIMED OUT'} (${rec.id}, attached ${rec.attached}x)${decision ? '' : ` — toast retract dismissed=${retract}`}`),+});+const blockedOnUser = _consentMod.blockedOnUser;+const _pendingConsents = consent.map; // consentId -> { since, verb, what, decided, ... } // Terminal frame for a finished open (percent is 100 by definition; actuals + startKind ride along). function openProgressFrame(t0) { const launched = _lastLaunchAt >= t0;@@ -964,10 +973,10 @@ function activeProgress() { if (rd && rd.installing) rows.push({ id: 'cft-download', verb: 'pup_prewarm', phase: 'download', stepLabel: 'Downloading Chrome for Testing', percent: rd.installProgressPct || 0, elapsedSec: rd.downloadElapsedSec, estimatedSec: (rd.downloadElapsedSec != null && rd.downloadEtaSec != null) ? rd.downloadElapsedSec + rd.downloadEtaSec : null, etaSec: rd.downloadEtaSec, confidence: rd.downloadEtaSec != null ? 'measured' : 'estimated', startKind: 'cold' }); } catch (e) {}- for (const [id, c] of _pendingConsents) rows.push({ id, verb: c.verb, phase: 'consent', stepLabel: c.what, percent: null, elapsedSec: Math.round((now - c.since) / 1000), estimatedSec: null, etaSec: null, confidence: 'measured', blockedBy: 'your Approve/Deny click on the desktop toast' });+ for (const row of consent.rows(now)) rows.push(row); try { const dlg = findAnyPendingDialog();- if (dlg) rows.push({ id: 'dialog-' + (dlg.sessionId || 'any'), verb: 'pup_dialog', phase: 'dialog', stepLabel: 'A browser dialog is up' + (dlg.sessionId ? ` in ${dlg.sessionId}` : ''), percent: null, confidence: 'measured', blockedBy: 'a browser dialog (pup_dialog to answer it)' });+ if (dlg) rows.push({ id: 'dialog-' + (dlg.sessionId || 'any'), verb: 'pup_dialog', phase: 'dialog', blockedOnUser: true, stepLabel: 'A browser dialog is up' + (dlg.sessionId ? ` in ${dlg.sessionId}` : ''), percent: null, confidence: 'measured', blockedBy: 'a browser dialog (pup_dialog to answer it)' }); } catch (e) {} return rows; }@@ -1033,6 +1042,27 @@ function findAnyPendingDialog() { } // The one pending native dialog on a session, if any. A modal blocks the WHOLE window, so it does // not matter which tab owns it: the first one found is the one the caller is stuck behind.+// v2.0.397 (#92): the walls a page puts up that only a human can pass. Cheap DOM heuristics,+// read in the background; used by pup_login after a submit. Returns { kind, evidence } | null.+async function detectHumanChallenge(page) {+ if (!page) return null;+ const r = await Promise.race([+ page.evaluate(() => {+ const q = (sel) => { try { return document.querySelector(sel); } catch (e) { return null; } };+ const vis = (el) => { if (!el) return false; const r = el.getBoundingClientRect(); return r.width > 0 && r.height > 0; };+ const otp = q('input[autocomplete="one-time-code"]') || q('input[name*="otp" i]') || q('input[id*="otp" i]') || q('input[name*="verification" i][inputmode="numeric"]') || q('input[name*="mfa" i]') || q('input[name*="totp" i]') || q('input[name*="2fa" i]');+ if (vis(otp)) return { kind: 'two_factor', evidence: 'one-time-code input: ' + (otp.getAttribute('name') || otp.getAttribute('id') || otp.getAttribute('autocomplete')) };+ const capFrame = Array.from(document.querySelectorAll('iframe')).find(f => /recaptcha|hcaptcha|turnstile|arkose|funcaptcha/i.test(f.src || ''));+ if (capFrame && vis(capFrame)) return { kind: 'captcha', evidence: 'captcha iframe: ' + new URL(capFrame.src).hostname };+ const t = (document.body && document.body.innerText || '').slice(0, 20000);+ if (/verification code|two[- ]factor|2-step verification|authenticator app|enter the code we sent|security code|one-time (pass)?code/i.test(t) && !q('input[type="password"]')) return { kind: 'two_factor', evidence: 'page text mentions a verification code and no password field is present' };+ if (/i'?m not a robot|prove you are human|verify you are human|security check/i.test(t)) return { kind: 'captcha', evidence: 'page text asks for a human check' };+ return null;+ }),+ new Promise((rs) => setTimeout(() => rs(null), 4000)),+ ]);+ return r || null;+} function findPendingDialog(session) { if (!session || !Array.isArray(session.tabs)) return null; for (const t of session.tabs) {@@ -12007,8 +12037,8 @@ const VERB_META = { pup_record_stop: { hint: 'Stops recording and returns the saved file path.', related: ['pup_record_start', 'pup_record_list'], pitfalls: ['no-op if nothing is recording for this session'] }, pup_record_status: { hint: 'Whether a recording is active for the session.', related: ['pup_record_start', 'pup_record_stop'], pitfalls: [] }, pup_record_list: { hint: 'Lists saved page recordings.', related: ['pup_record_start'], pitfalls: [] },- desktop_recorder_open: { hint: 'Opens the desktop screen-recorder UI window.', related: ['desktop_record_start', 'desktop_recorder_close'], pitfalls: [] },- desktop_record_start: { hint: 'Starts a full desktop screen recording.', related: ['desktop_record_stop', 'desktop_record_status'], pitfalls: ['only one desktop recording at a time'] },+ desktop_recorder_open: { hint: 'Opens the desktop screen-recorder UI window. OWNERSHIP: the desktop_ namespace is ab-core\'s; pup serves these on ab\'s behalf only until ab-core\'s native whole-desktop recorder lands (adom-bridge#143), then they leave pup. Tab recording (pup_record_*) is pup\'s own.', related: ['desktop_record_start', 'desktop_recorder_close'], pitfalls: [] },+ desktop_record_start: { hint: 'Starts a full desktop screen recording (served by pup for ab-core until adom-bridge#143 lands; needs ffmpeg on the box for the duration header).', related: ['desktop_record_stop', 'desktop_record_status'], pitfalls: ['only one desktop recording at a time'] }, desktop_record_stop: { hint: 'Stops the desktop recording and returns the file.', related: ['desktop_record_start', 'desktop_record_list'], pitfalls: [] }, pup_readiness: { hint: 'Cold-start readiness probe: {ready, chromeForTestingInstalled, installing, installProgressPct}. On a fresh PC, poll this until ready:true before pup_open_window — the bridge auto-installs Chrome for Testing on its own.', related: ['pup_prewarm', 'pup_open_window'], pitfalls: ['ready:false with installing:true just means Chrome for Testing is still downloading — wait + poll, do NOT treat it as an error', 'installPhase:"failed" → surface lastError to the user (usually network/proxy blocking the download, or low disk)'] }, pup_prewarm: { hint: 'Installs Chrome for Testing (the browser pup drives) WITHOUT opening a window — the warm-before-first-use path. Returns fast; poll pup_readiness for completion.', related: ['pup_readiness', 'pup_open_window'], pitfalls: ['the ~150 MB download needs network access to storage.googleapis.com', 'does not open a window — call pup_open_window once readiness reports ready:true'] },@@ -14970,7 +15000,7 @@ const server = http.createServer(async (req, res) => { }; sendJSON(res, { success: true, bridge: 'puppeteer', // v2.0.383 (#59/#64): the measured-progress contract, per the Bridge SDK.- progressContract: { statusVerb: 'pup_readiness', activeKey: 'active', slowVerbs: ['pup_open_window', 'pup_prewarm', 'pup_use'], blockedByPhases: ['consent', 'dialog'], timings: pupTimingsSummary() },+ progressContract: { statusVerb: 'pup_readiness', activeKey: 'active', slowVerbs: ['pup_open_window', 'pup_prewarm', 'pup_use'], blockedByPhases: ['consent', 'dialog'], blockedOnUser: { errorCode: 'blocked_on_user', rule: 'ab BRIDGE_SDK 2.1.30: a verb parked on a HUMAN never bare-timeouts; it answers this shape', stillRunning: 'true = the wait continues in the background: re-call the SAME verb with the SAME args to re-attach (no second prompt) or watch pup_readiness.active[]; false = the human must act first, then re-issue the verb', inVerbWaitSec: Math.round(consent.inVerbMs / 1000), promptLifetimeSec: consent.totalSec }, timings: pupTimingsSummary() }, verbs: [ V('pup_my_windows', 'START HERE for window management: everything YOUR thread owns — windows, tabs, urls, titles, state — with ready-to-run next steps. Zero args. You NEVER need an hwnd, desktop_find_window, or desktop_screenshot_window for a pup window: pup keys every window to its owning --ai-thread name and resolves ids for you — omit sessionId in any pup verb to target your own window, or pass "ai-thread" to address a thread by name (the CLI\'s --ai-thread flag, same spelling). pup_screenshot {} screenshots it; pup_reload {} refreshes it.', { 'ai-thread': 'optional str — inspect another thread\'s windows (read-only listing); default = the calling thread. The CLI flag\'s exact spelling; aiThread also accepted' }, 10, {}), V('pup_open_window', 'Open a generic pup window — a fresh isolated Chrome/Edge process (never your real signed-in browser), fully drivable via CDP. ALWAYS opens in the BACKGROUND (does not steal focus; never disturbs the user\'s main work; still fully screenshottable/drivable) UNLESS you pass foreground:true — do that ONLY when the user explicitly wants to watch. Pass width+height (+ optional x,y, screen px) to place it at a specific size/position for screenshot framing — this NO LONGER foregrounds it (sizing stays in the background too; default unsized is maximized). profile defaults to the durable shared \'adom-you\' profile (logins/localStorage persist + are shared across all pup windows); isolated:true for a private throwaway. OWNERSHIP: sessionIds are shared across ALL AI threads — use a task-prefixed sessionId + owner:"<your-task>" so pup can refuse other threads stealing your window (reusing an existing id with a new url navigates it away; declared cross-owner reuse errors with session_owned_by_another_thread unless takeover:true). (For the user\'s REAL Chrome/Edge, use the Adom extension\'s nbrowser_* verbs.)', { sessionId: 'optional str — OMIT IT (recommended): your thread gets its OWN window keyed on your --ai-thread name; the first open creates it, later opens become TABS in it, and no other thread can touch it. Pass an explicit id only when you need a SEPARATE window (task-prefix it, e.g. "mytask-web")', url: 'optional str', profile: 'optional str — a named durable jar; DEFAULT is the shared "adom-you" profile (signed-in state shared across windows). isolated:true overrides to a throwaway', isolated: 'optional bool — private throwaway profile: fresh-login tests, acting as a different user. REASON-GATED (v2.0.101): requires isolationReason or the open is refused — every window defaults to the shared adom-you profile', isolationReason: 'required WITH isolated:true — why this window must NOT ride the shared adom-you profile (min 10 chars; logged, auditable). webSecurity:false and wikiView:public carry their own built-in reason', webSecurity: 'optional false — dev escape hatch for cross-origin eval; FORCES isolation so insecure flags never touch the credentialed profile', owner: 'optional str — your thread/task label; lets pup protect this window from other threads', takeover: 'optional bool — claim a window owned by another thread (only when the user explicitly asks)', foreground: 'optional bool — request showing the window on the user\'s screen. GATED (v1.8.70): requires foregroundReason too, else the open succeeds but stays BACKGROUND with foregroundDenied set. Only when the USER explicitly asked to watch.', foregroundReason: 'required WITH foreground:true — quote/paraphrase what the user said that justifies putting a window on their screen (min 10 chars; logged + recorded in lastAgentUpdate, auditable)', width: 'optional int px — sizes for screenshot framing, stays background', height: 'optional int px', x: 'optional int px', y: 'optional int px', wikiView: 'optional "authed"|"public" — for wiki.adom.inc URLs. THREE distinct cases: (1) OMIT = the shared adom-you profile, LOGGED IN as the user if they have signed into the wiki (the usual case); this is NOT anonymous. (2) "public" = the SERVER renders the page exactly as a signed-out visitor sees it via the ?view=public param (adom/wiki#126: request user nulled, byte-identical to logged-out, private pages 404) — the window STAYS on the shared profile, session untouched, no isolated jar; use this to verify a publish did not leak private source. (3) "authed" = the explicit logged-in view (30-day SSO; first use may need a one-time login — response says wikiLoginNeeded). To compare "what I see" vs "what the world sees", open one "authed" (or omit) AND one "public" tab — do NOT rely on omit for the public view.', appName: 'optional str — the APP this window is driving (e.g. "shotlog"); shown in the hover tooltip', appIconB64: 'optional base64 PNG — override the app icon worn in the taskbar overlay. Usually UNNECESSARY: pup auto-fetches the page\'s own <link rel="icon"> favicon (the adom-ui-design standard artifact every app must serve) and wears it automatically. An app-looking URL with NO favicon gets a reprimand hint.', updateNote: 'optional str — a short "what I did" note, surfaced back in list_tabs/list_windows as lastAgentUpdate so the user knows what changed', autoFlash: 'optional bool (default true) — mutating verbs auto-flash this window\'s taskbar (debounced, never steals focus) so the user notices the agent touched it; pass false to make this window silent by default', silent: 'optional bool — suppress the auto-flash for just this call' }, 60, { sessionId: 'mytask-web', owner: 'mytask', url: 'https://example.com', width: 960, height: 540 }),@@ -15011,8 +15041,8 @@ const server = http.createServer(async (req, res) => { V('pup_record_stop', 'Stop recording; returns the saved file.', { sessionId: 'required str' }, 30, { sessionId: 'work' }), V('pup_record_status', 'Recording status for a session.', { sessionId: 'required str' }, 10, { sessionId: 'work' }), V('pup_record_list', 'List recordings.', {}, 10, {}),- V('desktop_recorder_open', 'Open the desktop screen-recorder UI.', {}, 20, {}),- V('desktop_record_start', 'Start a desktop screen recording.', { outPath: 'optional str' }, 20, {}),+ V('desktop_recorder_open', 'Open the desktop screen-recorder UI. (ab-core capability served by pup on its behalf until adom-bridge#143; moving out of pup.)', {}, 20, {}),+ V('desktop_record_start', 'Start a desktop screen recording. (ab-core capability served by pup on its behalf until adom-bridge#143; moving out of pup.)', { outPath: 'optional str' }, 20, {}), V('desktop_record_stop', 'Stop the desktop screen recording; returns the file.', {}, 30, {}), V('pup_readiness', 'Readiness + which browser pup will drive (installed Chrome/Edge, or cached Chrome for Testing) + the full candidate list and current default. READ-ONLY; never downloads.', {}, 10, {}), V('pup_prewarm', 'Install Chrome for Testing WITHOUT opening a window, and pin it as the default. Use when you specifically want CfT rather than the machine\'s installed Chrome/Edge.', { wait: 'optional bool — block until installed', setDefault: 'optional bool (default true) — pin CfT as the default' }, 240, {}),@@ -18085,10 +18115,13 @@ const server = http.createServer(async (req, res) => { session._foreground = true; session._lastForeground = { ts: Date.now(), verb: command, reason: _reason }; try { await osRaiseSessionWindow(session, resolvedId, null); } catch (e) {} // assisted_click narrates its own captions const hwnd = await sessionHwnd(resolvedId);- const ready = await waitForUserReady(resolvedId, hwnd, { maxMs: Number(args.maxWaitMs) || 120000 });+ // v2.0.397 (#92): the in-verb wait is capped under every ab budget; a longer human+ // absence answers blocked_on_user (the caller re-issues the click when the user is back).+ const _maxWait = Math.min(Number(args.maxWaitMs) || 120000, 45000);+ const ready = await waitForUserReady(resolvedId, hwnd, { maxMs: _maxWait }); if (!ready) {- await pupCaption('Adom stopped waiting: you did not return to the window. Ask me to try again when ready.', 7000);- sendJSON(res, { success: false, errorCode: 'user_not_ready', error: 'Timed out waiting for you to focus the window. Nothing was clicked.', _hint: 'The user alt-tabbed away and did not return within the wait window. Re-issue pup_assisted_click when they are ready.' });+ await pupCaption('Adom is still waiting for you to come back to this window; your AI will ask again when you are ready.', 7000);+ sendJSON(res, blockedOnUser({ stillRunning: false, reason: 'user_not_ready', blockedBy: 'The user must bring the pup window to the foreground (click it or Alt-Tab to it) so the real-cursor click lands in the right window', waitedSec: Math.round(_maxWait / 1000), error: 'Waited for the user to focus the window; they did not. Nothing was clicked.', _hint: 'Not a failure: the human was not at the window. Tell them what you need clicked and where, then re-issue pup_assisted_click with the same args when they say they are ready; it waits again (up to 45s per call).' })); return; } await pupCaption(`Adom is clicking ${_label} now.`, 5000);@@ -19719,35 +19752,31 @@ const server = http.createServer(async (req, res) => { try { cred = await credentialVault.getCredentialForUrl('https://' + host); } catch (e) { cred = null; } if (!cred) { sendJSON(res, { success: false, errorCode: 'no_credential', error: `no saved login for ${host}`, _hint: 'pup_credentials {} lists the hosts the vault knows (host + username only).' }); return; } const who = _callerThread || 'an unnamed local caller';- const toastId = `credreveal-${Date.now().toString(36)}`;- _pendingConsents.set(toastId, { since: Date.now(), verb: 'pup_credential_reveal', what: `Waiting for the user to allow revealing the ${host} login` });- console.log(`[credential] reveal REQUESTED for ${host} (user ${cred.username}) by "${who}" — asking the user on the desktop`);- let decision = null;- try {- const r = await chrome.abCommand('notify_user', {- id: toastId, level: 'warning', scenario: 'reminder',+ const _ask = await consent.ask({+ key: `pup_credential_reveal|${host}|${_callerThread || 'local'}`, verb: 'pup_credential_reveal',+ what: `Waiting for the user to allow revealing the ${host} login to ${who}`,+ toast: {+ idPrefix: 'credreveal', level: 'warning', scenario: 'reminder', title: `Reveal your ${host} login to ${who}?`,- body: `${who} is asking pup for the saved login for ${host} (username ${cred.username}). Reason given: "${reason}". Reveal sends the password to that caller ONCE. Deny sends nothing. This request expires in 2 minutes.`,+ body: `${who} is asking pup for the saved login for ${host} (username ${cred.username}). Reason given: "${reason}". Reveal sends the password to that caller ONCE. Deny sends nothing. This request came from a process on THIS desktop.`, buttons: [{ label: 'Reveal' }, { label: 'Deny' }],- wait: true, timeoutSec: 120,- }, { timeoutMs: 130000, callerThread: _callerThread || undefined, callerReason: reason });- const p = abPayload(r) || {};- decision = p.action || (r && r.action) || null;- } catch (e) { decision = null; }- _pendingConsents.delete(toastId);+ },+ callerThread: _callerThread || undefined, callerReason: reason,+ });+ if (_ask.pending) {+ console.log(`[credential] reveal for ${host} by "${who}" still waiting on the user after ${Math.round(consent.inVerbMs / 1000)}s (${_ask.consentId}) — answering blocked_on_user, toast stays up ${_ask.remainingSec}s more`);+ sendJSON(res, blockedOnUser({+ stillRunning: true, consentId: _ask.consentId, blockedBy: `The user must click Reveal or Deny on the desktop toast "Reveal your ${host} login to ${who}?"`, remainingSec: _ask.remainingSec,+ error: 'waiting on the user, not failed: the consent toast is still on their screen',+ _hint: `Not an error. The toast stays up for ${_ask.remainingSec}s more. Call pup_credential_reveal again with the SAME args to re-attach and collect the decision (no second toast is shown), or poll pup_readiness {} and watch active[] for consentId ${_ask.consentId}. Do not tell the user it failed.`,+ }));+ return;+ }+ const decision = _ask.decision; if (decision !== 'Reveal') { const code = decision ? 'consent_declined' : 'consent_timeout';- // v2.0.376 (John): a reminder toast is STICKY — ab never auto-dismisses it, and- // timeoutSec only bounds OUR wait. Without this retract, an expired consent toast- // stays on screen and a late click looks like an authorization pup already stopped- // listening for. On decline the user's own click dismissed it; on timeout WE must.- let _dis = 'n/a';- if (!decision) {- try { const rr = await chrome.abCommand('notify_dismiss', { id: toastId }, { timeoutMs: 8000 }); const pp = abPayload(rr) || {}; _dis = String(pp.dismissed !== undefined ? pp.dismissed : (rr && rr.dismissed)); }- catch (e) { _dis = 'error: ' + e.message; }- }- console.log(`[credential] reveal ${code.toUpperCase()} for ${host} by "${who}"${decision ? '' : ` — toast retract dismissed=${_dis}`}`);- sendJSON(res, { success: false, errorCode: code, error: code === 'consent_declined' ? 'the user declined on the desktop; nothing was revealed' : 'the user did not answer the consent toast within 120s; nothing was revealed', _hint: 'Do not retry in a loop. If the user wants this, they will click Reveal on the next ask.' });+ console.log(`[credential] reveal ${code.toUpperCase()} for ${host} by "${who}"`);+ sendJSON(res, { success: false, errorCode: code, error: code === 'consent_declined' ? 'the user declined on the desktop; nothing was revealed' : `the user did not answer the consent toast within ${consent.totalSec}s; nothing was revealed`, _hint: 'Do not retry in a loop: the user saw the request. Ask them in chat whether they want to reveal it, then call once more if they say yes.' }); return; } // Plaintext ONCE. The audit line below is the only record and it carries no secret.@@ -19802,32 +19831,31 @@ const server = http.createServer(async (req, res) => { // deliberately allowed for RELAYED callers (unlike pup_credential_reveal): the digits // never reach the caller, and the human click is the authorization for this one fill. console.log(`[card] fill REQUESTED ${nickname} (…${meta.last4}) on ${merchant}${amount ? ' for ' + amount : ''} by "${who}" — asking the user`);- let decision = null;- const cardToastId = `cardfill-${Date.now().toString(36)}`;- _pendingConsents.set(cardToastId, { since: Date.now(), verb: 'pup_fill_card', what: `Waiting for the user to approve the ${meta.brand} …${meta.last4} fill on ${merchant}` });- try {- const r = await chrome.abCommand('notify_user', {- id: cardToastId, level: 'warning', scenario: 'reminder',+ const _ask = await consent.ask({+ key: `pup_fill_card|${nickname}|${merchant}|${_callerThread || 'anon'}`, verb: 'pup_fill_card',+ what: `Waiting for the user to approve the ${meta.brand} …${meta.last4} fill on ${merchant}`,+ toast: {+ idPrefix: 'cardfill', level: 'warning', scenario: 'reminder', title: `Pay with ${meta.brand} …${meta.last4} on ${merchant}?`,- body: `${who} wants to fill your card "${nickname}" (${meta.brand} ending ${meta.last4}) into the checkout on ${merchant}${amount ? ` for ${amount}` : ''}. Approve fills the card fields in the pup window so you (or the AI) can submit. Deny fills nothing. This request expires in 2 minutes.`,+ body: `${who} wants to fill your card "${nickname}" (${meta.brand} ending ${meta.last4}) into the checkout on ${merchant}${amount ? ` for ${amount}` : ''}. Approve fills the card fields in the pup window (the digits never leave this machine). Deny fills nothing.`, buttons: [{ label: 'Approve' }, { label: 'Deny' }],- wait: true, timeoutSec: 120,- }, { timeoutMs: 130000, callerThread: _callerThread || undefined, callerReason: `fill card ${nickname} on ${merchant}` });- const p = abPayload(r) || {};- decision = p.action || (r && r.action) || null;- } catch (e) { decision = null; }- _pendingConsents.delete(cardToastId);+ },+ callerThread: _callerThread || undefined, callerReason: `fill card ${nickname} on ${merchant}`,+ });+ if (_ask.pending) {+ console.log(`[card] fill ${nickname} on ${merchant} by "${who}" still waiting on the user after ${Math.round(consent.inVerbMs / 1000)}s (${_ask.consentId}) — answering blocked_on_user, toast stays up ${_ask.remainingSec}s more`);+ sendJSON(res, blockedOnUser({+ stillRunning: true, consentId: _ask.consentId, blockedBy: `The user must click Approve or Deny on the desktop toast "Pay with ${meta.brand} …${meta.last4} on ${merchant}?"`, remainingSec: _ask.remainingSec,+ error: 'waiting on the user, not failed: the approval toast is still on their screen',+ _hint: `Not an error. The toast stays up for ${_ask.remainingSec}s more. Call pup_fill_card again with the SAME args to re-attach and collect the decision (no second toast is shown), or poll pup_readiness {} and watch active[] for consentId ${_ask.consentId}. Do not tell the user it failed.`,+ }));+ return;+ }+ const decision = _ask.decision; if (decision !== 'Approve') { const code = decision ? 'consent_declined' : 'consent_timeout';- // v2.0.376: retract the sticky toast on timeout (see the credential_reveal note) —- // an expired approval offer must not sit on screen looking clickable.- let _dis = 'n/a';- if (!decision) {- try { const rr = await chrome.abCommand('notify_dismiss', { id: cardToastId }, { timeoutMs: 8000 }); const pp = abPayload(rr) || {}; _dis = String(pp.dismissed !== undefined ? pp.dismissed : (rr && rr.dismissed)); }- catch (e) { _dis = 'error: ' + e.message; }- }- console.log(`[card] fill ${code.toUpperCase()} ${nickname} on ${merchant} by "${who}"${decision ? '' : ` — toast retract dismissed=${_dis}`}`);- sendJSON(res, { success: false, errorCode: code, error: code === 'consent_declined' ? 'the user declined the card fill on the desktop; nothing was filled' : 'the user did not answer the approval toast within 120s; nothing was filled', _hint: 'Do not retry in a loop. Tell the user what you were trying to buy and let them re-initiate.' });+ console.log(`[card] fill ${code.toUpperCase()} ${nickname} on ${merchant} by "${who}"`);+ sendJSON(res, { success: false, errorCode: code, error: code === 'consent_declined' ? 'the user declined the card fill on the desktop; nothing was filled' : `the user did not answer the approval toast within ${consent.totalSec}s; nothing was filled`, _hint: 'Do not retry in a loop: the user saw the request. Ask them in chat whether they want to pay with this card, then call once more if they say yes.' }); return; } let card = null;@@ -19909,6 +19937,19 @@ const server = http.createServer(async (req, res) => { // React-controlled MASKED field (Gusto etc.) silently ignored the set — the classic // "submits empty / Password is required" trap. Coach the caller to the fix instead of // letting it submit an empty password.+ // v2.0.397 (#92): after a SUBMIT, look for the walls only a human can pass (one-time+ // code, authenticator, captcha). That is blocked_on_user, not success.+ if (args.submit && r.submitted) {+ let _wall = null;+ try { await new Promise(rs => setTimeout(rs, 1800)); _wall = await detectHumanChallenge(tab.page); } catch (e) { _wall = null; }+ if (_wall) {+ console.log(`[login] ${liveHost || curUrl}: submitted, now on a ${_wall.kind} wall (${_wall.evidence}) — blocked_on_user`);+ sendJSON(res, blockedOnUser({ stillRunning: false, reason: _wall.kind, blockedBy: _wall.kind === 'captcha' ? 'The user must solve the captcha in the pup window' : 'The user must enter the one-time / two-factor code in the pup window', evidence: _wall.evidence,+ output: JSON.stringify({ ok: false, url: (() => { try { return tab.page.url(); } catch (e) { return curUrl; } })(), submitted: true, filledUsername: r.filledUsername, filledPassword: r.filledPassword, wall: _wall.kind }),+ _hint: `The login was filled and submitted; the site now needs the HUMAN (${_wall.kind}). Raise the window with pup_raise_os_window {foregroundReason:"finish the ${_wall.kind} for ${liveHost || 'this site'}"} and tell them what to do, then continue (screenshot to confirm you are past it). Do not re-submit the password.` }));+ return;+ }+ } const reactMaskedMiss = !useKeystrokes && r.filledPassword === false; sendJSON(res, { success: true, output: JSON.stringify({ ok: true, url: curUrl, credentialSource: source,