← Commit history

Fix browser_close leaving core-pegging renderer orphans that froze the desktop. The teardown reap was fire-and-forget with an 8s full-Win32_Process enumeration that TIMED OUT exactly when the box was CPU-pegged by the runaway renderers it existed to kill (AdomLapper: 88-98% CPU, dwm starved, forced reboot; 6 CfT procs survived a close, 4 renderers near 100%). Now the reap is AWAITED, VERIFIED, and load-tolerant: WMI image-name filter is pushed server-side so only chrome/msedge rows materialize (fast at 98% CPU), close() awaits the profile-scoped kill of the parent tree PLUS any reparented renderer orphans, and every close response returns chromeProcsRemaining (should be 0). Adds a boot + periodic orphan sweep (reaps pup chrome whose profile has no live session, spared by profile) and a verified close-all sweep.

John Lauer ·8f6e8a8b4c ·3mo ago ·parent bc46537
4 files changed +121−58
adom-bridge-puppeteer-manifest.json+4−4
@@ -1,10 +1,10 @@ {   "manifest_version": 1,   "name": "puppeteer",-  "version": "1.9.88",-  "url": "https://wiki.adom.inc/download/adom/adom-desktop-puppeteer-bridge/1.9.88/adom-bridge-puppeteer-v1.9.88.zip",-  "sha256": "61bd6d99d0dae80d5a80ab9b82946dc18a0bd3642b6ca6d52ebc80431a399096",-  "size": 360597,+  "version": "1.9.89",+  "url": "https://wiki.adom.inc/download/adom/adom-desktop-puppeteer-bridge/1.9.89/adom-bridge-puppeteer-v1.9.89.zip",+  "sha256": "799b042ea9f5fefdbd4f644bc293f77b372f5ef69f0a4eab3b1ea9cca9ba5810",+  "size": 362354,   "verbPrefixes": [     "browser_",     "desktop_recorder_",
src/BRIDGE_VERSION+1−1
@@ -1 +1 @@-1.9.88+1.9.89
src/bridge.json+1−1
@@ -2,7 +2,7 @@   "manifest_version": 1,   "name": "puppeteer",   "displayName": "Puppeteer (Chrome for Testing + Native)",-  "version": "1.9.88",+  "version": "1.9.89",   "description": "Drives a browser ALREADY on the machine \u2014 installed Chrome, else Microsoft Edge (every Windows PC ships with Edge) \u2014 launched with a FRESH isolated profile, so the common case has NO ~150 MB Chrome-for-Testing download and it never touches the user's real tabs/logins. Binds its local server to LOOPBACK (127.0.0.1) only, so no Windows Firewall 'allow Node through the firewall' prompt ever pops. Chrome for Testing is fetched only as a LAST RESORT if the machine has no Chromium browser at all. SPAWN-VERIFIED fallback: pup tries each detected browser (Chrome \u2192 Edge \u2192 CfT) and actually launches it, falling through to the next if one won't start (and handling browsers like Edge that relaunch onto another PID), so a corrupt/partial install never hard-fails an open. The browser that launches is cached as the default (persisted across restarts) so it isn't re-probed each time. On spawn the bridge SELF-WARMS: detects browsers + disk, and only if the box has no Chrome/Edge and no cached CfT does it background-fetch CfT so the first open is instant. browser_use pins a specific browser (chrome | edge | cft | auto); {browser:'chrome',install:true} downloads & installs REAL Google Chrome (user-scoped, no admin) on an Edge-only box. browser_readiness (READ-ONLY) reports which browser will be used + the full candidate list + the default + free disk. Every failure returns a specific errorCode + _hint (chrome_install_no_disk, chrome_download_network, chrome_install_permission, chrome_stable_not_detected, \u2026). Launches with FULL browser permissions by default for full debug (strictPermissions:true opts out). Headful automation: open/close windows+tabs, navigate, screenshot, eval JS, record. pup is a GENERIC, non-profile browser only \u2014 every session is a fresh isolated Chrome/Edge process, never the user's real signed-in browser; profile defaults to sessionId. To drive the user's REAL Chrome/Edge (their profile, cookies, SSO), use the Adom browser EXTENSION's nbrowser_* verbs \u2014 a separate bridge, not pup's job. macOS (v1.9.74+): full parity with mac-native mechanics \u2014 background-by-default is the app-level hide (windows stay drivable/screenshottable while hidden), raise/lower map to unhide+activate/hide, browser_alert_window posts a native toast whose click focuses the window, browser_use {browser:'chrome',install:true} installs Chrome from the official dmg, and taskbar identity/badging remains Windows-only.",   "homepage": "https://wiki.adom.inc/adom/adom-desktop",   "author": "Adom Inc.",
src/server.js+115−52
@@ -1441,13 +1441,82 @@ function setupErrorCollection(pg, errorsArr) { }  // Kill a Chrome browser process forcefully by PID-// Falls back to browser.close() if PID isn't available.-// `browser` may be null when we're just cleaning up an orphaned lock file-// and have no live puppeteer handle.+// ─── Reliable renderer/orphan reaping (close leaving core-pegging renderers → froze the desktop) ──+//+// v1.9.89 — the teardown reap is now AWAITED + VERIFIED + load-tolerant. The old sweep was+// FIRE-AND-FORGET with an 8s timeout on a FULL Win32_Process enumeration, which TIMES OUT precisely+// when the box is pegged by the runaway renderers it exists to kill (measured on AdomLapper: overall+// CPU 88-98% on 16 cores, dwm starved to 38-63%, mouse frozen, forced reboot; 6 CfT procs survived a+// browser_close, 4 --type=renderer children near 100%). Three fixes: (1) push the image-name filter to+// WMI server-side (-Filter "Name='chrome.exe' OR Name='msedge.exe'") so ONLY browser rows are+// materialized — fast even at 98% CPU; (2) AWAIT the reap so close() VERIFIES nothing survives before+// returning ok; (3) return chromeProcsRemaining so the caller KNOWS the teardown completed. Still+// non-blocking: async execFile inside an awaited Promise suspends only THIS request's async chain,+// never the event loop (the v1.9.81 rule was against execFileSync/execSync on the close path).+const _isWin = process.platform === 'win32';+// Enumerate chrome/msedge {pid, cmd} matching a PowerShell -Where clause on CommandLine. The -Filter+// makes WMI return only browser processes server-side, so this stays cheap under heavy CPU load.+function _psProcs(whereClause) {+  return new Promise((resolve) => {+    if (!_isWin) { resolve([]); return; }+    const ps = `Get-CimInstance Win32_Process -Filter "Name='chrome.exe' OR Name='msedge.exe'" | Where-Object { ${whereClause} } | Select-Object ProcessId,CommandLine | ConvertTo-Json -Compress`;+    try {+      execFile('powershell.exe', ['-NoProfile', '-ExecutionPolicy', 'Bypass', '-Command', ps],+        { encoding: 'utf8', timeout: 7000, windowsHide: true, maxBuffer: 8 * 1024 * 1024 }, (err, out) => {+          if (err || !out) { resolve([]); return; }+          let parsed; try { parsed = JSON.parse(out); } catch { parsed = null; }+          const items = Array.isArray(parsed) ? parsed : (parsed ? [parsed] : []);+          resolve(items.map(i => ({ pid: i.ProcessId, cmd: i.CommandLine || '' })).filter(x => x.pid));+        });+    } catch (e) { resolve([]); }+  });+}+function _taskkillPids(pids) {+  return Promise.all([...new Set(pids || [])].map(pid => new Promise((resolve) => {+    try { execFile('taskkill', ['/F', '/T', '/PID', String(pid)], { windowsHide: true, timeout: 5000 }, () => resolve()); }+    catch (e) { resolve(); }+  })));+}+const _psLike = (p) => String(p).replace(/'/g, "''");+// Reap EVERY chrome/msedge (the parent AND any reparented --type=renderer orphan the tree-kill missed)+// still bound to this profile's userDataDir. Awaited + verified, one retry for a straggler mid-spawn.+// Profile-scoped so it can never touch the user's real browser or another session's window.+async function reapProfileChromes(profileName) {+  if (!_isWin || !profileName) return { killed: 0, remaining: 0 };+  const where = `$_.CommandLine -like '*${_psLike(path.join(PROFILES_DIR, profileName))}*'`;+  let procs = await _psProcs(where);+  const killed = procs.length;+  if (procs.length) { await _taskkillPids(procs.map(p => p.pid)); await new Promise(r => setTimeout(r, 350)); }+  let remaining = await _psProcs(where);+  if (remaining.length) { await _taskkillPids(remaining.map(p => p.pid)); await new Promise(r => setTimeout(r, 300)); remaining = await _psProcs(where); }+  if (killed || remaining.length) console.log(`[reap] profile "${profileName}": killed ${killed}, remaining ${remaining.length}`);+  return { killed, remaining: remaining.length };+}+// Reap ORPHANED pup chrome/msedge — any process under a pup-profiles dir whose profile has NO live+// session (parent AND renderer children). Live sessions are spared by PROFILE (not just parent PID),+// so a session's own renderers are never touched. Used by browser_close (all) + boot/periodic sweeps.+async function reapOrphanPupChromes() {+  if (!_isWin) return { killed: 0, remaining: 0 };+  const liveDirs = new Set();+  for (const [, s] of sessions) { if (s && s.profileName) liveDirs.add(path.join(PROFILES_DIR, s.profileName).toLowerCase()); }+  const where = `$_.CommandLine -like '*${_psLike(PROFILES_DIR)}*'`;+  const isOrphan = (p) => { const c = (p.cmd || '').toLowerCase(); for (const d of liveDirs) { if (c.includes(d)) return false; } return true; };+  let orphans = (await _psProcs(where)).filter(isOrphan);+  const killed = orphans.length;+  if (orphans.length) { await _taskkillPids(orphans.map(p => p.pid)); await new Promise(r => setTimeout(r, 350)); }+  let remaining = (await _psProcs(where)).filter(isOrphan);+  if (remaining.length) { await _taskkillPids(remaining.map(p => p.pid)); await new Promise(r => setTimeout(r, 300)); remaining = (await _psProcs(where)).filter(isOrphan); }+  if (killed || remaining.length) console.log(`[reap-orphan] killed ${killed} orphan pup process(es), remaining ${remaining.length} (spared ${liveDirs.size} live profile(s))`);+  return { killed, remaining: remaining.length };+}++// Tear down the browser for a profile and VERIFY it. Returns {killed, remaining}; remaining MUST be 0.+// Falls back to browser.close() if PID isn't available. `browser` may be null when we're just cleaning+// up an orphaned lock file and have no live puppeteer handle. async function killBrowserProcess(browser, profileName) {   if (!profileName) {     console.log('killBrowserProcess called without profileName — skipping');-    return;+    return { killed: 0, remaining: 0 };   }   let pid = null;   if (browser) {@@ -1455,10 +1524,10 @@ async function killBrowserProcess(browser, profileName) {     if (!pid) {       try { pid = browser.process()?.pid; } catch (e) { /* connected browser has no process() */ }     }-    // Try graceful close first-    try { await browser.close(); } catch (e) {-      console.log(`Graceful close failed for profile "${profileName}": ${e.message}`);-    }+    // Graceful close, but never let a wedged CDP make close() hang the whole teardown (the reap+    // below is what actually guarantees the processes die, so a slow/hung close must not block it).+    try { await Promise.race([browser.close(), new Promise(r => setTimeout(r, 3000))]); }+    catch (e) { console.log(`Graceful close failed for profile "${profileName}": ${e.message}`); }   }    // If no browser handle, try to recover PID from session files on disk@@ -1469,51 +1538,21 @@ async function killBrowserProcess(browser, profileName) {     } catch {}   } -  // Then force-kill the PID if it's still around+  // Tree-kill the tracked parent first (fast; catches the common case). Non-blocking async taskkill.   if (pid) {-    try {-      if (process.platform === 'win32') {-        execSync(`taskkill /F /T /PID ${pid}`, { stdio: 'ignore' });-      } else {-        process.kill(pid, 'SIGKILL');-      }-      console.log(`Force-killed Chrome PID ${pid} for profile "${profileName}"`);-    } catch (e) {-      // PID already gone — that's fine-    }+    if (_isWin) { await _taskkillPids([pid]); }+    else { try { process.kill(pid, 'SIGKILL'); } catch (e) {} }+    console.log(`Force-killed Chrome PID ${pid} for profile "${profileName}"`);   } -  // v1.9.78 (wiki issue #16) — REAP ORPHANED / CRASHED RENDERERS. taskkill /T kills the process-  // TREE, but a crashed renderer can be reparented (orphaned) and survive the tree kill, lingering-  // at 8-12 MB. They are unambiguous to find: any chrome/CfT process still launched with THIS-  // profile's --user-data-dir. Scoping the sweep to the profile dir makes it impossible to touch-  // the user's real browser or another session's window.-  if (process.platform === 'win32') {-    // v1.9.81 — MUST BE NON-BLOCKING. The v1.9.78 reap ran this CIM query with execFileSync (up to-    // 8s) SYNCHRONOUSLY on the close path, freezing Node's ENTIRE event loop for the duration.-    // PowerShell is flaky on some boxes (CIM can be slow; a bad profile drops PS to interactive and-    // it hits the timeout), so every window close could freeze the bridge for seconds. Repeated-    // freezes make the bridge unresponsive to AD's health check, which then restarts it ("the pup-    // bridge crashed") — self-inflicted, seen live after closing several test windows in a row.-    // Fire-and-forget async: the close returns immediately and the sweep runs in the background-    // without ever blocking the loop.-    const dir = path.join(PROFILES_DIR, profileName);-    const ps = `Get-CimInstance Win32_Process | Where-Object { $_.Name -match 'chrome|msedge' -and $_.CommandLine -match 'user-data-dir' -and $_.CommandLine -like '*${dir.replace(/'/g, "''")}*' } | ForEach-Object { $_.ProcessId }`;-    try {-      execFile('powershell.exe', ['-NoProfile', '-ExecutionPolicy', 'Bypass', '-Command', ps],-        { encoding: 'utf8', timeout: 8000, windowsHide: true }, (err, out) => {-          if (err) return; // best-effort; a timed-out/failed sweep must never affect the bridge-          const pids = (out || '').split(/\r?\n/).map(x => parseInt(x, 10)).filter(Boolean);-          if (!pids.length) return;-          for (const rp of pids) { try { execFile('taskkill', ['/F', '/PID', String(rp)], { windowsHide: true }, () => {}); } catch (e) {} }-          console.log(`[reap] profile "${profileName}": swept ${pids.length} orphaned/crashed process(es) that survived the tree kill`);-        });-    } catch (e) { /* never let scheduling the reap throw on the close path */ }-  }+  // Then the RELIABLE, AWAITED, VERIFIED profile-scoped reap. taskkill /T kills the parent's tree, but+  // a crashed/backgrounded renderer can be reparented and survive it, lingering — or pegging a full+  // core. This awaited sweep catches those and confirms none remain, which is what makes close() honest.+  const rep = await reapProfileChromes(profileName); -  // Also check if Chrome is still locking the profile dir-  const lockFile = path.join(PROFILES_DIR, profileName, 'SingletonLock');-  try { fs.unlinkSync(lockFile); } catch { /* doesn't exist or can't remove */ }+  // Remove any stale SingletonLock so the next open on this profile starts clean.+  try { fs.unlinkSync(path.join(PROFILES_DIR, profileName, 'SingletonLock')); } catch { /* absent/locked */ }+  return rep; }  // ─── Profile session-restore prevention ──────────────────────────────────@@ -6230,12 +6269,13 @@ const server = http.createServer(async (req, res) => {           unregisterPupAumid(_closingAppId).catch(() => {}); // v1.9.2/#207 cleanup (appId captured pre-delete)            // Decrement refCount; kill browser if no sessions left using this profile+          let _reap = { killed: 0, remaining: 0 };           const be = browsers.get(profileName);           if (be) {             be.refCount = Math.max(0, be.refCount - 1);             if (be.refCount <= 0) {               console.log(`No more sessions using profile "${profileName}" — killing browser`);-              await killBrowserProcess(be.browser, profileName);+              _reap = await killBrowserProcess(be.browser, profileName);               browsers.delete(profileName);             }           }@@ -6244,7 +6284,11 @@ const server = http.createServer(async (req, res) => {             activeSessionId = sessions.size > 0 ? sessions.keys().next().value : null;           }           console.log(`Session "${sessionId}" closed (all tabs). Remaining sessions: ${sessions.size}`);-          sendJSON(res, { success: true, output: JSON.stringify({ ok: true, remaining: sessions.size, activeSession: activeSessionId }) });+          sendJSON(res, { success: true, output: JSON.stringify({+            ok: true, remaining: sessions.size, activeSession: activeSessionId,+            chromeProcsKilled: _reap.killed, chromeProcsRemaining: _reap.remaining,+            ...(_reap.remaining ? { _hint: `WARNING: ${_reap.remaining} chrome/renderer process(es) for this profile SURVIVED teardown (should be 0) and may peg a CPU core. The box may be under heavy load — retry browser_close_window; if it persists it is a reap bug worth reporting.` } : {}),+          }) });           return;         } @@ -8048,12 +8092,13 @@ const server = http.createServer(async (req, res) => {             const otherSessionsOnProfile = [...sessions.entries()].filter(               ([id, s]) => id !== args.sessionId && s.profileName === profileName             );+            let _reap = { killed: 0, remaining: 0 };             try {               if (otherSessionsOnProfile.length === 0) {                 // Only this session uses this Chrome — kill the whole browser.                 const be = browsers.get(profileName);                 if (be) {-                  await killBrowserProcess(be.browser, profileName);+                  _reap = await killBrowserProcess(be.browser, profileName);                   browsers.delete(profileName);                 }               } else {@@ -8075,7 +8120,10 @@ const server = http.createServer(async (req, res) => {               sessionId: args.sessionId,               message: `Session "${args.sessionId}" closed`,               chromeKilled: otherSessionsOnProfile.length === 0,+              chromeProcsKilled: _reap.killed,+              chromeProcsRemaining: _reap.remaining,               remainingSessions: sessions.size,+              ...(_reap.remaining ? { _hint: `WARNING: ${_reap.remaining} chrome/renderer process(es) SURVIVED teardown (should be 0) and may peg a CPU core — retry if it persists.` } : {}),             })});             return;           }@@ -8096,10 +8144,17 @@ const server = http.createServer(async (req, res) => {           activeSessionId = null;           // Belt-and-braces: sweep any orphaned Adom.Pup.* keys now that nothing is live.           try { pruneStaleAumids(); } catch (e) {}+          // Final VERIFIED sweep — with no live session left, reap EVERY pup chrome/renderer still+          // alive (parent or reparented orphan). This is what makes "close all" actually leave zero+          // core-pegging processes behind, and reports the count so the caller knows it's clean.+          const _sweep = await reapOrphanPupChromes();           sendJSON(res, { success: true, output: JSON.stringify({             ok: true,             message: 'All sessions and browsers closed',             remainingSessions: 0,+            chromeProcsKilled: _sweep.killed,+            chromeProcsRemaining: _sweep.remaining,+            ...(_sweep.remaining ? { _hint: `WARNING: ${_sweep.remaining} pup chrome process(es) survived the close-all sweep (should be 0) and may peg CPU — the box may be heavily loaded; retry browser_close.` } : {}),           })});           return;         }@@ -8550,6 +8605,14 @@ server.listen(PORT, BIND_HOST, async () => {   setTimeout(() => { reclaimUnmanagedWindows().catch(() => {}); }, 12000);   setInterval(() => { reclaimUnmanagedWindows().catch(() => {}); }, 90000); +  // v1.9.89: reap ORPHANED pup chrome/renderers — any process under a pup-profiles dir whose profile+  // has no live session (a previous bridge crash left them, or renderers that outlived a close and are+  // pegging a core). Boot sweep runs AFTER recovery has settled so it never kills a window recovery+  // just reconnected; then a slow periodic net. Spares every live session BY PROFILE, so it can only+  // ever hit true orphans — never a session's own renderer children.+  setTimeout(() => { reapOrphanPupChromes().catch(() => {}); }, 15000);+  setInterval(() => { reapOrphanPupChromes().catch(() => {}); }, 180000);+   // Periodic cleanup: every 5 minutes, remove stale session files with dead PIDs   setInterval(async () => {     const files = listSessionFiles();