← Commit history

Crash self-heal: proactively relaunch _lostBrowser sessions after a Chrome crash (signed-in, background, at last URL) with a 2-attempt backoff so a crash-on-load page cannot loop; keep the hung-CDP probe as a second path into the same relaunch. Fixes the frozen-pup-after-crash the v1.9.170 browsers-probe could not catch (entry is already deleted on crash).

John Lauer ·0f536c004d ·3mo ago ·parent cad61fd
4 files changed +66−16
adom-bridge-puppeteer-manifest.json+4−4
@@ -1,10 +1,10 @@ {   "manifest_version": 1,   "name": "puppeteer",-  "version": "1.9.170",-  "url": "https://wiki.adom.inc/download/adom/adom-desktop-puppeteer-bridge/1.9.170/adom-bridge-puppeteer-v1.9.170.zip",-  "sha256": "8831034c2f26a73b46e8d32568328bbe41b4a5d2911599891c740464c669b398",-  "size": 408105,+  "version": "1.9.171",+  "url": "https://wiki.adom.inc/download/adom/adom-desktop-puppeteer-bridge/1.9.171/adom-bridge-puppeteer-v1.9.171.zip",+  "sha256": "5f378f2be0d8d06145e1f1c20f4c38a7f89b65adbc64d05440a8cf05424fb9ea",+  "size": 409244,   "verbPrefixes": [     "browser_",     "desktop_recorder_",
src/BRIDGE_VERSION+1−1
@@ -1 +1 @@-1.9.170+1.9.171
src/bridge.json+1−1
@@ -2,7 +2,7 @@   "manifest_version": 1,   "name": "puppeteer",   "displayName": "Puppeteer (Chrome for Testing + Native)",-  "version": "1.9.170",+  "version": "1.9.171",   "description": "Drives a browser ALREADY on the machine \u2014 installed Chrome, else Microsoft Edge (every Windows PC ships with Edge) \u2014 launched with a FRESH isolated profile, so the common case has NO ~150 MB Chrome-for-Testing download and it never touches the user's real tabs/logins. Binds its local server to LOOPBACK (127.0.0.1) only, so no Windows Firewall 'allow Node through the firewall' prompt ever pops. Chrome for Testing is fetched only as a LAST RESORT if the machine has no Chromium browser at all. SPAWN-VERIFIED fallback: pup tries each detected browser (Chrome \u2192 Edge \u2192 CfT) and actually launches it, falling through to the next if one won't start (and handling browsers like Edge that relaunch onto another PID), so a corrupt/partial install never hard-fails an open. The browser that launches is cached as the default (persisted across restarts) so it isn't re-probed each time. On spawn the bridge SELF-WARMS: detects browsers + disk, and only if the box has no Chrome/Edge and no cached CfT does it background-fetch CfT so the first open is instant. browser_use pins a specific browser (chrome | edge | cft | auto); {browser:'chrome',install:true} downloads & installs REAL Google Chrome (user-scoped, no admin) on an Edge-only box. browser_readiness (READ-ONLY) reports which browser will be used + the full candidate list + the default + free disk. Every failure returns a specific errorCode + _hint (chrome_install_no_disk, chrome_download_network, chrome_install_permission, chrome_stable_not_detected, \u2026). Launches with FULL browser permissions by default for full debug (strictPermissions:true opts out). Headful automation: open/close windows+tabs, navigate, screenshot, eval JS, record. pup is a GENERIC, non-profile browser only \u2014 every session is a fresh isolated Chrome/Edge process, never the user's real signed-in browser; profile defaults to sessionId. To drive the user's REAL Chrome/Edge (their profile, cookies, SSO), use the Adom browser EXTENSION's nbrowser_* verbs \u2014 a separate bridge, not pup's job. macOS (v1.9.74+): full parity with mac-native mechanics \u2014 background-by-default is the app-level hide (windows stay drivable/screenshottable while hidden), raise/lower map to unhide+activate/hide, browser_alert_window posts a native toast whose click focuses the window, browser_use {browser:'chrome',install:true} installs Chrome from the official dmg, and taskbar identity/badging remains Windows-only.",   "homepage": "https://wiki.adom.inc/adom/adom-desktop",   "author": "Adom Inc.",
src/server.js+60−10
@@ -9974,16 +9974,33 @@ server.listen(PORT, BIND_HOST, async () => {   // net, so an unbranded window can never persist even if its window-opened event was missed.   setTimeout(() => { reclaimUnmanagedWindows().catch(() => {}); }, 12000);   setInterval(() => { reclaimUnmanagedWindows().catch(() => {}); }, 90000);-  // v1.9.170 — BROWSER-HEALTH WATCHDOG. When the shared Chrome crashes or hangs (measured repeatedly-  // on the Autodesk sign-in flow), the browsers-map entry becomes a DEAD handle: the next verb fails-  // 'Connection closed' and pup LOOKS frozen (John: 'pup seems frozen again'). This proactively probes-  // each browser with a real CDP call; a browser that fails TWO consecutive probes (~30s — a real-  // crash/hang, not a transient slow beat) is discarded, its zombie Chromes reaped, its sessions-  // marked _lostBrowser so the next verb relaunches them, and the recovery is REPORTED via the cleanup-  // queue. So pup self-heals instead of stranding the user, and the next open relaunches clean.-  const _healthStrikes = new Map();+  // v1.9.171 — CRASH SELF-HEAL. The shared Chrome dies two ways on heavy/hostile pages (measured+  // repeatedly on the Autodesk sign-in flow), and pup must recover from BOTH without a human:+  //+  //   (1) CRASHED — Chrome's process exits. puppeteer fires 'disconnected', handleBrowserDisconnect+  //       DELETES the browsers entry and marks each affected session `_lostBrowser`. So by the time+  //       any watchdog runs the browsers map is already EMPTY — probing it finds nothing (this is why+  //       the v1.9.170 browsers-probe never fired). The real signal is the orphaned `_lostBrowser`+  //       sessions, and the real fix is to RELAUNCH them from disk. On the one durable profile they+  //       come back SIGNED IN at their last URL, in the background frame (no foreground steal).+  //   (2) HUNG — Chrome is alive but its CDP socket stops answering (no 'disconnected' fires, so the+  //       entry LINGERS as a dead handle; the next verb returns 'Connection closed' and passive verbs+  //       return null → pup LOOKS frozen, John: 'pup seems frozen again'). A real CDP probe with a+  //       timeout catches this; two consecutive misses (~30s, not a transient slow beat) = discard ++  //       reap + mark sessions lost, which then feed into the same (1) relaunch path.+  //+  // Backoff is mandatory: a page that crashes-Chrome-on-load (Kepler) would otherwise spin an endless+  // relaunch loop and cook the laptop. Cap auto-relaunches per session; after the cap, hands off and+  // tells the user via the cleanup queue that the page itself looks unstable.+  const _healthStrikes = new Map();       // profileName -> consecutive hung-probe misses+  const _healRelaunch = new Map();        // sessionId -> { count, last, gaveUp }+  const _healingIds = new Set();          // sessionIds with a relaunch in flight (guards the 15s re-tick)+  const MAX_HEAL_ATTEMPTS = 2;+  const HEAL_COOLDOWN_MS = 5 * 60 * 1000;   setInterval(async () => {     if (process.platform !== 'win32') return;+    // (2) HUNG detection — probe each LIVE browser entry; on 2 misses, tear it down so its sessions+    // become `_lostBrowser` and fall into the relaunch path below.     for (const [profileName, entry] of [...browsers]) {       let alive = false;       try {@@ -9998,13 +10015,46 @@ server.listen(PORT, BIND_HOST, async () => {       _healthStrikes.set(profileName, strikes);       if (strikes < 2) { console.log(`[health] "${profileName}" failed a liveness probe (strike ${strikes}/2) — watching`); continue; }       _healthStrikes.delete(profileName);-      console.log(`[health] "${profileName}" DEAD/HUNG (2 strikes) — discarding + recovering so pup self-heals`);-      recordCleanup(`pup detected the "${profileName}" Chrome had crashed or hung and cleaned it up — your window(s) will relaunch signed-in on the next open (logins are safe on disk).`);+      console.log(`[health] "${profileName}" HUNG (2 strikes) — discarding the dead handle so its windows can relaunch`);       try { await entry.browser.close(); } catch (e) {}       browsers.delete(profileName);       try { await killOrphanChromesForProfile(profileName); } catch (e) {}       for (const [, sess] of sessions) { if (sess.profileName === profileName) sess._lostBrowser = true; }     }+    // (1) CRASH recovery — proactively relaunch every `_lostBrowser` session (from a crash OR from the+    // hung-teardown above) so the user's windows come back on their own, signed in, where they left off.+    for (const [sid, sess] of [...sessions]) {+      if (!sess._lostBrowser || _healingIds.has(sid)) continue;+      const rec = _healRelaunch.get(sid) || { count: 0, last: 0, gaveUp: false };+      const now = Date.now();+      if (rec.count >= MAX_HEAL_ATTEMPTS) {+        if (now - rec.last < HEAL_COOLDOWN_MS) {+          if (!rec.gaveUp) {+            rec.gaveUp = true; _healRelaunch.set(sid, rec);+            console.log(`[heal] "${sid}" exceeded ${MAX_HEAL_ATTEMPTS} relaunch attempts — backing off ${Math.round(HEAL_COOLDOWN_MS/60000)}m (the page looks like it destabilizes Chrome)`);+            recordCleanup(`pup tried to bring your "${sid}" window back after a Chrome crash but it kept crashing on load — that page may be destabilizing Chrome, so pup has stopped retrying it. Reopen it manually when you're ready; your logins are safe on disk.`);+          }+          continue;+        }+        rec.count = 0; rec.gaveUp = false;   // cooldown elapsed — allow a fresh round+      }+      _healingIds.add(sid);+      rec.count += 1; rec.last = now; _healRelaunch.set(sid, rec);+      console.log(`[heal] "${sid}" lost to a Chrome crash — proactively relaunching (attempt ${rec.count}/${MAX_HEAL_ATTEMPTS}) at its last URL`);+      try {+        const r = await recoverOrRelaunchSession({ sessionId: sid });+        if (r && r.session) {+          console.log(`[heal] "${sid}" relaunched OK — window is back signed-in`);+          recordCleanup(`pup auto-recovered your "${sid}" window after a Chrome crash — it relaunched in the background, signed in, at where it left off (logins are safe on disk).`);+        } else {+          console.log(`[heal] "${sid}" relaunch did not resolve a live session yet — will retry next tick`);+        }+      } catch (e) {+        console.log(`[heal] "${sid}" relaunch failed: ${e.message} — will retry next tick`);+      } finally {+        _healingIds.delete(sid);+      }+    }   }, 15000);   // v1.9.166: taskbar-progress reconciler — clear any bar that is lit but shouldn't be.   setInterval(async () => {