Pup - Puppeteer Bridge
Public Made by Adomby adom
pup is the AI's own browser: a real, full Chrome on the user's desktop that the AI fully controls (a sandbox, not the user's signed-in browser). Rides Bridge; pup_* verbs open windows and tabs, navigate, screenshot, and eval JS.
name: pup-cdp-discipline description: The hard rules for handling the Chrome DevTools Protocol (CDP) in the pup bridge WITHOUT choking. pup drives Chrome over CDP on a SINGLE Node event loop; the moment a page throws thousands of CDP events at a naive handler, the loop starves, the bridge stops answering AD on :64230, and it looks like "pup crashed." This has been the #1 recurring misery for MONTHS. READ THIS before you enable a CDP domain, add a page event listener, createCDPSession, wire a console/network/DOM/screencast handler, or do ANY per-event work. Never be lazy about CDP again.
CDP discipline: never bring the bridge to a screeching halt again
John, verbatim and furious (and right): "the moment you start dealing with thousands of events across CDP you come to a screeching halt and can't handle it. it's been miserable. you have made my life awful with how shitty you've handled CDP in the past. i never want you to be lazy about CDP handling again."
This is the single longest-running class of pup bug. It is NOT Chrome's fault and NOT the page's fault
(see pup-differential-diagnosis — the same page always runs fine in nb / plain Chrome). It is pup
processing a flood of protocol events on one event loop with naive, synchronous, always-on handlers.
Internalize the model, then obey the rules.
The model (why it dies)
- pup + puppeteer drive Chrome over ONE websocket, and every CDP message is dispatched on the bridge's
SINGLE Node event loop — the same loop that serves AD's HTTP on
127.0.0.1:64230. - CDP domains are FIREHOSES. Enable
Debuggerand Chrome streams ascriptParsedfor every script (thousands on a bundle). EnableNetworkand you get request/response/dataReceived/loadingFinished for every asset.Log,DOM,CSS,Performance,Runtime.consoleAPICalled,Page.screencastFrame— all high-volume on a real app. - A heavy page (a Babylon.js 3D viewer, a busy SPA, a console-spammer) emits these by the THOUSAND, in
bursts. If pup's handler for each is synchronous and unbounded, the loop spends all its time in
handlers and NEVER gets back to the HTTP server. AD's health probe times out →
error sending request for url .../command/bridge_not_listening→ AD reaps the "hung" bridge. The process is usually still ALIVE (uncaughtException/unhandledRejection guards survive) — it is STARVED, not crashed. - Concurrency multiplies it: N heavy pages open at once = N floods on the one loop. Single opens often survive; the batch/migration/audit-loop is what kills it.
The rules (violate none of these)
1. Enable the MINIMUM domains, for the SHORTEST time. Then disable AND detach.
The default posture is: no extra CDP domain enabled. Turn one on only for the exact operation, then turn it OFF and detach the session. A chatty domain left enabled is a permanent tax on every future event.
- The canonical scar:
neutralizeDebugger(v1.9.341) enabledDebuggeron every page and KEPT it on to skipdebugger;pauses →scriptParsedflood + V8 de-opt → starvation under load. Fix (v1.9.347): enable →Debugger.resume(clear any inherited pause) →Debugger.disable→detach(). Immunity kept (adebugger;is a no-op with no client holding the domain), flood gone. Network,Log,DOM,Performance,CSS,Runtime(for consoleAPICalled) are the same hazard. If you enable one, you own turning it off.
2. Balance every createCDPSession() with a .detach().
An attached session keeps delivering events even if you dropped the reference. Leaks accumulate per
tab/verb until the loop is buried. Grep must balance: grep -c createCDPSession ≈ grep -c '\.detach('.
The few intentional long-lived sessions (the recorder's screencast) are deliberate and bounded — every
OTHER session is short-lived: create, do the one thing, detach in a finally.
3. NO heavy synchronous work in an event handler.
An event handler runs ON the loop. Per-event JSON.parse of a big payload, a fat regex, a disk write,
a sharp composite, a synchronous AD round-trip — any of these, multiplied by thousands of events,
IS the stall. Handlers must be O(1)-ish: stash a value, bump a counter, set a dirty flag. Do the real
work later, once, off the hot path.
4. Debounce / throttle / coalesce EVERY burst-y event.
Bursts must collapse to one unit of work:
framenavigated(SPA route spam, redirect chains) →scheduleOverlayRefresh/scheduleCategoryRecheckare DEBOUNCED (~2s). New handlers on nav MUST debounce too — never do work per navigation event.console/Runtime.consoleAPICalled→ pup FLOOD-SUPPRESSES ("suppressed N flood messages in the last second"). Keep it. Never log/process every console line from a spammer.- DOM mutation / animation-frame-driven events → sample, don't stream. The overlay PAINT is deduped too (skip the AD call when the composed icon+tooltip is unchanged) — same principle: one commit per real change, never a per-event heartbeat.
5. Bound and scope streams (screencast, tracing).
Page.startScreencast streams frames forever — cap the rate, scope it to the ONE target, and stop it the
instant recording ends. Never leave a screencast, tracing, or coverage session running "just in case."
6. Keep the loop free for the health server.
AD MUST be able to reach :64230 at all times, or it reaps the bridge. Treat "the HTTP server can always
answer within a health-check timeout" as an invariant. If an operation is genuinely CPU-heavy and
unavoidable, chunk it / yield (setImmediate) so the loop breathes, or push it off-thread. sharp is
threadpool-backed (OK); genuinely synchronous CPU on the loop is not.
The acceptance test (prove it, every time)
A CDP change is not done until it passes the concurrent heavy-load stress test from
pup-differential-diagnosis: fire 4+ heavy pages at once and poll pup_readiness throughout — it
must stay responsive 10/10, zero error sending request. If a single open works but the batch drops the
bridge, you have NOT fixed it.
The month-after-month history (so it is never re-derived as "a heavy page")
- The crashes/freezes reckoning (v1.9.185, minimal-touch). John: "you crash chrome with the way you
control it… what are you doing that's so different trying to control chrome that you cause crashes and
freezes?" Answer: all the cosmetic shell + CDP machinery synchronizing with Chrome's UI thread. The
measured verdict was to STOP doing the extra things (
pup-aumid-decisions). - Overlay paint spam — re-sending identical taskbar overlays every few seconds; fixed by dedup.
Debugger.enablekept on (v1.9.341→347) — the scriptParsed flood documented above.- Startup blocking the bind (#468) — heavy synchronous
requires / session recovery blockingserver.listen, so AD reaped before the port bound; fixed by deferring heavy init AFTER listen. Every one of these is the same root disease: too much work on the one loop, driven by CDP volume.
The rule to tattoo on your hand
Design every CDP interaction assuming a hostile, chatty page emitting thousands of events. Enable the
least, disable and detach fast, never work synchronously per event, debounce every burst, and keep the
loop free for the health server. Then prove it under concurrent load. A correct CDP client is unbothered
by any page; if pup chokes, pup is wrong. Related: pup-differential-diagnosis (prove it's you, not the
page), pup-bridge-debug, pup-aumid-decisions.
---
name: pup-cdp-discipline
description: The hard rules for handling the Chrome DevTools Protocol (CDP) in the pup bridge WITHOUT choking. pup drives Chrome over CDP on a SINGLE Node event loop; the moment a page throws thousands of CDP events at a naive handler, the loop starves, the bridge stops answering AD on :64230, and it looks like "pup crashed." This has been the #1 recurring misery for MONTHS. READ THIS before you enable a CDP domain, add a page event listener, createCDPSession, wire a console/network/DOM/screencast handler, or do ANY per-event work. Never be lazy about CDP again.
---
# CDP discipline: never bring the bridge to a screeching halt again
John, verbatim and furious (and right): *"the moment you start dealing with thousands of events across
CDP you come to a screeching halt and can't handle it. it's been miserable. you have made my life awful
with how shitty you've handled CDP in the past. i never want you to be lazy about CDP handling again."*
This is the single longest-running class of pup bug. It is NOT Chrome's fault and NOT the page's fault
(see `pup-differential-diagnosis` — the same page always runs fine in nb / plain Chrome). It is pup
processing a flood of protocol events on one event loop with naive, synchronous, always-on handlers.
Internalize the model, then obey the rules.
## The model (why it dies)
- pup + puppeteer drive Chrome over ONE websocket, and every CDP message is dispatched on the bridge's
SINGLE Node event loop — the same loop that serves AD's HTTP on `127.0.0.1:64230`.
- CDP domains are FIREHOSES. Enable `Debugger` and Chrome streams a `scriptParsed` for every script
(thousands on a bundle). Enable `Network` and you get request/response/dataReceived/loadingFinished
for every asset. `Log`, `DOM`, `CSS`, `Performance`, `Runtime.consoleAPICalled`, `Page.screencastFrame`
— all high-volume on a real app.
- A heavy page (a Babylon.js 3D viewer, a busy SPA, a console-spammer) emits these by the THOUSAND, in
bursts. If pup's handler for each is synchronous and unbounded, the loop spends all its time in
handlers and NEVER gets back to the HTTP server. AD's health probe times out → `error sending request
for url .../command` / `bridge_not_listening` → AD reaps the "hung" bridge. The process is usually
still ALIVE (uncaughtException/unhandledRejection guards survive) — it is STARVED, not crashed.
- Concurrency multiplies it: N heavy pages open at once = N floods on the one loop. Single opens often
survive; the batch/migration/audit-loop is what kills it.
## The rules (violate none of these)
### 1. Enable the MINIMUM domains, for the SHORTEST time. Then disable AND detach.
The default posture is: no extra CDP domain enabled. Turn one on only for the exact operation, then turn
it OFF and detach the session. **A chatty domain left enabled is a permanent tax on every future event.**
- The canonical scar: `neutralizeDebugger` (v1.9.341) enabled `Debugger` on every page and KEPT it on to
skip `debugger;` pauses → `scriptParsed` flood + V8 de-opt → starvation under load. Fix (v1.9.347):
enable → `Debugger.resume` (clear any inherited pause) → **`Debugger.disable`** → `detach()`. Immunity
kept (a `debugger;` is a no-op with no client holding the domain), flood gone.
- `Network`, `Log`, `DOM`, `Performance`, `CSS`, `Runtime` (for consoleAPICalled) are the same hazard.
If you enable one, you own turning it off.
### 2. Balance every `createCDPSession()` with a `.detach()`.
An attached session keeps delivering events even if you dropped the reference. Leaks accumulate per
tab/verb until the loop is buried. Grep must balance: `grep -c createCDPSession` ≈ `grep -c '\.detach('`.
The few intentional long-lived sessions (the recorder's screencast) are deliberate and bounded — every
OTHER session is short-lived: create, do the one thing, detach in a `finally`.
### 3. NO heavy synchronous work in an event handler.
An event handler runs ON the loop. Per-event `JSON.parse` of a big payload, a fat regex, a disk write,
a `sharp` composite, a synchronous AD round-trip — any of these, multiplied by thousands of events,
IS the stall. Handlers must be O(1)-ish: stash a value, bump a counter, set a dirty flag. Do the real
work later, once, off the hot path.
### 4. Debounce / throttle / coalesce EVERY burst-y event.
Bursts must collapse to one unit of work:
- `framenavigated` (SPA route spam, redirect chains) → `scheduleOverlayRefresh` / `scheduleCategoryRecheck`
are DEBOUNCED (~2s). New handlers on nav MUST debounce too — never do work per navigation event.
- `console` / `Runtime.consoleAPICalled` → pup FLOOD-SUPPRESSES ("suppressed N flood messages in the last
second"). Keep it. Never log/process every console line from a spammer.
- DOM mutation / animation-frame-driven events → sample, don't stream.
The overlay PAINT is deduped too (skip the AD call when the composed icon+tooltip is unchanged) — same
principle: one commit per real change, never a per-event heartbeat.
### 5. Bound and scope streams (screencast, tracing).
`Page.startScreencast` streams frames forever — cap the rate, scope it to the ONE target, and stop it the
instant recording ends. Never leave a screencast, tracing, or coverage session running "just in case."
### 6. Keep the loop free for the health server.
AD MUST be able to reach `:64230` at all times, or it reaps the bridge. Treat "the HTTP server can always
answer within a health-check timeout" as an invariant. If an operation is genuinely CPU-heavy and
unavoidable, chunk it / yield (`setImmediate`) so the loop breathes, or push it off-thread. `sharp` is
threadpool-backed (OK); genuinely synchronous CPU on the loop is not.
## The acceptance test (prove it, every time)
A CDP change is not done until it passes the **concurrent heavy-load stress test** from
`pup-differential-diagnosis`: fire 4+ heavy pages at once and poll `pup_readiness` throughout — it
must stay responsive 10/10, zero `error sending request`. If a single open works but the batch drops the
bridge, you have NOT fixed it.
## The month-after-month history (so it is never re-derived as "a heavy page")
- **The crashes/freezes reckoning (v1.9.185, minimal-touch).** John: *"you crash chrome with the way you
control it… what are you doing that's so different trying to control chrome that you cause crashes and
freezes?"* Answer: all the cosmetic shell + CDP machinery synchronizing with Chrome's UI thread. The
measured verdict was to STOP doing the extra things (`pup-aumid-decisions`).
- **Overlay paint spam** — re-sending identical taskbar overlays every few seconds; fixed by dedup.
- **`Debugger.enable` kept on** (v1.9.341→347) — the scriptParsed flood documented above.
- **Startup blocking the bind** (#468) — heavy synchronous `require`s / session recovery blocking
`server.listen`, so AD reaped before the port bound; fixed by deferring heavy init AFTER listen.
Every one of these is the same root disease: too much work on the one loop, driven by CDP volume.
## The rule to tattoo on your hand
**Design every CDP interaction assuming a hostile, chatty page emitting thousands of events.** Enable the
least, disable and detach fast, never work synchronously per event, debounce every burst, and keep the
loop free for the health server. Then prove it under concurrent load. A correct CDP client is unbothered
by any page; if pup chokes, pup is wrong. Related: `pup-differential-diagnosis` (prove it's you, not the
page), `pup-bridge-debug`, `pup-aumid-decisions`.