← Commit history
testing-the-bridge: the daily 82-check verb battery (found 3 real bugs on day one)
4 files changed
+573
dev-skills/testing-the-bridge/SKILL.mdadded+66@@ -0,0 +1,66 @@+---+name: testing-the-bridge+description: "DEV skill: the DAILY end-to-end verb battery for the nb bridge - tools/verb-battery.mjs exercises ALL 81 nbrowser_* verbs through the adom-bridge CLI against a real desktop and real signed-in browser, exactly as users drive them, using the fixture page + sample files in demo/. Read this to run the daily test, interpret a red run, add coverage for a new verb (the CI gate FORCES a coverage decision), or manage the quarantine list. Built 2026-08-14 for the public release; its first three runs found three real product bugs."+---+Parent skill: **adom-native-browser** (maintainers). Companions: developing-the-bridge, publishing-the-bridge.++# The daily verb battery++```bash+node tools/verb-battery.mjs --target AdomLapper --profile chrome:[email protected]+# exit 0 = nothing NEW is broken. Report: tools/verb-battery-report.json (per-verb PASS/FAIL/mode/note)+```++One command, ~4 minutes, end-to-end: container -> relay -> ab -> bridge -> native host -> extension+-> a real window in the real signed-in profile. It opens ONE registered session (background, badged),+drives the fixture page through every input/read/tab/record/download verb, verifies every guard still+refuses, closes with zero litter, and self-checks litter via `owned` + the os inventory.++## The four coverage modes (every verb MUST have one - the gate enforces it)++| mode | meaning |+|---|---|+| `live` | executed for real, strict assertions on observable effects (DOM changed, file landed, mp4 exists) |+| `guard` | executed expecting a SPECIFIC refusal. Guards are the product: a guard that stops refusing is a WORSE regression than a feature that stops working. Currently: autologin-without-confirm, close-all-without-confirm, record-picker-without-fallbackConfirmed, force_close-on-a-live-session (owned_by_other) |+| `quarantine:<issue>` | a KNOWN product bug: the test still RUNS every day and prints "KNOWN BUG still present", but does not fail the exit - a red run must always mean something NEW broke. If it prints "NOT REPRODUCED - consider un-quarantining", the bug got fixed: verify, flip to live, close the issue |+| `skip:<reason>` | consciously never auto-run (install_chrome, register_totp, dev_reload, uninstall) - each prints its reason so the hole stays visible |++`tools/check-guard-coverage.mjs` fails the build if any VERB_META verb lacks a COVERAGE entry, so a+new verb cannot ship untested - adding a verb forces the "how do we test this" decision at the PR.++## The fixture (demo/test-harness.html + sample files)++Served live at `https://wiki.adom.inc/blob/app/nb-bridge/demo/test-harness.html`. Every element is+load-bearing: unique/duplicate click targets (ambiguity guard), a LABELED select (nbrowser_select+finds triggers near visible labels - it takes `optionText`, NOT selector+value), a SEGMENTED+spinbutton date widget (nbrowser_set_date's real react-aria target; it TIMES OUT on native+`<input type=date>` - that is contract, not a bug), hover-reveal, delayed element (wait_for),+target=_blank spawn, download + upload sample files (MARKER strings asserted), a fake login form,+confirm()/error buttons. Change the fixture and the battery together - they are one artifact.++## Reading a red run++1. `status`/`ping` failing with an `_ad_note` about a recent respawn = cold start after an ab+ self-update; the warm-up phase should absorb it - if not, extend the poll.+2. `open_window` refusing `profile_mismatch` mid-battery = a stale battery session (crashed prior+ run). The teardown-first phase closes it; since 0.2.6 close_window always FORGETS registration,+ so this self-heals. (Pre-0.2.6 a session pinned to a `pending-*` pseudo-profile was poisoned+ FOREVER - battery finding #3, fixed in the bridge.)+3. A cascade of failures after one early failure usually shares its root - fix the first, rerun.+4. NEVER pass an unverified `--profile`: a typo'd profile races resolution and used to poison the+ sessionId (see 2). The warm-up phase wakes and verifies the profile before anything grades.++## What the first day of battery runs caught (why this exists)++- **nb-bridge#20**: click-by-text intermittently misses under battery pacing (3/3 battery runs,+ 0/3 manual) - dispatch races the finder. Quarantined.+- **nb-bridge#21**: a confirm() raised by a DRIVEN click wedges the page - auto-dismiss does not+ fire, handle_dialog cannot resolve it, evals block until window close. Quarantined.+- **Fixed same-day (0.2.6)**: sessionIds poisoned by pending-* profile pins with no un-poison path.++## Daily operation++Run it from any container with the repo + relay access. For an unattended daily, wire a cron or ask+John's thread to /loop it; escalate to the issue tracker only on NEW failures. After every RELEASE:+run the battery BEFORE announcing (publishing-the-bridge step 7b), plus the manual skips that release+touches (uninstall cascade after any bridge.json change; dev_reload after extension changes).
publish-skills/publishing-the-bridge/SKILL.md+1@@ -72,6 +72,7 @@ Deps: none (core `http`/`net`), so the payload is source-only (no `node_modules` 6. **Land it:** `refresh_bridges {name:"native-browser"}` → verify `nbrowser_readiness` routes and `bridge_check_updates`/`bridge_list` show `liveVersion==<ver>` (no `staleProcess`). *(bridge_check_updates intermittently omits native-browser after a Bridge self-update — trust readiness + bridge_list.)*+7b. **RUN THE VERB BATTERY before announcing** (testing-the-bridge): `node tools/verb-battery.mjs --target <box>` must exit 0 - it catches what the static gates cannot (a verb that parses, declares, describes, and then fails against a real browser). 7. **NEVER `repo rm` a runtime zip while the manifest still points at it.** The old `/files` `.bin` uploads are the dead anti-pattern (they piled up to 22 stale versions once) — do NOT re-add them. If any linger, migrate as a SEQUENCE: release-upload → repoint manifest → verify `/download/` serves → THEN delete the
tools/verb-battery-report.jsonadded+501@@ -0,0 +1,501 @@+{+ "at": "2026-08-14T11:48:18.214Z",+ "target": "AdomLapper",+ "profile": "chrome:[email protected]",+ "pass": 82,+ "fail": 0,+ "results": [+ {+ "verb": "status",+ "mode": "live",+ "ok": true,+ "note": "ext:nbe v0.2.0"+ },+ {+ "verb": "readiness",+ "mode": "live",+ "ok": true,+ "note": "ready:true state:ready"+ },+ {+ "verb": "ping",+ "mode": "live",+ "ok": true,+ "note": "pong"+ },+ {+ "verb": "whoami",+ "mode": "live",+ "ok": true,+ "note": "keys:_hint,_timeoutHint,durationMs,error,output"+ },+ {+ "verb": "profiles",+ "mode": "live",+ "ok": true,+ "note": "4 profiles"+ },+ {+ "verb": "resolve_profiles",+ "mode": "live",+ "ok": true,+ "note": "resolved"+ },+ {+ "verb": "chrome_status",+ "mode": "live",+ "ok": true,+ "note": "chrome installed, phase:ready"+ },+ {+ "verb": "describe",+ "mode": "live",+ "ok": true,+ "note": "describe:81 == VERB_META:81"+ },+ {+ "verb": "activity",+ "mode": "live",+ "ok": true,+ "note": "5 entries"+ },+ {+ "verb": "os_windows",+ "mode": "live",+ "ok": true,+ "note": "16 windows"+ },+ {+ "verb": "wake_profile",+ "mode": "live",+ "ok": true,+ "note": "woke:true"+ },+ {+ "verb": "rescan",+ "mode": "live",+ "ok": true,+ "note": "rescanned"+ },+ {+ "verb": "resync_sessions",+ "mode": "live",+ "ok": true,+ "note": "resynced"+ },+ {+ "verb": "open_window",+ "mode": "live",+ "ok": true,+ "note": "opened hwnd:14223728 (+ registration guard verified)"+ },+ {+ "verb": "pin_session",+ "mode": "live",+ "ok": true,+ "note": "pinned"+ },+ {+ "verb": "use_profile",+ "mode": "live",+ "ok": true,+ "note": "active:chrome:[email protected]"+ },+ {+ "verb": "list_windows",+ "mode": "live",+ "ok": true,+ "note": "my session listed"+ },+ {+ "verb": "window_state",+ "mode": "live",+ "ok": true,+ "note": "1200x860"+ },+ {+ "verb": "switch_window",+ "mode": "live",+ "ok": true,+ "note": "switched to own"+ },+ {+ "verb": "navigate",+ "mode": "live",+ "ok": true,+ "note": "navigated"+ },+ {+ "verb": "reload",+ "mode": "live",+ "ok": true,+ "note": "reloaded"+ },+ {+ "verb": "back",+ "mode": "live",+ "ok": true,+ "note": "went back"+ },+ {+ "verb": "forward",+ "mode": "live",+ "ok": true,+ "note": "went forward"+ },+ {+ "verb": "wait",+ "mode": "live",+ "ok": true,+ "note": "waited 3231ms"+ },+ {+ "verb": "wait_for",+ "mode": "live",+ "ok": true,+ "note": "late element appeared"+ },+ {+ "verb": "keepalive",+ "mode": "live",+ "ok": true,+ "note": "keepalive verb responds"+ },+ {+ "verb": "eval",+ "mode": "live",+ "ok": true,+ "note": "read page title"+ },+ {+ "verb": "evaluate",+ "mode": "live",+ "ok": true,+ "note": "alias works"+ },+ {+ "verb": "screenshot",+ "mode": "live",+ "ok": true,+ "note": "1568x939"+ },+ {+ "verb": "screenshot_full_res",+ "mode": "live",+ "ok": true,+ "note": "saved om-nbrowser-shots\\shot-1786708035773.png"+ },+ {+ "verb": "errors",+ "mode": "live",+ "ok": true,+ "note": "console error captured"+ },+ {+ "verb": "fetch_url",+ "mode": "live",+ "ok": true,+ "note": "fetched sample content"+ },+ {+ "verb": "click",+ "mode": "quarantine",+ "ok": true,+ "note": "KNOWN BUG still present (nb-bridge#20 click-by-text misses under battery pacing)"+ },+ {+ "verb": "double_click",+ "mode": "live",+ "ok": true,+ "note": "dblclick landed"+ },+ {+ "verb": "right_click",+ "mode": "live",+ "ok": true,+ "note": "contextmenu fired"+ },+ {+ "verb": "hover",+ "mode": "live",+ "ok": true,+ "note": "hover menu revealed"+ },+ {+ "verb": "type",+ "mode": "live",+ "ok": true,+ "note": "typed into input"+ },+ {+ "verb": "press_key",+ "mode": "live",+ "ok": true,+ "note": "key dispatched"+ },+ {+ "verb": "select",+ "mode": "live",+ "ok": true,+ "note": "option selected by visible text"+ },+ {+ "verb": "set_date",+ "mode": "live",+ "ok": true,+ "note": "segmented date typed (MDY keydowns)"+ },+ {+ "verb": "input_dispatch",+ "mode": "live",+ "ok": true,+ "note": "low-level dispatch ok"+ },+ {+ "verb": "set_file_input",+ "mode": "live",+ "ok": true,+ "note": "file attached to input"+ },+ {+ "verb": "caption",+ "mode": "live",+ "ok": true,+ "note": "caption painted"+ },+ {+ "verb": "cursor",+ "mode": "live",+ "ok": true,+ "note": "cursor shown"+ },+ {+ "verb": "focus",+ "mode": "live",+ "ok": true,+ "note": "tab selected, no foreground steal"+ },+ {+ "verb": "flash",+ "mode": "live",+ "ok": true,+ "note": "flashed"+ },+ {+ "verb": "flash_stop",+ "mode": "live",+ "ok": true,+ "note": "flash stopped"+ },+ {+ "verb": "flash_mode",+ "mode": "live",+ "ok": true,+ "note": "flashMode:quiet"+ },+ {+ "verb": "taskbar",+ "mode": "live",+ "ok": true,+ "note": "progress on/off"+ },+ {+ "verb": "fullscreen",+ "mode": "live",+ "ok": true,+ "note": "fullscreen roundtrip"+ },+ {+ "verb": "open_tab",+ "mode": "live",+ "ok": true,+ "note": "tab:1743935506"+ },+ {+ "verb": "list_tabs",+ "mode": "live",+ "ok": true,+ "note": "2 tabs"+ },+ {+ "verb": "switch_tab",+ "mode": "live",+ "ok": true,+ "note": "switched"+ },+ {+ "verb": "close_tab",+ "mode": "live",+ "ok": true,+ "note": "tab closed"+ },+ {+ "verb": "record_start",+ "mode": "live",+ "ok": true,+ "note": "rec:rec-808cfe262bb44ec7bf9e6abb1d7e8aec"+ },+ {+ "verb": "record_status",+ "mode": "live",+ "ok": true,+ "note": "{\"_hint\":\"recording started in the BACKGROUND - the user keeps working; do NOT f"+ },+ {+ "verb": "record_stop",+ "mode": "live",+ "ok": true,+ "note": "mp4 produced"+ },+ {+ "verb": "record_list",+ "mode": "live",+ "ok": true,+ "note": "listed"+ },+ {+ "verb": "download",+ "mode": "live",+ "ok": true,+ "note": "downloadId:425"+ },+ {+ "verb": "download_wait",+ "mode": "live",+ "ok": true,+ "note": "landed: C:\\Users\\john\\Downloads\\sample-download (8).txt (cleaned)"+ },+ {+ "verb": "login_state",+ "mode": "live",+ "ok": true,+ "note": "login_state responds"+ },+ {+ "verb": "login_form",+ "mode": "live",+ "ok": true,+ "note": "detected form:undefined"+ },+ {+ "verb": "credentials",+ "mode": "live",+ "ok": true,+ "note": "no secret material in response"+ },+ {+ "verb": "totp",+ "mode": "live",+ "ok": true,+ "note": "refused unknown host, no seed leak"+ },+ {+ "verb": "clear_cookies",+ "mode": "live",+ "ok": true,+ "note": "removed:0"+ },+ {+ "verb": "add_extension_origin",+ "mode": "live",+ "ok": true,+ "note": "idempotent re-add ok"+ },+ {+ "verb": "open_os_window",+ "mode": "live",+ "ok": true,+ "note": "layer-1 open + close_window os path (WM_CLOSE)"+ },+ {+ "verb": "guard_autologin",+ "mode": "guard",+ "ok": true,+ "note": "refused without confirm:true"+ },+ {+ "verb": "guard_record_fallback",+ "mode": "guard",+ "ok": true,+ "note": "picker fallback refused without fallbackConfirmed"+ },+ {+ "verb": "guard_force_close",+ "mode": "guard",+ "ok": true,+ "note": "live-session hwnd refused (owned_by_other) - ownership absolute even via force path"+ },+ {+ "verb": "guard_close_all",+ "mode": "guard",+ "ok": true,+ "note": "close-all demands confirmCloseAll"+ },+ {+ "verb": "profile_block",+ "mode": "live",+ "ok": true,+ "note": "block refused a drive; unblock restored"+ },+ {+ "verb": "profile_unblock",+ "mode": "live",+ "ok": true,+ "note": "driving works again after unblock"+ },+ {+ "verb": "events_and_owned",+ "mode": "live",+ "ok": true,+ "note": "spawn tracked (0 owned.spawned)"+ },+ {+ "verb": "handle_dialog",+ "mode": "quarantine",+ "ok": true,+ "note": "KNOWN BUG still present (nb-bridge#21 confirm() wedges the page)"+ },+ {+ "verb": "cleanup",+ "mode": "live",+ "ok": true,+ "note": "spawns closed (was 0)"+ },+ {+ "verb": "owned",+ "mode": "live",+ "ok": true,+ "note": "exercised inside events_and_owned + close_window"+ },+ {+ "verb": "close_window",+ "mode": "live",+ "ok": true,+ "note": "window closed, zero litter"+ },+ {+ "verb": "install_chrome",+ "mode": "skip",+ "ok": true,+ "note": "starts a real Chrome install on the desktop — exercised manually on fresh boxes"+ },+ {+ "verb": "register_totp",+ "mode": "skip",+ "ok": true,+ "note": "writes a real TOTP seed into the vault — contract covered by totp read-path test"+ },+ {+ "verb": "dev_reload",+ "mode": "skip",+ "ok": true,+ "note": "restarts the SW + native host and is throttled 1/90s — disruptive to live sessions"+ },+ {+ "verb": "uninstall",+ "mode": "skip",+ "ok": true,+ "note": "the ab-2.0.35 cascade hook — deletes host registrations; tested at release time only"+ }+ ]+}
tools/verb-battery.mjs+5@@ -243,6 +243,11 @@ if (import.meta.url === `file://${process.argv[1]}`) { // Idempotency means every run starts from zero. call('nbrowser_close_window', { sessionId: SID, includeSpawned: true }); call('nbrowser_close_window', { sessionId: SID + '-l1' });+ // WARM-UP: a daily run can land seconds after ab auto-respawned the bridge (self-update), before+ // any extension SW has re-dialed. Wake the profile and give it up to 30s - readiness cold-starts+ // are an ab lifecycle fact, not a verb regression.+ call('nbrowser_wake_profile', { profile: PROFILE });+ for (let i = 0; i < 10; i++) { const r = call('nbrowser_readiness'); if (r.ready === true) { console.log(`warm-up: ready after ${i} poll(s)\n`); break; } execFileSync('sleep', ['3']); } const order = ['status','readiness','ping','whoami','profiles','resolve_profiles','chrome_status','describe','activity','os_windows','wake_profile','rescan','resync_sessions', 'open_window','pin_session','use_profile','list_windows','window_state','switch_window', 'navigate','reload','back','forward','wait','wait_for','keepalive',