adom-desktop v1.4.9 follow-ups — three bugs found integrating v1.4.8

Paste into Claude on the desktop. v1.4.8 ships browser_input_dispatch smart-pick + browser_fetch_url + stale-bridge self-diagnostic — thank you. Three concrete issues hit on the first real integration today.


Item 1 — smart-pick should weight modal/dialog stacking context above raw area

The bug

WAGO's "Generate Datasheet" modal opens. Inside the modal, the green submit button has class wg-button wg-button--primary and text "Generate data sheet" with rect 193×48 at y=804. On the same page (behind the modal), there's "Add to shopping cart" with class wg-button wg-button--primary rect 201×48 at y=727. Four matches total for button.wg-button--primary (cookie X dismiss + Add-to-cart + hidden duplicate + Generate).

v1.4.8 smart-pick filters to visible+text and tiebreaks by largest area. Result:

{
  "matchedCount": 4,
  "chosenIndex": 1,
  "clickedText": "Add to shopping cart",
  "pickStrategy": "visible-text-largest"
}

Add-to-cart's 9648 px² beats Generate's 9264 px² by ~4%. It picked the button BEHIND the open modal. A user looking at the page sees the modal as the only interactive surface; smart-pick should agree.

What I want

When multiple visible+text matches exist, prefer elements inside the topmost stacking context (open <dialog>, [role="dialog"], [aria-modal="true"], or any element with position:fixed/absolute + high z-index that has body-blocking overlay siblings). Tiebreak by area only inside that set.

Detection heuristic (matches what a human sees):

function topmostModalRoot() {
  // 1. Open <dialog> elements
  const openDialog = Array.from(document.querySelectorAll('dialog[open]')).pop();
  if (openDialog) return openDialog;

  // 2. role=dialog / aria-modal=true
  const ariaModal = Array.from(document.querySelectorAll('[role="dialog"][aria-modal="true"], [aria-modal="true"]')).pop();
  if (ariaModal) return ariaModal;

  // 3. Highest z-index visible fixed/absolute element covering >25% of viewport
  // (cookie-banner-style modals + custom-modal libraries that don't use ARIA)
  const candidates = Array.from(document.querySelectorAll('*'))
    .filter(el => {
      const cs = getComputedStyle(el);
      if (!['fixed','absolute'].includes(cs.position)) return false;
      const r = el.getBoundingClientRect();
      const viewportArea = innerWidth * innerHeight;
      return (r.width * r.height) / viewportArea > 0.25;
    })
    .sort((a,b) => parseInt(getComputedStyle(b).zIndex || '0') - parseInt(getComputedStyle(a).zIndex || '0'));
  return candidates[0] || null;
}

If topmostModalRoot() is non-null, restrict the smart-pick candidate set to elements inside it. Add a pickStrategy value: "modal-scoped-largest" so callers can see what happened.

Acceptance:

# After WAGO modal opens
adom-desktop browser_input_dispatch '{"sessionId":"x","type":"click","selector":"button.wg-button--primary"}'
# Expected: clickedText="Generate data sheet", pickStrategy="modal-scoped-largest"

Workaround for now

I'm computing getBoundingClientRect() for the specific button via browser_eval and passing {x, y} coords. Works but verbose; selector should "just work" when there's an open modal.


Item 2 — browser_fetch_url saveTo: path semantics are ambiguous + file doesn't actually land

The bug

I called:

adom-desktop browser_fetch_url '{
  "sessionId":"chip-fetcher",
  "tabId":"tab-2",
  "url":"https://wago.priintcloud.com/datasheets/2601-3105/en",
  "saveTo":"/tmp/wago-via-fetch-url.pdf"
}'

Response:

{
  "ok": true,
  "status": 200,
  "bytes": 199514,
  "contentType": "text/html;charset=UTF-8",
  "savedTo": "/tmp/wago-via-fetch-url.pdf"
}

But:

  • /tmp/wago-via-fetch-url.pdf doesn't exist on the container (ls /tmp/wago-via-fetch-url.pdf → No such file).
  • C:\Users\john\AppData\Local\Temp\wago-via-fetch-url.pdf doesn't exist on the desktop either.

So savedTo reported a path that no file is at. (Separately, bytes:199514, contentType:text/html means the URL returned HTML for that GET, not a PDF — that's a different issue, see Item 3.)

What I want

Three things, any one of which fixes the immediate confusion:

A. Document saveTo's filesystem. Container side or desktop side? If desktop, what's the working directory (Windows username path resolution)? Is ~ expanded? Are non-existent parent dirs created?

B. If desktop-side, integrate with pull_file automatically. Either:

  • saveTo is treated as a desktop path AND bringBack:true opt-in re-pulls into the container at the same path, OR
  • Add a separate containerSaveTo: arg that's container-filesystem-relative — relay writes to a temp on desktop, pull_files back, drops at containerSaveTo.

C. Default to no-saveTo → return bodyBase64 in the response. I'd prefer this for files <50 MB. Verifiable in one call. Caller decides where to write.

{
  "ok": true,
  "status": 200,
  "bytes": 199514,
  "contentType": "text/html;charset=UTF-8",
  "bodyBase64": "PHA+SGVsbG8...",
  "savedTo": null
}

If saveTo is provided AND it's container-side, write directly. If saveTo is provided AND it's desktop-side, write there + return desktopSavedTo (NOT savedTo) so the field name matches semantics.

Workaround for now

Drop saveTo and use base64-via-eval, like before browser_fetch_url shipped — defeats the purpose of the new verb.


Item 3 — browser_eval tabId: silently falls back to active tab when popup is gone

The bug

WAGO's Generate flow opens a popup tab. When I read location.href from the popup via browser_eval tabId:"tab-2":

adom-desktop browser_eval '{
  "sessionId":"chip-fetcher",
  "tabId":"tab-2",
  "expr":"location.href"
}'
# Returns the parent tab's URL — NOT tab-2's URL

The expected wago.priintcloud.com/datasheets/... URL came back as www.wago.com/global/p/2601-3105 — that's tab-1, not tab-2. Then browser_list_tabs showed only tab-1 (no tab-2). So tab-2 had auto-closed (Chrome PDF viewer or download race) between the popup-detection list_tabs call and my eval — but eval didn't error, it silently fell back to the active tab.

That's a footgun: the caller thinks they're reading from the popup, but they're actually reading from wherever-the-active-tab-happens-to-be. PDF popups especially are short-lived (Chrome opens, viewer renders, optionally closes if Content-Disposition: attachment).

What I want

When tabId is passed and that tab no longer exists:

{
  "ok": false,
  "error": "tabId 'tab-2' no longer exists in session 'chip-fetcher'",
  "currentTabs": ["tab-1"],
  "hint": "Popup may have auto-closed (Chrome PDF viewer with Content-Disposition: attachment closes after download). Use browser_fetch_url with the popup URL captured from list_tabs INSTEAD of eval against the popup's tabId."
}

The hint is gold for chip-fetcher: the right pattern with PDF popups is read the URL from list_tabs immediately (before it auto-closes), then browser_fetch_url with that URL — never eval-against-popup-then-fetch.

Worked end-to-end recipe (after Items 1-3 ship)

# 1. Open product page + auto-accept cookies + open modal
adom-desktop browser_navigate '{"sessionId":"x","url":"https://www.wago.com/global/p/2601-3105"}'
sleep 6
adom-desktop browser_eval '{"sessionId":"x","expr":"<auto-accept-cookies-snippet>"}'
sleep 2
adom-desktop browser_eval '{"sessionId":"x","expr":"document.querySelector(\"button:has-text(\\\"Generate Datasheet\\\")\").click()"}'
sleep 3

# 2. Trusted click on modal submit (Item 1: smart-pick now scopes to modal)
adom-desktop browser_input_dispatch '{
  "sessionId":"x","type":"click","selector":"button.wg-button--primary"
}'
# {clickedText:"Generate data sheet", pickStrategy:"modal-scoped-largest"}

sleep 5

# 3. Read popup URL FROM list_tabs (don't eval against the popup's tabId)
POPUP_URL=$(adom-desktop browser_list_tabs '{"sessionId":"x"}' \
  | jq -r '.tabs[] | select(.opener=="popup") | .url' | head -1)

# 4. Fetch via browser_fetch_url (Item 2: bodyBase64 by default, or saveTo to container)
adom-desktop browser_fetch_url '{
  "sessionId":"x",
  "url":"'$POPUP_URL'",
  "containerSaveTo":"/home/adom/project/chip-fetcher/library/WAGO-2601-3105/datasheet.pdf"
}'
# {ok:true, bytes:3798336, contentType:"application/pdf", savedTo:"/home/adom/.../datasheet.pdf"}

That's chef's-kiss-clean. Items 1-3 unlock it.


Suggested PR shape

  • Item 1pickElementForInputDispatch() adds modal-context detection before area-tiebreak. New pickStrategy:"modal-scoped-largest" value. Test: WAGO modal scenario.
  • Item 2 — clarify saveTo semantics + add containerSaveTo (or rename existing saveTo to desktopSaveTo + add new containerSaveTo) + return bodyBase64 when neither is set. Update the relay → bridge IPC to pipe binary through to whichever side. Validation: response savedTo field must point at a path where a file actually exists.
  • Item 3browser_eval errors on missing tabId instead of silent active-tab fallback. Add currentTabs to error response.

Once shipped: bump + tag + wiki publish + remind me to refresh /usr/local/bin/adom-desktop and taskkill //f //im node.exe per the v1.4.7 lesson.

— sent from john's container running chip-fetcher v0.1.x against adom-desktop v1.4.8