app

Adom Bridge (macOS)

Public Made by Adom

by adom

Adom Bridge for macOS: the menu-bar daemon that connects AI tools to your Mac.

adom-bridgemacosbridgedesktop-automationmenu-bardaemon
Overview Files Skills Discover Packages Releases Dependencies Issues1 Breadcrumbs Pull Requests Activity
adom-bridge-macos / skills / adom-desktop-window-capture / SKILL.md
Download

name: adom-desktop-window-capture description: >- The complete reference for Adom Bridge screenshots on macOS: every capture verb, the exact JSON response shape, and how the CLI automatically stores each shot on disk (the window/ and screen/ buckets, the ISO-stamped filenames, the single-vs-full/safe pair, and the .json sidecar). Read this to capture an app window without stealing focus, to grab the screen, to understand the Screen Recording permission requirement, or to build a tool (like shotlog) that renders every screenshot shape Adom Bridge can return. Covers desktop_screenshot_window, desktop_screenshot_screen, kicad_screenshot_all, desktop_find_window, desktop_list_windows, desktop_list_monitors, and the exact on-disk sidecar schema.

Adom Bridge screenshots (macOS): full capability, response shape, and on-disk contract

This is the deep reference for everything Adom Bridge can capture on a Mac and everything it writes to disk. Two audiences:

  • An AI driving a GUI app: jump to "The verb map". The one thing to internalize: desktop_screenshot_window {hwnd} captures ONE window by its window id, even when it is behind other windows, without stealing focus.
  • A tool that renders Adom Bridge screenshots (e.g. shotlog): jump to "On-disk storage" and "The sidecar schema". Every shot lands in a predictable bucket with a self-describing <stem>.json next to it; you never have to parse the response envelope, just watch the folder.

1. The verb map: what each capture verb returns

Verb Captures Foreground? Shape
desktop_screenshot_window {hwnd} ONE window by its CGWindowID (screencapture -l) No focus steal; works when occluded one image (full/safe pair when large)
desktop_screenshot_screen {} the MAIN display n/a (whatever is on screen) one image
kicad_screenshot_all {} every open KiCad window (main + dialogs) as separate images No an ARRAY of sibling-window images
browser_screenshot a browser PAGE / viewport (pup bridge, CDP; not a window grab) No one image (page pixels, not window chrome)

Rule of thumb: for one app window, ALWAYS prefer desktop_screenshot_window {hwnd} (resolve the id with desktop_find_window {titleContains}). It captures the window's own content even when other windows cover it, and it never steals focus. Use desktop_screenshot_screen only when you genuinely want "what is on the user's screen right now."

macOS mechanics and limits (all verified in src-tauri/src/screenshot.rs):

  • The hwnd field is the CGWindowID from CGWindowListCopyWindowInfo. Window enumeration (desktop_list_windows, desktop_find_window) lists on-screen windows only: a MINIMIZED window, or one on another Space, does not enumerate and cannot be captured. Occluded-but-on-screen windows are fine.
  • Capture runs through the system screencapture tool (-l <id> per window; -x no shutter sound, -o no drop shadow, PNG output). Both window and screen captures are downscaled at capture time to at most 1568 px on the longest side, so a Retina grab is never shipped raw.
  • App modal dialogs: a sheet attached to a window renders INSIDE that window's capture. A separate dialog window is its own on-screen top-level window: find it with desktop_list_windows / desktop_find_window and capture it by its own id. There is no owned-popup screenshots[] array on macOS (that was a Windows mechanism); kicad_screenshot_all is the one verb that returns a multi-window array.
  • Multi-monitor: desktop_screenshot_screen captures the MAIN display only. There are no {monitor:N} or {perMonitor:true} options on macOS. Enumerate displays with desktop_list_monitors (below).
  • No video recording on macOS. The native recording verbs return a clean recording_unsupported error. Recording a browser TAB is the pup bridge's own CDP feature (browser_record_start), unrelated to this skill.

2. Permission: Screen Recording (required)

macOS gates all window and screen capture behind the Screen Recording TCC permission. Grant it once:

System Settings > Privacy & Security > Screen Recording > enable "Adom Bridge", then relaunch the app.

Symptoms of a missing grant:

  • Captures come back black or empty, or screencapture errors ("Check Screen Recording permission" appears in the error message).
  • Window TITLES are missing: without the permission, kCGWindowName is not populated, so desktop_list_windows falls back to showing the owning APP's name as the title. If every window's title is just the app name, the permission is not granted.

Driving controls in the background (desktop_find_control, desktop_ui_click, desktop_ui_set) additionally needs the Accessibility permission (System Settings > Privacy & Security > Accessibility); a missing grant returns errorCode: ax_not_trusted.

3. The response shape

Every screenshot verb returns the same core envelope. Field by field, as produced by the app plus the CLI's auto-pull enrichment:

{
  "success": true,
  "output": "{...}",          // JSON string: { message, data: { image: <base64 PNG>, format, encoding, sizeBytes } }
  "error": "",

  // The image, on the DESKTOP (host) side: $TMPDIR/adom-desktop-shots/
  "fullPath":  ".../adom-desktop-shots/window-<hwnd>-<ts>.png",   // or a .full.png/.safe.png pair
  "safePath":  ".../adom-desktop-shots/window-<hwnd>-<ts>.png",
  "fullWidth": 1568, "fullHeight": 980,     // capture size (already capped at <=1568)
  "safeWidth": 1400, "safeHeight": 875,     // downscaled to <=resizeMax when the capture exceeded it
  "fullBytesHost": 402931, "safeBytesHost": 88104,
  "resizeMax": 1400,

  // The image, PULLED to the CALLER (container) side by the CLI:
  "fullPathHost": "...", "safePathHost": "...",   // host-path aliases
  "localFullPath": "/home/adom/project/screenshots/adom-desktop/window/2026-08-10T01-14-16.937-986710.png",
  "localSafePath": "/home/adom/project/screenshots/adom-desktop/window/2026-08-10T01-14-16.937-986710.png",
  "localFullExists": true, "localSafeExists": true,
  "savedTo": ".../window/2026-08-10T01-14-16.937-986710.png",   // = the safe copy (fallback full)
  "fullBytes": 402931, "safeBytes": 88104,   // bytes on the CALLER side
  "source": "desktop_screenshot_window",
  "narrative": "Captured and auto-pulled ONE PNG to .../window/ ...",
  "_cacheHint": "...",                        // where shots live + janitor recipe

  "drive": "background",                      // this capture never foregrounded the window
  "_related": "..."                           // sibling verbs
}

Args for desktop_screenshot_window: hwnd (required, from desktop_find_window / desktop_list_windows) and optional resizeMax (default 1400, clamped 64..4096): the longest side of the safe-resized copy.

Clicking what you see: pass SCREEN coordinates to desktop_click / desktop_double_click / desktop_right_click / desktop_hover ({x, y} in points, the same space as desktop_list_windows rects and desktop_list_monitors bounds). The Windows-only image-space contract ({space:"image", shotId} and the coordMap block) is NOT available on macOS: responses carry no coordMap/shotId. To click a control without doing coordinate math at all, prefer the Accessibility path: desktop_find_control {hwnd, name|contains, role?} returns the control's on-screen rect (centerX/centerY), and desktop_ui_click presses it in the background via AXPress.

4. Monitors: desktop_list_monitors

desktop_list_monitors (alias display) is the authoritative display map, via CGDisplay:

{
  "monitors": [
    {
      "index": 0, "primary": true, "device": "CGDisplay-1",
      "bounds":        { "x": 0, "y": 0, "width": 1512, "height": 982 },  // POINTS, same space as clicks + window rects
      "workArea":      { ... },            // == bounds on macOS (menu bar / Dock not excluded)
      "logicalBounds": { ... },            // == bounds
      "pixelSize":     { "width": 3024, "height": 1964 },  // native pixels
      "scale": 2.0,                        // Retina backing scale
      "dpi": 144                           // 72 * scale
    }
  ]
}

Ordering is stable: primary first, then left to right. On macOS everything the input and window verbs speak is POINTS: bounds here, desktop_list_windows rects, and desktop_click {x,y} all share one coordinate space, so no scale math is ever needed. pixelSize/scale exist only to tell you the display is Retina.

5. On-disk storage: where the CLI auto-pulls every shot (the shotlog contract)

The CLI auto-pulls every screenshot to the CALLER's machine and self-janitors the cache. This is the part a visualizer watches.

Directory (resolved once, per machine):

$ADOM_SHOTS_DIR                                  # explicit override, else:
~/project/screenshots/adom-desktop/              # when ~/project exists (the container convention)
~/.adom/screenshots/                             # fallback (no ~/project)

Never /tmp. The CLI deliberately does NOT touch ~/project/screenshots/*.png (timestamp-named) or ~/project/screenshots/shotlog/: those belong to other tools.

Buckets (subfolders by kind):

<shots_dir>/window/     # a specific window / app capture  (desktop_screenshot_window, kicad_*, browser_*)
<shots_dir>/screen/     # a full-screen grab               (desktop_screenshot_screen)

Filenames: human-readable, sortable:

window/2026-08-10T01-14-16.937-986710.png   # <ISO-stamp>-<windowId>.png (the folder says "window", so no prefix)
screen/2026-08-10T01-05-08.625.png          # a screen grab (no window id)

The ISO stamp is YYYY-MM-DDThh-mm-ss.mmm with no colons, so it is filename-safe AND lexicographic order == chronological order. Entries of a multi-window array (kicad_screenshot_all) keep each window's own name and gain localPath/savedTo per entry.

Single file vs full/safe pair:

  • If the capture was already within the safe size cap (resizeMax, the common case on macOS since captures are pre-capped at 1568) it is written ONCE as <stem>.png, and localFullPath == localSafePath point at that one file.
  • Only a shot LARGE enough to be downscaled gets the pair: <stem>.full.png + <stem>.safe.png. A renderer should prefer .safe for display and .full for pixel-exact work; when only <stem>.png exists, that IS both.

Janitor (automatic): the pull cache self-prunes (defaults: 7-day age cap, 2 GB size cap, 4000-file cap, oldest deleted first, newest always kept; a shot's sidecar follows its PNG on delete, plus an orphan-sidecar sweep). No cleanup is needed caller-side. Inspect with adom-desktop janitor status; force a sweep with adom-desktop janitor '{"sub":"run"}'. Tune with ADOM_SHOTS_DIR, ADOM_SHOTS_MAX_AGE_DAYS, ADOM_SHOTS_MAX_GB, ADOM_SHOTS_MAX_FILES, ADOM_JANITOR=0.

6. The sidecar schema: <stem>.json next to every shot

Every pulled shot gets a <stem>.json sidecar (a full+safe pair shares ONE sidecar). This is the self-describing record a visualizer should read instead of parsing the response envelope.

{
  "source": "desktop_screenshot_window",     // the verb that produced it
  "timestamp": "2026-08-10T01:14:16.937Z",   // RFC3339 UTC
  "kind": "window",                          // "window" | "screen"
  "hwnd": 986710,                            // CGWindowID; null for a "screen" grab
  "title": "KiCad 9.0 - myboard.kicad_pcb",  // window title (null for screen)
  "full": { "path": ".../window/<stem>.png", "w": 1568, "h": 980, "bytes": 402931 },
  "safe": { "path": ".../window/<stem>.png", "w": 1568, "h": 980, "bytes": 402931 },
  //  (when the shot was under the cap, full.path == safe.path == the single <stem>.png)
  "coordMap": null,                          // always null on macOS (Windows-only field)
  "windowRect": null                         // always null on macOS
}

Array-entry shots (each window of a kicad_screenshot_all) get the same sidecar plus parent/popupIndex slots (null when the response has no top-level parent, which is the kicad_screenshot_all case) and their own rect when the entry carried one.

How a renderer groups a capture from disk alone:

  1. List <shots_dir>/window/ and <shots_dir>/screen/, newest-first (filenames sort chronologically).
  2. Read each <stem>.json for source, kind, window id, title, and dims.
  3. Prefer safe.path for thumbnails/inline display, full.path for a zoom view; if full.path == safe.path, there is a single file.

7. Driving without disturbing the user

Capture is background by default (no focus steal, no cursor move). To also ACT without foregrounding:

  • desktop_find_control {hwnd, name|contains, role?} then desktop_ui_click / desktop_ui_set: background press / set-value by accessible name via the macOS Accessibility API (AXPress / AXValue). Requires the Accessibility permission; action verbs are gated by shell approval. When an element exposes no usable AX action, the fallback is a CGEvent click, which foregrounds and is reported as pathTaken:"cgevent-fallback". Every response carries a foreground block whose observed/raised are MEASURED (frontmost app before/after), never asserted.
  • Coordinate clicks (desktop_click {x, y} in points) as the fallback when a control is not in the AX tree.
  • desktop_ui_watch / desktop_ui_events / desktop_ui_selftest are not yet implemented on macOS and return a clean not_implemented_macos error.
  • desktop_find_window {titleContains|className}: resolve a window id; both are case-insensitive substrings, and className matches the owning APP's name on macOS. Topmost match wins.
  • desktop_list_windows: enumerate all on-screen, titled windows with ids + rects (front-to-back Z-order).
  • desktop_list_monitors: the display map (points bounds, pixel size, Retina scale, primary).
  • desktop_screenshot_screen: main-display grab.
  • adom-desktop janitor status: verify the auto-pull cache is being managed (so a tool never builds its own cleaner).