main
John Lauer Publish 0.1.0 d8a45ab

Hydrogen: Screen Capture and Sharing (the AI-consent surface)

This is the surface you (the AI) trigger to see Hydrogen's own screen. To screenshot the editor (panels, the workspace) into the workspace, a share must be active first: you request it, the user consents (Hydrogen auto-approves in about 3 s), a short countdown plays, then you capture.

What a capture is inside Hydrogen: no getDisplayMedia and no browser screen picker. Starting a share only flips the store's isSharing; each frame is taken by Hydrogen itself through POST /screenshot on the control API. On Linux that is WebKit's snapshot of Hydrogen's main window (iframes included); a panel capture crops it to the panel's selector. It never shows other apps or the rest of the desktop. The getDisplayMedia / Element Capture path (restrictCaptureTo) runs only in web Hydrogen in a browser.

This is distinct from hydrogen-permissions (the webview allowing every mic, camera and notification request), a different layer.

The Screen Capture dropdown (editor toolbar)

Source: src/lib/components/editor/EditorNav.svelte (~L2227–2332). Store: src/lib/stores/screenCaptureStore.ts.

When not yet sharing, the dropdown offers two ways to start:

Option Description
Share this tab "Panels and workspace screenshots"
Share entire screen Sets the surface to monitor; inside Hydrogen it still captures only Hydrogen's window

While sharing, it shows Sharing <surface> ("entire screen" / "window" / "this tab") plus capture actions:

Action Description
Workspace screenshot Save to container's screenshots folder
Full screen screenshot Save to container's screenshots folder (only when the surface is a monitor/window)
Stop sharing Ends the share

Window mode toggle (Webview vs Pup)

Below the share options is a Window mode toggle with two buttons — Webview and Pup (windowModeStore). It selects which window surface capture/recording targets: Hydrogen's own embedded Webview, or a Pup (Puppeteer) browser window. Webview = "see Hydrogen itself"; Pup = "see the AI-driven browser."

Pup works where Adom Bridge runs: this computer once Hydrogen has installed it (when a Linux build is published for the user's account), or another machine through adom-bridge --target.

Capture scopes & how a frame is taken

screenCaptureStore.ts exposes the scope type and the capture functions:

type CaptureScope = 'screen' | 'workspace' | 'panel';
Scope Function What you get on Linux
panel capturePanel(panelId) POST /screenshot {target:'shell', selector:'[data-pane-id="<id>"] .panel-body'}: that panel, cropped from the window snapshot
workspace captureWorkspace() the whole Hydrogen window
screen captureScreen() the whole Hydrogen window (not the desktop)

Other store entry points you'll see: startTabCapture(), startScreenCapture(), stopScreenCapture(), captureFrame(), isCaptureActive(), promptScreenCapture(message?) (asks the user to enable sharing).

State shape: { isSharing, displaySurface, hasElementCapture, promptMessage }, where displaySurface is 'browser' (tab) / 'window' / 'monitor'.

You can also call POST /screenshot on the control API directly (b64:true returns the PNG inline as dataUrl); see hydrogen-self-screenshot.

The AI sharing-approval flow

When you (or a bridge) need screen/mic access, Hydrogen shows a consent gate before any capture. Source: src/lib/components/editor/SharingApprovalDialog.svelte, store src/lib/stores/sharingApprovalStore.ts.

The request you send carries:

{ bridgeKey, reason, shareType: 'tab' | 'screen' | null, audio: boolean }
  • The dialog header reads "AI wants access" and shows your reason in a highlighted box. Send a clear, specific reason — it's the only thing the user sees to decide. ("Screenshot the schematic panel to verify the layout," not "capture screen.")
  • Hydrogen auto-approves after ~3 s (AUTO_MS = 3000), showing "Auto permission will be given in N…". This is the Hydrogen-only fast path so you're not blocked on hands-free flows.
  • The user can override before it auto-fires: flip share-type between Share this tab and Share entire screen, toggle the mic, or pick a mic device. Any interaction cancels the auto-countdown so they stay in control.
  • On approve, Hydrogen starts the screen share first (tab or screen), then mic if requested, and returns a result (status: 'approved', with sharing + mic details) to the requesting bridge.

CountdownToast — the cancelable lead-in

Source: src/lib/components/editor/CountdownToast.svelte. Before capture/recording actually starts, a countdown toast ("Recording in N…", default 3) plays, optionally showing the sources and reason. It has a cancel (×) — the user can abort right up to the last second. Don't assume capture started the instant you requested it; the toast is a real, abortable delay.

What this means for you (the AI)

  1. To screenshot Hydrogen's editor you need an active share. If you're not sharing, call promptScreenCapture(...) / send a sharing request with a reason; don't silently fail a capturePanel/captureWorkspace.
  2. Write the reason for a human. It's your one chance to justify the access.
  3. Respect the override + countdown. The user may switch share-type, deny mic, or cancel the countdown — read the returned result before claiming success.
  4. Pick the right scope: panel for one panel, workspace for the window. screen adds nothing on Linux.
  5. For a picture of the user's whole desktop or another app, Hydrogen has no way on Linux; Adom Bridge's desktop_screenshot_* verbs do it only where Adom Bridge runs.
  • hydrogen-self-screenshot: per-panel screenshots and the shotlog verify loop
  • hydrogen-recording: recording (no native recorder on Linux)
  • hydrogen-permissions: the webview's permission layer
  • hydrogen-adom-bridge: Adom Bridge's desktop screenshots, where it runs