main
John Lauer Publish 0.1.0 d8a45ab

name: hydrogen-annotate version: 1.3.0 description: >- Hydrogen's Annotate tool: the Title Bar pencil, the tray's Annotate screen, or Super+Alt+A lets the user draw on a frozen picture of the screen (pen, arrow, box, snip) and puts a PNG on the clipboard for the AI thread. Use when asked to annotate, circle it, mark up the screen, snip a region, or about POST /annotate.

Annotate: draw on the screen, paste it into your AI thread

Desktop annotate on Linux (freeze-frame)

Wayland lets no app hook the mouse globally or lay a click-through layer over other apps, so annotate on Linux works on a still picture:

  • The pencil in the Title Bar, the tray's Annotate screen, or Super+Alt+A first captures the screen, then shows that picture full screen.
  • A plain drag draws. Enter copies, Esc cancels (it works even if the page failed to load), Ctrl+Z undoes. The toolbar (Pen, Arrow, Box, Snip, colours, Auto/Multi, Undo, Clear, Done) sits at the top of the picture.
  • What you annotate is the screen as it was when you pressed the key: open the menu or dialog first, then start annotate.
  • Hydrogen captures the screen itself, never through Adom Bridge: on X11 from the root window (no prompt), on Wayland through the desktop portal's Screenshot. If the desktop asks first (GNOME asks once whether Hydrogen may take screenshots), that is the desktop's own dialog; the overlay opens only after the answer. A refusal, or a desktop with no screenshot portal, falls back to annotating inside Hydrogen's window (below). An overlay whose page does not come up within 8 seconds closes itself.

The shortcut is a desktop keybinding

GNOME 46 (Ubuntu 24.04) has no global-shortcuts portal, so Hydrogen adds a custom keyboard shortcut "Adom Hydrogen: Annotate" in the desktop's own settings (GNOME/Ubuntu: Settings > Keyboard > Custom Shortcuts; Cinnamon: Keyboard > Shortcuts > Custom). It runs ~/.local/share/adom-hydrogen/annotate-hotkey, which asks Hydrogen to start. If Super+Alt+A is already bound, Hydrogen leaves it alone and GET /annotate/hotkey says taken: true with taken_by. Settings > Hydrogen > Annotate turns it off (removing the keybinding); uninstalling Hydrogen removes it too. method is gsettings (GNOME family), gsettings-cinnamon, or none (other desktops: use the pencil or the tray). POST /annotate/hotkey {"enabled": false} turns it off from the API.

The result and the clipboard

The capture is the area around the drawing with the drawing on it, saved in ~/.config/hydrogen-desktop/annotations and put on the clipboard. The clipboard copy is held by a small helper (adom-hydrogen --clipboard-hold, its own systemd-run --user unit adom-hydrogen-clipboard-<ms>), so it survives Hydrogen restarting; it lets go when another app copies, or after 12 hours. It offers image/png plus the saved file (text/uri-list, x-special/gnome-copied-files), because a WebKitGTK chat box (the Claude panel in Hydrogen) receives a pasted picture only as a file. The card says "on your clipboard" only after Hydrogen read the clipboard back and saw image/png; otherwise it says "Not copied" and the file is still saved. Check from a desktop terminal: GDK_BACKEND=x11 adom-hydrogen --clipboard-check (exit 0 = an image is there).

Inside Hydrogen's window

With scope: "window" (or when no desktop capture is possible) annotate puts a canvas over the whole Hydrogen window (the editor, the wiki, every panel; they are all under the same canvas), with the same toolbar pup uses. The human draws, and the area around the drawing is captured and written to the clipboard as a PNG. The point is the next step: paste it into an AI thread and say "this one".

Tool What it does
Pen Draw freehand. With Auto on, the capture fires the moment the mouse is released.
Arrow Drag from tail to tip; the arrowhead lands where the mouse is released.
Box Frame a region with a rectangle, corner to corner.
Text Click where the note starts, type, press Enter (window scope only).
Snip Capture exactly a region: drag a marquee, it is copied on release, no strokes.

Colours: red, blue, yellow on the bar, more one click deeper. Two capture modes:

  • Auto (default): one shot. Draw one thing, lift the mouse, and it is captured, copied, and annotate mode closes. This is the "red circle" case.
  • Multi: compose. Draw several shapes; nothing is captured until Done. The colour advances after each shape so every shape is distinct.

The capture is the composited window with the drawing on it, cropped tight around everything drawn. A toast confirms it: "Captured 640×320, on your clipboard. Paste it into an AI thread."

Keys: Enter = Done (capture and close), Esc = leave without capturing, Ctrl+Z = undo the last shape. Undo and Clear are on the bar too.

What the AI does with it

You do not need annotate to see the screen. To look at Hydrogen yourself, take a screenshot: POST /screenshot on the control API (the hydrogen-self-screenshot skill). Annotate exists for the human to point at something.

When the user pastes an annotated PNG into your thread, treat the drawing as the pointer: the circled, boxed or arrowed thing is what they mean, and a text note is their comment. Read the image before answering.

You can open the overlay for the human when that is the fastest way to get an answer ("circle the part you mean"):

CTRL="$(cat ~/.adom/hydrogen-control-url)"
curl -s -X POST "$CTRL/annotate" -H 'Content-Type: application/json' -d '{"tool":"pen","mode":"auto"}'

tool is pen | arrow | box | text | snip (desktop freeze-frame takes pen, arrow, box, snip), mode is auto | multi, scope is desktop (default) or window, close: true dismisses it. The result lands on their clipboard, not in your context, so ask them to paste it. Do not open it while they are typing or during a recording without saying so first; it covers the screen until Esc.

Where it lives

  • The pencil: src/lib/components/editor/EditorNav.svelte (button next to the meters).
  • The overlay and toolbar: src/lib/components/AnnotateOverlay.svelte, driven by src/lib/stores/annotate.ts (startAnnotate(tool, mode), closeAnnotate()).
  • The route: POST /annotate in src-tauri/crates/hydrogen-control/src/lib.rs.
  • The capture itself is POST /screenshot with target=full, cropped in the overlay.
  • Desktop mode: src-tauri/crates/hydrogen-app/src/desktop_annotate.rs (module linux: the freeze-frame, clipboard holder, keybinding), src/lib/components/annotate/DesktopOverlay.svelte, shared tools in src/lib/annotate/draw.ts.

Related: hydrogen-self-screenshot (the AI's own screenshots), hydrogen-captions (text overlays the AI puts on screen for narration; not for marking things up), hydrogen-pup (pup's annotate, the same toolbar in the browser window).