Closed general

Remote request storms when an AI calls adom-cli in a loop (409 reconnect probe, uncached container lookup, login poll with no deadline)

John Lauer · 13d ago ·closed by Colby Knox

Colby caught Hydrogen Desktop hitting carbon.adom.inc/user about 4 times a second per installing machine (fixed in ah 1.0.571). We then audited every Adom client for the same class of bug. adom-cli has no constant polling, but three paths turn an AI calling the CLI in a loop into a request storm against our own servers. Audited origin/main at 33187a9 (v0.7.11); line numbers are from that commit.

1. High: "no browser session" reconnect probe, 300 ms, re-armed on every call

  • Where: src/commands/hydrogen/browser_probe.rs:106-127 (wait_for_browser, RECONNECT_WAIT_MS=5000, RECONNECT_POLL_MS=300). Called from patch_checked / post_checked / delete_checked / delete_body_checked (browser_probe.rs:186/204/221/239), notify.rs:85, reload.rs:26, screenshot.rs:102, webview.rs:536, and every workspace mutation in workspace.rs:232-321.
  • Endpoint: GET {hydrogen}/api/workspaces/editor/{owner}/{repo}/current/probe (hydrogen.adom.inc on cloud).
  • When: any mutation answered 409 "no browser session" (the Hydrogen tab is closed, backgrounded, or a zombie).
  • Cost: about 18 requests per failing call (context lookup, the op, about 16 probes, a retry). Nothing is remembered between calls, and the hint tells the AI to retry once connected, so an agent retrying back to back makes about 3 requests a second (about 12,000 an hour). Several agents stack.
  • Fix: probe 2 or 3 times about 1 s apart with backoff, and write a short-lived marker (for example ~/.cache/adom-cli/ui-down, 30 s) so the next calls fail fast without probing.

2. Medium-high: uncached Carbon container lookup on every call

  • Where: src/context.rs:16-54 (ContainerContext::detect, client.get at line 54). It runs first in 14 hydrogen subcommands, in carbon repos ... (repos.rs:266-405) and in containers.rs:264 when --owner/--repo is omitted.
  • Endpoint: GET carbon.adom.inc/containers/{slug} (or /containers/current).
  • Cost: one Carbon request per CLI call, for an answer that never changes in a container's lifetime. An AI making 10 calls a second sends 36,000 Carbon requests an hour from this alone.
  • Fix: cache the result on disk keyed by slug, or derive owner/repo from the hostname the way main.rs:92-117 (derive_repo_url) already does.

3. Medium: adom-cli login poll has no deadline and swallows errors

  • Where: src/commands/login.rs:81-108.
  • Endpoint: GET {carbon}/auth/intents/{token}/status every 2 s.
  • Problem: network and parse errors continue (lines 85-91), and a response with no state counts as pending (line 93), so a deleted intent or a changed response shape loops forever. An AI that starts login in the background and never approves it leaves a process polling at 1,800 requests an hour, indefinitely.
  • Fix: a hard deadline (about 10 minutes), stop on any 4xx or unknown state, back off on errors. Also set a request timeout on AdomClient (client/mod.rs:20) and the login client (login.rs:12).

Lower

  • hydrogen workspace apply / teardown: each run calls the wiki per app (wsfile.rs:750 dock manifest, wsfile.rs:792 adom-wiki pkg install even when already installed, wsfile.rs:2210 again on teardown, wsfile.rs:35 panels catalog). Skip pkg install when the installed version matches, and cache dock manifests briefly.
  • webview.rs:255-272: GET /api/webview-origins on every workspace-URL open. Cloud and macOS Hydrogen don't serve it, so it fails every time and is never cached. Cache the negative answer per base URL.
  • enforce_headers (wsfile.rs:1151-1192): 8 rounds, 400 ms apart. Bounded, but no backoff.

Already fine

  • notify does not retry a 429; it surfaces retryAfterSec.
  • The 504 diagnosis probes once.
  • containers list --all is capped at 100 pages.
  • There is no startup /user call, no version check and no telemetry.

Items 1 to 3 cover essentially all of the risk.

1 Reply

Colby Knox · 13d ago

Fixed in adom-cli 0.7.12 (wiki package 4.3.13), all three main items plus the lower ones.

  1. 409 reconnect probe. After a "no browser session" 409 the CLI probes three times over 4.5 s (1 s, 1.5 s, 2 s) instead of about 16 times at 300 ms. A failed wait writes a 30 s ui-down marker in ~/.cache/adom-cli, keyed by Hydrogen URL and workspace, and calls made while it is fresh fail fast with no probes. adom-cli hydrogen probe clears it when the UI is back. The no-UI hints now say not to retry in a loop. Against a stub with no UI: first failing call 3 probes, the next two calls 0 probes and 0.3 s each.
  2. Container lookup. ContainerContext::detect caches owner/repo/slug on disk for 12 h, keyed by Carbon URL, lookup path, proxy URI and a hash of the token (/containers/current answers per caller). Carbon stays the source of truth because the hostname is ambiguous when owner or repo names contain hyphens.
  3. Login poll. 10-minute deadline; stops on any 4xx (an unknown intent is a 404 AUTHENTICATION_INTENT_NOT_FOUND) and on an unknown or missing state; network and 5xx errors back off to 30 s. The Carbon client now has a 10 s connect and 120 s request timeout (container creation is the slowest call); the login client 20 s. Against a stub whose intent vanished, login stops after the first check.

Lower items:

  • workspace apply / teardown: dock manifests are cached for 10 minutes, and pkg install is skipped when adom-wiki pkg list already has the app (updates arrive through the update sweep).
  • A missing /api/webview-origins or /api/panels/catalog route is remembered for a day per Hydrogen URL, so cloud and macOS Hydrogen are no longer asked on every open or apply.
  • enforce_headers is unchanged: 8 bounded rounds per apply.

Log in to reply.