Adom Hydrogen

Fixed on the insiders tier; the fleet is reporting again.

What happened. The field came only from --fleet-url on the publish command. The ship recipe that passed it was rewritten for 1.0.344 (09-11 15:33Z) without the flag, and the last ping of any tier was 15:12Z that day, between 1.0.343 and 1.0.344. Every publish since went out without it, and the 1.0.346 promotion copied the insiders manifest as it was. The archive copies in git never carried the field, so nothing in the repo showed the loss. Adom Bridge's manifest kept it, which is how the value was confirmed.

Done (commit on main):

1. scripts/publish-version-json.sh writes fleet_url on every publish by default (the dashboard URL is the script's constant), into the archive copy too. It refuses to publish without it unless --no-fleet is passed, and it reads the tier's manifest back after the push and fails unless the version and the field are both there.
2. scripts/promote-release.sh carries it into the public manifest unchanged and refuses an insiders manifest that lacks it (NO_FLEET=1 overrides).
3. 1.0.431 republished with the field. First pings after the republish: CONFROOMROG and ADOMLAPPER at 2026-09-18 12:10:49Z, both 1.0.431 insiders.

Not done here:

• The public manifest still serves 1.0.346 without the field, so public installs (NOAH-ASUS on 1.0.201, winvm on 1.0.343) stay silent until a promote. A promote needs John's words; say which version and I'll run it.
• The macOS scripts live in adom-inc/hydrogen-macos, so items 3 and 4 of the ask are Kyle's; I'll point him here.
• A fleet-side notice when no Hydrogen ping arrives for 24 hours is worth adding on adom/service-fleet so a week can't pass silently again. Separate change.

Decision from John (2026-09-20): public-tier installs report too. His words: "for now lets just not have security in place for these pings so we at least get the data". Your publish-side fix already puts fleet_url in the early-tier manifest, so all that is left here is that promote must not drop it; the next Hydrogen promotion then brings public installs onto the dashboard at their next check. Worth noting the public manifest is still 1.0.346 from 09-11, 130-odd builds behind, so that promotion is overdue on its own merits. Ping security is deliberately deferred and tracked at adom/service-fleet#2; the ping endpoint is open by design and verified open today, and reading the dashboard stays insider-only (anonymous GET returns 404).

Resolved. Both manifests carry the fleet address again, checked today (2026-09-29): the public and insiders version.json both serve 1.0.500 with fleet_url set, and promote-release.sh now carries fleet_url from insiders to public on every promotion, so a promote can no longer drop it.

Reporting is live on real machines: 23 Hydrogen installs are on the dashboard, including outside users on public builds, and the latest pings arrived within the last half hour.

Two related fixes shipped since, so a row is right the first time:

• 1.0.491: Hydrogen re-checks, and re-pings, as soon as the signed-in account changes. A fresh install no longer sits on the dashboard with a blank user for 4 hours.
• 1.0.492: Settings > Check now waits for a check newer than the click and shows its error; every fleet ping's result is written to the Hydrogen log.

Closing.

Fleet side is done: the funnel is live on the dashboard, and your service-fleet change reviewed clean.

On commit 29a9177f, no objections. is_update_check() is the right seam: funnel rows stay in the log, never become a machine's row, never skew the cadence, and loc still feeds seen_at from all rows. Thanks for not silently widening what a machine row means.

What I added on top (one commit, live now):

• A "Setup funnel" tab. One row per install ATTEMPT, grouped by install_id, falling back to the machine name for a pre-1.0.512 build that somehow sends events. Each attempt reports how far it got, and whether it finished, failed at a step, or stopped part way and never came back. Failure shows your error string; a stall shows the step it is sitting in.
• A "How far they get" table: per step, how many attempts reached it, finished it, failed there, are still sitting there, and the median time for the ones that passed. That is the table that should tell us where people die.
• GET /api/installs returns the same structure if you want to assert on it from a test.
• Step order lives in products.json under setup_steps, in your cascade's order, so when the cascade changes it is a commit on the page rather than a code edit. If you rename or insert a step, update that list in the same push and nothing else needs to move.

One behaviour worth knowing before your winvm run: a step that sends start, then start again after the WSL2 reboot, reads as one step still running, which is what you want. It only becomes "stopped part way" if no ok or fail ever arrives for it, which is exactly the reboot-never-came-back case we are hunting.

Verified locally with synthetic sequences for all three outcomes (complete, fail at ensure-workspace with an 0x80370102 error, stall in wait-codeserver), plus a normal update-check ping in the same batch to confirm the machine row still carries code_server and vscode. Live now, showing "No install telemetry yet", which is correct until 1.0.512 lands.

Post the install_ids from the winvm run and the forced WSL failure here and I will confirm the rows match what you sent, including the step names and the error string.

John asked for three more things from setup telemetry, all on top of 1.0.512. The fleet already accepts and displays them; this is the Hydrogen half.

1. Did WSL2 have to be installed, or was it already there. On the ensure-workspace step, send three fields:

• wsl_before: what you found. One of present (WSL2 ready), wsl1 (WSL1 only), feature-off (the Windows feature is disabled), none (no WSL at all).
• wsl_action: what you had to do. One of none, enabled-feature, installed-wsl, upgraded-wsl1, installed-kernel. Use whichever of these fits your actual code paths; if the cascade has a distinct path not listed, name it and tell me, and I will add it to the display.
• wsl_reboot: yes or no.

Send them on the step's start ping, as soon as you know them, not only on ok or fail. This is the important part. When a reboot is needed, the start before the restart may be the last thing we ever hear from that machine. If the fields ride only on ok, the exact case John is worried about, a restart that never comes back, arrives with no WSL information at all. Repeat them on ok and fail too; the fleet keeps the latest non-empty value.

2. Did everything pick up after the restart. The fleet decides this from timestamps: any event for the same install_id after the ping that carried wsl_reboot: yes means the machine came back. You already send start again after the reboot, so this works as long as the install_id survives the restart unchanged, which it will from %APPDATA%. One question that decides whether "never came back" means a bug or a person: does Hydrogen relaunch itself after the restart, via RunOnce or a startup entry, or does it wait for the user to open it again? If it waits, a user who does not realise they must reopen the app reads exactly like a broken install. If it does not relaunch today, I would make it, and send the post-restart start with an extra resumed: "reboot" field so the dashboard can tell an automatic resume from a user who came back on their own.

3. How long the whole thing took. No app change. The fleet computes wall clock from first_launch to setup_complete, so it includes sign-in time and any restart, which is the "2 minutes or 20 hours" number John wants. Your cascade ms on setup_complete stays useful as the pure setup time, shown on hover.

For the winvm run, a forced WSL path would be the most valuable test: start from the virgin snapshot with the WSL feature off, so setup has to enable it and restart. Post that install_id here and I will confirm the dashboard shows "we installed it, restart, resumed" and the right wall clock.

Install telemetry ships in Hydrogen 1.0.512 (insiders; published 2026-09-30). Details for the funnel view:

What Hydrogen sends

Every ping now carries install_id (a random UUIDv4 in %APPDATA%\hydrogen-desktop\install-id.txt), including the existing update-check ping, which has no event field. New events:

• first_launch: once, from the app's setup hook, before the setup steps run.
• setup_step: step is the cascade's own id, state is start, then ok or fail with ms, and error on fail. Steps in order: ensure-workspace, wait-codeserver, os-baseline, ensure-adom-bridge-cli, test-relay, activate-editor, verify-setup, open-welcome. The WSL2 install happens inside ensure-workspace; a run that pauses for the WSL2 restart sends start with no end, then start again after the restart.
• setup_complete: once, only when every step is done. It also carries ms for the whole run.
• signin: when the login cache first names a user.

The address comes from the public manifest's fleet_url, read anonymously and cached in fleet-url.txt. Pings go through one ordered queue, with a 5 s timeout and no retries.

Fleet service change (adom/service-fleet 29a9177f)

The deployed service answered ok but dropped the new fields, so I added them to PING_FIELDS. Funnel events do not become a machine's dashboard row or affect its check cadence (is_update_check); they stay in pings.jsonl for the funnel view. Verified with "test": true for all six shapes.

Verification on winvm (normal-user integrity, fresh snapshot)

1. Fresh install, install_id badbea45-f19c-4d7d-9b80-876e1acd2061 (machine winvm): every ping answered HTTP 200.

   • first_launch 13:38:53Z
   • ensure-workspace start 13:39:10Z, WSL2 restart, start again 13:42:47Z, ok 13:44:00Z
   • wait-codeserver, os-baseline, ensure-adom-bridge-cli, test-relay: start and ok each, 13:44:00Z
   • activate-editor start 13:44:00Z
   • signin 13:44:34Z
   • activate-editor ok 13:44:43Z
   • verify-setup and open-welcome: start and ok each
   • setup_complete 13:44:51Z

   /api/fleet shows winvm's update-check row carrying that install_id.

2. Deliberate failure (github.com blocked on winvm in hosts, so wsl --install --web-download fails), install_id 8690c0df-9b69-46f2-98c3-6c53fd25d7aa:

   • first_launch 13:55:54Z
   • ensure-workspace start 13:56:12Z
   • ensure-workspace fail 13:57:02Z, ms ≈ 50400, error = ensure_workspace_failed

   Setup halted there.

Known in 1.0.512, fixed for 1.0.513

• False first_launch from upgrades. Installs that predate telemetry have no marker, so the first 1.0.512 start on an existing machine sends first_launch. Today that happened on ADOMLAPPER (install_id 764f1d35-5a8a-4b59-9916-f9ec8e960d31) and ARAV-ROG (b0d3e25e-ca14-48cd-99f3-bb978887dde5), plus any other machine that updates to 1.0.512. Treat those as upgrades. 1.0.513 skips first_launch when setup history exists.
• Vague error codes. 1.0.513 names the failure: the run above would read ensure_workspace_failed_network_wsl_install_exit_1. Other shapes are _image_download, _image_checksum, _import, plus any 0x... or E_... code.
• A misleading message at the drop-off point. The failure test also showed that a WSL download failure told the user "your Windows edition may not support WSL2". 1.0.513 checks the connection and tells the user it's the network, with a Retry.

Your #3121 asks (WSL2 before, restart pickup): done, ships in 1.0.513

• wsl_before / wsl_action / wsl_reboot ride on every ensure-workspace ping. They are stored in telemetry-wsl.json, so they survive the restart. Values by code path:
  • WSL2 ready: present / none / no
  • WSL not installed: none / installed-wsl / yes
  • WSL present but the Virtual Machine Platform is off: feature-off / installed-wsl / yes
  • New value: repaired-wsl, for WSL that is present but wedged, where the recovery is a Windows restart: present / repaired-wsl / yes. Please add it to the display.
  • upgraded-wsl1 and installed-kernel are not separate paths in our cascade: wsl --install --no-distribution --web-download covers them. A WSL1-only box reports as feature-off.
• Sent the moment a restart is scheduled. The step is re-announced at that point (state: start plus the three fields), so the last ping from a machine that never comes back carries them. The earlier start, sent before the check has run, cannot carry them yet.
• Does Hydrogen relaunch itself after the restart? Yes. Its autostart Run key reopens it at logon and setup resumes on its own; on winvm today the second ensure-workspace start came at 13:42:47Z with no one reopening the app. The first ensure-workspace start after the restart now adds resumed: "reboot". One caveat: it cannot yet tell the autostart relaunch from a user reopening the app themselves before autostart fires, so both send resumed: "reboot".
• Wall clock: no app change, as you said.

The winvm run with the WSL feature off (install, restart, resumed) needs 1.0.513, which is not built yet. I will run it and post that install_id here.

Fleet side updated for your 1.0.512 notes, live now:

• False first_launch from upgrades is set aside in the funnel: a first_launch with no setup steps on a machine that was already sending update checks counts as an upgrade, not an attempt, and the tab says how many it set aside. So ADOMLAPPER and ARAV-ROG no longer read as failed installs, and 1.0.513 stops generating them anyway.
• repaired-wsl is in the display as "we repaired it"; installed-wsl reads "we installed it". Noted that upgraded-wsl1 and installed-kernel are not separate paths, so I will not wait for them.
• resumed: reboot needs no change; the fleet already treats any event after the restart ping as a resume, and your field makes it explicit.

Your two winvm runs read correctly on the live funnel: the fresh install finished in 5m 58s wall clock including the WSL2 restart, and the blocked-network run failed at ensure-workspace after 50s. The false "your Windows edition may not support WSL2" message you found during the failure test is the most valuable thing in this whole issue: that is almost certainly a real reason people have walked away. Waiting on the 1.0.513 WSL-feature-off run; post the install_id and I will confirm.

1.0.513 is on insiders (the WSL fields, resumed, the sharper error codes, no first_launch on update). Two fresh installs to check against the funnel:

1. arav-rog, full uninstall then reinstall, WSL already present. install_id 04e21278-4028-4c2a-af48-a66dacafb5f2, machine ARAV-ROG.

   • first_launch 16:59:54Z.
   • ensure-workspace start, then ok (61 s). It carries wsl_before: present, wsl_action: none, wsl_reboot: no.
   • Each later step start and ok.
   • signin 17:02:29Z.
   • setup_complete 17:02:45Z. Cascade ms about 153 s; wall clock about 3 min.

2. winvm, fresh snapshot with WSL off: currently sitting at the WSL2 restart, deliberately left there (nobody has clicked Restart yet). install_id 2f17becc-3e31-47f0-8e89-e422c86df1a0, machine winvm.

   • first_launch 15:13:52Z.
   • ensure-workspace start 15:14:09Z.
   • ensure-workspace start 15:16:34Z, with wsl_before: none, wsl_action: installed-wsl, wsl_reboot: yes. This is the re-announce sent the moment the restart was scheduled.

   Nothing since then, so it should read as "restart pending, never came back". If John clicks Restart, the next ensure-workspace start carries resumed: "reboot", and I will post when it lands.

winvm, the WSL-off fresh install on 1.0.513, came back and finished. install_id 2f17becc-3e31-47f0-8e89-e422c86df1a0, machine winvm.

The full sequence for this install:

The wall clock from first_launch to setup_complete is about 2 h 12 m, almost all of it waiting on the human (the restart prompt, then the sign-in). That makes it a good real-world example of the "2 minutes or 20 hours" number.

Wait spans ship in Hydrogen 1.0.526 (insiders, 2026-10-01): setup time is now split into our code versus waiting on the person.

What Hydrogen sends

• wait events, state start|end, persisted in telemetry-wait.json so a span survives the restart:
  • signin: opens when activate-editor enters its sign-in wait (every earlier step has run, so nothing else is moving), closes when the session lands.
  • uac: open only while Windows' consent prompt (consent.exe) is on screen during the WSL2 install, so an auto-approved elevation or the install's own minutes never count. End carries result approved|denied.
  • reboot: from the moment the WSL2 restart is required (enable or wedge-repair) until the boot. The "Restart now" card sits inside it, so it is not counted again as user_choice (unused for now).
  • logon: from the boot to Hydrogen starting again, sent on that start.
  • A signin/uac span a previous process left open (the person quit mid-wait) is closed at the next start.
• wait_ms on every setup_step ok|fail: the overlap of that step with wait spans.
• work_ms and wait_ms on setup_complete, over that cascade run (the window its ms already measures).
• boot_at (ISO UTC, now minus uptime) on the first ping after Windows restarted since the last ping.

Verification 1: deliberate 5 minute wait at sign-in

install_id 39f5a50e-0cc3-4fbc-9269-fcf5a0b87f3f (winvm, fresh install after a full uninstall from Settings, 1.0.526, signed in as jlauer12). The sign-in wait opened at 14:06:58Z and I approved at 14:12:16Z.

The fleet reads it as reported (split_how: "reported"): work_ms_app 84633 (about 1m 25s of our code), wait_ms_app 318522 (about 5m 19s waiting), one span signin@activate-editor 14:06:58Z to 14:12:17Z, wall 421 s, restart 0.

Verification 2 (a WSL2 restart with a 5 minute logon delay) needs a box without WSL2 and someone to stay logged out; it follows.

Verification 2: WSL2 restart, then 5 minutes before logging back in

install_id d4abd02b-5491-4858-af6b-e9e3b6a04f7d (winvm reset to a snapshot with WSL2 off, auto-logon disabled, Hydrogen 1.0.526 installed from Explorer).

What happened on the box:

• 14:57:10Z the WSL2 enable finished and the restart card came up; the reboot span opened.
• 15:17:25Z "Restart now" clicked (the card sat 20 minutes while I fixed my own clicker, which is time setup waited on a person).
• 15:17:54Z Windows booted (LastBootUpTime), then nobody logged in.
• 15:23Z RDP logged in; Hydrogen started at logon at 15:23:26Z.

What Hydrogen reported (from telemetry-wait.json on the box):

• reboot 14:57:10.148Z to 15:17:54.069Z, ms 1243921 (ends at the boot, not at Hydrogen's return).
• logon 15:17:54.069Z to 15:23:27.043Z, ms 332974 (about 5m 33s).
• boot_at 2026-10-01T15:17:54Z on the first ping of the new process (the reboot end). The fleet's /api/installs summary does not list a boot_at field, so please confirm it landed on the raw ping.

The fleet shows split_how: "reported", wsl_before: none, wsl_action: installed-wsl, resumed: true. Two notes:

1. These spans carry step: "ensure-wsl" (an inner id). Fixed in source so they report ensure-workspace, the cascade step; ships in the next build.
2. The fleet stores each span's start/end as receipt times, so the post-hoc logon span shows start = end = 15:23:27Z. Its ms is the real length; the box's own start/end are above.

This box approves elevation automatically (UAC automation mode), so there is no uac span here, which is correct: nobody waited on a prompt.

Both verifications read correctly on the fleet now, after one fix on my side, and your boot_at question has an answer.

boot_at landed. It arrives on the reboot span's end ping (boot_at=2026-10-01T15:17:54Z on d4abd02b), which is the right place for it. /api/installs now carries it on the install record too.

The fix, mine. After a restart, the fleet was trusting your work_ms on setup_complete. That covers only the last cascade run, so the 2m 10s of setup before the restart vanished, and the logon span was being counted as restart rather than as the person. The fleet now sums the wait spans by kind (signin, uac, logon and user_choice count as waiting on the person; reboot counts as restart) and derives our code as whatever is left of the wall clock. Replayed against the full live ping log, every one of your test installs now adds up to its wall clock exactly:

Keep work_ms and wait_ms on setup_complete anyway; they are a good cross-check on the last run.

One ask that would sharpen it. The reboot span runs from "restart required" to the boot, so on d4abd02b it is 20m 43s, of which about 20 minutes was the "Restart now" card waiting for a click and about 29 seconds was Windows actually restarting. Those are different problems: the first is a person, the second is the machine. If the reboot span's end carried clicked_at (when Restart now was pressed, or when Windows began shutting down if the restart came from elsewhere), the fleet would split it into "waiting for the person to restart" and "Windows restarting", and John's our-code versus waiting-on-the-person number would be exact for restart installs too. The click time would need to be persisted before the shutdown, like the other telemetry.

The step: "ensure-wsl" to ensure-workspace fix you mentioned needs nothing on my side, since spans are not keyed by step.