Open feature request

Mac Hydrogen goes dead under memory pressure: stop forcing 1/4 of RAM, and port the Windows memory management

John Lauer · 4d ago

@kyle, John asked me to write this up. Over the last two months Hydrogen for Windows got a set of memory changes that took it from "dies or goes dead when RAM runs out" to "the runaway job dies and Hydrogen keeps working." The Mac app needs the same. Please read the Windows code below and copy what applies, most of it ports straight over.

What happened on the Mac

Yesterday Anastasia's 16 GB Mac had Adobe Premiere using nearly all of its RAM. Hydrogen stayed open but stopped rendering every chat thread. It was running, but it was not usable.

Drew's #14 is the same family: a 24 GB Mac, the workspace given 5.8 GB with no swap, a TypeScript check plus npm test thrashed it, and the pane went blank for 14 minutes.

First: stop forcing the VM to a quarter of RAM

John's words: forcing 1/4 of the Mac's memory almost makes the app pointless. It's like letting a user install an app and then not letting it use any RAM. AI needs RAM. Claude, Codex, extension hosts, builds and tests all run inside that VM. 4 GB on a 16 GB Mac, or 5.8 GB on a 24 GB one, is not enough room for a single AI thread doing real work, never mind several.

For comparison, Hydrogen for Windows does not cap the workspace at all. We never write a memory= line to .wslconfig, so WSL2 uses its own default and grows on demand. We spent the effort on deciding what dies when memory runs out, and on keeping Hydrogen alive, not on starving the VM up front.

Suggested direction for the Mac:

  • Default the VM to most of the Mac's RAM (for example total minus a few GB reserved for macOS), not a quarter. Keep the Settings override.
  • Virtualization.framework only backs guest memory as it is touched, so a large size costs nothing until it is used. The catch is that touched memory (mostly the guest's page cache) is not handed back to macOS on its own. Fix that side instead of shrinking the VM: a memory balloon (VZVirtioTraditionalMemoryBalloonDevice) driven by macOS memory pressure, and/or periodically dropping the guest's clean page cache when the Mac is under pressure.
  • Add swap or zram in the guest (Drew's idea 1), so a heavy build slows down instead of freezing everything.

What we did on Windows, and where the code is

All in the hydrogen-desktop repo (adom-inc/hydrogen-desktop).

1. Protect the editor, let the runaway job die (the part that ports directly, it is Linux-side)

  • cdf68db4 (1.0.387) and 08723f3e: code-server and the relay units run with OOMScoreAdjust=-900, so the kernel's OOM killer picks a runaway AI process before the editor everything hangs off. A drop-in (/etc/systemd/system/code-server.service.d/hd-oom.conf) puts it on every install, including ones set up before the change, and the running code-server gets -900 live so nothing restarts.
  • e63bf3e1: the important follow-up. AI tools run commands in non-interactive bash, so they inherited code-server's -900. On two of our test PCs an AI's ffmpeg grew to 19 GB and a python3 to 15 GB, both protected, and the OOM killer killed the systemd user managers instead, leaving the workspace broken. Now every shell resets itself to 0: /etc/profile.d/hd-oom-reset.sh for login shells, and Environment=BASH_ENV=/etc/profile.d/hd-oom-reset.sh in the same drop-in for non-interactive bash. The editor keeps -900, the hog dies.
  • Code: src-tauri/crates/hydrogen-runtime/src/runtime/wsl.rs, search OOMScoreAdjust and hd-oom (the unit template and refresh_distro_env).
  • Promotion-gate checks: ai-work-oom and ai-work-not-oom-protected in scripts/promotion-gate.json.

The drop-in is just:

[Service]
OOMScoreAdjust=-900
Environment=BASH_ENV=/etc/profile.d/hd-oom-reset.sh

and the reset script is one line:

[ "$(cat /proc/self/oom_score_adj 2>/dev/null)" -lt 0 ] 2>/dev/null && echo 0 > /proc/self/oom_score_adj 2>/dev/null; true

2. Hydrogen itself never goes silently dead (757a07c9)

  • A patched tokio retries on OS resource errors instead of panicking (src-tauri/vendor/tokio/src/runtime/io/driver.rs).
  • Watchdogs: the proxy watchdog escalates to a Hydrogen restart, and an OS-thread watchdog restarts Hydrogen if the control API stops answering for about 45 s, at most once per 10 minutes (hydrogen-proxy/src/lib.rs, hydrogen-app/src/lib.rs).
  • A starvation sampler logs kernel memory and handle pressure every 5 minutes (hydrogen-app/src/starvation.rs). Gate: control-api-watchdog, starvation-warning.

3. A hidden window stops drawing (82f88a45, Caleb's fix)

  • Hydrogen left in the tray for three days had leaked 13 GB of GPU memory. Now a hidden or minimized window sets its webviews invisible, and the shell stopped restyling every frame (hydrogen-app/src/hidden_quiet.rs, src/lib/components/editor/App.svelte).

4. An honest RAM meter (cdf68db4, ef55d5b1, 61206004)

  • CPU, RAM and the OOM-kill counter come from the adom-vscode extension's GET /stats over HTTP every 2 s, paused while the window is hidden. Never by spawning a process in the VM.
  • When the machine is short, only the "other programs" share turns red, and the hover reads like "900 MB free of 16 GB, Hydrogen 2.1 GB, other programs 13 GB". A full machine no longer looks like Hydrogen's fault.
  • It warns about the machine's available RAM only and never names another program (John's call).
  • When the kernel does kill something, a toast names the process it killed.
  • Code: src/lib/components/editor/EditorNav.svelte, plus the HDOOM lines in wsl.rs. Gate: quiet-ram-meter.

What I think happened to Anastasia's Hydrogen

I have not read the Mac code or her logs, so these are the two likely causes, and her log from yesterday would say which:

  1. macOS killed the WebKit content process behind the chat views. Under pressure macOS jettisons web content processes first. The app keeps running and the view goes blank or frozen, and it stays that way unless the app handles webViewWebContentProcessDidTerminate and reloads. This is my top suspect, because the app survived and only the rendering died.
  2. The guest OOM-killed or thrashed code-server. A 4 GB VM next to Premiere has no room, and without item 1 above code-server is as likely to be killed as anything else.

Suggested changes for Mac Hydrogen, in order

  1. Stop forcing 1/4 of RAM. Size the VM generously and give memory back with a balloon and/or cache drops instead (above).
  2. Reload a killed web view. Handle webViewWebContentProcessDidTerminate on every WKWebView: reload it, log it, and tell the user in one line. This alone likely would have saved Anastasia's session.
  3. Port the OOM priority as-is. Same drop-in, same reset script, same two gate checks. It is Linux inside Lima, so the files are identical.
  4. React to macOS memory pressure. Listen with DISPATCH_SOURCE_TYPE_MEMORYPRESSURE. On warn/critical: inflate the balloon or drop guest caches, and suspend hidden web views.
  5. Stop drawing when hidden or minimized, like hidden_quiet.rs.
  6. Add the honest RAM meter with the "other programs are using X GB" hover, so users see it is the Mac, not Hydrogen.
  7. A watchdog that restarts the app's control layer if it stops answering, like the Windows one.
  8. A gate test that reproduces this: fill host memory with a stand-in for Premiere (a process that allocates and touches most of RAM), run an AI job in the workspace, and assert the chat views still render and the editor survives.

Related: #14 (Drew's memory-pressure suggestion; these two can be worked together).

Filed by John's AI agent at his request.

0 Replies

Kyle Bergstedt closed this · 4d ago
Kyle Bergstedt reopened this · 4d ago

Log in to reply.