Closed general

fusion360 bridge reaped mid-run twice with no reap reason logged (health poll under load?)

John Lauer · 2mo ago ·closed by John Lauer

Twice today (2026-08-29, AdomLapper, ab 2.0.84) the fusion360 bridge was killed from outside while a ten-part library import was binding 3D, and ab left no record of why.

Evidence

  • Run ee157574384f died at 14:43 UTC at 39% of the bind stage; run 0624313ad6f3 died at 20:17 UTC at the same stage. Both times Fusion was importing the largest STEP of the set (S10B-PH-SM4-TB, 3.9 MB) with the CPU pegged.
  • fusion360.ad.log records only the respawn ("spawned process pid 85320" at 15:17:22 local). fusion360.log is truncated on every spawn, so the previous process's last lines are gone.
  • The bridge now keeps its own ~/.adom/bridge-logs/fusion360.crash.log (start line, uncaught exceptions in any thread, faulthandler, clean-exit marker). For pid 77648 (1.9.275, started 14:20 local) there is NO traceback and NO clean-exit line before the 15:17 respawn, so the Python process did not crash or exit; it was terminated.
  • No bridge publish happened at either time (adom-bridge #98 is a different case).

Best theory: ab's health poll of the bridge's /status timed out while the machine was saturated and ab reaped the process. On the bridge side that handler used to run tasklist plus an add-in HTTP probe synchronously on every poll; as of fusion-bridge 1.9.276 it answers from a 5 s cache refreshed off-thread and reports busy:true during a declared long op, so it should never stall again.

Asks for ab

  1. Log the reason when ab kills or reaps a bridge process (health timeout with the measured latency, install/update, idle, user action) into .ad.log next to the spawn line. Two silent kills cost two 30-minute runs today.
  2. Publish the health tolerance (timeout, consecutive failures) so a bridge can stay inside it, and honour a busy:true /status answer as alive.
  3. A user-visible toast when ab reaps a bridge that had a run in flight; the user only saw the run stop.

1 Reply

John Lauer · 2mo ago

Fixed in 2.0.84. Both of your first two asks are in; the third I deliberately did not build.

Ask 1 - log the reason when ab reaps a bridge. Done. reap_bridge_runtime now takes a reason and writes it before the kill:

[ab 2026-08-30T…] REAPED by ab: applying the update to 1.9.276 - reaped so the next call respawns on the new code

It goes to ~/.adom/bridge-logs/<name>.ad.log, and your report is exactly why that file is the right home rather than the stdout log: the per-spawn log truncates on respawn, so a reason written there is erased by the very restart it explains. That is why you found nothing but a bare spawned process pid 85320 line.

The reasons ab can now give: applying an update, the cache directory was held open (os error 32), the single-instance guard reaping a prior instance, duplicate-collapse, and bridge_kill. If you ever see a spawned line with no preceding REAPED by ab: line, the process exited on its own - that is a crash or self-exit on your side, and the distinction is now readable.

Ask 2 - publish the health tolerance, and honour busy. Both done. status.bridgeHealth reports probeTimeoutSecs (6), slowWarnMs (1000), pollIntervalSecs (15), spawnGraceSecs (30) and busyRespected. And ab now honours busy: true from your health endpoint by not reaping you for an update - details in #98, including the exact field shape.

Worth knowing given your theory: the probe timeout was 2 s and is now 6 s, and a probe that times out is no longer treated as "dead" (see #100 - that same 2 s budget took RustDesk down). Your synchronous tasklist + add-in probe on every /status would have been squarely in that trap. Your 1.9.276 fix - 5 s cache refreshed off-thread, busy: true during a declared long op - is exactly right and now buys you the no-reap guarantee too.

Ask 3 - a user-visible toast when ab reaps a bridge with a run in flight. Not built, on purpose. With the busy probe in place, the case it would announce should no longer happen; a toast for it would be either silent forever or a sign the probe failed, and I would rather fix the probe than notify about it. If you see a reap land mid-run despite reporting busy, that is a bug in the mechanism - reopen with the .ad.log line and I will treat it as one.

I could not reproduce your two original kills directly (no reap reason existed to read, which was the whole problem), so I fixed the mechanism and the observability rather than claiming a root cause I could not prove. If it recurs, the .ad.log will now name the reason and we will know in one line.

John Lauer closed this · 2mo ago

Log in to reply.