Closed bug report

Concurrent installs corrupt each other: pkg install extracts in place with no lock (needs atomic extract + install lock)

Colby Knox · 5d ago ·closed by Colby Knox

Repro'd on prod today. A container's auto-update failed with:

cp: cannot stat 'skills/floating-windows/SKILL.md': No such file or directory
error: script exited with exit status: 1: .../adom_modules/adom/adom-ui-design/install.sh

First read said "broken package." It isn't: the tarball ships that file, and the same install succeeds on a clean retry (verified; the package sits at current 1.3.6 and installs fine). The file was missing at the moment install.sh ran because a second installer had just wiped and was mid-repopulating the same directory.

The mechanism (src/commands/pkg/install.rs:742-751)

let module_dir = state::module_dir_for(owner, slug);
if module_dir.exists() { remove_dir_all(&module_dir); }   // wipe
create_dir_all(&module_dir)?;
// tar xzf <cache> -C <module_dir>                        // extract IN PLACE, non-atomic
// ...then run install.sh with CWD = module_dir

Two installs of the same package interleave freely: B's remove_dir_all + partial extract lands between A's extract and A's install.sh, so A's script sees a half-written tree. No per-package lock, no process-level lock, and the extract is visible to other processes from its first file onward. (Same shape on the uninstall path at lines 173-179.)

Why this now happens in normal life

adom/hook 1.1.8 added a cron trigger (*/30 + @reboot) alongside UserPromptSubmit, so a cron tick can land on a prompt and both fire pkg update + pkg install adom/core into the same module dirs. Hook 1.2.0 serializes the hook's own runs with a flock, so the scheduled triggers no longer race each other. But that only patches one caller: a manual pkg install racing the hook -- or any two CLI invocations -- still corrupts. The vulnerability is in the CLI, and only the CLI can close it for all callers.

Proposed fix (two layers, belt and suspenders)

  1. Atomic extract. Extract to a sibling temp dir, then rename into place: tar xzf -C <module_dir>.tmp-<pid> then remove_dir_all(module_dir) then rename(tmp, module_dir). Rename is atomic on the same filesystem (a sibling path guarantees that), so no process ever observes a half-written tree -- the dir is either the old version or the new one, complete. Sweep stray .tmp-* from crashed installs on the way in.

  2. A per-invocation install lock, so two concurrent pkg install/pkg update runs serialize instead of interleaving wipe/extract/script phases. Simplest correct form: one exclusive flock on a file next to .installed.json (state::prefix().join(".install.lock")) held for the whole operation. Coarse, but these runs take seconds and correctness beats parallelism here. It also protects the shared .installed.json / .lock.json maps (src/core/state.rs:26-29), which two concurrent writers can currently clobber last-write-wins. Prefer blocking with a "waiting for another install" note over failing, so the second invocation completes instead of erroring.

Layer 1 without 2 still lets two runs fight over the same dir (each atomic, but flapping); layer 2 without 1 leaves a crash mid-extract visible as a broken tree. Together: any number of concurrent callers, each package dir always complete, last completed install wins.

Repro

Launch two pkg install adom/adom-ui-design --force (any script-bearing package) simultaneously. Today one intermittently dies with cp: cannot stat from its install.sh. After the fix: both complete (one waits), and the module dir is never observable in a partial state.

One more datapoint

This failure is loud now only because hook 1.1.7+ records update exit codes. Before that, the race landed silently: half-installed skill dirs with no error anywhere. It has probably been happening at low frequency for a while; we just could not see it.

1 Reply

Colby Knox · 5d ago

Fixed in adom-wiki-cli 1.0.39. Both layers shipped as specified.

Reproduced first. Isolated prefix (ADOMPKG_PREFIX) plus a faked HOME, so the real adom_modules and ~/.claude/skills were never touched (worth noting: install.sh copies into $HOME/.claude/skills, and a mid-extract source file can be cp'd over a good skill, so the repro as filed is not safe to run against a live container). 30 concurrent installs of adom/adom-ui-design: 23 failed.

Two distinct faces, not one. The reported cp: cannot stat 'skills/<x>/SKILL.md', and also tar itself dying with 'failed to extract tarball' when its own target dir was wiped mid-extract by the other installer.

After the fix: 0/30, with 20 'waiting for another install' notes confirming the lock serialized, no stray staging dirs, and a complete 13-file tree with all 7 skills.

Layer 1, atomic extract. Stage into a sibling <module_dir>.tmp-<pid>, then rename into place. Sibling guarantees same filesystem, so the rename is atomic and the dir is only ever the old version or the new one, whole. Stale staging dirs from crashed installs are swept on entry (safe because the lock means no other installer is live).

Layer 2, install lock. One exclusive lock on PREFIX/.install.lock, taken at every mutating entry point (install/add/uninstall/update/ci/bootstrap) and held for the process, blocking with a note rather than failing. This also closes the last-write-wins clobbering of .installed.json / .lock.json. Idempotent via a OnceLock, because flock is per-descriptor and locking the same file twice in one process would deadlock against itself. A filesystem that cannot flock warns loudly and continues rather than bricking the install, since layer 1 keeps each dir whole on its own.

Used std::fs::File::lock (stable since 1.89), so no new dependency and Windows is covered (libc in this crate is unix-only).

Two notes on the issue itself: pkg install has no --force flag, and it skips when the package is already installed at that version, so the repro line needs the install state cleared each round to actually re-extract. Your diagnosis of the mechanism and the proposed two-layer fix were exactly right.

Log in to reply.