Closed general

Page create is not crash-atomic: an orphan bare repo bricks the slug (409 on create, 404 on push)

Colby Knox · 1mo ago ·closed by Colby Knox

Found on staging while verifying #149. Page create is not crash-atomic: it creates the bare repo on disk before inserting the page row. The staging container restarted (redeploy) mid-request, after createRepo and before the INSERT, and the slug is now bricked both ways:

  • POST create again: 409 "kcknox/component/ztest-qa-149 already exists" (the repo layer sees the orphan bare repo)
  • repo push: 404 "Page not found" (no page row)

No recovery path exists short of shell access to rm the orphan repo dir. The orphan kcknox/component/ztest-qa-149.git is still sitting on staging's disk as a live repro.

Two fixes worth considering, smallest first:

  1. Self-heal on create: when createRepo hits an existing bare repo whose owner/type/slug has NO page row (and the repo has no commits, or only commits by the same caller), treat it as the orphan it is: adopt or remove it and proceed. This unbricks the slug with no migration.
  2. Order the work so the window closes: insert the page row first (or wrap repo creation and the insert so a crash leaves either nothing or a row that a retry completes). The repo is the expensive-to-orphan half, so it should come second.

Either way a sweep that lists bare repos with no matching page row would find any existing orphans on prod.

1 Reply

Colby Knox · 1mo ago

Shipped to prod (tag aa1366a), fix 1 plus the sweep:

Self-heal on create. When create hits an existing bare repo whose type/slug has NO pages row under ANY owner (not even soft-deleted), the orphan is removed and the create proceeds. The any-owner, include-soft-deleted check is what makes the removal safe: an undeleted page must keep its repo, and resolveBare's legacy fallback means an owner-scoped create can land on a legacy-path repo another owner's unmigrated row still references. A repo with no row at all was never pushable (push 404s without the row), so nothing reachable can be lost. 20-assertion suite covers: planted orphan heals to a 201, a live row still 409s, a soft-deleted page's repo survives a recreate attempt.

The sweep. GET /api/admin/orphan-repos (admin, list-only) walks both repo layouts and lists repos with no matching row. Run against prod it found exactly ONE orphan fleet-wide: repos/skill/core.git on the legacy path, no row ever (looks like pre-migration residue of core's early skill-type incarnation, long before the owner-keyed layout). Nothing deletes it today, but note the interaction: if anyone ever creates a skill named core, the heal will treat that legacy repo as the orphan it is and remove it. If its history matters to you, archive it manually first; otherwise it can sit.

Fix 2 (reordering) deliberately not taken. Inserting the row first just flips which half orphans on a crash, and the codebase already tolerates row-without-repo everywhere (tarball-only pages), so the heal closes the user-visible brick without creating new intermediate states.

The staging repro has evolved out from under the issue: kcknox/ztest-qa-149 now HAS a row, soft-deleted by kcknox on 2026-08-21, so the slug is no longer bricked, it is an ordinary deleted page (undelete restores it). The new code was verified against it live anyway: the recreate attempt 409s through the protected branch rather than healing, which is exactly right for a soft-deleted page.

Log in to reply.