master
Drew Owens Publish 1.6.9 3961820 2mo ago

Handoff: wiki-notify-gchat — drop "ghost" issues (deleted/renumbered) during status sync

Owner of this package: Drew (adom/wiki-notify-gchat, published v1.4.0, auto-updates via adom-hook) File to change: bin/wiki-notify-gchat — function sync_issue_status() (currently ~lines 259–319) Severity: minor. Board over-counts "open" by however many ghost items have accumulated. No data loss, no crash.


Symptom

The "Wiki Notifications" board card showed 30 open · 0 closed when only 28 issues were actually open. Two tracked items — adom/molecule-publish#164 and #165 — no longer exist on the page (molecule-publish's live discussions are #1–#18; those were deleted/renumbered on the wiki after the board ingested them from email). They sat as permanent phantom "open" rows.

I hand-cleaned these two from state.json (took the run.lock first, backup at state.json.bak-*), so the live board is correct right now. This handoff is the durable code fix so it can't recur.

Root cause

sync_issue_status() reconciles board items against the wiki DB via adom-wiki issue list <page> --json. Its cleanup logic only drops items when the whole page is gone:

for page, its in by_page.items():
    code, out, _ = run_cmd(["adom-wiki", "issue", "list", page, "--json"])
    if code != 0:
        if "PAGE_NOT_FOUND" in (out or ""):      # <-- only whole-page deletion is handled
            for it in its:
                items.pop(f"{it['page']}#{it['num']}", None)
            changed = True
        continue                                  # transient failure -> correctly skip (never delete)
    try:
        j = json.loads(out)
        discs = {str(dd.get("number")): dd
                 for dd in (j.get("data", j) or {}).get("discussions", [])}
    except Exception:
        continue
    # >>> GAP: page listed fine, but a tracked issue NUMBER is absent from discs.
    for it in its:
        dd = discs.get(str(it["num"]))
        if not dd or dd.get("status") not in ("open", "closed"):
            continue    # <-- ghost item lands here every pass and is skipped forever

When the page lists successfully but a tracked num isn't in discs (issue-level delete, or the issue was renumbered), the item is silently continue-skipped — never flipped, never dropped. It lives forever as "open."

The fix

When we have an authoritative, complete discussion list for the page (code == 0, JSON parsed), treat a tracked item whose num is absent from discs as gone → drop it (same as the page-deleted branch), with a log line.

Suggested change to the per-item loop:

    for it in its:
        dd = discs.get(str(it["num"]))
        if dd is None:
            # Page listed fine but this issue number is gone (deleted or renumbered).
            # The list is authoritative, so this is a definitive absence, not a
            # transient failure -> drop the ghost so it stops inflating the board.
            items.pop(f"{it['page']}#{it['num']}", None)
            changed = True
            log_line(f"OK status sync: {it['page']}#{it['num']} dropped (issue not on page)")
            continue
        if dd.get("status") not in ("open", "closed"):
            continue
        ... # existing open/closed flip logic unchanged

CRITICAL caveat the implementer MUST verify first — pagination

The "absent ⇒ deleted" inference is only safe if adom-wiki issue list <page> --json returns all discussions for the page, not a first page. If the endpoint paginates (e.g. default limit 20/50), a legitimately-present issue on page 2 would be wrongly dropped.

Before shipping:

  1. Check whether adom-wiki issue list --json caps results. Find a page with >50 issues (or inspect the CLI/help). The data.discussions array must be the full set.
  2. If it caps: page through it (or pass an explicit high --limit) and build discs from the complete union before applying the drop. Do not apply the drop against a truncated list.
  3. Confirm issue list includes closed issues, not just open. (It does today: adom/molecule-publish returned status counts {open:7, closed:11} — closed are present. Keep this assumption tested, since the whole close-detection path depends on it.)

Also keep the existing safety property intact: transient failures (network/auth, code != 0) must never delete state — only a clean code == 0 + parsed list authorizes a drop. Don't loosen that.

Optional hardening (nice-to-have, not required)

  • Re-ingest guard: the ghost originally arrived via an email whose URL used a number that doesn't match the API's per-page number. If email-URL numbering and API number can diverge, dropped ghosts could re-ingest from a re-seen email. The processed email-id dedup set already blocks re-processing the same email, so this is low-risk — but worth a note in the PR if you see URL-vs-API number skew.
  • Age gate (belt-and-suspenders): only drop if the item is older than, say, 10 minutes, to avoid racing an issue created seconds ago that a stale read hasn't surfaced. Given the list is live (not cached), this is likely unnecessary — implementer's call.

Test plan

  1. Unit-ish: seed state.json with a fake item adom/<realpage>#99999 (a number that doesn't exist), run wiki-notify-gchat run, assert the item is gone from state.json and a dropped (issue not on page) line is logged.
  2. No-false-drop: seed a real open issue, run, assert it is retained with status: open.
  3. Transient safety: temporarily break adom-wiki (e.g. bad PATH/creds so code != 0), run, assert no items are dropped.
  4. Pagination (if applicable): on a page with more issues than one API page, assert an issue only reachable on page 2 is retained.

Ship

  • Bump package.json version (patch), update README/changelog with a one-liner ("status sync now drops issues deleted/renumbered off a page, not just whole-page deletions").
  • Republish to adom/wiki-notify-gchat. It auto-updates on installed instances via adom-hook.

Context / provenance

Diagnosed 2026-08-05. The board also legitimately shows "0 closed" most of the time by design — closed items are purged ~1 day after closing (purgeHour). And the board can look "frozen" when idle because --quiet runs emit no log line on a no-op; that's expected, not a fault. Neither of those is a bug — the ghost-item drop is the only real defect here.