← Commit history

docs: DESIGN.md - design rationale, dead ends, and ops lessons for future devs

Drew Owens ·5b206724b0 ·3mo ago ·parent e05c867
3 files changed +161−1
README.md+10
@@ -172,6 +172,16 @@ All commands accept `--json` for a `{status, data, hints}` envelope. | `~/.local/state/wiki-notify-gchat/activity.log` | one line per ingest/board/purge | | `~/.local/state/wiki-notify-gchat/cron.log` | cron pass output | +## For future devs++[docs/DESIGN.md](docs/DESIGN.md) records why things are built this way: the+Chat API constraints (webhooks are create-only; only apps can send cards;+the delete-repost swap and the deleted-message-tombstone trap), the card+layouts we tried and killed (pipe tables, grids), the Gmail search quirks,+the two-source involvement line, the cron/cold-boot lessons, and why a+central service container was deliberately rejected. Read it before+redesigning anything.+ ## Troubleshooting  - `health` says not authorized: run `adom-google auth`.
docs/DESIGN.mdadded+150
@@ -0,0 +1,150 @@+# Design notes - why wiki-notify-gchat is built the way it is++Context for future devs: the constraints we hit, the dead ends we tried, and+the reasoning behind the current shape. Built 2026-07-16 by Drew + Claude in+one long iterative session, live against real mail and a real Chat space.+Read this before "improving" something - several obvious-looking alternatives+were tried and killed for non-obvious reasons.++## The problem, and the pivot++The original ask was a generic "forward matching Gmail to a Chat space and+delete the email" tool (`mail-gchat`, v1.0.0 - shipped and replaced the same+day). Per-message forwarding turned out to be the wrong model: it moves the+clutter from the inbox into Chat. What Drew actually wanted was a **standing+board**: open wiki issues stay in front of you until resolved, closed items+linger long enough to enjoy, then disappear (6am the next day). That pivot -+from a stateless forwarder to a stateful item board - drove every design+decision below.++## Google Chat constraints (the load-bearing discoveries)++These were all verified empirically against the live API; the errors are+quoted so you can recognize them.++1. **Incoming webhooks are create-only.** PUT/PATCH/DELETE with the+   webhook's `key+token` fail with `401 "API keys are not supported by this+   API"`. The docs confirm: webhooks post messages, full stop. So a+   "continuously updated card" cannot be updated in place via webhook.+2. **Only apps can send cards.** User OAuth (`chat.messages`) can create and+   edit its own *text* messages but cannot send `cardsV2` at all. So "post+   the card as the user and edit it" is also impossible.+3. **The escape hatch: delete-repost swap.** A user CAN delete the webhook's+   messages - including card messages - **if they are a space manager**,+   which they are, because they created the space. So every board change is:+   webhook POSTs the fresh card, then `adom-google` deletes the previous one.+   Net effect: exactly one board card, always the newest message in the+   space. This is why the setup docs insist each member uses a space *they+   created*.+4. **Trap: deleted messages still GET 200.** A deleted Chat message returns+   HTTP 200 with `deletionMetadata` set (a tombstone); re-deleting it returns+   403 "Permission denied ... or the resource doesn't exist". We initially+   misread this as "cards can't be deleted" and nearly redesigned around a+   nonexistent limitation. Check `deletionMetadata`, not the status code.++## Card layout - what we tried and killed++Chat cards have **no table widget**, the `columns` widget hard-caps at **2**+(extras silently dropped, no nesting), and text is a proportional font with+roughly 55-60 characters per line at desktop card width.++- **Single-line pipe table** (`STATUS | ITEM | TITLE | BY | OPENED | LAST |+  LINK`, one line per row): unreadable in practice - rows wrap mid-cell and+  columns smear together. Looked fine in JSON, mush on screen. Killed on+  sight.+- **Grid widget as a table**: `grid` officially supports any number of+  columns, but cells are equal-width tiles (TITLE gets the same width as+  STATUS), plain strings only (no color, no links), one click target per+  grid. We posted a live demo card with 5-col and 3-col variants; verdict+  was "gross". Killed.+- **Monospace code block in a text message** (would give true alignment):+  links are not clickable inside code blocks, no color, and it reads like a+  terminal dump. Never shipped.+- **What survived**: a 2-column `columns` widget per row with a `divider`+  between rows. Left: bold `page #N` with colored OPEN/CLOSED beneath.+  Right: description with the View link |'d at its end, then Created-by,+  last-event, and involvement lines. Column structure with *controlled* line+  breaks - a wrap can only happen inside the description, never across+  fields. The older per-field layouts remain as `style: "blocks"`.++## Gmail findings++- **Gmail search cannot match quoted URLs.** `"wiki.adom.inc"` returns zero+  hits and the bare keyword `wiki` matches URLs unreliably (we measured 5 of+  18 real notifications). Precision therefore comes from a broad sender+  query (`from:[email protected] in:inbox`) plus a `requireInBody`+  substring check on the fetched body. Do not "fix" the query by quoting.+- **`adom-google api` prints its `HTTP <status>` line on stderr**, stdout is+  pure JSON. Our first parser assumed stdout and silently truncated the+  body.+- **Safe-mode scope is enough.** `gmail.modify` covers trash, archive, and+  labeling (and `messages.insert`, which is how the e2e test injects a+  synthetic notification). No full-mode auth needed.++## The involvement line (and why it needed two data sources)++The email footer ("You're receiving this because ...") is **issue-scoped**:+if someone else opens an issue on a page you *created*, the wiki only says+"you watch this page" - page creators are just auto-watchers in its+notification model. That made the board claim Drew merely "watches" pages he+authored. Fix: cross-reference `adom-wiki page stats <page>` `author_name`+(cached per page in state) against the user's `fullName`. Precedence:+owns this fix (assigned) > opened this > owns this page > was mentioned /+commented > watches this page. Related: the footer is *authoritative* about+self-involvement - "you opened the issue" beats our creator guess, which is+only "actor of the earliest email we hold" and can be wrong when the+creation email predates the retention window.++## Ops lessons (each of these bit us)++- **cron runs with a bare PATH.** Every scheduled pass failed for hours with+  `adom-google: not found` while the board looked healthy (manual runs kept+  feeding it). The tool now resolves helper CLIs from `~/.local/bin` /+  `/usr/local/bin` explicitly. Test scheduled code under+  `env -i PATH=/usr/bin:/bin`, not your shell.+- **Cold boots stop cron, and humans never notice.** Fix is `ensure`:+  restart cron via passwordless sudo, else run a detached watch loop+  (retired automatically when cron returns). Wired three ways by+  `install-cron` - cron `@reboot`, a `~/.bashrc` snippet, and a Claude Code+  `UserPromptSubmit` hook (the adom-hook pattern) - so recovery rides+  whatever the user does first. Never ship "run sudo when you notice"+  as the recovery story.+- **Concurrent passes are real** (cron + watch fallback + ensure). `run`+  takes a non-blocking flock; a second pass exits quietly.+- **Auto-update is not this tool's job.** adom-hook's throttled+  `adom-wiki pkg update` already refreshes every installed package. We+  explicitly did not build an updater.++## The road not taken: a central service container++Seriously considered: one always-on multi-tenant container polling for the+whole team ("always ready to update"). Rejected because this tool wields+**personal** credentials - it reads your Gmail, trashes your email, and+deletes messages as you. Centralizing means every member's Google refresh+token on one box; the per-user design keeps each token 0600 on its owner's+container. The resilience goals were met locally instead (self-healing cron++ adom-hook auto-update). If you revisit this, know it was a deliberate+trade, not an oversight - the blocker is credential centralization, not+engineering effort.++## Testing approach++- `--dry-run` everywhere; `board --dry-run` prints the exact card JSON.+- Purge logic is unit-testable by importing the script+  (`SourceFileLoader("wng", "bin/wiki-notify-gchat")`).+- Full e2e without waiting for real wiki traffic: build an RFC822 message in+  the real notification format, inject it with Gmail `messages.insert`+  (`labelIds: ["INBOX","UNREAD"]`), run a pass under a cron-identical bare+  environment, then verify the board row and the `TRASH` label. Clean up by+  deleting the item from state and force-reposting the board.++## State model (quick orientation)++`~/.local/state/wiki-notify-gchat/state.json`: items keyed `page#num`;+events fold in with timestamp guards (listing order is newest-first, so+every mutation checks `ts` before overwriting); `processed` message-id map+dedupes ingestion; `lastRenderHash` (hash of items only, not timestamps)+prevents no-change reposts; `boardMsg` is the current card; `pageAuthors`+caches the ownership lookups. Purge drops closed items at `purgeHour` (in+the configured IANA `timezone` - containers are UTC, so leaving it unset+makes "6am" fire overnight in the US) on the day after `closedAt`.
package.json+1−1
@@ -2,7 +2,7 @@   "slug": "wiki-notify-gchat",   "title": "Wiki Notify for Google Chat",   "type": "app",-  "version": "1.4.0",+  "version": "1.4.1",   "description": "Turn wiki.adom.inc notification emails into a live Google Chat board. One card, one row per issue - newest first, OPEN/CLOSED status, origination date, actor, and a View link - continuously updated in your own Chat space via your own webhook. Emails are swept out of your inbox (trash, archive, or label). Closed items linger until 6am the next day, then drop off. Delegates all auth to adom-google; the only secret is your webhook URL, kept in your private config.",   "tags": [     "cli",