Cross-repo, author-scoped commit activity feed

by adom

How the Adom Wiki (wiki.adom.inc) works and how to drive it with the adom-wiki CLI: the two storage layers (git page repo vs package registry), hosting source by pushing files (not a tarball), making packages, cutting per-platform releases, and the three-tier skill layout (user skills ship, dev-skills/ and publish-skills/ stay source-only). adom-wiki is the current tool; adompkg is deprecated. Read this first before publishing or installing.

Overview Files Skills Discover Packages Releases Dependencies Issues8 Breadcrumbs Pull Requests Activity

Closed feature request

Colby Knox · 1mo ago ·closed by Colby Knox

Problem

There's no way to see "everything a given author committed" across every page/repo they touch, without going to GitHub and manually querying + merging per repo. That's what I've been doing by hand for weekly activity reports: query GitHub's commit search per repo, filter by author, merge and sort, then cross-check the wiki's own release history to catch cases where a shared package (adom-lbr, adom-parts-search) has commits from more than one person mixed together.

Pain points with the current state:

GitHub's search API has a secondary rate limit that bites on repeated author-filtered queries in one session
No cross-repo aggregation exists anywhere, on the wiki or on GitHub. Each report means N separate queries, one per repo, stitched together by hand
A page's own git log on the wiki (adom-wiki repo log <owner>/<slug>) is too shallow to help here. It only records one commit per release (Publish X.Y.Z), not real dev history, since publishers typically only push a release snapshot rather than mirror every commit

Proposal

An internal, author-scoped activity feed, not a public one.

Ingest commits from each page's linked GitHub source via webhook (or periodic authenticated sync) instead of ad hoc search calls. Push-based and cached, so no per-query rate limit
Key entries by author + page + timestamp; store enough to categorize (repo, message, files touched)
Surface it the way the existing /users/<handle> publisher dashboards already work: visible to the account owner (and admins), not blasted onto the public page
Span every page/repo an author touches, so the cross-repo merge that's tedious today happens once, server-side

Why not just mirror full GitHub history into the page's public git log instead

Public per-page history should probably stay as-is (release-only commits). Wiki already shipped "Stop shipping internal engineering commentary to visitors," which strips postmortems, security reasoning and internal issue numbers out of what ships to visitors. Mirroring full raw commit messages onto a page's visible history would undo that same principle: dev commits often carry exactly that kind of internal detail. The activity feed should be a private, author-scoped view built from ingested commit data, not a change to what a page's public git log shows.

Scope note

Happy to sketch the data model (commit ingest table, dedup-by-sha, author resolution against Carbon identity) if useful before implementation starts.

3 Replies

Colby Knox · 1mo ago

Picking it up. The sketch I am building against, so you can redirect early:

Data

commit_activity: one row per (repo, sha): repo (owner/name on GitHub), sha, branch, author login, name, email, resolved wiki user id (nullable), committed_at, message (first 2000 chars), files touched (JSON, when the source carries it), source (webhook or sync), ingested_at. Dedup by (repo, sha).
activity_repos: the repos we ingest, optionally mapped to a wiki page (from page.json repository.url, or set by an admin), with the last sync cursor.
activity_identities: how a commit author becomes a wiki user: claimed GitHub logins and extra emails per account (adom-wiki-side claim, admin can set for anyone), plus an automatic match on the account email. Unresolved authors stay keyed by login so nothing is lost; resolving later backfills.

Ingest

POST /api/webhooks/github: GitHub push events, HMAC verified with GITHUB_WEBHOOK_SECRET. One org-level webhook on adom-inc covers every repo, including ones with no wiki page. Idempotent.
A periodic sync (when GITHUB_TOKEN is set) walks the registered repos with since= cursors, so a webhook gap self-heals; admin can trigger it. No per-query search calls, no secondary rate limit.

Read side (private)

GET /api/users/<handle>/activity?since&until&repo&format=json|md: owner and admins only. JSON for tooling; md is the weekly-report shape, grouped by repo with dates and first lines, so the hand-merged report becomes one call.
/users/<handle> gets an Activity section when you are looking at your own page (or you are an admin): range picker, repo groups with counts, commit rows linking to GitHub, copy-as-Markdown, and your claimed identities. Nothing on public pages; the page git log stays release-only as you argued.

Hints: a page.json repository.url on GitHub gets ACTIVITY_SOURCE_LINKED on push and publish (whether ingest is configured, and how to claim your login if commits show unresolved).

Ops after it lands: two env vars on prod (GITHUB_WEBHOOK_SECRET, optional GITHUB_TOKEN) and the org webhook at adom-inc pointing at the endpoint. I will hand you the exact settings with the ship note.

Colby Knox · 1mo ago

Shipped (prod tag b65d51e). It follows the sketch above; the page's public git log is untouched.

What is live

POST /api/webhooks/github ingests GitHub push events (HMAC checked, idempotent by repo and sha). Branch, author login/name/email, message and the files touched are kept per commit.
With GITHUB_TOKEN set, a sync every 30 minutes walks the registered repos with a since= cursor (first pass: the last 90 days), so a webhook gap heals itself. POST /api/admin/activity/sync runs it on demand.
Attribution: account email, plus claimed GitHub logins and extra emails (POST /api/users/me/activity/identities { "kind": "github_login", "value": "<login>" }; a claim backfills earlier unattributed commits). Unattributed authors are listed for admins and in the ACTIVITY_UNRESOLVED_AUTHORS hint when your range is empty.
GET /api/users/me/activity?range=last-week (also week, 7d, 30d, 90d, or since=/until= days), JSON or &format=md for the weekly report grouped by repo with dated first lines. Owner or admin only; anyone else gets a 404.
/users/<handle> shows a Commit activity section to the owner and admins: range picker, per-repo groups with counts, commit rows linking to GitHub, the Markdown report link, your identities.
A page.json repository.url on GitHub links the repo to its page (ACTIVITY_SOURCE_LINKED on push and publish); admins can also map repos by hand (POST /api/admin/activity/repos { repo, page }), and the feed spans repos with no page at all.
Docs: docs/activity-feed.md and the API reference section.

What you need to do once, on prod

In Dokploy, set GITHUB_WEBHOOK_SECRET (any long random string) and, if you want the sync, GITHUB_TOKEN (a fine-grained token with Contents: read on the adom-inc repos). Redeploy.
On GitHub, adom-inc org settings, Webhooks, add one: Payload URL https://wiki.adom.inc/api/webhooks/github, content type application/json, the same secret, just the push event. GitHub's ping on save should answer { "pong": true }.
Claim your GitHub login with the curl above so commits authored under a noreply address or a non-account email count as yours. Until the secret is set the route answers 503 and the section says ingest is not configured.

The CLI side (adom-wiki activity [--range] [--md] wrapping the read route, and an activity claim <login> verb) is with the CLI thread as adom/adom-wiki-cli#52.

Colby Knox · 1mo ago

Corrected after Colby's note: the feed uses nothing outside this wiki. The GitHub webhook receiver and token sync that shipped first are removed (prod tag 2f9053b); no request to GitHub was ever made and nothing was configured.

Source now: this wiki's page repos. A push through the wiki (repo push, the editor, the files API) is recorded the moment it lands and attributed to the account that pushed it, exactly, no email guesswork. A publish's materialization commit and a merged PR's merge commit are recorded the same way. A sweep every 10 minutes (admins can run it on demand) reads any page repo that moved since the feed last saw it and attributes older history by author email, so the last 90 days of existing history fill in on their own after this deploy.

Read side (unchanged): GET /api/users/me/activity?range=last-week (also week, 7d, 30d, 90d, or since=/until= days), JSON or &format=md for the weekly report grouped by page with dated first lines; owner or admin only. /users/<handle> shows the Commit activity section to the owner and admins: range picker, per-page groups with counts, commit rows linking to the page's commit log, the Markdown report link, claimed emails.

Attribution: account email, plus any address you claim (POST /api/users/me/activity/identities { "kind": "email", "value": "<address>" }; a claim backfills). Unattributed author emails show on the admin overview (GET /api/admin/activity) and in the ACTIVITY_UNRESOLVED_AUTHORS hint when a range comes back empty.

Caveat you named in the issue: page repos carry what gets pushed to them, which for many packages is one commit per publish. The feed is complete for everything that happens on the wiki and only that; dev history that never reaches a page repo is out of scope by design.

Nothing to configure. Docs: docs/activity-feed.md and the API reference section. CLI verbs: adom/adom-wiki-cli#52 (updated).