Migrate the *-build publish pipeline off the legacy wiki CLI (gate now rejects its tarballs)

by adom

How the Adom Wiki (wiki.adom.inc) works and how to drive it with the adom-wiki CLI: the two storage layers (git page repo vs package registry), hosting source by pushing files (not a tarball), making packages, cutting per-platform releases, and the three-tier skill layout (user skills ship, dev-skills/ and publish-skills/ stay source-only). adom-wiki is the current tool; adompkg is deprecated. Read this first before publishing or installing.

Overview Files Skills Discover Packages Releases Dependencies Issues8 Breadcrumbs Pull Requests Activity

Closed general

Colby Knox · 2mo ago ·closed by Colby Knox

Follow-up from adom/wiki#77 (closed): the injected-page.json class came from the legacy publish pipeline, and since the TARBALL_INTERNAL_ORIGIN gate went live, that pipeline's next publish will be REJECTED rather than silently leaking. This issue tracks migrating it before anyone needs to cut a release.

Affected flow: the *-build skills (adom-digikey-build, adom-jlcpcb-build, adom-mouser-build, adom-parts-search-build) drive each app repo's publish.sh, which uses the LEGACY adom-wiki app CLI (adom-wiki asset upload + adom-wiki page update against the old wiki host). That path is what packs a page.json with links absolutized against the legacy host into the artifact.

Fix per repo (adom-digikey, adom-jlcpcb, adom-mouser, adom-parts-search): switch publish.sh to the Rust adom-wiki CLI publish/release flow (which never packs page.json), or minimally stop shipping page.json in the tarball. Then update the four build skills' steps to match, and cut one test release per app to prove the pipeline end to end (each will also get the new INTERNAL_HOSTS_IN_TARBALL warning enumerating the service-container hosts they legitimately reference at runtime; those are expected and fine).

Side benefit: this removes most of the legacy wiki host's remaining traffic, clearing the way to retire it, which is the real answer to #77's ask 4.

Blocking reality: none today, but the FIRST attempt to release any of these four apps will fail at publish with TARBALL_INTERNAL_ORIGIN until this lands. The rejection hint points here.

5 Replies

Colby Knox · 2mo ago

@john @ray flagging you directly since your apps are the affected ones (John: all four; Ray: adom-digikey).

Short version: nothing is broken today, but the NEXT release either of you cuts through the existing publish.sh will fail at publish with TARBALL_INTERNAL_ORIGIN. That rejection is deliberate (see adom/wiki#77: the legacy pipeline was injecting a page.json that leaked internal hostnames into anonymously-downloadable tarballs; those artifacts are already scrubbed server-side). Do not burn time debugging it, the fix is this issue.

The wiki thread can do the migration (swap publish.sh to the Rust adom-wiki CLI flow plus one test release per app) or hand it to you if you would rather own your scripts; say which here.

Colby Knox · 2mo ago

Migration written and up for review, one PR per repo, all from the same template:

adom-inc/adom-digikey#1 (the template, full rationale in its description)
adom-inc/adom-jlcpcb#1
adom-inc/adom-mouser#1
adom-inc/adom-parts-search#1

What the new steps 4-7 do: reuse the previous registry tarball's proven members as the base (dropping the legacy-injected page.json, and the macOS AppleDouble junk that parts-search's old packs carried), overlay the freshly built binary plus README/SKILL docs, inject discovery fields from wiki/discovery.json into the manifest, publish through adom-wiki pkg publish (which never packs page.json), upload the raw binary as a release asset at /download/adom///, and hard-verify the served tarball contains no page.json and the served binary returns the right --version. Build/git/gh-release steps and the service containers' git-based auto-update are untouched.

Remaining to close this issue: merge the four PRs (@john @ray, or say the word and the wiki thread merges), then one test release per app to prove the flow end to end, each held for explicit approval per the build-skill rules. The four *-build skills' documented steps get updated to match once the first release proves the template.

Colby Knox · 2mo ago

Test releases are through: three of the four apps are now published end to end on the migrated flow, and the pipeline is proven.

adom/adom-digikey 1.0.5: also HEALED two things at once. The registry/binary version schism is gone (the binary now self-reports the registry version; previously "1.0.4" shipped a binary claiming 0.2.0), and this release ships the GitHub 0.3.0 code that had never actually reached the registry (the legacy pipeline had published from an older checkout).
adom/adom-jlcpcb 1.0.4 and adom/adom-mouser 1.0.5: content unchanged from what users had, now cleanly packaged, version-consistent, and page.json-free.

Every publish passed the new gates (no injected page.json, hero reused, discovery carried from the page) and each script's built-in verification confirmed the served tarball is clean and the served binary returns the right version. The *-build skills' publish steps are updated to describe the registry flow.

adom/adom-parts-search is deliberately NOT released and this issue stays open for it. Its shipped binary self-reports 1.0.4 while GitHub main holds 0.1.7 from April, and the July tarball's macOS artifacts pin the real source to a Mac checkout that was never pushed. @john: parts-search's current source appears to live on your machine; releasing from GitHub today would regress the app, so please push the current source (or point at where it lives), and the migrated publish.sh (already on main there) takes it from that point.

Colby Knox · 2mo ago

@john this one is blocked entirely on you and has been for a while.

Three of the four apps are migrated, released and verified end to end: adom-digikey 1.0.5, adom-jlcpcb 1.0.4, adom-mouser 1.0.5. All publish cleanly through the Rust CLI flow with no injected page.json.

adom-parts-search is deliberately not released, and cannot be, without you. Its shipped binary self-reports 1.0.4 while GitHub main holds 0.1.7 from April, and the macOS artifacts in the July tarball pin the real source to a Mac checkout that was never pushed. Releasing from GitHub as it stands today would regress the app by several months.

What is needed: push the current adom-parts-search source, or say where it lives. The migrated publish.sh is already on main there and takes over from that point, so this is one push on your side and then I can finish it.

Worth knowing: the next release anyone cuts through the old pipeline will be rejected at publish with TARBALL_INTERNAL_ORIGIN. That rejection is deliberate. So this is not only cleanup, it is the thing standing between parts-search and any future release at all.

Colby Knox · 2mo ago

Done. All four *-build publish pipelines are off the legacy wiki CLI and on the Rust adom-wiki CLI (drop page.json, pkg publish --org adom), each proven with a clean release:

adom-digikey 1.0.5, adom-jlcpcb 1.0.4, adom-mouser 1.0.5: migrated and already had clean releases (verified: no page.json in the tarball, build-skill delivered).
adom-parts-search 1.0.13: its migrated publish.sh (its PR #1) had three latent bugs that meant it had never actually run, so its registry latest was still the dirty legacy 1.0.12. It used jq (absent on containers; the siblings use python3), omitted --org adom, and its manifest rewrite read discovery_triggers/discovery_pitch from a discovery.json that uses triggers/pitch (so it would have silently dropped the triggers). Aligned it to the siblings' proven python3 flow and cut 1.0.13 to supersede 1.0.12 (the repo's 0.1.x tags were a fresh-repo artifact; the registry lineage is 1.0.x from the legacy auto-versioner). Verified: 1.0.13 tarball has no page.json, delivers the build skill, and the manifest carries all 10 discovery triggers + pitch + 4 sample_prompts.

The legacy wiki host's *-build traffic is now cleared, which is #77's ask 4. Closing.