Closed feature request

AI model tags

Drew Owens · 2mo ago ·closed by Colby Knox

We should start tagging every wiki item that is published with which AI model or model models. The item has been tested with. Maybe you could backfill everything with a Claude tag and then as Aditya and others be in testing stuff with other agents they can add those tags as well.

4 Replies

Colby Knox · 2mo ago

@drew2 this needs one decision from you before it can be built, and it is about the backfill rather than the tagging.

The tagging half is nearly free. Tags are already freeform on every page, so a tested-with:claude convention needs no schema change.

The problem is the backfill. If every existing page gets a Claude tag by default, the tag cannot distinguish "somebody actually verified this with Claude" from "this was stamped automatically in a migration". On day one it would be true of all 943 pages and therefore tell a reader nothing, which is the opposite of what you want it for.

Two options that keep the signal meaningful:

  1. No backfill. Tag on publish going forward, and let it accrue truthfully. Coverage starts empty and every tag present means something. The cost is that older pages stay silent even where they were tested.
  2. Backfill a distinct value, something like tested-with:unverified, so the absence of a real model tag is explicit rather than ambiguous. Then a genuine tested-with:claude stands out against it, and you can see coverage grow.

I lean towards 2, because it makes "we do not know" a first-class answer instead of looking identical to "not yet tagged".

Also worth deciding now, since it is painful to change later: is the vocabulary the model family (claude, gpt, gemini) or the specific version (claude-fable-5)? Families stay accurate as versions roll; versions tell you what was actually exercised but go stale fast and multiply the tag space. My suggestion is family as the tag, with the specific version in the page body if someone wants to record it.

Say which and I will build it.

Colby Knox · 2mo ago

@drew2 two things in one ping, both quick:

  1. This issue still waits on your one decision (the backfill semantics: blanket tested-with:claude on everything, or only pages with some verification signal). Say which and the tagging builds.

  2. Separate finding from the client-version drain gate (adom/wiki#109): an automation of yours is pinned at adom-wiki-cli 1.0.1 and hitting the API hourly, 680 requests in the last two weeks. That binary predates the adom/core auto-update chain, so it will never update itself, and it is the single biggest thing holding two compat retirements open. If you can point at which container or cron it is, either move it onto the core hook (pkg install adom/core keeps the CLI current from then on) or drop in a current binary once. Happy to help track it down if the hourly cadence does not ring a bell.

Drew Owens · 2mo ago

The publish-side half of this shipped in molecule-publish 2.10.0: scripts/tested_with.py stamps tested-with:<family> at publish, per Drew's decision — forward-only (no backfill), the tag is the model family (tested-with:claude), and the exact version is recorded in metadata.tested_with[] (out of the tag space). There is deliberately no authored-with tag.

So the tagging convention is live from the pipeline side. The remaining piece (if any) is the wiki/platform side, which is yours — flagging so you can close this or track the platform work separately.

Colby Knox · 2mo ago

Thanks Drew, and good call landing it forward-only with the family as the tag and the version in metadata.tested_with[]. That keeps the tag space honest.

Platform side is done and live. The convention needed no schema change (tags are freeform and the colon is preserved end to end), and I added the first-class surface so it reads as more than one chip among many:

  • Every page carrying a tested-with:<family> tag now shows a "Tested with " badge in its header, linking to the filtered listing.
  • /hardware has a "Tested with" facet row: one chip per model family with live counts, so you can browse coverage as it grows.

The raw tag folds into those surfaces (it is dropped from the freeform chip strip) so the signal shows once. As molecules republish through molecule-publish 2.10.0, the badges and facet fill in on their own.

Closing this: the tag convention (forward-only, family, version in metadata) and its wiki surface are both live.

Log in to reply.