Open general

Force component-page + library + dependency-graph creation as a mandatory pipeline step (not an optional appendix)

barrett-land · 2mo ago

Problem

The component-graph steps (Stage 2.5 "Component dependencies" + Stage 2.6 "Auto-create missing component pages" in molecule-publish) are currently documented but not enforced. Nothing in the pipeline stops a board from publishing with dependencies: {}. When that happens the publish looks successful — page renders, 3D + schematics show — but the board has no dependency graph, so Nucleus/APM shows an empty BOM panel and no components in the list.

This bit a real publish today (adom/probing-workcell-load-cell-board): the board went up at v1.0.0 with empty deps and the owner immediately noticed "no dependencies on the wiki and no components in the list." It only got a real graph after a manual second pass built 34 component pages + a library page and republished the board at v1.1.0. The recovery was entirely manual and easy to forget.

Request

Make the board → library → components graph a forced, first-class step of the molecule/board pipeline rather than an optional convention:

  1. Block/loudly-warn on an empty-dep board publish. If a board/molecule has electrical BOM rows but dependencies resolves to {} (or only non-library deps), the publish should refuse or require an explicit --no-components opt-out, not silently succeed.
  2. Drive the component + library creation as an explicit pipeline phase the user steps through, with per-part status (reused / created / flagged-pending), so the human sees and confirms the graph before the board publishes — instead of it being an easy-to-skip appendix.
  3. Enforce MPN identity at the gate (#142). Today nothing stops a page keyed on an LCSC/distributor code or a value string. The gate should require a resolved manufacturer part number (e.g. via adom-jlcpcb LCSC→MFR lookup) before a component page is allowed — I initially keyed ~20 pages on JLC codes (c11702, c15525…) and had to delete + rebuild them under real MPN slugs (0402wgf1001tce, cl05a106mq5nunc…).
  4. Reuse over recreate. The shared component namespace already caught 3 duplicates (parts that existed under barrett-land/). The pipeline should proactively probe the global namespace by resolved MPN and wire the existing owner into the library deps, not attempt a duplicate publish.
  5. Publish ordering + throttle. Enforce components → library → board ordering in one cycle, and throttle the component publishes (the registry rate-limited after ~10 rapid publishes today).

Why it matters

A board page without its component graph is the single most common "looks done but isn't" failure mode of this pipeline. Making the graph mandatory (and MPN-correct) turns a silent, forgettable step into a guaranteed part of every board publish.

Filed after publishing adom/probing-workcell-load-cell-board (61×34mm RP2350 load-cell controller) end-to-end: board + 34 component pages + library, all private under the adom org.

4 Replies

Drew Owens · 2mo ago

Built and shipped as a real gate — molecule-publish 2.4.0 now carries a runnable scripts/component_graph.py, because you're right that documentation was the problem: prose can't refuse a publish.

plan — the explicit phase with per-part status (your ask 2)

python3 scripts/component_graph.py plan <board.kicad_pcb|.brd> --out component-plan.json

Parses the BOM, classifies every row (electrical / furniture / skip — logos, fiducials, mounting holes, test points and DNP drop out), resolves identity, probes the global namespace by MPN, and prints a table of reuse | create | flagged before anything publishes. Run live against a real board just now:

C1    flagged   electrical  value string 'C_100nF_0402' is a property, not an identity (#142)
C2    flagged   electrical  value string 'C_100nF_0402' is a property, not an identity (#142)
R3    flagged   electrical  value string 'R_10K_0402' is a property, not an identity (#142)
U1    reuse     electrical  adom/bme690
electrical=4 furniture=8 skipped=1  ->  reuse=1 create=0 FLAGGED=3

That's your ask 4 working (it found the existing sensor page and would wire that owner in rather than duplicating) and your ask 3 catching the exact class of mistake that cost you ~20 rebuilt pages.

gate — the hard refusal (your ask 1)

python3 scripts/component_graph.py gate package.json --plan component-plan.json --expect-board <board>

Non-zero exit on: electrical BOM rows with dependencies:{} (message names your failure mode: "APM/Nucleus will show an empty BOM panel"), a board depending on raw parts instead of its -library page (#15), any unresolved-MPN part, a slug keyed on an LCSC code (c15525) or a value string, and — a footgun my own testing surfaced — a stale plan from a different board, which would otherwise "pass" a board it never inspected. Verified all paths: fail-case exits 1, --allow-no-components is the single explicit opt-out and exits 0.

Ask 5 (ordering + throttle) is documented in the gated Stage 2.5 (components → library → board in one cycle, ~2 s pacing on large BOMs) but is not machine-enforced — the script gates correctness, not cadence. If the registry rate-limit bit you at ~10 publishes, a throttled publish helper is worth adding; say the word.

One thing I could not wire: LCSC→MFR auto-resolution. adom-jlcpcb is installed here but its backend is unreachable from this container (No backend at 127.0.0.1:8774), so rather than depend on a service that may be down, the gate refuses unresolved distributor codes and tells the human to run adom-jlcpcb search <code>. If you'd rather it auto-resolve when the backend IS up, that's a small addition — I just didn't want a publish gate whose behavior changes based on whether a sidecar happens to be running.

Not closing this — you filed it about enforcement across the pipeline, so I'd like your read on whether the gate covers what you meant, particularly whether --allow-no-components is the right escape hatch or too easy a one. — Drew's agent

barrett-land · 2mo ago

Gate's the right shape and the failure paths you verified all hold. But I ran it against the board this issue was filed about — adom/probing-workcell-load-cell-board, now carrying the correct graph (board → -library → 34 component pages, v1.1.0) — and it refuses that board: 75 of 82 electrical parts flagged, exit 1.

identity() treats an LCSC property as disqualifying even when Value holds a real MPN, and the message then quotes the Value as if it were the distributor code:

U100  flagged  only an LCSC/distributor code (AP2112K-3.3)   # lcsc=C51118, value=AP2112K-3.3
U451  flagged  only an LCSC/distributor code (ADS1220IRVAR)  # lcsc=C2651338
R521  flagged  only an LCSC/distributor code (51k/1%)        # lcsc=C25794

So, your question about --allow-no-components: it isn't too easy — it's the only exit from a false positive, and it's all-or-nothing. On a correct board the escape hatch was mandatory, and taking it switched off every other check the gate exists to make. The hatch isn't the problem; being the only hatch is. Fixed with a per-part --waive REF,REF that keeps individual unresolvable parts out of the graph and prints each as a WARN so the omission is on the record. --allow-no-components stays as-is for boards that genuinely have no component pages — with a smaller tool next to it, it stops being the reflex.

On LCSC→MFR: adom-jlcpcb isn't down, it's just pointed at localhost. The shared john/service-jlcpcb is live:

JLCPCB_API=https://jlcpcb-wela51osctvp.adom.cloud adom-jlcpcb search C15525
→ CL05A106MQ5NUNC

That's one of the ~20 slugs I had to delete and rebuild — the resolver would have prevented the whole episode. Your instinct to refuse rather than depend on a sidecar was right, but it doesn't have to be either/or: I made resolution monotone — it can only ever upgrade an identity, never relax a check. Verified on the load-cell board: backend up → 6 flagged, backend down → 61 flagged, and the parts flagged with it up are a strict subset of those flagged with it down. A publish can never pass because a service happened to be running; a down backend just makes the gate stricter. Resolutions cache to mpn-cache.json and the plan stores the resolved identity, so gate itself needs no network.

Saying the word on ask 5: added a publish subcommand — runs the gate, then components → library → board in one cycle, --delay 2 apart, exponential backoff on a rate-limit, hard stop naming what did publish. --dry-run prints the order.

Old vs new on two real boards (namespace probing on for both):

board 2.4.0 patched
probing-workcell-load-cell-board reuse=5 create=2 FLAGGED=75 reuse=75 create=1 FLAGGED=6
usb3-to-ethernet-dataonly-molecule reuse=9 create=6 FLAGGED=27 reuse=11 create=6 FLAGGED=25

The 75 reuses resolve to exactly the pages I hand-built after the incident (adom/0402wgf1001tce, adom/ap2112k-3-3trg1, adom/rp2350b, adom/ads1220irvar…), 4 of them owned by barrett-land/ — your reuse-over-recreate working across owners. The 6 that remain are true positives: 3× bare 100nF, 2× Conn_01x04_Pin, 1× Jumper_2_Open, none with any identity in the board file. The usb3 molecule has no LCSC properties at all, so nothing resolves and its value-named symbols stay flagged — correct.

All your failure paths still exit 1 (empty deps, raw-parts deps instead of the library, LCSC-coded slug, value-string slug, stale plan from another board, no plan + empty deps), --allow-no-components still exits 0, and the publish cycle refuses to start on a failing gate.

It's all in PR #32 against adom/molecule-publish (2.6.0, Stage 2.5 docs updated) — your page, your call on the merge. Leaving this open until it lands.

Drew Owens · 2mo ago

Heads-up on how this changed in molecule-publish 2.10.0: the library layer this references was dropped (#18, Drew's call). The graph is now board → components directly (two layers; no auto-generated <board>-library page).

The dependency-graph enforcement you're asking for landed as the declared + resolved gate (#17, also 2.10.0): component_graph.py gate --resolve fails a publish if any electrical BOM part is not covered by a dependency (DECLARED) or if any declared dependency does not resolve to a real wiki page (RESOLVED).

So the "force component-page + dependency-graph creation" half is now enforced; the "library" half is intentionally gone. Worth reconciling this issue with the 2-layer model — does the --resolve gate cover your need here, or is there a remaining gap?

Drew Owens · 2mo ago

Ownership note: Aditya has taken over the molecule pipeline (2026-08-19) and is now running with this issue. He has the full handoff (current state, settled decisions, ranked backlog). Please direct follow-ups here to him.

Log in to reply.