discover list --limit silently truncates at 9,998 rows while reporting total 10,841; manual --offset returns the rest

Overview Files Skills Discover Packages Releases Dependencies Issues5 Breadcrumbs Pull Requests Activity

Closed general

Ray · 1mo ago ·closed by Colby Knox

discover list --limit N walks offsets internally (per the --limit help text, adom-wiki-cli#34), but the walk stops at 9,998 rows while the very same response reports total: 10841. No warning, no error, exit 0.

Reproduction

adom-wiki discover list --type component --limit 30000 --json
#   rows returned: 9998
#   data.total:   10841

Asking for more does not help: --limit 20000 and --limit 30000 both return exactly 9,998.

Manual pagination reaches the missing rows perfectly well, so the data is there and only the internal walk is short:

adom-wiki discover list --type component --limit 200 --offset 10200 --json   # 200 rows
adom-wiki discover list --type component --limit 200 --offset 10600 --json   # 200 rows

Walking --offset by hand in 200-row pages returns 10,841, exactly the reported total.

Why this one bites harder than the count suggests

The 843 rows it drops are not random: --sort defaults put the most recently updated pages at one end, so a truncated read silently hides the newest pages. Those are precisely the ones any freshness check is asking about.

Concretely, the component-page factory uses this call to answer "does this part already have a page?" before spending a full build on it. A short read there means rebuilding a page that already exists and only finding out at pkg publish (already published), after the datasheet fetch, the LLM extraction, symbol, footprint, STEP, GLB, lbr, hero, build and lint have all been paid for. That failure has already cost 889 full pipeline runs from a different stale-cache bug; this call was the intended fix, and it would have quietly reintroduced 843 of the same collisions.

Suggested behaviour

Any of these closes it; the first is the least surprising:

keep walking until the accumulated count reaches total, or the server returns a short page;
if the walk stops early, return a non-zero exit or a hints[] entry saying the result is partial and naming the count and the total;
failing both, document the ceiling in --limit's help so callers know to paginate themselves.

Silent truncation is the worst of the three, because the caller cannot tell a complete answer from a partial one, and the response carries the number that proves it is partial.

Environment

adom-wiki-cli via adom-wiki on ROG-Zephyrus, adom-cli 0.5.18, 2026-09-10.

1 Reply

Colby Knox · 1mo ago

Fixed in adom-wiki 1.7.40, with one correction to the report and your option 1 as the fix.

The fix: 9,998 was the walk's runaway bound (50 requests x 200-row server pages, minus 2 dedupe collisions), written when it sat "far above the fleet"; the component fleet outgrew it and it became a silent ceiling exactly where your freshness check looks. The bound is now a true runaway guard (1000 requests / 200k rows) and the walk's real stops are your suggested behavior 1: the requested limit, a short page, or the reported total. Verified against your exact command: discover list --type component --limit 30000 --json returns 10,848 rows of total 10,848 in 55 requests.

The correction: "no warning, no error" does not hold on current releases. The WALK_CAPPED warning (naming the row count and the resume --offset) has fired on this truncation since the #34 work, and reproducing your command on 1.7.39 shows it in hints[] alongside TRUNCATED_LIST carrying showing-9998-of-10848. Your suggested behavior 2 was already in place; the ceiling itself was the bug, and the environment line says adom-cli 0.5.18 but not which adom-wiki build ran the repro, so a stale binary is the likely reader of a hint-free response. Either way behavior 1 now makes the hint unreachable in practice: WALK_CAPPED remains for the 200k guard, still naming the resume offset.

For the factory's does-this-page-exist check specifically: the full walk is now correct, but 55 requests per check is heavy at your volume; a single page stats / (uniform 404 for absent) per candidate is one request and immune to list pagination entirely. Closing; shout if any walk comes back short of total again.