files listing: recursive mode (?recursive=1) to serve the whole tree in one response

by adom

How the Adom Wiki (wiki.adom.inc) works and how to drive it with the adom-wiki CLI: the two storage layers (git page repo vs package registry), hosting source by pushing files (not a tarball), making packages, cutting per-platform releases, and the three-tier skill layout (user skills ship, dev-skills/ and publish-skills/ stay source-only). adom-wiki is the current tool; adompkg is deprecated. Read this first before publishing or installing.

Overview Files Skills Discover Packages Releases Dependencies Issues8 Breadcrumbs Pull Requests Activity

Closed general

Colby Knox · 1mo ago ·closed by Colby Knox

Context: the CLI's repo pull is now incremental (1.7.24): it diffs the listing's per-file git blob hash against local bytes and fetches only changed files. The remaining cost is enumerating the tree: GET /api/pages/:owner/:slug/files has no recursive mode, so the CLI walks one request per DIRECTORY (pooled 5-wide). adom/altium-bridge has 36 directories, so a no-change pull still spends its whole 2.3s on 36 listing calls that could be one.

Ask: accept ?recursive=1 (ignored today, verified: same top-level rows come back for recursive=1/true and deep=1) and return the full tree in one response, rows carrying the same fields the flat listing already serves (name or path, type, hash, executable/mode). Flat remains the default; no existing client changes behavior.

CLI side is ready to adopt: the walk collapses to a single GET when the flag is honored, and the version-skew story is the usual one (older server ignores the param, CLI sees subdirectory trees absent from the response and falls back to the walk).

Related but separately owned: the intermittent whole-process stalls being root-caused now multiply per-request patterns; fewer requests helps every client either way.

3 Replies

Colby Knox · 1mo ago

Shipped on staging (git-wiki 9c71732; rides the next prod tag).

GET /api/pages/<owner>/<slug>/files?recursive=1 (also deep=1) returns the whole subtree in one response. Entries carry their repo-relative path and lastCommit like the flat listing. It is files only (git ls-tree -r), so derive directories from the paths. The response carries "recursive": true so a client can tell the mode was honored. Memoized per commit like the flat listing. Verified on staging against a 406-file repo.

Colby Knox · 1mo ago

Adopted in adom-wiki 1.7.27. The incremental pull probes ?recursive=1 first; a response carrying recursive: true replaces the whole walk, and an old server's flat response SEEDS the pooled per-directory walk so the probe request is never wasted. The PULL_INCREMENTAL hint now reports the count: staging shows "0 changed file(s) fetched, 4 unchanged (blob-hash diff over 1 listing request(s): recursive listing)" against a repo with a subdirectory. Closing the loop on what this issue asked for.

Colby Knox · 1mo ago

On prod as of git-wiki aba6ac4 (18:25 today). GET .../files?recursive=1 serves the whole subtree in one response. Closing; reopen if it does not hold for you.