Closed bug report

repo clone: slow per-file fallback when unzip is missing, and noisy default output

kaleymer · 1mo ago ·closed by Colby Knox

Summary

adom-wiki repo clone is slow and its default output is confusing on a minimal container. Two issues:

  1. Slow: hard dependency on an external unzip. The whole-repo archive path shells out to unzip; when unzip isn't installed (common on minimal containers — this one has no unzip) it falls back to fetching every file individually. For adom/drone-rf that's 514 separate fetches, which is noticeably slow.
  2. Weird output. Without --json, the command still prints a raw JSON result object, interleaved with hint [...] lines, rather than a clean human-readable summary.

Environment

  • adom-wiki CLI, registry https://wiki.adom.inc
  • Minimal container: command -v unzip → not installed

Repro

adom-wiki repo clone adom/drone-rf

(on a host without unzip)

Actual output

hint [CLONE_ARCHIVE_FALLBACK]: whole-repo archive unavailable (`unzip` unavailable or failed); falling back to per-file fetch
{
  "cloned": "adom/drone-rf",
  "dir": "drone-rf",
  "files": 514,
  "complete": true
}
hint [EXEC_BIT_NORMALIZED]: chmod +x applied to 2 file(s) ...

The clone succeeds (complete: true), but the archive fallback makes it slow, and the default output mixes JSON + hints.

Expected

  1. Don't depend on an external unzip for the fast path — extract the archive with a bundled/library zip (or tar) decoder, or offer a git-based clone, so the fast archive path works on hosts without unzip. Failing that, the per-file fallback could fetch in parallel.
  2. In non---json mode, print a concise human summary (e.g. cloned adom/drone-rf -> ./drone-rf (514 files)); reserve the JSON envelope for --json.

Note

Not blocking — the clone completed. Filing for the slowness + output clarity. The EXEC_BIT_NORMALIZED hint is already tracked (wiki#165 / adom-wiki-cli#40) and is not part of this report.

Reproduced 2026-09-14.

1 Reply

Colby Knox · 1mo ago

Fixed in adom-wiki 1.7.45.

  1. No more external unzip: the whole-repo archive is extracted in-process (zip crate, deflate only). Entry names are confined to the target directory and the unix exec bit is honored. The same extractor serves repo download --extract. Measured here on adom/drone-rf: 514 files in 3.6 s wall via the archive path, on any host, with or without unzip installed. The per-file walk remains only as the fallback for a server without the archive route (or a truncated stream); since the fast path no longer has a host dependency, I left it serial rather than adding parallel fetch against the per-request rate buckets.

  2. Terminal output is now one line, cloned adom/drone-rf -> drone-rf (514 files, via archive), with hints below it (the same for the fallback's completion line, which says PARTIAL when files failed); the JSON envelope is reserved for --json, whose shape is unchanged (cloned, dir, files, complete, via).

The EXEC_BIT_NORMALIZED hint you saw stays as tracked elsewhere.

Log in to reply.