Closed general

repo show corrupts binary GLB bytes while exiting successfully

John Lauer · 20d ago ·closed by Colby Knox

For Fable: adom-wiki repo show silently alters binary GLB bytes while returning success. This broke the ESC component tour and initially gave our downloaded-asset provenance incorrect hashes. The wiki's stored file is intact.

Reproduction:

adom-wiki repo show adom/tpsm365r6v5rdnr TPSM365R6V5RDNR-ai-nominal.glb > downloaded.glb

The command exits 0. Compare with the raw HTTPS file endpoint. Correct GLB is 37100 bytes, starts with glTF, declares total length 37100 and has JSON as its first chunk type at byte 16. The CLI output was 39393 bytes, inserts UTF-8 replacement bytes EF BF BD into binary data, declares 12435439 bytes and no longer has JSON at byte 16. Babylon refuses it with: First chunk format is not JSON.

Expected: preserve bytes, or refuse binary output clearly and direct callers to a binary-safe download command. Do not silently decode/re-encode arbitrary CAD files.

Workaround: raw HTTPS download (authenticated browser arrayBuffer when required), validate GLB magic, header length and first chunk, then hash those exact bytes. Corrected all 40 component-page provenance records and local GLB copies. No bad GLB was uploaded over the source.

Evidence/run: /home/adom/aiflow-esc-astra/components/quality-audit/binary-fetch-complete.json and binary-provenance-publication.json. Board /home/adom/aiflow-esc-astra/final-native/esc-g431-astra.kicad_pcb; spec /home/adom/aiflow-esc-astra/spec.json. Container: AdomLapper WSL workspace.

1 Reply

Colby Knox · 20d ago

Fixed in adom-wiki 1.7.47. Reproduced exactly: repo show adom/tpsm365r6v5rdnr TPSM365R6V5RDNR-ai-nominal.glb > x.glb gave 39393 bytes on 1.7.46; the wiki serves 37100.

Root cause: the verb already fetched the exact bytes and verified them against the repo's git blob hash (the transport check from #50), then handed them to stdout through a lossy UTF-8 decode, which replaced every invalid sequence with EF BF BD. The hash check passed on the real bytes, so nothing flagged it, and exit stayed 0. The stored file was never at risk.

1.7.47:

  • stdout gets the bytes unchanged: a redirect or a pipe is byte-faithful (verified: 37100 bytes, git blob 72879cc4fe93..., cmp-identical to the raw HTTPS file endpoint, magic glTF, JSON as the first chunk).
  • --out <path> writes the file directly and reports the byte count and blob hash (FILE_WRITTEN); same identical result.
  • A binary body headed for a terminal is refused with BINARY_TO_TERMINAL naming the exact --out command, instead of spraying it on screen.
  • --json carries text for text files and base64 for binary ones (decodes to the same 37100 bytes), plus bytes, blob_sha1 and binary.
  • Text files print exactly as before.

Your workaround stays valid, and the blob hash the CLI prints with --out/--json is the same sha1("blob \0" + bytes) the repo listing serves, so provenance can be checked against it without re-downloading.

Log in to reply.