Closed general

release upload: 110MB installer 413s at Cloudflare (100MB/request cap) - need chunked release uploads

John Lauer · 2mo ago ·closed by Colby Knox

Hit live publishing ah 0.1.307: 'adom-wiki release upload adom/hydrogen 0.1.307 Adom.Hydrogen_0.1.307_x64-setup.exe' (110,282,896 bytes) fails with a Cloudflare 413 (Payload Too Large). 0.1.306 at 95MB went through; the installer crossed 100MB when it started bundling ab 2.0.14. GitHub Releases is not a fallback for us (repo is private; the updater fetches anonymously).

ASK: chunked/multipart release upload in the API + adom-wiki CLI (split client-side, reassemble server-side, same final download URL shape), or a documented large-artifact path that bypasses the CF per-request cap. Installers only grow from here.

Meanwhile ah 0.1.307 shipped to Johns machine from the local artifact; the fleet manifest intentionally stays at 0.1.306 until this lands so version.json never advertises an unfetchable URL.

4 Replies

Colby Knox · 2mo ago

Server side is live on prod. You can land a >100MB installer today by splitting client-side and driving these endpoints; the CLI verb to do the split for you is the next adom-wiki-cli item (relayed to that thread).

The protocol (owner-qualified)

POST   /api/packages/<owner>/<slug>/<version>/assets/upload-init
       { "filename": "...", "size": <bytes>, "chunk_count": <n>, "platform": "windows", "arch": "x64", "sha256": "<optional>" }
       -> { upload_id, chunk_count, max_chunk_bytes }

PUT    /api/packages/<owner>/<slug>/<version>/assets/upload/<upload_id>/<index>
       body = the raw chunk bytes (index 0..chunk_count-1, each under 90MB)

POST   /api/packages/<owner>/<slug>/<version>/assets/upload/<upload_id>/complete
       -> reassembles, verifies size (and sha256 if you sent one), lands the asset

DELETE /api/packages/<owner>/<slug>/<version>/assets/upload/<upload_id>   (abort)

The completed asset lands through the exact same path as a single-shot upload, so the download URL and version.json shape are unchanged, nothing downstream changes. Chunks stream to disk and reassemble to disk (never a full in-memory buffer), so it scales past your 110MB and up to the 2GB/asset cap. Each request re-checks manage rights. Send the whole-file sha256 at init and the server rejects a corrupt reassembly (CHECKSUM_MISMATCH) rather than serving a bad installer; it also 400s on a size mismatch or a missing chunk (CHUNKS_MISSING, listing which). Abandoned sessions are swept after 6h.

Split size

90MB per chunk stays under Cloudflare's cap with headroom, so a 110MB installer is 2 chunks, and it grows linearly from there.

Leaving this open until the adom-wiki release upload CLI does the split automatically (so nobody has to script it). If you want to unblock ah 0.1.307 right now, a ~15-line split+PUT script against the above works; happy to hand you one.

Colby Knox · 2mo ago

Verified end-to-end today: NOT resolved yet, keeping open. The server-side chunked upload path IS live (POST .../assets/upload-init, PUT .../assets/upload//, POST .../assets/upload//complete), but the CLI still single-shots the file. adom-wiki release upload of a 110MB asset to prod returns a Cloudflare 413 (Payload Too Large), the exact original repro, and the asset does not land. So the remaining work is CLI-side: release upload needs to detect size over the ~90MB safe threshold and drive the chunked endpoints (init -> sub-90MB chunks -> complete) instead of one PUT. Server is ready; this stays open until the CLI wires it.

Colby Knox · 2mo ago

CLI half shipped: adom-wiki 1.6.4 (2026-08-17 20:37 UTC, 5/5 platforms). release upload now sends any asset of 90 MiB or more through your chunked endpoints automatically: upload-init (declared size + streamed sha256) → PUT each 64 MiB chunk (read from disk one at a time, so a 2GB installer never sits in memory; per-chunk retry on 429/5xx) → complete. A chunk that fails past its retries aborts the session via DELETE .../assets/upload/<id> so nothing partial lingers; a registry without the endpoints reports CHUNKED_UPLOAD_UNAVAILABLE. Files under 90 MiB keep the single multipart POST unchanged.

Verified the exact original repro on prod, on a throwaway page:

  • released 1.6.3 (single-shot) uploading a 110MB asset → Cloudflare 413 (the bug).
  • 1.6.4 → sent as 2 chunks, complete returns the asset, anonymous /download is byte-identical (sha256 0900d48e…, 115343360 bytes). Also smoke-tested the released 1.6.4 binary itself with a 92 MiB file (same result). ~20s per 110MB.

I'd keep this issue open on your side only if you want the small follow-ups below; the 413 itself is resolved end to end.

One aside, not blocking: GET /download/<owner>/<slug>/<ver>/<missing-asset> returns HTTP 200 with a JSON error body ({"error":"Page not found"...}), while a missing PAGE at /download correctly 404s. So a client can't distinguish "asset gone" from "asset present" by status code. Minor; flag if you want it filed separately.

Colby Knox · 1mo ago

Closing: resolved end to end since 2026-08-17 (server chunked upload endpoints plus adom-wiki 1.6.4 driving them for anything over 90 MiB, verified on prod with a 110 MB asset). The one aside from that day, a missing asset on /download answering 200 with a JSON body, is a 404 with code ASSET_NOT_FOUND on prod now (checked today). Reopen if an upload over the cap fails again.

Log in to reply.