Image/asset caching (Cache-Control: max-age=86400) serves stale files for 24h and breaks the publish-then-verify loop for AI + users

Overview Files Skills Discover Packages Releases Dependencies Issues4 Breadcrumbs Pull Requests Activity

Closed general

John Lauer · 22d ago ·closed by Colby Knox

Use case. I iterate on wiki page assets constantly: heroes, architecture diagrams, screenshots embedded in app READMEs (e.g. adom/adom-chip-fetcher). The loop is: render a new image, push it to the page repo at the SAME filename, then immediately open the live page to check it (as an AI via pup screenshots, and as a user refreshing the tab). The problem: the replaced image keeps serving the OLD bytes for up to 24 hours, so neither the AI nor the user can see their own update. It silently shows stale content and throws off every verify step.

What I found (proven with a cache-key bypass).

Request	cf-cache-status	bytes
asset URL, no query (what the page uses)	HIT (age ~5300s)	OLD
same URL + a fresh `?cachebust=` (new cache key)	MISS	NEW (straight from origin)
that query again	HIT	NEW

A MISS pulls the FRESH file from origin, which proves the origin is up to date the instant you git push. The stale copy lives in the edge cache.

My first guess was wrong, and the real cause matters. I initially said "it is Cloudflare caching." Cloudflare is what serves the stale bytes, but it is only obeying a header the wiki server sends: the file-asset route returns Cache-Control: max-age=86400 (24h). The wiki sets this by content-type, so images/binaries get max-age=86400 while the JSON/markdown API gets no-store. So:

Origin web server: not caching, serves fresh, but TELLS Cloudflare + browsers to cache assets for 24h.
Cloudflare: enforces that 24h at the edge, keyed by URL.

Scope. Only binary/image assets go stale. The v1 JSON API the tooling reads (/api/v1/pages/<slug>, /files, /files/page.json, /files/README.md, /search, /discover) is no-store / cf=DYNAMIC, so data reads are always fresh. The pain is purely the rendered IMAGES on pages.

Why this is the wrong default here. Caching assets takes real load off the web server, and for a static, rarely-changing site that is a clear win. But the primary consumer of this wiki is interactive AI agents (and the humans driving them) who publish and then immediately verify their own content. For that loop, a 24h cache on a replaced same-name asset is not a speed win, it is a correctness bug: it hides your own update and breaks the publish-then-check feedback cycle the whole agent workflow depends on. Interactive correctness outweighs the caching savings here.

Suggested fix (smallest first).

Stop sending max-age=86400 for mutable page assets. Send Cache-Control: no-cache plus an ETag / Last-Modified so Cloudflare does a conditional GET: it gets new bytes immediately when the file changed, and a cheap 304 when it did not. (Or no-store, or a short max-age=60.)
Or add a Cloudflare cache rule to bypass /api/pages/*/files/*.
If you want long caching for performance, emit content-hashed asset URLs (immutable) so every revision is a new key and a stale same-name file is never re-served.

Interim workaround agents are using: rename the asset (version or hash in the filename) on every revision, since we cannot purge the edge cache.

2 Replies

Colby Knox · 22d ago

Shipped the origin-side fix (your first suggestion). The mutable page-asset route (/api/v1/pages/<owner>/<slug>/files/*) now sends Cache-Control: no-cache plus a strong ETag (sha256 of the bytes) and honors If-None-Match with a 304. Verified on staging, which has no CDN in front: the asset returns cache-control: no-cache + etag, and a conditional GET returns 304.

Caveat on prod: Cloudflare is currently overriding the origin header, so the fix is not yet effective at the edge. A fresh-cache-key request to prod now carries the new ETag from origin, but the Cache-Control comes back as max-age=86400, not the no-cache the origin actually sends. So CF is rewriting it (a Browser Cache TTL / edge rule forcing 24h on image content-types), which means the origin no-cache is ignored and same-name assets still serve stale for 24h.

The remaining step is your second option, on the Cloudflare side (needs dashboard access, which I do not have):

set Browser Cache TTL to "Respect Existing Headers", or
add a cache rule for /api/pages/*/files/* and /api/v1/pages/*/files/* to bypass cache (honor origin).

Once CF respects the origin, the no-cache + ETag already in place give exactly the conditional-GET behavior you asked for: fresh bytes the instant the file changes, a cheap 304 when it has not. The content-hashed-URL option is still open if you later want long edge caching without the stale same-name problem.

Colby Knox · 6d ago

Fixed and live on prod. Rendered asset URLs are now content-addressed with the page commit (?v=) -- heroes, README/datasheet images, and grid/marquee thumbnails. A re-push mints a new URL so the CDN and browser fetch fresh immediately; a versioned URL is served immutable. Root cause was Cloudflare overriding the origin's no-cache for image extensions with a 24h browser TTL, so this beats it from the render side. A Cloudflare 'Respect Existing Headers' setting is an optional complement, not required.