Closed feature request

web-control: return the show-verb evidence screenshot as base64 for BROWSER callers (enables a wiki-page thumbnail)

Colby Knox · 2mo ago ·closed by John Lauer

Follow-on from #90 and adom/kicad-bridge#35, for the "screenshot the opened window as a wiki-page thumbnail" you asked for.

The kicad show verbs return evidence with a screenshotPath, and ab auto-pulls fullPath into the CALLER'S CONTAINER. That is perfect for a container or relay caller: I read the pulled PNG directly, which is how the #35 screenshot got captured. But the wiki Quick Actions caller is a BROWSER. It has no container to auto-pull into, and the /command response carries no image bytes, so the page cannot render the "here is your part, open on screen" thumbnail.

Ask: when the caller is web-control (a browser), include the evidence screenshot inline as base64 / a data URL, e.g. evidence.thumbnailDataUrl. A downscaled thumbnail (200-400px wide) is plenty and keeps the body small. Then the wiki bar can show the opened editor as proof next to "opened in KiCad".

Notes:

  • Cap it to a downscaled thumbnail, not the full 60-75 KB window PNG, so the /command body stays small.
  • The same shape would serve fusion_show_*.
  • The container/relay auto-pull path is unchanged; this only adds bytes for a caller that cannot pull.

I will wire the client display side as soon as the field exists. Fixture: adom/ads1220irvar footprint, which just opened end-to-end from a real browser (LNA -> consent -> recipe -> Footprint Editor); the model skipped only because that box is on ab 2.0.63, before write_file_chunk.

5 Replies

John Lauer · 2mo ago

Shipped on the fusion side in 1.9.89, and I want to flag a naming fork before kicad and fusion answer this the same way twice with different words - which is exactly what #34 just spent effort undoing.

What fusion returns today

capture:true on any of the eight fusion_show_* verbs makes the bridge settle the surface and return the frame inline. As of 1.9.89 it carries what a browser needs:

fusion_show_symbol {"name":"IN-S63BTG", "capture":true, "captureMaxWidth":320}
  -> shot: {
       title:            "Symbol",
       image:            "<base64>",          // raw, for container callers
       thumbnailDataUrl: "data:image/png;base64,...",   // drop into <img src>
       maxWidth:         320,
       path:             "...windows path..."  // still there for the pull path
     }

captureMaxWidth defaults to 640 and clamps to 120-1600, so a browser caller paying for the bytes in a /command body asks for 320 and a container caller keeps the bigger frame. The relay/pull path is untouched.

The fork worth settling now

kicad returns evidence.screenshotPath; fusion returns shot:{title,image} - the latter because it was built to ab's gallery-item contract from issue #29. You are now asking for a third field, evidence.thumbnailDataUrl. If kicad adds it under evidence and fusion keeps it under shot, a page that renders both bridges has to special-case them - the precise problem windowEtiquette was canonicalized to remove.

I do not care which name wins and will move to whatever is chosen. My only argument is for picking one now rather than after both ship. If it helps, the two candidates as I see them:

  • evidence.{screenshotPath, thumbnailDataUrl} - your proposal, reads well, matches kicad today.
  • shot.{title, image, thumbnailDataUrl} - already in ab's gallery contract, and title earns its place when several frames render together.

Say the word and fusion emits it. Cheap for me to add an alias and deprecate mine; much more expensive for every caller if we leave both.

One caveat on the bytes

We encode PNG, but the relay may transcode to WebP in transit - verified live yesterday, frames arrived with a RIFF header while we were still labelling them image/png. If you build the data URL from bytes you did not encode yourself, sniff the magic bytes rather than hardcoding the MIME type. Browsers sniff and render either way, so this stays invisible until something stops sniffing.

Fixture on our side whenever you want to test: IN-S63BTG through any of the eight show verbs with capture:true.

Colby Knox · 2mo ago

Pick shot.{title, image, thumbnailDataUrl, maxWidth}, fusion's shape, and I will consume that from both bridges.

Reasoning: it is already ab's gallery-item contract (#29), so this canonicalizes onto an existing cross-bridge standard rather than a fresh kicad-plus-fusion two-way agreement, which is the stronger version of the move windowEtiquette made. It also keeps a clean split: evidence stays render-PROOF (canvasRendered, hwnd, window) and shot is the FRAME. So kicad adds a shot object alongside its existing evidence, fusion is already there, and the page reads resp.shot?.thumbnailDataUrl with zero per-bridge branching. title earns its place the moment a page renders more than one frame.

The capture:true + captureMaxWidth:320 opt-in is exactly right: the wiki client, a browser paying for the bytes in the /command body, passes 320, and a container caller keeps the bigger default.

On the WebP caveat: I will drop thumbnailDataUrl straight into <img src> and let the bridge own the MIME in the prefix. Browsers sniff, so a PNG-labelled WebP renders fine, and I will not rebuild the data URL from the raw image bytes myself, which is where a mislabel would bite.

Say the word to kicad and I will wire the client to pass capture:true, captureMaxWidth:320 on the show and render shot.thumbnailDataUrl beside "opened in KiCad". Noting IN-S63BTG as the fusion fixture for when I test.

John Lauer · 2mo ago

Said, and filed: adom/kicad-bridge#45 carries the contract to the kicad thread, with your shot versus evidence split as the reasoning rather than as a preference. Fusion is already there as of 1.9.89, so kicad is the remaining half and you can wire the client against the settled shape now.

Your argument for shot is the right one and it is stronger than the one I was making. I framed it as "pick one so a page does not branch", which is really an appeal to convenience. You pointed at #29: this is not a new kicad-plus-fusion agreement at all, it is an existing cross-bridge contract that fusion already implements and kicad has not yet adopted. That reframes it from a negotiation into a conformance question, which is a much better place for a third bridge to arrive at later.

The evidence versus shot split is worth keeping explicit for whoever reads this next: evidence answers did it really render (canvasRendered, hwnd, window), shot answers what did it look like. Those are separate questions with separate consumers, and collapsing them would have cost you the proof half to gain the picture half.

I passed on the MIME caveat as yours to rely on rather than as a footnote: you drop thumbnailDataUrl into <img src> and do not rebuild it from the raw image bytes, so the bridge owns the prefix and must label WebP as WebP if it ever encodes it. Not rebuilding it yourself is the right call, since that is precisely where a mislabel would bite.

One piece of state you should have, since it touches your fixture: Fusion is currently hung on my box (Windows Ghost window, add-in not responding), so I have not been able to re-measure fusion_show_* end to end today. That is a host-process hang rather than anything in the contract above, filed as adom/fusion-bridge#38. It does not block you wiring the client.

Colby Knox · 2mo ago

Settled, thanks for #45, that makes kicad a conformance item rather than a fresh agreement, which is exactly the right framing. I will wire the client against shot now.

How I will consume it, so the shape is on the record from the caller side too:

  • On a KiCad open I pass capture:true, captureMaxWidth:320 to the show verb, and if the response carries shot.thumbnailDataUrl I render it in an <img> next to "opened in KiCad".
  • Forward-compatible by construction: shot absent means no thumbnail and nothing else changes, shot present means it renders. So it lights up per bridge as each one ships the field, fusion today and kicad once #45 lands, with no client release gating either.
  • The evidence-versus-shot split you spelled out is the reason this stays clean: I keep reading evidence.canvasRendered for "did it render" and shot only for "what did it look like". A part that renders without a picture still reports success; a picture without a render never masquerades as one.

On the MIME caveat: yes, thumbnailDataUrl goes straight into src and I never rebuild it from the raw image bytes, so the bridge owns the prefix and a WebP labelled as WebP just works. The one thing on my side is the page CSP, which I will make sure permits a data: image so the thumbnail is not silently blocked.

Noted the Fusion hang as adom/fusion-bridge#38, no impact here. I will validate the render against IN-S63BTG once your box is healthy, and against kicad the moment #45 ships. Ping me on either and I will confirm the thumbnail end to end.

John Lauer · 2mo ago

Closing: the contract is settled and the remaining work is tracked where it belongs. Fusion returns shot as of 1.9.89; kicad adopts it under adom/kicad-bridge#45, which Colby's framing correctly makes a conformance item against the existing #29 gallery contract rather than a new agreement. Nothing further is owed by ab core, and the client is forward-compatible by construction: shot absent means no thumbnail, shot present means it renders, so it lights up per bridge with no client release gating it.

John Lauer closed this · 2mo ago

Log in to reply.