KiCad - the KiCad Bridge

Closes wiki issue #37 (Colby Knox): kicad_install_symbol reported 'No symbols found in the provided file content' for a valid KiCad 10 symbol file, because the extraction regex required at least one leading whitespace character before (symbol, and in that file the top-level symbol sits at column 0 while its sub-units are tab-indented. So only the sub-units matched, and they were then correctly dropped by the _N_N suffix filter, leaving nothing. His diagnosis and his fix, verbatim. Worth recording that this regex has now failed in BOTH directions - an earlier version keyed on exactly two spaces and missed every tab-indented library - which is the lesson: indentation is not semantic in s-expressions and must not be load-bearing. Both regexes now accept any leading whitespace including none, and the sub-unit suffix filter does the real work. Verified against both shapes: a column-0 top-level symbol with tab-indented sub-units, and the classic fully indented layout.

Follows Colby's 0.9.186 retest on wiki issue #34, where kicad_bridge_status showed two plugin instances with exeName python, both alive:false, both on port 8879. Root cause: usercustomize.py had no process guard, so ANY Python with KiCad's user-site on its path started the reverse-bridge RPC server - including the short-lived python.exe probes the bridge itself spawns, which pass the wx check because KiCad's bundled Python ships wx. Each one registered a discovery file, bound a port, and exited moments later, leaving a graveyard that made every plugin tier report plugin_not_running while looking at dead entries. The loader now starts the server only inside real KiCad GUI processes (kicad, eeschema, pcbnew, gerbview, pl_editor, bitmap2component); the .pth loader already had this guard, usercustomize did not. Separately, kicad_bridge_status now prunes discovery files whose PID no longer exists by default, so the status reflects live instances instead of history.

Closes the second half of wiki issue #34 (Colby Knox): from a genuinely cold start, a user with no board open could not open the Footprint Editor at all. The bridge already opens a board in the background to make KiCad scan its plugin directory (0.9.173), but that assumed a board existed somewhere on the machine. On a fresh box there is none, so the path gave up. It now generates a minimal throwaway board in TEMP purely to trigger the scan, which costs a few kilobytes and makes the cold footprint path work on a machine that has never had a KiCad project.

Two fixes. (1) John, watching a live launch: the KiCad project manager came to the foreground and the bridge never even attempted to put it behind. Cause: the idle rule added in 0.9.172 says that with no verb running, a KiCad window reaching the foreground is the user's doing and must be left alone - correct for a window the user opened, wrong for one the bridge spawned seconds earlier that took its time appearing. The difference is now bounded by time since OUR spawn: for 60 seconds after the bridge starts a KiCad process, a window of ours that appears in front gets the single background action; after that, hands off. Still exactly one action per window. (2) Wiki issue #35 (Colby Knox): kicad_show_symbol cost about 56 seconds on EVERY call, even when the Symbol Editor was already open on that exact symbol, because the verb always ran the cold path and KiCad re-indexes every symbol library when that editor opens. The show verbs now check first: if an editor is already displaying the requested part, they return it immediately instead of relaunching and rescanning. Same for the Footprint Editor.

Removes the machine-wide low-level input hooks that 0.9.182 installed. They were added to detect a taskbar click or alt-tab, and the effect was that every mouse event and keystroke on the user's computer passed through this bridge's callback - which degraded his cursor badly enough to cost him four reboots. They are deleted, not repaired, and the prohibition is written into the source: this bridge never installs a system-wide input hook of any kind, and any future idea that seems to need one gets proposed to the user BEFORE it is written, not shipped and observed. Nothing depended on them anyway: the bridge no longer blocks activation at all (WS_EX_NOACTIVATE is gone as of 0.9.181), so a click or alt-tab simply works the way it does for any other window, and there is nothing to detect. Backgrounding remains one SetWindowPos to the bottom of the z-order, once per window, only if that window took the foreground.

Urgent fix for a regression I introduced in 0.9.182 and John caught within minutes: his cursor was breaking. The user-intent detection added in that release used a WH_MOUSE_LL hook, which is correct in principle, but the callback did real work inline - WindowFromPoint, a cross-process class-name read, a process-image query, and then allow_activation_all() which calls SetWindowLongPtr and therefore SENDS WM_STYLECHANGING cross-process, blocking for as long as the target takes to answer. A low-level hook callback runs inside the system's input path, so every one of those delays the mouse for the entire machine, and Windows silently unhooks a callback that overruns LowLevelHooksTimeout. The callback now does exactly one thing - append a timestamp to a small queue - and returns. A worker thread off the input path does the window classification and the activation release. The lesson, written into the code so it is not repeated: a hook may record a fact and nothing else.

Two things. (1) John's requirement: if the bridge ever blocks activation it MUST know when the human asks for a window, or the whole experience is broken. The old detector polled GetAsyncKeyState every 120ms, so a taskbar click that began and ended between two polls was invisible - which is exactly how a deliberate click got treated as a self-raise. Detection is now low-level hooks (WH_MOUSE_LL and WH_KEYBOARD_LL) armed at bridge boot: Windows delivers every button and keystroke to them synchronously, so there is no sampling gap and nothing to miss. A press anywhere on the shell (taskbar, Start, tray, alt-tab switcher), a press on a KiCad window, or Alt+Tab immediately releases every activation block, and while the user is driving the bridge performs no background action at all. (2) Wiki issue #34 (Colby Knox): the plugin auto-install cached a false no_kicad_installed for the entire bridge boot on a machine with KiCad 10 installed - the explicit kicad_install_plugin found it moments later - so the reverse bridge never came up on its own and every plugin-dependent verb silently degraded. A failed auto-install is no longer cached, and when the boot hands it an empty version list it re-detects from scratch rather than trusting a list that may simply have been populated too late.

The backgrounding algorithm, rewritten to John's rules and reduced to a single call. For a KiCad window that actually takes the foreground, the bridge now performs exactly one action for the entire life of that window: SetWindowPos to HWND_BOTTOM with SWP_NOACTIVATE, SWP_NOMOVE and SWP_NOSIZE. Nothing else, ever. Removed: minimize-bounce, because a minimized window cannot be screenshotted and every screenshot-based verb depends on that; off-screen parking, which moved windows to -32000 and back and never delivered what it promised; WS_EX_NOACTIVATE, because it blocks the USER's own click and a user may want the window in front the very next second - their click must work immediately with no hold-off; and all repetition, including the sweep branch that re-parked quiet windows every 50ms and the guardian that re-demoted on every tick. A window that opens quietly behind the user's work is now left completely untouched. Boot also clears any WS_EX_NOACTIVATE left behind by versions up to 0.9.180, which could persist indefinitely and silently break taskbar clicks.

John, twice, and Colby once: the KiCad window blips - foreground, background, foreground, background. The previous fixes capped how MANY times the bridge did this (3 rounds in 0.9.175 and 0.9.179) which was the wrong shape of fix, because the mechanism itself was visible. Backgrounding used a minimize-bounce (SW_MINIMIZE, then SW_SHOWNOACTIVATE, then z-bottom) and parking moved windows to -32000 and back: a minimize is one visible frame and the restore is another, so every call was a blip by construction and no cap could reach zero. Backgrounding now changes only things the compositor does not animate: WS_EX_NOACTIVATE so Windows refuses the window's own activation attempts, and one SetWindowPos to the bottom of the z-order with SWP_NOACTIVATE|NOMOVE|NOSIZE. No minimize, no restore, no move, no resize, ever. If a window still holds the foreground after that, it is LEFT there - briefly in front beats strobing. Off-screen parking is retired to a shim, unparking no longer touches geometry, and WS_EX_NOACTIVATE is cleared the moment the bridge goes idle so a taskbar click always works.

Fixes wiki issue #33 (Colby Knox): the focus guardian demoted a self-raising KiCad window on every 100ms tick with no debounce, so a freshly-launched wx frame that Raise()es itself while loading produced a ~10x/sec front-and-back flicker and burned CPU until KiCad was closed. His diagnosis was exactly right, and his suggested fix is what shipped: after three rounds the guardian stops re-demoting a given window, holds it once at the bottom of the z-order with activation suppressed, and leaves it there - one demote that sticks is enough. Note this is a SECOND loop with the same failure mode as the one capped in 0.9.175: that release capped the parking sweep, this one caps the guardian, and both now record every round in kicad_state.focusDebug (parkFights, parkGaveUp, guardianDemotes, guardianDemoteLimit, parkFightLog) so the next occurrence is measurable rather than anecdotal.

The progress block is now attached to every instrumented verb rather than only to runs that exceeded three seconds. A warm run finishing in half a second is precisely the fact issue #32 wants surfaced - cold and warm differ by 10-50x and the user used to get one pessimistic sentence covering both - and reporting it also feeds the warm history, which is what lets the next estimate be measured instead of guessed.

Hotfix for 0.9.176: the progress phase map was defined inside the dispatcher AFTER its first use, so Python raised 'cannot access local variable _PHASES where it is not associated with a value' on every verb - the bridge answered nothing but that error. The map is now module-level, defined once at import. My mistake and my apologies; the measured-progress feature itself is unchanged.

Implements the measured-progress contract from issue #32. Every verb with a p50 over ~3s now returns a progress block (namespaced phase, stepLabel, monotonic percent, elapsedSec, estimatedSec, etaSec, confidence, startKind) and its terminal frame carries the actuals plus a measured{} record. Estimates come from a rolling per-machine history persisted in the bridge's own state dir (last 5 samples per phase, split cold vs warm, p50 and p90) - never a hardcoded literal, which is the thing that made 'can take a couple of minutes' useless. Confidence is honest about sample count: 0 samples reports unknown and seeds from a shipped default, 1-2 reports typical, 3 or more reports measured from this machine's own p50. Both amendments I proposed on the issue are implemented: phase ids are namespaced (kicad.cold_start) so a UI can group histories across bridges, and blockedBy names a human-controlled wait (a modal, a UAC prompt) so a bar can say 'waiting for you' instead of animating through something that cannot progress. Cold versus warm is detected per call rather than assumed. kicad_describe now exposes the whole timing history plus the contract version, so a UI can say 'typically Ns cold, Ms warm on THIS machine' before the user clicks. Instrumentation lives in the dispatcher rather than in each handler, so a verb added tomorrow is covered by default.

John: 'why do you blink them like 20 times in a row? if you cannot really control the natural behavior of kicad, you should not fight it as hard as you are - you literally look like you are broken.' He is right, and this concedes the argument. KiCad reasserts its own geometry repeatedly while it loads, and a 50ms parking sweep that re-parks on every reassertion turns into 20-30 visible blinks. The sweep now counts fights per window and after 3 rounds STOPS: it concedes the position and keeps only what actually matters - the window stays behind, unfocused, with its taskbar button. A window that is quiet and slightly visible beats one that is invisible and seizuring. Every round is recorded (kicad_state.focusDebug now carries parkFights, parkGaveUp, parkFightLog and parkFightLimit) plus a stderr line naming the window and the round it conceded on, so this is measurable rather than anecdotal.

The last piece of John's footprint report. With the plugin bound and the correct menu id, the command posts fine and KiCad does open the Footprint Editor - but that frame loads the footprint libraries as it opens and routinely needs 10-30 seconds on a cold pcbnew, while the tier only waited 6. So it declared the background path failed, fell through to paths that could not work, and reported failure while the editor was quietly opening behind it. The wait is now the same 30s ceiling the other tiers use. Verified by firing the same command by hand: the Footprint Editor appears, just later than the old wait allowed.

Applies the Adom automation doctrine to the part surfaces. 0.9.172 made show_footprint fail honestly when its window never appeared, which was correct but useless: the usual reason is that the reverse-bridge plugin is not bound, and on KiCad 10.0.5 the plugin only loads when KiCad scans its plugin directory - which happens when a board editor comes up, not when the project manager alone is running. So the surface now opens a board in the background itself, waits for the bind, and then does what was asked. The user never sees an error they would have fixed by pressing a button the bridge could have pressed.

Two things John hit live. (1) 'When I click the taskbar icon to bring KiCad to the foreground, you don't let me.' Bouncing exists only to stop a window flashing into the user's face WHILE the bridge is working, but the sentinel applied it whenever it could not positively attribute the raise to a click - and a fast taskbar click lands entirely between two 120ms polls, so it was invisible. New idle rule: with no verb in flight and no guard armed, a KiCad window reaching the foreground is the USER, and is left alone unconditionally. Click detection also now uses the pressed-since-last-query bit rather than only down-right-now, and a bounce suppression lasts 3s instead of 10s. (2) 'I asked for a footprint and the evidence was the project window.' A show_* verb could report success while its window never appeared, and the caller then screenshotted whatever was on screen - manufacturing wrong evidence rather than reporting a failure. show_footprint, show_symbol and show_3d_chip now verify their own window is present and fail with surface_window_missing when it is not, and the dashboard refuses to render a different window as proof of the requested surface.

Closes the footprint half of issue #31. The bridge deliberately never trusts hardcoded menu ids across KiCad versions - it asks the live process via the plugin's get_menu_ids RPC - but the parser only understood one response shape (frames[].menus[]). On 10.0.5 pcbnew that RPC returns a FLAT list of {id,label,topMenu}, so the lookup silently found nothing, fell back to the hardcoded 20572, and posted a command that reported success while opening no window. The resolver now walks whatever shape comes back and collects every {id,label} pair; on 10.0.5 it resolves pcbnew's Footprint Editor to 20574 and eeschema's Symbol Editor to 20391.

The loader now starts the payload on a DEFERRED thread. Diagnosis came straight from the log trail added in 0.9.168: 'loading in pcbnew.exe ... import adom_bridge ... ModuleNotFoundError: No module named urllib', raised deep inside http.server importing email.utils out of KiCad's own Lib. KiCad scans its scripting/plugins directory before the embedded interpreter's sys.path is complete, so importing the payload inline could never work in pcbnew; it only worked in kicad.exe because that scan happens later. The plugin now arms a background thread, retries the import every 5 seconds for a minute, starts the bridge as soon as the payload is importable, and logs the outcome. Dashboard in the same release: every LED gains a live busy state (pulsing, with a running label such as 'launching KiCad in the background' or 'waiting for the plugin to bind'), pushed over the live channel the instant work starts, so background activity never reads as a dead indicator.

Third and last gate fix in this chain: the installer's already-current check compared the payload by hash but the GENERATED loaders only by existence. So when 0.9.168 rewrote the plugin-dir loader, every box that already had 0.9.166's copy reported already-current and kept running the broken one. The gate now content-compares every generated artifact (the plugin-dir loader, the boot module and the .pth) against what this release would write.