Fusion - the Fusion 360 Bridge

@@ -0,0 +1,80 @@+---+name: fusion-verb-tests+description: The standard end-to-end verb test suite for the fusion bridge - run the daily tier every day against a real box, the weekly tier before releases, triage failures, and keep the plan in lockstep with the verb catalog so no verb ships untested+---++# fusion-verb-tests - the standard daily verb test++Every described verb of this bridge, tested end-to-end the way a user's AI actually calls+it: through `adom-bridge --target <box>` over the real relay, against real Fusion, real+APS, and the canonical BQ25792 board. Built for daily runs ahead of the public release.++## Run it++```bash+cd ~/project/adom-desktop-fusion-bridge+python3 tests/run_verb_tests.py --target AdomLapper --tier daily    # every day, ~5-10 min+python3 tests/run_verb_tests.py --target AdomLapper --tier weekly   # before releases+python3 tests/run_verb_tests.py --target AdomLapper --only fusion_aps_search   # one verb+```++Exit code 0 = all green. Reports land in `tests/reports/verb-test-<stamp>.json`.+Use the ai-thread the runner sets (`fusion-verb-tests`) when investigating by hand, so+the user's Activity Log shows the test run as one coherent actor.++## The tiers, and why a verb is where it is++| Tier | What qualifies | Examples |+|---|---|---|+| **daily** | Read-only, or state-changing but reversible with the runner restoring what it touched | readiness, all APS reads, open BQ25792 + the three views, inspect_bodies, assembly_bom, run_modeling_script returning a constant, uninstall dryRun |+| **negative** | Verbs that MUST refuse - a refusal is the PASS | the two deprecated crash verbs; the `expectDocument` guard fed a wrong name on both a mutating and a READ verb |+| **weekly** | Heavy or write-bearing flows: exports to the scratch dir, library opens, electron view commands, cloud reads that walk hubs | export_step/gerbers/bom, open_lbr, generate_package, list_cloud_projects |+| **manual** | Anything that needs a human present or touches their session: sign-in flows, notify toasts, real uninstall, prefs UI driving, doc-closing verbs that could eat their tabs | fusion_signin, fusion_kill, fusion_close_all_documents, fusion_demo |++**Catalog drift is a failing test.** The runner diffs `describe._T` against its own plan;+a verb described but untriaged fails the run with `untriaged`. Adding a verb to the+bridge means adding a line to the plan in the same commit.++## What the runner cannot judge - the AI's part of the ritual++The runner asserts response shapes. Four things still need eyes each daily run:++1. **The board actually rendered.** After the open + view stages, take a background+   `desktop_screenshot_window` of Fusion and LOOK: BQ25792 in the 3D view, dark canvas,+   not a dialog, not an empty tab.+2. **Timing drift.** Each result carries `sec`. A verb that passed but took 5x its+   usual time is tomorrow's failure; compare against the previous report.+3. **The box was left as found.** Fusion running-state restored, the test board tab+   closed, no stray dialogs. `fusion_get_app_state` at the end should match the start.+4. **Failures get filed, not just logged.** A red verb gets an issue on+   adom/fusion-bridge with the runner's JSON snippet, the bridge + add-in versions+   (`fusion_readiness`), and repro args, per the issue-reporting skill.++## Fixtures++- **Cloud**: `Molecules/XRP/Power/BQ25792.fprj` in project `Main` - the canonical board.+  Nothing in the daily tier writes to the cloud.+- **Local samples**: `examples/ADS8588SIPM.step` + `examples/ADS8588SIPM.fusion.lbr`+  (weekly library flows; staged to the box via `send_files` at run time).+- **Scratch on the box**: `%TEMP%/fusion-verb-tests` - all export outputs; safe to wipe.+- **Cloud writes** (weekly library builds only) go to the Adom AI Workspace folder and+  are cleaned with `fusion_cleanup_cloud_files`; never the project root, per the+  fusion-cloud-hygiene skill.++## Reading a failure++- `wrong_document` where a PASS was expected: the guard fired - which document was+  actually active? The runner opens BQ25792 first; a failure here usually means the+  open itself failed upstream.+- `bridge_restarting`: ab respawned the bridge mid-run (normal right after an install);+  re-run the verb before calling it a failure.+- `fusion_addin_not_responding` + a dialogs array: a modal is up; screenshot it - that+  is the real finding.+- Every refusal from the negative tier passing means the safety rails hold; a+  deprecated verb suddenly SUCCEEDING is a sev-1 finding, not a pass.++## Cadence++Daily: `--tier daily` (a cron on the container works; see the cron-watchdogs skill).+Before any release: `--tier weekly` plus a by-hand pass of the manual tier's+release-rehearsal subset (make_3d_package, build_library_3d) on a box you can watch.

@@ -0,0 +1,233 @@+#!/usr/bin/env python3+"""End-to-end verb test runner for the fusion bridge (dev-skills/fusion-verb-tests).++Runs IN THE CONTAINER, through the real relay, exactly the way a user's AI calls verbs:+    adom-bridge --target <box> --ai-thread "fusion-verb-tests" <verb> '<json>'++Tiers:+  daily    read-only + reversible state (safe every day, leaves the box as found)+  weekly   heavy flows (library builds, cloud writes into the test workspace, exports)+  manual   human-present only (sign-in, uninstall-for-real, notify, install, demo)+  negative always run: verbs that MUST refuse (deprecated verbs, the document guard)++Usage:+  python3 tests/run_verb_tests.py --target AdomLapper [--tier daily] [--only verb1,verb2]++Every described verb MUST appear in PLAN. If describe.py grows a verb the plan does not+know, the run FAILS with 'untriaged' - catalog drift becomes a failing test, not a gap.+"""+import argparse, datetime, json, os, pathlib, subprocess, sys, time++REPO = pathlib.Path(__file__).resolve().parent.parent+sys.path.insert(0, str(REPO))+import describe  # noqa: E402++THREAD = "fusion-verb-tests"+SCRATCH = "C:/Users/john/AppData/Local/Temp/fusion-verb-tests"++def call(target, verb, args=None, timeout=90):+    cmd = ["adom-bridge", "--target", target, "--ai-thread", THREAD, verb, json.dumps(args or {})]+    t0 = time.time()+    try:+        p = subprocess.run(cmd, capture_output=True, text=True, timeout=timeout)+        raw = p.stdout.strip() or p.stderr.strip()+        d = json.loads(raw[raw.index("{"):]) if "{" in raw else {"error": raw[:200]}+    except subprocess.TimeoutExpired:+        return {"_testTimeout": True}, time.time() - t0+    except Exception as e:+        return {"error": f"runner: {e}"}, time.time() - t0+    # unwrap the nested output envelope when present+    out = d.get("output")+    if isinstance(out, str) and out.strip().startswith("{"):+        try:+            inner = json.loads(out)+            merged = dict(d); merged.update(inner if not isinstance(inner.get("data"), dict) else inner["data"])+            if isinstance(inner.get("data"), dict): merged.update(inner["data"])+            d = merged+        except Exception:+            pass+    if isinstance(d.get("data"), dict):+        merged = dict(d); merged.update(d["data"]); d = merged+    return d, time.time() - t0++def expect(d, spec):+    """spec: {'ok':True} needs success/status ok; {'fields':[...]} need present;+    {'errorCode':'x'} exact; {'pred':name} named predicate."""+    errs = []+    if spec.get("ok"):+        if not (d.get("success") is True or d.get("status") == "ok" or d.get("ok") is True):+            errs.append("expected success, got error=%s code=%s" % (str(d.get("error"))[:80], d.get("errorCode")))+    for f in spec.get("fields", []):+        if f not in d: errs.append("missing field %r" % f)+    if "errorCode" in spec and d.get("errorCode") != spec["errorCode"]:+        errs.append("expected errorCode=%s got %s (error=%s)" % (spec["errorCode"], d.get("errorCode"), str(d.get("error"))[:60]))+    if spec.get("pred") == "count_gt_0" and not (d.get("count", 0) > 0 or len(d.get("results") or []) > 0):+        errs.append("expected >0 results")+    if spec.get("pred") == "verbs_match_plan":+        got = {v["name"] if isinstance(v, dict) else v for v in (d.get("verbs") or [])}+        planned = set(PLAN.keys())+        if got and (got - planned): errs.append("verbs described but UNTRIAGED in test plan: %s" % sorted(got - planned)[:6])+    if d.get("_testTimeout"): errs.append("timed out")+    return errs++# tier, args, expectation, [cleanup verb, cleanup args]+P = {}+def t(verb, tier, args=None, exp=None, note=""):+    P[verb] = {"tier": tier, "args": args or {}, "exp": exp or {"ok": True}, "note": note}++# ── stage 0: bridge + catalog ────────────────────────────────────────────────+t("fusion_readiness", "daily", {}, {"ok": True, "fields": ["bridgeVersion"]})+t("fusion_describe", "daily", {}, {"ok": True, "pred": "verbs_match_plan"})+t("fusion_get_app_state", "daily", {}, {})  # shape only; content depends on Fusion state+# must-refuse (deprecated, hard-blocked):+t("fusion_search_cloud_files", "negative", {"query": "x"}, {"errorCode": None, "pred": None}, "expect refusal text")+t("fusion_walk_cloud_tree", "negative", {}, {}, "expect refusal text")++# ── stage 1: APS, no Fusion needed ───────────────────────────────────────────+t("fusion_aps_status", "daily", {}, {"ok": True})+t("fusion_aps_get_browser", "daily", {}, {"ok": True})+t("fusion_aps_search", "daily", {"query": "BQ25792", "limit": 5}, {"ok": True, "pred": "count_gt_0"})+t("fusion_aps_recent", "daily", {"limit": 5}, {"ok": True, "pred": "count_gt_0"})+t("fusion_aps_browse", "daily", {}, {"ok": True, "fields": ["items"]})+t("fusion_aps_file_info", "daily", {"query": "BQ25792"}, {"ok": True, "fields": ["versionCount"]})+t("fusion_aps_versions", "weekly", {"query": "BQ25792"}, {"ok": True, "fields": ["versionCount"]})+t("fusion_aps_download", "weekly", {"query": "BQ25792", "saveDir": SCRATCH}, {"ok": True})+t("fusion_aps_get", "daily", {"path": "/project/v1/hubs"}, {"ok": True})+t("fusion_uninstall", "daily", {"dryRun": True}, {"ok": True, "fields": ["removed", "skipped", "dryRun"]})+# APS mutations: only into the AI workspace, weekly, with cleanup+t("fusion_aps_create_folder", "manual", note="needs projectId/parentFolderId of the test workspace")+t("fusion_aps_upload", "manual", note="cloud write; exercised via build flows weekly")+t("fusion_aps_set_client_id", "manual", note="would clobber org auth config")+t("fusion_aps_set_browser", "manual"); t("fusion_aps_forget_browser", "manual")+t("fusion_aps_signin", "manual", note="opens a browser on the user's machine")++# ── stage 2: Fusion lifecycle + prefs + MCP ─────────────────────────────────+t("fusion_start", "daily", {}, {"ok": True}, "runner restores prior running-state at the end")+t("fusion_get_preferences", "daily", {}, {"ok": True, "fields": ["current"]})+t("fusion_set_preference", "daily", {"theme": "darkblue"}, {"ok": True}, "sets the value it already has")+t("fusion_mcp_status", "daily", {}, {"ok": True, "fields": ["enabled"]})+t("fusion_mcp_tools", "weekly", {}, {"ok": True}, "only meaningful when MCP enabled")+t("fusion_mcp_call", "weekly", {"tool": "fusion_mcp_read", "arguments": {"queryType": "summary"}}, {"ok": True})+t("fusion_mcp_resources", "weekly", {}, {"ok": True})+t("fusion_mcp_enable", "manual", note="drives the prefs UI, takes foreground")+t("fusion_prefs_open", "manual", note="opens a dialog on the user's screen")+t("fusion_prefs_close", "manual")+t("fusion_set_auto_update", "daily", {}, {"ok": True}, "no args = read-only")++# ── stage 3: document flows on the canonical board ──────────────────────────+OPEN = {"fileName": "BQ25792", "projectName": "Main", "folderPath": "Molecules/XRP/Power"}+t("fusion_open_cloud_file", "daily", OPEN, {"ok": True}, "the canonical demo board")+t("fusion_show_schematic", "daily", {}, {"ok": True})+t("fusion_show_2d_board", "daily", {}, {"ok": True})+t("fusion_show_3d_board", "daily", {}, {"ok": True})+t("fusion_board_info", "daily", {}, {"ok": True, "fields": ["rawXml"]})+t("fusion_inspect_bodies", "daily", {}, {"ok": True, "fields": ["bodies", "units"]})+t("fusion_assembly_bom", "daily", {}, {"ok": True, "fields": ["parts", "collapsedUnits", "possibleUnderCollapse"]})+t("fusion_physical_properties", "daily", {}, {"ok": True})+t("fusion_get_design_info", "daily", {}, {"ok": True})+t("fusion_get_parameters", "daily", {}, {"ok": True})+t("fusion_document_info", "daily", {}, {"ok": True})+t("fusion_list_cloud_projects", "weekly", {}, {"ok": True}, "slow hub walk")+t("fusion_list_cloud_files", "weekly", {"projectName": "Main"}, {"ok": True})+# THE GUARD (issue #289): mismatch must refuse - on a READ verb too+t("fusion_run_modeling_script#guard", "negative",+  {"script": "result=1", "expectDocument": "ZZZ_NoSuchDoc_TEST"}, {"errorCode": "wrong_document"})+t("fusion_assembly_bom#guard", "negative",+  {"expectDocument": "ZZZ_NoSuchDoc_TEST"}, {"errorCode": "wrong_document"})+t("fusion_run_modeling_script", "daily",+  {"script": "result = {'ok': True}", "expectDocument": "BQ25792"}, {"ok": True})+t("fusion_execute_text_command", "daily", {"command": "Commands.List", "expectDocument": "BQ25792"}, {"ok": True})+t("fusion_electron_run", "weekly", {"command": "WINDOW FIT;"}, {"ok": True}, "board view must be active")+t("fusion_electron_zoom", "weekly", {"factor": 1.0}, {"ok": True})+t("fusion_electron_pan", "weekly", {"dx": 0, "dy": 0}, {"ok": True})+t("fusion_electron_select", "manual", note="selection state is user-visible")+t("fusion_open_by_urn", "weekly", note="URN fetched from aps_search at runtime", args={"_dynamic": "urn_from_search"})+t("fusion_aps_open", "weekly", {"query": "BQ25792", "wait": False}, {"ok": True})+t("fusion_open_design", "manual", note="local-file open; superseded by cloud flows")+t("fusion_close_window", "daily", {}, {"ok": True}, "closes the test board tab, restores tab state")+t("fusion_close_document", "manual"); t("fusion_close_all_documents", "manual", note="would close the user's own tabs")++# ── stage 4: exports (local writes to scratch) ──────────────────────────────+for fmt in ("step", "stl", "3mf", "f3d", "iges", "obj", "sat"):+    t(f"fusion_export_{fmt}", "weekly", {"outputPath": f"{SCRATCH}/verbtest.{fmt}"}, {"ok": True},+      "needs a DESIGN doc active; runner opens the 3D PCB first")+for fmt in ("fbx", "usdz", "dxf", "dwg", "skp"):+    t(f"fusion_export_{fmt}", "manual", note="format needs specific doc types; verify quarterly")+t("fusion_export_gerbers", "weekly", {"outputDir": SCRATCH}, {"ok": True}, "board view active")+t("fusion_export_bom", "weekly", {"outputPath": f"{SCRATCH}/bom.csv"}, {"ok": True})+t("fusion_export_cpl", "weekly", {"outputPath": f"{SCRATCH}/cpl.csv"}, {"ok": True})+t("fusion_export_optimized_glb", "weekly", {"outputPath": f"{SCRATCH}/board.glb"}, {"ok": True})+t("fusion_set_design_rules", "manual", note="mutates the board's DRC rules")+t("fusion_import_step", "weekly", {"_dynamic": "staged_step"}, {"ok": True}, "examples/ADS8588SIPM.step staged to the box")+t("fusion_web_export", "manual")++# ── stage 5: libraries (weekly; cloud writes cleaned by cleanup_cloud_files) ─+t("fusion_open_lbr", "weekly", {"_dynamic": "staged_lbr"}, {"ok": True})+t("fusion_capture_library_views", "weekly", {}, {"ok": True}, "after open_lbr")+t("fusion_generate_package", "weekly", {"type": "chip", "params": {"D": 1.6, "E": 0.8, "A": 0.45}}, {"ok": True})+t("fusion_attach_3d_package", "manual", note="modal Save dialog needs a human eye")+t("fusion_make_3d_package", "manual", note="cloud upload; run in release rehearsals")+t("fusion_build_library_3d", "manual", note="long multi-part build; release rehearsals")+t("fusion_cleanup_cloud_files", "weekly", {"dryRun": True}, {"ok": True})+t("fusion_board_stackup", "weekly", {}, {"ok": True})+t("fusion_new_electronics_from_eagle", "manual", note="drives native dialogs")+t("fusion_update_libraries", "manual")++# ── never in automation ──────────────────────────────────────────────────────+for v in ("fusion_stop", "fusion_kill", "fusion_signin", "fusion_signin_2fa", "fusion_demo",+          "fusion_install_fusion", "fusion_notify_owner", "fusion_open_by_urn_note"):+    t(v, "manual", note="lifecycle/human-wall verbs; manual tier")++PLAN = P++def main():+    ap = argparse.ArgumentParser()+    ap.add_argument("--target", required=True)+    ap.add_argument("--tier", default="daily", choices=["daily", "weekly", "all"])+    ap.add_argument("--only", default="")+    a = ap.parse_args()+    tiers = {"daily": {"daily", "negative"}, "weekly": {"daily", "weekly", "negative"},+             "all": {"daily", "weekly", "negative"}}[a.tier]+    only = set(x.strip() for x in a.only.split(",") if x.strip())++    described = {v[0] for v in describe._T}+    untriaged = described - {k.split("#")[0] for k in PLAN}+    results, t0 = [], time.time()+    for name in sorted(untriaged):+        results.append({"verb": name, "status": "FAIL", "why": "described but untriaged in tests/run_verb_tests.py"})++    for name, spec in PLAN.items():+        if spec["tier"] == "manual" or spec["tier"] not in tiers or (only and name not in only):+            if spec["tier"] == "manual" and not only:+                results.append({"verb": name, "status": "MANUAL", "why": spec.get("note", "")})+            continue+        if "_dynamic" in spec["args"]:+            results.append({"verb": name, "status": "SKIP", "why": "dynamic fixture not staged this run"})+            continue+        verb = name.split("#")[0]+        d, dt = call(a.target, verb, spec["args"])+        errs = expect(d, spec["exp"])+        if name.startswith("fusion_search_cloud_files") or name.startswith("fusion_walk_cloud_tree"):+            # must refuse: success:true here is the FAILURE+            errs = [] if (d.get("success") is False or "disabled" in str(d.get("error", "")).lower()+                          or "DISABLED" in str(d.get("error", ""))) else ["deprecated verb did NOT refuse"]+        results.append({"verb": name, "status": "PASS" if not errs else "FAIL",+                        "why": "; ".join(errs), "sec": round(dt, 1)})+        print("  %-42s %s %s" % (name, "PASS" if not errs else "FAIL", ("- " + "; ".join(errs)) if errs else ""))++    passed = sum(1 for r in results if r["status"] == "PASS")+    failed = [r for r in results if r["status"] == "FAIL"]+    stamp = datetime.datetime.now().strftime("%Y-%m-%d_%H%M")+    report = {"date": stamp, "target": a.target, "tier": a.tier,+              "pass": passed, "fail": len(failed), "results": results,+              "durationSec": round(time.time() - t0, 1)}+    out = REPO / "tests" / "reports" / f"verb-test-{stamp}.json"+    out.write_text(json.dumps(report, indent=1))+    print(f"\n{passed} PASS, {len(failed)} FAIL "+          f"({sum(1 for r in results if r['status']=='MANUAL')} manual-tier, "+          f"{sum(1 for r in results if r['status']=='SKIP')} skipped) -> {out}")+    for r in failed: print("  FAIL:", r["verb"], "-", r["why"])+    sys.exit(1 if failed else 0)++if __name__ == "__main__":+    main()