The CSWA benchmark (0.11.0; community parts, Step 4 and the live gate 0.11.4)

"Pull 3-5 random parts from CSWA exams as part of our e2e testing/validation for each release. We can also use those as benchmarks for new feature developments." — Drew, 2026-10-15

Each spec in resources/cswa/ is ONE exam-style part: the published answer key, the material, and the swx_run_script that builds it. The specs ship in the release zip (resources/cswa/) and are embedded in the exe.

Running it

cswa-check.py follows tools/dsl-live-check.py's rules:

• every creation says purpose:"bridge";
• by default every part is built in ONE bridge-made part (reuseDocument, whose leading new_part becomes clear_part); --fresh gives each spec its own part;
• it STOPS on a user_present, low_commit_memory or dialog refusal, and never creates another part after one.

The release gate

tools/release.sh --canary                                          # release candidate <ver>-rc<N> + manifests/canary.json
adom-bridge --target adom_drew2 bridge_install '{"manifestUrl":"https://wiki.adom.inc/api/v1/pages/solidworks-bridge/files/manifests/canary.json"}'
python3 tools/cswa-check.py --target adom_drew2 --report auto      # live, on THAT build; names its report
tools/release.sh --require-live tools/cswa/reports/<that report> --publish   # promotes the candidate's exact zip

cswa-check records WHICH BUILD it ran in bridge (read-only, through ab): exeSha256, the installed exe read back with read_file and hashed; installedFrom and zipSha256, the manifest ab installed from and the zip it named; installPath, version. The publish ships the canary's zip without rebuilding it, and only when bridge.exeSha256 is the exe inside it.

Since 0.11.4 EVERY publish goes through it: --publish alone is refused before anything is built. --require-live refuses to go on unless the report (tools/check-live-report.py):

• comes from cswa-check;
• was made by THIS bridge version;
• ran EVERY spec shipped in resources/cswa/ (0.11.4: a part may be skipped only for an op the bridge lacks; a run narrowed with --only is refused) and passed every part it ran;
• did not stop;
• is newer than the last commit that touched src/, resources/ or Cargo.toml;
• ran the exe being published (bridge.exeSha256), from the canary's zip if it says (bridge.zipSha256).

The one exception is --docs-only --publish, for a release that changes no code (tools/check-docs-only.py: nothing in src/ or resources/, and nothing in Cargo.toml or Cargo.lock beyond the package's version, since the last publish). Its changelog says the live gate was skipped. The ritual is in the main README, under "Live validation". Commit each live report in reports/ beside the earlier ones.

Spec schema

{
  "id": "cswa-05-…",                      // also the file name
  "title": "…",
  "source": "the URL of the drawing and the key",
  "official": true,                        // Dassault's own sample exam; false = community
  "units": "MMGS",                         // the DSL is millimetres; convert IPS parts first
  "material": {"name": "AISI 1020", "densityKgM3": 7900},
  "expected": {
    "massG": 939.54,                       // REQUIRED: the published answer
    "volumeMm3": null,                     // optional
    "centerOfMassMm": [x, y, z],           // optional, in the part's own frame
    "surfaceAreaMm2": null,                // optional
    "tolerancePct": 1.0                    // CSWA accepts about 1 %
  },
  "keyBasis": "where the numbers came from, when it is not simply the published key",
  "derivation": [["V", "120*80*12 - ..."], ["volumeMm3", "V"], ["massG", "V*2.7e-3"]],  // a DERIVED key, re-derivable (0.11.4)
  "exercises": ["revolve.360", "cut.regions"],   // the op variants it builds (a test holds the simulator to it)
  "simulatorSkips": "why only SOLIDWORKS can build it",  // optional: skipped in the simulator, run live
  "prediction": {"massG": 434.715, "how": "..."},        // optional: what the rebuild should weigh, when it is not the key
  "needsOps": [],                          // ops the bridge lacks: the part is SKIPPED, never half-built
  "variables": {"A": 81}, "derived": [["xR", "A-14"]],   // optional, see below
  "extends": "<another spec's id>",        // optional, see below
  "script": [ … swx_run_script steps … ]
}

Rules:

• The key is the PUBLISHED number, never one the bridge computed. A spec whose key was derived (spec 0's mass is Michael's measured volume times the density of ABS) says so in keyBasis.
• A COMMUNITY part's key is derived analytically, and the derivation ships with it (0.11.4). derivation is a list of [name, Python expression] pairs, evaluated in order with math's functions and the spec's variables in scope. It must yield massG, plus volumeMm3 and centerOfMassMm where the key has them, each within 1e-6 of the key: python3 tools/cswa-check.py --derivations checks every one, and so does cargo test. Such a key is exact, so the spec holds SOLIDWORKS to 0.1 %, not 1 %. Each spec states that 0.1 % tells its readings apart.
• A derivedAppend name may not shadow one the base derives. If it did, every "=name" in the base's script would silently take the new value: Step 4's tpx once redrew Step 1's R5 fillet. Both expanders refuse it.
• A runnable spec must parse — cargo test parses every one (src/cswa.rs).
• A spec that needs a missing op lists it in needsOps. That is the backlog: when the op lands, the spec runs.

Variables, derived values and extends

An exam often reuses one drawing with new values (the Tool Block's steps). A spec may give variables (numbers), derived ([name, expression] pairs, evaluated in order) and write any number of its script or key as "=expression". extends: "<id>" takes another spec whole and overrides its variables; scriptAppend / derivedAppend add steps and values after the base's. Expressions: + - * / ^, parentheses, pi, sqrt sin cos tan asin acos atan atan2 abs min max rad deg. This is the BENCHMARK's grammar, in src/cswa.rs and tools/cswa-check.py (a test holds the two to the same numbers); the bridge itself is only ever sent numbers.

The parts

The recipes, ambiguities and the reviewer's numeric rebuilds are in SOURCES.md (tb2.py, tb3b.py, tb4c.py beside it). In the simulator every runnable part lands within 0.01 % of its key, or of its prediction (cargo test --test cswa, CSWA_PRINT=1 prints the table).

Step 4's reading. SOURCES.md Part 4, read literally, gives 434.72 g (+0.49 %). The front face's outer loop is offset 1 mm in. A 1-mm wall stays outside the slot's R41 silhouette (R42). The pocket outside that wall goes to 12 from the back face. The ear inside R41 is recessed to 1 mm short of the slot's front wall, leaving a 1-mm skin. The reviewer's best case, 432.86 g, cuts the ear through to the slot instead, and keeps a 1-mm ring round the Ø11. The drawing cannot tell the two apart (ambiguity 4), and both are within the exam's 1 %.

Roadmap: placeholders (drawings only, not transcribed)

These are one line each: the source and its published key. The geometry stays in the source; no exam or book content is copied here.

• 2007 sample exam, Q1: https://cdn.goengineer.com/cswasampleexam2007.pdf (key 1280 g).
• 2007 sample exam, Q6: https://cdn.goengineer.com/cswasampleexam2007.pdf (key 2040.57 g).
• Planchard, Official CSWA Examination Guide, 3-3: https://static.sdcpublications.com/pdfsample/978-1-58503-656-1-3.pdf (key 302.40 g, CoM (0, 19.20, 0)).
• Official chain-link assembly, 2017 sample exam: https://www.solidworks.com/sites/default/files/2017-12/CSWASampleExam.pdf (key CoM (348.66, −88.48, −91.40) at A 25° B 125° C 130°; it needs assemblies with mates, which the DSL lacks).

Spec 0

cswa-00-lego-2x4: the LEGO 2x4 acceptance brick from issue #1. It is not an exam part. Its volume, 2073.3615208 mm³, is the one SOLIDWORKS 2021 measured for Michael's brick; the mass is that volume in ABS (1020 kg/m³).