Chip Fetcher (cf)

@@ -0,0 +1,177 @@+# Email-delivered CAD: the JST flow (and how to add any email-gated manufacturer)++Most manufacturers serve CAD as a direct download. A growing few **email it to+you** instead. **JST (J.S.T. Mfg. Co., Ltd.)** is the reference case: since+**April 2025**, requesting a STEP/IGES file on jst-mfg.com sends it as an+**email attachment** rather than a download. This page documents the whole+flow end-to-end so any chip-fetcher user, on any AI, can reproduce it, and so+non-Gmail users know exactly what to build to participate.++> Proven end-to-end on **PHR-2** (2026-06-27): native browser → agreement form+> → JST emailed `PHR-2.zip` (containing `PHR-2.STEP`) → fetched from Gmail →+> imported. No JST account required.++This is the human-readable companion to the agent playbooks shipped in the+skill: `playbooks/jst.md` and `playbooks/jst-email-retrieval.md`.++---++## TL;DR++1. **Use a real browser, not a headless/pup one.** jst.com (USA) is+   Cloudflare-protected and blocks automated browsers. Drive your real,+   signed-in Chrome via the Adom **native-browser bridge**+   ([adom-browser-extension](https://wiki.adom.inc/adom/adom-browser-extension)).+2. JST's CAD lives on the **global site jst-mfg.com**, organized by **series**+   (PH = 199, SH = 231, … ~600 series). jst.com just routes there.+3. A part's **Download** button does **not** return a file — it returns a+   **License Agreement form**. **Accept cookies first**, fill+   company/section/name/address/tel/**email**, and submit ("I agree").+4. JST **emails** the `.zip` (STEP or IGES) to that address within ~1 minute.+5. **Retrieve the attachment from email** and import it. There is **no+   account login** for CAD.++## Why a real browser is mandatory++jst.com sits behind Cloudflare's bot challenge ("Just a moment… / Performing+security verification"). Automated/headless browsers (including pup) get stuck+on it. The native-browser bridge drives your **actual** Chrome window, which+already carries Cloudflare's human-trust, so it passes cleanly. If you don't+have the bridge installed, set it up first:+[adom-browser-extension](https://wiki.adom.inc/adom/adom-browser-extension).++(jst.com also shows a "Distributor Login" — that's for distributors, **not** a+CAD gate. You never need it for CAD.)++## Step 1 — find the part's series++JST groups CAD by **series**, not by individual part number.++- **Browse all series:** the Series Alphabet Search,+  `https://www.jst-mfg.com/product/index.php?type=5`. Every series is a+  `index.php?series=<id>` link. This is also your **scope list** if you ever+  batch the catalog.+- **Or search a known part:** `https://www.jst-mfg.com/search/?q=<part>`.+  Strip ordering suffixes first: drop the lead-free/plating `(LF)(SN)` and any+  `(…)` option group, plus a trailing suction-cap `T` on SMT headers. Example:+  `B10B-PH-K-S(LF)(SN)` → search `B10B-PH-K-S`.++## Step 2 — read the CAD format columns++Open the series page. Each part row has **Download** buttons under format+columns. The `doc=` number in the link is **per-series** — read the **column+header**, don't assume:++| Column (PH series, 199) | `doc=` | What you get |+|---|---|---|+| IGES   | 1 | 3D model, `.IGS` |+| STEP   | 2 | 3D model, `.STEP` (prefer this) |+| 3D-PDF | 3 | 3D PDF |+| 2D-PDF | 4 | dimensioned drawing PDF |++⚠️ **Coverage varies.** Some older series offer **only IGES** (no STEP). If+there's no STEP, take IGES and convert downstream (`step2glb` /+`service-kicad`).++The download link itself looks like:+`https://www.jst-mfg.com/product/index.php?type=10&series=<S>&doc=<D>&filename=<PART>.zip`++## Step 3 — the agreement form (the gate)++Opening that link does two things on JST's server: it **remembers which file+you asked for** (in your session) and shows a **License Agreement** page with+required fields:++- Company name, Section name, Name, Address, Telephone, **E-Mail** (all+  required), URL (optional).++**Accept the cookie banner first** — JST states *"accept cookies in order to+submit any of our forms."* Then fill the fields and click **"I agree"**. The+page confirms: **"The data has been sent to your email address."**++Because the requested file is tied to your **browser session**, the page you+open (which sets it) and the submit must happen in the **same browser** — which+is automatic when you drive it in one window.++For a **large batch**, request to a filtered alias (e.g.+`[email protected]`) so the attachments are easy to isolate and your main+inbox isn't buried.++## Step 4 — retrieve the emailed file++This is the part that needs an **email-reading capability**, which is+**account-specific** and therefore not bundled into chip-fetcher. Pick the path+for your provider.++### Gmail / Google Workspace → `adom-google`++[`adom-google`](https://wiki.adom.inc/adom/adom-google) reads Gmail via the+API (no browser automation):++```bash+adom-google status                                   # confirm Gmail scope is authorized+# find the JST mail (strip the leading "HTTP 200 …" line before parsing JSON):+adom-google api "https://gmail.googleapis.com/gmail/v1/users/me/messages" \+    -q q="JST newer_than:1d" -q maxResults=5+adom-google gmail attachments <messageId>            # lists "<PART>.zip … id: <attId>"+adom-google gmail read <messageId> <attId> -o <PART>.zip+```++Tip: IGES and STEP both arrive as `<PART>.zip` — tell them apart by the+**inner** file (`.STEP` vs `.IGS`) and take the **newest** matching message.++### Not a Gmail user? Build it, share it, breadcrumb it++If you're on Microsoft 365/Outlook, Fastmail, a corporate IMAP server, etc.,+`adom-google` doesn't apply. You need a small equivalent that (1) connects to+your mailbox (Microsoft Graph, IMAP, JMAP…), (2) finds the manufacturer mail,+and (3) saves the `.zip` attachment — the same three steps as above.++Then **share it so nobody re-solves it**:++- Publish your fetcher as a small CLI/skill on the [Adom Wiki](https://wiki.adom.inc)+  (the way `adom-google` is published). Keep secrets local; mirror the+  `attachments` / `read` verb shape so this doc's steps still read true.+- **Drop a chip-fetcher [breadcrumb](https://wiki.adom.inc/adom/wiki-breadcrumbs)**+  pointing at your fetcher, so it auto-surfaces from this page for the next+  person on your provider — no edit to chip-fetcher required.++That keeps chip-fetcher **provider-agnostic**: the email layer is a pluggable,+community-extensible piece, and every new provider integration is discoverable+right here.++## Step 5 — datasheet + import++JST has no per-part datasheet; use the **series catalog PDF**, a plain direct+download (no gate): `https://www.jst-mfg.com/product/pdf/eng/e<SERIES>.pdf`+(e.g. `ePH.pdf`). The per-part **2D-PDF** drawing is the mechanical print.++```bash+adom-chip-fetcher import <PART>.STEP --mpn <PART>+adom-chip-fetcher import <catalog>.pdf --mpn <PART>+```++## What JST gives you — and what it doesn't++JST supplies **3D models (STEP/IGES) + drawings + the series catalog datasheet+only**. It does **not** supply EDA **symbol** (`.kicad_sym`) or **footprint**+(`.kicad_mod`). Those still come from chip-fetcher's normal ladder+(**SnapEDA/SnapMagic → Mouser → DigiKey → UL/CSE**). `adom-chip-fetcher gate`+will correctly keep a JST card **incomplete** until the symbol + footprint land+— that's expected, not a bug.++## Batch hazards (read before crawling the catalog)++- The full catalog is ~600 series × tens of thousands of parts → that many+  form submits **and emails**. Throttle hard (human-like 2–5s delays), route to+  a filtered alias, and expect JST to rate-limit. **Scope deliberately** (a+  priority set of series, or per-BOM) rather than blasting everything.+- One email per file; STEP and IGES are separate requests/emails.++## See also++- Agent playbooks in the skill: `playbooks/jst.md`,+  `playbooks/jst-email-retrieval.md`+- [adom-browser-extension](https://wiki.adom.inc/adom/adom-browser-extension) ·+  [adom-google](https://wiki.adom.inc/adom/adom-google) ·+  [wiki breadcrumbs](https://wiki.adom.inc/adom/wiki-breadcrumbs)

@@ -0,0 +1,148 @@+# Playbook: JST connectors (email-gated CAD)++**Read this for any JST part** (PHR-/XHP-/ZHR-/SHR-/EHR-/VHR-/PAP-/GHR-/NHR-+housings, or B\*B-/S\*B-/BM\*B-/SM\*B- board/SMT headers). JST is the+canonical example of a **manufacturer that emails you the CAD** instead of+serving it as a download. Proven end-to-end on PHR-2, 2026-06-27.++> One-line shape: native browser → jst-mfg.com download link → accept cookies+> → fill agreement form → JST **emails** the `.zip` → fetch the email+> attachment → import. There is **no account login** for CAD.++---++## 0. Why JST is special (don't skip)++- **jst.com (USA) is Cloudflare-walled against pup.** A pup navigation to any+  deep product page returns a "Just a moment… / Performing security+  verification" interstitial that never clears. **You MUST use the+  native-browser bridge** (`nbrowser_*`, the user's real signed-in Chrome via+  `adom-browser-extension`) — it carries human-trust and passes Cloudflare.+  If the bridge is absent, tell the user to install it+  (https://wiki.adom.inc/adom/adom-browser-extension); pup will not work here.+- jst.com is only marketing + spec tables + a **"Distributor Login"** (that+  login is for distributors, NOT a CAD gate). The actual CAD + catalogs live+  on the **global site jst-mfg.com**, which jst.com links to.+- **No login is required to get CAD.** The gate is a per-download **License+  Agreement form** whose submit triggers an **email**.++## 1. Resolve the part → series++JST CAD is organized by **series**, not by individual MPN. Each series has a+numeric id (PH = 199, PH high-box = 200, SH = 231, …; ~600+ series).++- Master index ("Series Alphabet Search"):+  `https://www.jst-mfg.com/product/index.php?type=5` — every series is an+  `index.php?series=<id>` link. This page is also your **scope list** for a+  batch (one row per series).+- Or keyword-search a known MPN: `https://www.jst-mfg.com/search/?q=<MPN>`+  (`sources::jst::search_url`). Strip ordering suffixes first with+  `sources::jst::base_part` — drop `(LF)(SN)` and any `(…)` option group, and+  a trailing suction-cap `T` on SMT headers+  (`B10B-PH-K-S(LF)(SN)` → `B10B-PH-K-S`).++## 2. Read the CAD columns on the series page++Open the series page in the native browser. Each part row has **Download**+buttons under format columns. Map them by reading the **column header** — the+`doc=` number is **per-series**, not universal:++| Common (PH, series=199) | doc= | format | inner file |+|---|---|---|---|+| IGES  | 1 | 3D    | `<PART>.IGS`  |+| STEP  | 2 | 3D    | `<PART>.STEP` ← **prefer this** |+| 3D-PDF| 3 | 3D pdf| `<PART>.pdf`  |+| 2D-PDF| 4 | drawing pdf | `<PART>.pdf` |++⚠️ **Coverage varies**: some older series offer **only IGES** (no STEP). If a+series has no STEP column, take IGES and convert with `step2glb`/service-kicad+downstream. Always read the header — never assume `doc=2` is STEP.++The link itself is built by `sources::jst::download_url(series, doc, filename)`:+`…/product/index.php?type=10&series=<S>&doc=<D>&filename=<PART>.zip`++## 3. The download link returns a FORM, not a file++GET-ing the download link does two things server-side:+1. binds `(series, doc, filename)` into the **PHP session**, and+2. returns a **License Agreement** HTML page (no file).++The page has required fields **company, section, name, address, tel, email**+(+ optional **url**) and an **"I agree"** button wired to a JS `download()`+function. `download()` collects every form value and POSTs them to+`ajax_download.php` (jsonp). The server reads the file to send from the+**session set in step 1** — so the GET and the POST **must share the browser+session** (same native-browser window = automatic).++**Accept cookies first.** The cookie banner literally says *"accept cookies in+order to submit any of our forms."* Click "Allow all cookies" before filling.++### Driving it (native browser)+1. `nbrowser_navigate` to the `download_url` → wait ~2.5s.+2. `nbrowser_eval`: accept cookies, set the 7 fields by id (dispatch+   `input`+`change`), then call `download()` (or click "I agree"). The success+   callback fires `alert("The data has been sent to your email address")` then+   navigates to `index.php?series=<S>` — hook `window.alert` into+   `sessionStorage` first since navigation wipes globals, then read it back.++Use the user's real business profile for the fields (see+`user_profile.md`). For a **batch**, set the email to a plus-alias like+`[email protected]` with a Gmail filter/label so the inbox isn't buried (the+file still arrives; adom-google can still read it).++## 4. Retrieve the emailed file (the part most tools miss)++Since **April 2025** JST sends the CAD as an **email attachment** (`<PART>.zip`,+inner `<PART>.STEP`/`.IGS`), from a JST send-only address, ~30-60s later. This+is where chip-fetcher needs an **email-reading capability** —+see [`jst-email-retrieval.md`](jst-email-retrieval.md) for the Gmail+(adom-google) path and the **non-Gmail** build-your-own path.++Gmail quick path (adom-google):+```bash+adom-google api "https://gmail.googleapis.com/gmail/v1/users/me/messages" \+  -q q="JST newer_than:1d" -q maxResults=5          # strip leading "HTTP 200…" line before JSON+adom-google gmail attachments <messageId>            # find <PART>.zip + its attId+adom-google gmail read <messageId> <attId> -o <PART>.zip+```+Pick the **newest** message whose inner file matches the format you requested+(IGES and STEP both arrive as `<PART>.zip` — distinguish by the inner+`.STEP` vs `.IGS`).++## 5. Datasheet = series catalog (DIRECT download, no gate)++JST has no per-part datasheet. Use the **series catalog PDF**, a plain direct+download: `sources::jst::catalog_pdf_url("PH")` →+`https://www.jst-mfg.com/product/pdf/eng/ePH.pdf`. Fetch with+`nbrowser_fetch_url` (returns base64 for binary — decode before writing) and+import as the part's `.pdf`. The 2D-PDF drawing (doc=4) is the per-part+mechanical print if you want it too.++## 6. Import + what JST does NOT give you++```bash+adom-chip-fetcher import <PART>.STEP --mpn <PART>+adom-chip-fetcher import <catalog>.pdf --mpn <PART>+```+JST supplies **3D (STEP/IGES) + drawings + catalog datasheet only**. It does+**NOT** supply KiCad `.kicad_sym` / `.kicad_mod`. Those still come from the+normal ladder — **SnapEDA/SnapMagic → Mouser → DigiKey → UL/CSE** — so after+the JST manufacturer-direct grab, keep walking the ladder for symbol ++footprint. `adom-chip-fetcher gate` will (correctly) keep the card INCOMPLETE+until sym+mod land.++## 7. Batch notes / hazards++- **The full catalog is ~600 series × tens of thousands of parts** = that many+  form submits + emails. Throttle hard (2-5s human-like delays, per critical+  rule #9), route mail to a filtered alias, and expect JST to rate-limit. Do+  NOT blast the whole catalog without the user scoping it.+- One email per file, one file per `(series, doc, filename)` GET→POST. STEP and+  IGES are separate requests/emails.+- Heartbeat each stage: `adom-chip-fetcher heartbeat <PART> "probing jst-mfg.com"`+  / `"requested STEP — awaiting email"` / `"imported"`.++## Source plugin++`src/sources/jst.rs` — `series_index_url`, `series_url`, `search_url`,+`download_url`, `catalog_pdf_url`, `DocKind`, `FORM_FIELDS`, `base_part`.

@@ -0,0 +1,145 @@+//! JST (J.S.T. Mfg. Co., Ltd.) — connector CAD source plugin.+//!+//! JST is unusual: its CAD is **email-gated**. Hitting a download link does+//! NOT return the file — it returns a License Agreement form, and on submit+//! (since April 2025) JST EMAILS the CAD as a `.zip` attachment. The full+//! flow (native-browser bridge, form fill, Gmail retrieval) lives in+//! `playbooks/jst.md`. This module is just the URL/part-number plumbing.+//!+//! Two sites:+//!   * jst.com (USA) — Cloudflare-walled against pup; spec tables only; it+//!     routes to jst-mfg.com for CAD. Drive it with the native-browser+//!     bridge, never pup.+//!   * jst-mfg.com (global) — where the CAD + catalogs actually live.+//!+//! No account login is required for CAD — only the agreement form + a+//! reachable email address.++/// The master index of every JST series ("Series Alphabet Search"). Each+/// series on this page is an `index.php?series=<id>` link. Start here when+/// you don't already know a part's series id.+pub fn series_index_url() -> &'static str {+    "https://www.jst-mfg.com/product/index.php?type=5"+}++/// A series listing page (all parts + their CAD download buttons).+pub fn series_url(series_id: u32) -> String {+    format!("https://www.jst-mfg.com/product/index.php?series={series_id}")+}++/// Global-site keyword search — use to resolve an MPN to its series when the+/// series id is unknown.+pub fn search_url(mpn: &str) -> String {+    let mut q = String::with_capacity(mpn.len());+    for b in mpn.bytes() {+        match b {+            b'A'..=b'Z' | b'a'..=b'z' | b'0'..=b'9' | b'-' | b'_' | b'.' | b'~' => q.push(b as char),+            _ => q.push_str(&format!("%{:02X}", b)),+        }+    }+    format!("https://www.jst-mfg.com/search/?q={q}")+}++/// JST CAD document kinds. The `doc=` query param selects the format, BUT the+/// numeric mapping is per-series — newer series expose STEP, older ones only+/// IGES. ALWAYS read the column header on the series page; treat these as the+/// common-case defaults (verified on the PH series, series=199), not gospel.+#[derive(Debug, Clone, Copy, PartialEq, Eq)]+pub enum DocKind {+    Iges,  // doc=1 (common) — 3D, .IGS+    Step,  // doc=2 (common) — 3D, .STEP  ← preferred for chip-fetcher+    Pdf3d, // doc=3 (common) — 3D-PDF+    Pdf2d, // doc=4 (common) — 2D-PDF drawing+}++impl DocKind {+    /// The common-case `doc=` number for this kind. Verify against the page.+    pub fn default_doc(&self) -> u32 {+        match self {+            DocKind::Iges => 1,+            DocKind::Step => 2,+            DocKind::Pdf3d => 3,+            DocKind::Pdf2d => 4,+        }+    }+}++/// The email-gated download link for one file. GET-ing this in the+/// native browser:+///   1. sets the PHP session binding for (series, doc, filename), and+///   2. returns the License Agreement HTML form.+/// You then accept cookies, fill the form, and call the page's `download()`+/// JS — JST emails the `.zip`. `filename` is the part's zip name, e.g.+/// "PHR-2.zip".+pub fn download_url(series_id: u32, doc: u32, filename: &str) -> String {+    format!(+        "https://www.jst-mfg.com/product/index.php?type=10&series={series_id}&doc={doc}&filename={filename}"+    )+}++/// Series catalog/datasheet PDF — a DIRECT download (no gate). `series_slug`+/// is the short uppercase series tag, e.g. "PH" → ePH.pdf. JST has no+/// per-part datasheet; use the catalog + the 2D-PDF drawing.+pub fn catalog_pdf_url(series_slug: &str) -> String {+    format!(+        "https://www.jst-mfg.com/product/pdf/eng/e{}.pdf",+        series_slug.to_ascii_uppercase()+    )+}++/// The required License Agreement form fields, in page order. Field ids on+/// the page match these keys. `url` is optional; the rest are mandatory.+pub const FORM_FIELDS: &[&str] = &["company", "section", "name", "address", "tel", "email", "url"];++/// Normalize a JST ordering MPN to its catalog part number for searching:+/// strip the lead-free/plating suffix `(LF)(SN)` and any trailing+/// parenthetical option groups, plus a trailing `T` suction-cap option on+/// SMT headers (per JST + DigiKey TechForum guidance). E.g.+///   "B10B-PH-K-S(LF)(SN)"  -> "B10B-PH-K-S"+///   "BM03B-SRSS-TBT(LF)(SN)" -> "BM03B-SRSS-TB"+pub fn base_part(mpn: &str) -> String {+    // Drop everything from the first '(' onward (all option groups).+    let mut s = match mpn.find('(') {+        Some(i) => mpn[..i].to_string(),+        None => mpn.to_string(),+    };+    s = s.trim().to_string();+    // Trailing suction-cap "T" on SMT headers (…-TBT -> …-TB). Only strip a+    // lone trailing T that follows another uppercase letter, to avoid eating+    // a real terminal T that is part of the code.+    if s.ends_with('T') {+        let chars: Vec<char> = s.chars().collect();+        if chars.len() >= 2 && chars[chars.len() - 2].is_ascii_uppercase() {+            s.pop();+        }+    }+    s+}++#[cfg(test)]+mod tests {+    use super::*;++    #[test]+    fn strips_plating_and_option_suffixes() {+        assert_eq!(base_part("B10B-PH-K-S(LF)(SN)"), "B10B-PH-K-S");+        assert_eq!(base_part("BM03B-SRSS-TBT(LF)(SN)"), "BM03B-SRSS-TB");+        assert_eq!(base_part("PHR-2"), "PHR-2");+        assert_eq!(base_part("S2B-PH-K-S(LF)(SN)"), "S2B-PH-K-S");+    }++    #[test]+    fn doc_defaults() {+        assert_eq!(DocKind::Step.default_doc(), 2);+        assert_eq!(DocKind::Iges.default_doc(), 1);+    }++    #[test]+    fn urls() {+        assert_eq!(+            download_url(199, 2, "PHR-2.zip"),+            "https://www.jst-mfg.com/product/index.php?type=10&series=199&doc=2&filename=PHR-2.zip"+        );+        assert_eq!(catalog_pdf_url("ph"), "https://www.jst-mfg.com/product/pdf/eng/ePH.pdf");+    }+}