# Mouser + Component Search Engine (chip-fetcher playbook)

Mouser is a first-class playbook (NOT a fallback tier) — it covers the long tail of chips without dedicated mfr CAD pages. Read this when manufacturer-direct doesn't host CAD, OR when you need stock + pricing + ECAD in one go.

**Heartbeat every stage** per [`dashboard-flow.md`](dashboard-flow.md). The slide-CAPTCHA escalation is a particularly important moment to heartbeat: post `mouser slide-CAPTCHA — user help needed` so the dashboard surfaces the stuck state, then raise the scraping window with `browser_raise_os_window` and let the user drag the slider. Don't silently wait — the user should see on the dashboard that we're stuck on Mouser.

## Mouser CSE Library Loader — the universal fallback (don't miss this)

**Critical pattern:** when a manufacturer has no CSE portal of their own (`<mfr>.componentsearchengine.com` returns `DNS_PROBE_FINISHED_NXDOMAIN`) AND no UL deep-link AND nothing on the mfr's own site, **check Mouser CSE**.

Pattern: `https://mouser.componentsearchengine.com/entry_u_newDesign.php?mna=<MFR>&mpn=<MPN>&pna=mouser&vrq=multi&fmt=zip&o3=0&lang=en-GB`

Mouser proxies CSE for any chip in their catalog, not just chips whose manufacturer paid for a dedicated CSE portal. It hits the same SamacSys library backend, returns the same mfr-uploaded STEP/symbol/footprint, with the same single CSE login.

### ⚠ TWO traps that make `entry_u_newDesign.php` return NO partID (both verified 2026-06-23, TI ADCs)

The entry endpoint returns one of three pages. Only one has a `partID`:
- **"SamacSys Part Preview"** (len ~50KB) → contains `partID=<N>`. ✅ This is what you want.
- **"SamacSys Part Selector" / "Pick One"** (len ~8–21KB) → a disambiguation page, **no partID**. ❌

Two independent causes push you onto the no-partID pages:

1. **Use the ORDERABLE MPN, not the base part.** `mpn=ADS1263` → "Pick One" (the base part has many package variants, so CSE can't resolve one). `mpn=ADS1263IPWR` (orderable, with package suffix) → "Part Preview", partID 755745. **Always pass the orderable MPN with its package/reel suffix.** If you only have the base MPN, a distributor parametric lookup (`adom-mouser`/`adom-digikey`) gives the orderable variant, or try the common suffix (TSSOP `…IPWR`, QFN `…IRTER`).

2. **Do NOT append `&logo=&epm=0` to the entry URL.** Those two extra params silently flip CSE into the "Part Selector" page (no partID) **even for a valid orderable MPN**. Use the minimal param set above. This cost an afternoon of "no-partID" debugging — the bug was a malformed URL, not the chip or the browser.

### `chip-fetcher cse <MPN> --mfr "<Manufacturer>"` — the automated path (use this first)

Does the whole CSE chain in code: navigate clean → entry (orderable MPN) → partID → preview → zipForm → `ga/model.php` → import STEP+symbol+footprint. Retries the cold-call flake and keeps the dashboard "working on" bar live via a heartbeat ticker. Prefer it over hand-driving URLs.

### How the download works — same-origin XHR + Basic-auth header (the breakthrough)

CSE downloads are HTTP Basic Auth. Inline-cred URLs (`https://user:pass@host/…`) are REJECTED by `fetch()`, and a plain navigate pops Chrome's invisible Basic-auth dialog. **The robust technique: a same-origin `XMLHttpRequest` (from a tab already on `mouser.componentsearchengine.com`) with `xhr.setRequestHeader('Authorization','Basic '+btoa(user+':'+pass))`.** XHR (unlike fetch) accepts the Authorization header AND pulls the bytes into JS as an arraybuffer — so the zip never hits Chrome's download policy (which silently blocks the 2nd+ download). This is what `cse_fetch_zip` in `pup.rs` does. **Vault-auth works in ANY browser** — the auth is the XHR header from chip-fetcher's vault, not a browser cookie, so CSE runs fine in the user's Chrome even if their saved CSE login lives in Edge.

**Real example:** Espressif has no CSE portal. Probing `espressif.componentsearchengine.com` gave NXDOMAIN. UL portal `vendor.ultralibrarian.com/espressif/embedded` gave "No Access is defined." But Mouser's product page for ESP32-S3FN8 had an "ECAD Model" block linking to `mouser.componentsearchengine.com/...` — and that path served a complete bundle. Same trick worked for the ESP32-S3-WROOM-1-N4 module.

## ⚠ HARD RULE — CSE downloads need credentials embedded in the URL

**Before you click any "Download CAD Models" button on `*.componentsearchengine.com`, embed credentials inline.** The `/signin` form-login session cookie is USELESS for downloads — those endpoints issue a separate HTTP Basic Auth challenge that pops a native Chrome dialog **invisible to `browser_eval`**. Symptom: page lands on `chrome-error://chromewebdata/`, no file in Downloads, nothing in DOM to indicate why. (Verified painful 2026-05-04.)

```bash
# Pull creds from chip-fetcher's vault, urlencode, prepend to URL:
CRED=$(curl -s http://127.0.0.1:8786/api/credentials | python3 -c '
import sys,json,urllib.parse
d=json.load(sys.stdin)
c=next((x for x in d["credentials"] if "componentsearchengine" in x["host"]), None)
if c: print(f"{urllib.parse.quote(c[\"username\"],safe=\"\")}:{urllib.parse.quote(c[\"password\"],safe=\"\")}")
')
URL="https://${CRED}@mouser.componentsearchengine.com/preview_newDesign.php?o3=0&partID=<ID>&ev=0&fmt=zip&pna=Mouser"
adom-desktop browser_navigate "{\"sessionId\":\"chip-fetcher\",\"url\":\"$URL\"}"
```

Always urlencode the username (`@` → `%40`) and password — special chars are common in CSE passwords. **Use this pattern by default for every CSE click that triggers a download.** Full diagnostic, OS-keychain alternative, and the two-vault sync requirement: see [`credentials-and-logins.md`](credentials-and-logins.md) § HARD RULE — CSE downloads use HTTP Basic Auth.

**Discovery flow when you can't find CAD on a vendor's site:**
1. Use `adom-mouser part <MPN>` to get the Mouser product URL.
2. Navigate pup to that Mouser page.
3. Look for "ECAD Model" / "Library Loader" link/text on the product page.
4. Follow the CSE URL into `mouser.componentsearchengine.com/...` directly.
5. Drive the standard CSE Download flow (same as NXP CSE).

## Mouser — first-class playbook

**Mouser is the most important external resource chip-fetcher has after the manufacturer's own site.** It hosts the datasheet AND the ECAD bundle for nearly every chip in their catalog — and for the long tail of chips whose manufacturers don't publish CAD (Vishay passives, niche connectors, smaller-vendor SoCs), Mouser is the *only* source.

Direct user quote: *"mouser is a key resource here so we have to get into the skill how to work with their website."*

**What's on every Mouser product page (right sidebar):**

- **`Datasheet:`** block — a `<MPN> Datasheet (PDF)` link. Either Mouser-CDN-hosted (`https://www.mouser.com/datasheet/<…>.pdf`) or a redirect to the manufacturer's CDN. Behind Akamai's bot wall, so use the in-page `fetch()` pattern, not naked curl.
- **`ECAD Model:`** block — `PCB Symbol, Footprint & 3D Model` link. Routes into `mouser.componentsearchengine.com/...` and serves a `LIB_<MPN>.zip` with `.step` + `.kicad_sym` + `.kicad_mod`.
- Extra reference docs (app notes, training PDFs) listed as additional `<a href="*.pdf">` rows further down — scrape them all.

## Operating manual — Mouser

1. **Always go through pup Chrome** — never naked curl. Their CDN serves a 174 KB HTML denial page (`"Access to this page has been denied because we believe you are using automation tools"`) for any non-browser request.
2. **Use the auto-cookie-accept helper** on every navigation — Mouser's cookie banner blocks downstream clicks via z-index.
3. **Use in-page `fetch()` with `{credentials:"include"}`** to pull PDFs (Chrome session cookies + JA3 fingerprint pass Akamai). Decode base64 result back into the library. `cf-mouser-pdf-pull.sh` is the reference implementation.
4. **For ECAD bundles, drive the full CSE Download flow inside the pup window.** Mouser-hosted CSE shares the same single-sign-on cookie as `componentsearchengine.com` proper, so once the user is signed in there, every Mouser ECAD link works without re-auth.
5. **Mouser's Search API (`adom-mouser`) gives you the canonical product URL + `datasheet_url` field for every variant.** Walk all variants — the version with the populated `datasheet_url` is sometimes a different package SKU than the MPN you searched for.

## Mouser's slide-to-verify CAPTCHA

After ~5–10 automated requests in a session, Mouser shows a slide-to-verify CAPTCHA. **Stop and ask the user to drag the slider.** Use `browser_raise_os_window` + `browser_screenshot` to surface the obstacle, then say: *"Mouser hit me with a slide-CAPTCHA — please drag the slider on the pup window to clear it."* Resume the flow once the CAPTCHA clears.

**DO NOT pivot to a different source for ECAD bundles** — there is no other source for those. Direct user pushback: *"that was a good test case … you can't just abandon the website if that's the only way to get the step file and symbol/footprint and the datasheet."*

**Cooldown:** Mouser's verification token persists for 30+ minutes after the user clears the slider. Batch your Mouser fetches during that window so the user only ever has to slide once per session.

**Critical caveat:** Mouser's CDN is Akamai-protected and rejects naked `curl` (returns 200 OK with a "Access to this page has been denied" HTML page). To pull a Mouser-hosted PDF you must either (a) navigate pup Chrome there and trigger a download (file lands in `~/Downloads`, then `chip-fetcher pull`), or (b) use in-page `fetch()` via `browser_eval` with `credentials:"include"` — Chrome's session cookies pass Akamai's bot check and the response can be base64-encoded back through `awaitPromise:true`.

## When the slider hits — escalation order

1. **Try `browser_input_dispatch type:"move"` along the slider track followed by `type:"click"` at the right edge.** Trusted CDP input has `isTrusted: true` and may pass GeeTest's basic acceleration check; some Mouser slider variants accept it.
2. **Pivot to vendor-direct ONLY if Mouser is fungible for this chip's docs.** TI lit IDs, ST `/resource/`, NXP `/docs/` — there are non-Mouser paths for documentation. Not for ECAD bundles.
3. **For ECAD bundles, there is no fungible path.** Hand off via `desktop_open_url` to the user's native browser (real Chrome with browsing-history reputation, extensions, real input). They slide once, the CAPTCHA token persists 30+ minutes, then the rest of the session works through pup again.
4. **Don't silently abandon a website if it's the only path.**

## CSE auth — single login covers all `<mfr>.componentsearchengine.com` subdomains

Sign in once at `https://componentsearchengine.com/signin` with the user's email + password, and the cookie covers every per-vendor subdomain (nxp, mouser, etc.). On first visit to a subdomain, an HTTP **Basic Auth** popup may appear once — it accepts the same CSE creds. After that, downloads on subdomain pages work.

The CSE STEP files we get for NXP/Mouser CSE are **mfr-authored, FreeCAD-exported** with named PRODUCTs (`ASSEMBLY`, `Body`, `ThermalPin`, `PinsArrayLR`, `PinsArrayTB`) — same authoring pipeline as TI's Fusion-electronics exports. So CSE-distributed STEPs match UL-distributed STEPs in scene-graph quality.

**Filename pattern: CSE downloads land as `LIB_<MPN>.zip` (vs UL's `ul_<MPN>.zip`).** The `cf-autowatch.sh` script's filename match needs to handle both — regex is `^(ul_|LIB_).*\.zip$`.

## Chrome's "wants to Download multiple files" SILENT KILLER

When CSE fires a download, Chrome detects multiple downloads (symbol + footprint + 3D + datasheet often arrive as separate transfers) and shows an Allow/Block dialog the first time per origin per session. **Programmatic `.click()` and `MouseEvent` dispatch CANNOT click Allow** — it's a Chromium browser-chrome dialog, not a page element.

Symptom: download button "works" (no error, page doesn't change), but `~/Downloads` stays empty for minutes.

**Cure: pre-grant the `automatic_downloads` permission to `*.componentsearchengine.com` BEFORE opening the page.** Two known approaches:
- (a) use Chrome DevTools Protocol `Browser.grantPermissions` via pup before the first navigation
- (b) launch the pup profile with `--auto-open-devtools-for-tabs` plus a setup script that flips the permission

**Do NOT ask the user to click Allow** — see "chip-fetcher does ALL the clicking" rule. Cloudflare/CSE reject curl-with-cookies attempts (HTTP 403 → CF anti-bot challenge → redirect to `/register`), so Chrome inside pup is the only viable path. Once the permission is granted at profile-load, all CSE clicks in that session run unattended.

(`chip-fetcher/scripts/adom-desktop-chrome-permissions-prompt.md` is the open feature request to bake `automatic_downloads: granted` into every pup `browser_open_window` by default.)

## Anti-detection / browse as a real user

Even when you go through pup Chrome, sites still profile *behavior*. Burst-clicking 5 product pages in 30 seconds with no scrolling, no mouse-movement, no referrer chain looks like a bot regardless of TLS fingerprint. The remedies, applied always:

1. **Pre-warm with a referrer chain.** Don't deep-link straight to a product page. Visit `<vendor>.com/` first, wait 3 seconds, then click into a search → results → product. Anti-bot heuristics weight `Referer` chains heavily.
2. **Add 2–5 second random delays between navigations** (`sleep $((2 + RANDOM % 4))`).
3. **Scroll the page after load.** Even one `window.scrollTo({top:600,behavior:'smooth'})` then a pause then `scrollTo({top:1400})` shifts behavioral profile out of bot territory and triggers lazy-loaded content.
4. **Move the mouse.** A few `dispatchEvent(new MouseEvent('mousemove', {clientX:..., clientY:..., bubbles:true}))` calls between clicks help.
5. **Don't fetch the same domain in tight bursts.** When backfilling 14 chips, space them ≥4 seconds apart and reload one tab — don't open 14 tabs at once.
6. **Use a single persistent profile (`chip-fetcher`)** so cookies, fingerprint, and visit history accumulate.
7. **Detection escape valve:** if a site still 403s despite all of the above, slow down and ask the user to slide whatever CAPTCHA appears.

## Always browse as a real user

**Every PDF fetch goes through pup Chrome with a real session, not naked `curl`.** Vendor CDNs (Mouser/Akamai, Cloudflare, ST/Akamai, Nordic/Cloudflare, NXP/Cloudflare) actively profile traffic for "automation tools" — TLS fingerprint, header order, missing `sec-ch-ua`/`sec-fetch-*`, no JS execution, no cookies, no referrer chain.

**The rule: every fetch flows through pup.** Use one of these in priority order:

1. **In-page `fetch()` via `browser_eval`** with `{credentials:"include"}` and `awaitPromise:true`, base64-encoding the response body back through the JSON channel.
2. **Programmatic `<a>` click in the rendered page** — Chrome handles the download natively, file lands in `~/Downloads`, then `chip-fetcher pull <path>`.
3. **Naked `curl` ONLY for vendor-doc CDNs that are confirmed not behind a bot wall** — TI's `ti.com/lit/pdf/<id>` is one of the few that works. **Always validate by checking the first 4 bytes:**
   ```bash
   head -c 4 "$OUT" | grep -q "%PDF" || { echo "not a real PDF"; rm -f "$OUT"; }
   ```
   If the magic check fails, the vendor served a denial page. Re-route through pup.

**Never write a backfill script that does `curl ... > out.pdf` without the `%PDF` magic-byte gate.** A 174KB HTML denial page mis-stored as a `.pdf` is worse than a missing file.

Direct user pushback: *"make sure we never get errors like this. meaning always browse as a real user would."*