app

Adom Chip Fetcher

Public Made by Adom

by adom

Your whole parts library — manufacturer-grade chip CAD (symbol, footprint, 3D) one tap from your EDA tool.

edacadpartscomponentspcbcli
Overview Files Skills Discover Packages Releases Dependencies Issues1 Breadcrumbs Pull Requests Activity
adom-chip-fetcher / playbooks / ultra-librarian.md
Download

Ultra Librarian (chip-fetcher playbook)

UL deep-link patterns, form quirks, captcha handling, atomic-submit pattern, watcher race-safety. Read this when the manufacturer's product page links to a vendor.ultralibrarian.com/<slug>/embedded URL.

TL;DR — three flows, pick by URL host

URL pattern Flow Captcha? Driver script Auth
vendor.ultralibrarian.com/<vendor>/embedded Embedded captcha-iframe (the public, no-login route from manufacturer product pages) reCAPTCHA gauntlet, iterative escalation cf-ul-fast.sh --phase pre-captcha/post-captcha none required, but anonymous
app.ultralibrarian.com/details/<uuid>/<vendor>/<MPN> UL Pro tick-and-click (logged-in user; URL contains ?uid=…) none for logged-in users cf-ul-pro-batch.sh <MPN> UL Pro form-login, cookie in pup profile
app.ultralibrarian.com/details/<uuid>/... (no ?uid=) UL Pro detail, anonymous yes — falls back to embedded captcha cf-ul-fast.sh (after page transition) none

Best practice 2026-05-04: sign in to UL Pro once via chip-fetcher login ultralib (see credentials-and-logins.md). Then cf-ul-pro-batch.sh drives every subsequent fetch with zero captcha — no iterative-grid AI-vision burn, no token-timeout race. Captcha-driven cf-ul-fast.sh is now a fallback for parts UL Pro doesn't carry, or for anonymous flows.

UL Pro logged-in tick-and-click flow (cf-ul-pro-batch.sh)

When the chip-fetcher pup profile has a live app.ultralibrarian.com cookie (Remember My Login was ticked at sign-in time → 30-day persistent session), UL Pro detail pages serve a direct download form instead of a captcha-iframe.

The exact two-click pattern (this is the bug-prone part)

There are TWO elements on the page both labeled "Download Now":

  • <button class="btn btn-dark..."> — opens the format-picker panel. Clicking this does NOT trigger a download.
  • <a class="export-trigger"> — actually submits the picked formats and triggers the download. Initially hidden (collapsed under the closed picker); only offsetParent !== null after the BUTTON has been clicked.

Calling the script in the right order:

  1. Click the BUTTON to open the format picker. URL gets ?open=exports.
  2. Tick the format checkboxes (Altium Designer, Fusion360 PCB, KiCAD v6+, etc.) — these are now visible in the open picker.
  3. Click the A.export-trigger to submit. UL Pro server-side processes the format conversion; the "Starting Your Download" modal appears.

A single-click pattern (just clicking whichever Download Now is visible first) fails silently — the BUTTON click opens the picker but the download never fires because the submit element wasn't clicked. Verified painful 2026-05-04: 6/8 chips timed out waiting for a download that was never submitted.

Full flow

  1. Search: https://app.ultralibrarian.com/search?queryText=<MPN> → first a[href*=details] is the part page.
  2. Navigate to part page → URL becomes https://app.ultralibrarian.com/details/<uuid>/<vendor>/<MPN>?uid=<userId>. The ?uid= confirms authenticated session — if absent, the cookie didn't take and you'll get re-routed through embedded captcha.
  3. Click <button>Download Now</button> (visible BUTTON) — opens picker.
  4. Tick checkboxes by label text: Altium Designer, Fusion360 PCB, KiCAD v6+, STEP, etc. The checkboxes share name="exports" and have an integer value identifying the format. There are ~40 export formats; pick what you need.
  5. Optionally tick any consent checkbox the manufacturer requires (TI: name="consent-TIInfoShare"). Verified not actually required for the download to succeed — UL Pro proceeds without it. But ticking it satisfies the form's UI completeness check.
  6. Click <a class="export-trigger">Download Now</a> (visible A) — submits.
  7. UL Pro shows a "Starting Your Download" modal — server-side format conversion takes 2-15 seconds typically (claims "up to 2 minutes").
  8. Poll ~/Downloads/ (or Windows Downloads via adom-desktop shell_execute powershell ...) for ul_<MPN>.zip newer than the click timestamp.
  9. Pull via chip-fetcher pull <Windows-path> --mpn <MPN> to copy zip into incoming/.
  10. Extract *.SchLib, *.PcbLib, *.lbr from the zip and copy to library/<MPN>/<MPN>.{SchLib,PcbLib,lbr}. The chip-fetcher pull step already imports .kicad_sym, .kicad_mod, .step — the manual distribute step is just for Altium + Fusion files (chip-fetcher's importer doesn't handle them yet).
./scripts/cf-ul-pro-batch.sh BQ25798RQMR
# Posts activity to /api/activity at every stage so the dashboard reflects progress.
# Output:  url, picker opened, ticked N, submitted via A, download <name>, distributed SchLib PcbLib lbr, OK <MPN>

Coverage caveats — UL Pro doesn't carry every package variant

UL Pro is per-package, not per-MPN. Common gaps (verified 2026-05-04):

  • TI ADS131M04IPBSR (TQFP-32 PBS package) — UL has IPWR/IPWT (TSSOP-32 PW package) but not the PBS variant.
  • TI MCF8316A1RRYR — not on UL Pro at all (very new part).

Don't ship a wrong-package Altium file. When the search returns a different package suffix, mark it in info.json (altium: "no UL Pro coverage for <package>") and try the manufacturer's product page (TI's "EDA Symbols and Footprints" link sometimes routes to a different UL flow that has more packages). For non-coverage parts, accept partial state — KiCad-only is better than KiCad + wrong-package Altium.

Use the fast two-phase captcha script for the embedded flow — DO NOT drive UL step-by-step from chat

chip-fetcher/scripts/cf-ul-fast.sh is the canonical UL driver. It collapses the chained-LLM-call problem (each chat-side browser_eval adds ~2-5s of latency, easily eating the captcha's 2-minute token budget) into TWO calls with one AI vision step in between. Tight bash cadences (200ms token-poll, parallel tile dispatch, 0.5s post-Verify wait) match what a Rust implementation could do; the bottleneck post-vision is the adom-desktop HTTP relay, not the script.

# Phase 1 (one shell call): navigate → preset form → trusted-click I'm-not-a-robot
#                           → if image grid: screenshot iframe + crop with DPR scaling
./scripts/cf-ul-fast.sh --phase pre-captcha --url '<UL-URL>'
# Returns JSON: { stage:"image_grid", iframe_png:"/tmp/...", round:1, _hint:"..." }
#       OR:    { stage:"silent_pass" }

# AI does ONE vision step: read iframe_png_2x, identify (a) grid size, (b) prompt, (c) tiles.

# Phase 2 (one shell call): click tiles → click Verify → check round-N escalation
#                           → atomic-eval Submit on token → poll Downloads
#                           → pull → distribute to family MPNs
./scripts/cf-ul-fast.sh --phase post-captcha --tiles 4,9 --mpns A,B,C \
    --grid 3x3 --round 1
# Returns: { stage:"done", ul_zip:..., distributed_to:["A","B","C"] }
#  OR (iterative captcha): { stage:"image_grid_round_2", iframe_png:..., _hint:"..." }
#  → AI does another vision pass, calls phase 2 again with --round 2.

The script handles family-distribution: when one UL zip covers multiple MPNs (common for indicator passives where the package body is shared), pass --mpns A,B,C and the script copies sym/mod into each library/<MPN>/.

UL rotates captcha approaches — adapt, don't hardcode

UL deliberately varies captcha to throw off AI vision:

  • Grid size: 3×3 (9 tiles, 1.5× larger per tile) vs 4×4 (16 tiles, denser). AI MUST detect from screenshot and pass --grid 3x3 or --grid 4x4 to phase 2.
  • Type: "Select all images with X" (one-shot — click matching tiles, click Verify, done) vs "Click verify once there are none left" (iterative — click matches, then NEW images appear in clicked tiles, may need multiple rounds).
  • Subject difficulty: easy (cars, fire hydrants, traffic lights) vs harder (boats, bicycles, chimneys, palm trees) — AI may need to be aggressive about ambiguous tiles.
  • Image noise overlay: some replacement tiles in iterative mode have heavy noise — usually means "newly-shown after a click; classify based on what's visible through the noise."

cf-ul-fast.sh --phase post-captcha detects iterative escalation automatically (post-Verify check: bframe still visible AND token still empty → re-screenshot → output image_grid_round_N+1 for AI to do another vision pass). AI loops until token populates or captcha hard-fails.

/compact your conversation BEFORE driving UL

A captcha token has a ~120-second life from the I'm-not-a-robot click. AI vision response time scales with conversation context size. If your context is large (200K+ tokens), each vision turn can take 8-15s — that compounds across iterative captcha rounds and burns the token budget.

Before any non-trivial UL fetch, recommend the user run /compact to trim context. Smaller context = faster vision responses = the captcha pass succeeds before the token expires.

Heartbeat protocol is mandatory — see dashboard-flow.md. cf-ul-fast.sh calls post_act automatically at every stage so the dashboard reflects live progress.

DO NOT drive UL with separate browser_eval chat-side calls

Anti-pattern (the slow path):

[chat] browser_navigate UL URL
[chat] browser_eval — open Choose CAD Formats
[chat] browser_eval — preset form
[chat] browser_input_dispatch — click captcha
[chat] browser_screenshot — see image grid
[chat] AI vision — pick tiles
[chat] browser_input_dispatch — click tile 1
[chat] browser_input_dispatch — click tile 2
[chat] browser_input_dispatch — click Verify
[chat] browser_eval — poll for token (×N)
[chat] browser_eval — Submit
[chat] shell_execute — poll Downloads
[chat] chip-fetcher pull

Each line carries 2-5s of LLM/network latency. By the time Submit fires, the token is dead.

The right path: cf-ul-fast.sh --phase pre-captcha, AI vision, cf-ul-fast.sh --phase post-captcha. Two CLI calls, one vision step.

Auto-driving pattern (the win)

chip-fetcher/scripts/cf-ul-fetch.sh <MPN> <UL-deep-link> is the canonical end-to-end driver. It implements the atomic-submit pattern verified working across TI/ST/ADI/WAGO/Microchip flows.

chip-fetcher/scripts/cf-drive.sh <MPN> opens the TI product page, finds the embedded UL link (carries session-tied URL pre-loaded with right gpn + package + pin), navigates to it, clicks "Choose CAD Formats & Download", then sets the form via JS:

document.getElementById("MfrThreeDModel").click();   // STEP
document.getElementById("KiCADv6").click();          // KiCAD v6+
document.getElementById("TermsAndConditions").click();
const sel = document.querySelectorAll("select").find(s => /Metric/i.test(s.options[1].text));
sel.value = "2-2"; sel.dispatchEvent(new Event("change",{bubbles:true}));  // mm

chip-fetcher/scripts/cf-autowatch.sh <MPN> (background) does two things:

  1. Polls g-recaptcha-response token every 2s. When length > 50, the user has solved CAPTCHA. Verifies the active tab URL still matches the MPN's base part (race-safety) before auto-removing disabled from #SubmitLink and clicking it.
  2. Polls user's ~/Downloads every 2s. When a ul_*.zip whose filename matches the MPN base appears, calls adom-desktop pull_file + chip-fetcher import --mpn <MPN>.

A PID lock at /tmp/cf-autowatch.lock prevents two watchers running simultaneously.

The user only has to click one CAPTCHA per part. Everything else is automated.

Typical batch flow

cd /home/adom/project/chip-fetcher
./scripts/cf-drive.sh <MPN>
./scripts/cf-autowatch.sh <MPN> &
adom-desktop browser_raise_os_window '{"sessionId":"chip-fetcher"}'
# → user clicks CAPTCHA on the pup window (one click)
# → watcher auto-clicks Submit, polls Downloads, pulls + imports

While the watcher's running, curl https://www.ti.com/lit/gpn/<base-mpn> to grab the datasheet PDF in parallel — that endpoint is open, no auth.

Vendor-specific UL form quirks

Different UL portals use different checkbox IDs:

Portal STEP checkbox ID KiCAD v6+ ID
TI MfrThreeDModel KiCADv6
Microchip MfrThreeDModel KiCADv6
ST MfrThreeDModel KiCADv6
ADI ThreeDModel KiCADv6

Don't rely on IDs — use the visible label "STEP" / "KiCAD v6+" with byLabel(re => /^STEP$/i) lookup. The cf-drive.sh script already does this.

On every UL download form, check the popular-EDA format checkboxes — not just KiCAD. The user's current project may target one EDA tool, but Adom adds support for new EDA tools over time, and re-fetching every board for a different EDA tool would mean another reCAPTCHA toll per part. UL also caps free downloads at ~5 format selections per submission, so picking smartly matters — going for "ALL formats" doesn't work.

The canonical four-pick that covers >95% of user EDA tooling:

  • ☑ STEP (3D model — always, every download)
  • ☑ KiCAD v6+
  • ☑ Altium Designer
  • ☑ Fusion 360 PCB / Eagle
  • ☑ T&C checkbox (mandatory)

Direct user direction: "i think the names i gave you are the right choice for now. we don't need ALL formats of other EDA's. just the popular ones … fusion 360 and altium and kicad."

If the user later expands the supported EDA list (OrCAD, DesignSpark, Mentor, Pulsonix, Quadcept, TARGET, Zuken, CADSTAR, eCadstar — each is a separate UL checkbox), add to this list, but stay within UL's per-submission cap. For chips that need an EDA format outside the canonical four, do a second reCAPTCHA-gated fetch later (worth the toll).

Footprint Units: always set to Metric (mm) when the dropdown exists. Most EDA tools internally use mm; importing as mil and converting is a precision-loss tax.

STEP picker: if the page shows a "Cadenas 3D Model" dropdown (WAGO-style UL forms), pick STEP AP214 (universal compatibility) or STEP AP242 (newer, with PMI). If STEP is a checkbox (TI-style UL forms), just check it.

# Driver helper: pick the canonical four + T&C (no overflow)
adom-desktop browser_eval '{"sessionId":"chip-fetcher","expr":"(()=>{const cbs=Array.from(document.querySelectorAll(\"input[type=checkbox]\")); const want=[/^STEP$/i,/kicad v6/i,/^altium designer/i,/^fusion.?360/i,/agree|terms/i]; for(const re of want){const c=cbs.find(c=>re.test((c.parentElement?.textContent||c.closest(\"label\")?.textContent||\"\").trim())); if(c&&!c.checked) c.click();} return \"done\";})()"}'

If UL silently unchecks one (the "max 5" enforcement), the canonical four already fits — but if a vendor adds extra mandatory checkboxes (e.g. WAGO had Cadenas 3D Model dropdown which counts toward the cap), drop one of the EDA formats (typically Eagle/Fusion last) to stay under.

reCAPTCHA v2 auto-click — try first, fall back to user only on image-grid

With browser_input_dispatch (adom-desktop ≥ v1.4.7), the agent CAN dispatch a trusted CDP click on the reCAPTCHA "I'm not a robot" checkbox iframe — isTrusted is no longer the limiter. What stops a clean silent pass is Google's session-reputation scoring: pup's Chrome-for-Testing has no browsing-history, no Google account session, no consistent IP usage pattern, so reCAPTCHA escalates to the image-grid challenge for low-reputation sessions.

The empirical pattern (verified 2026-05-03 on TI UL):

  1. Move cursor toward the checkbox via browser_input_dispatch type:"move" with intermediate steps (helps fingerprint look human).
  2. Click the checkbox at iframe (left+30, top+40) — that's where "I'm not a robot" sits inside the iframe.
  3. Wait 3–5s.
  4. Read [name=g-recaptcha-response]'s value.
    • Non-empty + >20 chars → silent pass, proceed to Submit.
    • Empty + an image-challenge iframe (iframe[src*=bframe]) appears → escalation, hand off to user.
# 1. Find iframe coords
RECT=$(adom-desktop browser_eval '{"sessionId":"X","expr":"(()=>{const f=document.querySelector(\"iframe[src*=recaptcha]\"); if(!f)return null; const r=f.getBoundingClientRect(); return JSON.stringify({x:r.x|0,y:r.y|0});})()"}' \
  | jq -r '.result')

# 2. Move + click (humanlike approach)
IX=$(($(echo $RECT | jq .x) + 30))
IY=$(($(echo $RECT | jq .y) + 40))
adom-desktop browser_input_dispatch "{\"sessionId\":\"X\",\"type\":\"move\",\"x\":$((IX-200)),\"y\":$((IY-100)),\"steps\":5}"
sleep 0.5
adom-desktop browser_input_dispatch "{\"sessionId\":\"X\",\"type\":\"move\",\"x\":$IX,\"y\":$IY,\"steps\":15}"
sleep 0.4
adom-desktop browser_input_dispatch "{\"sessionId\":\"X\",\"type\":\"click\",\"x\":$IX,\"y\":$IY}"
sleep 4

# 3. Detect outcome: passed vs escalated
RESULT=$(adom-desktop browser_eval '{"sessionId":"X","expr":"JSON.stringify({tokenLen:(document.querySelector(\"[name=g-recaptcha-response]\")?.value||\"\").length, challenge:!!document.querySelector(\"iframe[src*=bframe]\")})"}')
# tokenLen >= 20 → passed, agent continues
# tokenLen == 0 && challenge true → image grid, AI vision can try, else user

If escalated to image grid, AI vision CAN often solve it. Verified working 2026-05-03 on TI UL: solved a 4×4 "traffic lights" grid + 3×3 "motorcycles" grid in two passes, token populated cleanly. Technique:

  1. Screenshot the iframe region. The bframe iframe's bounding rect is exposed via document.querySelector("iframe[src*=bframe]").getBoundingClientRect(). Crop to (rect.x * dpr, rect.y * dpr, rect.w * dpr, rect.h * dpr) where dpr = window.devicePixelRatio (typically 1.5).
  2. Resize the crop 2× via ImageMagick (convert input -resize 200% out.png).
  3. Read the prompt + grid. "Select all squares with X" = 4×4 grid (16 tiles, click each square containing any pixel). "Select all images with X" = 3×3 grid (9 tiles, only the primary subject).
  4. Compute tile centers in browser CSS coords. For 4×4 grid in iframe (273,127,400,580): grid area top≈237 left≈298, tile size ≈89×105 → tile (r,c) center = (298 + (c-0.5)*89, 237 + (r-0.5)*105). For 3×3 grid: tile size ≈118×140.
  5. Click each correct tile via browser_input_dispatch type:click with ~600ms between clicks (humanlike pacing).
  6. Click Verify at iframe-bottom-right.
  7. Watch [name=g-recaptcha-response] — token populating with >20 chars = passed.

Token expires within ~2 minutes — Submit IMMEDIATELY after token populates. Do NOT re-set form, do NOT add sleep 8 between checks; that latency burns through the captcha lifetime and you'll have to solve it again.

CRITICAL pattern — ATOMIC token-check + Submit-click in one browser_eval

The winning pattern for UL captcha submits, verified end-to-end on TPS25751 2026-05-03. Every prior attempt failed because the bash-side polling was eval → check → eval with 200–500ms round-trip between detection and Submit, which was enough for UL's backend to re-validate and reject the captcha token.

Fix: do the check AND the click in a single browser_eval.

(() => {
  const t = (document.querySelector('[name=g-recaptcha-response]')?.value || '');
  if (t.length > 20) {
    const a = Array.from(document.querySelectorAll('a, button, input[type=submit]'))
      .find(e => /^submit$/i.test((e.textContent || e.value || '').trim()));
    if (a) { a.click(); return 'SUBMITTED_' + a.tagName; }
    return 'NO_BTN';
  }
  return 'WAIT';
})()

Bash polls this one eval at 1-second cadence — when it returns SUBMITTED_* you know Submit fired in the same browser tick the token was detected. Net round-trip latency from token-populate to Submit-click: ~0 ms.

Other prerequisites that go with the atomic eval:

  1. Pre-set the form BEFORE the captcha (STEP + KiCAD v6+ + Altium + Fusion 360 + mm + T&C all in one eval). Zero work between token-detect and Submit.
  2. UL's Submit is an <a> tag, not <button> — selector must scan 'a, button, input[type=submit]'.
  3. Cache the Downloads-before snapshot BEFORE clicking reCAPTCHA, not after Submit — else you race the download and miss the new file.
  4. Use the manufacturer's session-tied UL deep-link, not ?vdrPN= guesses. TI's product page has a CAD link that includes gpn=<MPN>&package=<PKG>&pin=<N>&sid=<token>. Without sid, UL silently routes to a wrong/related part. Verified: ?vdrPN=tps65987d routed to F280041RSHSR (totally different chip).
  5. Verify part identity after navigation — read document.body.textContent and confirm the MPN appears.

The full winning recipe lives in chip-fetcher/scripts/cf-ul-fetch.sh. Direct user reaction when this pattern landed the bundle on first attempt: "you did it!!!!!! put that in the skill and/or make a script to be ready next time."

Even tighter timing for AI-vision-solved image grids

Verified 2026-05-03: even when AI vision passes a multi-stage image-grid challenge (4×4 → 3×3 → 4×4 → 3×3, ~3 challenges deep), the cumulative time spent on screenshot + ImageMagick crop + Read + click cycles can exceed the 2-minute token window by the time the FINAL Submit posts to the vendor backend.

Mitigation when going AI-vision-solve route:

  1. Pre-set the form completely before triggering reCAPTCHA. STEP, KiCAD v6+, Altium, Fusion 360, Metric (mm), T&C — all set BEFORE you click "I'm not a robot."
  2. Keep ALL between-click latency under 1s. No sleep 4 for screenshots — use sleep 0.6 between tile clicks.
  3. Skip screenshots after Verify. Read [name=g-recaptcha-response] directly — that's the only signal that matters. If .length > 20 → fire Submit in the SAME bash chain.
  4. Accept that 3+ challenge stages probably exceed the budget. Plan for a redo, but expect the user to need to step in if Google keeps escalating. Direct user pushback on slow re-tries: "why are you going so slow again? it'll timeout!!!"

The hard fact: AI vision works for 1-stage image grids (~70% of trusted-click escalations). For 3+-stage chained challenges, the round-trip time of screenshot → crop → read → identify → click → verify is the bottleneck, not vision accuracy. For those, hand off to the user immediately.

Reputation-building tips for boosting silent-pass rate over time:

  • Have the user sign into Google in the pup profile once.
  • Visit a few high-reputation domains (google.com search, gmail.com) before the gated form.
  • Use the same chip-fetcher profile across all sessions so cookies+history accumulate.

In practice, expect ~30–50% silent-pass rate on first attempts and increasing over time.

Watcher race-safety (as of 2026-04-30)

scripts/cf-autowatch.sh enforces:

  • PID lock at /tmp/cf-autowatch.lock — only one watcher at a time.
  • URL match — verifies the active pup tab URL contains the expected MPN base before clicking Submit.
  • Filename match — the Downloads-folder poll filters for filenames containing the MPN base, AND filename matches ^(ul_|LIB_).*\.zip$.

Past incident: two watchers running simultaneously both saw the same CAPTCHA solve token and both raced to download/import the same UL zip — one ended up imported into the wrong MPN folder.

UL gotchas

  1. UL download tokens are SINGLE-USE. After Chrome consumes the URL once, server returns Token Expired. Re-fetching via browser_eval + fetch() will not work. The only reliable path is pull_file from the user's Downloads folder.
  2. UL ships PACKAGE-named files inside the zip, not MPN-named. Inside ul_TPS62840YBGR.zip the STEP is DSBGA6_1P4XP9_TEX.step. Always pass --mpn <MPN> to import so files get renamed to <MPN>.step.
  3. UL ships multiple footprint variants (default, -L, -M). Current import overwrites them all to one <MPN>.kicad_mod. TODO: disambiguate suffixes.
  4. TI does NOT self-host STEPs on ti.com. Every TI part's "CAD/CAE Symbols" link is a UL deep-link with a session-tied URL.