Web Serial HMI + USB CDC Firmware (Octolux ↔ RP-series)

{
  "schema_version": 1,
  "slug": "octolux-web-serial-hmi",
  "type": "skill",
  "version": "1.0.0",
  "title": "Web Serial HMI + USB CDC Firmware (Octolux ↔ RP-series)",
  "brief": "Build a paired NTX HMI (Chromium kiosk on Octolux) + firmware (USB CDC on RP2040/RP2350) that auto-connects over USB with no permission picker. End-to-end recipe: Pico SDK firmware template, HMI Web Serial via getPorts() + VID filter, Chromium SerialAllowAllPortsForUrls policy, udev MODE=0666 for /dev/ttyACM*, SCP + chromium-restart + screencast-bounce deploy, CDP verification, visible-debug-strip diagnostics, and a 12-row failure-mode table.",
  "content": "## Known errors this skill resolves\n\nUntil the underlying bugs ship a fix, this skill is the reference for diagnosing and working around the items below. Strike a line off this list when the upstream fix lands.\n\n| | Error / symptom | Where |\n|---|---|---|\n| ☐ | Web Serial picker shows *\"No compatible devices found\"* even though `lsusb` lists the USB CDC device | Symptom #1 |\n| ☐ | `navigator.serial.requestPort()` rejects with `NotFoundError: No port selected by the user` every session | Symptom #2 |\n| ☐ | `navigator.serial.getPorts()` returns ports but `port.open()` throws *\"Failed to execute 'open' on 'SerialPort': Failed to open serial port\"* | Symptom #3 |\n| ☐ | `getPorts().length === 0` even with the policy installed and the device on the kernel bus | Symptom #4 |\n| ☐ | Tap on the Connect button does nothing visible — and you can't see why, because Chromium permission dialogs don't render in the VNC stream | Diagnostics |\n| ☐ | `octolux deploy push` times out (the SCP + chromium-restart + screencast-bounce shell sequence is more reliable) | Deploy |\n| ☐ | `ps aux \\| grep screencast` returns nothing on the device, but screencast IS running on port 8085 (busybox truncates cmdlines) | Deploy |\n| ☐ | Walked `/proc/[0-9]*/net/tcp` to find the screencast PID, `xargs kill` then killed system PIDs and lost SSH | NOT-TO-DO |\n| ☐ | `/tmp/screencast.js` or `/tmp/node_modules/ws/` missing after a bad kill on the device — needs SCP restore from the container repo | Deploy |\n| ☐ | Dashboard VNC tile freezes on a pre-deploy frame after any Chromium restart (screencast's CDP binding to the dead target) | hmi-deploy-reset-vnc |\n| ☐ | `error code: 502` no-body from the device's dashboard endpoint (Adom HTTPS proxy strips 5xx bodies — return 200 + `{ok:false}` instead) | cross-ref [octolux-app-server-debug] |\n\n\n---\nname: web-serial-hmi\ndescription: >\n  Build a paired NTX HMI (Chromium kiosk HTML on an Octolux device) +\n  firmware (USB CDC on an RP2040 / RP2350) that auto-connects over USB\n  with no permission picker. Covers the whole loop: project layout under\n  `~/ntx-embedded/customers/<id>/`, Pico SDK USB CDC firmware template\n  with line-based ASCII protocol, HMI-side Web Serial using `getPorts()`\n  + VID filter + auto-connect polling, the Chromium enterprise policy\n  `SerialAllowAllPortsForUrls` (the key enabler — without it\n  `requestPort()` always shows a picker), the udev rule that makes\n  `/dev/ttyACM*` openable by the kiosk's Chromium user, the SCP +\n  chromium-restart + screencast-bounce deploy sequence, a visible\n  page-side debug strip so you can diagnose what Chromium UI dialogs\n  (which don't render in VNC) are doing, and a CDP-via-SSH-tunnel\n  pattern for live page inspection. Includes a working firmware +\n  HMI template (~80 lines of C, ~80 lines of JS) plus a failure-mode\n  reference table. Scope: NTX Octolux devices + RP-series targets.\n  Trigger words: build hmi firmware pair, hmi over usb, web serial\n  hmi, web serial kiosk, octolux web serial, chromium serial port,\n  Web Serial no devices, navigator.serial.getPorts, requestPort vs\n  getPorts, SerialAllowAllPortsForUrls, DefaultSerialGuardSetting,\n  no port selected by user, no compatible devices found, web serial\n  picker empty, auto-connect serial port, rp2040 usb cdc, rp2350 usb\n  cdc, pico_enable_stdio_usb, kiosk udev ttyACM, /dev/ttyACM\n  permissions, chromium policy serial, octolux device deploy reset\n  vnc, visible debug strip, CDP page reload screencast stale, port\n  open failed serial, ntx hmi firmware template, new customer hmi\n  project, ports[0] open fails.\n---\n\n# web-serial-hmi — NTX HMI ↔ USB CDC firmware pattern\n\n> **Scope.** Building a Chromium-kiosk HMI on an **NTX Octolux device** that auto-connects to a **USB-attached RP2040 or RP2350** target over CDC-ACM, with **no Web Serial picker** ever showing. End-to-end working pattern, not a per-customer recipe — instantiate it for any new project under `~/ntx-embedded/customers/<id>/`.\n>\n> Out of scope: Android USB Host (different stack); Web Bluetooth / WebHID; non-RP-series USB devices (mostly the same pattern, just substitute the VID); CAN-FD bridges; the SWD/flash side of the firmware lifecycle — that's covered by `rp2350-flash` and the `target/rp2040.cfg` openocd patterns and the local `firmware/` build is just plain `cmake + make`.\n\n## When to use this skill\n\n- Starting a new customer HMI project that needs to control hardware over a USB-attached MCU.\n- Porting an Android-tablet HMI app to an Octolux Linux/Chromium device and you want the same \"just plug it in\" feel.\n- The Web Serial picker is showing — *\"No compatible devices found\"*, *\"No port selected by user\"*, or just appearing at all — and the project shouldn't have a picker.\n- `requestPort()` works once but the operator has to tap-and-pick every session and that's wrong for a kiosk.\n- The HMI page can't see what's happening because Chromium permission dialogs don't render in VNC.\n\n## Architecture\n\n```\n┌─────────────────────────────────┐      USB-C       ┌──────────────────────┐\n│ NTX Octolux device (10.0.10.x)  │ ◄──────────────► │ RP2040 / RP2350 board│\n│ ─────────────────────────────── │ CDC-ACM, 115200  │ ───────────────────  │\n│ Chromium 109 kiosk @ localhost  │   8N1 line ASCII │ Pico SDK USB CDC     │\n│  ├─ Web Serial API              │                  │  ├─ getchar polling  │\n│  ├─ SerialAllowAllPortsForUrls  │                  │  ├─ ASCII line parser│\n│  └─ getPorts() filtered by VID  │                  │  └─ GPIO drive       │\n│ Apache at /                     │                  │                      │\n│ /dev/ttyACM0 (root:root, 0666)  │                  │                      │\n└─────────────────────────────────┘                  └──────────────────────┘\n        ▲                                                       ▲\n        │  SSH from container (deploy + udev + policy)          │  SWD via PiProbe (flash)\n        │                                                       │\n   Adom Docker container                                  see rp2350-flash\n   ~/ntx-embedded/customers/<id>/{hmi,firmware}\n```\n\nTwo physical links from the workcell to the target board:\n\n- **USB-C** between Octolux and target — runtime command path (this skill).\n- **SWD** between the workcell's PiProbe and the target — flash path (`rp2350-flash`).\n\nThe container is the build host for both: HMI HTML edits go SCP-ed to Octolux; firmware ELF is built locally and flashed via the PiProbe.\n\n## Required components\n\nA working NTX HMI+firmware pair has **five** moving parts. Skip any one and connect won't happen. Pieces 3 and 4 are one-time setup *per Octolux device*; pieces 1, 2, 5 are per-project.\n\n| # | Layer | Where it lives | Per-… |\n|---|---|---|---|\n| 1 | **Firmware** — USB CDC stdio, command parser, GPIO drive | `customers/<id>/firmware/src/main.c` (in container) | project |\n| 2 | **HMI HTML** — connect logic + UI | `customers/<id>/hmi/index.html` (in container, deployed to device) | project |\n| 3 | **Chromium policy** — `SerialAllowAllPortsForUrls` | `/etc/chromium/policies/managed/octolux-serial.json` (on device) | device |\n| 4 | **udev rule** — `MODE=\"0666\"` for VID 0x2e8a | `/etc/udev/rules.d/99-rp-cdc-acm.rules` (on device) | device |\n| 5 | **Deploy** — SCP HMI + chromium restart + screencast bounce | shell pattern run from container | project |\n\n## Quick start — new project, end to end\n\nRun from the Adom container. Substitute `<id>` (e.g. `acme-control`), `<device-ip>` (e.g. `10.0.10.98`), and your domain commands.\n\n### 1. Scaffold\n\n```bash\nID=acme-control\nDEVICE_IP=10.0.10.98\nDEVICE_MODEL='Octolux 15.6\"'\nVIEW_W=1920 VIEW_H=1080\n\nmkdir -p ~/ntx-embedded/customers/$ID/{hmi,firmware/src}\n\ncat > ~/ntx-embedded/customers/$ID/project.json <<EOF\n{\n  \"id\": \"$ID\",\n  \"name\": \"Acme Control\",\n  \"customer\": \"Acme\",\n  \"summary\": \"One-line description of what this HMI does.\",\n  \"accent\": \"#e87a1e\",\n  \"accentDeep\": \"#a64f10\",\n  \"deviceModel\": \"$DEVICE_MODEL\",\n  \"viewport\": { \"w\": $VIEW_W, \"h\": $VIEW_H },\n  \"orientation\": \"landscape\",\n  \"defaultIP\": \"$DEVICE_IP\",\n  \"tagline\": \"one-line operator-facing tagline\"\n}\nEOF\n```\n\n### 2. Drop in the firmware + HMI templates\n\nUse the templates further down this doc verbatim, customizing only:\n\n- **Firmware:** `FW_VERSION_STRING` (the `<NAME> FW x.y.z` boot banner), the `dispatch()` command list, and your specific GPIO pin numbers / domain logic.\n- **HMI:** the visible controls (buttons, sliders, etc.), the brand colors from your `project.json`, the response-line handlers in `handleResponse()`.\n\nKeep the Web Serial connect logic and the visible debug strip **unchanged** — that's the point of this skill.\n\n### 3. Build + flash the firmware (first time)\n\n```bash\nexport PICO_SDK_PATH=$HOME/pico-sdk\ncd ~/ntx-embedded/customers/$ID/firmware\nmkdir -p build && cd build && cmake .. && make -j$(nproc)\n\n# Flash via PiProbe over SWD — pick the cfg for your chip:\nopenocd -f interface/cmsis-dap.cfg -f target/rp2040.cfg \\\n  -c \"adapter speed 5000; program ${ID//-/_}.elf verify reset exit\"\n# (or target/rp2350.cfg for an RP2350 — see rp2350-flash)\n```\n\n### 4. One-time device setup (Chromium policy + udev rule)\n\nSkip if this Octolux device has been set up for Web Serial before. Verify by SSH'ing in: if `/etc/chromium/policies/managed/octolux-serial.json` and `/etc/udev/rules.d/99-rp-cdc-acm.rules` both exist, you're done.\n\n```bash\nssh root@$DEVICE_IP 'bash -s' <<'REMOTE'\nset -e\n# Chromium policy: SerialAllowAllPortsForUrls turns getPorts() into a\n# blanket enumeration for http://localhost; DefaultSerialGuardSetting=2\n# also disables requestPort() pickers as a safety net.\nmkdir -p /etc/chromium/policies/managed\ncat > /etc/chromium/policies/managed/octolux-serial.json <<'POL'\n{\n  \"SerialAllowAllPortsForUrls\": [\"http://localhost\"],\n  \"DefaultSerialGuardSetting\": 2\n}\nPOL\n\n# udev: /dev/ttyACM* is root-only by default on the Yocto distro, and\n# the kiosk Chromium can't open it. MODE=0666 fixes that for any\n# Raspberry Pi (RP2040/RP2350) USB CDC device.\ncat > /etc/udev/rules.d/99-rp-cdc-acm.rules <<'UDEV'\nSUBSYSTEM==\"tty\", ATTRS{idVendor}==\"2e8a\", MODE=\"0666\"\nUDEV\nudevadm control --reload-rules\n\n# Re-process any ALREADY-plugged-in device so the new rule applies now.\n# (Without this, the perms only apply to *future* plug events.)\nBUSPORT=$(lsusb -t 2>/dev/null | awk '/Pico|2e8a/ {sub(/^[ \\t]+/, \"\"); print}' | head -1)\necho \"USB device tree row: ${BUSPORT:-(none seen)}\"\n# Cycle the parent USB device (substitute bus-port from `lsusb -t` if your\n# topology differs; 4-1 is the common Octolux internal hub port).\necho 4-1 > /sys/bus/usb/drivers/usb/unbind 2>/dev/null || true\nsleep 1\necho 4-1 > /sys/bus/usb/drivers/usb/bind 2>/dev/null || true\n\nsystemctl restart octolux-chromium\nREMOTE\n\n# Per the [[hmi-deploy-reset-vnc]] rule: chromium restart leaves the\n# screencast bound to a dead CDP target. Bounce the screencast on the\n# device — see the Deploy sequence below for the exact commands.\n```\n\n### 5. Deploy the HMI to the device\n\nSee the \"Standard deploy sequence\" section below — it's the same shell pattern you'll run on every HMI edit.\n\n### 6. Verify it auto-connected (CDP eval from container)\n\n```bash\nssh -fN -L 9222:localhost:9222 -o ExitOnForwardFailure=yes root@$DEVICE_IP\n\npython3 - <<'PY'\nimport json, urllib.request, websocket\nt = json.loads(urllib.request.urlopen(\"http://localhost:9222/json/list\").read())\npage = next(x for x in t if x.get(\"type\") == \"page\")\nws = websocket.create_connection(page[\"webSocketDebuggerUrl\"])\ndef evx(expr, await_=False):\n    ws.send(json.dumps({\"id\": 1, \"method\": \"Runtime.evaluate\",\n        \"params\": {\"expression\": expr, \"returnByValue\": True,\n                   \"awaitPromise\": await_}}))\n    while True:\n        r = json.loads(ws.recv())\n        if r.get(\"id\") == 1: return r[\"result\"][\"result\"][\"value\"]\nprint(\"connected:\", evx(\"state.connected\"))\nprint(\"fwVersion:\", evx(\"state.fwVersion\"))\nprint(\"ports    :\", evx(\"navigator.serial.getPorts().then(p => p.length)\", await_=True))\nPY\n```\n\nExpected once the firmware is running and the cable is plugged in:\n\n```\nconnected: True\nfwVersion: 1.0.0\nports    : 9          # 8 platform UARTs + your 1 RP-series CDC\n```\n\nIf `connected: True` and `fwVersion` is populated → you're done; the HMI is talking to the MCU. If not, jump to the failure table.\n\n## Firmware template (RP2040 / RP2350 USB CDC)\n\n`firmware/CMakeLists.txt`:\n\n```cmake\ncmake_minimum_required(VERSION 3.13)\nset(PICO_BOARD pico CACHE STRING \"\")            # or \"pico2\" for RP2350\nset(PICO_PLATFORM rp2040 CACHE STRING \"\")       # or \"rp2350\"\ninclude($ENV{PICO_SDK_PATH}/external/pico_sdk_import.cmake)\nproject(<id_underscored> C CXX ASM)\npico_sdk_init()\nadd_executable(<id_underscored> src/main.c)\ntarget_link_libraries(<id_underscored> pico_stdlib hardware_gpio)\npico_enable_stdio_usb(<id_underscored>  1)      # CDC-ACM on USB-C\npico_enable_stdio_uart(<id_underscored> 0)      # no UART — USB only\npico_add_extra_outputs(<id_underscored>)\n```\n\n`firmware/src/main.c`:\n\n```c\n#include <stdio.h>\n#include <string.h>\n#include <ctype.h>\n#include <stdarg.h>\n#include \"pico/stdlib.h\"\n#include \"pico/stdio_usb.h\"      // stdio_usb_connected()\n#include \"hardware/gpio.h\"\n\n#define FW_VERSION_STRING \"<NAME> FW 1.0.0\"\n#define CMD_BUF_SIZE 96\n\n// Status LEDs (wire your board's status LEDs to these or change as needed).\n// LED_USB: solid ON while USB CDC is connected to a host.\n// LED_ACT: 50 ms blink on every valid command.\n// LED_ERR: solid ON when the last response was ERR; clears on next OK.\n// LED_IDLE: solid ON when system is in its safe / inactive state.\n#define LED_USB   9\n#define LED_ACT  10\n#define LED_ERR  16\n#define LED_IDLE 18\n\nstatic char cmd_buf[CMD_BUF_SIZE];\nstatic int  cmd_len = 0;\nstatic bool act_on = false;\nstatic absolute_time_t act_off_at;\n\nstatic void blink_activity(void) {\n    gpio_put(LED_ACT, 1);\n    act_on = true;\n    act_off_at = make_timeout_time_ms(50);\n}\nstatic void say_ok(const char *fmt, ...) {\n    va_list a; va_start(a, fmt);\n    fputs(\"OK \", stdout); vprintf(fmt, a); fputs(\"\\r\\n\", stdout);\n    va_end(a);\n    gpio_put(LED_ERR, 0);\n    blink_activity();\n}\nstatic void say_err(const char *m) {\n    printf(\"ERR %s\\r\\n\", m);\n    gpio_put(LED_ERR, 1);\n    blink_activity();\n}\nstatic void to_upper(char *s) { for (; *s; s++) *s = (char)toupper((unsigned char)*s); }\n\n// === Domain commands ====================================================\n//\n// Replace these with the commands your project actually needs. Keep\n// the canonical built-ins (PING, VERSION, STATUS, HELP) — the HMI\n// template's handshake assumes they exist.\n//\nstatic void dispatch(char *line) {\n    char *tok = strtok(line, \" \\t\");\n    if (!tok) return;\n\n    if (!strcmp(tok, \"PING\"))    { say_ok(\"PONG\"); return; }\n    if (!strcmp(tok, \"VERSION\")) { printf(\"%s\\r\\n\", FW_VERSION_STRING); blink_activity(); return; }\n    if (!strcmp(tok, \"STATUS\"))  { /* print one-line state snapshot */ blink_activity(); return; }\n    if (!strcmp(tok, \"HELP\"))    { /* print \"OK …\" lines */ blink_activity(); return; }\n\n    // your-domain command parsing here, e.g.:\n    //   if (!strcmp(tok, \"DO\")) { … say_ok(\"DO %d\", arg); return; }\n\n    say_err(\"unknown command\");\n}\n\n// === Boilerplate main =================================================\n\nint main(void) {\n    stdio_init_all();\n\n    // Init your GPIOs here (LEDs + outputs). Drive outputs LOW first\n    // so the system boots in a safe state regardless of host behavior.\n    const uint leds[] = { LED_USB, LED_ACT, LED_ERR, LED_IDLE };\n    for (size_t i = 0; i < sizeof(leds)/sizeof(leds[0]); i++) {\n        gpio_init(leds[i]); gpio_set_dir(leds[i], GPIO_OUT); gpio_put(leds[i], 0);\n    }\n    gpio_put(LED_IDLE, 1);\n\n    bool was_conn = false, banner_done = false;\n\n    while (true) {\n        bool conn = stdio_usb_connected();\n        gpio_put(LED_USB, conn ? 1 : 0);\n        if (conn && !was_conn) banner_done = false;\n        was_conn = conn;\n\n        // Boot banner on every CDC connect — the HMI parses this to\n        // populate the FW version pill.\n        if (conn && !banner_done) {\n            sleep_ms(50);                     // give the host's reader a beat\n            printf(\"%s\\r\\n\", FW_VERSION_STRING);\n            puts(\"TYPE 'HELP' FOR COMMANDS\");\n            banner_done = true;\n        }\n\n        // Drain available input into cmd_buf; dispatch on EOL.\n        for (int budget = 0; budget < 32; budget++) {\n            int c = getchar_timeout_us(0);\n            if (c == PICO_ERROR_TIMEOUT) break;\n            if (c == '\\n' || c == '\\r') {\n                if (cmd_len > 0) {\n                    cmd_buf[cmd_len] = '\\0';\n                    to_upper(cmd_buf);\n                    dispatch(cmd_buf);\n                    cmd_len = 0;\n                }\n            } else if (cmd_len < CMD_BUF_SIZE - 1) {\n                cmd_buf[cmd_len++] = (char)c;\n            } else { cmd_len = 0; say_err(\"command too long\"); }\n        }\n\n        // Activity LED timeout.\n        if (act_on && absolute_time_diff_us(get_absolute_time(), act_off_at) <= 0) {\n            gpio_put(LED_ACT, 0); act_on = false;\n        }\n    }\n}\n```\n\n### Protocol conventions that work well\n\n- **Newline-terminated ASCII** — accept `\\n` and `\\r\\n`. Case-insensitive (uppercase before dispatch).\n- **One line in, exactly one line out** — `OK …`, `ERR …`, or a single-token status line. One-to-one keeps the HMI's send/receive trivial.\n- **Standard built-ins always present:**\n  - `PING → PONG` (liveness)\n  - `VERSION → <NAME> FW x.y.z` (capability detection + version pill)\n  - `STATUS → STATE …` (one-line state snapshot)\n  - `HELP → multi-line OK …` (operator self-discovery)\n- **Boot banner on every CDC connect.** Print `<NAME> FW x.y.z\\r\\n` first thing after `stdio_usb_connected()` flips true. The HMI matches `/^([\\w-]+) FW (\\S+)/i` to populate the version pill.\n- **LED1 (USB) mirrors `stdio_usb_connected()`** — instant visual confirmation that the host opened the port.\n- **LED2 (activity) blinks 50 ms on every valid command** — independent visual signal that the firmware *parsed* a command.\n- **Default-safe on boot** — all outputs LOW, critical actuators OFF, idle LED ON. Don't rely on the host to send `ALL OFF` first.\n\n## HMI template\n\n```html\n<script>\n\"use strict\";\n\nconst state = {\n  port: null, writer: null, reader: null,\n  connected: false, fwVersion: null,\n  autoConnectTimer: null,\n};\n\n// ─── Visible debug strip ──────────────────────────────────────────────\n// Mirror every UI event + Web Serial step into the page DOM, because\n// Chromium permission dialogs (the Web Serial picker, geo prompts,\n// etc.) render above the page surface and DON'T appear in the\n// screencast / VNC feed. Without this strip, you'd be blind to what's\n// happening on the kiosk from your dashboard.\nfunction dbg(line) {\n  const strip = document.getElementById('debugStrip');\n  if (!strip) return console.log(line);\n  const d = document.createElement('div');\n  d.textContent = new Date().toISOString().slice(11,23) + '  ' + line;\n  strip.appendChild(d);\n  while (strip.children.length > 5) strip.removeChild(strip.firstChild);\n}\n\n// ─── Line-splitter transformer for the reader stream ──────────────────\nfunction lineSplitter() {\n  let buf = '';\n  return {\n    transform(chunk, ctrl) {\n      buf += chunk;\n      let i;\n      while ((i = buf.search(/\\r?\\n/)) >= 0) {\n        ctrl.enqueue(buf.slice(0, i));\n        buf = buf.slice(i + (buf[i] === '\\r' ? 2 : 1));\n      }\n    },\n  };\n}\n\nasync function openExisting(port) {\n  await port.open({\n    baudRate: 115200, dataBits: 8, stopBits: 1, parity: 'none',\n    bufferSize: 256, flowControl: 'none',\n  });\n  state.port = port;\n\n  const enc = new TextEncoderStream();\n  enc.readable.pipeTo(port.writable);\n  state.writer = enc.writable.getWriter();\n\n  const dec = new TextDecoderStream();\n  port.readable.pipeTo(dec.writable);\n  state.reader = dec.readable\n    .pipeThrough(new TransformStream(lineSplitter()))\n    .getReader();\n\n  state.connected = true;\n  readLoop();\n  handshake();\n}\n\nasync function tryAutoConnect() {\n  if (state.connected || !('serial' in navigator)) return state.connected;\n  const ports = await navigator.serial.getPorts();\n  // The kiosk policy returns the FULL set of /dev/tty* devices the\n  // kernel sees — typically ~8 platform UARTs (empty getInfo()) plus\n  // your real USB CDC. Always filter; never open ports[0] blindly.\n  // 0x2e8a is the Raspberry Pi VID (RP2040 + RP2350). Substitute your\n  // chip's VID if you use a non-RP target.\n  const rp     = ports.filter(p => (p.getInfo() || {}).usbVendorId === 0x2e8a);\n  const anyUsb = ports.filter(p => typeof (p.getInfo() || {}).usbVendorId === 'number');\n  const candidates = rp.length ? rp : anyUsb;\n  dbg(`getPorts() → ${ports.length} total, ${rp.length} RP-series`);\n  for (const port of candidates) {\n    try {\n      await openExisting(port);\n      const info = port.getInfo();\n      dbg(`auto-connected to VID:${(info.usbVendorId||0).toString(16)} PID:${(info.usbProductId||0).toString(16)}`);\n      return true;\n    } catch (e) {\n      dbg('  open failed on one candidate: ' + e.message);\n    }\n  }\n  return false;\n}\n\nwindow.addEventListener('load', async () => {\n  // Install global capture-phase event listeners so we can see every\n  // tap on the page, even ones that don't land on a button.\n  ['pointerdown','pointerup','touchstart','touchend',\n   'mousedown','mouseup','click'].forEach(k => {\n    document.addEventListener(k, e => {\n      const tgt = e.target ? (e.target.id || e.target.tagName) : '?';\n      dbg(`${k}  (${(e.clientX||0)|0},${(e.clientY||0)|0}) → ${tgt}  trusted=${e.isTrusted}`);\n    }, /* useCapture */ true);\n  });\n  dbg('debug listeners installed');\n\n  if (!(await tryAutoConnect())) {\n    // Keep polling — if the cable is plugged in *after* the page loads,\n    // the next tick picks it up. Cheap (one promise every 2 s).\n    state.autoConnectTimer = setInterval(async () => {\n      if (await tryAutoConnect() && state.autoConnectTimer) {\n        clearInterval(state.autoConnectTimer);\n        state.autoConnectTimer = null;\n      }\n    }, 2000);\n  }\n});\n\nasync function readLoop() {\n  while (state.reader) {\n    try {\n      const { value, done } = await state.reader.read();\n      if (done) { state.connected = false; break; }\n      if (value) handleResponse(value.trim());\n    } catch { state.connected = false; break; }\n  }\n}\n\nasync function handshake() {\n  await send('PING');\n  await send('VERSION');\n  await send('STATUS');\n}\n\nasync function send(cmd) {\n  if (!state.writer) return;\n  dbg('→ ' + cmd);\n  await state.writer.write(cmd + '\\n');\n}\n\nfunction handleResponse(line) {\n  dbg('← ' + line);\n  const m = line.match(/^([\\w-]+) FW (\\S+)/i);\n  if (m) state.fwVersion = m[2];\n  // Your domain handlers here (parse STATUS reply, react to OK/ERR, etc.)\n}\n</script>\n```\n\nThe HMI's HTML body and CSS are entirely up to your project — see `octolux-hmi-design` for the brand-language guide (dark mode, accent colors, touch-target sizes, Inter font). The only DOM element this template requires is `<div id=\"debugStrip\"></div>` somewhere visible during development. You can hide it (or remove the global tap listeners) after the project ships.\n\n## One-time Chromium policy — the single biggest enabler\n\n> Without `SerialAllowAllPortsForUrls`, `requestPort()` shows a picker, the picker isn't visible through VNC, and the kiosk is unreliable. With it, `getPorts()` returns every kernel serial device with no user gesture.\n\n```json\n{\n  \"SerialAllowAllPortsForUrls\": [\"http://localhost\"],\n  \"DefaultSerialGuardSetting\": 2\n}\n```\n\n- **`SerialAllowAllPortsForUrls: [\"http://localhost\"]`** — pages served from `http://localhost` (which is where the Octolux's Apache serves the HMI) get blanket permission to enumerate and open serial ports. `getPorts()` now returns the full list.\n- **`DefaultSerialGuardSetting: 2`** — `2` means *\"Do not allow any site to request access\"*. Belt-and-suspenders: if your code accidentally calls `requestPort()`, the picker is a no-op instead of presenting UI nobody can see.\n\nPath: `/etc/chromium/policies/managed/octolux-serial.json`. Restart Chromium after install (`systemctl restart octolux-chromium`). After restart, you **must** also bounce the screencast (see `hmi-deploy-reset-vnc`).\n\nVerify the policy is live via CDP eval:\n\n```javascript\nnavigator.serial.getPorts().then(p => p.length)\n// Expected: a positive integer (typically 9 on Octolux 15.6\") when an RP\n// device is plugged in. Zero means the policy didn't load — re-check\n// the file path and the chromium restart.\n```\n\n## One-time udev rule\n\nThe Octolux Yocto distro ships `/dev/ttyACM0` as `-rw-r--r-- root:root`. The kiosk Chromium can't open it. Fix with a rule that targets the RP-series VID:\n\n```bash\ncat > /etc/udev/rules.d/99-rp-cdc-acm.rules <<'EOF'\nSUBSYSTEM==\"tty\", ATTRS{idVendor}==\"2e8a\", MODE=\"0666\"\nEOF\nudevadm control --reload-rules\n```\n\nFor an **already-plugged** device, the rule doesn't retroactively re-permission. Either physically unplug+replug, or cycle the USB driver from a shell:\n\n```bash\necho 4-1 > /sys/bus/usb/drivers/usb/unbind\nsleep 1\necho 4-1 > /sys/bus/usb/drivers/usb/bind\n```\n\n(`4-1` is the typical Octolux internal hub port; substitute from `lsusb -t` if your topology differs.)\n\nNote: the Yocto distro lacks the `usb_id` udev helper, so `udevadm info /dev/ttyACM0` returns *\"Unknown device: Inappropriate ioctl\"*. **That's fine.** Chromium reads `getInfo().usbVendorId` from the underlying USB descriptor, not from udev's metadata database. You only need the permission rule, not metadata population.\n\n## Standard deploy sequence\n\nEvery HMI edit goes through this. Don't skip the screencast bounce — see `hmi-deploy-reset-vnc`.\n\n```bash\nID=acme-control\nDEVICE_IP=10.0.10.98\n\n# 1. Push the HTML\nscp ~/ntx-embedded/customers/$ID/hmi/index.html \\\n    root@$DEVICE_IP:/usr/share/apache2/default-site/htdocs/index.html\n\n# 2. Reload the kiosk — restart chromium pulls the new HTML AND\n#    reloads any new chromium policy / udev rule deployed alongside.\nssh root@$DEVICE_IP 'systemctl restart octolux-chromium'\n\n# 3. Wait for the kiosk to come back\nsleep 10\n\n# 4. Bounce the screencast — chromium restart killed its CDP target\n#    binding and the screencast won't auto-reattach. Without this,\n#    the dashboard VNC tile freezes on a pre-deploy frame.\n#    NOTE: busybox `ps aux` truncates cmdlines so `ps aux | grep\n#    screencast` returns NOTHING even when it's running. Walk /proc\n#    instead. And NEVER walk /proc/PID/net/tcp to find port owners —\n#    those views are namespace-wide and you will mass-kill systemd.\nssh root@$DEVICE_IP 'bash -s' <<'REMOTE'\nfor d in /proc/[0-9]*; do\n  pid=${d##*/}\n  grep -q 'screencast' \"$d/cmdline\" 2>/dev/null && kill -9 \"$pid\" 2>/dev/null\ndone\nsleep 2\ncd /tmp && nohup node screencast.js --quality 80 --pipeline 4 > /tmp/screencast.log 2>&1 &\nREMOTE\n\n# 5. (optional) confirm the dashboard VNC is alive — successive snaps\n#    should differ unless the page is genuinely static.\ncurl -s http://$DEVICE_IP:8085/snapshot -o /tmp/a.jpg -w \"%{size_download}\\n\"\nsleep 2\ncurl -s http://$DEVICE_IP:8085/snapshot -o /tmp/b.jpg -w \"%{size_download}\\n\"\n```\n\nIf `/tmp/screencast.js` or `/tmp/node_modules/ws/` ever go missing on the device (they live in tmpfs and can be wiped by a bad kill spree), SCP them back from the container:\n\n```bash\nscp ~/ntx-embedded/apps/octolux/ui/screencast/screencast.js \\\n    root@$DEVICE_IP:/tmp/screencast.js\ncd ~/ntx-embedded/apps/octolux/node_modules && tar cz ws | \\\n  ssh root@$DEVICE_IP 'mkdir -p /tmp/node_modules && cd /tmp/node_modules && tar xz'\n```\n\n## Live diagnostics — CDP from the container\n\nThe Octolux kiosk exposes Chrome DevTools at `localhost:9222`. Tunnel it back:\n\n```bash\nssh -fN -L 9222:localhost:9222 -o ExitOnForwardFailure=yes root@$DEVICE_IP\n\n# install python websocket client once\npip install --user --break-system-packages websocket-client\n```\n\nThen any time you want to inspect the live page state:\n\n```python\nimport json, urllib.request, websocket\nt = json.loads(urllib.request.urlopen(\"http://localhost:9222/json/list\").read())\npage = next(x for x in t if x.get(\"type\") == \"page\")\nws = websocket.create_connection(page[\"webSocketDebuggerUrl\"])\ndef cdp(method, params=None, _id=[0]):\n    _id[0] += 1\n    ws.send(json.dumps({\"id\": _id[0], \"method\": method, \"params\": params or {}}))\n    while True:\n        r = json.loads(ws.recv())\n        if r.get(\"id\") == _id[0]: return r\ndef evx(expr, await_=False):\n    return cdp(\"Runtime.evaluate\", {\"expression\": expr, \"returnByValue\": True, \"awaitPromise\": await_})[\"result\"][\"result\"][\"value\"]\n\n# Useful queries:\nevx(\"state.connected\")\nevx(\"state.fwVersion\")\nevx(\"navigator.serial.getPorts().then(p => p.length)\", await_=True)\nevx(\"Array.from(document.querySelectorAll('#debugStrip > div')).map(d=>d.textContent).join('\\\\n')\")\n\n# Force a reload (and remember: bounce screencast afterwards!):\ncdp(\"Page.reload\", {\"ignoreCache\": True})\n```\n\n### Hit-test if a tap doesn't fire the button\n\n```javascript\nconst b = document.getElementById('btnConnect');\nconst r = b.getBoundingClientRect();\nconst hit = document.elementFromPoint(r.left + r.width/2, r.top + r.height/2);\nJSON.stringify({ same: hit === b, hit_id: hit?.id, hit_tag: hit?.tagName });\n```\n\nIf `same` is false, something is covering the button.\n\n## Failure-mode reference\n\nEvery row in this table corresponds to a real failure observed during this pattern's development. Use it as the diagnostic flow.\n\n| Symptom | Root cause | Fix |\n|---|---|---|\n| Picker shows *\"No compatible devices found\"* | Chromium policy not in place AND/OR `/dev/ttyACM*` not world-readable | Install both the policy and the udev rule; restart chromium; bounce screencast |\n| Picker shows the device but every connect ends in *NotFoundError: No port selected by user* | User dismissed the picker, or `requestPort()` was called from a non-user-gesture context (programmatic CDP `.click()` triggers this) | Don't use `requestPort()` at all — use the `getPorts()` auto-connect pattern in the HMI template |\n| `getPorts()` returns ports but `port.open()` throws *Failed to execute 'open' on 'SerialPort': Failed to open serial port* | You opened `ports[0]`; it's a platform UART without a backing device | Filter by `getInfo().usbVendorId === 0x2e8a` before opening |\n| `getPorts().length === 0` even though `lsusb` shows the device on the kernel side | Policy file missing, malformed, or wrong path; OR Chromium wasn't restarted after the policy install; OR `/dev/ttyACM0` perms are still root-only | Re-check `/etc/chromium/policies/managed/octolux-serial.json` is valid JSON; restart chromium; `ls -la /dev/ttyACM0` should be `0666` |\n| Tap on a button does nothing — no visible page reaction | Touch input might not be reaching the page; check via the debug strip (it should log `pointerdown` / `click`). If those don't appear → input layer (Weston/touchscreen) issue. If they do appear but `connect() called` doesn't → the click hit something covering the button — use `elementFromPoint` | Diagnose via debug strip + hit-test |\n| The debug strip shows `connect() called → requestPort()...` and hangs — then eventually fails *\"No port selected by user\"* | The HMI still has `requestPort()` somewhere | Replace with `getPorts()` + VID filter (HMI template) |\n| Page reloaded via CDP `Page.reload` — VNC dashboard tile froze on the pre-reload frame | The screencast bound to the old CDP target ID; reload creates a new one; the binding is dead | Bounce screencast (Standard deploy sequence step 4). Applies to ANY Chromium target change — restart, reload, navigate |\n| `octolux deploy push` times out | Octolux server's deploy API is flaky on this build | Use the SCP + restart bash sequence directly (it's what the API does internally anyway) |\n| `ps aux \\| grep screencast` returns nothing but screencast IS running and bound to port 8085 | Busybox `ps aux` truncates cmdlines | Walk `/proc/[0-9]*/cmdline` to find PIDs by argv match |\n| Killed a system PID and lost SSH / had to recover the device | You walked `/proc/[0-9]*/net/tcp` to find owners of port 8085 — those views are namespace-wide and match every PID | **NEVER** identify port owners via `/proc/PID/net/tcp` on this distro. Use `/proc/PID/cmdline` matching instead |\n| `/tmp/screencast.js` or `/tmp/node_modules/ws/` missing after a bad kill / reboot | They live in tmpfs and the original placement path doesn't auto-restore | SCP from `~/ntx-embedded/apps/octolux/ui/screencast/screencast.js` and tar+SCP `~/ntx-embedded/apps/octolux/node_modules/ws/` |\n| Adom HTTPS proxy returns `error code: 502` with no body when the HMI's own dashboard endpoint fails | The Adom proxy strips bodies on 5xx (Cloudflare-style stock error page) | Have the HMI server return HTTP 200 with `{ok:false, error, hint}` instead of 5xx — see `octolux-app-debug` Symptom 6 |\n| Dashboard section button (VNC / Deploy / Stream) does nothing | Different issue — Hydrogen browser session disconnected or server.js handler — see `octolux-app-debug` Symptoms 5 and 6 | Not in this skill |\n\n## Things NOT to do\n\n- **Don't blindly `open(ports[0])`** — always filter by VID. The kiosk policy returns the full set of `/dev/tty*` and most of them are platform UARTs that won't open.\n- **Don't rely on `requestPort()`** — it's user-gesture-gated and shows a picker, and the picker is invisible to VNC, and the operator shouldn't have to tap anyway.\n- **Don't walk `/proc/[0-9]*/net/tcp`** to find what's bound to a port. Namespace-wide view → `xargs kill` hits systemd PIDs.\n- **Don't `pkill node`** on the kiosk. node-red, the screencast, and other helpers all run as `node`. Same rule that applies on the Adom container.\n- **Don't restart Chromium without resetting screencast** — see `hmi-deploy-reset-vnc`. Same applies to CDP `Page.reload` and `Page.navigate`.\n- **Don't return 5xx** for app errors that need an explanatory body — the Adom HTTPS proxy strips bodies on 5xx. Return HTTP 200 with `{ok: false, error, hint}`.\n- **Don't assume udev metadata is populated** on the Yocto distro — `udevadm info /dev/ttyACM0` returns *\"Inappropriate ioctl\"*. Doesn't matter; Chromium reads VID/PID from the USB descriptor via `getInfo()`.\n\n## Related skills\n\n- `hmi-deploy-reset-vnc` (auto-memory) — the screencast-bounce rule. Triggered by ANY Chromium target change.\n- `octolux-hmi-design` — visual design language (dark mode, accent colors, touch targets, Inter font) for the HTML body and CSS that wraps this skill's connect logic.\n- `octolux-platform` — Octolux device-side reference (Weston, Chromium kiosk launcher, Apache config, paths).\n- `octolux-deploy` — the wider HMI-deploy story; this skill's deploy sequence is a more reliable shell-equivalent of `octolux deploy push`.\n- `octolux-app-debug` — when the issue is the Octolux Node server / dashboard, not the HMI/firmware pair.\n- `rp2350-flash` — the SWD/flash side of the firmware lifecycle. This skill is the USB-runtime side; that one is the flash side.\n",
  "skill_source": "## Known errors this skill resolves\n\nUntil the underlying bugs ship a fix, this skill is the reference for diagnosing and working around the items below. Strike a line off this list when the upstream fix lands.\n\n| | Error / symptom | Where |\n|---|---|---|\n| ☐ | Web Serial picker shows *\"No compatible devices found\"* even though `lsusb` lists the USB CDC device | Symptom #1 |\n| ☐ | `navigator.serial.requestPort()` rejects with `NotFoundError: No port selected by the user` every session | Symptom #2 |\n| ☐ | `navigator.serial.getPorts()` returns ports but `port.open()` throws *\"Failed to execute 'open' on 'SerialPort': Failed to open serial port\"* | Symptom #3 |\n| ☐ | `getPorts().length === 0` even with the policy installed and the device on the kernel bus | Symptom #4 |\n| ☐ | Tap on the Connect button does nothing visible — and you can't see why, because Chromium permission dialogs don't render in the VNC stream | Diagnostics |\n| ☐ | `octolux deploy push` times out (the SCP + chromium-restart + screencast-bounce shell sequence is more reliable) | Deploy |\n| ☐ | `ps aux \\| grep screencast` returns nothing on the device, but screencast IS running on port 8085 (busybox truncates cmdlines) | Deploy |\n| ☐ | Walked `/proc/[0-9]*/net/tcp` to find the screencast PID, `xargs kill` then killed system PIDs and lost SSH | NOT-TO-DO |\n| ☐ | `/tmp/screencast.js` or `/tmp/node_modules/ws/` missing after a bad kill on the device — needs SCP restore from the container repo | Deploy |\n| ☐ | Dashboard VNC tile freezes on a pre-deploy frame after any Chromium restart (screencast's CDP binding to the dead target) | hmi-deploy-reset-vnc |\n| ☐ | `error code: 502` no-body from the device's dashboard endpoint (Adom HTTPS proxy strips 5xx bodies — return 200 + `{ok:false}` instead) | cross-ref [octolux-app-server-debug] |\n\n\n---\nname: web-serial-hmi\ndescription: >\n  Build a paired NTX HMI (Chromium kiosk HTML on an Octolux device) +\n  firmware (USB CDC on an RP2040 / RP2350) that auto-connects over USB\n  with no permission picker. Covers the whole loop: project layout under\n  `~/ntx-embedded/customers/<id>/`, Pico SDK USB CDC firmware template\n  with line-based ASCII protocol, HMI-side Web Serial using `getPorts()`\n  + VID filter + auto-connect polling, the Chromium enterprise policy\n  `SerialAllowAllPortsForUrls` (the key enabler — without it\n  `requestPort()` always shows a picker), the udev rule that makes\n  `/dev/ttyACM*` openable by the kiosk's Chromium user, the SCP +\n  chromium-restart + screencast-bounce deploy sequence, a visible\n  page-side debug strip so you can diagnose what Chromium UI dialogs\n  (which don't render in VNC) are doing, and a CDP-via-SSH-tunnel\n  pattern for live page inspection. Includes a working firmware +\n  HMI template (~80 lines of C, ~80 lines of JS) plus a failure-mode\n  reference table. Scope: NTX Octolux devices + RP-series targets.\n  Trigger words: build hmi firmware pair, hmi over usb, web serial\n  hmi, web serial kiosk, octolux web serial, chromium serial port,\n  Web Serial no devices, navigator.serial.getPorts, requestPort vs\n  getPorts, SerialAllowAllPortsForUrls, DefaultSerialGuardSetting,\n  no port selected by user, no compatible devices found, web serial\n  picker empty, auto-connect serial port, rp2040 usb cdc, rp2350 usb\n  cdc, pico_enable_stdio_usb, kiosk udev ttyACM, /dev/ttyACM\n  permissions, chromium policy serial, octolux device deploy reset\n  vnc, visible debug strip, CDP page reload screencast stale, port\n  open failed serial, ntx hmi firmware template, new customer hmi\n  project, ports[0] open fails.\n---\n\n# web-serial-hmi — NTX HMI ↔ USB CDC firmware pattern\n\n> **Scope.** Building a Chromium-kiosk HMI on an **NTX Octolux device** that auto-connects to a **USB-attached RP2040 or RP2350** target over CDC-ACM, with **no Web Serial picker** ever showing. End-to-end working pattern, not a per-customer recipe — instantiate it for any new project under `~/ntx-embedded/customers/<id>/`.\n>\n> Out of scope: Android USB Host (different stack); Web Bluetooth / WebHID; non-RP-series USB devices (mostly the same pattern, just substitute the VID); CAN-FD bridges; the SWD/flash side of the firmware lifecycle — that's covered by `rp2350-flash` and the `target/rp2040.cfg` openocd patterns and the local `firmware/` build is just plain `cmake + make`.\n\n## When to use this skill\n\n- Starting a new customer HMI project that needs to control hardware over a USB-attached MCU.\n- Porting an Android-tablet HMI app to an Octolux Linux/Chromium device and you want the same \"just plug it in\" feel.\n- The Web Serial picker is showing — *\"No compatible devices found\"*, *\"No port selected by user\"*, or just appearing at all — and the project shouldn't have a picker.\n- `requestPort()` works once but the operator has to tap-and-pick every session and that's wrong for a kiosk.\n- The HMI page can't see what's happening because Chromium permission dialogs don't render in VNC.\n\n## Architecture\n\n```\n┌─────────────────────────────────┐      USB-C       ┌──────────────────────┐\n│ NTX Octolux device (10.0.10.x)  │ ◄──────────────► │ RP2040 / RP2350 board│\n│ ─────────────────────────────── │ CDC-ACM, 115200  │ ───────────────────  │\n│ Chromium 109 kiosk @ localhost  │   8N1 line ASCII │ Pico SDK USB CDC     │\n│  ├─ Web Serial API              │                  │  ├─ getchar polling  │\n│  ├─ SerialAllowAllPortsForUrls  │                  │  ├─ ASCII line parser│\n│  └─ getPorts() filtered by VID  │                  │  └─ GPIO drive       │\n│ Apache at /                     │                  │                      │\n│ /dev/ttyACM0 (root:root, 0666)  │                  │                      │\n└─────────────────────────────────┘                  └──────────────────────┘\n        ▲                                                       ▲\n        │  SSH from container (deploy + udev + policy)          │  SWD via PiProbe (flash)\n        │                                                       │\n   Adom Docker container                                  see rp2350-flash\n   ~/ntx-embedded/customers/<id>/{hmi,firmware}\n```\n\nTwo physical links from the workcell to the target board:\n\n- **USB-C** between Octolux and target — runtime command path (this skill).\n- **SWD** between the workcell's PiProbe and the target — flash path (`rp2350-flash`).\n\nThe container is the build host for both: HMI HTML edits go SCP-ed to Octolux; firmware ELF is built locally and flashed via the PiProbe.\n\n## Required components\n\nA working NTX HMI+firmware pair has **five** moving parts. Skip any one and connect won't happen. Pieces 3 and 4 are one-time setup *per Octolux device*; pieces 1, 2, 5 are per-project.\n\n| # | Layer | Where it lives | Per-… |\n|---|---|---|---|\n| 1 | **Firmware** — USB CDC stdio, command parser, GPIO drive | `customers/<id>/firmware/src/main.c` (in container) | project |\n| 2 | **HMI HTML** — connect logic + UI | `customers/<id>/hmi/index.html` (in container, deployed to device) | project |\n| 3 | **Chromium policy** — `SerialAllowAllPortsForUrls` | `/etc/chromium/policies/managed/octolux-serial.json` (on device) | device |\n| 4 | **udev rule** — `MODE=\"0666\"` for VID 0x2e8a | `/etc/udev/rules.d/99-rp-cdc-acm.rules` (on device) | device |\n| 5 | **Deploy** — SCP HMI + chromium restart + screencast bounce | shell pattern run from container | project |\n\n## Quick start — new project, end to end\n\nRun from the Adom container. Substitute `<id>` (e.g. `acme-control`), `<device-ip>` (e.g. `10.0.10.98`), and your domain commands.\n\n### 1. Scaffold\n\n```bash\nID=acme-control\nDEVICE_IP=10.0.10.98\nDEVICE_MODEL='Octolux 15.6\"'\nVIEW_W=1920 VIEW_H=1080\n\nmkdir -p ~/ntx-embedded/customers/$ID/{hmi,firmware/src}\n\ncat > ~/ntx-embedded/customers/$ID/project.json <<EOF\n{\n  \"id\": \"$ID\",\n  \"name\": \"Acme Control\",\n  \"customer\": \"Acme\",\n  \"summary\": \"One-line description of what this HMI does.\",\n  \"accent\": \"#e87a1e\",\n  \"accentDeep\": \"#a64f10\",\n  \"deviceModel\": \"$DEVICE_MODEL\",\n  \"viewport\": { \"w\": $VIEW_W, \"h\": $VIEW_H },\n  \"orientation\": \"landscape\",\n  \"defaultIP\": \"$DEVICE_IP\",\n  \"tagline\": \"one-line operator-facing tagline\"\n}\nEOF\n```\n\n### 2. Drop in the firmware + HMI templates\n\nUse the templates further down this doc verbatim, customizing only:\n\n- **Firmware:** `FW_VERSION_STRING` (the `<NAME> FW x.y.z` boot banner), the `dispatch()` command list, and your specific GPIO pin numbers / domain logic.\n- **HMI:** the visible controls (buttons, sliders, etc.), the brand colors from your `project.json`, the response-line handlers in `handleResponse()`.\n\nKeep the Web Serial connect logic and the visible debug strip **unchanged** — that's the point of this skill.\n\n### 3. Build + flash the firmware (first time)\n\n```bash\nexport PICO_SDK_PATH=$HOME/pico-sdk\ncd ~/ntx-embedded/customers/$ID/firmware\nmkdir -p build && cd build && cmake .. && make -j$(nproc)\n\n# Flash via PiProbe over SWD — pick the cfg for your chip:\nopenocd -f interface/cmsis-dap.cfg -f target/rp2040.cfg \\\n  -c \"adapter speed 5000; program ${ID//-/_}.elf verify reset exit\"\n# (or target/rp2350.cfg for an RP2350 — see rp2350-flash)\n```\n\n### 4. One-time device setup (Chromium policy + udev rule)\n\nSkip if this Octolux device has been set up for Web Serial before. Verify by SSH'ing in: if `/etc/chromium/policies/managed/octolux-serial.json` and `/etc/udev/rules.d/99-rp-cdc-acm.rules` both exist, you're done.\n\n```bash\nssh root@$DEVICE_IP 'bash -s' <<'REMOTE'\nset -e\n# Chromium policy: SerialAllowAllPortsForUrls turns getPorts() into a\n# blanket enumeration for http://localhost; DefaultSerialGuardSetting=2\n# also disables requestPort() pickers as a safety net.\nmkdir -p /etc/chromium/policies/managed\ncat > /etc/chromium/policies/managed/octolux-serial.json <<'POL'\n{\n  \"SerialAllowAllPortsForUrls\": [\"http://localhost\"],\n  \"DefaultSerialGuardSetting\": 2\n}\nPOL\n\n# udev: /dev/ttyACM* is root-only by default on the Yocto distro, and\n# the kiosk Chromium can't open it. MODE=0666 fixes that for any\n# Raspberry Pi (RP2040/RP2350) USB CDC device.\ncat > /etc/udev/rules.d/99-rp-cdc-acm.rules <<'UDEV'\nSUBSYSTEM==\"tty\", ATTRS{idVendor}==\"2e8a\", MODE=\"0666\"\nUDEV\nudevadm control --reload-rules\n\n# Re-process any ALREADY-plugged-in device so the new rule applies now.\n# (Without this, the perms only apply to *future* plug events.)\nBUSPORT=$(lsusb -t 2>/dev/null | awk '/Pico|2e8a/ {sub(/^[ \\t]+/, \"\"); print}' | head -1)\necho \"USB device tree row: ${BUSPORT:-(none seen)}\"\n# Cycle the parent USB device (substitute bus-port from `lsusb -t` if your\n# topology differs; 4-1 is the common Octolux internal hub port).\necho 4-1 > /sys/bus/usb/drivers/usb/unbind 2>/dev/null || true\nsleep 1\necho 4-1 > /sys/bus/usb/drivers/usb/bind 2>/dev/null || true\n\nsystemctl restart octolux-chromium\nREMOTE\n\n# Per the [[hmi-deploy-reset-vnc]] rule: chromium restart leaves the\n# screencast bound to a dead CDP target. Bounce the screencast on the\n# device — see the Deploy sequence below for the exact commands.\n```\n\n### 5. Deploy the HMI to the device\n\nSee the \"Standard deploy sequence\" section below — it's the same shell pattern you'll run on every HMI edit.\n\n### 6. Verify it auto-connected (CDP eval from container)\n\n```bash\nssh -fN -L 9222:localhost:9222 -o ExitOnForwardFailure=yes root@$DEVICE_IP\n\npython3 - <<'PY'\nimport json, urllib.request, websocket\nt = json.loads(urllib.request.urlopen(\"http://localhost:9222/json/list\").read())\npage = next(x for x in t if x.get(\"type\") == \"page\")\nws = websocket.create_connection(page[\"webSocketDebuggerUrl\"])\ndef evx(expr, await_=False):\n    ws.send(json.dumps({\"id\": 1, \"method\": \"Runtime.evaluate\",\n        \"params\": {\"expression\": expr, \"returnByValue\": True,\n                   \"awaitPromise\": await_}}))\n    while True:\n        r = json.loads(ws.recv())\n        if r.get(\"id\") == 1: return r[\"result\"][\"result\"][\"value\"]\nprint(\"connected:\", evx(\"state.connected\"))\nprint(\"fwVersion:\", evx(\"state.fwVersion\"))\nprint(\"ports    :\", evx(\"navigator.serial.getPorts().then(p => p.length)\", await_=True))\nPY\n```\n\nExpected once the firmware is running and the cable is plugged in:\n\n```\nconnected: True\nfwVersion: 1.0.0\nports    : 9          # 8 platform UARTs + your 1 RP-series CDC\n```\n\nIf `connected: True` and `fwVersion` is populated → you're done; the HMI is talking to the MCU. If not, jump to the failure table.\n\n## Firmware template (RP2040 / RP2350 USB CDC)\n\n`firmware/CMakeLists.txt`:\n\n```cmake\ncmake_minimum_required(VERSION 3.13)\nset(PICO_BOARD pico CACHE STRING \"\")            # or \"pico2\" for RP2350\nset(PICO_PLATFORM rp2040 CACHE STRING \"\")       # or \"rp2350\"\ninclude($ENV{PICO_SDK_PATH}/external/pico_sdk_import.cmake)\nproject(<id_underscored> C CXX ASM)\npico_sdk_init()\nadd_executable(<id_underscored> src/main.c)\ntarget_link_libraries(<id_underscored> pico_stdlib hardware_gpio)\npico_enable_stdio_usb(<id_underscored>  1)      # CDC-ACM on USB-C\npico_enable_stdio_uart(<id_underscored> 0)      # no UART — USB only\npico_add_extra_outputs(<id_underscored>)\n```\n\n`firmware/src/main.c`:\n\n```c\n#include <stdio.h>\n#include <string.h>\n#include <ctype.h>\n#include <stdarg.h>\n#include \"pico/stdlib.h\"\n#include \"pico/stdio_usb.h\"      // stdio_usb_connected()\n#include \"hardware/gpio.h\"\n\n#define FW_VERSION_STRING \"<NAME> FW 1.0.0\"\n#define CMD_BUF_SIZE 96\n\n// Status LEDs (wire your board's status LEDs to these or change as needed).\n// LED_USB: solid ON while USB CDC is connected to a host.\n// LED_ACT: 50 ms blink on every valid command.\n// LED_ERR: solid ON when the last response was ERR; clears on next OK.\n// LED_IDLE: solid ON when system is in its safe / inactive state.\n#define LED_USB   9\n#define LED_ACT  10\n#define LED_ERR  16\n#define LED_IDLE 18\n\nstatic char cmd_buf[CMD_BUF_SIZE];\nstatic int  cmd_len = 0;\nstatic bool act_on = false;\nstatic absolute_time_t act_off_at;\n\nstatic void blink_activity(void) {\n    gpio_put(LED_ACT, 1);\n    act_on = true;\n    act_off_at = make_timeout_time_ms(50);\n}\nstatic void say_ok(const char *fmt, ...) {\n    va_list a; va_start(a, fmt);\n    fputs(\"OK \", stdout); vprintf(fmt, a); fputs(\"\\r\\n\", stdout);\n    va_end(a);\n    gpio_put(LED_ERR, 0);\n    blink_activity();\n}\nstatic void say_err(const char *m) {\n    printf(\"ERR %s\\r\\n\", m);\n    gpio_put(LED_ERR, 1);\n    blink_activity();\n}\nstatic void to_upper(char *s) { for (; *s; s++) *s = (char)toupper((unsigned char)*s); }\n\n// === Domain commands ====================================================\n//\n// Replace these with the commands your project actually needs. Keep\n// the canonical built-ins (PING, VERSION, STATUS, HELP) — the HMI\n// template's handshake assumes they exist.\n//\nstatic void dispatch(char *line) {\n    char *tok = strtok(line, \" \\t\");\n    if (!tok) return;\n\n    if (!strcmp(tok, \"PING\"))    { say_ok(\"PONG\"); return; }\n    if (!strcmp(tok, \"VERSION\")) { printf(\"%s\\r\\n\", FW_VERSION_STRING); blink_activity(); return; }\n    if (!strcmp(tok, \"STATUS\"))  { /* print one-line state snapshot */ blink_activity(); return; }\n    if (!strcmp(tok, \"HELP\"))    { /* print \"OK …\" lines */ blink_activity(); return; }\n\n    // your-domain command parsing here, e.g.:\n    //   if (!strcmp(tok, \"DO\")) { … say_ok(\"DO %d\", arg); return; }\n\n    say_err(\"unknown command\");\n}\n\n// === Boilerplate main =================================================\n\nint main(void) {\n    stdio_init_all();\n\n    // Init your GPIOs here (LEDs + outputs). Drive outputs LOW first\n    // so the system boots in a safe state regardless of host behavior.\n    const uint leds[] = { LED_USB, LED_ACT, LED_ERR, LED_IDLE };\n    for (size_t i = 0; i < sizeof(leds)/sizeof(leds[0]); i++) {\n        gpio_init(leds[i]); gpio_set_dir(leds[i], GPIO_OUT); gpio_put(leds[i], 0);\n    }\n    gpio_put(LED_IDLE, 1);\n\n    bool was_conn = false, banner_done = false;\n\n    while (true) {\n        bool conn = stdio_usb_connected();\n        gpio_put(LED_USB, conn ? 1 : 0);\n        if (conn && !was_conn) banner_done = false;\n        was_conn = conn;\n\n        // Boot banner on every CDC connect — the HMI parses this to\n        // populate the FW version pill.\n        if (conn && !banner_done) {\n            sleep_ms(50);                     // give the host's reader a beat\n            printf(\"%s\\r\\n\", FW_VERSION_STRING);\n            puts(\"TYPE 'HELP' FOR COMMANDS\");\n            banner_done = true;\n        }\n\n        // Drain available input into cmd_buf; dispatch on EOL.\n        for (int budget = 0; budget < 32; budget++) {\n            int c = getchar_timeout_us(0);\n            if (c == PICO_ERROR_TIMEOUT) break;\n            if (c == '\\n' || c == '\\r') {\n                if (cmd_len > 0) {\n                    cmd_buf[cmd_len] = '\\0';\n                    to_upper(cmd_buf);\n                    dispatch(cmd_buf);\n                    cmd_len = 0;\n                }\n            } else if (cmd_len < CMD_BUF_SIZE - 1) {\n                cmd_buf[cmd_len++] = (char)c;\n            } else { cmd_len = 0; say_err(\"command too long\"); }\n        }\n\n        // Activity LED timeout.\n        if (act_on && absolute_time_diff_us(get_absolute_time(), act_off_at) <= 0) {\n            gpio_put(LED_ACT, 0); act_on = false;\n        }\n    }\n}\n```\n\n### Protocol conventions that work well\n\n- **Newline-terminated ASCII** — accept `\\n` and `\\r\\n`. Case-insensitive (uppercase before dispatch).\n- **One line in, exactly one line out** — `OK …`, `ERR …`, or a single-token status line. One-to-one keeps the HMI's send/receive trivial.\n- **Standard built-ins always present:**\n  - `PING → PONG` (liveness)\n  - `VERSION → <NAME> FW x.y.z` (capability detection + version pill)\n  - `STATUS → STATE …` (one-line state snapshot)\n  - `HELP → multi-line OK …` (operator self-discovery)\n- **Boot banner on every CDC connect.** Print `<NAME> FW x.y.z\\r\\n` first thing after `stdio_usb_connected()` flips true. The HMI matches `/^([\\w-]+) FW (\\S+)/i` to populate the version pill.\n- **LED1 (USB) mirrors `stdio_usb_connected()`** — instant visual confirmation that the host opened the port.\n- **LED2 (activity) blinks 50 ms on every valid command** — independent visual signal that the firmware *parsed* a command.\n- **Default-safe on boot** — all outputs LOW, critical actuators OFF, idle LED ON. Don't rely on the host to send `ALL OFF` first.\n\n## HMI template\n\n```html\n<script>\n\"use strict\";\n\nconst state = {\n  port: null, writer: null, reader: null,\n  connected: false, fwVersion: null,\n  autoConnectTimer: null,\n};\n\n// ─── Visible debug strip ──────────────────────────────────────────────\n// Mirror every UI event + Web Serial step into the page DOM, because\n// Chromium permission dialogs (the Web Serial picker, geo prompts,\n// etc.) render above the page surface and DON'T appear in the\n// screencast / VNC feed. Without this strip, you'd be blind to what's\n// happening on the kiosk from your dashboard.\nfunction dbg(line) {\n  const strip = document.getElementById('debugStrip');\n  if (!strip) return console.log(line);\n  const d = document.createElement('div');\n  d.textContent = new Date().toISOString().slice(11,23) + '  ' + line;\n  strip.appendChild(d);\n  while (strip.children.length > 5) strip.removeChild(strip.firstChild);\n}\n\n// ─── Line-splitter transformer for the reader stream ──────────────────\nfunction lineSplitter() {\n  let buf = '';\n  return {\n    transform(chunk, ctrl) {\n      buf += chunk;\n      let i;\n      while ((i = buf.search(/\\r?\\n/)) >= 0) {\n        ctrl.enqueue(buf.slice(0, i));\n        buf = buf.slice(i + (buf[i] === '\\r' ? 2 : 1));\n      }\n    },\n  };\n}\n\nasync function openExisting(port) {\n  await port.open({\n    baudRate: 115200, dataBits: 8, stopBits: 1, parity: 'none',\n    bufferSize: 256, flowControl: 'none',\n  });\n  state.port = port;\n\n  const enc = new TextEncoderStream();\n  enc.readable.pipeTo(port.writable);\n  state.writer = enc.writable.getWriter();\n\n  const dec = new TextDecoderStream();\n  port.readable.pipeTo(dec.writable);\n  state.reader = dec.readable\n    .pipeThrough(new TransformStream(lineSplitter()))\n    .getReader();\n\n  state.connected = true;\n  readLoop();\n  handshake();\n}\n\nasync function tryAutoConnect() {\n  if (state.connected || !('serial' in navigator)) return state.connected;\n  const ports = await navigator.serial.getPorts();\n  // The kiosk policy returns the FULL set of /dev/tty* devices the\n  // kernel sees — typically ~8 platform UARTs (empty getInfo()) plus\n  // your real USB CDC. Always filter; never open ports[0] blindly.\n  // 0x2e8a is the Raspberry Pi VID (RP2040 + RP2350). Substitute your\n  // chip's VID if you use a non-RP target.\n  const rp     = ports.filter(p => (p.getInfo() || {}).usbVendorId === 0x2e8a);\n  const anyUsb = ports.filter(p => typeof (p.getInfo() || {}).usbVendorId === 'number');\n  const candidates = rp.length ? rp : anyUsb;\n  dbg(`getPorts() → ${ports.length} total, ${rp.length} RP-series`);\n  for (const port of candidates) {\n    try {\n      await openExisting(port);\n      const info = port.getInfo();\n      dbg(`auto-connected to VID:${(info.usbVendorId||0).toString(16)} PID:${(info.usbProductId||0).toString(16)}`);\n      return true;\n    } catch (e) {\n      dbg('  open failed on one candidate: ' + e.message);\n    }\n  }\n  return false;\n}\n\nwindow.addEventListener('load', async () => {\n  // Install global capture-phase event listeners so we can see every\n  // tap on the page, even ones that don't land on a button.\n  ['pointerdown','pointerup','touchstart','touchend',\n   'mousedown','mouseup','click'].forEach(k => {\n    document.addEventListener(k, e => {\n      const tgt = e.target ? (e.target.id || e.target.tagName) : '?';\n      dbg(`${k}  (${(e.clientX||0)|0},${(e.clientY||0)|0}) → ${tgt}  trusted=${e.isTrusted}`);\n    }, /* useCapture */ true);\n  });\n  dbg('debug listeners installed');\n\n  if (!(await tryAutoConnect())) {\n    // Keep polling — if the cable is plugged in *after* the page loads,\n    // the next tick picks it up. Cheap (one promise every 2 s).\n    state.autoConnectTimer = setInterval(async () => {\n      if (await tryAutoConnect() && state.autoConnectTimer) {\n        clearInterval(state.autoConnectTimer);\n        state.autoConnectTimer = null;\n      }\n    }, 2000);\n  }\n});\n\nasync function readLoop() {\n  while (state.reader) {\n    try {\n      const { value, done } = await state.reader.read();\n      if (done) { state.connected = false; break; }\n      if (value) handleResponse(value.trim());\n    } catch { state.connected = false; break; }\n  }\n}\n\nasync function handshake() {\n  await send('PING');\n  await send('VERSION');\n  await send('STATUS');\n}\n\nasync function send(cmd) {\n  if (!state.writer) return;\n  dbg('→ ' + cmd);\n  await state.writer.write(cmd + '\\n');\n}\n\nfunction handleResponse(line) {\n  dbg('← ' + line);\n  const m = line.match(/^([\\w-]+) FW (\\S+)/i);\n  if (m) state.fwVersion = m[2];\n  // Your domain handlers here (parse STATUS reply, react to OK/ERR, etc.)\n}\n</script>\n```\n\nThe HMI's HTML body and CSS are entirely up to your project — see `octolux-hmi-design` for the brand-language guide (dark mode, accent colors, touch-target sizes, Inter font). The only DOM element this template requires is `<div id=\"debugStrip\"></div>` somewhere visible during development. You can hide it (or remove the global tap listeners) after the project ships.\n\n## One-time Chromium policy — the single biggest enabler\n\n> Without `SerialAllowAllPortsForUrls`, `requestPort()` shows a picker, the picker isn't visible through VNC, and the kiosk is unreliable. With it, `getPorts()` returns every kernel serial device with no user gesture.\n\n```json\n{\n  \"SerialAllowAllPortsForUrls\": [\"http://localhost\"],\n  \"DefaultSerialGuardSetting\": 2\n}\n```\n\n- **`SerialAllowAllPortsForUrls: [\"http://localhost\"]`** — pages served from `http://localhost` (which is where the Octolux's Apache serves the HMI) get blanket permission to enumerate and open serial ports. `getPorts()` now returns the full list.\n- **`DefaultSerialGuardSetting: 2`** — `2` means *\"Do not allow any site to request access\"*. Belt-and-suspenders: if your code accidentally calls `requestPort()`, the picker is a no-op instead of presenting UI nobody can see.\n\nPath: `/etc/chromium/policies/managed/octolux-serial.json`. Restart Chromium after install (`systemctl restart octolux-chromium`). After restart, you **must** also bounce the screencast (see `hmi-deploy-reset-vnc`).\n\nVerify the policy is live via CDP eval:\n\n```javascript\nnavigator.serial.getPorts().then(p => p.length)\n// Expected: a positive integer (typically 9 on Octolux 15.6\") when an RP\n// device is plugged in. Zero means the policy didn't load — re-check\n// the file path and the chromium restart.\n```\n\n## One-time udev rule\n\nThe Octolux Yocto distro ships `/dev/ttyACM0` as `-rw-r--r-- root:root`. The kiosk Chromium can't open it. Fix with a rule that targets the RP-series VID:\n\n```bash\ncat > /etc/udev/rules.d/99-rp-cdc-acm.rules <<'EOF'\nSUBSYSTEM==\"tty\", ATTRS{idVendor}==\"2e8a\", MODE=\"0666\"\nEOF\nudevadm control --reload-rules\n```\n\nFor an **already-plugged** device, the rule doesn't retroactively re-permission. Either physically unplug+replug, or cycle the USB driver from a shell:\n\n```bash\necho 4-1 > /sys/bus/usb/drivers/usb/unbind\nsleep 1\necho 4-1 > /sys/bus/usb/drivers/usb/bind\n```\n\n(`4-1` is the typical Octolux internal hub port; substitute from `lsusb -t` if your topology differs.)\n\nNote: the Yocto distro lacks the `usb_id` udev helper, so `udevadm info /dev/ttyACM0` returns *\"Unknown device: Inappropriate ioctl\"*. **That's fine.** Chromium reads `getInfo().usbVendorId` from the underlying USB descriptor, not from udev's metadata database. You only need the permission rule, not metadata population.\n\n## Standard deploy sequence\n\nEvery HMI edit goes through this. Don't skip the screencast bounce — see `hmi-deploy-reset-vnc`.\n\n```bash\nID=acme-control\nDEVICE_IP=10.0.10.98\n\n# 1. Push the HTML\nscp ~/ntx-embedded/customers/$ID/hmi/index.html \\\n    root@$DEVICE_IP:/usr/share/apache2/default-site/htdocs/index.html\n\n# 2. Reload the kiosk — restart chromium pulls the new HTML AND\n#    reloads any new chromium policy / udev rule deployed alongside.\nssh root@$DEVICE_IP 'systemctl restart octolux-chromium'\n\n# 3. Wait for the kiosk to come back\nsleep 10\n\n# 4. Bounce the screencast — chromium restart killed its CDP target\n#    binding and the screencast won't auto-reattach. Without this,\n#    the dashboard VNC tile freezes on a pre-deploy frame.\n#    NOTE: busybox `ps aux` truncates cmdlines so `ps aux | grep\n#    screencast` returns NOTHING even when it's running. Walk /proc\n#    instead. And NEVER walk /proc/PID/net/tcp to find port owners —\n#    those views are namespace-wide and you will mass-kill systemd.\nssh root@$DEVICE_IP 'bash -s' <<'REMOTE'\nfor d in /proc/[0-9]*; do\n  pid=${d##*/}\n  grep -q 'screencast' \"$d/cmdline\" 2>/dev/null && kill -9 \"$pid\" 2>/dev/null\ndone\nsleep 2\ncd /tmp && nohup node screencast.js --quality 80 --pipeline 4 > /tmp/screencast.log 2>&1 &\nREMOTE\n\n# 5. (optional) confirm the dashboard VNC is alive — successive snaps\n#    should differ unless the page is genuinely static.\ncurl -s http://$DEVICE_IP:8085/snapshot -o /tmp/a.jpg -w \"%{size_download}\\n\"\nsleep 2\ncurl -s http://$DEVICE_IP:8085/snapshot -o /tmp/b.jpg -w \"%{size_download}\\n\"\n```\n\nIf `/tmp/screencast.js` or `/tmp/node_modules/ws/` ever go missing on the device (they live in tmpfs and can be wiped by a bad kill spree), SCP them back from the container:\n\n```bash\nscp ~/ntx-embedded/apps/octolux/ui/screencast/screencast.js \\\n    root@$DEVICE_IP:/tmp/screencast.js\ncd ~/ntx-embedded/apps/octolux/node_modules && tar cz ws | \\\n  ssh root@$DEVICE_IP 'mkdir -p /tmp/node_modules && cd /tmp/node_modules && tar xz'\n```\n\n## Live diagnostics — CDP from the container\n\nThe Octolux kiosk exposes Chrome DevTools at `localhost:9222`. Tunnel it back:\n\n```bash\nssh -fN -L 9222:localhost:9222 -o ExitOnForwardFailure=yes root@$DEVICE_IP\n\n# install python websocket client once\npip install --user --break-system-packages websocket-client\n```\n\nThen any time you want to inspect the live page state:\n\n```python\nimport json, urllib.request, websocket\nt = json.loads(urllib.request.urlopen(\"http://localhost:9222/json/list\").read())\npage = next(x for x in t if x.get(\"type\") == \"page\")\nws = websocket.create_connection(page[\"webSocketDebuggerUrl\"])\ndef cdp(method, params=None, _id=[0]):\n    _id[0] += 1\n    ws.send(json.dumps({\"id\": _id[0], \"method\": method, \"params\": params or {}}))\n    while True:\n        r = json.loads(ws.recv())\n        if r.get(\"id\") == _id[0]: return r\ndef evx(expr, await_=False):\n    return cdp(\"Runtime.evaluate\", {\"expression\": expr, \"returnByValue\": True, \"awaitPromise\": await_})[\"result\"][\"result\"][\"value\"]\n\n# Useful queries:\nevx(\"state.connected\")\nevx(\"state.fwVersion\")\nevx(\"navigator.serial.getPorts().then(p => p.length)\", await_=True)\nevx(\"Array.from(document.querySelectorAll('#debugStrip > div')).map(d=>d.textContent).join('\\\\n')\")\n\n# Force a reload (and remember: bounce screencast afterwards!):\ncdp(\"Page.reload\", {\"ignoreCache\": True})\n```\n\n### Hit-test if a tap doesn't fire the button\n\n```javascript\nconst b = document.getElementById('btnConnect');\nconst r = b.getBoundingClientRect();\nconst hit = document.elementFromPoint(r.left + r.width/2, r.top + r.height/2);\nJSON.stringify({ same: hit === b, hit_id: hit?.id, hit_tag: hit?.tagName });\n```\n\nIf `same` is false, something is covering the button.\n\n## Failure-mode reference\n\nEvery row in this table corresponds to a real failure observed during this pattern's development. Use it as the diagnostic flow.\n\n| Symptom | Root cause | Fix |\n|---|---|---|\n| Picker shows *\"No compatible devices found\"* | Chromium policy not in place AND/OR `/dev/ttyACM*` not world-readable | Install both the policy and the udev rule; restart chromium; bounce screencast |\n| Picker shows the device but every connect ends in *NotFoundError: No port selected by user* | User dismissed the picker, or `requestPort()` was called from a non-user-gesture context (programmatic CDP `.click()` triggers this) | Don't use `requestPort()` at all — use the `getPorts()` auto-connect pattern in the HMI template |\n| `getPorts()` returns ports but `port.open()` throws *Failed to execute 'open' on 'SerialPort': Failed to open serial port* | You opened `ports[0]`; it's a platform UART without a backing device | Filter by `getInfo().usbVendorId === 0x2e8a` before opening |\n| `getPorts().length === 0` even though `lsusb` shows the device on the kernel side | Policy file missing, malformed, or wrong path; OR Chromium wasn't restarted after the policy install; OR `/dev/ttyACM0` perms are still root-only | Re-check `/etc/chromium/policies/managed/octolux-serial.json` is valid JSON; restart chromium; `ls -la /dev/ttyACM0` should be `0666` |\n| Tap on a button does nothing — no visible page reaction | Touch input might not be reaching the page; check via the debug strip (it should log `pointerdown` / `click`). If those don't appear → input layer (Weston/touchscreen) issue. If they do appear but `connect() called` doesn't → the click hit something covering the button — use `elementFromPoint` | Diagnose via debug strip + hit-test |\n| The debug strip shows `connect() called → requestPort()...` and hangs — then eventually fails *\"No port selected by user\"* | The HMI still has `requestPort()` somewhere | Replace with `getPorts()` + VID filter (HMI template) |\n| Page reloaded via CDP `Page.reload` — VNC dashboard tile froze on the pre-reload frame | The screencast bound to the old CDP target ID; reload creates a new one; the binding is dead | Bounce screencast (Standard deploy sequence step 4). Applies to ANY Chromium target change — restart, reload, navigate |\n| `octolux deploy push` times out | Octolux server's deploy API is flaky on this build | Use the SCP + restart bash sequence directly (it's what the API does internally anyway) |\n| `ps aux \\| grep screencast` returns nothing but screencast IS running and bound to port 8085 | Busybox `ps aux` truncates cmdlines | Walk `/proc/[0-9]*/cmdline` to find PIDs by argv match |\n| Killed a system PID and lost SSH / had to recover the device | You walked `/proc/[0-9]*/net/tcp` to find owners of port 8085 — those views are namespace-wide and match every PID | **NEVER** identify port owners via `/proc/PID/net/tcp` on this distro. Use `/proc/PID/cmdline` matching instead |\n| `/tmp/screencast.js` or `/tmp/node_modules/ws/` missing after a bad kill / reboot | They live in tmpfs and the original placement path doesn't auto-restore | SCP from `~/ntx-embedded/apps/octolux/ui/screencast/screencast.js` and tar+SCP `~/ntx-embedded/apps/octolux/node_modules/ws/` |\n| Adom HTTPS proxy returns `error code: 502` with no body when the HMI's own dashboard endpoint fails | The Adom proxy strips bodies on 5xx (Cloudflare-style stock error page) | Have the HMI server return HTTP 200 with `{ok:false, error, hint}` instead of 5xx — see `octolux-app-debug` Symptom 6 |\n| Dashboard section button (VNC / Deploy / Stream) does nothing | Different issue — Hydrogen browser session disconnected or server.js handler — see `octolux-app-debug` Symptoms 5 and 6 | Not in this skill |\n\n## Things NOT to do\n\n- **Don't blindly `open(ports[0])`** — always filter by VID. The kiosk policy returns the full set of `/dev/tty*` and most of them are platform UARTs that won't open.\n- **Don't rely on `requestPort()`** — it's user-gesture-gated and shows a picker, and the picker is invisible to VNC, and the operator shouldn't have to tap anyway.\n- **Don't walk `/proc/[0-9]*/net/tcp`** to find what's bound to a port. Namespace-wide view → `xargs kill` hits systemd PIDs.\n- **Don't `pkill node`** on the kiosk. node-red, the screencast, and other helpers all run as `node`. Same rule that applies on the Adom container.\n- **Don't restart Chromium without resetting screencast** — see `hmi-deploy-reset-vnc`. Same applies to CDP `Page.reload` and `Page.navigate`.\n- **Don't return 5xx** for app errors that need an explanatory body — the Adom HTTPS proxy strips bodies on 5xx. Return HTTP 200 with `{ok: false, error, hint}`.\n- **Don't assume udev metadata is populated** on the Yocto distro — `udevadm info /dev/ttyACM0` returns *\"Inappropriate ioctl\"*. Doesn't matter; Chromium reads VID/PID from the USB descriptor via `getInfo()`.\n\n## Related skills\n\n- `hmi-deploy-reset-vnc` (auto-memory) — the screencast-bounce rule. Triggered by ANY Chromium target change.\n- `octolux-hmi-design` — visual design language (dark mode, accent colors, touch targets, Inter font) for the HTML body and CSS that wraps this skill's connect logic.\n- `octolux-platform` — Octolux device-side reference (Weston, Chromium kiosk launcher, Apache config, paths).\n- `octolux-deploy` — the wider HMI-deploy story; this skill's deploy sequence is a more reliable shell-equivalent of `octolux deploy push`.\n- `octolux-app-debug` — when the issue is the Octolux Node server / dashboard, not the HMI/firmware pair.\n- `rp2350-flash` — the SWD/flash side of the firmware lifecycle. This skill is the USB-runtime side; that one is the flash side.\n",
  "author": {
    "name": "Caleb",
    "email": "[email protected]"
  },
  "tags": [],
  "license": "proprietary",
  "visibility": {
    "public": true
  },
  "hero": null,
  "sample_prompts": [
    {
      "label": "Build HMI+firmware pair",
      "prompt": "Start a new HMI+firmware project that talks over USB on Octolux"
    },
    {
      "label": "Auto-connect no picker",
      "prompt": "My HMI's Web Serial picker keeps appearing — how do I auto-connect on load?"
    },
    {
      "label": "Picker shows no devices",
      "prompt": "Web Serial picker says 'No compatible devices found' but lsusb sees the device"
    },
    {
      "label": "HMI port.open fails",
      "prompt": "getPorts() returns ports but port.open() throws 'Failed to open serial port'"
    },
    {
      "label": "Chromium serial policy",
      "prompt": "Where do I install SerialAllowAllPortsForUrls on an Octolux device?"
    },
    {
      "label": "Reference firmware template",
      "prompt": "Show me the RP2040 USB CDC firmware template with the standard line protocol"
    }
  ],
  "discovery_triggers": [
    "build hmi firmware pair",
    "hmi over usb",
    "web serial hmi",
    "web serial kiosk",
    "octolux web serial",
    "chromium serial port",
    "Web Serial no devices",
    "navigator.serial.getPorts",
    "requestPort vs getPorts",
    "SerialAllowAllPortsForUrls",
    "DefaultSerialGuardSetting",
    "no port selected by user",
    "no compatible devices found",
    "web serial picker empty",
    "auto-connect serial port",
    "rp2040 usb cdc",
    "rp2350 usb cdc",
    "pico_enable_stdio_usb",
    "kiosk udev ttyACM",
    "/dev/ttyACM permissions",
    "chromium policy serial",
    "octolux device deploy reset vnc",
    "visible debug strip",
    "CDP page reload screencast stale",
    "Failed to execute open on SerialPort",
    "ntx hmi firmware template",
    "new customer hmi project",
    "ports[0] open fails"
  ],
  "discovery_pitch": "Build a paired NTX HMI (Chromium kiosk on Octolux) + firmware (USB CDC on RP2040 / RP2350) that auto-connects over USB with no permission picker. The single biggest enabler is the Chromium `SerialAllowAllPortsForUrls` policy; the rest is `getPorts()` with VID filter (NOT `requestPort()`), the udev rule for /dev/ttyACM* under Yocto, the visible-debug-strip pattern for Chromium dialogs invisible to VNC, and the SCP + chromium-restart + screencast-bounce deploy sequence. Includes working templates for both firmware (~80 lines C) and HMI (~80 lines JS) plus a 12-row failure-mode table.",
  "metadata": {},
  "created_at": "2026-06-04T16:30:45.036Z",
  "updated_at": "2026-06-04T17:03:36.876Z",
  "source_path": "SKILL.md",
  "sub_skills": [],
  "parent_app": null,
  "readme": "## Known errors this skill resolves\n\nUntil the underlying bugs ship a fix, this skill is the reference for diagnosing and working around the items below. Strike a line off this list when the upstream fix lands.\n\n| | Error / symptom | Where |\n|---|---|---|\n| ☐ | Web Serial picker shows *\"No compatible devices found\"* even though `lsusb` lists the USB CDC device | Symptom #1 |\n| ☐ | `navigator.serial.requestPort()` rejects with `NotFoundError: No port selected by the user` every session | Symptom #2 |\n| ☐ | `navigator.serial.getPorts()` returns ports but `port.open()` throws *\"Failed to execute 'open' on 'SerialPort': Failed to open serial port\"* | Symptom #3 |\n| ☐ | `getPorts().length === 0` even with the policy installed and the device on the kernel bus | Symptom #4 |\n| ☐ | Tap on the Connect button does nothing visible — and you can't see why, because Chromium permission dialogs don't render in the VNC stream | Diagnostics |\n| ☐ | `octolux deploy push` times out (the SCP + chromium-restart + screencast-bounce shell sequence is more reliable) | Deploy |\n| ☐ | `ps aux \\| grep screencast` returns nothing on the device, but screencast IS running on port 8085 (busybox truncates cmdlines) | Deploy |\n| ☐ | Walked `/proc/[0-9]*/net/tcp` to find the screencast PID, `xargs kill` then killed system PIDs and lost SSH | NOT-TO-DO |\n| ☐ | `/tmp/screencast.js` or `/tmp/node_modules/ws/` missing after a bad kill on the device — needs SCP restore from the container repo | Deploy |\n| ☐ | Dashboard VNC tile freezes on a pre-deploy frame after any Chromium restart (screencast's CDP binding to the dead target) | hmi-deploy-reset-vnc |\n| ☐ | `error code: 502` no-body from the device's dashboard endpoint (Adom HTTPS proxy strips 5xx bodies — return 200 + `{ok:false}` instead) | cross-ref [octolux-app-server-debug] |\n\n\n---\n\n---\nname: web-serial-hmi\ndescription: >\n  Build a paired NTX HMI (Chromium kiosk HTML on an Octolux device) +\n  firmware (USB CDC on an RP2040 / RP2350) that auto-connects over USB\n  with no permission picker. Covers the whole loop: project layout under\n  `~/ntx-embedded/customers/<id>/`, Pico SDK USB CDC firmware template\n  with line-based ASCII protocol, HMI-side Web Serial using `getPorts()`\n  + VID filter + auto-connect polling, the Chromium enterprise policy\n  `SerialAllowAllPortsForUrls` (the key enabler — without it\n  `requestPort()` always shows a picker), the udev rule that makes\n  `/dev/ttyACM*` openable by the kiosk's Chromium user, the SCP +\n  chromium-restart + screencast-bounce deploy sequence, a visible\n  page-side debug strip so you can diagnose what Chromium UI dialogs\n  (which don't render in VNC) are doing, and a CDP-via-SSH-tunnel\n  pattern for live page inspection. Includes a working firmware +\n  HMI template (~80 lines of C, ~80 lines of JS) plus a failure-mode\n  reference table. Scope: NTX Octolux devices + RP-series targets.\n  Trigger words: build hmi firmware pair, hmi over usb, web serial\n  hmi, web serial kiosk, octolux web serial, chromium serial port,\n  Web Serial no devices, navigator.serial.getPorts, requestPort vs\n  getPorts, SerialAllowAllPortsForUrls, DefaultSerialGuardSetting,\n  no port selected by user, no compatible devices found, web serial\n  picker empty, auto-connect serial port, rp2040 usb cdc, rp2350 usb\n  cdc, pico_enable_stdio_usb, kiosk udev ttyACM, /dev/ttyACM\n  permissions, chromium policy serial, octolux device deploy reset\n  vnc, visible debug strip, CDP page reload screencast stale, port\n  open failed serial, ntx hmi firmware template, new customer hmi\n  project, ports[0] open fails.\n---\n\n# web-serial-hmi — NTX HMI ↔ USB CDC firmware pattern\n\n> **Scope.** Building a Chromium-kiosk HMI on an **NTX Octolux device** that auto-connects to a **USB-attached RP2040 or RP2350** target over CDC-ACM, with **no Web Serial picker** ever showing. End-to-end working pattern, not a per-customer recipe — instantiate it for any new project under `~/ntx-embedded/customers/<id>/`.\n>\n> Out of scope: Android USB Host (different stack); Web Bluetooth / WebHID; non-RP-series USB devices (mostly the same pattern, just substitute the VID); CAN-FD bridges; the SWD/flash side of the firmware lifecycle — that's covered by `rp2350-flash` and the `target/rp2040.cfg` openocd patterns and the local `firmware/` build is just plain `cmake + make`.\n\n## When to use this skill\n\n- Starting a new customer HMI project that needs to control hardware over a USB-attached MCU.\n- Porting an Android-tablet HMI app to an Octolux Linux/Chromium device and you want the same \"just plug it in\" feel.\n- The Web Serial picker is showing — *\"No compatible devices found\"*, *\"No port selected by user\"*, or just appearing at all — and the project shouldn't have a picker.\n- `requestPort()` works once but the operator has to tap-and-pick every session and that's wrong for a kiosk.\n- The HMI page can't see what's happening because Chromium permission dialogs don't render in VNC.\n\n## Architecture\n\n```\n┌─────────────────────────────────┐      USB-C       ┌──────────────────────┐\n│ NTX Octolux device (10.0.10.x)  │ ◄──────────────► │ RP2040 / RP2350 board│\n│ ─────────────────────────────── │ CDC-ACM, 115200  │ ───────────────────  │\n│ Chromium 109 kiosk @ localhost  │   8N1 line ASCII │ Pico SDK USB CDC     │\n│  ├─ Web Serial API              │                  │  ├─ getchar polling  │\n│  ├─ SerialAllowAllPortsForUrls  │                  │  ├─ ASCII line parser│\n│  └─ getPorts() filtered by VID  │                  │  └─ GPIO drive       │\n│ Apache at /                     │                  │                      │\n│ /dev/ttyACM0 (root:root, 0666)  │                  │                      │\n└─────────────────────────────────┘                  └──────────────────────┘\n        ▲                                                       ▲\n        │  SSH from container (deploy + udev + policy)          │  SWD via PiProbe (flash)\n        │                                                       │\n   Adom Docker container                                  see rp2350-flash\n   ~/ntx-embedded/customers/<id>/{hmi,firmware}\n```\n\nTwo physical links from the workcell to the target board:\n\n- **USB-C** between Octolux and target — runtime command path (this skill).\n- **SWD** between the workcell's PiProbe and the target — flash path (`rp2350-flash`).\n\nThe container is the build host for both: HMI HTML edits go SCP-ed to Octolux; firmware ELF is built locally and flashed via the PiProbe.\n\n## Required components\n\nA working NTX HMI+firmware pair has **five** moving parts. Skip any one and connect won't happen. Pieces 3 and 4 are one-time setup *per Octolux device*; pieces 1, 2, 5 are per-project.\n\n| # | Layer | Where it lives | Per-… |\n|---|---|---|---|\n| 1 | **Firmware** — USB CDC stdio, command parser, GPIO drive | `customers/<id>/firmware/src/main.c` (in container) | project |\n| 2 | **HMI HTML** — connect logic + UI | `customers/<id>/hmi/index.html` (in container, deployed to device) | project |\n| 3 | **Chromium policy** — `SerialAllowAllPortsForUrls` | `/etc/chromium/policies/managed/octolux-serial.json` (on device) | device |\n| 4 | **udev rule** — `MODE=\"0666\"` for VID 0x2e8a | `/etc/udev/rules.d/99-rp-cdc-acm.rules` (on device) | device |\n| 5 | **Deploy** — SCP HMI + chromium restart + screencast bounce | shell pattern run from container | project |\n\n## Quick start — new project, end to end\n\nRun from the Adom container. Substitute `<id>` (e.g. `acme-control`), `<device-ip>` (e.g. `10.0.10.98`), and your domain commands.\n\n### 1. Scaffold\n\n```bash\nID=acme-control\nDEVICE_IP=10.0.10.98\nDEVICE_MODEL='Octolux 15.6\"'\nVIEW_W=1920 VIEW_H=1080\n\nmkdir -p ~/ntx-embedded/customers/$ID/{hmi,firmware/src}\n\ncat > ~/ntx-embedded/customers/$ID/project.json <<EOF\n{\n  \"id\": \"$ID\",\n  \"name\": \"Acme Control\",\n  \"customer\": \"Acme\",\n  \"summary\": \"One-line description of what this HMI does.\",\n  \"accent\": \"#e87a1e\",\n  \"accentDeep\": \"#a64f10\",\n  \"deviceModel\": \"$DEVICE_MODEL\",\n  \"viewport\": { \"w\": $VIEW_W, \"h\": $VIEW_H },\n  \"orientation\": \"landscape\",\n  \"defaultIP\": \"$DEVICE_IP\",\n  \"tagline\": \"one-line operator-facing tagline\"\n}\nEOF\n```\n\n### 2. Drop in the firmware + HMI templates\n\nUse the templates further down this doc verbatim, customizing only:\n\n- **Firmware:** `FW_VERSION_STRING` (the `<NAME> FW x.y.z` boot banner), the `dispatch()` command list, and your specific GPIO pin numbers / domain logic.\n- **HMI:** the visible controls (buttons, sliders, etc.), the brand colors from your `project.json`, the response-line handlers in `handleResponse()`.\n\nKeep the Web Serial connect logic and the visible debug strip **unchanged** — that's the point of this skill.\n\n### 3. Build + flash the firmware (first time)\n\n```bash\nexport PICO_SDK_PATH=$HOME/pico-sdk\ncd ~/ntx-embedded/customers/$ID/firmware\nmkdir -p build && cd build && cmake .. && make -j$(nproc)\n\n# Flash via PiProbe over SWD — pick the cfg for your chip:\nopenocd -f interface/cmsis-dap.cfg -f target/rp2040.cfg \\\n  -c \"adapter speed 5000; program ${ID//-/_}.elf verify reset exit\"\n# (or target/rp2350.cfg for an RP2350 — see rp2350-flash)\n```\n\n### 4. One-time device setup (Chromium policy + udev rule)\n\nSkip if this Octolux device has been set up for Web Serial before. Verify by SSH'ing in: if `/etc/chromium/policies/managed/octolux-serial.json` and `/etc/udev/rules.d/99-rp-cdc-acm.rules` both exist, you're done.\n\n```bash\nssh root@$DEVICE_IP 'bash -s' <<'REMOTE'\nset -e\n# Chromium policy: SerialAllowAllPortsForUrls turns getPorts() into a\n# blanket enumeration for http://localhost; DefaultSerialGuardSetting=2\n# also disables requestPort() pickers as a safety net.\nmkdir -p /etc/chromium/policies/managed\ncat > /etc/chromium/policies/managed/octolux-serial.json <<'POL'\n{\n  \"SerialAllowAllPortsForUrls\": [\"http://localhost\"],\n  \"DefaultSerialGuardSetting\": 2\n}\nPOL\n\n# udev: /dev/ttyACM* is root-only by default on the Yocto distro, and\n# the kiosk Chromium can't open it. MODE=0666 fixes that for any\n# Raspberry Pi (RP2040/RP2350) USB CDC device.\ncat > /etc/udev/rules.d/99-rp-cdc-acm.rules <<'UDEV'\nSUBSYSTEM==\"tty\", ATTRS{idVendor}==\"2e8a\", MODE=\"0666\"\nUDEV\nudevadm control --reload-rules\n\n# Re-process any ALREADY-plugged-in device so the new rule applies now.\n# (Without this, the perms only apply to *future* plug events.)\nBUSPORT=$(lsusb -t 2>/dev/null | awk '/Pico|2e8a/ {sub(/^[ \\t]+/, \"\"); print}' | head -1)\necho \"USB device tree row: ${BUSPORT:-(none seen)}\"\n# Cycle the parent USB device (substitute bus-port from `lsusb -t` if your\n# topology differs; 4-1 is the common Octolux internal hub port).\necho 4-1 > /sys/bus/usb/drivers/usb/unbind 2>/dev/null || true\nsleep 1\necho 4-1 > /sys/bus/usb/drivers/usb/bind 2>/dev/null || true\n\nsystemctl restart octolux-chromium\nREMOTE\n\n# Per the [[hmi-deploy-reset-vnc]] rule: chromium restart leaves the\n# screencast bound to a dead CDP target. Bounce the screencast on the\n# device — see the Deploy sequence below for the exact commands.\n```\n\n### 5. Deploy the HMI to the device\n\nSee the \"Standard deploy sequence\" section below — it's the same shell pattern you'll run on every HMI edit.\n\n### 6. Verify it auto-connected (CDP eval from container)\n\n```bash\nssh -fN -L 9222:localhost:9222 -o ExitOnForwardFailure=yes root@$DEVICE_IP\n\npython3 - <<'PY'\nimport json, urllib.request, websocket\nt = json.loads(urllib.request.urlopen(\"http://localhost:9222/json/list\").read())\npage = next(x for x in t if x.get(\"type\") == \"page\")\nws = websocket.create_connection(page[\"webSocketDebuggerUrl\"])\ndef evx(expr, await_=False):\n    ws.send(json.dumps({\"id\": 1, \"method\": \"Runtime.evaluate\",\n        \"params\": {\"expression\": expr, \"returnByValue\": True,\n                   \"awaitPromise\": await_}}))\n    while True:\n        r = json.loads(ws.recv())\n        if r.get(\"id\") == 1: return r[\"result\"][\"result\"][\"value\"]\nprint(\"connected:\", evx(\"state.connected\"))\nprint(\"fwVersion:\", evx(\"state.fwVersion\"))\nprint(\"ports    :\", evx(\"navigator.serial.getPorts().then(p => p.length)\", await_=True))\nPY\n```\n\nExpected once the firmware is running and the cable is plugged in:\n\n```\nconnected: True\nfwVersion: 1.0.0\nports    : 9          # 8 platform UARTs + your 1 RP-series CDC\n```\n\nIf `connected: True` and `fwVersion` is populated → you're done; the HMI is talking to the MCU. If not, jump to the failure table.\n\n## Firmware template (RP2040 / RP2350 USB CDC)\n\n`firmware/CMakeLists.txt`:\n\n```cmake\ncmake_minimum_required(VERSION 3.13)\nset(PICO_BOARD pico CACHE STRING \"\")            # or \"pico2\" for RP2350\nset(PICO_PLATFORM rp2040 CACHE STRING \"\")       # or \"rp2350\"\ninclude($ENV{PICO_SDK_PATH}/external/pico_sdk_import.cmake)\nproject(<id_underscored> C CXX ASM)\npico_sdk_init()\nadd_executable(<id_underscored> src/main.c)\ntarget_link_libraries(<id_underscored> pico_stdlib hardware_gpio)\npico_enable_stdio_usb(<id_underscored>  1)      # CDC-ACM on USB-C\npico_enable_stdio_uart(<id_underscored> 0)      # no UART — USB only\npico_add_extra_outputs(<id_underscored>)\n```\n\n`firmware/src/main.c`:\n\n```c\n#include <stdio.h>\n#include <string.h>\n#include <ctype.h>\n#include <stdarg.h>\n#include \"pico/stdlib.h\"\n#include \"pico/stdio_usb.h\"      // stdio_usb_connected()\n#include \"hardware/gpio.h\"\n\n#define FW_VERSION_STRING \"<NAME> FW 1.0.0\"\n#define CMD_BUF_SIZE 96\n\n// Status LEDs (wire your board's status LEDs to these or change as needed).\n// LED_USB: solid ON while USB CDC is connected to a host.\n// LED_ACT: 50 ms blink on every valid command.\n// LED_ERR: solid ON when the last response was ERR; clears on next OK.\n// LED_IDLE: solid ON when system is in its safe / inactive state.\n#define LED_USB   9\n#define LED_ACT  10\n#define LED_ERR  16\n#define LED_IDLE 18\n\nstatic char cmd_buf[CMD_BUF_SIZE];\nstatic int  cmd_len = 0;\nstatic bool act_on = false;\nstatic absolute_time_t act_off_at;\n\nstatic void blink_activity(void) {\n    gpio_put(LED_ACT, 1);\n    act_on = true;\n    act_off_at = make_timeout_time_ms(50);\n}\nstatic void say_ok(const char *fmt, ...) {\n    va_list a; va_start(a, fmt);\n    fputs(\"OK \", stdout); vprintf(fmt, a); fputs(\"\\r\\n\", stdout);\n    va_end(a);\n    gpio_put(LED_ERR, 0);\n    blink_activity();\n}\nstatic void say_err(const char *m) {\n    printf(\"ERR %s\\r\\n\", m);\n    gpio_put(LED_ERR, 1);\n    blink_activity();\n}\nstatic void to_upper(char *s) { for (; *s; s++) *s = (char)toupper((unsigned char)*s); }\n\n// === Domain commands ====================================================\n//\n// Replace these with the commands your project actually needs. Keep\n// the canonical built-ins (PING, VERSION, STATUS, HELP) — the HMI\n// template's handshake assumes they exist.\n//\nstatic void dispatch(char *line) {\n    char *tok = strtok(line, \" \\t\");\n    if (!tok) return;\n\n    if (!strcmp(tok, \"PING\"))    { say_ok(\"PONG\"); return; }\n    if (!strcmp(tok, \"VERSION\")) { printf(\"%s\\r\\n\", FW_VERSION_STRING); blink_activity(); return; }\n    if (!strcmp(tok, \"STATUS\"))  { /* print one-line state snapshot */ blink_activity(); return; }\n    if (!strcmp(tok, \"HELP\"))    { /* print \"OK …\" lines */ blink_activity(); return; }\n\n    // your-domain command parsing here, e.g.:\n    //   if (!strcmp(tok, \"DO\")) { … say_ok(\"DO %d\", arg); return; }\n\n    say_err(\"unknown command\");\n}\n\n// === Boilerplate main =================================================\n\nint main(void) {\n    stdio_init_all();\n\n    // Init your GPIOs here (LEDs + outputs). Drive outputs LOW first\n    // so the system boots in a safe state regardless of host behavior.\n    const uint leds[] = { LED_USB, LED_ACT, LED_ERR, LED_IDLE };\n    for (size_t i = 0; i < sizeof(leds)/sizeof(leds[0]); i++) {\n        gpio_init(leds[i]); gpio_set_dir(leds[i], GPIO_OUT); gpio_put(leds[i], 0);\n    }\n    gpio_put(LED_IDLE, 1);\n\n    bool was_conn = false, banner_done = false;\n\n    while (true) {\n        bool conn = stdio_usb_connected();\n        gpio_put(LED_USB, conn ? 1 : 0);\n        if (conn && !was_conn) banner_done = false;\n        was_conn = conn;\n\n        // Boot banner on every CDC connect — the HMI parses this to\n        // populate the FW version pill.\n        if (conn && !banner_done) {\n            sleep_ms(50);                     // give the host's reader a beat\n            printf(\"%s\\r\\n\", FW_VERSION_STRING);\n            puts(\"TYPE 'HELP' FOR COMMANDS\");\n            banner_done = true;\n        }\n\n        // Drain available input into cmd_buf; dispatch on EOL.\n        for (int budget = 0; budget < 32; budget++) {\n            int c = getchar_timeout_us(0);\n            if (c == PICO_ERROR_TIMEOUT) break;\n            if (c == '\\n' || c == '\\r') {\n                if (cmd_len > 0) {\n                    cmd_buf[cmd_len] = '\\0';\n                    to_upper(cmd_buf);\n                    dispatch(cmd_buf);\n                    cmd_len = 0;\n                }\n            } else if (cmd_len < CMD_BUF_SIZE - 1) {\n                cmd_buf[cmd_len++] = (char)c;\n            } else { cmd_len = 0; say_err(\"command too long\"); }\n        }\n\n        // Activity LED timeout.\n        if (act_on && absolute_time_diff_us(get_absolute_time(), act_off_at) <= 0) {\n            gpio_put(LED_ACT, 0); act_on = false;\n        }\n    }\n}\n```\n\n### Protocol conventions that work well\n\n- **Newline-terminated ASCII** — accept `\\n` and `\\r\\n`. Case-insensitive (uppercase before dispatch).\n- **One line in, exactly one line out** — `OK …`, `ERR …`, or a single-token status line. One-to-one keeps the HMI's send/receive trivial.\n- **Standard built-ins always present:**\n  - `PING → PONG` (liveness)\n  - `VERSION → <NAME> FW x.y.z` (capability detection + version pill)\n  - `STATUS → STATE …` (one-line state snapshot)\n  - `HELP → multi-line OK …` (operator self-discovery)\n- **Boot banner on every CDC connect.** Print `<NAME> FW x.y.z\\r\\n` first thing after `stdio_usb_connected()` flips true. The HMI matches `/^([\\w-]+) FW (\\S+)/i` to populate the version pill.\n- **LED1 (USB) mirrors `stdio_usb_connected()`** — instant visual confirmation that the host opened the port.\n- **LED2 (activity) blinks 50 ms on every valid command** — independent visual signal that the firmware *parsed* a command.\n- **Default-safe on boot** — all outputs LOW, critical actuators OFF, idle LED ON. Don't rely on the host to send `ALL OFF` first.\n\n## HMI template\n\n```html\n<script>\n\"use strict\";\n\nconst state = {\n  port: null, writer: null, reader: null,\n  connected: false, fwVersion: null,\n  autoConnectTimer: null,\n};\n\n// ─── Visible debug strip ──────────────────────────────────────────────\n// Mirror every UI event + Web Serial step into the page DOM, because\n// Chromium permission dialogs (the Web Serial picker, geo prompts,\n// etc.) render above the page surface and DON'T appear in the\n// screencast / VNC feed. Without this strip, you'd be blind to what's\n// happening on the kiosk from your dashboard.\nfunction dbg(line) {\n  const strip = document.getElementById('debugStrip');\n  if (!strip) return console.log(line);\n  const d = document.createElement('div');\n  d.textContent = new Date().toISOString().slice(11,23) + '  ' + line;\n  strip.appendChild(d);\n  while (strip.children.length > 5) strip.removeChild(strip.firstChild);\n}\n\n// ─── Line-splitter transformer for the reader stream ──────────────────\nfunction lineSplitter() {\n  let buf = '';\n  return {\n    transform(chunk, ctrl) {\n      buf += chunk;\n      let i;\n      while ((i = buf.search(/\\r?\\n/)) >= 0) {\n        ctrl.enqueue(buf.slice(0, i));\n        buf = buf.slice(i + (buf[i] === '\\r' ? 2 : 1));\n      }\n    },\n  };\n}\n\nasync function openExisting(port) {\n  await port.open({\n    baudRate: 115200, dataBits: 8, stopBits: 1, parity: 'none',\n    bufferSize: 256, flowControl: 'none',\n  });\n  state.port = port;\n\n  const enc = new TextEncoderStream();\n  enc.readable.pipeTo(port.writable);\n  state.writer = enc.writable.getWriter();\n\n  const dec = new TextDecoderStream();\n  port.readable.pipeTo(dec.writable);\n  state.reader = dec.readable\n    .pipeThrough(new TransformStream(lineSplitter()))\n    .getReader();\n\n  state.connected = true;\n  readLoop();\n  handshake();\n}\n\nasync function tryAutoConnect() {\n  if (state.connected || !('serial' in navigator)) return state.connected;\n  const ports = await navigator.serial.getPorts();\n  // The kiosk policy returns the FULL set of /dev/tty* devices the\n  // kernel sees — typically ~8 platform UARTs (empty getInfo()) plus\n  // your real USB CDC. Always filter; never open ports[0] blindly.\n  // 0x2e8a is the Raspberry Pi VID (RP2040 + RP2350). Substitute your\n  // chip's VID if you use a non-RP target.\n  const rp     = ports.filter(p => (p.getInfo() || {}).usbVendorId === 0x2e8a);\n  const anyUsb = ports.filter(p => typeof (p.getInfo() || {}).usbVendorId === 'number');\n  const candidates = rp.length ? rp : anyUsb;\n  dbg(`getPorts() → ${ports.length} total, ${rp.length} RP-series`);\n  for (const port of candidates) {\n    try {\n      await openExisting(port);\n      const info = port.getInfo();\n      dbg(`auto-connected to VID:${(info.usbVendorId||0).toString(16)} PID:${(info.usbProductId||0).toString(16)}`);\n      return true;\n    } catch (e) {\n      dbg('  open failed on one candidate: ' + e.message);\n    }\n  }\n  return false;\n}\n\nwindow.addEventListener('load', async () => {\n  // Install global capture-phase event listeners so we can see every\n  // tap on the page, even ones that don't land on a button.\n  ['pointerdown','pointerup','touchstart','touchend',\n   'mousedown','mouseup','click'].forEach(k => {\n    document.addEventListener(k, e => {\n      const tgt = e.target ? (e.target.id || e.target.tagName) : '?';\n      dbg(`${k}  (${(e.clientX||0)|0},${(e.clientY||0)|0}) → ${tgt}  trusted=${e.isTrusted}`);\n    }, /* useCapture */ true);\n  });\n  dbg('debug listeners installed');\n\n  if (!(await tryAutoConnect())) {\n    // Keep polling — if the cable is plugged in *after* the page loads,\n    // the next tick picks it up. Cheap (one promise every 2 s).\n    state.autoConnectTimer = setInterval(async () => {\n      if (await tryAutoConnect() && state.autoConnectTimer) {\n        clearInterval(state.autoConnectTimer);\n        state.autoConnectTimer = null;\n      }\n    }, 2000);\n  }\n});\n\nasync function readLoop() {\n  while (state.reader) {\n    try {\n      const { value, done } = await state.reader.read();\n      if (done) { state.connected = false; break; }\n      if (value) handleResponse(value.trim());\n    } catch { state.connected = false; break; }\n  }\n}\n\nasync function handshake() {\n  await send('PING');\n  await send('VERSION');\n  await send('STATUS');\n}\n\nasync function send(cmd) {\n  if (!state.writer) return;\n  dbg('→ ' + cmd);\n  await state.writer.write(cmd + '\\n');\n}\n\nfunction handleResponse(line) {\n  dbg('← ' + line);\n  const m = line.match(/^([\\w-]+) FW (\\S+)/i);\n  if (m) state.fwVersion = m[2];\n  // Your domain handlers here (parse STATUS reply, react to OK/ERR, etc.)\n}\n</script>\n```\n\nThe HMI's HTML body and CSS are entirely up to your project — see `octolux-hmi-design` for the brand-language guide (dark mode, accent colors, touch-target sizes, Inter font). The only DOM element this template requires is `<div id=\"debugStrip\"></div>` somewhere visible during development. You can hide it (or remove the global tap listeners) after the project ships.\n\n## One-time Chromium policy — the single biggest enabler\n\n> Without `SerialAllowAllPortsForUrls`, `requestPort()` shows a picker, the picker isn't visible through VNC, and the kiosk is unreliable. With it, `getPorts()` returns every kernel serial device with no user gesture.\n\n```json\n{\n  \"SerialAllowAllPortsForUrls\": [\"http://localhost\"],\n  \"DefaultSerialGuardSetting\": 2\n}\n```\n\n- **`SerialAllowAllPortsForUrls: [\"http://localhost\"]`** — pages served from `http://localhost` (which is where the Octolux's Apache serves the HMI) get blanket permission to enumerate and open serial ports. `getPorts()` now returns the full list.\n- **`DefaultSerialGuardSetting: 2`** — `2` means *\"Do not allow any site to request access\"*. Belt-and-suspenders: if your code accidentally calls `requestPort()`, the picker is a no-op instead of presenting UI nobody can see.\n\nPath: `/etc/chromium/policies/managed/octolux-serial.json`. Restart Chromium after install (`systemctl restart octolux-chromium`). After restart, you **must** also bounce the screencast (see `hmi-deploy-reset-vnc`).\n\nVerify the policy is live via CDP eval:\n\n```javascript\nnavigator.serial.getPorts().then(p => p.length)\n// Expected: a positive integer (typically 9 on Octolux 15.6\") when an RP\n// device is plugged in. Zero means the policy didn't load — re-check\n// the file path and the chromium restart.\n```\n\n## One-time udev rule\n\nThe Octolux Yocto distro ships `/dev/ttyACM0` as `-rw-r--r-- root:root`. The kiosk Chromium can't open it. Fix with a rule that targets the RP-series VID:\n\n```bash\ncat > /etc/udev/rules.d/99-rp-cdc-acm.rules <<'EOF'\nSUBSYSTEM==\"tty\", ATTRS{idVendor}==\"2e8a\", MODE=\"0666\"\nEOF\nudevadm control --reload-rules\n```\n\nFor an **already-plugged** device, the rule doesn't retroactively re-permission. Either physically unplug+replug, or cycle the USB driver from a shell:\n\n```bash\necho 4-1 > /sys/bus/usb/drivers/usb/unbind\nsleep 1\necho 4-1 > /sys/bus/usb/drivers/usb/bind\n```\n\n(`4-1` is the typical Octolux internal hub port; substitute from `lsusb -t` if your topology differs.)\n\nNote: the Yocto distro lacks the `usb_id` udev helper, so `udevadm info /dev/ttyACM0` returns *\"Unknown device: Inappropriate ioctl\"*. **That's fine.** Chromium reads `getInfo().usbVendorId` from the underlying USB descriptor, not from udev's metadata database. You only need the permission rule, not metadata population.\n\n## Standard deploy sequence\n\nEvery HMI edit goes through this. Don't skip the screencast bounce — see `hmi-deploy-reset-vnc`.\n\n```bash\nID=acme-control\nDEVICE_IP=10.0.10.98\n\n# 1. Push the HTML\nscp ~/ntx-embedded/customers/$ID/hmi/index.html \\\n    root@$DEVICE_IP:/usr/share/apache2/default-site/htdocs/index.html\n\n# 2. Reload the kiosk — restart chromium pulls the new HTML AND\n#    reloads any new chromium policy / udev rule deployed alongside.\nssh root@$DEVICE_IP 'systemctl restart octolux-chromium'\n\n# 3. Wait for the kiosk to come back\nsleep 10\n\n# 4. Bounce the screencast — chromium restart killed its CDP target\n#    binding and the screencast won't auto-reattach. Without this,\n#    the dashboard VNC tile freezes on a pre-deploy frame.\n#    NOTE: busybox `ps aux` truncates cmdlines so `ps aux | grep\n#    screencast` returns NOTHING even when it's running. Walk /proc\n#    instead. And NEVER walk /proc/PID/net/tcp to find port owners —\n#    those views are namespace-wide and you will mass-kill systemd.\nssh root@$DEVICE_IP 'bash -s' <<'REMOTE'\nfor d in /proc/[0-9]*; do\n  pid=${d##*/}\n  grep -q 'screencast' \"$d/cmdline\" 2>/dev/null && kill -9 \"$pid\" 2>/dev/null\ndone\nsleep 2\ncd /tmp && nohup node screencast.js --quality 80 --pipeline 4 > /tmp/screencast.log 2>&1 &\nREMOTE\n\n# 5. (optional) confirm the dashboard VNC is alive — successive snaps\n#    should differ unless the page is genuinely static.\ncurl -s http://$DEVICE_IP:8085/snapshot -o /tmp/a.jpg -w \"%{size_download}\\n\"\nsleep 2\ncurl -s http://$DEVICE_IP:8085/snapshot -o /tmp/b.jpg -w \"%{size_download}\\n\"\n```\n\nIf `/tmp/screencast.js` or `/tmp/node_modules/ws/` ever go missing on the device (they live in tmpfs and can be wiped by a bad kill spree), SCP them back from the container:\n\n```bash\nscp ~/ntx-embedded/apps/octolux/ui/screencast/screencast.js \\\n    root@$DEVICE_IP:/tmp/screencast.js\ncd ~/ntx-embedded/apps/octolux/node_modules && tar cz ws | \\\n  ssh root@$DEVICE_IP 'mkdir -p /tmp/node_modules && cd /tmp/node_modules && tar xz'\n```\n\n## Live diagnostics — CDP from the container\n\nThe Octolux kiosk exposes Chrome DevTools at `localhost:9222`. Tunnel it back:\n\n```bash\nssh -fN -L 9222:localhost:9222 -o ExitOnForwardFailure=yes root@$DEVICE_IP\n\n# install python websocket client once\npip install --user --break-system-packages websocket-client\n```\n\nThen any time you want to inspect the live page state:\n\n```python\nimport json, urllib.request, websocket\nt = json.loads(urllib.request.urlopen(\"http://localhost:9222/json/list\").read())\npage = next(x for x in t if x.get(\"type\") == \"page\")\nws = websocket.create_connection(page[\"webSocketDebuggerUrl\"])\ndef cdp(method, params=None, _id=[0]):\n    _id[0] += 1\n    ws.send(json.dumps({\"id\": _id[0], \"method\": method, \"params\": params or {}}))\n    while True:\n        r = json.loads(ws.recv())\n        if r.get(\"id\") == _id[0]: return r\ndef evx(expr, await_=False):\n    return cdp(\"Runtime.evaluate\", {\"expression\": expr, \"returnByValue\": True, \"awaitPromise\": await_})[\"result\"][\"result\"][\"value\"]\n\n# Useful queries:\nevx(\"state.connected\")\nevx(\"state.fwVersion\")\nevx(\"navigator.serial.getPorts().then(p => p.length)\", await_=True)\nevx(\"Array.from(document.querySelectorAll('#debugStrip > div')).map(d=>d.textContent).join('\\\\n')\")\n\n# Force a reload (and remember: bounce screencast afterwards!):\ncdp(\"Page.reload\", {\"ignoreCache\": True})\n```\n\n### Hit-test if a tap doesn't fire the button\n\n```javascript\nconst b = document.getElementById('btnConnect');\nconst r = b.getBoundingClientRect();\nconst hit = document.elementFromPoint(r.left + r.width/2, r.top + r.height/2);\nJSON.stringify({ same: hit === b, hit_id: hit?.id, hit_tag: hit?.tagName });\n```\n\nIf `same` is false, something is covering the button.\n\n## Failure-mode reference\n\nEvery row in this table corresponds to a real failure observed during this pattern's development. Use it as the diagnostic flow.\n\n| Symptom | Root cause | Fix |\n|---|---|---|\n| Picker shows *\"No compatible devices found\"* | Chromium policy not in place AND/OR `/dev/ttyACM*` not world-readable | Install both the policy and the udev rule; restart chromium; bounce screencast |\n| Picker shows the device but every connect ends in *NotFoundError: No port selected by user* | User dismissed the picker, or `requestPort()` was called from a non-user-gesture context (programmatic CDP `.click()` triggers this) | Don't use `requestPort()` at all — use the `getPorts()` auto-connect pattern in the HMI template |\n| `getPorts()` returns ports but `port.open()` throws *Failed to execute 'open' on 'SerialPort': Failed to open serial port* | You opened `ports[0]`; it's a platform UART without a backing device | Filter by `getInfo().usbVendorId === 0x2e8a` before opening |\n| `getPorts().length === 0` even though `lsusb` shows the device on the kernel side | Policy file missing, malformed, or wrong path; OR Chromium wasn't restarted after the policy install; OR `/dev/ttyACM0` perms are still root-only | Re-check `/etc/chromium/policies/managed/octolux-serial.json` is valid JSON; restart chromium; `ls -la /dev/ttyACM0` should be `0666` |\n| Tap on a button does nothing — no visible page reaction | Touch input might not be reaching the page; check via the debug strip (it should log `pointerdown` / `click`). If those don't appear → input layer (Weston/touchscreen) issue. If they do appear but `connect() called` doesn't → the click hit something covering the button — use `elementFromPoint` | Diagnose via debug strip + hit-test |\n| The debug strip shows `connect() called → requestPort()...` and hangs — then eventually fails *\"No port selected by user\"* | The HMI still has `requestPort()` somewhere | Replace with `getPorts()` + VID filter (HMI template) |\n| Page reloaded via CDP `Page.reload` — VNC dashboard tile froze on the pre-reload frame | The screencast bound to the old CDP target ID; reload creates a new one; the binding is dead | Bounce screencast (Standard deploy sequence step 4). Applies to ANY Chromium target change — restart, reload, navigate |\n| `octolux deploy push` times out | Octolux server's deploy API is flaky on this build | Use the SCP + restart bash sequence directly (it's what the API does internally anyway) |\n| `ps aux \\| grep screencast` returns nothing but screencast IS running and bound to port 8085 | Busybox `ps aux` truncates cmdlines | Walk `/proc/[0-9]*/cmdline` to find PIDs by argv match |\n| Killed a system PID and lost SSH / had to recover the device | You walked `/proc/[0-9]*/net/tcp` to find owners of port 8085 — those views are namespace-wide and match every PID | **NEVER** identify port owners via `/proc/PID/net/tcp` on this distro. Use `/proc/PID/cmdline` matching instead |\n| `/tmp/screencast.js` or `/tmp/node_modules/ws/` missing after a bad kill / reboot | They live in tmpfs and the original placement path doesn't auto-restore | SCP from `~/ntx-embedded/apps/octolux/ui/screencast/screencast.js` and tar+SCP `~/ntx-embedded/apps/octolux/node_modules/ws/` |\n| Adom HTTPS proxy returns `error code: 502` with no body when the HMI's own dashboard endpoint fails | The Adom proxy strips bodies on 5xx (Cloudflare-style stock error page) | Have the HMI server return HTTP 200 with `{ok:false, error, hint}` instead of 5xx — see `octolux-app-debug` Symptom 6 |\n| Dashboard section button (VNC / Deploy / Stream) does nothing | Different issue — Hydrogen browser session disconnected or server.js handler — see `octolux-app-debug` Symptoms 5 and 6 | Not in this skill |\n\n## Things NOT to do\n\n- **Don't blindly `open(ports[0])`** — always filter by VID. The kiosk policy returns the full set of `/dev/tty*` and most of them are platform UARTs that won't open.\n- **Don't rely on `requestPort()`** — it's user-gesture-gated and shows a picker, and the picker is invisible to VNC, and the operator shouldn't have to tap anyway.\n- **Don't walk `/proc/[0-9]*/net/tcp`** to find what's bound to a port. Namespace-wide view → `xargs kill` hits systemd PIDs.\n- **Don't `pkill node`** on the kiosk. node-red, the screencast, and other helpers all run as `node`. Same rule that applies on the Adom container.\n- **Don't restart Chromium without resetting screencast** — see `hmi-deploy-reset-vnc`. Same applies to CDP `Page.reload` and `Page.navigate`.\n- **Don't return 5xx** for app errors that need an explanatory body — the Adom HTTPS proxy strips bodies on 5xx. Return HTTP 200 with `{ok: false, error, hint}`.\n- **Don't assume udev metadata is populated** on the Yocto distro — `udevadm info /dev/ttyACM0` returns *\"Inappropriate ioctl\"*. Doesn't matter; Chromium reads VID/PID from the USB descriptor via `getInfo()`.\n\n## Related skills\n\n- `hmi-deploy-reset-vnc` (auto-memory) — the screencast-bounce rule. Triggered by ANY Chromium target change.\n- `octolux-hmi-design` — visual design language (dark mode, accent colors, touch targets, Inter font) for the HTML body and CSS that wraps this skill's connect logic.\n- `octolux-platform` — Octolux device-side reference (Weston, Chromium kiosk launcher, Apache config, paths).\n- `octolux-deploy` — the wider HMI-deploy story; this skill's deploy sequence is a more reliable shell-equivalent of `octolux deploy push`.\n- `octolux-app-debug` — when the issue is the Octolux Node server / dashboard, not the HMI/firmware pair.\n- `rp2350-flash` — the SWD/flash side of the firmware lifecycle. This skill is the USB-runtime side; that one is the flash side.\n"
}