← Commit history

Initial commit: skill/pcl-data-profile

Kyle Bergstedt ·9b73732cdb ·17d ago
2 files changed +262
README.mdadded+215
@@ -0,0 +1,215 @@+# Data Generation at the Adom PCL Node++**Adom Industries — NSF PCL Test Bed node, Fort Worth, Texas.**++This page is the physical shape of the data our node produces: every instrument+class that generates data, the file types each one writes, and the rate at which+it writes them. It is written for the people planning storage, transport, and+deposition for the PCL network, and it is kept current as instruments come+online.++Our node is an electronics-prototyping cloud lab. The instruments below are the+ones attached to our automated workcells (a UV fiber laser cell and a+self-driving probing cell) and to the InstaPCB design-build-test loop. Every+capture lands on the Adom Wiki alongside the skill that explains how to read it,+so a dataset and its interpreter are published as one unit.++---++## Summary++| | |+|---|---|+| Site uplink | **28 Gbps** (factory, Fort Worth) |+| Highest single-instrument burst rate | **5 Gbps** (spectrum analyzer I/Q) |+| Dominant steady-state producer | Continuous 4K60 video, per camera |+| Dominant burst producer | Spectrum analyzer and oscilloscope streaming captures |+| Publication target | Adom Wiki page + skill per dataset (`wiki.adom.inc`) |++---++## Instruments and file types++### Video — high-resolution live cameras++Cameras record continuously during use and simultaneously stream a lower+resolution to remote users.++| | |+|---|---|+| Record | 4K (3840×2160) at 60 fps |+| Stream to users | 1080p or 1440p at 60 fps |+| Codec | H.264 / AVC |+| Container | `.mkv`, `.ts` |++The stream is a live view and is not retained; the 4K60 record is the archived+artifact.++### Oscilloscopes — MXR-series++| | |+|---|---|+| Interface rate | **1 Gbps** streaming |+| Files | `.csv` (waveform tables), `.wav` (waveform as sampled audio-rate stream), `.png` (screen captures) |++Used in the probing cell for per-board bring-up and characterization. Output is+bursty: short high-rate acquisitions around each device under test, not a+continuous stream.++### Thermal cameras — FLIR++| | |+|---|---|+| Files | `.jpg` (radiometric) |++Used for thermal imaging of boards under load. Frame-rate is set per experiment,+from single shots to continuous capture during a thermal soak.++### Spectrum analyzers++| | |+|---|---|+| Interface rate | **5 Gbps** streaming |+| Native | custom binary format (`.bin`) |+| Exports | `.csv`, `.sta` |++Our highest-rate instrument. Native `.bin` is the capture format; `.csv` and+`.sta` are the exported, portable forms we publish.++### Microphones++| | |+|---|---|+| Sample rate | 44.1 kHz |+| Data rate | 176.4 kB/s |+| Files | `.wav` |++Acoustic capture of the workcells — a useful and cheap channel for detecting+mechanical anomalies during automated runs.++### Power sensors++| | |+|---|---|+| Files | `.wav` |++Power is captured as an audio-rate waveform stream, so the same tooling that+reads microphone and oscilloscope `.wav` output reads power traces too.++### Environmental — Bosch BME690++Barometric pressure, temperature, humidity.++| | |+|---|---|+| Files | `.csv`, `.txt`, `.bin`, `.json`, `.jsonl` |++Low-rate ambient context logged alongside every run.++### Motion — Bosch BMI270++6-DOF IMU (3-axis accelerometer + 3-axis gyroscope).++| | |+|---|---|+| Files | `.csv`, `.txt`, `.bin`, `.json`, `.jsonl` |++Vibration and motion of the workcells and of devices under test.++### Precision voltage and current sensing++| | |+|---|---|+| Resolution | **µV / µA** |+| Files | `.csv`, `.bin`, `.json`, `.jsonl`, `.wav` |++High-resolution electrical measurement across the probing cell. `.wav` is used+when the measurement is a continuous waveform; `.csv`/`.jsonl` when it is a+sampled log.++### Custom user sensors++Because our node exists to let other labs prototype electronics, a user can put+**any** component on a board we fabricate and read it back through our stack. The+instrument list above is our fixed inventory; the sensors a *user* brings are+open-ended, and their formats follow the same menu (`.csv`, `.bin`, `.json`,+`.jsonl`, `.wav`, and images).++---++## Rate estimates++Every figure in this section is an **estimate**, derived from the instrument+rates above and our expected duty cycles. Actual volume depends on how much of+each day the workcells are running. We are publishing our real measured figures+once InstaPCB goes live in January.++Assumptions: "continuous" = 24 h/day; "active" = 8 h/day of workcell operation.++![Estimated data volume per instrument, GB/day on a log scale](docs/rates.png)++| Instrument | Rate | Per active hour | Per day |+|---|---|---|---|+| 4K60 camera (per camera) | ~50–80 Mbps H.264 | ~25–35 GB | ~200–290 GB active, ~650 GB continuous |+| Spectrum analyzer | 5 Gbps peak | 2.25 TB if sustained | burst-limited; tens of GB/day typical, TB/day if streamed |+| Oscilloscope (MXR) | 1 Gbps peak | 450 GB if sustained | burst-limited; single-GB to tens of GB/day typical |+| FLIR thermal | ~0.5–1.5 MB/frame | ~2–5 GB at 1 fps | MB/day for single shots to ~40 GB/day continuous |+| Microphone (per mic) | 176.4 kB/s | ~0.6 GB | ~15 GB continuous |+| Power sensor (per channel) | audio-rate `.wav` | ~0.6–2 GB | ~15–50 GB continuous |+| µV/µA voltage + current | sample-rate dependent | ~0.1–3 GB | sub-GB logging to ~70 GB/day at 100 kHz |+| BMI270 IMU (per unit) | 16 B/sample binary | ~1.5 MB at 100 Hz, ~150 MB at 1.6 kHz | ~140 MB to ~2.2 GB continuous (binary); ~10× that as JSONL |+| BME690 (per unit) | ~12 B/sample binary | negligible | ~1 MB/day at 1 Hz binary; ~90 MB/day at 10 Hz JSONL |+| Custom user sensors | user-defined | — | unbounded |++### What this adds up to++The daily total is set almost entirely by two decisions: how many cameras are+recording, and whether the spectrum analyzer and scope are streaming or+snapshotting.++- **Typical instrumented day** (a handful of cameras, burst scope/analyzer+  captures, continuous low-rate sensors): **~1–3 TB/day**.+- **Heavy day** (all cameras continuous, sustained analyzer streaming):+  **10+ TB/day**, bounded by the 28 Gbps uplink rather than by the instruments.++The 28 Gbps uplink is the real ceiling: 28 Gbps is ~302 TB/day if saturated, so+the network is not the binding constraint on a typical day, but a single+sustained 5 Gbps spectrum capture consumes about a fifth of it.++---++## How we publish it++Every dataset above is published on the Adom Wiki as a page with a persistent+identifier, named authors, a per-page license, git-backed version history,+captured provenance, public comments and pull requests, and machine-readable+metadata behind a REST API. Datasets go up the moment they exist rather than+when a paper is written.++Critically, each dataset ships with the **skill** that explains it — the+instructions any AI or person needs to read the format and reuse the data. That+is how we avoid imposing a single format mandate across ten instrument classes+with very different native outputs.++Browse the live wiki at [wiki.adom.inc/skills](https://wiki.adom.inc/skills).++---++## Open questions we would welcome help on++These are the gaps between what we do today and a full open-science pipeline:++- We do not yet mint DOIs.+- We do not yet deposit to Zenodo, protocols.io, or The Stacks.+- We have no funder or ROR field and no output register.+- Licenses default to MIT for code; we have nothing in place yet for CC0 data or+  CC BY protocols.+- Experiments are not yet packaged as publishable units (method, result,+  resource, negative data) with contributor roles and a citation.++We would like the deposition pipeline in place before InstaPCB goes live in+January and the experiment volume amplifies.++---++*Contact: Kyle Bergstedt (co-PI, AI lead) — [email protected] · John Lauer (PI) — [email protected]*
page.jsonadded+47
@@ -0,0 +1,47 @@+{+  "slug": "pcl-data-profile",+  "title": "Adom PCL Node Data Profile",+  "type": "skill",+  "version": "0.1.0",+  "description": "The physical shape of the data produced by the Adom Industries NSF PCL Test Bed node: every instrument that generates data, the file types each writes, and the rate it writes them at.",+  "tags": [+    "nsf-pcl",+    "open-science",+    "data",+    "instruments",+    "metrology"+  ],+  "dependencies": {},+  "discovery_triggers": [+    "what data does the adom node produce",+    "instrument file formats at adom",+    "how much data per day does the lab generate",+    "adom pcl data rates",+    "nsf pcl test bed data profile"+  ],+  "discovery_pitch": "Reach for this when you need the instrument inventory, file formats, or data-rate estimates for the Adom Industries NSF PCL Test Bed node.",+  "sample_prompts": [+    {+      "label": "Formats",+      "prompt": "what file formats does the oscilloscope at the Adom node write?"+    },+    {+      "label": "Rates",+      "prompt": "how many GB per day does the Adom PCL node generate?"+    }+  ],+  "hero": {+    "type": "image",+    "path": "docs/hero.png"+  },+  "scripts": {+    "install": "./install.sh",+    "uninstall": "./uninstall.sh"+  },+  "visibility": "public",+  "org": "adom",+  "author": {+    "name": "Kyle Bergstedt",+    "email": "[email protected]"+  }+}\ No newline at end of file