← Commit history

Publish 0.1.3

Kyle Bergstedt ·55ad96048a ·17d ago ·parent e3b972f
4 files changed +85−178
README.md+83−176
@@ -2,213 +2,120 @@  **Adom Industries — NSF PCL Test Bed node, Fort Worth, Texas.** -This page is the physical shape of the data our node produces: every instrument-class that generates data, the file types each one writes, and the rate at which-it writes them. It is written for the people planning storage, transport, and-deposition for the PCL network, and it is kept current as instruments come-online.--Our node is an electronics-prototyping cloud lab. The instruments below are the-ones attached to our automated workcells (a UV fiber laser cell and a-self-driving probing cell) and to the InstaPCB design-build-test loop. Every-capture lands on the Adom Wiki alongside the skill that explains how to read it,-so a dataset and its interpreter are published as one unit.+The instruments at our node that generate data, the file types each produces, and+the rate at which they produce it — today, and projected as we scale. ----+Our node is an electronics-prototyping cloud lab. Instruments fall into two+groups, and the distinction is what drives every number below: -## Summary+- **Per-workcell instruments** — one of each per workcell. These scale linearly+  with workcell count, and they dominate our volume.+- **Bench instruments** — shared across the lab. These are bursty, not+  continuous, and they do not scale with workcell count. -| | |-|---|---|-| Site uplink | **28 Gbps** (factory, Fort Worth) |-| Highest single-instrument burst rate | **5 Gbps** (spectrum analyzer I/Q) |-| Dominant steady-state producer | Continuous 4K60 video, per camera |-| Dominant burst producer | Spectrum analyzer and oscilloscope streaming captures |-| Publication target | Adom Wiki page + skill per dataset (`wiki.adom.inc`) |+**We run 20 workcells today, each with one camera.** We anticipate thousands,+plausibly tens of thousands.  ---  ## Instruments and file types -### Video — high-resolution live cameras--Cameras record continuously during use and simultaneously stream a lower-resolution to remote users.--| | |-|---|---|-| Record | 4K (3840×2160) at 60 fps |-| Stream to users | 1080p or 1440p at 60 fps |-| Codec | H.264 / AVC |-| Container | `.mkv`, `.ts` |--The stream is a live view and is not retained; the 4K60 record is the archived-artifact.--### Oscilloscopes — MXR-series--| | |-|---|---|-| Interface rate | **1 Gbps** streaming |-| Files | `.csv` (waveform tables), `.wav` (waveform as sampled audio-rate stream), `.png` (screen captures) |--Used in the probing cell for per-board bring-up and characterization. Output is-bursty: short high-rate acquisitions around each device under test, not a-continuous stream.--### Thermal cameras — FLIR--| | |-|---|---|-| Files | `.jpg` (radiometric) |--Used for thermal imaging of boards under load. Frame-rate is set per experiment,-from single shots to continuous capture during a thermal soak.--### Spectrum analyzers--| | |-|---|---|-| Interface rate | **5 Gbps** streaming |-| Native | custom binary format (`.bin`) |-| Exports | `.csv`, `.sta` |--Our highest-rate instrument. Native `.bin` is the capture format; `.csv` and-`.sta` are the exported, portable forms we publish.--### Microphones--| | |-|---|---|-| Sample rate | 44.1 kHz |-| Data rate | 176.4 kB/s |-| Files | `.wav` |--Acoustic capture of the workcells — a useful and cheap channel for detecting-mechanical anomalies during automated runs.--### Power sensors--| | |-|---|---|-| Files | `.wav` |--Power is captured as an audio-rate waveform stream, so the same tooling that-reads microphone and oscilloscope `.wav` output reads power traces too.+### Per-workcell instruments -### Environmental — Bosch BME690+| Instrument | File types | Rate |+|---|---|---|+| High-resolution live camera | `.mkv`, `.ts` (H.264/AVC) | records 4K (3840×2160) at 60 fps, ~50–80 Mbps; simultaneously streams 1080p or 1440p at 60 fps to users |+| Microphone | `.wav` | 44.1 kHz, 176.4 kB/s |+| Power sensor | `.wav` | audio-rate waveform stream |+| BME690 — barometric pressure, temperature, humidity | `.csv`, `.txt`, `.bin`, `.json`, `.jsonl` | low rate, typically 1–10 Hz |+| BMI270 — 6-DOF IMU (accelerometer + gyroscope) | `.csv`, `.txt`, `.bin`, `.json`, `.jsonl` | up to 1.6 kHz ODR | -Barometric pressure, temperature, humidity.+The camera stream is a live view for remote users and is not retained; the 4K60+record is the archived artifact. -| | |-|---|---|-| Files | `.csv`, `.txt`, `.bin`, `.json`, `.jsonl` |--Low-rate ambient context logged alongside every run.--### Motion — Bosch BMI270--6-DOF IMU (3-axis accelerometer + 3-axis gyroscope).--| | |-|---|---|-| Files | `.csv`, `.txt`, `.bin`, `.json`, `.jsonl` |+### Bench instruments -Vibration and motion of the workcells and of devices under test.+| Instrument | File types | Rate |+|---|---|---|+| Oscilloscopes — MXR series | `.csv`, `.wav`, `.png` | 1 Gbps streaming |+| Spectrum analyzers | custom binary (`.bin`); exports `.csv`, `.sta` | 5 Gbps streaming |+| FLIR thermal cameras | `.jpg` (radiometric) | frame rate set per experiment |+| µV/µA voltage and current sensing | `.csv`, `.bin`, `.json`, `.jsonl`, `.wav` | sample-rate dependent | -### Precision voltage and current sensing--| | |-|---|---|-| Resolution | **µV / µA** |-| Files | `.csv`, `.bin`, `.json`, `.jsonl`, `.wav` |--High-resolution electrical measurement across the probing cell. `.wav` is used-when the measurement is a continuous waveform; `.csv`/`.jsonl` when it is a-sampled log.+These are our highest-rate instruments by interface speed, but they capture in+short bursts around a device under test rather than streaming continuously, so+their contribution to daily volume is far smaller than the camera fleet's.  ### Custom user sensors -Because our node exists to let other labs prototype electronics, a user can put-**any** component on a board we fabricate and read it back through our stack. The-instrument list above is our fixed inventory; the sensors a *user* brings are-open-ended, and their formats follow the same menu (`.csv`, `.bin`, `.json`,-`.jsonl`, `.wav`, and images).+Our node exists to let other labs prototype electronics, so a user can put **any**+component on a board we fabricate and read it back through our stack. The lists+above are our fixed inventory; the sensors a *user* brings are open-ended. Their+formats follow the same menu: `.csv`, `.bin`, `.json`, `.jsonl`, `.wav`, and+images.  --- -## Rate estimates+## Rates -Every figure in this section is an **estimate**, derived from the instrument-rates above and our expected duty cycles. Actual volume depends on how much of-each day the workcells are running. We are publishing our real measured figures-once InstaPCB goes live in January.+Per-unit, per-day estimates for each instrument. Assumes 8 active hours per day. -Assumptions: "continuous" = 24 h/day; "active" = 8 h/day of workcell operation.+![Estimated data volume per instrument, GB/day per unit on a log scale](docs/rates.png) -![Estimated data volume per instrument, GB/day on a log scale](docs/rates.png)--| Instrument | Rate | Per active hour | Per day |-|---|---|---|---|-| 4K60 camera (per camera) | ~50–80 Mbps H.264 | ~25–35 GB | ~200–290 GB active, ~650 GB continuous |-| Spectrum analyzer | 5 Gbps peak | 2.25 TB if sustained | burst-limited; tens of GB/day typical, TB/day if streamed |-| Oscilloscope (MXR) | 1 Gbps peak | 450 GB if sustained | burst-limited; single-GB to tens of GB/day typical |-| FLIR thermal | ~0.5–1.5 MB/frame | ~2–5 GB at 1 fps | MB/day for single shots to ~40 GB/day continuous |-| Microphone (per mic) | 176.4 kB/s | ~0.6 GB | ~15 GB continuous |-| Power sensor (per channel) | audio-rate `.wav` | ~0.6–2 GB | ~15–50 GB continuous |-| µV/µA voltage + current | sample-rate dependent | ~0.1–3 GB | sub-GB logging to ~70 GB/day at 100 kHz |-| BMI270 IMU (per unit) | 16 B/sample binary | ~1.5 MB at 100 Hz, ~150 MB at 1.6 kHz | ~140 MB to ~2.2 GB continuous (binary); ~10× that as JSONL |-| BME690 (per unit) | ~12 B/sample binary | negligible | ~1 MB/day at 1 Hz binary; ~90 MB/day at 10 Hz JSONL |-| Custom user sensors | user-defined | — | unbounded |--### What this adds up to--The daily total is set almost entirely by two decisions: how many cameras are-recording, and whether the spectrum analyzer and scope are streaming or-snapshotting.--- **Typical instrumented day** (a handful of cameras, burst scope/analyzer-  captures, continuous low-rate sensors): **~1–3 TB/day**.-- **Heavy day** (all cameras continuous, sustained analyzer streaming):-  **10+ TB/day**, bounded by the 28 Gbps uplink rather than by the instruments.--The 28 Gbps uplink is the real ceiling: 28 Gbps is ~302 TB/day if saturated, so-the network is not the binding constraint on a typical day, but a single-sustained 5 Gbps spectrum capture consumes about a fifth of it.+| Instrument | Per unit, per active day |+|---|---|+| 4K60 camera | ~216 GB (~650 GB if run continuously) |+| Power sensor | ~17 GB |+| Microphone | ~5 GB |+| BMI270 IMU | ~46 MB at 100 Hz, ~740 MB at 1.6 kHz (binary; ~10× as JSONL) |+| BME690 | under 30 MB |+| **Per-workcell subtotal** | **~240 GB** |+| Oscilloscope (MXR) | burst-limited; single-GB to tens of GB |+| Spectrum analyzer | burst-limited; tens of GB, TB-scale if streamed |+| FLIR thermal | MB for single shots, up to ~40 GB continuous |+| µV/µA sensing | sub-GB logging to ~70 GB at 100 kHz |+| Custom user sensors | user-defined; unbounded |  --- -## How we publish it--Every dataset above is published on the Adom Wiki as a page with a persistent-identifier, named authors, a per-page license, git-backed version history,-captured provenance, public comments and pull requests, and machine-readable-metadata behind a REST API. Datasets go up the moment they exist rather than-when a paper is written.+## Current vs. projected -Critically, each dataset ships with the **skill** that explains it — the-instructions any AI or person needs to read the format and reuse the data. That-is how we avoid imposing a single format mandate across ten instrument classes-with very different native outputs.+Per-workcell instruments dominate, so total volume tracks workcell count almost+linearly. -Browse the live wiki at [wiki.adom.inc/skills](https://wiki.adom.inc/skills).+| | Today | 1,000 workcells | 10,000 workcells |+|---|---|---|---|+| Workcells | **20** | 1,000 | 10,000 |+| Cameras | **20** | 1,000 | 10,000 |+| Mics / power / BME690 / BMI270 | 20 each | 1,000 each | 10,000 each |+| Per-workcell volume | ~4.8 TB/day | ~240 TB/day | ~2.4 PB/day |+| Bench instruments | ~0.05–0.5 TB/day | ~0.05–0.5 TB/day | ~0.05–0.5 TB/day |+| **Total** | **~5 TB/day** | **~240 TB/day** | **~2.4 PB/day** |+| Sustained average | ~0.5 Gbps | ~22 Gbps | ~222 Gbps |++### Uplink++**28 Gbps is our potential uplink once the factory is fully built out**, not what+is installed today.++That number is worth holding next to the table above. A 28 Gbps uplink is ~302+TB/day if saturated, which covers roughly **1,000 workcells** — our projected+volume there is ~22 Gbps sustained, comfortably inside it. At 10,000 workcells we+would need on the order of 222 Gbps, roughly eight times the fully-built-out+figure. Somewhere past a few thousand workcells, egress becomes the binding+constraint rather than the instruments, and the question shifts from "how much+can we generate" to "what do we keep, what do we reduce at the edge, and what do+we ship."  --- -## Open questions we would welcome help on--These are the gaps between what we do today and a full open-science pipeline:--- We do not yet mint DOIs.-- We do not yet deposit to Zenodo, protocols.io, or The Stacks.-- We have no funder or ROR field and no output register.-- Licenses default to MIT for code; we have nothing in place yet for CC0 data or-  CC BY protocols.-- Experiments are not yet packaged as publishable units (method, result,-  resource, negative data) with contributor roles and a citation.+## On these numbers -We would like the deposition pipeline in place before InstaPCB goes live in-January and the experiment volume amplifies.+The instrument inventory, the file types, and the native interface rates are+firm. The daily volumes are **estimates**: they assume 8 active hours per day and+one of each per-workcell instrument per workcell. Actual volume depends on real+duty cycles, which we will have once InstaPCB goes live in January. This page is+versioned and will be updated in place with measured figures then.  --- 
docs/rates.png
⋯ 1 unchanged line ⋯
package.json+1−1
@@ -2,7 +2,7 @@   "slug": "pcl-data-profile",   "title": "Adom PCL Node Data Profile",   "type": "skill",-  "version": "0.1.2",+  "version": "0.1.3",   "description": "The physical shape of the data produced by the Adom Industries NSF PCL Test Bed node: every instrument that generates data, the file types each writes, and the rate it writes them at.",   "license": "CC-BY-4.0",   "tags": [
page.json+1−1
@@ -2,7 +2,7 @@   "slug": "pcl-data-profile",   "title": "Adom PCL Node Data Profile",   "type": "skill",-  "version": "0.1.2",+  "version": "0.1.3",   "description": "The physical shape of the data produced by the Adom Industries NSF PCL Test Bed node: every instrument that generates data, the file types each writes, and the rate it writes them at.",   "tags": [     "nsf-pcl",