EdgeStream-GW
An ultra-lightweight industrial software gateway: an intelligent, resilient bridge between physical perception nodes (sensors, meters, PLCs) and the Cloud. It intercepts real-time streams, processes them locally and prevents bottlenecks in harsh industrial environments.
Project card
- Developed by — Helveticore OÜ (Estonia)
- Hardware target — Puzhi ZU2CG / ZU3EG (Zynq UltraScale+), PS/PL partitioning
- Maturity — CMMI Level 3 “Defined”
- Source document — SDD IEEE 1016 (document register)
Positioning
Energy efficiency and bandwidth optimization for smart metering networks (Smart Grid). Positioned in the “hero layer” of a three-layer Edge ecosystem.
Impact KPIs
Modular pipeline
Collection → Normalization → Analytics → Publication, decoupled into layers:
The eight sub-projects
south-drivers/— Modbus RTU/TCP, DLMS/COSEM collection REUSEprotocol-adapters/— normalization into theFrameobject (Fail-Fast) BUILDedge-analytics/— hardware filtering (PL) + software aggregation (PS)ingestion-core/— central asynchronous broker routingFramesnorthbound-publishers/— MQTT (Sparkplug B) / OPC UA to Cloud and SCADAlocal-action-engine/— < 50 ms local loop (PL threshold detection + PS dispatch)store-and-forward/— offline resilience (SQLite) and replay REUSEdeployment/— Docker “zero-configuration” containerization and OTA
Fast path & safety queue
Alongside the main flow, two lateral loops guarantee reactivity and resilience:
- Fast path —
local-action-enginetriggers anActionCommandin < 50 ms with no Cloud dependency. - Northbound safety queue —
store-and-forwardbuffers locally in SQLite when offline, then replays sequentially with throttling.
PS/PL partitioning detail
edge-analytics
- PL (
pl/) — digital noise filtering + FFT on the FPGA fabric, continuously; AXI stream/memory-mapped towards the PS. Planned: Verilog / SystemVerilog RTL and/or Vitis HLS. - PS (
ps/) — containerized Python microservice applying time-window aggregation (RMS, min/max, deduplication), built with TDD.
local-action-engine
- PL (
pl/) — continuous hardware threshold evaluation in real time; triggers a hardware interrupt on critical breach (sub-millisecond, deterministic). - PS (
ps/) — receives the interrupt and routes the pre-configuredActionCommandto the southbound actuator driver in < 50 ms, with action logging.
Requirements
Consolidated extracts from the IEEE 1016 SDD (RDM area). IDs stay aligned with the source document.
Core requirements
- REQ-001a — hardware data reduction (PL): noise filtering and FFT on the FPGA fabric, continuously.
- REQ-001b — software data reduction (PS): time-window aggregation (RMS, min/max, deduplication).
- REQ-002 — offline resilience: SQLite buffering + sequential, throttled replay on reconnection.
- REQ-003 — actuation < 50 ms: local PS/PL loop with no Cloud dependency.
Derived requirements
- REQ-004 — Fail-Fast: corrupted payload rejected (
CorruptedPayloadError), never silently “repaired”. - REQ-005 — zero-configuration deployment (Docker stack).
- REQ-006 — secure OTA updates, without stopping collection.
- REQ-007 — northbound interoperability: MQTT + Sparkplug B (Protobuf) and OPC UA (client-server + PubSub).
- REQ-008 — embedded footprint compatible with Zynq UltraScale+ (lightweight RAM / storage).
Out of scope: Cloud / back-office management, and development of proprietary Modbus/DLMS stacks (reused, cf. decision 1).
Architecture decisions (DAR)
Decision 1 — Protocol collection (South Drivers)
- Verdict
- REUSE —
pymodbus, Gurux DLMS - Rationale
- Rebuilding compliant Modbus/DLMS stacks brings no strategic value and risks non-compliance.
- Requirement
- REQ-004, REQ-008
Decision 2 — Data normalization (Protocol Adapters)
- Verdict
- BUILD FROM SCRATCH
- Rationale
- Open-source parsers silently fix bad data or let incomplete arrays through; in-house adapters raise
CorruptedPayloadErrorand guarantee CMMI L3 quality. - Requirement
- REQ-004
Decision 3 — Store-and-Forward database
- Verdict
- REUSE SQLite
- Rationale
- MongoDB is too heavy for the PS RAM limits; binary files risk corruption on power loss; SQLite is serverless, ACID and edge-appropriate.
- Requirement
- REQ-002, REQ-008
Decision 4 — HW/SW partitioning boundary (Zynq UltraScale+)
- Verdict
- PS/PL SPLIT
- Rationale
- Routing high-frequency signals through Linux (PS) introduces non-deterministic OS latency; the PL guarantees sub-millisecond detection, leaving the PS to route only the asynchronous
ActionCommand. - Requirement
- REQ-003
Decision 5 — First module for detailed specification
- Verdict
- Protocol Adapters — pure
bytes → Frame, TDD-friendly. - Rationale
- No hardware/network/database dependency; defines the central
Framecontract consumed downstream (mandated by Decision 2). - Requirement
- REQ-004, REQ-008
Decision 6 — Schema validation engine
- Verdict
- Pydantic v2 — strict mode,
extra="forbid". - Rationale
- Type-safe, fast, first-class with pytest/hypothesis; strict bounded ranges.
Decision 7 — First protocol to implement
- Verdict
- Modbus RTU first (FC 03), then Modbus TCP, then DLMS/COSEM.
- Rationale
- Simplest and matches the 50-meter bench;
pymodbusprovides reference vectors (Decision 1).
Decision 8 — TDD tooling
- Verdict
- pytest + hypothesis + pydantic + pytest-cov + ruff.
- Rationale
hypothesisenforces the “no invalidFrame” invariant;pytest-covfor branch coverage.
Decision 9 — DLMS/COSEM adapter
- Verdict
- DEFER to a dedicated specification phase.
- Rationale
- A rushed parser would violate the Fail-Fast guarantees (HDLC vs COSEM-TCP, A-XDR, OBIS mapping remain to be framed).
Decision 10 — Edge Analytics PS aggregation output
- Verdict
- Primary +
.min/.maxper quantity; reducerrms/mean/last. - Rationale
- Keeps the canonical
Framestable; N frames → 1 realizes the −75% volume KPI. - Requirement
- REQ-001b
Requirements traceability matrix (RTM)
| Requirement | Design (§ SDD) | Component (src/) | Verification | KPI |
|---|---|---|---|---|
REQ-001a | §4 Edge Analytics PL | edge-analytics/pl/ | RTL / HLS simulation (FFT + filter) | −75% volume |
REQ-001b | §4 Edge Analytics PS | edge-analytics/ps/ | aggregation tests (TDD) | −75% volume |
REQ-002 | §5.2 Store-and-Forward | store-and-forward/ | simulated outage + replay | resilience |
REQ-003 | §3.4 & §5.1 Action Engine | local-action-engine/{pl,ps}/ | < 50 ms latency bench | < 50 ms |
REQ-004 | §4 Protocol Adapters | protocol-adapters/ | corrupted payloads (TDD) | quality |
REQ-005 | §3.1 Containerization | deployment/ | docker-compose deployment | zero-config |
REQ-006 | §3.1 Containerization | deployment/ | OTA update test | continuity |
REQ-007 | §3.2 North boundary | northbound-publishers/ | Mosquitto + OPC UA client | IT/OT interop |
REQ-008 | §6 Decision 3 | store-and-forward/ | RAM / storage measurement | footprint |
Each row must be verified and checked off in the performance report before any release.
Milestones & risks
Milestones (MILESTONES.md)
- M0 · Baseline — done (66 tests pass; technical report regenerable).
- M1 · Volume-reduction KPI — done (96.07% measured, target ≥ 75%).
- M2 · Ingestion Core (async broker, TDD) — next.
- M3–M6 · Northbound MQTT + Sparkplug B, Store-and-Forward SQLite, Local Action Engine (< 50 ms), Deployment.
- M7 · DLMS/COSEM adapter (Decision 9) — M8 · Edge Analytics PL + PS/PL integration.
- M9 · Fog-Crypto Core spec — M10 · NetSentry-IoT spec.
Key risks (RSK)
- PS/PL latency determinism — mitigated by hardware partitioning (decision 4).
- Data corruption during power outages — mitigated by SQLite ACID (decision 3).
- Modbus/DLMS protocol compliance — mitigated by reusing proven stacks (decision 1).
Virtual test bench (VV)
Emulation
- 50 simultaneous Modbus meters simulated locally (realistic cadence, telemetry + noise).
- Mocked PS/PL interfaces: AXI contract + mocked interrupts (TDD of the software layers).
- Local MQTT broker (Eclipse Mosquitto) and OPC UA client for the north boundary.
Planned tooling
pytest, hypothesis,
docker-compose, Modbus/DLMS simulators, Mosquitto, OPC UA client.
Metrics: latency (< 50 ms), reduction rate
(≥ 75%), container RAM/CPU/storage — feeding performance/.
protocol-adapters (Modbus RTU + TCP into the canonical Frame),
edge-analytics/ps (window aggregation + dedup) and the 50-meter Modbus virtual
test bench. 66 automated tests pass (99% coverage on the adapters, 100% on Edge Analytics
PS). Milestone 1 is sealed: the end-to-end −75% volume KPI is measured at 96.07%.
Next: Ingestion Core (M2), then DLMS (M7) and the PL/FPGA layer (M8).