Project 1 of the triptych

EdgeStream-GW

An ultra-lightweight industrial software gateway: an intelligent, resilient bridge between physical perception nodes (sensors, meters, PLCs) and the Cloud. It intercepts real-time streams, processes them locally and prevents bottlenecks in harsh industrial environments.

Phase: implementation — Milestone 1 sealed (KPI −96% measured)

Project card

  • Developed by — Helveticore OÜ (Estonia)
  • Hardware target — Puzhi ZU2CG / ZU3EG (Zynq UltraScale+), PS/PL partitioning
  • Maturity — CMMI Level 3 “Defined”
  • Source document — SDD IEEE 1016 (document register)

Positioning

Energy efficiency and bandwidth optimization for smart metering networks (Smart Grid). Positioned in the “hero layer” of a three-layer Edge ecosystem.

Impact KPIs

−96 %
northbound volume reduction measured (M1) — target ≥ 75%
< 50 ms
local actuation latency (fast path, no Cloud round-trip)

Modular pipeline

Collection → Normalization → Analytics → Publication, decoupled into layers:

south-driversCollection (REUSE)
→
protocol-adaptersNormalization (BUILD)
→
edge-analyticsPS/PL reduction
→
ingestion-coreCentral broker
→
northbound-publishersMQTT / OPC UA

The eight sub-projects

  • south-drivers/ — Modbus RTU/TCP, DLMS/COSEM collection REUSE
  • protocol-adapters/ — normalization into the Frame object (Fail-Fast) BUILD
  • edge-analytics/ — hardware filtering (PL) + software aggregation (PS)
  • ingestion-core/ — central asynchronous broker routing Frames
  • northbound-publishers/ — MQTT (Sparkplug B) / OPC UA to Cloud and SCADA
  • local-action-engine/ — < 50 ms local loop (PL threshold detection + PS dispatch)
  • store-and-forward/ — offline resilience (SQLite) and replay REUSE
  • deployment/ — Docker “zero-configuration” containerization and OTA

Fast path & safety queue

Alongside the main flow, two lateral loops guarantee reactivity and resilience:

  • Fast path — local-action-engine triggers an ActionCommand in < 50 ms with no Cloud dependency.
  • Northbound safety queue — store-and-forward buffers locally in SQLite when offline, then replays sequentially with throttling.

PS/PL partitioning detail

edge-analytics

  • PL (pl/) — digital noise filtering + FFT on the FPGA fabric, continuously; AXI stream/memory-mapped towards the PS. Planned: Verilog / SystemVerilog RTL and/or Vitis HLS.
  • PS (ps/) — containerized Python microservice applying time-window aggregation (RMS, min/max, deduplication), built with TDD.

local-action-engine

  • PL (pl/) — continuous hardware threshold evaluation in real time; triggers a hardware interrupt on critical breach (sub-millisecond, deterministic).
  • PS (ps/) — receives the interrupt and routes the pre-configured ActionCommand to the southbound actuator driver in < 50 ms, with action logging.

Requirements

Consolidated extracts from the IEEE 1016 SDD (RDM area). IDs stay aligned with the source document.

Core requirements

  • REQ-001a — hardware data reduction (PL): noise filtering and FFT on the FPGA fabric, continuously.
  • REQ-001b — software data reduction (PS): time-window aggregation (RMS, min/max, deduplication).
  • REQ-002 — offline resilience: SQLite buffering + sequential, throttled replay on reconnection.
  • REQ-003 — actuation < 50 ms: local PS/PL loop with no Cloud dependency.

Derived requirements

  • REQ-004 — Fail-Fast: corrupted payload rejected (CorruptedPayloadError), never silently “repaired”.
  • REQ-005 — zero-configuration deployment (Docker stack).
  • REQ-006 — secure OTA updates, without stopping collection.
  • REQ-007 — northbound interoperability: MQTT + Sparkplug B (Protobuf) and OPC UA (client-server + PubSub).
  • REQ-008 — embedded footprint compatible with Zynq UltraScale+ (lightweight RAM / storage).

Out of scope: Cloud / back-office management, and development of proprietary Modbus/DLMS stacks (reused, cf. decision 1).

Architecture decisions (DAR)

Decision 1 — Protocol collection (South Drivers)

Verdict
REUSE — pymodbus, Gurux DLMS
Rationale
Rebuilding compliant Modbus/DLMS stacks brings no strategic value and risks non-compliance.
Requirement
REQ-004, REQ-008

Decision 2 — Data normalization (Protocol Adapters)

Verdict
BUILD FROM SCRATCH
Rationale
Open-source parsers silently fix bad data or let incomplete arrays through; in-house adapters raise CorruptedPayloadError and guarantee CMMI L3 quality.
Requirement
REQ-004

Decision 3 — Store-and-Forward database

Verdict
REUSE SQLite
Rationale
MongoDB is too heavy for the PS RAM limits; binary files risk corruption on power loss; SQLite is serverless, ACID and edge-appropriate.
Requirement
REQ-002, REQ-008

Decision 4 — HW/SW partitioning boundary (Zynq UltraScale+)

Verdict
PS/PL SPLIT
Rationale
Routing high-frequency signals through Linux (PS) introduces non-deterministic OS latency; the PL guarantees sub-millisecond detection, leaving the PS to route only the asynchronous ActionCommand.
Requirement
REQ-003

Decision 5 — First module for detailed specification

Verdict
Protocol Adapters — pure bytes → Frame, TDD-friendly.
Rationale
No hardware/network/database dependency; defines the central Frame contract consumed downstream (mandated by Decision 2).
Requirement
REQ-004, REQ-008

Decision 6 — Schema validation engine

Verdict
Pydantic v2 — strict mode, extra="forbid".
Rationale
Type-safe, fast, first-class with pytest/hypothesis; strict bounded ranges.

Decision 7 — First protocol to implement

Verdict
Modbus RTU first (FC 03), then Modbus TCP, then DLMS/COSEM.
Rationale
Simplest and matches the 50-meter bench; pymodbus provides reference vectors (Decision 1).

Decision 8 — TDD tooling

Verdict
pytest + hypothesis + pydantic + pytest-cov + ruff.
Rationale
hypothesis enforces the “no invalid Frame” invariant; pytest-cov for branch coverage.

Decision 9 — DLMS/COSEM adapter

Verdict
DEFER to a dedicated specification phase.
Rationale
A rushed parser would violate the Fail-Fast guarantees (HDLC vs COSEM-TCP, A-XDR, OBIS mapping remain to be framed).

Decision 10 — Edge Analytics PS aggregation output

Verdict
Primary + .min/.max per quantity; reducer rms/mean/last.
Rationale
Keeps the canonical Frame stable; N frames → 1 realizes the −75% volume KPI.
Requirement
REQ-001b

Requirements traceability matrix (RTM)

RequirementDesign (§ SDD)Component (src/)VerificationKPI
REQ-001a§4 Edge Analytics PLedge-analytics/pl/RTL / HLS simulation (FFT + filter)−75% volume
REQ-001b§4 Edge Analytics PSedge-analytics/ps/aggregation tests (TDD)−75% volume
REQ-002§5.2 Store-and-Forwardstore-and-forward/simulated outage + replayresilience
REQ-003§3.4 & §5.1 Action Enginelocal-action-engine/{pl,ps}/< 50 ms latency bench< 50 ms
REQ-004§4 Protocol Adaptersprotocol-adapters/corrupted payloads (TDD)quality
REQ-005§3.1 Containerizationdeployment/docker-compose deploymentzero-config
REQ-006§3.1 Containerizationdeployment/OTA update testcontinuity
REQ-007§3.2 North boundarynorthbound-publishers/Mosquitto + OPC UA clientIT/OT interop
REQ-008§6 Decision 3store-and-forward/RAM / storage measurementfootprint

Each row must be verified and checked off in the performance report before any release.

Milestones & risks

Milestones (MILESTONES.md)

  • M0 · Baseline — done (66 tests pass; technical report regenerable).
  • M1 · Volume-reduction KPI — done (96.07% measured, target ≥ 75%).
  • M2 · Ingestion Core (async broker, TDD) — next.
  • M3–M6 · Northbound MQTT + Sparkplug B, Store-and-Forward SQLite, Local Action Engine (< 50 ms), Deployment.
  • M7 · DLMS/COSEM adapter (Decision 9) — M8 · Edge Analytics PL + PS/PL integration.
  • M9 · Fog-Crypto Core spec — M10 · NetSentry-IoT spec.

Key risks (RSK)

  • PS/PL latency determinism — mitigated by hardware partitioning (decision 4).
  • Data corruption during power outages — mitigated by SQLite ACID (decision 3).
  • Modbus/DLMS protocol compliance — mitigated by reusing proven stacks (decision 1).

Virtual test bench (VV)

Emulation

  • 50 simultaneous Modbus meters simulated locally (realistic cadence, telemetry + noise).
  • Mocked PS/PL interfaces: AXI contract + mocked interrupts (TDD of the software layers).
  • Local MQTT broker (Eclipse Mosquitto) and OPC UA client for the north boundary.

Planned tooling

pytest, hypothesis, docker-compose, Modbus/DLMS simulators, Mosquitto, OPC UA client.

Metrics: latency (< 50 ms), reduction rate (≥ 75%), container RAM/CPU/storage — feeding performance/.

Current state: three components are implemented test-first and verified — protocol-adapters (Modbus RTU + TCP into the canonical Frame), edge-analytics/ps (window aggregation + dedup) and the 50-meter Modbus virtual test bench. 66 automated tests pass (99% coverage on the adapters, 100% on Edge Analytics PS). Milestone 1 is sealed: the end-to-end −75% volume KPI is measured at 96.07%. Next: Ingestion Core (M2), then DLMS (M7) and the PL/FPGA layer (M8).