diff --git a/docs/milestones/README.md b/docs/milestones/README.md index e89c1f3..b507dda 100644 --- a/docs/milestones/README.md +++ b/docs/milestones/README.md @@ -50,6 +50,8 @@ Departures from the spec are recorded in [`docs/DEVIATIONS.md`](../DEVIATIONS.md - **Web push notifications (v2)**: a `webpush` receiver kind in `internal/notify` (RFC 8291 payload encryption, RFC 8292 VAPID, standard-library crypto), per-user subscriptions (needs M6 users), a service worker in the UI, and push endpoints in the outbound allow-list. Out of v1 per the spec's non-goals ("alerts go to webhooks and email only"). Owner deferred it to v2 on 2026-10-22. Until then, a webhook receiver pointed at ntfy or Gotify gives phone push. +- **Distributed poller (v2)**: remote pollers per site or rule that spool to disk while the master is down, send signed chunks over an outbound-only link (MQTT through a broker), and receive device credentials sealed to their own key. Assessed feasible on 2026-10-25 and deferred to v2 by the owner. Requirements, gaps and risks: [v2-distributed-poller.md](v2-distributed-poller.md). + - **BMC support**: SNMP profiles for Dell iDRAC, HPE iLO and Supermicro (health, temperatures, fans, PSUs, power, RAID) feeding the Health/Power/Environment tabs; a native Redfish collector later. Today, Redfish works only via an exporter pushing with a push token. Owner: "good enough for now" (2026-10-23). ## Post-v1 additions diff --git a/docs/milestones/v2-distributed-poller.md b/docs/milestones/v2-distributed-poller.md new file mode 100644 index 0000000..8f53078 --- /dev/null +++ b/docs/milestones/v2-distributed-poller.md @@ -0,0 +1,72 @@ +# v2: distributed poller (feasibility, deferred) + +Status: deferred to v2 by the owner on 2026-10-25. This is a feasibility assessment, not a plan; nothing is built. + +## Requirements (owner) + +1. A remote poller polls the devices of a site, or of a predefined rule. +2. It sends its data to the master only as chunked payloads. +3. While the master is down it keeps polling, and delivers when the master is back. +4. Each poller has its own private key; the master sends the target device credentials sealed to it. +5. It works over an outbound-only link (MQTT pub/sub through a broker). Neither side accepts a connection from + the other. Not a data diode yet, but the design should not rule one out. + +## Verdict + +Feasible, about one milestone. The poller is already injectable: `poller.Options` (`internal/poller/poller.go`) +takes the store, the metric writer, the reachability observer and the SNMP dialer, so the polling code can be +reused unchanged. The work is in what sits around it: where targets come from, where samples go, and how +alerting treats data that arrives late. + +## What exists and what is missing + +| # | Exists | Missing | +|---|---|---| +| 1 | `devices.site`, `store.DeviceFilter.Site`; the alert-policy matcher (`alertpolicy.Device{Slug,Profile,Role,Site,Kind}`) already selects devices by rule | a `pollers` table and assignment rules; reuse the policy matcher rather than a second rule engine | +| 2 | `metrics.Sample`; the vmsup queue batches 50k samples per POST | a chunk format (poller id, sequence, signature). Push ingest is **not** reusable: it strips `device`/`iface`, refuses `if*`/`nmon_` names and rejects timestamps more than 1 h from now. Poller data needs its own trusted path into the hot queue | +| 3 | the hot queue buffers in memory only (60 s / 2M samples, then drops) | a disk spool on the poller (append-only segments, bounded by size), and a local SQLite snapshot of its targets so it restarts without the master | +| 4 | symmetric AES-256-GCM keyring with versions (`internal/secrets`, `store/cred.go` `seal`); rotation in `internal/keyrotate` | a key pair per poller (X25519 + Ed25519): the master seals the credential bundle to the poller's public key (HPKE) and signs it; the poller signs its chunks; rotation follows the keyrotate pattern | +| 5 | none | an MQTT transport behind an interface (see below), with outbound HTTPS chunk upload as a second, brokerless option | + +## Transport (requirement 5) + +- Data up: `nmon/poller//data`, QoS 1. The master holds a persistent session, so the broker keeps chunks + while the master is down. +- Config and credentials down: a retained, sealed message on `nmon/poller//config`, read on every connect. +- An MQTT client library (e.g. paho.golang) departs from the spec's "standard library elsewhere" and needs a + DEVIATIONS row. + +Rules that keep a later data diode possible: + +1. The data path is already one-way. Chunks are self-describing, signed and sequence-numbered, and the master + drops duplicates by (poller, seq), so a resend is harmless. The only signal the poller waits for is the + broker's PUBACK, which frees spool space. Under a diode, spool release becomes age/size bound plus a periodic + resend window; the format does not change. +2. One downlink only: the sealed, signed config bundle. Make it a file format that arrives either as the + retained message or by hand (`nmon poller import bundle.sealed`). A diode drops the subscription and keeps + the import. +3. Interactive tools (poll-now, test poll, SNMP debug recording: `PollTools`) need a request and a reply. Over + the broker they work through a request topic with seconds of latency. Under a diode they are unavailable, + and the UI must say so for remote devices. + +## Risks + +1. **Late data looks like an outage.** `reach.Evaluator` runs on the master and expects live + `ObserveICMP`/`ObserveSNMP` (`internal/reach/types.go`); the API marks a device down after three silent + intervals. A site link drop would page every device behind it. Fix: model the poller as a dependency parent + of its devices (dependency suppression already exists) and replay spooled probes as history, never as live + state transitions. Needs a DEVIATIONS row. +2. **Inventory writes from the poller.** The poller writes poll state (`devloop.persistState` → + `SetPollState`), component rows and discontinuity bases. Remotely these go to the local snapshot and are + sent up as events. +3. **Discovery must run remotely.** The master usually cannot reach a site's devices. Interface ULIDs are the + frozen `iface` label; the poller can mint them (ULIDs are globally unique) and the master accepts them. + Traps, L2 tables and checks need the same scope decision up front. +4. **Late writes.** VictoriaMetrics accepts old timestamps within retention; dedup by (poller, seq) keeps a + resent chunk from being written twice. + +## Codebase constraints + +- A new package (e.g. `internal/remotepoll`) and an `nmon poller` subcommand wired in `internal/app`. +- Collection paths still do not import each other; time comes from `internal/clock`. +- API routes additive only (`scripts/openapi-check.sh`); new tables through a new migration.