19 Commits

Author SHA1 Message Date
Vibe Coder
61b5341774 Add Find host, the L2 tab, the BGP page and the Routing tab
Some checks failed
ci / lint (push) Has been cancelled
ci / test (ubuntu-24.04-arm) (push) Has been cancelled
ci / test (ubuntu-latest) (push) Has been cancelled
ci / postgres (push) Has been cancelled
ci / openapi (push) Has been cancelled
The UI for the MAC/IP lookup and BGP monitoring.

Find host (/find, and the header search when the text is a MAC, IP, CIDR or
vendor: query; a customer name or circuit id still goes to the port search)
answers "device X port Y, VLAN Z, last seen 3 min ago" with the uplinks and
history collapsed underneath, IPv6 addresses first. A vendor column and a
"private" badge for locally administered addresses appear wherever a MAC does.
The device page gets an L2 tab only when the job holds rows (MAC table with a
VLAN filter and history toggle, ARP/ND, VLAN list), and the Ports tab shows
each port's MAC count linking to that port's MACs.

The BGP page lists every session with the down ones first and state, AS,
family and text filters; a device's Routing tab shows its sessions with
click-to-graph on accepted prefixes and update rates. The Poller page shows the
L2 job status and non-alert notes such as oui_unavailable. The policy template
"Routers: BGP peer down" binds the new rule. The mock API and a new Playwright
spec cover all of it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T4PdJUPrakM8fHvN8jYv2x
2026-10-24 01:30:32 +00:00
Vibe Coder
7173360346 Add the MAC/ARP/FDB table job, the lookup API, BGP peer collection and bgp_peer_down
Owner request: finding where a host is plugged in from a MAC or IP is a must,
and BGP peer monitoring is wanted.

L2 job (internal/l2): a job of its own, off the 60 s poll path (its sessions
are path "l2"), walks BRIDGE-MIB, Q-BRIDGE-MIB, CISCO-VTP-MIB (a per-VLAN
context walk, enabled by the cisco-ios profile flag l2.per_vlan_context, not by
detection) and the IP-MIB neighbour tables (IPv4 ARP and IPv6 ND, IPv4-only
fallback) every l2.interval, with its own concurrency cap and a per-device
deadline. It yields to the poller like the rollup job, and a device that times
out is skipped for the cycle, counted and named on /poller/health. Migration 14
adds fdb_entries, arp_entries, vlans and l2_walks with first/last-seen history,
batched upserts, daily pruning by last_seen (l2.keep) and backup coverage.

API (additive): /l2/search ranks edge ports first (a port is an uplink above
l2.edge_max_macs MACs, with an LLDP neighbour, or as a LAG), plus the device
fdb/arp/l2/l2-ports tables and /interfaces/{id}/macs; vendor and mac_kind are
computed at query time from the optional registry.

BGP: profiles label peer rows by decoding the table index (index_label.layout),
so each peer, IPv4 or IPv6, is one component whatever MIB feeds it; AS, local
AS, router id and description are component attributes, never labels.
CISCO-BGP4 (cisco-ios, cisco-nxos), Juniper BGP4-V2 and Arista BGP4V2 replace
the optional generic BGP4-MIB metrics, which edgeos, fortigate, pfsense and vyos
switch on. /bgp/peers lists sessions down first. The built-in rule
bgp_peer_down (critical, not established for 2 minutes) fires only where a
policy binds it; its notification names the peer's AS and description.

Fixtures are generated (Q-BRIDGE with IPv4/IPv6 neighbours, BRIDGE-MIB only,
Cisco per-VLAN contexts via a new simagent contexts feature) and checked
against real LibreNMS recordings. DEVIATIONS 50-55 record the departures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T4PdJUPrakM8fHvN8jYv2x
2026-10-24 01:30:32 +00:00
Vibe Coder
c4d14a67d1 Add the optional IEEE vendor registry for MAC addresses and the l2/oui settings
The owner asked to see the vendor of each MAC where the IEEE registry can be
fetched, and never to depend on it. internal/oui downloads MA-L, MA-M and MA-S
weekly through the outbound allow-list (timeouts, a size cap and a row-count
sanity check, so a truncated or wrong download keeps the previous file), stores
one checksummed, atomically written file in <data-dir>/oui that nmon backup
includes, and looks MACs up by the longest prefix. A missing registry only
leaves the vendor empty. `nmon oui import DIR|FILE` loads the CSVs offline, and
the Ansible role can ship them next to the MIB sidecar.

internal/hostaddr parses the MAC, prefix, IP and CIDR forms the lookup accepts
and classifies a MAC as global, local or multicast. config gains the l2 and oui
sections (validated here; the L2 job that reads l2 follows).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T4PdJUPrakM8fHvN8jYv2x
2026-10-24 01:30:11 +00:00
Vibe Coder
06a3c85c69 e2e: point device-page specs at the Overview tab
992f09b removed DeviceHeader and OverviewSummary, but six mock specs and
the live gate screenshots still waited for their test ids (device-header,
summary-tile, generic-hint, device-warnings), so they failed and stopped
gate-m4 at its UI step. Assert on what the page shows now: the System
panel (role, profile, status), the Processors/Memory/Storage panels, and
the per-port unreliable flag on the Ports tab. The device-level warning
badge and the generic-profile hint no longer exist, so their assertions
are dropped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T4PdJUPrakM8fHvN8jYv2x
2026-10-23 23:29:41 +00:00
Vibe Coder
ddc8863a67 Add alert policies: alerts fire only where a policy says so
Preprod fires everything: every default rule applied to every device and
port, so a WiFi port that always sees errors was judged like a fiber
uplink where one error matters. WiFi errors are acceptable, fiber errors
are not, and one global threshold cannot say both.

An alert policy (new alert_policies table, migration 12) binds one rule
to a scope: groups plus device and port attribute matchers (ifType,
name/alias regexes, speed, site, role, profile, kind, the alert_down
bell). The highest priority decides, ignore beats alert on a tie, and a
policy's severity/threshold/for override the rule's (its threshold also
beats group overrides). With no policy nothing alerts: not metric rules,
not device_down/device_warning, not system alarms. Nothing is seeded, so
on upgrade open alerts without a binding close silently as retired.

The pure internal/alertpolicy package makes the decision; alert, reach
and the system alarms take a gate (Options.Policy, Options.Wants, a
system policy) that the app backs with a cache refreshed every 30 s and
invalidated on every API write, so policy edits apply live. New API:
/alert-policies CRUD, a viewer-level preview of what a scope covers, and
/alert-rules. The UI gets an Alert policies tab with templates and a
live preview; the port bell now says it only marks the port. Gate
scripts create the policies they need. DEVIATIONS row 41 and the
operations guide describe the new default.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T4PdJUPrakM8fHvN8jYv2x
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 18:41:25 +00:00
c4f5ee916b Add CLAUDE.md: agent onboarding (rules, layout, commands, how-tos, gotchas)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lx9FNLG2CqeBbVGGd9bFsP
2026-10-23 17:16:56 +00:00
992f09b97a Overview tab: use generated overview types, drop dead DeviceHeader/OverviewSummary
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lx9FNLG2CqeBbVGGd9bFsP
2026-10-23 14:38:16 +00:00
d92c6e1e08 Device overview endpoint and aggregate metric graphs
GET /api/v1/devices/{slug}/overview (traffic, ports, availability, rtt, events);
metrics/graph gains aggregate=avg|sum|max when component is omitted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lx9FNLG2CqeBbVGGd9bFsP
2026-10-23 14:34:37 +00:00
9b4ab7ccbb Device Overview tab: LibreNMS-style panel dashboard (system, ports, availability, ping, processors, memory, storage, supplies, sensors, events)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lx9FNLG2CqeBbVGGd9bFsP
2026-10-23 14:17:49 +00:00
457191accd Docs: BMC support deferred
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lx9FNLG2CqeBbVGGd9bFsP
2026-10-23 05:59:11 +00:00
61274c4f55 Docs: v1.0.1 device tabs contract
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lx9FNLG2CqeBbVGGd9bFsP
2026-10-23 05:46:26 +00:00
fe53998aa6 Device page tabs (Overview, Ports, per-category metrics, Printer and Wireless panels) and Printers fleet page
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lx9FNLG2CqeBbVGGd9bFsP
2026-10-23 05:44:54 +00:00
e25143881c Device metrics API, printer fleet, profile presentation schema (v1.0.1 backend)
Profiles gain category/label/of/display/scale_by/sentinels/enum_state and component_attrs;
every pollable non-interface metric is annotated and an orphan test enforces it. New routes
GET /devices/{slug}/metrics, /metrics/graph and /printers; printer supply type/colour read at
discovery into device_components.attrs; printer_supply_empty alert; supply_replaced audit event;
docs/coverage.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lx9FNLG2CqeBbVGGd9bFsP
2026-10-23 05:33:07 +00:00
ad220eae58 Interface-down alerts are opt-in: per port (bell toggle, bulk) and via groups
interfaces.alert_down / groups.alert_down (migration 9); interface_down rule OptIn; non-opted ports' open alerts close silently as retired.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lx9FNLG2CqeBbVGGd9bFsP
2026-10-23 05:04:41 +00:00
345daaa773 Profiles for EdgeSwitch, UniFi switch/AP and VyOS; profile pinning and re-match; whole-row navigation
Discovery re-matches unpinned devices every run and at startup; an admin can pin a
profile (PUT /devices/{slug} profile, GET /profiles, DeviceView profile_pinned and
profile_match). Rows on the device list, interface search and poller health navigate on
click and Enter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lx9FNLG2CqeBbVGGd9bFsP
2026-10-23 05:03:17 +00:00
bf6dbd4923 Add device management UI: add, edit, rediscover, retire
The UI had no way to add a device. Adds an admin-only Add device dialog
(IPv6/IPv4 addresses, SNMP v1/v2c/v3, synchronous discovery, reattach 409
flow) and a device-page Actions menu (Edit with parents and SNMP
replacement, Rediscover now, Retire). Purge now requires typing the name.

API: POST /api/v1/devices/{slug}/retire (admin, audited, 409 if already
retired). OpenAPI change is additive.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lx9FNLG2CqeBbVGGd9bFsP
2026-10-23 03:36:10 +00:00
d97dd1d44a Ignore local Ansible inventory and vars (vars.yml holds secrets)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lx9FNLG2CqeBbVGGd9bFsP
2026-10-23 03:14:18 +00:00
28f5444860 Playbook: self-signed cert names IP addresses as IP SANs; nmon_tls_extra_names for more names/IPs
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lx9FNLG2CqeBbVGGd9bFsP
2026-10-23 03:14:12 +00:00
6ae7729107 Ansible deployment playbook; fix: SIGHUP during startup no longer kills nmon
The playbook (deploy/ansible) builds the release binary, installs nmon as a hardened systemd service with
self-signed/files/ACME/insecure TLS modes, break-glass and optional OIDC, MIB sidecar, ping sysctl and nightly backups.
Tested end to end in a Debian 13 systemd container, idempotent on a second run.

Bug found by that test: Run installed its SIGHUP handler only after Start, so a 'systemctl reload' during the
seconds-long startup hit Go's default action and exited nmon. The handler is now installed before Start; an early
SIGHUP is queued and applied once startup finishes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lx9FNLG2CqeBbVGGd9bFsP
2026-10-23 03:07:56 +00:00
397 changed files with 32706 additions and 1029 deletions

3
.gitignore vendored
View File

@@ -11,3 +11,6 @@ coverage.out
/nmon-librenms-import
/internal/webui/dist/*
!/internal/webui/dist/.gitkeep
deploy/ansible/.mibs.tar.gz
deploy/ansible/inventory.ini
deploy/ansible/vars.yml

View File

@@ -100,6 +100,7 @@ linters:
- "**/internal/checks/**"
- "**/internal/rollup/**"
- "**/internal/alert/**"
- "**/internal/l2/**"
deny:
- pkg: github.com/v10networks/nmon/internal/poller
desc: collection paths share code via a lower package, not each other
@@ -107,6 +108,8 @@ linters:
desc: collection paths share code via a lower package, not each other
- pkg: github.com/v10networks/nmon/internal/traps
desc: collection paths share code via a lower package, not each other
- pkg: github.com/v10networks/nmon/internal/l2
desc: collection paths share code via a lower package, not each other
devtools-not-in-nmon:
# The product binary never links dev/test machinery. `nmon profile record` needs only the
# .snmprec format, which lives in internal/snmprec.

120
CLAUDE.md Normal file
View File

@@ -0,0 +1,120 @@
# nmon: agent onboarding
nmon is a lightweight LibreNMS replacement for ISPs and IT teams. It is one static Go binary that polls devices
(SNMP, ICMP, traps, push ingest, Proxmox, checks), stores metrics in two embedded VictoriaMetrics child processes
(hot: raw 30 d; long: rollups up to 10 y), keeps inventory in SQLite or Postgres via bun, alerts with dependency
suppression, and serves a React UI and JSON API from the same binary. Module: `github.com/v10networks/nmon`, Go 1.27.
## Read first (source of truth, in this order)
1. `docs/spec/spec.md` and `docs/spec/build-detail.md`: the owner's spec. The contract.
2. `docs/DEVIATIONS.md`: every approved departure from the spec, with the reason. Check it before "fixing" something to match the spec.
3. `docs/MAINTAINABILITY.md`: **hard rule**, enforced by lint. Read it before writing code.
4. `docs/milestones/`: per-milestone scope, decisions and gate logs (M1–M6, `v1.0.1-device-tabs.md`), plus deferred items and post-v1 additions in `README.md`.
5. `docs/operations.md` (running nmon), `docs/dev.md` (UI embedding, openapi check, devtools), `docs/coverage.md` (what each profile collects vs. the spec).
## Non-negotiables
- **`make lint` must be clean**, Go (golangci-lint v2.14.0 via `.golangci.yml`, plus `scripts/lint-filesize.sh`) and web (ESLint, `--max-warnings 0`).
- Size limits: functions ≤ 80 lines, cyclomatic ≤ 15, cognitive ≤ 20; Go files ≤ 600 lines, web files ≤ 400.
- `//nolint` only with the linter's name and a reason.
- **Layering (depguard):**
- No SQL or bun outside `internal/store`.
- Only `cmd/*` imports `internal/app`, and only `internal/app` imports `internal/api`.
- Collection paths don't import each other: `poller`, `discovery`, `traps`, `ingest`, `checks`, `rollup`, `alert`, `l2`. Shared code goes in a lower package (e.g. `snmptarget`, `longq`, `snmprec`).
- `cmd/nmon` must not link dev/test machinery (`simagent`, `fixturegen`, `fixtures`, `librenms`, `testutil`); `TestNmonDoesNotLinkDevMachinery` enforces this.
- **Time** comes only from `internal/clock` (forbidigo bans `time.Now`/`Sleep`/`After`… outside `clock`, `cmd/` and tests). Tests use `clock.NewFake`.
- **Frozen contracts:**
- Metric names and labels: SNMP series carry only `device=<slug>` and `iface=<ULID>`/`component`. Long-store rollups use `v` (highest wins; read via `internal/longq.LatestVersion`).
- API routes: `scripts/openapi-check.sh` fails on incompatible changes vs the last tag. Additive changes only.
- **Migrations are frozen:** never edit an applied migration. Add `internal/store/<YYYYMMDDNNNNNN>_<name>.go` with the next number, using frozen struct copies (see existing files). Add new store methods as a sub-interface in a new file, plus one embed line in `store.go`.
- **API types in the UI are generated, never hand-written:**
1. `go run ./cmd/nmon openapi > openapi.json`
2. `cd web && npm run gen:api` (writes `web/src/api/schema.d.ts`)
3. Use `client.GET/POST`.
- **Only `web/src/components/TrafficGraph.tsx` imports uPlot.** Single-series graphs use its `format` prop.
- **IPv6 first.** Everything works on v6 and v4; tests default to `::1`.
- **Every port is polled.** Only a profile's `exclude_ifname` drops one. 32-bit counters on fast links are flagged `unreliable` and excluded from sums, p95 and rollups, never graphed as trustworthy.
- **Rollups store rates, never raw counters.** A grep test enforces this.
## Layout
| Path | What |
|---|---|
| `cmd/nmon` | the product binary (`serve`, `migrate`, `discover`, `profile test/record/new`, `backup`, `restore`, `key rotate`, `unlock`, `store restart`, `secret set`, `rollup repair`, `openapi`) |
| `cmd/nmon-devtools` | dev-only binary (backfill, seed-demo, mocks for OIDC/Proxmox, webhook sink); never shipped |
| `cmd/nmon-librenms-import` | offline tool: LibreNMS definitions → draft profiles |
| `internal/app` | wiring: one `start*` per subsystem, stops in reverse order, SIGHUP reload |
| `internal/api` | huma v2 handlers, RFC 9457 errors, paging (default 100, max 1,000); slugs and ULIDs only, never numeric ids or secrets |
| `internal/auth` | OIDC (no login UI: browsers 302 to the IdP), sessions + CSRF, API tokens, break-glass (HTTP Basic on `/auth/breakglass`), route policy |
| `internal/store` | bun models, migrations, every SQL query |
| `internal/vmsup` | embedded VictoriaMetrics: extract, checksum, supervise, write queue, querier |
| `internal/{poller,discovery,traps,ingest,checks,proxmox,rollup,p95,groups,reach,deps,alert,notify}` | collection, storage jobs, reachability, alerting |
| `internal/{l2,hostaddr,oui}` | the MAC/ARP/FDB table job, MAC and IP parsing, the optional IEEE vendor registry |
| `internal/profile` + `profiles/*.yaml` | device profiles (YAML, numeric OIDs, `category`/`label`/`of`/`display` per metric) |
| `internal/mib` | SMI parser; MIBs are a sidecar in `mibs/`, never embedded |
| `internal/{simagent,snmprec,fixturegen,fixtures,profiletest}` | the SNMP simulator, `.snmprec` format, fixture generator, expectation harness |
| `web/` | React 19 + TS + Vite + Tailwind/shadcn + TanStack Query/Router; built into `internal/webui` |
| `testdata/snmprec/` | per-profile and behaviour fixtures, each with `*.expect.yaml` and optional `*.agent.yaml` |
| `deploy/ansible/` | playbook (systemd, TLS modes, break-glass/OIDC, MIB sidecar) |
## Commands
```
make fetch-vm # once: download and checksum the VictoriaMetrics binaries (needed for vmembed and integration)
make build # web build + static nmon with the UI and VictoriaMetrics embedded
make devtools importer # the dev and import binaries
make lint # Go + web; must be clean
go test -race ./... # unit tests (fast)
go test -race -tags 'integration vmembed' ./... # real VictoriaMetrics children
./scripts/test-pg.sh # store suite on a throwaway Postgres 16 (Docker; safe to run concurrently)
NMON_LIBRENMS_DIR=/home/svcuser/src/librenms go test ./internal/profile/... ./internal/fixtures/... # real-device recordings
go run ./testdata/gen # regenerate fixtures (a test fails on drift)
scripts/openapi-check.sh # API compatibility vs the last tag
make gate-m1 … gate-m6 # end-to-end gates (slow, ~10 min each)
cd web && npm run lint && npm run test && npm run build && npm run e2e # e2e = Playwright on a mocked API
```
## Adding things
- **A device profile:**
- Add `profiles/<name>.yaml` that `extends: generic`, with `match` (sysObjectID prefix and/or sysDescr regex) and `priority`. Higher wins; watch overlaps such as Linux-based sysDescrs and shared enterprise numbers.
- Use numeric OIDs, verified against `mibs/`, with a provenance comment. Every pollable non-interface metric needs a `category`; the orphan test enforces it.
- Then run `go run ./testdata/gen` for the fixture and expect file, add matcher tests, and check against LibreNMS recordings in `/home/svcuser/src/librenms/tests/snmpsim`.
- That repo is GPL and read-only reference material. Never copy code or YAML from it.
- **An API route:**
- Put the handler in a new file in `internal/api`. State-changing routes are admin by default; declare read-only POSTs as viewer on the auth policy (`internal/app/authwire.go`). `TestViewerCannotMutate` walks every route automatically.
- Regenerate `openapi.json` and the web schema, then run `scripts/openapi-check.sh`.
- **A UI page:**
- Use the generated client in `web/src/api/*Hooks.ts`, `NavRow` for clickable rows, and `RequireAdmin` or role checks for mutating controls.
- Mock handlers and fixtures live in `web/e2e/mock/` for the Playwright mock suite.
- **A deviation from the spec:** add a row to `docs/DEVIATIONS.md` explaining what and why.
## Gotchas
- **Concurrent agents:**
- Stage only your own paths, never `git add -A`.
- `openapi.json` and `web/src/api/schema.d.ts` are shared, so regenerate them just before committing.
- Two agents writing a migration must agree on numbers first.
- **Screenshots and Playwright:**
- Full Playwright runs rewrite the committed screenshots under `docs/screenshots/*`. Revert the ones you didn't mean to change (`git checkout -- docs/screenshots/<dir>`).
- Use a non-default `E2E_PORT` if a `vite preview` may already be running, and kill any preview server you start.
- **Gate scripts:**
- Gate scripts run `make lint` first; a lint finding anywhere fails every gate.
- `--insecure-http` with no sign-in configured runs **dev-open mode** (everyone is admin). The gates rely on it; never ship it.
- **Ingest:** reserved metric prefixes (`nmon_`, `if`, `hr`, `sys` + uppercase/digit) and the labels `device`/`iface`/`v`/`push` are stripped or refused.
- **ICMP:** `nmon serve --icmp off`, or an unsupported unprivileged ICMP socket, makes SNMP alone decide reachability, with an `icmp_unsupported` warning.
## Working style expected here
- Commit messages: imperative summary, a body explaining why, and the trailers the session asks for.
- Don't claim something passed unless you ran it. Report skipped checks explicitly.
- Owner preferences in force:
- Maintainability is a hard rule.
- MIBs are a sidecar.
- The LibreNMS importer and devtools are separate binaries.
- No login UI (OIDC redirect only).
- Interface-down alerts are opt-in (per port or via groups).
- The 2,000-device load test is skipped (approved deviation).
- Web push is v2.
- BMC profiles are deferred.

47
cmd/nmon/oui.go Normal file
View File

@@ -0,0 +1,47 @@
package main
import (
"context"
"fmt"
"path/filepath"
"github.com/v10networks/nmon/internal/config"
"github.com/v10networks/nmon/internal/oui"
)
func init() {
register("oui", "load the IEEE vendor registry from disk: nmon oui import DIR|FILE", ouiCmd)
}
const ouiUsage = "usage: nmon oui import <dir-or-file> --data-dir DIR\n\n" +
"Loads the IEEE MAC registry CSVs (oui.csv for MA-L, mam.csv for MA-M, oui36.csv for MA-S, downloaded from\n" +
"https://standards-oui.ieee.org/) into <data-dir>/oui/registry.gz, where a running nmon picks it up within an hour.\n" +
"Use it where nmon may not fetch (oui.fetch: false). The CSVs are public IEEE data and are not kept after import."
// ouiCmd is "nmon oui import": the offline way to give nmon its vendor registry.
func ouiCmd(_ context.Context, args []string) error {
if len(args) == 0 || args[0] != "import" {
return fmt.Errorf("%s", ouiUsage)
}
var f config.Flags
fs := newFlagSet("oui import", "oui import <dir-or-file> --data-dir DIR")
fs.StringVar(&f.DataDir, "data-dir", "", "nmon's data directory")
pos, err := parseInterspersed(fs, args[1:])
if err != nil {
return flagErr(err)
}
if len(pos) != 1 || f.DataDir == "" {
return fmt.Errorf("%s", ouiUsage)
}
// Only the MA-L registry is required: it holds nearly every vendor. MA-M and MA-S refine it when present.
reg, p, err := oui.Import(pos[0], oui.Minimums{MAL: oui.MinMAL})
if err != nil {
return err
}
dst := filepath.Join(config.OUIDir(f.DataDir), oui.FileName)
if err := oui.Write(dst, reg); err != nil {
return err
}
fmt.Printf("oui import: %d MA-L, %d MA-M and %d MA-S assignments written to %s\n", p.MAL, p.MAM, p.MAS, dst)
return nil
}

53
cmd/nmon/oui_test.go Normal file
View File

@@ -0,0 +1,53 @@
package main
import (
"context"
"fmt"
"os"
"path/filepath"
"strings"
"testing"
"github.com/v10networks/nmon/internal/config"
"github.com/v10networks/nmon/internal/oui"
)
func TestOuiImportWritesTheRegistryFileANmonLoads(t *testing.T) {
src, data := t.TempDir(), t.TempDir()
var b strings.Builder
b.WriteString("Registry,Assignment,Organization Name,Organization Address\n")
for i := 0; i < oui.MinMAL; i++ {
fmt.Fprintf(&b, "MA-L,%06X,Vendor %d,Somewhere\n", 0x100000+i, i)
}
if err := os.WriteFile(filepath.Join(src, "oui.csv"), []byte(b.String()), 0o644); err != nil {
t.Fatal(err)
}
if err := ouiCmd(context.Background(), []string{"import", src, "--data-dir", data}); err != nil {
t.Fatal(err)
}
reg, err := oui.Load(filepath.Join(config.OUIDir(data), oui.FileName))
if err != nil {
t.Fatal(err)
}
if v, ok := reg.Lookup("10:00:05:00:00:01"); !ok || v != "Vendor 5" {
t.Errorf("lookup = %q %v", v, ok)
}
}
func TestOuiImportRefusesATruncatedDirectoryAndUsageErrors(t *testing.T) {
src, data := t.TempDir(), t.TempDir()
if err := os.WriteFile(filepath.Join(src, "oui.csv"), []byte("Registry,Assignment,Organization Name,Organization Address\nMA-L,AABBCC,One,x\n"), 0o644); err != nil {
t.Fatal(err)
}
if err := ouiCmd(context.Background(), []string{"import", src, "--data-dir", data}); err == nil {
t.Error("a registry with one row was imported")
}
if _, err := os.Stat(filepath.Join(config.OUIDir(data), oui.FileName)); err == nil {
t.Error("a refused import wrote a file")
}
for _, args := range [][]string{nil, {"fetch"}, {"import"}, {"import", src}} {
if err := ouiCmd(context.Background(), args); err == nil {
t.Errorf("args %v accepted", args)
}
}
}

View File

@@ -276,7 +276,7 @@ func renderDraft(spec draftSpec, m *mib.Module, metrics []draftMetric) string {
var b strings.Builder
name := "draft-" + strings.ToLower(spec.Module)
fmt.Fprintf(&b, "# DRAFT compiled by `nmon profile new` from %s (%s).\n", spec.Module, filepath.Base(m.File))
fmt.Fprintf(&b, "# Save as %s.yaml. Replace every TODO (unit, interval, index_label) and add a match: section.\n", name)
fmt.Fprintf(&b, "# Save as %s.yaml. Replace every TODO (unit, interval, category, index_label) and add a match: section.\n", name)
b.WriteString("# The loader rejects TODO on purpose, so an unreviewed draft can never be polled.\n")
fmt.Fprintf(&b, "name: %s\nextends: generic\nprovenance: %q\nmetrics:\n", name,
"drafted by nmon profile new from "+spec.Module+"; review before use")
@@ -462,7 +462,8 @@ func writeDraftMetric(b *strings.Builder, dm draftMetric) {
}
fmt.Fprintf(b, " # %s\n", d)
}
fmt.Fprintf(b, " - name: %s\n oid: %s\n type: %s\n unit: %s\n interval: TODO\n", o.Name, oid, dm.typ, dm.unit)
fmt.Fprintf(b, " - name: %s\n oid: %s\n type: %s\n unit: %s\n", o.Name, oid, dm.typ, dm.unit)
b.WriteString(" interval: TODO\n category: TODO\n")
if dm.scale != 0 {
fmt.Fprintf(b, " scale: %s\n", strconv.FormatFloat(dm.scale, 'f', -1, 64))
}

View File

@@ -27,6 +27,7 @@ func bundledMIBs(t *testing.T) string {
}
func fillTODOs(s string) string {
s = strings.ReplaceAll(s, "category: TODO", "category: health")
s = strings.ReplaceAll(s, "interval: TODO", "interval: 5m")
return strings.ReplaceAll(s, "unit: TODO", "unit: count")
}

48
deploy/ansible/README.md Normal file
View File

@@ -0,0 +1,48 @@
# Deploying nmon with Ansible
Installs nmon as a systemd service on one Linux host (amd64 or arm64). Tested against Debian 13.
## Quick start
cd deploy/ansible
cp inventory.example.ini inventory.ini # your host
cp vars.example.yml vars.yml # set nmon_breakglass_password at least
ansible-playbook site.yml -e @vars.yml
The first play builds the release binary on your machine (`make build build-arm64`: needs Go, Node and network).
Pass `-e nmon_build=false` to reuse an existing `./nmon`. The second play installs it on the hosts in group `nmon`:
- `/usr/local/bin/nmon`, user `nmon`, data in `/var/lib/nmon`, key and TLS files in `/etc/nmon`, backups in `/var/backups/nmon`
- the MIB sidecar in `/usr/share/nmon/mibs`, used only by `nmon profile new --mib`
- optionally the IEEE vendor registry CSVs in `/usr/share/nmon/oui`, loaded with `nmon oui import` (`nmon_oui_download`, `nmon_oui_csv_dir`; by default nmon fetches the registry itself)
- `config.yaml` rendered from your vars, and a hardened systemd unit (`systemctl reload nmon` sends SIGHUP to reload config and certificates)
- `net.ipv4.ping_group_range` opened so nmon can ping without root
Then open `https://<host>:8443/`. Without OIDC you are sent to `/auth/breakglass` (a browser password prompt; user `admin` by default).
## TLS (`nmon_tls_mode`)
| Mode | What it does |
|---|---|
| `selfsigned` (default) | Generates a cert on the host. Good for trying it out; browsers will warn. |
| `files` | Copies `nmon_tls_cert_src` / `nmon_tls_key_src` from the control machine. |
| `acme` | Let's Encrypt for `nmon_server_name`; port 80 must reach the host. |
| `insecure` | Plain HTTP. With no sign-in configured, nmon runs in dev-open mode where everyone is admin. Never expose this. |
**Reverse proxy:** nmon can sit behind one. Keep the proxy-to-nmon hop on HTTPS (`selfsigned` is fine there), set
`nmon_trusted_proxies` to the proxy's addresses, and set the OIDC `redirect_url` to the public URL. Traps (UDP 162) and
Graphite (TCP 2003) go to nmon directly, not through the HTTP proxy.
## Sign-in
- **OIDC:** set `nmon_oidc` (issuer, client_id, client_secret, admin_group). The secret is stored encrypted with
`nmon secret set` and never written to `config.yaml`. Browsers go straight to the IdP; there is no login page.
- **Break-glass:** `nmon_breakglass_password` is bcrypt-hashed on the host. Use it when the IdP is down. Five wrong
attempts lock the source IP for 15 min; run `nmon unlock all --data-dir /var/lib/nmon` to clear.
## After installing
- **Back up `/etc/nmon/secret.key`** somewhere safe. Without it, a restore has no SNMP credentials.
- Add devices in the UI or with `POST /api/v1/devices`, and mint API or push tokens under Settings.
- The playbook owns `config.yaml`: edits made in the UI are overwritten on the next run, so put lasting settings in vars.
- More: `docs/operations.md`.

View File

@@ -0,0 +1,5 @@
[defaults]
inventory = inventory.ini
roles_path = roles
callback_result_format = yaml
host_key_checking = True

View File

@@ -0,0 +1,3 @@
# Copy to inventory.ini and edit. One nmon host is the design point.
[nmon]
nmon1.example.net ansible_user=admin ansible_become=true

View File

@@ -0,0 +1,43 @@
nmon_user: nmon
nmon_data_dir: /var/lib/nmon
nmon_etc_dir: /etc/nmon
nmon_key_file: "{{ nmon_etc_dir }}/secret.key"
nmon_backup_dir: /var/backups/nmon
nmon_mibs_dir: /usr/share/nmon/mibs
nmon_install_mibs: true
# MAC vendors (optional IEEE registry). nmon downloads it itself unless nmon_oui_fetch is false; to ship it instead,
# set nmon_oui_download (the host downloads the CSVs) or nmon_oui_csv_dir (a directory of the three CSVs on the control
# machine). Either way they are kept next to the MIB sidecar and loaded with `nmon oui import`.
nmon_oui_dir: /usr/share/nmon/oui
nmon_oui_fetch: true
nmon_oui_download: false
nmon_oui_csv_dir: ""
nmon_oui_urls:
- https://standards-oui.ieee.org/oui/oui.csv
- https://standards-oui.ieee.org/oui28/mam.csv
- https://standards-oui.ieee.org/oui36/oui36.csv
# MAC/ARP table job (see docs/operations.md); empty uses the defaults.
nmon_l2: {}
nmon_listen: "[::]:8443"
nmon_trap_listen: "[::]:162"
nmon_graphite_listen: "[::]:2003"
nmon_icmp: auto
nmon_timezone: UTC
nmon_tls_mode: selfsigned # selfsigned | files | acme | insecure
nmon_server_name: "{{ inventory_hostname }}"
nmon_acme_email: ""
nmon_tls_extra_names: [] # more DNS names / IPs for the self-signed cert
nmon_breakglass_user: admin
nmon_breakglass_password: "" # empty = no break-glass file (then OIDC must be configured)
nmon_oidc: {}
nmon_alerting: {}
nmon_trusted_proxies: []
nmon_backup_at: "02:30"
nmon_backup_keep: 7
nmon_retention:
hot: 31d
fiveminute: 120d
long: 2y

View File

@@ -0,0 +1,14 @@
- name: Restart nmon
ansible.builtin.systemd_service:
name: nmon
state: restarted
daemon_reload: true
- name: Reload nmon
ansible.builtin.systemd_service:
name: nmon
state: reloaded
- name: Apply sysctl
ansible.builtin.command: sysctl -p /etc/sysctl.d/60-nmon.conf
changed_when: true

View File

@@ -0,0 +1,182 @@
- name: Check the platform (nmon ships linux/amd64 and linux/arm64)
ansible.builtin.assert:
that:
- ansible_system == "Linux"
- ansible_architecture in ["x86_64", "aarch64"]
fail_msg: "nmon v1 supports linux/amd64 and linux/arm64 only"
- name: Require a sign-in method when serving HTTPS (nmon refuses to start otherwise)
ansible.builtin.assert:
that: nmon_tls_mode == "insecure" or nmon_breakglass_password | length > 0 or nmon_oidc
fail_msg: "Set nmon_breakglass_password and/or nmon_oidc, or use nmon_tls_mode=insecure for a dev-open playground"
- name: Install host packages (openssl for the self-signed cert, htpasswd for bcrypt, acl for become_user)
ansible.builtin.package:
name: "{{ ['openssl', 'apache2-utils', 'acl', 'procps'] if ansible_os_family == 'Debian' else ['openssl', 'httpd-tools', 'acl', 'procps-ng'] }}"
state: present
- name: Create the nmon system user
ansible.builtin.user:
name: "{{ nmon_user }}"
system: true
shell: /usr/sbin/nologin
home: "{{ nmon_data_dir }}"
create_home: false
- name: Create directories
ansible.builtin.file:
path: "{{ item.path }}"
state: directory
owner: "{{ nmon_user }}"
group: "{{ nmon_user }}"
mode: "{{ item.mode }}"
loop:
- { path: "{{ nmon_data_dir }}", mode: "0750" }
- { path: "{{ nmon_etc_dir }}", mode: "0750" }
- { path: "{{ nmon_backup_dir }}", mode: "0750" }
- name: Ensure /etc/sysctl.d exists (minimal images lack it)
ansible.builtin.file:
path: /etc/sysctl.d
state: directory
mode: "0755"
- name: Allow unprivileged ICMP echo (nmon pings without root)
ansible.builtin.copy:
content: "net.ipv4.ping_group_range = 0 2147483647\n"
dest: /etc/sysctl.d/60-nmon.conf
mode: "0644"
notify: Apply sysctl
- name: Install the nmon binary
ansible.builtin.copy:
src: "{{ playbook_dir }}/../../{{ 'nmon' if ansible_architecture == 'x86_64' else 'nmon-arm64' }}"
dest: /usr/local/bin/nmon
mode: "0755"
notify: Restart nmon
- name: Install the MIB sidecar
when: nmon_install_mibs | bool
block:
- name: Create the MIB directory
ansible.builtin.file:
path: "{{ nmon_mibs_dir }}"
state: directory
mode: "0755"
- name: Unpack MIBs
ansible.builtin.unarchive:
src: "{{ playbook_dir }}/.mibs.tar.gz"
dest: "{{ nmon_mibs_dir }}"
creates: "{{ nmon_mibs_dir }}/IF-MIB"
- name: Ship the IEEE vendor registry (optional)
when: nmon_oui_download | bool or nmon_oui_csv_dir | length > 0
block:
- name: Create the registry directory
ansible.builtin.file:
path: "{{ nmon_oui_dir }}"
state: directory
mode: "0755"
- name: Download the registry CSVs on the host
when: nmon_oui_download | bool
ansible.builtin.get_url:
url: "{{ item }}"
dest: "{{ nmon_oui_dir }}/{{ item | basename }}"
mode: "0644"
timeout: 60
loop: "{{ nmon_oui_urls }}"
register: nmon_oui_fetched
- name: Copy the registry CSVs from the control machine
when: nmon_oui_csv_dir | length > 0
ansible.builtin.copy:
src: "{{ nmon_oui_csv_dir }}/"
dest: "{{ nmon_oui_dir }}/"
mode: "0644"
register: nmon_oui_copied
- name: Load the registry into nmon's data directory
when: (nmon_oui_fetched is changed) or (nmon_oui_copied is changed)
become_user: "{{ nmon_user }}"
ansible.builtin.command:
argv: [/usr/local/bin/nmon, oui, import, "{{ nmon_oui_dir }}", --data-dir, "{{ nmon_data_dir }}"]
changed_when: true
- name: TLS certificate
ansible.builtin.include_tasks: tls.yml
when: nmon_tls_mode in ["selfsigned", "files"]
- name: Break-glass admin (bcrypt hash; the password never lands on disk)
when: nmon_breakglass_password | length > 0
block:
- name: Hash the break-glass password
ansible.builtin.command:
argv: [htpasswd, -nbBC, "12", "{{ nmon_breakglass_user }}", "{{ nmon_breakglass_password }}"]
register: nmon_bg_hash
changed_when: false
no_log: true
- name: Write the break-glass file
ansible.builtin.copy:
content: "{{ nmon_bg_hash.stdout | trim }}\n"
dest: "{{ nmon_etc_dir }}/breakglass"
owner: "{{ nmon_user }}"
group: "{{ nmon_user }}"
mode: "0600"
no_log: true
# Re-hashing gives a new salt each run; only replace the file when it is missing.
args:
force: false
- name: Store the OIDC client secret encrypted (creates the key file on a new install)
ansible.builtin.shell:
cmd: >-
set -o pipefail;
printf '%s\n' "$NMON_OIDC_SECRET" |
/usr/local/bin/nmon secret set oidc-client --data-dir {{ nmon_data_dir }} --secret-key-file {{ nmon_key_file }}
executable: /bin/bash
environment:
NMON_OIDC_SECRET: "{{ nmon_oidc.client_secret }}"
become: true
become_user: "{{ nmon_user }}"
when: nmon_oidc.client_secret is defined
changed_when: true
no_log: true
notify: Restart nmon
- name: Write config.yaml
ansible.builtin.template:
src: config.yaml.j2
dest: "{{ nmon_data_dir }}/config.yaml"
owner: "{{ nmon_user }}"
group: "{{ nmon_user }}"
mode: "0640"
notify: Reload nmon
- name: Install the systemd unit
ansible.builtin.template:
src: nmon.service.j2
dest: /etc/systemd/system/nmon.service
mode: "0644"
notify: Restart nmon
- name: Enable and start nmon
ansible.builtin.systemd_service:
name: nmon
enabled: true
state: started
daemon_reload: true
- name: Wait for nmon to answer
ansible.builtin.uri:
url: "{{ 'http' if nmon_tls_mode == 'insecure' else 'https' }}://localhost:{{ nmon_listen | regex_replace('^.*:', '') }}/healthz"
validate_certs: false
status_code: 200
register: nmon_health
retries: 30
delay: 2
until: nmon_health.status == 200
- name: Where to go next
ansible.builtin.debug:
msg:
- "nmon is up: {{ 'http' if nmon_tls_mode == 'insecure' else 'https' }}://{{ nmon_server_name }}:{{ nmon_listen | regex_replace('^.*:', '') }}/"
- "{{ 'Signs in via your IdP; break-glass at /auth/breakglass' if nmon_oidc else 'Sign in at /auth/breakglass as ' + nmon_breakglass_user }}"
- "Back up the key file {{ nmon_key_file }} somewhere safe; without it a restore has no SNMP credentials."

View File

@@ -0,0 +1,51 @@
- name: Work out the certificate's names (IP addresses need IP:, names need DNS:)
ansible.builtin.set_fact:
nmon_tls_sans: >-
{{ (([nmon_server_name] + nmon_tls_extra_names) | unique
| map('regex_replace', '^(?=[0-9.]+$|.*:)', 'IP:')
| map('regex_replace', '^(?!IP:)', 'DNS:') | list)
+ ['DNS:localhost', 'IP:127.0.0.1', 'IP:::1'] }}
when: nmon_tls_mode == "selfsigned"
- name: Generate a self-signed certificate (quick start; replace for production)
ansible.builtin.command:
argv:
- openssl
- req
- -x509
- -newkey
- ec
- -pkeyopt
- ec_paramgen_curve:prime256v1
- -nodes
- -days
- "825"
- -subj
- "/CN={{ nmon_server_name }}"
- -addext
- "subjectAltName={{ nmon_tls_sans | join(',') }}"
- -keyout
- "{{ nmon_etc_dir }}/tls.key"
- -out
- "{{ nmon_etc_dir }}/tls.crt"
creates: "{{ nmon_etc_dir }}/tls.crt"
when: nmon_tls_mode == "selfsigned"
notify: Restart nmon
- name: Install the provided certificate and key
ansible.builtin.copy:
src: "{{ item.src }}"
dest: "{{ nmon_etc_dir }}/{{ item.dest }}"
mode: "{{ item.mode }}"
loop:
- { src: "{{ nmon_tls_cert_src }}", dest: tls.crt, mode: "0644" }
- { src: "{{ nmon_tls_key_src }}", dest: tls.key, mode: "0600" }
when: nmon_tls_mode == "files"
notify: Reload nmon
- name: Certificate ownership
ansible.builtin.file:
path: "{{ nmon_etc_dir }}/{{ item }}"
owner: "{{ nmon_user }}"
group: "{{ nmon_user }}"
loop: [tls.crt, tls.key]

View File

@@ -0,0 +1,38 @@
# Managed by Ansible (deploy/ansible). Admin edits in the UI are overwritten on the next run.
retention:
hot: {{ nmon_retention.hot }}
fiveminute: {{ nmon_retention.fiveminute }}
long: {{ nmon_retention.long }}
timezone: {{ nmon_timezone }}
{% if nmon_l2 %}
l2:
{{ nmon_l2 | to_nice_yaml(indent=2) | indent(2) }}
{% endif %}
{% if not nmon_oui_fetch | bool %}
oui:
fetch: false
{% endif %}
{% if nmon_alerting %}
alerting:
{{ nmon_alerting | to_nice_yaml(indent=2) | indent(2) }}
{% endif %}
auth:
{% if nmon_oidc %}
oidc:
issuer: {{ nmon_oidc.issuer }}
client_id: {{ nmon_oidc.client_id }}
client_secret_name: oidc-client
redirect_url: {{ nmon_oidc.redirect_url | default('https://' + nmon_server_name + ':' + (nmon_listen | regex_replace('^.*:', '')) + '/auth/callback') }}
groups_claim: {{ nmon_oidc.groups_claim | default('groups') }}
admin_group: {{ nmon_oidc.admin_group | default('nmon-admins') }}
{% endif %}
{% if nmon_breakglass_password | length > 0 %}
break_glass_file: {{ nmon_etc_dir }}/breakglass
{% endif %}
{% if nmon_trusted_proxies %}
trusted_proxies: {{ nmon_trusted_proxies | to_json }}
{% endif %}
backup:
at: "{{ nmon_backup_at }}"
dir: {{ nmon_backup_dir }}
keep: {{ nmon_backup_keep }}

View File

@@ -0,0 +1,47 @@
# Managed by Ansible (deploy/ansible).
[Unit]
Description=nmon network monitor
Documentation=https://claude.ai/artifact/7FTnfA5hReTjY6cfArfJ2Z
After=network-online.target
Wants=network-online.target
[Service]
User={{ nmon_user }}
Group={{ nmon_user }}
ExecStart=/usr/local/bin/nmon serve \
--data-dir {{ nmon_data_dir }} \
--secret-key-file {{ nmon_key_file }} \
--listen {{ nmon_listen }} \
--trap-listen {{ nmon_trap_listen }} \
--graphite-listen {{ nmon_graphite_listen }} \
--icmp {{ nmon_icmp }} \
{% if nmon_tls_mode == 'insecure' %}
--insecure-http
{% elif nmon_tls_mode == 'acme' %}
--acme-host {{ nmon_server_name }} --acme-email {{ nmon_acme_email }}
{% else %}
--tls-cert {{ nmon_etc_dir }}/tls.crt --tls-key {{ nmon_etc_dir }}/tls.key
{% endif %}
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure
RestartSec=5
TimeoutStopSec=90
LimitNOFILE=1048576
# Ports below 1024: SNMP traps (162) and the ACME challenge (80).
AmbientCapabilities=CAP_NET_BIND_SERVICE
CapabilityBoundingSet=CAP_NET_BIND_SERVICE
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
PrivateTmp=true
PrivateDevices=true
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectControlGroups=true
RestrictSUIDSGID=true
LockPersonality=true
ReadWritePaths={{ nmon_data_dir }} {{ nmon_etc_dir }} {{ nmon_backup_dir }}
[Install]
WantedBy=multi-user.target

26
deploy/ansible/site.yml Normal file
View File

@@ -0,0 +1,26 @@
# Deploys nmon to the hosts in group "nmon". Run from the repo root's deploy/ansible:
# ansible-playbook site.yml -e @vars.yml
- name: Build nmon locally
hosts: localhost
connection: local
gather_facts: false
tags: [build]
tasks:
- name: Build the release binary (UI and VictoriaMetrics embedded)
ansible.builtin.command:
cmd: make build build-arm64
chdir: "{{ playbook_dir }}/../.."
changed_when: true
when: nmon_build | default(true) | bool
- name: Pack the MIB sidecar # noqa: command-instead-of-module (archive needs community.general; plain tar keeps this collection-free)
ansible.builtin.command:
cmd: tar -czf {{ playbook_dir }}/.mibs.tar.gz -C {{ playbook_dir }}/../../mibs .
creates: "{{ playbook_dir }}/.mibs.tar.gz"
when: nmon_install_mibs | default(true) | bool
- name: Install and run nmon
hosts: nmon
become: true
roles:
- nmon

View File

@@ -0,0 +1,38 @@
# Copy to vars.yml (keep it out of git, or encrypt with ansible-vault) and pass with -e @vars.yml.
# TLS: selfsigned (quick start), files (your own cert), acme (Let's Encrypt), or insecure (plain HTTP, dev only).
nmon_tls_mode: selfsigned
nmon_server_name: "{{ inventory_hostname }}" # a DNS name or an IP; it goes in the self-signed cert
# nmon_tls_extra_names: ["10.0.0.5", "nmon.lan"]
# nmon_tls_cert_src: files/nmon.crt # for tls_mode=files (paths on the control machine)
# nmon_tls_key_src: files/nmon.key
# nmon_acme_email: noc@example.net # for tls_mode=acme; nmon answers http-01 on :80
# Break-glass local admin: the way in when the IdP is down (and the only sign-in until OIDC is set).
nmon_breakglass_user: admin
nmon_breakglass_password: "change-me-please"
# OIDC (optional). Unauthenticated browsers are redirected to the IdP; there is no login page.
# nmon_oidc:
# issuer: https://idp.example.net/realms/net
# client_id: nmon
# client_secret: "s3cret" # stored encrypted via `nmon secret set`, never in config.yaml
# groups_claim: groups
# admin_group: nmon-admins
# Alert receivers (optional): webhook and/or SMTP, plus the outbound allow-list they need.
# nmon_alerting:
# receivers:
# - name: noc-webhook
# kind: webhook
# url: https://hooks.example.net/nmon
# outbound_allow: ["hooks.example.net"]
nmon_timezone: UTC
# MAC/ARP table job and optional IEEE vendor registry (see docs/operations.md).
# nmon_l2: {interval: 15m, keep: 90d, edge_max_macs: 8}
# nmon_oui_fetch: false # never let nmon download the registry...
# nmon_oui_download: true # ...have this play download the CSVs on the host instead,
# nmon_oui_csv_dir: files/oui # ...or ship the three CSVs from the control machine
nmon_icmp: auto # off = decide reachability by SNMP alone

View File

@@ -42,3 +42,13 @@ conscious, reviewed exception.
| 35 | Port descriptions imported from CSV | Interfaces have a new operator-owned `description` column (migration 8). The CSV import and the API write it; discovery rewrites `ifalias` from the device on every run and never touches `description`. Interface search `q` matches ifname, ifalias and description. | A description typed into `ifalias` would be overwritten at the next discovery. | M6 |
| 36 | OIDC login with a browser flow; break-glass as a local login | **No login UI** (owner decision). Every route except `/auth/*`, `/healthz` and push ingest needs a credential, the SPA's files included: an unauthenticated page load is a 302 to `/auth/login?return=<url>` (the OIDC flow; the return target must be a same-site path), an API call without credentials is a 401 problem+json. Break-glass is `GET /auth/breakglass` with HTTP Basic, so the browser's own prompt appears; success sets a session and redirects to `return` or `/` (`POST /auth/breakglass` with JSON stays for scripts). When no IdP is configured the redirect goes to the break-glass prompt. Logout is `POST /auth/logout`, then `/auth/signedout` (static HTML). `/docs` and `/openapi*` need the viewer role. A missing OIDC client secret at start leaves OIDC off with a warning. | The IdP is the login page; a second one would be a second thing to secure. | M6 |
| 37 | Disk guard: "ingest throttled below the reserve" | Throttle answers push ingest with 429 + Retry-After below the 5% reserve; the poller keeps writing (its volume is bounded by the inventory). Above 90% used, new series are dropped on the nmon side (poller, push and checks) while known series continue; disk alarms are re-raised each minute while the condition holds and resolve themselves 3 minutes after it clears. | VictoriaMetrics series limits are start-up flags; the guard is nmon-side admission. | M6 |
| 38 | `interface_down` is a default rule that fires for every down port | `interface_down` is **opt-in**: it alerts only for ports whose own `alert_down` flag is on, or that belong to a group with `alert_down` on (membership in force, `GroupMember.Covers`). Migration 9 adds the columns, default off. Opting a port out closes its open alert silently as `retired` (never a resolved notification). On an access switch most unused ports are admin-up and oper-down, so the default flooded the owner with alerts. `interface_errors` is unchanged. | Owner feedback, first hands-on session. | post-v1 (v1.0.1) |
| 39 | Spec: a profile that was matched stays ("non-generic is sticky"); no way to choose one by hand | Discovery re-matches **every unpinned device on every run**, so a newer or better profile is picked up, and at startup nmon queues a discovery for each unpinned device whose stored profile differs from what `Set.Match` picks now. An admin can pin a profile (`PUT /api/v1/devices/{slug}` with `profile`, or the Edit dialog); a pinned device keeps it, and a pinned profile that no longer exists gives generic plus the `profile_missing` warning as before. `profile: ""` returns to automatic. Migration 10 adds `devices.profile_pinned`; `GET /api/v1/profiles` lists the loaded profiles; `DeviceView` gains `profile_pinned` and `profile_match`. Profile changes are audited (`device.profile` from discovery, `device.edit` from the API). | The sticky rule meant a device matched once as generic stayed generic after an upgrade added its profile. | post-v1 (v1.0.1) |
| 40 | Profiles shipped: the M2 set | Three more shipped profiles for the owner's gear: `ubiquiti-edgeswitch` (sysObjectID `1.3.6.1.4.1.4413` or sysDescr `^EdgeSwitch`, priority 30), `ubiquiti-unifi-switch` (sysDescr `USW`/`US-`/`USL`/`UniFi`, priority 40, so it beats edgeswitch for UniFi switches that report enterprise 4413), `vyos` (sysObjectID `1.3.6.1.4.1.44641` or sysDescr `VyOS`, extends `linux-netsnmp`, priority 20), and `ubiquiti-unifi-ap` (sysDescr `UAP`/`U6`/`U7`/`UAL`/`UK`/`E7`/`U-LTE`, UBNT-UniFi-MIB per-VAP clients, channel and tx power, per-radio channel utilisation; the MIB has no noise floor, and no per-radio channel or tx power, so those are per VAP). EdgeSwitch CPU is a text string and its temperature table has no label column, so neither is collected. | The owner's switches, routers and access points matched only `generic`. | post-v1 (v1.0.1) |
| 41 | Default rules apply everywhere (spec §Alerting) | Alerts fire only where an **alert policy** binds a rule to a scope (groups plus device and port attribute matchers), stored in the `alert_policies` table (migration 12), edited through `/api/v1/alert-policies` and the Alerts > Alert policies tab, and applied live. With no policies nothing alerts: not metric rules, not `device_down`/`device_warning`, not system alarms (`system` policy, scope `all` only). An `ignore` policy beats an `alert` one at equal priority. Upgrading seeds nothing, so open alerts without a binding close silently as `retired` with no resolved notice. Supersedes #38's mechanism (the bell now only feeds the `alert_down` matcher) and makes `Rule.OptIn` a no-op. | Owner decision 2026-10-23: preprod alerted on everything, and a WiFi port that always sees errors is judged by the same rule as a fiber uplink where one error matters. | v1.1 |
| 50 | The spec's collection table lists BGP-4-MIB but no profile fills it, and it names no MAC/ARP lookup | v1.3 (owner request 2026-10-24) adds both: BGP peers in the `routing` category (CISCO-BGP4 for cisco-ios/cisco-nxos, Juniper BGP4-V2, Arista BGP4V2, BGP4-MIB for the IPv4-only profiles that answer it) with a Routing tab, a BGP page, `GET /api/v1/bgp/peers` and the built-in `bgp_peer_down` rule; and a MAC/ARP/FDB lookup with a Find host page. Additive API: `/l2/search`, `/devices/{slug}/{l2,fdb,arp,l2/ports}`, `/interfaces/{id}/macs`, `/bgp/peers`; `/poller/health` gains `l2` and `notes`. | The owner wants to find where a host is plugged in and to see BGP sessions. | v1.3 |
| 51 | (not specified) table walks run in discovery or not at all; the poll path never walks | A separate **L2 table job** (`internal/l2`, SNMP path label `l2` in `nmon_pdus_total`) walks BRIDGE-MIB, Q-BRIDGE-MIB, CISCO-VTP-MIB (per-VLAN context, profile flag `l2.per_vlan_context`, set on cisco-ios only) and IP-MIB neighbour tables every `l2.interval` (default 15m, minimum 5m) with its own concurrency cap (`l2.concurrency`, 4), a per-device deadline (`l2.device_timeout`, 2m) and the rollup job's yield rule (it pauses while the hot write queue is over half full). A device that times out or fails is skipped for the cycle, recorded on `/poller/health`, and never delays the others. Rows live in the inventory database (migration 14: `fdb_entries`, `arp_entries`, `vlans`, and `l2_walks`, one row per device holding the time of its last walk, which is how a row is "current"), are in `nmon backup` (JSONL export too) and are pruned daily by `last_seen` against `l2.keep` (90d). `interface_id` is the empty string, not NULL, when a bridge port has no ifIndex, because the unique key must not contain NULL. | The hot poll path must not walk tables; the history behind "where was this MAC last week" is inventory, not a metric. | v1.3 |
| 52 | The component label of a table metric is a column of the table | Profiles may label rows by decoding the **row index** (`index_label.layout`, tokens `skip`, `skipinet`, `inet`, `skipaddr`, `addr`, `ipv4`, `peerindex`, `afisafi`), because BGP peer tables are indexed by an address. The label is the peer address (IPv6 compressed); a per-address-family prefix count is labelled `<peer>\|<afi>-<safi>`, for example `2001:db8::1\|ipv6-unicast`. Juniper's prefix table is keyed by a peer index, so its rows resolve through the `peer_index` attribute of the peer table (`index_label.via`). Peer AS, local AS, router id and description are component attributes (migration 11), never labels, so the SNMP label contract (`device` + `component`) is unchanged. Other profile additions: `enable_metrics` (switch on optional parent metrics), `wire: counter32` (CISCO-BGP4 declares accepted prefixes as a Counter32 but they are a count), attribute `format: ipbytes`, and `l2.per_vlan_context`. | A BGP peer is identified by its address, which only the index carries; the alternatives (a label column that does not exist, or labelling by row number) would break "each peer appears once" and the alert text. | v1.3 |
| 53 | `bgp_peer_down` summary names the peer, AS and description | The rule's own summary names the device and the peer. The AS, router id and description are appended to the **notification** (Summary, Description and labels) when it is delivered, from the component attributes; the stored alert row keeps the plain summary. | The metric label contract allows only `device` and `component`, and the rule templates see only those; extending the evaluator's template data belongs to the v1.2 rules plumbing. | v1.3 |
| 54 | (not specified) vendor of a MAC | An optional IEEE OUI registry (MA-L, MA-M, MA-S) shows the vendor of each MAC and flags locally administered (`local`, shown "Private / randomised") and multicast addresses. It is fetched weekly through the outbound allow-list (`oui.fetch: false` turns it off; `nmon oui import DIR` loads the CSVs offline), kept as `<data-dir>/oui/registry.gz` (checksummed, written atomically, in `nmon backup`), and looked up at query time by the longest prefix, so a refresh relabels history. Without a registry the vendor is empty, nothing fails, and `/poller/health` carries an `oui_unavailable` note (never an alert). The CSVs are public IEEE data and are not in the repository. | Owner request 2026-10-24: nice to have, never required. | v1.3 |
| 55 | Lookup ranks edge ports "the port with the fewest MACs, then the most recent last_seen" | Within a MAC's locations the order is: edge ports before uplinks, then rows the device's last walk still saw before history, then fewest MACs, then most recent. A port is an uplink above `l2.edge_max_macs` MACs (8), with an LLDP neighbour (the lldpRemTable local port number is taken to be the ifIndex), or as a LAG (ifType 161). A port's MAC count is the MACs on it at or after the row's `last_seen`, so for a current row it is the port's MACs in the last walk. | A host that moved should show its new port first even when the old, now empty, port had fewer MACs. | v1.3 |

91
docs/coverage.md Normal file
View File

@@ -0,0 +1,91 @@
# Collection coverage
What the spec's "What to collect, by device class" table (docs/spec/spec.md) asks for, against what the shipped
profiles in `profiles/` collect today. A metric listed here is polled, stored raw and shown on a device tab (every
non-interface metric names a `category`; the orphan test in `internal/profile/present_test.go` enforces it). Gaps are
stated plainly; nothing here is promised that is not collected.
Legend: **yes** collected and shown; **partial** some of it; **no** not collected.
| Class | Spec item | Status | Profile and metric |
|---|---|---|---|
| Every device | sysUpTime | yes | generic: sysUpTime, hrSystemUptime |
| Every device | hrProcessorLoad per core | yes | generic: hrProcessorLoad |
| Every device | hrStorage size/used per filesystem and RAM | yes | generic: hrStorageUsed / hrStorageSize x hrStorageAllocationUnits (bytes, percent bar) |
| Every device | memTotalReal / memAvailReal (UCD) | yes | generic: memTotalReal, memAvailReal |
| Servers | UCD load average | yes | linux-netsnmp, ubiquiti-edgeos: laLoadInt |
| Servers | swap | yes | linux-netsnmp: memTotalSwap, memAvailSwap |
| Routers, switches | IF-MIB counters, ifOperStatus | yes | generic (Ports tab) |
| Routers, switches | ENTITY-SENSOR-MIB temp/fan/PSU | partial | generic entPhySensorValue is the raw reading only: its type, scale and precision are not applied, so units are mixed (environment tab, labelled raw). No fan or PSU state. cisco-ios/cisco-nxos entSensorValue is the same |
| Routers, switches | POWER-ETHERNET-MIB PoE per port | no | not in any profile |
| Routers, switches | LLDP neighbours | yes | generic discovery table only (no metric) |
| Routers, switches | BGP-4-MIB peer state and prefixes | partial | see "BGP peers" below: state, uptime, accepted prefixes per address family and update counters where the device answers a BGP MIB; nothing on MikroTik or Linux hosts |
| Routers, switches | optics DOM | partial | juniper-junos jnxDom* are collected but are interface metrics, so they are not on a device tab (there is no optics tab data yet); cisco entSensor DOM is not separated from the raw sensor reading |
| Routers, switches | STP topology changes | no | |
| Routers, switches | MAC/ARP table size | yes | the tables themselves, not just their size: the L2 table job (`internal/l2`) reads FDB, VLAN names and ARP/ND from every SNMP device (see "MAC, ARP and FDB" below) |
| Routers, switches | cpmCPUTotal, jnxOperatingCPU | yes | cisco-ios, cisco-nxos: cpmCPUTotal5minRev; juniper-junos: jnxOperatingCPU, jnxOperatingTemp, jnxOperatingBuffer |
| Routers, switches | Cisco memory pools | yes | cisco-ios: ciscoMemoryPoolUsed / Free (not as a percent; there is no total) |
| Switches (Ubiquiti) | memory | partial | ubiquiti-edgeswitch, ubiquiti-unifi-switch: free / available only; CPU is a text string in that MIB and is not collected |
| Firewalls | session count | partial | fortigate: fgSysSesCount (no maximum); pfsense has no vendor metrics |
| Firewalls | VPN tunnel state | no | |
| Firewalls | HA state | partial | fortigate: fgHaSystemMode only |
| Firewalls | policy throughput (fgSys, panSession, pfSense) | no | fortigate CPU and memory only |
| APs, controllers | client count per AP and SSID | yes | ubiquiti-unifi-ap: unifiVapNumStations per SSID |
| APs, controllers | channel utilization | yes | ubiquiti-unifi-ap: unifiRadioCu* per radio |
| APs, controllers | noise floor | no | the UniFi MIB has no such object |
| APs, controllers | tx power | yes | ubiquiti-unifi-ap: unifiVapTxPower |
| APs, controllers | AP up/down from controller | no | no controller integration |
| WISP radios, CPE | signal, SNR, modulation, link capacity | no | no Ubiquiti airMAX, Cambium or Mimosa profile |
| Servers, hypervisors | Proxmox push | yes | push receiver and the proxmox integration (not a profile) |
| Servers, hypervisors | node_exporter, Ceph, SMART | no | |
| UPS, PDU | battery charge, runtime, input/output voltage, load, output status | yes | apc-ups: UPS-MIB and PowerNet upsAdv/upsBasic metrics (power tab) |
| UPS, PDU | per-outlet current (PDU MIBs), generator status | no | |
| Printers | supply levels per prtMarkerSupplies | yes | printer: prtMarkerSuppliesLevel of prtMarkerSuppliesMaxCapacity; -3 "some remaining" and -2 "unknown" are stored and shown as words |
| Printers | supply type and colour | yes | printer: prtMarkerSuppliesType and prtMarkerColorantValue, read at discovery as component attributes |
| Printers | page count | yes | printer: prtMarkerLifeCount |
| Printers | status and alerts | partial | printer: prtAlertSeverityLevel (severity only, not the alert description or code) |
| Printers | paper tray levels | yes | printer: prtInputCurrentLevel of prtInputMaxCapacity, labelled by prtInputName |
| Printers | hp, Brother, Xerox serial and model | no | serial and model are not collected; no vendor quirk profiles (printer-brother, printer-ricoh) are shipped, see below |
| Environment | ENTITY-SENSOR-MIB | partial | as for routers: raw reading only |
| Environment | temperature, humidity, door, water (NetBotz, Raritan) | no | no profile |
| Environment | MikroTik health | yes | mikrotik-routeros: voltage, board and CPU temperature |
| Services | ping, HTTP, DNS, TLS expiry | yes | checks (not profiles) |
Profiles with no vendor metrics beyond generic: arista-eos, pfsense, vyos, windows-snmp.
## Alerts named in the spec
| Alert | Status |
|---|---|
| cpu_high, memory_high, disk_full | yes |
| ups_on_battery, ups_runtime_low | yes |
| printer_supply_low (< 10%) | yes |
| printer_supply_empty (level 0, critical) | yes (added in v1.0.1) |
| temp_high (per-profile threshold) | yes, for MikroTik, UPS battery and Junos sensors only |
| "any prtAlert of severity critical" | no rule; prtAlertSeverityLevel is collected and shown, not alerted on |
## Printer quirk profiles (Brother, Ricoh)
Not shipped. The LibreNMS recordings (`tests/snmpsim/brother*.snmprec`, `ricoh*.snmprec`) were not used to derive a
quirk set, so none is claimed. The generic `printer` profile matches Brother (2435) and Ricoh (367) by enterprise number.
## BGP peers (v1.3)
The component is the peer address (IPv6 compressed). Metrics: `bgp_peer_state` (idle, connect, active, opensent, openconfirm, established; anything but established is critical), `bgp_peer_uptime`, `bgp_peer_in_updates` / `bgp_peer_out_updates` (counters, shown as a rate) and, where the vendor reports prefixes per address family, `bgp_prefixes_accepted` with the component `<peer>|<afi>-<safi>`. Peer AS, local AS, router id (and the description, where the MIB has one) are component attributes from discovery. A peer is reported once: a profile with a vendor table replaces the BGP4-MIB metrics of the same names.
| Profile | Source | IPv6 peers | Prefixes | Evidence |
|---|---|---|---|---|
| cisco-ios | CISCO-BGP4-MIB cbgpPeer2Table and cbgpPeer2AddrFamilyPrefixTable | yes | per AFI/SAFI | IOS XE and IOS XR recordings (`iosxe`, `iosxr`, `iosxr_asr9904`) |
| cisco-nxos | CISCO-BGP4-MIB, as cisco-ios | yes | per AFI/SAFI | **not verified**: the NX-OS recordings carry no BGP table; kept because Cisco documents the MIB on NX-OS, and a device that does not answer simply has no peers |
| juniper-junos | BGP4-V2-MIB-JUNIPER jnxBgpM2PeerTable, jnxBgpM2PrefixCountersTable | yes | per AFI/SAFI (resolved through jnxBgpM2PeerIndex) | `junos_mx`, `junos_vmx`, `junos_rpm`; the peer index has no length octets, as Junos sends it |
| arista-eos | ARISTA-BGP4V2-MIB aristaBgp4V2PeerTable, aristaBgp4V2PrefixTable | yes | per AFI/SAFI | `arista_eos`, `arista_eos_7280r`; description is collected |
| ubiquiti-edgeos, fortigate, pfsense, vyos | BGP4-MIB bgpPeerTable (generic metrics switched on with `enable_metrics`) | no (RFC 4273 is IPv4 only) | no | `edgeos`, `fortigate_60f3g4g`, `pfsense_frr-bgp` (FRR's agentx BGP4-MIB); VyOS uses FRR the same way but has no recording |
| generic and every other profile | BGP4-MIB metrics exist but are off (`optional`) | no | no | a router profile turns them on with `enable_metrics` |
| mikrotik-routeros | **nothing** | | | MIKROTIK-MIB has no BGP objects and none of the RouterOS recordings answers a BGP table; RouterOS BGP is not visible over SNMP |
| linux-netsnmp, windows-snmp, apc-ups, printer, ubiquiti switches and APs | **nothing** | | | not routers |
FRR on Cumulus (`cumulus_bgp`) and Dell OS (`dnos_bgp`) answer BGP4-MIB, but no shipped profile matches those devices, so their peers are collected only once a profile for them enables the generic metrics.
## MAC, ARP and FDB (v1.3)
Read by the L2 table job for every SNMP device, from BRIDGE-MIB (`dot1dBasePortIfIndex`, `dot1dTpFdbPort`), Q-BRIDGE-MIB (`dot1qTpFdbPort`, `dot1qVlanFdbId`, `dot1qVlanStaticName`), CISCO-VTP-MIB (`vtpVlanState`, `vtpVlanName`, only with the cisco-ios flag `l2.per_vlan_context`, walking the bridge table once per VLAN with community `<community>@<vlan>` or v3 context `vlan-<vlan>`) and IP-MIB (`ipNetToPhysicalPhysAddress` for IPv4 ARP and IPv6 ND, `ipNetToMediaPhysAddress` as the IPv4-only fallback). Devices without a bridge MIB (servers, firewalls without a switch, most APs) have no FDB rows. The FDB is VLAN-aware only through Q-BRIDGE or the Cisco per-VLAN walk; BRIDGE-MIB alone gives VLAN 0.

View File

@@ -18,6 +18,7 @@ its **Gate log** is filled in when the gate passes. A milestone is done only whe
| M4 | Alerting | stopping a parent fixture sends one notification naming it and its children | [M4](M4-alerting.md) | **passed**, tagged `m4` |
| M5 | Push ingest, rollup, checks | a replayed Proxmox sample shows under its node and a 2 y query hits the 1 h tier | [M5](M5-ingest-rollup-checks.md) | **passed**, tagged `m5` |
| M6 | Auth and ops | a viewer gets 403 on PUT (2,000-device loadtest skipped, approved deviation) | [M6](M6-auth-and-ops.md) | **passed**, tagged `m6` |
| v1.3 | BGP peers and MAC/ARP/FDB lookup | MAC/IP lookup, BGP peers and `bgp_peer_down`, see the gate list in the doc | [v1.3](v1.3-bgp-and-l2-tables.md) | **built**, gate log filled in (not tagged) |
| M7 | gNMI streaming | after v1; not built in this run | — | out of scope |
## How the work is run
@@ -49,6 +50,14 @@ Departures from the spec are recorded in [`docs/DEVIATIONS.md`](../DEVIATIONS.md
- **Web push notifications (v2)**: a `webpush` receiver kind in `internal/notify` (RFC 8291 payload encryption, RFC 8292 VAPID, standard-library crypto), per-user subscriptions (needs M6 users), a service worker in the UI, and push endpoints in the outbound allow-list. Out of v1 per the spec's non-goals ("alerts go to webhooks and email only"). Owner deferred it to v2 on 2026-10-22. Until then, a webhook receiver pointed at ntfy or Gotify gives phone push.
- **BMC support**: SNMP profiles for Dell iDRAC, HPE iLO and Supermicro (health, temperatures, fans, PSUs, power, RAID) feeding the Health/Power/Environment tabs; a native Redfish collector later. Today, Redfish works only via an exporter pushing with a push token. Owner: "good enough for now" (2026-10-23).
## Post-v1 additions
- **Device management UI** (2026-10-23): the UI had no way to add a device. Added an admin-only "Add device" dialog on the device list (IPv6/IPv4 addresses, SNMP v1/v2c/v3, synchronous discovery with the reattach 409 flow) and an Actions menu on the device page (Edit with parents and SNMP replacement, Rediscover now, Retire; Purge now asks for the device name). New API route `POST /api/v1/devices/{slug}/retire` (admin, audited, additive). The SNMP fields are one shared component (`web/src/components/SnmpFields.tsx`). Screenshots: `docs/screenshots/devices-mgmt/`.
- **Profiles, profile pinning and row navigation** (2026-10-23): new shipped profiles `ubiquiti-edgeswitch`, `ubiquiti-unifi-switch`, `ubiquiti-unifi-ap` and `vyos`; discovery re-matches unpinned devices every run (and at startup) and an admin can pin a profile from the Edit dialog (DEVIATIONS 39, 40); the device header explains why a device is generic. On the device list, the interface search results and the poller health table, a click anywhere in a row (or Enter on a focused row) opens the device or graph; the name stays a real link. Screenshots: `docs/screenshots/v1.0.1/`.
## Review artifacts
- From M3 on, each UI milestone's gate captures Playwright screenshots of every page into `docs/screenshots/<milestone>/` for the user to review.

View File

@@ -0,0 +1,70 @@
# v1.0.1: device tabs (Health, Wireless, Printer, …) and a printer fleet view
Owner request (2026-10-23): non-interface metrics (CPU, memory, storage, sensors, UPS, wireless, printer supplies) are
collected but not shown anywhere. Build LibreNMS-style device tabs before production, driven by profile metadata so
that any device class shows up without per-vendor UI code.
## Profile schema additions (internal/profile)
Per metric, all optional except `category`, which is **required for every pollable metric** (enforced by a test over all
shipped profiles and by the loader for new ones; `imported: librenms` drafts default to `health`):
- `category`: `system | health | power | environment | wireless | printer | routing | optics | firewall`
- `label`: human name ("CPU load", "Toner level"); defaults to the metric name
- `of: <metric>`: the capacity/total this metric is a fraction of (hrStorageUsed `of: hrStorageSize`,
prtMarkerSuppliesLevel `of: prtMarkerSuppliesMaxCapacity`, memAvailReal `of: memTotalReal`). The UI shows a percent bar,
computed at query time (raw stays raw, per the spec).
- `display`: `gauge` (default for gauge), `percent` (implied by `of`), `state` (implied by `type: status`), `rate`
(implied by counter types), `text`
- `scale_by: <metric>` (optional): multiply by another column, e.g. hrStorageUsed × hrStorageAllocationUnits for bytes
- `sentinels: {<value>: <text>}` (optional): values shown as words instead of numbers, e.g. printer `-3: "some remaining"`,
`-2: "unknown"`. These are polled and stored (not dropped as not_available).
Every shipped profile is annotated. Printer gains `prtMarkerSuppliesType` (enum) and `prtMarkerColorantValue`, collected at
discovery as component attributes, plus the paper trays (`prtInputCurrentLevel` `of: prtInputMaxCapacity`, label
`prtInputName`). Brother and Ricoh quirks go in small vendor profiles extending `printer` (verified against the LibreNMS
recordings where they exist).
## API (additive)
- `GET /api/v1/devices/{slug}/metrics`
```
{ categories: [ { name: "health", label: "Health",
groups: [ { metric: "hrProcessorLoad", label: "CPU load", unit: "percent", display: "gauge",
components: [ { component: "CPU 0", value: 12.0, max: null, text: null, state: null, ts: "…" } ] } ] } ] }
```
Latest values from hot (one batched query per device). `max` is set for `of` metrics, `state` is `ok|warn|crit|unknown`
for enums (mapping in the profile `enum_state` or a default by name), and `text` is used for sentinels and text displays.
Categories appear only when the device has data in them, in the order system, health, power, environment, wireless, printer, routing, optics, firewall.
- `GET /api/v1/devices/{slug}/metrics/graph?metric=&component=&from=&to=&step=` returns `{unit, display, series: [[unix_s, value|null]], max_series?}`
for gauges (raw), with counters as a rate, using the same step rules as `/graph`. Percent displays return percent.
- `GET /api/v1/printers?page=` (viewer): the fleet view, with
`items: [{device{slug,name,site}, reachability, pages_total, supplies: [{component, type, color, percent|null, text|null, state}], trays: [...]}]`,
sorted by the lowest supply percent first.
- Alerts: add `printer_supply_empty` (critical, level 0 or "empty"). Record a "supply replaced" audit event when a supply's
percent rises by 20 points or more between polls.
## UI
Device page tabs: **Overview** (header, small CPU/memory/storage summary, active alerts) · **Ports** (today's table) ·
then one tab per category the device has (Health, Power, Environment, Wireless, Printer, Routing, Optics, Firewall).
- Generic renderers, chosen by `display`:
- gauge: value, unit and a sparkline
- percent: bar with value/max
- state: coloured badge, also shown by icon or shape so it doesn't rely on colour alone
- rate: value per second
- text: plain text
Clicking a row opens the metric graph, a TrafficGraph-like component for single series. Components are grouped by metric.
- Bespoke panels:
- **Printer**: a card per supply with a colour swatch (C/M/Y/K from colorant), a percent bar and "some remaining" text,
paper trays, and a page count graph.
- **Wireless**: per-SSID table (clients, channel, tx power) with totals, and per-radio utilisation bars.
- New nav page **Printers** (fleet), sorted by the lowest supply.
- Viewers see everything; nothing here mutates.
## Guards
- **Orphan test:** every pollable metric in every shipped profile has a category, and its unit and display have a renderer.
- **Coverage matrix:** `docs/coverage.md` maps the spec's "What to collect" table to what each profile collects today, with gaps listed plainly.

View File

@@ -0,0 +1,113 @@
# v1.1: alert policies (alerts fire only where a policy says so)
Owner request (2026-10-23): preprod alerts on everything. Rules exist, but each one applies to every device and port,
so a WiFi port that always sees errors and a fiber uplink, where one error matters, are judged by the same rule.
The owner's decisions:
- **Nothing fires without a policy.** This covers metric rules, reachability (`device_down`, `device_warning`) and
nmon's own system alarms. A fresh install records no alerts until an admin adds policies.
- **Scope is groups + matchers.** Matchers pick ports and devices by attribute (so new ports are covered
automatically), and groups handle the hand-curated exceptions.
- **Policies live in the database** (a new bun migration) and are edited through the API and UI. They apply live, with no
restart. Rule *definitions* (the MetricsQL, templates) stay in config as today.
## Model
A **policy** binds one rule to a scope:
| Field | Meaning |
|---|---|
| `id` | ULID |
| `name` | unique, human ("Fiber uplinks: errors") |
| `rule` | a metric rule name (`interface_errors`, …), `device_down`, `device_warning`, or `system` (all subsystem alarms, `alert.SystemRule`) |
| `action` | `alert` or `ignore`. An `ignore` policy that matches wins over any `alert` policy of the same priority or lower. |
| `priority` | int, default 0. For one entity and rule, the matching policy with the highest priority decides; on a tie `ignore` beats `alert`, then the lowest name wins. |
| `enabled` | bool |
| `severity`, `threshold`, `for` | optional overrides (null = the rule's). Threshold resolution: policy > the rule's group override > profile `ThresholdFor` > rule default. |
| `scope` | see below; must be non-empty (`all: true` is the explicit "everything") |
**Scope** fields are ANDed together; within a list, any entry may match. An empty field is unconstrained.
- `all: bool`
- `groups: [group id]`. A port matches if it is a member now (`GroupMember.Covers`); a device matches if any of its ports is.
- `devices`: `slugs` (globs), `profiles`, `roles`, `sites`, `kinds`
- `interfaces`: `if_types` (ints, e.g. 6 ethernetCsmacd, 71 ieee80211), `if_name` (RE2), `if_alias` (RE2),
`min_speed_mbps`/`max_speed_mbps`, `alert_down` (true = the port's bell or a group's alert_down is on; this keeps the
existing opt-in UI meaningful)
An interface matcher on a device-level rule (`cpu_high`, `device_down`) never matches, and validation refuses it. A
`system` policy accepts only `all: true`, because system alarms have no device.
The examples the owner gave:
```
{name: "Fiber + uplinks: errors", rule: interface_errors, action: alert, threshold: 0,
scope: {interfaces: {if_types: [6], if_alias: "(?i)fib|uplink|core"}}}
{name: "WiFi: no error alerts", rule: interface_errors, action: ignore, priority: 10,
scope: {interfaces: {if_types: [71]}}}
```
## Packages
- `internal/alertpolicy` (new, pure: no store, no SQL): `Policy`, `Scope`, `Validate`, `Subject` (device attrs, iface
attrs, group ids) and `Decide(policies, rule, Subject) (Binding, bool)`. Unit tests carry the precedence table.
- `internal/store`: migration `20261023000012_alert_policies.go` (table `alert_policies`, scope as JSON text, frozen
struct copy), plus a sub-interface `AlertPolicies` in `alert_policies.go` (List/Get/Create/Update/Delete; unique name ->
ErrConflict) and one embed line in `store.go`.
- `internal/alert`: `Options.Policy func(Entity) (alertpolicy.Binding, bool)`. A series without a binding is skipped,
and its open row closes silently as `retired`, like the existing opt-in. The binding's threshold, severity and for
apply. `Rule.OptIn` and `Options.OptedIn` are superseded by the `alert_down` matcher. Keep the yaml field so old
configs still parse, but make it a no-op and say so in the doc comment.
- `internal/reach`: `Options.Wants func(rule, slug string) bool`. No binding means no new row, and an open row closes
quietly.
- `internal/app`: a policy cache, loaded at start, refreshed every 30 s and invalidated on every API write. It builds the
`Subject` from TTL-cached device and interface lookups and gates `sysAlarms` on a `system` policy.
- `internal/api`, in a new file:
- `GET/POST /api/v1/alert-policies` and `GET/PUT/DELETE /api/v1/alert-policies/{id}` (admin).
- `POST /api/v1/alert-policies/preview` (viewer, read-only) returns the counts of devices and ports a draft scope covers,
with up to 20 samples of each.
- `GET /api/v1/alert-rules` (viewer) lists the rule names, their level (`device`, `interface`, `component`, `system`),
default threshold, op and severity, for the UI dropdown.
- All additive; regenerate `openapi.json` and the web schema.
- `web/`: an **Alert policies** page under Alerts (list, create and edit form with a live preview count, enable toggle;
`RequireAdmin` for mutation). It has a "Start from template" menu with UI-side constants: all devices `device_down`,
bell-opted ports `interface_down`, uplinks `interface_errors`, wireless ignore `interface_errors`, `system` all. The
bell tooltip changes to say that a policy using the `alert_down` matcher is needed. Add a mock and an e2e test.
## Upgrade behaviour
No policies are seeded. On the first cycle after upgrade, every open alert without a binding closes silently as
`retired` and sends no "resolved" notifications, which is how the preprod flood clears. The startup log warns once when
the policy table is empty ("no alert policies: nothing will alert").
## Docs
- DEVIATIONS: a new row, "Default rules apply everywhere; spec §Alerting", which this change replaces with policy-gated
rules (owner decision 2026-10-23). It supersedes #38's mechanism but keeps the bell.
- `docs/operations.md`: an alert policies section with the two examples above.
## Gate
- `make lint` is clean, and `go test -race ./...`, `-tags 'integration vmembed'` for `internal/alert`, `internal/reach`,
`internal/app` and `internal/store` all pass, plus `./scripts/test-pg.sh`, `scripts/openapi-check.sh` and the web
lint/test/build/e2e.
- Tests prove the following:
- With no policies, nothing is recorded (metric, reach and system).
- The WiFi ignore and fiber alert pair decides a 71 port and a 6 port correctly.
- Removing a policy retires open rows silently.
- Policy threshold beats group override.
- A policy write takes effect on the next cycle without a restart.
## Gate log
2026-10-23, run by the implementing agent:
- `make lint`: 0 Go issues, filesize script clean, ESLint `--max-warnings 0` clean.
- `go test -race ./...`: all 43 packages ok.
- `go test -race -tags 'integration vmembed' ./internal/alert/... ./internal/reach/... ./internal/app/... ./internal/store/...`: ok.
- `./scripts/test-pg.sh` (store suite on Postgres 16, includes migration 12): ok.
- `scripts/openapi-check.sh`: compatible with m6 (additive only). `openapi.json` and `web/src/api/schema.d.ts` regenerated.
- Web: `npm run lint`, `npm run test` (140 tests) and `npm run build` pass. `E2E_PORT=4791 npm run e2e`: 149 passed, 6 failed. The 6 failures are device-page specs (`device-tabs`, `devices-mgmt` edit, `profiles-nav` pin, `screenshots-v101` overview, `screenshots` 32-bit warning, `ui` warning explanation) that look for the `summary-tile`, `device-header`, `generic-hint` and `device-warnings` test ids, which no longer exist in `web/src` since commit 992f09b (Overview tab). They are not caused by this change and were left alone. The new `alert-policies.spec.ts` (4 tests) and the updated `alert-down.spec.ts` pass. Screenshot dirs were reverted.
- `make gate-m4`: the whole backend run passes (45 PASS, including device_down, suppression, restart, ack and maintenance with the two gate-created policies). It then fails in its UI steps for the same stale test ids as above (mock UI specs, plus the live `device-down` screenshot waiting for `device-header`). The first run of the gate caught a missing `Wants:` wiring in `internal/app/alerting.go` (no device_down was recorded); fixed.
- `gate-m5.sh` and `gate-m6.sh` now create a `system` policy before the alarms they expect; only `bash -n` was run on them (not the full ~10 min gates).
- The 2,000-device load test and `make gate-m5`/`gate-m6` were not run.

View File

@@ -0,0 +1,229 @@
# v1.3: BGP peers and MAC/ARP/FDB lookup
Owner request (2026-10-24), after the LibreNMS comparison: **MAC/ARP/FDB lookup is a must, and BGP peer monitoring is
wanted.** WISP radio profiles are nice-to-have and not in this milestone.
Parts of this go beyond the spec. The spec lists BGP-4-MIB in the collection table but no profile fills it, and it
names no MAC/ARP lookup feature. Add one DEVIATIONS row per feature.
## Part A: MAC / ARP / FDB ("where is this host plugged in?")
### Collection
A new **L2 table job**, separate from the 60 s poll path (the spec forbids table walks on the hot path) and from the
twice-daily discovery:
- It walks every enabled SNMP device every `l2.interval` (default 15 min, minimum 5 min, in `config.yaml`; applied on
SIGHUP like the other job settings, or `restart_required` if that is simpler, and say which).
- It has its own concurrency cap and a per-device deadline, and yields to the poller under the same admission policy as
rollup.
- A device that times out is skipped for that cycle, with a counter and a poller-health note; it never blocks the job.
Tables, all numeric OIDs verified against `mibs/`, each with a provenance comment:
| What | Source | Notes |
|---|---|---|
| Bridge port → ifIndex | BRIDGE-MIB `dot1dBasePortIfIndex` | needed for every FDB row |
| FDB, VLAN-aware | Q-BRIDGE-MIB `dot1qTpFdbPort` (index: fdbId.mac) | Map fdbId to the VLAN with `dot1qVlanFdbId` where present; otherwise fdbId = VLAN id. |
| FDB, fallback | BRIDGE-MIB `dot1dTpFdbPort` | used when Q-BRIDGE returns nothing; the VLAN is unknown (0) |
| Cisco per-VLAN FDB | BRIDGE-MIB per VLAN | IOS only: list VLANs from CISCO-VTP-MIB `vtpVlanState`, then walk with community `<community>@<vlan>` (v2c) or context `vlan-<vlan>` (v3). Turn it on with a profile flag (`l2.per_vlan_context: true` on cisco-ios), not by detection. |
| VLAN names | Q-BRIDGE-MIB `dot1qVlanStaticName`, CISCO-VTP-MIB `vtpVlanName` | shown in the UI |
| ARP and IPv6 ND | IP-MIB `ipNetToPhysicalPhysAddress` (both IPv4 and IPv6, IPv6 first) | This is the primary source. Fall back to `ipNetToMediaPhysAddress` (IPv4 only) when it is empty. Skip entries of type invalid or incomplete. |
Recordings to verify against: `/home/svcuser/src/librenms/tests/snmpsim` (pick ones with these tables for cisco-ios,
junos, arista, edgeswitch, unifi-switch, mikrotik and a Linux bridge). Read them as reference data only; copy no code or
YAML from that GPL repository. Add `.snmprec` fixtures under `testdata/snmprec/` produced by our own generator (extend
`internal/fixturegen`), including one with Q-BRIDGE, one with only BRIDGE-MIB, one Cisco per-VLAN, and an ARP table with
IPv6 and IPv4 entries.
### Storage (inventory DB, not the metric stores)
Use migration `20261024000014_l2_tables.go`. Number 13 is taken by the concurrent v1.2 work; take 14 and nothing else.
- `fdb_entries`: device_id, interface_id (ULID, nullable when the bridge port has no ifIndex), vlan, mac (normalised
lower-case `aa:bb:cc:dd:ee:ff`), first_seen, last_seen.
- Unique on (device_id, interface_id, vlan, mac).
- A row seen again only bumps last_seen.
- Use batched upserts: a 48-port switch can carry thousands of MACs.
- `arp_entries`: device_id, interface_id (nullable), ip (canonical text, v6 compressed), mac, first_seen, last_seen.
Unique on (device_id, ip, mac).
- `vlans`: device_id, vlan, name. Replaced per walk.
- Retention is `l2.keep` (default 90 d) on last_seen, pruned daily. This history is what lets you ask "where was this MAC
last week".
- Add the new tables to `backupTables`, or mark them deliberately transient with a reason. Back them up.
- Only `internal/store` touches SQL. Add a store sub-interface in a new file plus one embed line in `store.go`. Every
query must be indexed: lookups are by mac, by ip, by interface and by device.
### Trunk awareness
A MAC is seen on every switch it passes through. For each FDB row, compute at query time the **MAC count of its port**
(per walk), and mark a port an **uplink** when it has more than `l2.edge_max_macs` MACs (default 8), when it is an LLDP
neighbour link, or when it is a LAG/parent. The lookup ranks **edge ports first**: the port with the fewest MACs, then
the most recent last_seen. That is where the host is plugged in.
### API (additive)
- `GET /api/v1/l2/search?q=` (viewer):
- `q` is a MAC in any common format (`aabb.ccdd.eeff`, `aa-bb-…`, `aabbccddeeff`, partial prefixes of at least 6 hex
digits), an IPv6 or IPv4 address, or a CIDR.
- It returns grouped hits:
`{ mac, ips: [{ip, device, last_seen}], locations: [{device, iface, vlan, vlan_name, edge: bool, port_mac_count, first_seen, last_seen}] }`,
edge locations first, paged like the other routes.
- `GET /api/v1/devices/{slug}/fdb` and `GET /api/v1/devices/{slug}/arp` (viewer): the paged tables, filterable by VLAN,
interface and `current` (seen in the last walk) or with history.
- `GET /api/v1/interfaces/{id}/macs` (viewer): the MACs on a port, current and history, each with its IPs from any ARP
table.
### UI
- A global search box (existing header search, or a new **Find host** page): paste a MAC or IP and get the answer
"device X port Y, VLAN Z, last seen 3 min ago", with the full path (uplinks) collapsed below it.
- On the device page, a new **L2** tab (FDB with a VLAN filter, ARP, VLAN list), shown only when the device has rows.
- On the port, a MAC count on the Ports tab row with a link to that port's MACs.
### OUI vendor lookup (optional; nmon must work fully without it)
Owner request (2026-10-24): show the vendor of each MAC, from the IEEE registry where it can be fetched. It is nice to
have and never required.
- **Source:** the IEEE public registries MA-L (`https://standards-oui.ieee.org/oui/oui.csv`), MA-M (`…/oui28/mam.csv`) and
MA-S (`…/oui36/oui36.csv`). Check the current URLs and CSV columns before coding against them.
- **Fetch:**
- a background job at start (if no database is present) and then every `oui.refresh` (default 7 d);
- through `internal/netpolicy`, so the outbound allow-list applies;
- with timeouts, a size cap, and a parse sanity check (a minimum row count) before replacing the current file.
- `oui.fetch: false` turns fetching off.
- **Offline:** `nmon oui import <dir-or-file>` loads the same CSVs from disk. The Ansible role can optionally download
and ship them next to the MIB sidecar.
- **Storage:** a compact file in the data dir (e.g. `oui/registry.bin`, or a gzipped normalised CSV), written atomically
with a checksum, and included in `nmon backup`. It is not in the inventory DB, since it is reference data. Loaded into
memory as three prefix maps (24-, 28- and 36-bit); a lookup takes the longest match.
- **Missing or stale data never fails anything.** With no file, vendor is empty and a `oui_unavailable` note appears on
poller health. There is no alert.
- **Vendor flag:**
- computed at query time and never stored per FDB or ARP row, so a refresh relabels history;
- API hits and table rows gain `vendor` (string, empty when unknown) and `mac_kind`: `global`, `local` (the
locally-administered bit, typical of randomised phone and VM MACs, with vendor shown as "Private / randomised"),
or `multicast`.
- **Search:** `q` may also be a vendor name substring (`vendor:cisco`). It returns the current MACs of that vendor,
paged, so "find every Ubiquiti device on VLAN 20" works.
- **UI:** a vendor column on the L2 tab, the port MAC list and the search results, plus a small "private" badge for
`local`.
- **Licensing:** the IEEE registry is public data. Note its source in `docs/operations.md`, and do not commit the CSVs
to the repo. Tests use a tiny generated fixture registry.
- **Tests:**
- the longest prefix wins (an MA-S entry beats MA-L);
- the local and multicast bits are detected;
- a truncated or garbage download keeps the previous file;
- no network and no file still lets the L2 job and search work, with an empty vendor.
## Part B: BGP peers
### Collection (profiles, 5 min interval; metrics in the `routing` category)
The component label is the peer address (canonical, IPv6 compressed). Peer AS, local AS, description/remote router ID
and AFI/SAFI are **component attributes** (migration 11's mechanism) collected at discovery, never metric labels. The
SNMP label contract stays `device` + `component` only. Where a vendor reports prefixes per address family, the
component is `<peer>|<afi>` for the prefix metrics only; document it.
| Source | Gives | Profiles |
|---|---|---|
| BGP4-MIB `bgpPeerState`, `bgpPeerFsmEstablishedTime`, `bgpPeerInUpdates`/`OutUpdates`, `bgpPeerRemoteAs` | IPv4 peers only | generic (optional, enabled on router profiles) |
| CISCO-BGP4-MIB `cbgpPeer2State`, `cbgpPeer2FsmEstablishedTime`, `cbgpPeer2AcceptedPrefixes` (per AFI/SAFI), `cbgpPeer2RemoteAs` | IPv4 and IPv6 | cisco-ios, cisco-nxos |
| BGP4-V2-MIB-JUNIPER `jnxBgpM2PeerState`, `jnxBgpM2PeerFsmEstablishedTime`, `jnxBgpM2PrefixInPrefixesAccepted` | IPv4 and IPv6 | juniper-junos |
| ARISTA-BGP4V2-MIB `aristaBgp4V2PeerState`, `…PrefixInPrefixesAccepted` | IPv4 and IPv6 | arista-eos |
- `state` is a `status` metric with enum `idle connect active opensent openconfirm established`;
`enum_state: established ok, others crit`.
- Use the vendor table where one exists, and drop the BGP4-MIB duplicate on that profile, so each peer appears once.
- VyOS/FRR, MikroTik, FortiGate and EdgeOS: check `mibs/` and the recordings (`cumulus_bgp`, `dnos_bgp`, vyos and
mikrotik ones). Add BGP only where the device actually answers. State per profile in `docs/coverage.md` what is
collected, and where nothing is, say so plainly.
- Verify against the LibreNMS recordings (`arista_eos*.snmprec`, `cumulus_bgp`, `dnos_bgp`, ios and junos ones) and add
our own generated fixtures with IPv6 peers.
### UI
- Device **Routing** tab:
- a peers table with peer, AS, description, state badge, uptime, prefixes accepted per AFI, and update rate;
- a click-to-graph on the prefixes and updates;
- built from the existing metrics-tab renderers where they fit.
- A **BGP** page in the main nav listing every peer on every device, with filters for state, AS and device, so you can
see which sessions are down right now.
- Back it with `GET /api/v1/bgp/peers` (viewer, paged; latest values from hot, with attributes from the inventory).
### Alerts
- New built-in rule `bgp_peer_down`:
- component-level;
- condition: state is not established for `for: 2m`;
- severity critical;
- its summary names the peer, AS and description.
- Like every rule, it fires only where a policy binds it (v1.1). Add a template "Routers: BGP peer down" to the policy
templates.
- A prefix-drop alert is left to the custom rule builder (v1.2). Add it to the operations doc as an example once v1.2 is
in.
## Constraints
- **v1.2 is being built concurrently.** It owns `internal/alert` rules plumbing, `internal/alertrule`, migration 13, and
the alert-rules API and UI.
- Do not edit those files, except to append `bgp_peer_down` to `DefaultRules()` in `internal/alert/rules.go` plus its
integration test row, and to add the policy template constant in `web/src/lib/policyModel.ts`.
- Rebase onto whatever v1.2 commits first, if it lands before you.
- Regenerate `openapi.json` and `web/src/api/schema.d.ts` just before committing.
- Every port is still polled the same way; this adds a job and does not change the poller.
- IPv6 first: ND and IPv6 BGP peers are first-class, and tests default to `::1`.
## Gate
- `make lint` clean. All of these pass:
- `go test -race ./...`
- `go test -race -tags 'integration vmembed'` for the touched packages
- `./scripts/test-pg.sh` and `scripts/openapi-check.sh`
- `NMON_LIBRENMS_DIR=/home/svcuser/src/librenms go test ./internal/profile/... ./internal/fixtures/...`
- `go run ./testdata/gen` with no drift
- web lint, test and build, plus only the new and changed Playwright specs
- Tests prove the following:
- MAC normalisation accepts every format above and refuses garbage.
- A MAC seen on an access port and on two uplinks resolves to the access port first.
- The Cisco per-VLAN context walk is used only when the profile flag is set.
- IPv6 ND entries are stored and searchable.
- Retention prunes by last_seen.
- An IPv6 BGP peer from a vendor table shows once (no BGP4-MIB duplicate).
- `bgp_peer_down` fires through a policy and resolves on established.
- The L2 job never runs on the poll path, and a slow device does not delay the others.
## Gate log
Built on branch `worktree-agent-a86e97dea1f9eae40` (from main at `06a3c85`), 2026-10-24. Run by the implementing agent; the
supervisor reruns the gates after merging with v1.2.
### Verification
| Check | Result |
|---|---|
| `make lint` (golangci-lint v2.14.0, `scripts/lint-filesize.sh`, ESLint `--max-warnings 0`) | 0 issues |
| `go test -race ./...` | all packages pass |
| `go test -race -tags 'integration vmembed'` on alert, api, app, backup, l2, discovery, fixtures, profile, vmsup | pass (the alert integration test runs `bgp_peer_down` against a real VictoriaMetrics: fires for the active IPv6 peer, not for the established one) |
| `./scripts/test-pg.sh` (Postgres 16, store suite incl. the L2 queries, upserts and backup/restore table list) | pass |
| `scripts/openapi-check.sh` | compatible with m6 (additive only) |
| `NMON_LIBRENMS_DIR=/home/svcuser/src/librenms go test ./internal/profile/... ./internal/fixtures/... ./internal/l2` | pass: real recordings produce each BGP peer exactly once (IOS XE and XR with IPv6 peers, Junos MX, Arista, EdgeOS, FortiGate, FRR on pfSense), no type mismatches; the L2 walker reads FDB and ARP/ND (IPv6 from `arista_eos_7280r` and `dell-os10`) from the Arista, Junos, IOS, UniFi/EdgeSwitch and RouterOS recordings |
| `go run ./testdata/gen` | no drift (`TestCommittedFixturesAreCurrent` passes); 9 expect files gained only BGP series, PDU budgets unchanged |
| `cd web && npm run lint && npm run test && npm run build` | clean; 165 tests pass |
| Playwright, only the new `e2e/specs/l2-bgp.spec.ts` (12 tests), plus `device-tabs` and `profiles-nav` as a regression check of the pages this touches | 27 passed; `docs/screenshots` reverted; no preview server left running |
Not run: the `make gate-m*` scripts, the Ansible tasks (no Ansible here; the YAML parses), and the full Playwright suite.
### What proves each gate line
| Line | Test |
|---|---|
| MAC normalisation accepts every format and refuses garbage | `internal/hostaddr` `TestParseMACAcceptsEveryCommonFormat`, `TestParseMACRefusesGarbage`, `TestClassifySortsEveryQueryShape`; API `TestSearchAcceptsEveryMACFormatAndPrefixAndRefusesGarbage` |
| A MAC on an access port and two uplinks resolves to the access port first | `internal/api` `TestAMACSeenOnAnAccessPortAndTwoUplinksResolvesToTheAccessPortFirst` (uplinks by MAC count and by LLDP), `TestEdgeLimitComesFromTheLiveSetting` |
| The Cisco per-VLAN walk is used only with the profile flag | `internal/l2` `TestCiscoPerVLANContextIsUsedOnlyWhenTheProfileFlagIsSet` (dial log: no `@vlan` community without the flag); `internal/profile` `TestOnlyIOSWalksBridgeTablesPerVLAN` |
| IPv6 ND entries are stored and searchable | `internal/l2` `TestIPv6NeighboursAreStoredAndSearchable`; store `TestL2SearchFindsMACsByPrefixIPv6AndCIDR`; API `TestIPv6AndIPv4AndCIDRSearchFindTheMAC` |
| Retention prunes by last_seen | store `TestPruneL2DeletesByLastSeen`; l2 `TestRetentionPrunesByLastSeenOnTheDailyPass` |
| An IPv6 BGP peer from a vendor table shows once | `internal/profile` `TestVendorBGPTablesReplaceTheBGP4MIBOnesSoAPeerAppearsOnce`; fixtures (every vendor fixture lists `2001:db8::2` once); the real-recording `checkPeers` |
| `bgp_peer_down` fires through a policy and resolves on established | `internal/alert` `TestBGPPeerDownFiresThroughAPolicyAndResolvesOnEstablished`, `TestBGPPeerDownFiresNowhereWithoutAPolicy`; integration row |
| The L2 job never runs on the poll path; a slow device does not delay the others | `internal/l2` (every dial is path `l2`; a poll session refuses walks, `internal/snmp`), `TestSlowDeviceTimesOutWithoutDelayingTheOthers`, `TestJobYieldsWhileTheWriteQueueIsMoreThanHalfFull` |
| OUI: longest prefix wins; local and multicast bits; truncated or garbage download keeps the previous file; no network and no file still works | `internal/oui` `TestLongestPrefixWins`, `TestVendorFlagsLocalAndMulticast`, `TestTruncatedOrGarbageDownloadKeepsThePreviousFile`, `TestMissingRegistryNeverBreaksALookup`; API `TestVendorSearchListsCurrentMACsAndWorksEmptyWithoutARegistry` |
### Decisions where the spec was silent
- **L2 settings and reload.** `l2.*` and `oui.*` apply from the next cycle after a reload (SIGHUP or a Settings save) with no restart, and are reported as `applied`. The job re-reads `l2.interval` every minute while waiting, so a shorter interval takes effect within a minute and a longer one on the wait in progress. Test: `TestRunStartsACycleAtOnceThenEveryIntervalAndReadsTheIntervalAgain`.
- **Ranking.** Edge before uplink, then rows the last walk still sees before history, then fewest MACs, then most recent (DEVIATIONS 55).
- **Per-VLAN fallback.** If the first VLAN context does not answer, the walker falls back to the plain tables instead of waiting out a timeout per VLAN.
- **`bgp_peer_down` text.** AS and description are added to the notification, not the stored alert row (DEVIATIONS 53).
- **BGP per profile.** See `docs/coverage.md`. Nothing for mikrotik-routeros (no BGP objects in MIKROTIK-MIB or the recordings) and the non-router profiles; cisco-nxos is unverified against a recording; VyOS is covered through BGP4-MIB on the strength of the FRR recording only.
- **Open item for v1.2.** The prefix-drop example in `docs/operations.md` is to be added once custom rules land.

View File

@@ -59,6 +59,27 @@ Edit in Settings (admin) or `config.yaml` + `SIGHUP`. An invalid document is rej
The supervisor restarts a crashed store; after three restarts in ten minutes it raises `store_failed` and stops. `nmon store restart [hot|long] --api URL --token T` (or `SIGHUP`) resumes it.
## Interface-down alerts are opt-in
Since v1.1 the bell and the group switch only **mark** ports: `interface_down` alerts for them only where an alert policy for `interface_down` uses the `alert_down` matcher (template "Bell-marked ports: interface_down"). Mark ports in one at a time (the bell on the device page's interface table), all at once (the "Alert on down" control in the interfaces card header, or `POST /api/v1/devices/{slug}/interfaces/alert-down`), or through a group (the group's "Alert when member ports go down" switch; `PUT /api/v1/interfaces/{id}` with `{"alert_down": true}` for one port). A port in a group with the switch on alerts for as long as its membership is in force. Changes reach the evaluator at once; opting a port out closes its open alert silently as `retired`.
## Alert policies: nothing alerts until you add one
Alerts fire only where an **alert policy** binds a rule to a scope. A fresh install has no policies and records no alerts at all: no metric rule, no `device_down` or `device_warning`, no system alarm (ingest caps, late p95, rollup, disk guard, break-glass, backup). The startup log says so once ("no alert policies: nothing will alert"). Upgrading from v1.0 seeds nothing: the first cycle closes every open alert that no policy binds, silently, as `retired` (no "resolved" notices go out).
Manage policies under Alerts > Alert policies (admins edit, viewers read) or with `GET/POST /api/v1/alert-policies` and `GET/PUT/DELETE /api/v1/alert-policies/{id}`. `GET /api/v1/alert-rules` lists the rule names. `POST /api/v1/alert-policies/preview` counts the devices and ports a draft scope covers. Changes apply on the next cycle with no restart. Removing or disabling a policy retires its open alerts quietly.
A policy has a rule (a metric rule, `device_down`, `device_warning`, or `system`), an action (`alert` or `ignore`), a priority, optional `severity`, `threshold` and `for` overrides, and a scope. Scope fields are ANDed and any entry of a list may match: `all`, `groups` (a port is in force as a member now; a device matches if any port does), `devices` (`slugs` globs, `profiles`, `roles`, `sites`, `kinds`) and `interfaces` (`if_types`, `if_name` and `if_alias` as RE2, `min_speed_mbps`, `max_speed_mbps`, `alert_down`). For one entity and rule the matching policy with the highest priority decides; on a tie `ignore` beats `alert`, then the lowest name. A policy threshold beats a rule's group override, the profile threshold and the default. Interface matchers are refused on device-level rules, and a `system` policy takes only `all: true`.
Fiber and uplinks alert on any error, WiFi never alerts on errors:
```
{"name": "Fiber + uplinks: errors", "rule": "interface_errors", "action": "alert", "threshold": 0,
"scope": {"interfaces": {"if_types": [6], "if_alias": "(?i)fib|uplink|core"}}}
{"name": "WiFi: no error alerts", "rule": "interface_errors", "action": "ignore", "priority": 10,
"scope": {"interfaces": {"if_types": [71]}}}
```
## Retired devices
`POST /api/v1/devices/{slug}/purge` (admin) deletes a retired device's series from both stores once it has been retired for `retired_purge_after` (default 1y).
@@ -66,3 +87,35 @@ The supervisor restarts a crashed store; after three restarts in ten minutes it
## CSV import
Devices (`name,address,site,role,credential,parents,...`) and port descriptions (`device,ifname,alias`) from Settings or `POST /api/v1/devices/import`. Port descriptions are stored in the operator-owned `description` field, which discovery never overwrites. A dependency cycle is rejected with its path; nothing is written unless the whole import is valid.
## Find host: MAC, ARP and FDB lookup
The **L2 table job** walks the bridge forwarding tables, VLAN names and ARP / IPv6 neighbour tables of every SNMP device every `l2.interval` and keeps them with first and last seen in the inventory, so the **Find host** page (and the header search, for a MAC, IP, CIDR or `vendor:name`) answers "which switch port is this host plugged in at, in which VLAN, and when did we last see it", with the uplinks that also saw it collapsed underneath. A MAC is seen on every switch it passes through, so the answer is the **edge** port: one that carries at most `l2.edge_max_macs` MACs, has no LLDP neighbour and is not a LAG.
```yaml
l2:
interval: 15m # default 15m, minimum 5m
keep: 90d # history kept by last_seen, pruned daily
edge_max_macs: 8 # a port with more MACs is an uplink
concurrency: 4 # devices walked at once
device_timeout: 2m # a slower device is skipped for the cycle
oui:
fetch: true # IEEE vendor registry; false never touches the network
refresh: 7d
```
All of these apply from the next cycle after a reload (Settings save or SIGHUP) with no restart. The job never runs on the 60 s poll path and yields to the poller the way the rollup job does. A device that does not answer in time is skipped for that cycle and named on the Poller page (`l2` status and an `l2_devices_skipped` note); it never delays the other devices. A Cisco IOS switch keeps one bridge table per VLAN: the `cisco-ios` profile sets `l2.per_vlan_context: true`, which lists the VLANs from CISCO-VTP-MIB and walks each with community `<community>@<vlan>` (v3: context `vlan-<vlan>`). Other devices use Q-BRIDGE-MIB, then BRIDGE-MIB (VLAN unknown).
API: `GET /api/v1/l2/search?q=`, `GET /api/v1/devices/{slug}/{l2,fdb,arp,l2/ports}`, `GET /api/v1/interfaces/{id}/macs`. On a device, the **L2** tab (MAC table with a VLAN filter, ARP / ND, VLAN list) appears once the job has read it, and the Ports tab shows each port's MAC count linking to that port's MACs.
## MAC vendors (optional IEEE registry)
If a registry is loaded, each MAC shows its vendor, a locally administered address (typical of randomised phone and VM MACs) shows a "private" badge, and `vendor:ubiquiti` lists the current MACs of a vendor. nmon works fully without it: the vendor is simply empty, and the Poller page notes `oui_unavailable` (not an alert).
The registry is the IEEE's public data: MA-L (`https://standards-oui.ieee.org/oui/oui.csv`), MA-M (`.../oui28/mam.csv`) and MA-S (`.../oui36/oui36.csv`), downloaded weekly through the outbound allow-list (`standards-oui.ieee.org` must be allowed when an allow-list is configured) with timeouts, a size cap and a row-count sanity check, so a truncated or wrong download keeps the previous file. The compact result is `<data-dir>/oui/registry.gz` (checksummed, written atomically, included in `nmon backup`). Where nmon may not fetch, set `oui.fetch: false` and load the CSVs by hand: `nmon oui import <dir-or-file> --data-dir DIR` (a running nmon notices within an hour). The CSVs are not part of this repository.
## BGP
Router profiles collect BGP session state, uptime, accepted prefixes per address family and update rates every 5 minutes (see `docs/coverage.md` for which profiles and MIBs). The **BGP** page lists every session on every device, the ones that are down first, filterable by state, AS, family and text; a device's **Routing** tab shows its sessions with click-to-graph on prefixes and updates.
The built-in rule `bgp_peer_down` (critical, state not established for 2 minutes) fires only where a policy binds it, like every rule: use the template "Routers: BGP peer down" under Alerts > Alert policies. Its notification names the peer and, from discovery, the peer's AS and description. A prefix-drop alert is a custom rule (v1.2); an example will be added here once custom rules are in.

Binary file not shown.

After

Width:  |  Height:  |  Size: 131 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 150 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 150 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 166 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 136 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 157 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 128 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 68 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 204 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 90 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 57 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 159 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 86 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 44 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 83 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 65 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 63 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 89 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 89 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 57 KiB

View File

@@ -0,0 +1,62 @@
package alert
import (
"strings"
"testing"
"time"
)
func bgpRule(t *testing.T) Rule {
t.Helper()
for _, r := range DefaultRules() {
if r.Name == "bgp_peer_down" {
return r
}
}
t.Fatal("no bgp_peer_down in DefaultRules")
return Rule{}
}
func TestBGPPeerDownIsAComponentLevelCriticalRuleHeldForTwoMinutes(t *testing.T) {
r := bgpRule(t)
if r.Severity != SevCritical || r.For != 2*time.Minute || r.Level() != "component" {
t.Errorf("rule = %+v level %s", r, r.Level())
}
if !strings.Contains(r.Expr, "bgp_peer_state") || !strings.Contains(r.Expr, "6") {
t.Errorf("expr = %q: the session is up only in state 6 (established)", r.Expr)
}
if err := r.Validate(); err != nil {
t.Fatal(err)
}
}
func TestBGPPeerDownFiresThroughAPolicyAndResolvesOnEstablished(t *testing.T) {
r := bgpRule(t)
g := newGate(nil, allPolicy("routers", "bgp_peer_down"))
h := gateHarness(t, g, r)
// The expression is 1 for a peer whose state is not 6: an IPv6 peer in "active" and a healthy IPv4 peer.
h.q.set(r.Expr, "rtr1|2001:db8::2|1", "rtr1|192.0.2.1|0")
h.step(30 * time.Second)
h.want("", "pending")
h.step(2 * time.Minute)
h.want("firing", "firing")
a := h.only()
if a.Component != "2001:db8::2" || a.Severity != SevCritical || !strings.Contains(a.Summary, "2001:db8::2") {
t.Fatalf("alert = %+v: only the down IPv6 peer may fire, and its summary names it", a)
}
h.q.set(r.Expr, "rtr1|2001:db8::2|0", "rtr1|192.0.2.1|0") // established again
h.step(30 * time.Second)
h.want("firing,resolved", "resolved")
}
func TestBGPPeerDownFiresNowhereWithoutAPolicy(t *testing.T) {
r := bgpRule(t)
h := gateHarness(t, newGate(nil), r)
h.q.set(r.Expr, "rtr1|192.0.2.1|1")
h.step(30 * time.Second)
h.step(5 * time.Minute)
h.want("", "")
if n := len(h.alerts()); n != 0 {
t.Fatalf("%d alert rows with no policy", n)
}
}

View File

@@ -9,6 +9,7 @@ import (
"sync"
"time"
"github.com/v10networks/nmon/internal/alertpolicy"
"github.com/v10networks/nmon/internal/clock"
"github.com/v10networks/nmon/internal/selfmon"
"github.com/v10networks/nmon/internal/store"
@@ -89,6 +90,10 @@ type Options struct {
// Unreliable reports an interface whose counters cannot be trusted
// (32-bit counters on fast links); rules with SkipUnreliable ignore it.
Unreliable func(iface string) bool
// Policy is the alert policy gate (required). It decides, for one entity of a rule, whether any alert policy
// binds the rule there; no binding means the series is skipped and its open row closes silently as retired.
// The binding's severity, threshold and for override the rule's, and its threshold beats group overrides.
Policy func(Entity) (alertpolicy.Binding, bool)
// Active reports whether an entity still exists (device and interface not
// retired). Inactive entities are closed as retired, silently.
Active func(Entity) bool
@@ -116,8 +121,8 @@ type Evaluator struct {
// New validates the rules and returns an Evaluator.
func New(o Options) (*Evaluator, error) {
if o.Store == nil || o.Querier == nil || o.Notifier == nil || o.Clock == nil {
return nil, errors.New("alert: Store, Querier, Notifier and Clock are required")
if o.Store == nil || o.Querier == nil || o.Notifier == nil || o.Clock == nil || o.Policy == nil {
return nil, errors.New("alert: Store, Querier, Notifier, Clock and Policy are required")
}
if o.Interval <= 0 {
o.Interval = DefaultInterval
@@ -168,6 +173,7 @@ func (ev *Evaluator) Run(ctx context.Context) error {
type reading struct {
ent Entity
value float64
bind alertpolicy.Binding
}
// EvalOnce runs one evaluation cycle at the clock's current time.
@@ -205,7 +211,7 @@ func (ev *Evaluator) sweepRows(ctx context.Context, now time.Time, rows []store.
ev.deliver(ctx, now, a, ent) // closed row with an undelivered notification
continue
}
if ruleByName[a.Rule] == nil || (ev.o.Active != nil && !ev.o.Active(ent)) {
if ruleByName[a.Rule] == nil || (ev.o.Active != nil && !ev.o.Active(ent)) || !ev.bound(ent) {
ev.retire(ctx, now, a)
continue
}
@@ -235,8 +241,8 @@ func (ev *Evaluator) evalRule(ctx context.Context, now time.Time, r *compiledRul
for _, rd := range readings {
key := rd.ent.Key()
seen[key] = true
thr := r.resolveThreshold(rd.ent.Device, ev.o.Groups, ev.o.ThresholdFor)
ev.apply(ctx, now, r, rd, thr, byKey[key])
rr, thr := r.bound(rd.bind, rd.ent.Device, ev.o.Groups, ev.o.ThresholdFor)
ev.apply(ctx, now, rr, rd, thr, byKey[key])
}
for _, a := range rows {
if !seen[a.SeriesKey] {
@@ -266,3 +272,11 @@ func (ev *Evaluator) SortedRuleNames() []string {
sort.Strings(n)
return n
}
// bound reports whether an alert policy binds the entity's rule there. A policy removed, disabled or
// narrowed away makes the entity unbound, and its open alert closes as retired, silently: switching an alert
// off is not a recovery.
func (ev *Evaluator) bound(ent Entity) bool {
_, ok := ev.o.Policy(ent)
return ok
}

View File

@@ -37,6 +37,9 @@ func (q *fakeQ) Query(_ context.Context, _ string, expr string, _ time.Time) ([]
if p[1] != "" {
m["component"] = p[1]
}
if len(p) > 3 {
m["iface"] = p[3]
}
res = append(res, map[string]any{"metric": m, "value": []any{1, p[2]}})
}
return json.Marshal(map[string]any{"status": "success", "data": map[string]any{"resultType": "vector", "result": res}})
@@ -112,7 +115,7 @@ func newHarness(t *testing.T, rules ...Rule) *harness {
t.Fatal(err)
}
h := &harness{t: t, st: st, clk: clk, q: &fakeQ{series: map[string][]string{}, err: map[string]error{}}, n: &fakeN{}, reg: selfmon.New()}
h.opts = Options{Store: st, Querier: h.q, Notifier: h.n, Clock: clk, Rules: rules, Metrics: h.reg}
h.opts = Options{Store: st, Querier: h.q, Notifier: h.n, Clock: clk, Rules: rules, Metrics: h.reg, Policy: allowAll}
h.restart()
return h
}
@@ -616,7 +619,7 @@ func TestDefaultRulesValid(t *testing.T) {
t.Errorf("%s summary %q", r.Name, out)
}
}
for _, n := range []string{"interface_down", "interface_errors", "cpu_high", "memory_high", "disk_full", "ups_on_battery", "ups_runtime_low", "printer_supply_low", "temp_high"} {
for _, n := range []string{"interface_down", "interface_errors", "cpu_high", "memory_high", "disk_full", "ups_on_battery", "ups_runtime_low", "printer_supply_low", "temp_high", "bgp_peer_down"} {
if !seen[n] {
t.Errorf("missing default %s", n)
}

View File

@@ -69,7 +69,11 @@ func TestIntegrationDefaultRulesAgainstRealStore(t *testing.T) {
add("prtMarkerSuppliesMaxCapacity", 100, "device", "prn1", "component", "Black Toner")
add("prtMarkerSuppliesLevel", -2, "device", "prn1", "component", "Cyan Toner")
add("prtMarkerSuppliesMaxCapacity", 100, "device", "prn1", "component", "Cyan Toner")
add("prtMarkerSuppliesLevel", 0, "device", "prn1", "component", "Magenta Toner")
add("prtMarkerSuppliesMaxCapacity", 100, "device", "prn1", "component", "Magenta Toner")
add("mtxrHlTemperature", 80, "device", "rb1")
add("bgp_peer_state", 3, "device", "rtr1", "component", "2001:db8::2") // active
add("bgp_peer_state", 6, "device", "rtr1", "component", "192.0.2.1") // established
add("ifInErrors", float64(i)*60, "device", "sw1", "iface", "if3")
add("ifOutErrors", 0, "device", "sw1", "iface", "if3")
add("ifHCInOctets", float64(i)*60*150000, "device", "sw1", "iface", "if3") // 2500 pkt/s at 1500 B... errors 1/s
@@ -101,7 +105,7 @@ func TestIntegrationDefaultRulesAgainstRealStore(t *testing.T) {
}
n := &fakeN{}
reg := selfmon.New()
ev, err := New(Options{Store: st, Querier: vmsup.NewQuerier(sup), Notifier: n, Clock: clk, Rules: rules, Metrics: reg})
ev, err := New(Options{Store: st, Querier: vmsup.NewQuerier(sup), Notifier: n, Clock: clk, Rules: rules, Metrics: reg, Policy: allowAll})
if err != nil {
t.Fatal(err)
}
@@ -109,7 +113,7 @@ func TestIntegrationDefaultRulesAgainstRealStore(t *testing.T) {
deadline := time.Now().Add(40 * time.Second)
for {
ev.EvalOnce(ctx)
if len(n.got) >= 10 || time.Now().After(deadline) {
if len(n.got) >= 12 || time.Now().After(deadline) {
break
}
time.Sleep(2 * time.Second)
@@ -135,14 +139,16 @@ func TestIntegrationDefaultRulesAgainstRealStore(t *testing.T) {
"ups_on_battery/ups1//",
"ups_runtime_low/ups1//",
"printer_supply_low/prn1//Black Toner",
"printer_supply_empty/prn1//Magenta Toner",
"temp_high/rb1//mtxrHlTemperature",
"bgp_peer_down/rtr1//2001:db8::2",
"interface_errors/sw1/if3/",
} {
if !fired[want] {
t.Errorf("did not fire: %s (fired: %v)", want, fired)
}
}
for _, not := range []string{"disk_full/disk1///", "disk_full/mem1//Physical memory", "interface_down/sw1/if2/", "printer_supply_low/prn1//Cyan Toner"} {
for _, not := range []string{"disk_full/disk1///", "disk_full/mem1//Physical memory", "interface_down/sw1/if2/", "printer_supply_low/prn1//Cyan Toner", "printer_supply_empty/prn1//Black Toner", "printer_supply_empty/prn1//Cyan Toner", "bgp_peer_down/rtr1//192.0.2.1"} {
if fired[not] {
t.Errorf("fired but should not: %s", not)
}

View File

@@ -0,0 +1,146 @@
package alert
import (
"sync"
"testing"
"time"
"github.com/v10networks/nmon/internal/alertpolicy"
"github.com/v10networks/nmon/internal/store"
)
// allowAll is the gate of tests that are not about policies: every entity is bound by a plain alert policy.
func allowAll(Entity) (alertpolicy.Binding, bool) { return alertpolicy.Binding{PolicyID: "all"}, true }
// policyGate stands in for the app's policy cache: a mutable policy set and a port directory.
type policyGate struct {
mu sync.Mutex
set *alertpolicy.Set
ifaces map[string]alertpolicy.Iface
}
func newGate(ifaces map[string]alertpolicy.Iface, ps ...alertpolicy.Policy) *policyGate {
g := &policyGate{ifaces: ifaces}
g.put(ps...)
return g
}
func (g *policyGate) put(ps ...alertpolicy.Policy) {
g.mu.Lock()
defer g.mu.Unlock()
g.set, _ = alertpolicy.NewSet(ps)
}
func (g *policyGate) decide(e Entity) (alertpolicy.Binding, bool) {
g.mu.Lock()
defer g.mu.Unlock()
subj := alertpolicy.Subject{Device: alertpolicy.Device{Slug: e.Device}}
if i, ok := g.ifaces[e.Iface]; ok {
subj.Iface = &i
}
return g.set.Decide(e.Rule, subj)
}
func allPolicy(name, rule string) alertpolicy.Policy {
return alertpolicy.Policy{ID: name, Name: name, Rule: rule, Action: alertpolicy.ActionAlert, Enabled: true,
Scope: alertpolicy.Scope{All: true}}
}
func errorsRule() Rule {
return Rule{Name: "interface_errors", Severity: SevWarning, Expr: testExpr, Op: OpGreater, Threshold: 0.1,
Labels: []string{"device", "iface"}, Summary: "s", Description: "d"}
}
func gateHarness(t *testing.T, g *policyGate, r Rule) *harness {
h := newHarness(t, r)
h.opts.Policy = g.decide
h.restart()
return h
}
func TestWithoutPoliciesNothingIsRecorded(t *testing.T) {
h := gateHarness(t, newGate(nil), errorsRule())
h.q.set(testExpr, "d1||5|p1")
h.step(30 * time.Second)
h.want("", "")
if n := len(h.alerts()); n != 0 {
t.Fatalf("%d alert rows with no policies", n)
}
}
func TestWifiIgnoreAndFiberAlertDecidePerPortType(t *testing.T) {
zero := 0.0
g := newGate(map[string]alertpolicy.Iface{
"fiber": {Type: 6, Alias: "Core fiber"}, "wifi": {Type: 71, Alias: "uplink radio"}, "lan": {Type: 6, Alias: "customer"},
},
alertpolicy.Policy{ID: "f", Name: "Fiber", Rule: "interface_errors", Action: alertpolicy.ActionAlert, Enabled: true, Threshold: &zero,
Scope: alertpolicy.Scope{Interfaces: alertpolicy.IfaceMatch{IfTypes: []int64{6}, IfAlias: "(?i)fib|uplink|core"}}},
alertpolicy.Policy{ID: "w", Name: "WiFi", Rule: "interface_errors", Action: alertpolicy.ActionIgnore, Priority: 10, Enabled: true,
Scope: alertpolicy.Scope{Interfaces: alertpolicy.IfaceMatch{IfTypes: []int64{71}}}})
h := gateHarness(t, g, errorsRule())
// 0.05 is under the rule's 0.1 but over the fiber policy's 0: only the fiber port, at threshold 0, fires.
h.q.set(testExpr, "d1||0.05|fiber", "d1||0.05|wifi", "d1||0.05|lan")
h.step(30 * time.Second)
h.want("firing", "firing")
if got := h.only(); got.Iface != "fiber" || got.Threshold != 0 {
t.Fatalf("alert = %+v, want fiber at threshold 0", got)
}
}
func TestRemovingAPolicyRetiresOpenRowsSilently(t *testing.T) {
g := newGate(nil, allPolicy("p", "interface_errors"))
h := gateHarness(t, g, errorsRule())
h.q.set(testExpr, "d1||5|p1")
h.step(30 * time.Second)
h.want("firing", "firing")
g.put() // the policy is deleted; no restart
h.step(30 * time.Second)
h.want("firing", "")
if got := h.only().State; got != store.AlertRetired {
t.Fatalf("state %q, want retired", got)
}
}
func TestPolicyThresholdBeatsGroupOverride(t *testing.T) {
r := errorsRule()
r.Overrides = []Override{{Group: "dc1", Threshold: 50}}
thr := 1.0
p := allPolicy("p", "interface_errors")
p.Threshold = &thr
h := gateHarness(t, newGate(nil, p), r)
h.opts.Groups = fakeGroups{"d1": {{"dc1", 10}}}
h.restart()
h.q.set(testExpr, "d1||5|p1") // over the policy's 1, under the group's 50
h.step(30 * time.Second)
h.want("firing", "firing")
if got := h.only().Threshold; got != 1 {
t.Fatalf("threshold %v, want the policy's 1", got)
}
}
func TestPolicySeverityAndForOverrideTheRule(t *testing.T) {
p := allPolicy("p", "interface_errors")
p.Severity = SevCritical
hold := 2 * time.Minute
p.For = &hold
h := gateHarness(t, newGate(nil, p), errorsRule())
h.q.set(testExpr, "d1||5|p1")
h.step(30 * time.Second)
h.want("", "") // pending: the rule's for is 0 but the policy holds 2 minutes
h.step(2 * time.Minute)
h.want("firing", "firing")
if got := h.only().Severity; got != SevCritical {
t.Fatalf("severity %q", got)
}
}
func TestPolicyWriteTakesEffectOnNextCycle(t *testing.T) {
g := newGate(nil)
h := gateHarness(t, g, errorsRule())
h.q.set(testExpr, "d1||5|p1")
h.step(30 * time.Second)
h.want("", "")
g.put(allPolicy("p", "interface_errors"))
h.step(30 * time.Second)
h.want("firing", "firing")
}

View File

@@ -7,6 +7,8 @@ import (
"math"
"strconv"
"time"
"github.com/v10networks/nmon/internal/alertpolicy"
)
// query runs a rule and returns one reading per active, trusted entity.
@@ -24,7 +26,7 @@ func (ev *Evaluator) query(ctx context.Context, r *compiledRule, now time.Time)
best := map[string]reading{}
var order []string
for _, s := range series {
ent, ok := ev.entityOfSeries(r, s)
ent, bind, ok := ev.entityOfSeries(r, s)
if !ok {
continue
}
@@ -35,7 +37,7 @@ func (ev *Evaluator) query(ctx context.Context, r *compiledRule, now time.Time)
}
// Duplicate series for one entity: keep the one nearer to alerting.
if !seen || r.nearer(s.value, cur.value) {
best[k] = reading{ent, s.value}
best[k] = reading{ent, s.value, bind}
}
}
out := make([]reading, 0, len(order))
@@ -54,14 +56,14 @@ func (r *compiledRule) nearer(a, b float64) bool {
}
// entityOfSeries maps a series to its entity. It reports false for series that
// lack a label the rule needs, are on unreliable interfaces, or are inactive.
func (ev *Evaluator) entityOfSeries(r *compiledRule, s sample) (Entity, bool) {
// lack a label the rule needs, are on unreliable interfaces, are inactive or that no alert policy binds.
func (ev *Evaluator) entityOfSeries(r *compiledRule, s sample) (Entity, alertpolicy.Binding, bool) {
ent := Entity{Rule: r.Name}
for _, l := range r.Labels {
v := s.labels[l]
if v == "" {
ev.log.Debug("series without label", "rule", r.Name, "label", l)
return Entity{}, false
return Entity{}, alertpolicy.Binding{}, false
}
switch l {
case "device":
@@ -73,12 +75,13 @@ func (ev *Evaluator) entityOfSeries(r *compiledRule, s sample) (Entity, bool) {
}
}
if r.SkipUnreliable && ent.Iface != "" && ev.o.Unreliable != nil && ev.o.Unreliable(ent.Iface) {
return Entity{}, false
return Entity{}, alertpolicy.Binding{}, false
}
if ev.o.Active != nil && !ev.o.Active(ent) {
return Entity{}, false
return Entity{}, alertpolicy.Binding{}, false
}
return ent, true
b, ok := ev.o.Policy(ent)
return ent, b, ok
}
type sample struct {

View File

@@ -8,9 +8,12 @@ import (
"bytes"
"errors"
"fmt"
"slices"
"strings"
"text/template"
"time"
"github.com/v10networks/nmon/internal/alertpolicy"
)
// Severities.
@@ -50,7 +53,10 @@ type Rule struct {
// SkipUnreliable drops series whose interface the Unreliable filter flags
// (32-bit counters on fast links wrap too quickly to trust).
SkipUnreliable bool `yaml:"skip_unreliable" json:"skip_unreliable"`
Disabled bool `yaml:"disabled" json:"disabled"`
// OptIn is a no-op kept so old configs still parse. Alert policies (the alert_down matcher) replaced it:
// a rule fires only where a policy binds it.
OptIn bool `yaml:"opt_in" json:"opt_in"`
Disabled bool `yaml:"disabled" json:"disabled"`
}
// Override replaces a rule's threshold for devices in a group.
@@ -254,6 +260,35 @@ func (r Rule) resolveThreshold(device string, g GroupLookup, thresholdFor func(d
return r.Threshold
}
// bound returns the rule as one entity's policy binding sees it (a copy with the policy's severity and for)
// and the threshold to compare with: the policy's, else the rule's group override, profile or default.
func (c *compiledRule) bound(b alertpolicy.Binding, device string, g GroupLookup,
thresholdFor func(device, rule string) (float64, bool)) (*compiledRule, float64) {
rr := *c
if b.Severity != "" {
rr.Severity = b.Severity
}
if b.For != nil {
rr.For = *b.For
}
if b.Threshold != nil {
return &rr, *b.Threshold
}
return &rr, c.resolveThreshold(device, g, thresholdFor)
}
// Level says what the rule's entity is: an interface, a component of a device, or a device. Policies use it to
// refuse matchers that cannot apply.
func (r Rule) Level() string {
switch {
case slices.Contains(r.Labels, "iface"):
return alertpolicy.LevelInterface
case slices.Contains(r.Labels, "component"):
return alertpolicy.LevelComponent
}
return alertpolicy.LevelDevice
}
// patchRule overlays the non-zero tunables of c onto d.
func patchRule(d, c Rule) Rule {
if c.Severity != "" {
@@ -335,6 +370,8 @@ const (
// or more disables the prediction half in effect.
// - printer_supply_low: negative levels (-1 other, -2 unknown, -3 "some
// remaining") are filtered out; the series then goes stale rather than low.
// - printer_supply_empty: a supply level of exactly 0 (critical); the negative
// sentinels are not 0, so they never fire it.
// - temp_high: the temperature metrics of the shipped profiles, each tagged
// with its metric name as component where the profile has no component
// label. entPhySensorValue is excluded, since it mixes units. Per-profile
@@ -421,6 +458,13 @@ func deviceRules(lab func(...string) []string) []Rule {
Summary: `{{.Device}} {{.Component}} at {{printf "%.0f" .Value}}%`,
Description: `Supply {{.Component}} on {{.Device}} is at {{printf "%.0f" .Value}}% (threshold {{printf "%.0f" .Threshold}}%).`,
},
{
Name: "printer_supply_empty", Severity: SevCritical, Op: OpGreater, Threshold: 0, For: 0,
Labels: lab("device", "component"),
Expr: `last_over_time(prtMarkerSuppliesLevel[1h]) == bool 0`,
Summary: `{{.Device}} {{.Component}} is empty`,
Description: `Supply {{.Component}} on {{.Device}} reports a level of 0 (empty).`,
},
{
Name: "temp_high", Severity: SevWarning, Op: OpGreater, Threshold: 70, For: 5 * time.Minute,
Labels: lab("device", "component"),
@@ -432,5 +476,14 @@ func deviceRules(lab func(...string) []string) []Rule {
Summary: `{{.Device}} {{.Component}} at {{printf "%.0f" .Value}} C`,
Description: `Temperature {{.Component}} on {{.Device}} is {{printf "%.1f" .Value}} C (threshold {{printf "%.0f" .Threshold}}).`,
},
{
// state 6 is established in every BGP MIB the profiles read (BGP4-MIB, CISCO-BGP4, Juniper BGP4-V2, Arista
// BGP4V2). The notification adds the peer's AS and description from the component attributes.
Name: "bgp_peer_down", Severity: SevCritical, Op: OpGreater, Threshold: 0, For: 2 * time.Minute,
Labels: lab("device", "component"),
Expr: `last_over_time(bgp_peer_state[15m]) != bool 6`,
Summary: `{{.Device}} BGP peer {{.Component}} is down`,
Description: `The BGP session to {{.Component}} on {{.Device}} has not been established for {{.For}}.`,
},
}
}

View File

@@ -0,0 +1,136 @@
package alertpolicy
import (
"path"
"regexp"
"slices"
"strings"
)
// Matcher is a compiled Scope.
type Matcher struct {
scope Scope
name, alias *regexp.Regexp
}
// Match reports whether the subject is in the scope. An interface matcher never matches a subject without a port.
func (m *Matcher) Match(s Subject) bool {
sc := m.scope
if sc.All {
return true
}
if len(sc.Groups) > 0 && !slices.ContainsFunc(sc.Groups, func(g string) bool { return slices.Contains(s.Groups, g) }) {
return false
}
if !matchDevice(sc.Devices, s.Device) {
return false
}
if sc.Interfaces.empty() {
return true
}
return s.Iface != nil && m.matchIface(*s.Iface)
}
func anyFold(list []string, v string) bool {
return len(list) == 0 || slices.ContainsFunc(list, func(x string) bool { return strings.EqualFold(x, v) })
}
func matchDevice(dm DeviceMatch, d Device) bool {
if len(dm.Slugs) > 0 && !slices.ContainsFunc(dm.Slugs, func(g string) bool { ok, _ := path.Match(g, d.Slug); return ok }) {
return false
}
return anyFold(dm.Profiles, d.Profile) && anyFold(dm.Roles, d.Role) && anyFold(dm.Sites, d.Site) && anyFold(dm.Kinds, d.Kind)
}
func (m *Matcher) matchIface(i Iface) bool {
im := m.scope.Interfaces
switch {
case len(im.IfTypes) > 0 && !slices.Contains(im.IfTypes, i.Type):
return false
case m.name != nil && !m.name.MatchString(i.Name):
return false
case m.alias != nil && !m.alias.MatchString(i.Alias):
return false
case im.MinSpeedMbps != nil && i.SpeedMbps < *im.MinSpeedMbps:
return false
case im.MaxSpeedMbps != nil && i.SpeedMbps > *im.MaxSpeedMbps:
return false
case im.AlertDown != nil && *im.AlertDown != i.AlertDown:
return false
}
return true
}
type entry struct {
p Policy
m *Matcher
}
// Set is a compiled list of policies.
type Set struct{ entries []entry }
// NewSet compiles the enabled policies. A policy that does not compile is left out, and the error says which;
// the rest still work, so one bad row cannot silence every other policy.
func NewSet(policies []Policy) (*Set, error) {
s := &Set{}
var bad []string
for _, p := range policies {
if !p.Enabled {
continue
}
m, err := p.Scope.Compile()
if err != nil {
bad = append(bad, p.Name+": "+err.Error())
continue
}
s.entries = append(s.entries, entry{p, m})
}
if len(bad) > 0 {
return s, &CompileError{Bad: bad}
}
return s, nil
}
// CompileError lists the policies NewSet skipped.
type CompileError struct{ Bad []string }
func (e *CompileError) Error() string { return "alert policies skipped: " + strings.Join(e.Bad, "; ") }
// Len is the number of enabled, compiled policies.
func (s *Set) Len() int { return len(s.entries) }
// Decide finds the policy that rules over (rule, subject). The matching policy with the highest priority
// decides; on a tie ignore beats alert, then the lowest name wins. It reports false when nothing matches or
// the decider is an ignore policy: the entity raises nothing.
func (s *Set) Decide(rule string, subj Subject) (Binding, bool) {
var best *Policy
for i := range s.entries {
e := &s.entries[i]
if e.p.Rule != rule || !e.m.Match(subj) {
continue
}
if best == nil || outranks(e.p, *best) {
best = &e.p
}
}
if best == nil || best.Action != ActionAlert {
return Binding{}, false
}
return Binding{PolicyID: best.ID, PolicyName: best.Name, Severity: best.Severity, Threshold: best.Threshold, For: best.For}, true
}
func outranks(a, b Policy) bool {
if a.Priority != b.Priority {
return a.Priority > b.Priority
}
if a.Action != b.Action {
return a.Action == ActionIgnore
}
return a.Name < b.Name
}
// Decide compiles policies and decides once; the evaluators keep a Set instead.
func Decide(policies []Policy, rule string, subj Subject) (Binding, bool) {
s, _ := NewSet(policies)
return s.Decide(rule, subj)
}

View File

@@ -0,0 +1,123 @@
package alertpolicy
import (
"testing"
"time"
)
func f64(v float64) *float64 { return &v }
func port(typ int64, alias string) Subject {
return Subject{Device: Device{Slug: "sw1"}, Iface: &Iface{Type: typ, Name: "e1", Alias: alias, SpeedMbps: 1000}}
}
func ownerPair() []Policy {
return []Policy{
{ID: "a", Name: "Fiber + uplinks: errors", Rule: "interface_errors", Action: ActionAlert, Enabled: true, Threshold: f64(0),
Scope: Scope{Interfaces: IfaceMatch{IfTypes: []int64{6}, IfAlias: "(?i)fib|uplink|core"}}},
{ID: "b", Name: "WiFi: no error alerts", Rule: "interface_errors", Action: ActionIgnore, Priority: 10, Enabled: true,
Scope: Scope{Interfaces: IfaceMatch{IfTypes: []int64{71}}}},
}
}
func TestOwnerExamplesDecideWifiAndFiber(t *testing.T) {
ps := ownerPair()
if b, ok := Decide(ps, "interface_errors", port(6, "Core Fiber to X")); !ok || b.Threshold == nil || *b.Threshold != 0 {
t.Fatalf("fiber port: %+v %v", b, ok)
}
if _, ok := Decide(ps, "interface_errors", port(71, "uplink wifi")); ok {
t.Fatal("wifi port must not alert")
}
if _, ok := Decide(ps, "interface_errors", port(6, "customer lan")); ok {
t.Fatal("unmatched ethernet port must not alert")
}
}
func TestNoPoliciesNothingAlerts(t *testing.T) {
if _, ok := Decide(nil, "device_down", Subject{}); ok {
t.Fatal("alerted without policies")
}
}
func TestPrecedence(t *testing.T) {
all := Scope{All: true}
mk := func(name, action string, prio int) Policy {
return Policy{ID: name, Name: name, Rule: "r", Action: action, Priority: prio, Enabled: true, Scope: all}
}
cases := []struct {
name string
ps []Policy
want string // "" = no alert
}{
{"higher priority alert beats ignore", []Policy{mk("i", ActionIgnore, 0), mk("a", ActionAlert, 5)}, "a"},
{"tie: ignore wins", []Policy{mk("a", ActionAlert, 1), mk("i", ActionIgnore, 1)}, ""},
{"tie alerts: lowest name", []Policy{mk("b", ActionAlert, 1), mk("a", ActionAlert, 1)}, "a"},
{"higher ignore beats alert", []Policy{mk("a", ActionAlert, 1), mk("i", ActionIgnore, 2)}, ""},
{"disabled skipped", []Policy{{ID: "x", Name: "x", Rule: "r", Action: ActionAlert, Scope: all}}, ""},
{"other rule skipped", []Policy{{ID: "x", Name: "x", Rule: "q", Action: ActionAlert, Enabled: true, Scope: all}}, ""},
}
for _, c := range cases {
b, ok := Decide(c.ps, "r", Subject{})
if got := b.PolicyID; (c.want == "") == ok || got != c.want {
t.Errorf("%s: got %q ok=%v, want %q", c.name, got, ok, c.want)
}
}
}
func TestMatchers(t *testing.T) {
yes := true
cases := []struct {
name string
scope Scope
subj Subject
want bool
}{
{"group member", Scope{Groups: []string{"3"}}, Subject{Groups: []string{"1", "3"}}, true},
{"group miss", Scope{Groups: []string{"3"}}, Subject{Groups: []string{"1"}}, false},
{"slug glob", Scope{Devices: DeviceMatch{Slugs: []string{"core-*"}}}, Subject{Device: Device{Slug: "core-1"}}, true},
{"role fold", Scope{Devices: DeviceMatch{Roles: []string{"CORE"}}}, Subject{Device: Device{Role: "core"}}, true},
{"iface matcher on device subject", Scope{Interfaces: IfaceMatch{IfTypes: []int64{6}}}, Subject{}, false},
{"alert_down", Scope{Interfaces: IfaceMatch{AlertDown: &yes}}, Subject{Iface: &Iface{AlertDown: true}}, true},
{"min speed", Scope{Interfaces: IfaceMatch{MinSpeedMbps: new(int64)}}, Subject{Iface: &Iface{}}, true},
{"and across fields", Scope{Groups: []string{"1"}, Devices: DeviceMatch{Kinds: []string{"switch"}}},
Subject{Groups: []string{"1"}, Device: Device{Kind: "router"}}, false},
}
for _, c := range cases {
m, err := c.scope.Compile()
if err != nil {
t.Fatal(err)
}
if got := m.Match(c.subj); got != c.want {
t.Errorf("%s: got %v", c.name, got)
}
}
}
func TestValidate(t *testing.T) {
levels := map[string]string{"interface_errors": LevelInterface, "cpu_high": LevelDevice, RuleSystem: LevelSystem}
ok := Policy{Name: "n", Rule: "interface_errors", Action: ActionAlert, Scope: Scope{Interfaces: IfaceMatch{IfTypes: []int64{6}}}}
if err := Validate(ok, levels); err != nil {
t.Fatal(err)
}
neg := -time.Second
bad := map[string]Policy{
"empty scope": {Name: "n", Rule: "cpu_high", Action: ActionAlert},
"unknown rule": {Name: "n", Rule: "nope", Action: ActionAlert, Scope: Scope{All: true}},
"iface on device": {Name: "n", Rule: "cpu_high", Action: ActionAlert, Scope: Scope{Interfaces: IfaceMatch{IfName: "x"}}},
"system not all": {Name: "n", Rule: RuleSystem, Action: ActionAlert, Scope: Scope{Groups: []string{"1"}}},
"bad regex": {Name: "n", Rule: "interface_errors", Action: ActionAlert, Scope: Scope{Interfaces: IfaceMatch{IfAlias: "("}}},
"all plus more": {Name: "n", Rule: "cpu_high", Action: ActionAlert, Scope: Scope{All: true, Groups: []string{"1"}}},
"bad action": {Name: "n", Rule: "cpu_high", Action: "x", Scope: Scope{All: true}},
"negative for": {Name: "n", Rule: "cpu_high", Action: ActionAlert, For: &neg, Scope: Scope{All: true}},
"missing name": {Rule: "cpu_high", Action: ActionAlert, Scope: Scope{All: true}},
"bad severity": {Name: "n", Rule: "cpu_high", Action: ActionAlert, Severity: "x", Scope: Scope{All: true}},
"min above max speed": {Name: "n", Rule: "interface_errors", Action: ActionAlert, Scope: Scope{Interfaces: IfaceMatch{MinSpeedMbps: i64(5), MaxSpeedMbps: i64(1)}}},
}
for n, p := range bad {
if Validate(p, levels) == nil {
t.Errorf("%s: accepted", n)
}
}
}
func i64(v int64) *int64 { return &v }

View File

@@ -0,0 +1,116 @@
// Package alertpolicy decides where alerts fire. A policy binds one rule to a scope (groups plus device and
// port attribute matchers); with no matching policy an entity raises nothing. The package is pure: it knows no
// store and no clock, so the evaluators, the API's preview and the tests all use the same decision.
package alertpolicy
import "time"
// Actions of a policy.
const (
ActionAlert = "alert"
ActionIgnore = "ignore"
)
// Rule names that are not metric rules. RuleSystem stands for every subsystem alarm (alert.SystemRule).
const (
RuleDeviceDown = "device_down"
RuleDeviceWarning = "device_warning"
RuleSystem = "system"
)
// Levels say what a rule's entity is, and so which matchers can apply to it.
const (
LevelDevice = "device"
LevelInterface = "interface"
LevelComponent = "component"
LevelSystem = "system"
)
// Policy binds one rule to a scope.
type Policy struct {
ID string `json:"id"`
Name string `json:"name"`
Rule string `json:"rule"`
Action string `json:"action"`
Priority int `json:"priority"`
Enabled bool `json:"enabled"`
// Severity, Threshold and For override the rule's; nil or empty keeps the rule's.
Severity string `json:"severity,omitempty"`
Threshold *float64 `json:"threshold,omitempty"`
For *time.Duration `json:"for,omitempty"`
Scope Scope `json:"scope"`
}
// Scope selects entities. Its fields are ANDed; within a list any entry may match; an empty field is
// unconstrained. A scope with no field set is invalid: All is the explicit "everything".
type Scope struct {
All bool `json:"all,omitempty"`
Groups []string `json:"groups,omitempty"`
Devices DeviceMatch `json:"devices,omitempty"`
Interfaces IfaceMatch `json:"interfaces,omitempty"`
}
// DeviceMatch picks devices by attribute. Slugs are path.Match globs; the others compare case-insensitively.
type DeviceMatch struct {
Slugs []string `json:"slugs,omitempty"`
Profiles []string `json:"profiles,omitempty"`
Roles []string `json:"roles,omitempty"`
Sites []string `json:"sites,omitempty"`
Kinds []string `json:"kinds,omitempty"`
}
// IfaceMatch picks ports by attribute. IfName and IfAlias are RE2 patterns searched anywhere in the text.
type IfaceMatch struct {
IfTypes []int64 `json:"if_types,omitempty"`
IfName string `json:"if_name,omitempty"`
IfAlias string `json:"if_alias,omitempty"`
MinSpeedMbps *int64 `json:"min_speed_mbps,omitempty"`
MaxSpeedMbps *int64 `json:"max_speed_mbps,omitempty"`
// AlertDown true matches ports whose bell, or a group's alert_down, is on; false matches the others.
AlertDown *bool `json:"alert_down,omitempty"`
}
// Binding is the outcome of a deciding "alert" policy: who decided and what it overrides.
type Binding struct {
PolicyID, PolicyName string
Severity string
Threshold *float64
For *time.Duration
}
// Subject is the thing a rule would alert on. Iface is nil for a device-level or component-level entity.
type Subject struct {
Device Device
Iface *Iface
// Groups are the ids of the groups the port is a member of now; for a device subject, the groups of any
// of its ports.
Groups []string
}
// Device holds the device attributes matchers read.
type Device struct{ Slug, Profile, Role, Site, Kind string }
// Iface holds the port attributes matchers read.
type Iface struct {
Type int64
Name string
Alias string
SpeedMbps int64
AlertDown bool
}
func (m IfaceMatch) empty() bool {
return len(m.IfTypes) == 0 && m.IfName == "" && m.IfAlias == "" && m.MinSpeedMbps == nil && m.MaxSpeedMbps == nil && m.AlertDown == nil
}
func (m DeviceMatch) empty() bool {
return len(m.Slugs)+len(m.Profiles)+len(m.Roles)+len(m.Sites)+len(m.Kinds) == 0
}
// Empty reports a scope that constrains nothing and is not All.
func (s Scope) Empty() bool {
return !s.All && len(s.Groups) == 0 && s.Devices.empty() && s.Interfaces.empty()
}
// HasInterfaceMatchers reports a scope that only a port can satisfy.
func (s Scope) HasInterfaceMatchers() bool { return !s.Interfaces.empty() }

View File

@@ -0,0 +1,24 @@
package alertpolicy
import (
"encoding/json"
"fmt"
)
// FormatScope renders a scope as the JSON the store keeps.
func FormatScope(s Scope) string {
b, err := json.Marshal(s)
if err != nil { // a Scope holds only strings, numbers and bools
panic(fmt.Sprintf("alertpolicy: marshal scope: %v", err))
}
return string(b)
}
// ParseScope reads what FormatScope wrote.
func ParseScope(raw string) (Scope, error) {
var s Scope
if err := json.Unmarshal([]byte(raw), &s); err != nil {
return Scope{}, fmt.Errorf("alert policy scope: %w", err)
}
return s, nil
}

View File

@@ -0,0 +1,103 @@
package alertpolicy
import (
"errors"
"fmt"
"math"
"path"
"regexp"
"strings"
)
// Validate checks a policy. levels maps every rule name a policy may name to its level (LevelDevice, ...);
// the caller builds it from the configured rules plus device_down, device_warning and system.
func Validate(p Policy, levels map[string]string) error {
var errs []error
add := func(f string, a ...any) { errs = append(errs, fmt.Errorf(f, a...)) }
if strings.TrimSpace(p.Name) == "" {
add("name is required")
}
level, known := levels[p.Rule]
if !known {
add("rule %q is not a known rule", p.Rule)
}
if p.Action != ActionAlert && p.Action != ActionIgnore {
add("action %q is not alert or ignore", p.Action)
}
switch p.Severity {
case "", "critical", "warning", "info":
default:
add("severity %q is not critical, warning or info", p.Severity)
}
if p.Threshold != nil && (math.IsNaN(*p.Threshold) || math.IsInf(*p.Threshold, 0)) {
add("threshold is not a number")
}
if p.For != nil && *p.For < 0 {
add("for is negative")
}
if known {
errs = append(errs, validateScopeFor(p.Scope, level)...)
}
errs = append(errs, validateScope(p.Scope)...)
return errors.Join(errs...)
}
// validateScopeFor checks the scope against what the rule's entity can be.
func validateScopeFor(s Scope, level string) []error {
var errs []error
if level == LevelSystem && !s.All {
errs = append(errs, errors.New("a system policy accepts only scope.all: true, because system alarms have no device"))
}
if level != LevelInterface && s.HasInterfaceMatchers() {
errs = append(errs, fmt.Errorf("interface matchers cannot apply to a %s rule", level))
}
return errs
}
func validateScope(s Scope) []error {
var errs []error
if s.Empty() {
return []error{errors.New("scope is empty: set all: true to mean everything")}
}
if s.All && (len(s.Groups) > 0 || !s.Devices.empty() || s.Interfaces.HasAny()) {
errs = append(errs, errors.New("scope.all excludes every other scope field"))
}
for _, g := range s.Groups {
if strings.TrimSpace(g) == "" {
errs = append(errs, errors.New("scope.groups holds an empty group id"))
}
}
for _, g := range s.Devices.Slugs {
if _, err := path.Match(g, ""); err != nil {
errs = append(errs, fmt.Errorf("device slug glob %q: %w", g, err))
}
}
if _, err := s.Compile(); err != nil {
errs = append(errs, err)
}
if mn, mx := s.Interfaces.MinSpeedMbps, s.Interfaces.MaxSpeedMbps; mn != nil && mx != nil && *mn > *mx {
errs = append(errs, errors.New("min_speed_mbps is above max_speed_mbps"))
}
return errs
}
// HasAny reports whether any interface matcher is set.
func (m IfaceMatch) HasAny() bool { return !m.empty() }
// Compile prepares a scope for matching; it fails on a bad regular expression.
func (s Scope) Compile() (*Matcher, error) {
m := &Matcher{scope: s}
var errs []error
var err error
if s.Interfaces.IfName != "" {
if m.name, err = regexp.Compile(s.Interfaces.IfName); err != nil {
errs = append(errs, fmt.Errorf("if_name: %w", err))
}
}
if s.Interfaces.IfAlias != "" {
if m.alias, err = regexp.Compile(s.Interfaces.IfAlias); err != nil {
errs = append(errs, fmt.Errorf("if_alias: %w", err))
}
}
return m, errors.Join(errs...)
}

155
internal/api/alert_down.go Normal file
View File

@@ -0,0 +1,155 @@
package api
import (
"context"
"net/http"
"github.com/danielgtaylor/huma/v2"
"github.com/v10networks/nmon/internal/store"
)
// AlertDownState says whether an interface alerts when it goes down. Own is the port's flag; Effective adds
// membership of a group with the flag on, and Via names the source ("port", or "group:<name>").
type AlertDownState struct {
AlertDown bool `json:"alert_down" doc:"The port's own opt-in for interface_down alerts."`
AlertDownEffective bool `json:"alert_down_effective" doc:"The port alerts when down: own flag, or a group it is in now."`
AlertDownVia string `json:"alert_down_via" doc:"Why it is effective: port, or group:<name>. Empty when it is not."`
}
func alertDownState(own bool, via string) AlertDownState {
return AlertDownState{AlertDown: own, AlertDownEffective: via != "", AlertDownVia: via}
}
// InterfaceAlertBody is the body of PUT /interfaces/{id}.
type InterfaceAlertBody struct {
AlertDown bool `json:"alert_down" doc:"Opt the port in (true) or out (false) of interface_down alerts."`
}
// DeviceAlertDownBody is the body of POST /devices/{slug}/interfaces/alert-down.
type DeviceAlertDownBody struct {
On bool `json:"on" doc:"Opt the ports in (true) or out (false)."`
Ifaces []string `json:"ifaces,omitempty" doc:"Interface ids of this device. Omitted means every active port of the device."`
}
// AlertDownResult reports how many ports a bulk change touched.
type AlertDownResult struct {
Changed int `json:"changed"`
}
func (s *server) registerAlertDown(api huma.API) {
huma.Register(api, huma.Operation{
OperationID: "set-interface-alert-down", Method: http.MethodPut, Path: "/api/v1/interfaces/{id}", Tags: []string{"interfaces"},
Summary: "Opt one interface in or out of interface_down alerts",
Description: "Admin only. interface_down alerts are opt-in: a down port raises nothing until its own flag is on " +
"or a group it belongs to has alert_down on.",
Errors: []int{http.StatusNotFound},
}, s.setInterfaceAlertDown)
huma.Register(api, huma.Operation{
OperationID: "set-device-alert-down", Method: http.MethodPost, Path: "/api/v1/devices/{slug}/interfaces/alert-down",
Tags: []string{"devices"},
Summary: "Opt a device's interfaces in or out of interface_down alerts",
Description: "Admin only. Without ifaces the change applies to every active port of the device. " +
"Turning a port off closes its open interface_down alert silently.",
Errors: []int{http.StatusNotFound, http.StatusUnprocessableEntity},
}, s.setDeviceAlertDown)
}
func (s *server) alertDownChanged() {
if s.OnAlertDownChange != nil {
s.OnAlertDownChange()
}
}
func (s *server) setInterfaceAlertDown(ctx context.Context, in *struct {
ID string `path:"id" doc:"Interface ULID."`
Body InterfaceAlertBody
}) (*struct{ Body InterfaceView }, error) {
err := s.Store.WithTx(ctx, func(tx store.Store) error {
if err := tx.SetInterfaceAlertDown(ctx, []string{in.ID}, in.Body.AlertDown); err != nil {
return err
}
return tx.AppendAudit(ctx, actorOf(ctx), "interface.alert_down", in.ID, nil, in.Body)
})
if err != nil {
return nil, s.storeErr(err)
}
s.alertDownChanged()
i, err := s.Store.GetInterface(ctx, in.ID)
if err != nil {
return nil, s.storeErr(err)
}
via, err := s.Store.InterfaceAlertDownVia(ctx, []string{in.ID})
if err != nil {
return nil, s.storeErr(err)
}
v := interfaceView(i, nil)
v.AlertDownState = alertDownState(i.AlertDown, via[in.ID])
return &struct{ Body InterfaceView }{v}, nil
}
func (s *server) setDeviceAlertDown(ctx context.Context, in *struct {
Slug string `path:"slug" doc:"Device slug."`
Body DeviceAlertDownBody
}) (*struct{ Body AlertDownResult }, error) {
d, err := s.Store.GetDeviceBySlug(ctx, in.Slug)
if err != nil || d.PurgedAt != nil {
return nil, huma.Error404NotFound("not found")
}
ifs, err := s.Store.ListInterfaces(ctx, d.ID, false)
if err != nil {
return nil, s.storeErr(err)
}
ids, err := alertDownTargets(ifs, in.Body.Ifaces)
if err != nil {
return nil, err
}
err = s.Store.WithTx(ctx, func(tx store.Store) error {
if err := tx.SetInterfaceAlertDown(ctx, ids, in.Body.On); err != nil {
return err
}
return tx.AppendAudit(ctx, actorOf(ctx), "device.interfaces.alert_down", d.Slug, nil,
map[string]any{"on": in.Body.On, "count": len(ids), "all": in.Body.Ifaces == nil})
})
if err != nil {
return nil, s.storeErr(err)
}
s.alertDownChanged()
return &struct{ Body AlertDownResult }{AlertDownResult{Changed: len(ids)}}, nil
}
// alertDownTargets picks the port ids a bulk change applies to: every active port, or the listed ones, which
// must all belong to the device.
func alertDownTargets(ifs []store.Interface, want []string) ([]string, error) {
have := make(map[string]bool, len(ifs))
all := make([]string, 0, len(ifs))
for _, i := range ifs {
have[i.ID] = true
all = append(all, i.ID)
}
if want == nil {
return all, nil
}
for _, id := range want {
if !have[id] {
return nil, huma.Error422UnprocessableEntity("interface " + id + " is not an active port of this device")
}
}
return want, nil
}
// withAlertDown fills the alert_down fields of interface views from one store query.
func (s *server) withAlertDown(ctx context.Context, ifs []store.Interface, views []InterfaceView) error {
ids := make([]string, len(ifs))
for i := range ifs {
ids[i] = ifs[i].ID
}
via, err := s.Store.InterfaceAlertDownVia(ctx, ids)
if err != nil {
return err
}
for i := range views {
views[i].AlertDownState = alertDownState(ifs[i].AlertDown, via[ifs[i].ID])
}
return nil
}

View File

@@ -0,0 +1,121 @@
package api_test
import (
"encoding/json"
"fmt"
"net/http"
"strings"
"testing"
"github.com/v10networks/nmon/internal/api"
"github.com/v10networks/nmon/internal/store"
)
func alertDownEnv(t *testing.T) (*env, *store.Device, store.Interface, store.Interface, *int) {
changes := new(int)
e := newEnvDeps(t, 0, func(d *api.Deps) { d.OnAlertDownChange = func() { *changes++ } })
d := e.device(t, "sw1", "yyz", "access")
return e, d, e.iface(t, d, 1, "e1", ""), e.iface(t, d, 2, "e2", ""), changes
}
func TestInterfaceAlertDownIsOffByDefaultAndTogglesPerPort(t *testing.T) {
e, d, a, b, changes := alertDownEnv(t)
var dv api.DeviceView
e.getJSON(t, "/api/v1/devices/"+d.Slug, &dv)
for _, i := range dv.Interfaces {
if i.AlertDown || i.AlertDownEffective || i.AlertDownVia != "" {
t.Fatalf("%s alerts by default: %+v", i.IfName, i.AlertDownState)
}
}
rec := e.do("PUT", "/api/v1/interfaces/"+a.ID, `{"alert_down":true}`)
if rec.Code != http.StatusOK {
t.Fatalf("PUT: %d %s", rec.Code, rec.Body)
}
var iv api.InterfaceView
if err := json.Unmarshal(rec.Body.Bytes(), &iv); err != nil || !iv.AlertDown || !iv.AlertDownEffective || iv.AlertDownVia != "port" {
t.Fatalf("PUT body %s (%v)", rec.Body, err)
}
if *changes != 1 {
t.Errorf("evaluator told %d times, want 1", *changes)
}
e.getJSON(t, "/api/v1/devices/"+d.Slug, &dv)
if !dv.Interfaces[0].AlertDown || dv.Interfaces[1].AlertDown {
t.Fatalf("device view flags: %+v %+v", dv.Interfaces[0].AlertDownState, dv.Interfaces[1].AlertDownState)
}
var hits api.InterfaceList
e.getJSON(t, "/api/v1/interfaces?q=e", &hits)
got := map[string]string{}
for _, h := range hits.Items {
got[h.ID] = h.AlertDownVia
}
if got[a.ID] != "port" || got[b.ID] != "" {
t.Fatalf("search hits via = %v", got)
}
isProblem(t, e.do("PUT", "/api/v1/interfaces/nope", `{"alert_down":true}`), http.StatusNotFound)
audit, _, _ := e.st.ListAudit(t.Context(), store.Page{})
if len(audit) == 0 || audit[0].Action != "interface.alert_down" {
t.Fatalf("audit = %+v", audit)
}
}
func TestDeviceBulkAlertDownAllOrListedPorts(t *testing.T) {
e, d, a, _, _ := alertDownEnv(t)
url := "/api/v1/devices/" + d.Slug + "/interfaces/alert-down"
var res api.AlertDownResult
rec := e.do("POST", url, `{"on":true}`)
if rec.Code != http.StatusOK || json.Unmarshal(rec.Body.Bytes(), &res) != nil || res.Changed != 2 {
t.Fatalf("all on: %d %s", rec.Code, rec.Body)
}
rec = e.do("POST", url, fmt.Sprintf(`{"on":false,"ifaces":[%q]}`, a.ID))
if rec.Code != http.StatusOK {
t.Fatalf("one off: %d %s", rec.Code, rec.Body)
}
var dv api.DeviceView
e.getJSON(t, "/api/v1/devices/"+d.Slug, &dv)
if dv.Interfaces[0].AlertDown || !dv.Interfaces[1].AlertDown {
t.Fatalf("flags after partial off: %+v %+v", dv.Interfaces[0].AlertDownState, dv.Interfaces[1].AlertDownState)
}
other := e.device(t, "sw2", "yyz", "access")
foreign := e.iface(t, other, 1, "x1", "")
isProblem(t, e.do("POST", url, fmt.Sprintf(`{"on":true,"ifaces":[%q]}`, foreign.ID)), http.StatusUnprocessableEntity)
isProblem(t, e.do("POST", "/api/v1/devices/nope/interfaces/alert-down", `{"on":true}`), http.StatusNotFound)
rec = e.do("POST", url, `{"on":false}`)
e.getJSON(t, "/api/v1/devices/"+d.Slug, &dv)
if rec.Code != http.StatusOK || dv.Interfaces[1].AlertDown {
t.Fatalf("all off: %d %s", rec.Code, rec.Body)
}
}
func TestGroupAlertDownMakesMemberPortsEffectiveVia(t *testing.T) {
e, d, a, _, changes := alertDownEnv(t)
rec := e.do("POST", "/api/v1/groups", `{"name":"Uplinks","kind":"role","alert_down":true}`)
var g api.GroupSummary
if rec.Code != http.StatusCreated || json.Unmarshal(rec.Body.Bytes(), &g) != nil || !g.AlertDown {
t.Fatalf("create: %d %s", rec.Code, rec.Body)
}
rec = e.do("POST", "/api/v1/groups/"+g.ID+"/members", fmt.Sprintf(`{"interface_id":%q,"direction":"toward"}`, a.ID))
if rec.Code != http.StatusCreated {
t.Fatalf("member: %d %s", rec.Code, rec.Body)
}
var dv api.DeviceView
e.getJSON(t, "/api/v1/devices/"+d.Slug, &dv)
if i := dv.Interfaces[0]; i.AlertDown || !i.AlertDownEffective || i.AlertDownVia != "group:Uplinks" {
t.Fatalf("member port: %+v", i.AlertDownState)
}
// A PUT without alert_down leaves the flag; an explicit false clears it.
rec = e.do("PUT", "/api/v1/groups/"+g.ID, `{"name":"Uplinks 2","kind":"role"}`)
if rec.Code != http.StatusOK || !strings.Contains(rec.Body.String(), `"alert_down":true`) {
t.Fatalf("rename: %d %s", rec.Code, rec.Body)
}
rec = e.do("PUT", "/api/v1/groups/"+g.ID, `{"name":"Uplinks 2","kind":"role","alert_down":false}`)
if rec.Code != http.StatusOK || !strings.Contains(rec.Body.String(), `"alert_down":false`) {
t.Fatalf("clear: %d %s", rec.Code, rec.Body)
}
e.getJSON(t, "/api/v1/devices/"+d.Slug, &dv)
if dv.Interfaces[0].AlertDownEffective {
t.Fatal("still effective after the group flag cleared")
}
if *changes < 4 {
t.Errorf("evaluator told %d times; group writes must tell it", *changes)
}
}

View File

@@ -0,0 +1,347 @@
package api
import (
"context"
"errors"
"fmt"
"net/http"
"strings"
"time"
"github.com/danielgtaylor/huma/v2"
"github.com/v10networks/nmon/internal/alertpolicy"
"github.com/v10networks/nmon/internal/store"
)
// AlertRuleInfo describes a rule a policy can bind, for the UI's dropdown.
type AlertRuleInfo struct {
Name string `json:"name"`
Level string `json:"level" enum:"device,interface,component,system" doc:"What the rule's entity is; port matchers need interface."`
Op string `json:"op,omitempty" enum:">,<" doc:"Comparison of a metric rule; empty for reachability and system rules."`
Threshold *float64 `json:"threshold" nullable:"true" doc:"The rule's default threshold; null for reachability and system rules."`
Severity string `json:"severity" enum:"critical,warning,info"`
}
// AlertRuleList is every rule a policy can bind.
type AlertRuleList struct {
Items []AlertRuleInfo `json:"items"`
}
// AlertPolicyHost is the running alerting engine as the API sees it.
type AlertPolicyHost interface {
// Rules lists the rules a policy may name.
Rules() []AlertRuleInfo
// Changed tells the evaluators a policy was written, so it applies on their next cycle.
Changed()
// Preview counts what a scope covers right now.
Preview(ctx context.Context, rule string, scope alertpolicy.Scope) (PolicyPreview, error)
}
// PolicyPreview is what a draft scope covers.
type PolicyPreview struct {
Devices PreviewDevices `json:"devices"`
Interfaces PreviewInterfaces `json:"interfaces"`
}
// PreviewDevices counts devices; for an interface rule it counts devices with a covered port.
type PreviewDevices struct {
Count int `json:"count"`
Samples []DeviceRef `json:"samples" doc:"Up to 20."`
}
// PreviewInterfaces counts ports; it is zero for rules that have no port.
type PreviewInterfaces struct {
Count int `json:"count"`
Samples []PreviewInterface `json:"samples" doc:"Up to 20."`
}
// PreviewInterface names a covered port.
type PreviewInterface struct {
ID string `json:"id"`
Device string `json:"device" doc:"Device slug."`
IfName string `json:"ifname"`
IfAlias string `json:"ifalias"`
}
// AlertPolicyView is a policy as the API returns it.
type AlertPolicyView struct {
ID string `json:"id"`
Name string `json:"name"`
Rule string `json:"rule"`
Action string `json:"action" enum:"alert,ignore"`
Priority int `json:"priority"`
Enabled bool `json:"enabled"`
Severity *string `json:"severity" nullable:"true" doc:"Overrides the rule's severity; null keeps it."`
Threshold *float64 `json:"threshold" nullable:"true" doc:"Overrides the rule's threshold; null keeps it."`
For *string `json:"for" nullable:"true" doc:"Overrides the rule's hold time, a duration such as 5m; null keeps it."`
Scope alertpolicy.Scope `json:"scope"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}
// AlertPolicyList is every policy, by name.
type AlertPolicyList struct {
Items []AlertPolicyView `json:"items"`
}
// AlertPolicyBody is the body of POST and PUT /alert-policies.
type AlertPolicyBody struct {
Name string `json:"name" minLength:"1" maxLength:"200"`
Rule string `json:"rule" minLength:"1" doc:"A name from GET /alert-rules."`
Action string `json:"action" enum:"alert,ignore" default:"alert"`
Priority int `json:"priority,omitempty" doc:"The highest priority decides; on a tie ignore wins, then the lowest name."`
Enabled *bool `json:"enabled,omitempty" doc:"Omitted: on for create, kept for update."`
Severity *string `json:"severity,omitempty" enum:"critical,warning,info"`
Threshold *float64 `json:"threshold,omitempty"`
For *string `json:"for,omitempty" doc:"A duration such as 5m."`
Scope alertpolicy.Scope `json:"scope" doc:"Must not be empty; all: true means everything."`
}
// AlertPolicyPreviewBody is the body of POST /alert-policies/preview.
type AlertPolicyPreviewBody struct {
Rule string `json:"rule" minLength:"1"`
Scope alertpolicy.Scope `json:"scope"`
}
func (s *server) registerAlertPolicies(api huma.API) {
tags := []string{"alerts"}
conflict := []int{http.StatusConflict, http.StatusUnprocessableEntity, http.StatusServiceUnavailable}
huma.Register(api, huma.Operation{OperationID: "list-alert-rules", Method: http.MethodGet, Path: "/api/v1/alert-rules", Tags: tags,
Summary: "Rules an alert policy can bind", Errors: []int{http.StatusServiceUnavailable}}, s.listAlertRules)
huma.Register(api, huma.Operation{OperationID: "list-alert-policies", Method: http.MethodGet, Path: "/api/v1/alert-policies", Tags: tags,
Summary: "List alert policies",
Description: "Alerts fire only where an enabled policy binds a rule to a scope; with no policies nothing alerts."}, s.listAlertPolicies)
huma.Register(api, huma.Operation{OperationID: "create-alert-policy", Method: http.MethodPost, Path: "/api/v1/alert-policies", Tags: tags,
DefaultStatus: http.StatusCreated, Summary: "Create an alert policy", Errors: conflict}, s.createAlertPolicy)
huma.Register(api, huma.Operation{OperationID: "preview-alert-policy", Method: http.MethodPost, Path: "/api/v1/alert-policies/preview",
Tags: tags, Summary: "Count the devices and ports a draft scope covers",
Errors: []int{http.StatusUnprocessableEntity, http.StatusServiceUnavailable}}, s.previewAlertPolicy)
huma.Register(api, huma.Operation{OperationID: "get-alert-policy", Method: http.MethodGet, Path: "/api/v1/alert-policies/{id}", Tags: tags,
Summary: "An alert policy", Errors: []int{http.StatusNotFound}}, s.getAlertPolicy)
huma.Register(api, huma.Operation{
OperationID: "update-alert-policy", Method: http.MethodPut, Path: "/api/v1/alert-policies/{id}", Tags: tags,
Summary: "Replace an alert policy", Errors: append([]int{http.StatusNotFound}, conflict...),
}, s.updateAlertPolicy)
huma.Register(api, huma.Operation{
OperationID: "delete-alert-policy", Method: http.MethodDelete, Path: "/api/v1/alert-policies/{id}",
Tags: tags, DefaultStatus: http.StatusNoContent, Summary: "Delete an alert policy",
Description: "Open alerts the policy bound close silently as retired on the next cycle.", Errors: []int{http.StatusNotFound},
}, s.deleteAlertPolicy)
}
func (s *server) policyHost() (AlertPolicyHost, error) {
if s.AlertPolicy == nil {
return nil, errNotConfigured("alert policies")
}
return s.AlertPolicy, nil
}
func (s *server) listAlertRules(_ context.Context, _ *struct{}) (*struct{ Body AlertRuleList }, error) {
h, err := s.policyHost()
if err != nil {
return nil, err
}
return &struct{ Body AlertRuleList }{AlertRuleList{Items: h.Rules()}}, nil
}
func policyView(r store.AlertPolicy) (AlertPolicyView, error) {
scope, err := alertpolicy.ParseScope(r.Scope)
if err != nil {
return AlertPolicyView{}, fmt.Errorf("policy %s: %w", r.ID, err)
}
v := AlertPolicyView{ID: r.ID, Name: r.Name, Rule: r.Rule, Action: r.Action, Priority: r.Priority, Enabled: r.Enabled,
Threshold: r.Threshold, Scope: scope, CreatedAt: r.CreatedAt, UpdatedAt: r.UpdatedAt}
if r.Severity != "" {
v.Severity = &r.Severity
}
if r.ForSeconds != nil {
d := (time.Duration(*r.ForSeconds) * time.Second).String()
v.For = &d
}
return v, nil
}
func (s *server) listAlertPolicies(ctx context.Context, _ *struct{}) (*struct{ Body AlertPolicyList }, error) {
rows, err := s.Store.ListAlertPolicies(ctx)
if err != nil {
return nil, s.storeErr(err)
}
out := AlertPolicyList{Items: make([]AlertPolicyView, 0, len(rows))}
for _, r := range rows {
v, err := policyView(r)
if err != nil {
s.Logger.Error("alert policy unreadable", "err", err)
continue
}
out.Items = append(out.Items, v)
}
return &struct{ Body AlertPolicyList }{out}, nil
}
func (s *server) getAlertPolicy(ctx context.Context, in *struct {
ID string `path:"id" doc:"Policy ULID."`
}) (*struct{ Body AlertPolicyView }, error) {
r, err := s.Store.GetAlertPolicy(ctx, in.ID)
if err != nil {
return nil, s.storeErr(err)
}
v, err := policyView(*r)
if err != nil {
return nil, s.storeErr(err)
}
return &struct{ Body AlertPolicyView }{v}, nil
}
// policyRow validates a body against the known rules and converts it to a store row. enabled is the value to
// keep when the body omits it.
func (s *server) policyRow(b AlertPolicyBody, enabled bool) (store.AlertPolicy, error) {
h, err := s.policyHost()
if err != nil {
return store.AlertPolicy{}, err
}
p := alertpolicy.Policy{Name: strings.TrimSpace(b.Name), Rule: b.Rule, Action: b.Action, Priority: b.Priority,
Enabled: enabled, Threshold: b.Threshold, Scope: b.Scope}
if b.Enabled != nil {
p.Enabled = *b.Enabled
}
if p.Action == "" {
p.Action = alertpolicy.ActionAlert
}
if b.Severity != nil {
p.Severity = *b.Severity
}
if b.For != nil {
d, err := time.ParseDuration(*b.For)
if err != nil {
return store.AlertPolicy{}, huma.Error422UnprocessableEntity("for: " + err.Error())
}
p.For = &d
}
if err := alertpolicy.Validate(p, ruleLevels(h.Rules())); err != nil {
return store.AlertPolicy{}, huma.Error422UnprocessableEntity(flatten(err))
}
row := store.AlertPolicy{Name: p.Name, Rule: p.Rule, Action: p.Action, Priority: p.Priority, Enabled: p.Enabled,
Severity: p.Severity, Threshold: p.Threshold, Scope: alertpolicy.FormatScope(p.Scope)}
if p.For != nil {
secs := int64(p.For.Seconds())
row.ForSeconds = &secs
}
return row, nil
}
func ruleLevels(rules []AlertRuleInfo) map[string]string {
m := make(map[string]string, len(rules))
for _, r := range rules {
m[r.Name] = r.Level
}
return m
}
func flatten(err error) string {
var parts []string
for _, line := range strings.Split(err.Error(), "\n") {
if line != "" {
parts = append(parts, line)
}
}
return strings.Join(parts, "; ")
}
func (s *server) policyChanged() {
if s.AlertPolicy != nil {
s.AlertPolicy.Changed()
}
}
func (s *server) createAlertPolicy(ctx context.Context, in *struct{ Body AlertPolicyBody }) (*struct{ Body AlertPolicyView }, error) {
row, err := s.policyRow(in.Body, true)
if err != nil {
return nil, err
}
err = s.Store.WithTx(ctx, func(tx store.Store) error {
if err := tx.CreateAlertPolicy(ctx, &row); err != nil {
return err
}
return tx.AppendAudit(ctx, actorOf(ctx), "alert_policy.create", row.ID, nil, in.Body)
})
if err != nil {
return nil, s.storeErr(err)
}
s.policyChanged()
return s.policyResult(row)
}
func (s *server) updateAlertPolicy(ctx context.Context, in *struct {
ID string `path:"id" doc:"Policy ULID."`
Body AlertPolicyBody
}) (*struct{ Body AlertPolicyView }, error) {
cur, err := s.Store.GetAlertPolicy(ctx, in.ID)
if err != nil {
return nil, s.storeErr(err)
}
row, err := s.policyRow(in.Body, cur.Enabled)
if err != nil {
return nil, err
}
row.ID, row.CreatedAt = cur.ID, cur.CreatedAt
err = s.Store.WithTx(ctx, func(tx store.Store) error {
if err := tx.UpdateAlertPolicy(ctx, &row); err != nil {
return err
}
return tx.AppendAudit(ctx, actorOf(ctx), "alert_policy.update", row.ID, cur, in.Body)
})
if err != nil {
return nil, s.storeErr(err)
}
s.policyChanged()
return s.policyResult(row)
}
func (s *server) policyResult(row store.AlertPolicy) (*struct{ Body AlertPolicyView }, error) {
v, err := policyView(row)
if err != nil {
return nil, s.storeErr(err)
}
return &struct{ Body AlertPolicyView }{v}, nil
}
func (s *server) deleteAlertPolicy(ctx context.Context, in *struct {
ID string `path:"id" doc:"Policy ULID."`
}) (*struct{}, error) {
err := s.Store.WithTx(ctx, func(tx store.Store) error {
cur, err := tx.GetAlertPolicy(ctx, in.ID)
if err != nil {
return err
}
if err := tx.DeleteAlertPolicy(ctx, in.ID); err != nil {
return err
}
return tx.AppendAudit(ctx, actorOf(ctx), "alert_policy.delete", in.ID, cur, nil)
})
if err != nil {
return nil, s.storeErr(err)
}
s.policyChanged()
return nil, nil
}
func (s *server) previewAlertPolicy(ctx context.Context, in *struct{ Body AlertPolicyPreviewBody }) (*struct{ Body PolicyPreview }, error) {
h, err := s.policyHost()
if err != nil {
return nil, err
}
draft := alertpolicy.Policy{Name: "preview", Rule: in.Body.Rule, Action: alertpolicy.ActionAlert, Scope: in.Body.Scope}
if err := alertpolicy.Validate(draft, ruleLevels(h.Rules())); err != nil {
return nil, huma.Error422UnprocessableEntity(flatten(err))
}
p, err := h.Preview(ctx, in.Body.Rule, in.Body.Scope)
if err != nil {
if errors.Is(err, store.ErrNotFound) {
return nil, huma.Error404NotFound("not found")
}
s.Logger.Error("alert policy preview", "err", err)
return nil, huma.Error500InternalServerError("internal error")
}
return &struct{ Body PolicyPreview }{p}, nil
}

View File

@@ -0,0 +1,112 @@
package api_test
import (
"context"
"encoding/json"
"net/http"
"testing"
"github.com/v10networks/nmon/internal/alertpolicy"
"github.com/v10networks/nmon/internal/api"
"github.com/v10networks/nmon/internal/store"
)
type fakePolicyHost struct {
changes int
preview api.PolicyPreview
}
func (f *fakePolicyHost) Rules() []api.AlertRuleInfo {
thr := 0.1
return []api.AlertRuleInfo{
{Name: "interface_errors", Level: "interface", Op: ">", Threshold: &thr, Severity: "warning"},
{Name: "cpu_high", Level: "device", Op: ">", Threshold: &thr, Severity: "warning"},
{Name: "device_down", Level: "device", Severity: "critical"},
{Name: "system", Level: "system", Severity: "warning"},
}
}
func (f *fakePolicyHost) Changed() { f.changes++ }
func (f *fakePolicyHost) Preview(context.Context, string, alertpolicy.Scope) (api.PolicyPreview, error) {
return f.preview, nil
}
func policyEnv(t *testing.T) (*env, *fakePolicyHost) {
h := &fakePolicyHost{preview: api.PolicyPreview{Devices: api.PreviewDevices{Count: 2, Samples: []api.DeviceRef{}}}}
return newEnvDeps(t, 0, func(d *api.Deps) { d.AlertPolicy = h }), h
}
const fiberPolicy = `{"name":"Fiber","rule":"interface_errors","action":"alert","threshold":0,"for":"5m","severity":"critical",
"scope":{"interfaces":{"if_types":[6],"if_alias":"(?i)fib"}}}`
func TestAlertPolicyCRUDIsAuditedAndTellsTheEngine(t *testing.T) {
e, host := policyEnv(t)
rec := e.do("POST", "/api/v1/alert-policies", fiberPolicy)
if rec.Code != http.StatusCreated {
t.Fatalf("POST: %d %s", rec.Code, rec.Body)
}
var v api.AlertPolicyView
if err := json.Unmarshal(rec.Body.Bytes(), &v); err != nil || v.ID == "" || !v.Enabled || *v.For != "5m0s" || *v.Threshold != 0 {
t.Fatalf("created = %s (%v)", rec.Body, err)
}
isProblem(t, e.do("POST", "/api/v1/alert-policies", fiberPolicy), http.StatusConflict)
upd := `{"name":"Fiber","rule":"interface_errors","action":"ignore","priority":5,"enabled":false,"scope":{"all":true}}`
rec = e.do("PUT", "/api/v1/alert-policies/"+v.ID, upd)
if rec.Code != http.StatusOK {
t.Fatalf("PUT: %d %s", rec.Code, rec.Body)
}
var list api.AlertPolicyList
e.getJSON(t, "/api/v1/alert-policies", &list)
if len(list.Items) != 1 || list.Items[0].Action != "ignore" || list.Items[0].Enabled || list.Items[0].Threshold != nil || !list.Items[0].Scope.All {
t.Fatalf("list = %+v", list)
}
if rec := e.do("DELETE", "/api/v1/alert-policies/"+v.ID, ""); rec.Code != http.StatusNoContent {
t.Fatalf("DELETE: %d %s", rec.Code, rec.Body)
}
isProblem(t, e.do("GET", "/api/v1/alert-policies/"+v.ID, ""), http.StatusNotFound)
if host.changes != 3 {
t.Errorf("engine told %d times, want 3", host.changes)
}
audit, _, _ := e.st.ListAudit(t.Context(), store.Page{})
if len(audit) < 3 || audit[0].Action != "alert_policy.delete" {
t.Fatalf("audit = %+v", audit)
}
}
func TestAlertPolicyValidationRefusesBadScopes(t *testing.T) {
e, _ := policyEnv(t)
for name, body := range map[string]string{
"empty scope": `{"name":"a","rule":"cpu_high","action":"alert","scope":{}}`,
"unknown rule": `{"name":"a","rule":"nope","action":"alert","scope":{"all":true}}`,
"interface on device": `{"name":"a","rule":"cpu_high","action":"alert","scope":{"interfaces":{"if_types":[6]}}}`,
"system needs all": `{"name":"a","rule":"system","action":"alert","scope":{"groups":["1"]}}`,
"bad regex": `{"name":"a","rule":"interface_errors","action":"alert","scope":{"interfaces":{"if_name":"("}}}`,
"bad duration": `{"name":"a","rule":"cpu_high","action":"alert","for":"soon","scope":{"all":true}}`,
"unknown action": `{"name":"a","rule":"cpu_high","action":"maybe","scope":{"all":true}}`,
} {
rec := e.do("POST", "/api/v1/alert-policies", body)
if rec.Code != http.StatusUnprocessableEntity {
t.Errorf("%s: %d %s", name, rec.Code, rec.Body)
}
}
}
func TestAlertRulesAndPreview(t *testing.T) {
e, _ := policyEnv(t)
var rules api.AlertRuleList
e.getJSON(t, "/api/v1/alert-rules", &rules)
if len(rules.Items) != 4 {
t.Fatalf("rules = %+v", rules)
}
rec := e.do("POST", "/api/v1/alert-policies/preview", `{"rule":"device_down","scope":{"all":true}}`)
var p api.PolicyPreview
if err := json.Unmarshal(rec.Body.Bytes(), &p); rec.Code != http.StatusOK || err != nil || p.Devices.Count != 2 {
t.Fatalf("preview = %d %s", rec.Code, rec.Body)
}
isProblem(t, e.do("POST", "/api/v1/alert-policies/preview", `{"rule":"device_down","scope":{}}`), http.StatusUnprocessableEntity)
}
func TestAlertPolicyRoutesAnswer503WithoutAnEngine(t *testing.T) {
e := newEnvDeps(t, 0, nil)
isProblem(t, e.do("GET", "/api/v1/alert-rules", ""), http.StatusServiceUnavailable)
isProblem(t, e.do("POST", "/api/v1/alert-policies", fiberPolicy), http.StatusServiceUnavailable)
}

View File

@@ -78,6 +78,9 @@ type Deps struct {
// OnMaintenanceChange is called after a maintenance window is created or deleted, so the
// evaluators see it at once instead of at their next refresh.
OnMaintenanceChange func(ctx context.Context)
// OnAlertDownChange is called after a write that can change which ports alert on interface_down (a port or
// group flag, group membership), so the evaluator sees it at once instead of after its cache expires.
OnAlertDownChange func()
// PollInterval is the normal poll cadence; a device unheard of for three of them is down. Default 60 s.
PollInterval time.Duration
@@ -117,8 +120,17 @@ type Deps struct {
Series SeriesDeleter
// PurgeAfter is how long a device must have been retired before it can be purged. Default one year.
PurgeAfter func() time.Duration
// AlertPolicy is the running policy engine; routes under /alert-policies and /alert-rules answer 503 without it.
AlertPolicy AlertPolicyHost
// OnImported is told the ids of devices a CSV import created, so they can be discovered at once.
OnImported func(ids []int64)
// v1.3 collaborators; all optional. Vendors names MAC vendors from the IEEE registry (nil: vendor is empty
// everywhere). L2EdgeMax returns l2.edge_max_macs. L2Health and HealthNotes feed GET /poller/health.
Vendors VendorLookup
L2EdgeMax func() int
L2Health func() *L2Health
HealthNotes func() []HealthNote
}
type server struct {
@@ -185,14 +197,20 @@ func (s *server) register(api huma.API) {
s.registerDevices(api)
s.registerDeviceList(api)
s.registerDeviceViews(api)
s.registerProfiles(api)
s.registerInterfaces(api)
s.registerAlertDown(api)
s.registerGraph(api)
s.registerDeviceMetrics(api)
s.registerOverview(api)
s.registerPrinters(api)
s.registerOps(api)
s.registerHealth(api)
s.registerReviews(api)
s.registerQuery(api)
s.registerMeta(api)
s.registerAlerts(api)
s.registerAlertPolicies(api)
s.registerMaintenance(api)
s.registerMetrics(api)
s.registerTokens(api)
@@ -207,6 +225,9 @@ func (s *server) register(api huma.API) {
s.registerSettings(api)
s.registerImport(api)
s.registerPurge(api)
s.registerL2(api)
s.registerBGP(api)
s.registerRetire(api)
s.registerAdmin(api)
s.registerAudit(api)
}

335
internal/api/bgp.go Normal file
View File

@@ -0,0 +1,335 @@
package api
import (
"context"
"encoding/json"
"fmt"
"net/http"
"net/netip"
"slices"
"strconv"
"strings"
"github.com/danielgtaylor/huma/v2"
"github.com/v10networks/nmon/internal/profile"
"github.com/v10networks/nmon/internal/store"
)
// The profile metrics BGP peers are built from. Every BGP profile uses these names whatever MIB feeds them, so
// the rule, the API and the UI never care which vendor a peer comes from.
const (
bgpState = "bgp_peer_state"
bgpUptime = "bgp_peer_uptime"
bgpPrefixes = "bgp_prefixes_accepted"
bgpInUpd = "bgp_peer_in_updates"
bgpOutUpd = "bgp_peer_out_updates"
stateEstablished = "established"
stateUnknown = "unknown"
// rateWindow is long enough to hold several 5-minute samples of an update counter.
rateWindow = "15m"
)
func (s *server) registerBGP(api huma.API) {
huma.Register(api, huma.Operation{
OperationID: "list-bgp-peers", Method: http.MethodGet, Path: "/api/v1/bgp/peers", Tags: []string{"bgp"},
Summary: "BGP sessions of every device",
Description: "Every BGP peer the profiles collect, with the latest state, uptime, accepted prefixes per address family and " +
"update rates from the hot store, and the peer AS, local AS, router ID and description from discovery. Sessions that " +
"are down come first. state is established, down (anything else) or an FSM state name.",
Errors: []int{http.StatusTooManyRequests, http.StatusServiceUnavailable},
}, s.listBGPPeers)
}
// bgpDevice is a device whose profile collects BGP, with its peer rows.
type bgpDevice struct {
dev store.Device
state profile.Metric
comps []store.Component
}
func (s *server) listBGPPeers(ctx context.Context, in *struct {
Device string `query:"device" doc:"Only this device (slug)."`
State string `query:"state" doc:"established, down, or an FSM state: idle, connect, active, opensent, openconfirm."`
AS string `query:"as" pattern:"^[0-9]{1,10}$" doc:"Only sessions with this remote AS."`
Family string `query:"family" enum:"ipv4,ipv6" doc:"Only IPv4 or only IPv6 peers."`
Q string `query:"q" doc:"Substring of the peer address, description or device name."`
PageParams
}) (*struct{ Body BGPPeerList }, error) {
devs, err := s.bgpDevices(ctx, in.Device)
if err != nil {
return nil, err
}
peers, err := s.bgpPeers(ctx, devs)
if err != nil {
return nil, err
}
kept := peers[:0]
var counts BGPCounts
for _, p := range peers {
if !bgpMatches(p, optInt(in.AS), in.Family, in.Q) {
continue
}
counts.Total++
if p.Established {
counts.Established++
} else {
counts.Down++
}
if bgpStateMatches(p, in.State) {
kept = append(kept, p)
}
}
sortPeers(kept)
total := int64(len(kept))
lo := min(in.offset(), total)
page := kept[lo:min(lo+in.limit(), total)]
out := BGPPeerList{Items: slices.Clone(page), Page: in.pageNo(), PerPage: in.limit(), Total: total, Counts: counts}
return &struct{ Body BGPPeerList }{out}, nil
}
// bgpDevices lists the devices whose profile collects BGP and which have peer rows.
func (s *server) bgpDevices(ctx context.Context, only string) ([]bgpDevice, error) {
if s.Profiles == nil {
return nil, nil
}
var out []bgpDevice
for off := int64(0); ; off += store.MaxLimit {
rows, n, err := s.Store.ListDevices(ctx, store.DeviceFilter{Kind: store.KindNetwork}, store.Page{Offset: off, Limit: store.MaxLimit})
if err != nil {
return nil, s.storeErr(err)
}
for _, d := range rows {
if only != "" && d.Slug != only {
continue
}
if bd, ok, err := s.bgpDeviceOf(ctx, d); err != nil {
return nil, s.storeErr(err)
} else if ok {
out = append(out, bd)
}
}
if off+store.MaxLimit >= n {
return out, nil
}
}
}
func (s *server) bgpDeviceOf(ctx context.Context, d store.Device) (bgpDevice, bool, error) {
res := s.Profiles.Get(d.Profile)
if res == nil {
return bgpDevice{}, false, nil
}
m, ok := res.Metric(bgpState)
if !ok || !m.Pollable() {
return bgpDevice{}, false, nil
}
all, err := s.Store.ListComponents(ctx, d.ID)
if err != nil {
return bgpDevice{}, false, err
}
bd := bgpDevice{dev: d, state: m}
for _, c := range all {
if c.TableName == m.TableKey() {
bd.comps = append(bd.comps, c)
}
}
return bd, len(bd.comps) > 0, nil
}
// bgpPeers builds the peers of the devices from the inventory and the hot store.
func (s *server) bgpPeers(ctx context.Context, devs []bgpDevice) ([]BGPPeer, error) {
if len(devs) == 0 {
return []BGPPeer{}, nil
}
slugs := make([]string, len(devs))
for i, d := range devs {
slugs[i] = d.dev.Slug
}
match := `device=~` + quote(regexAlt(slugs))
latest, err := s.queryLatest(ctx, match, []string{bgpState, bgpUptime, bgpPrefixes})
if err != nil {
return nil, err
}
rates, err := s.queryRates(ctx, match, []string{bgpInUpd, bgpOutUpd})
if err != nil {
return nil, err
}
out := []BGPPeer{}
for _, d := range devs {
out = append(out, peersOf(d, latest[d.dev.Slug], rates[d.dev.Slug])...)
}
return out, nil
}
// rateReadings is metric -> component -> per-second rate.
type rateReadings map[string]map[string]float64
// queryRates reads the per-second rate of counters over rateWindow for every device in match.
func (s *server) queryRates(ctx context.Context, match string, names []string) (map[string]rateReadings, error) {
parts := make([]string, len(names))
for i, n := range names {
parts[i] = `label_set(rate(` + n + `{` + match + `}[` + rateWindow + `]), "k", ` + quote(n) + `)`
}
release, err := s.admit(ctx, false)
if err != nil {
return nil, err
}
defer release()
qctx, cancel := context.WithTimeout(ctx, QueryTimeout)
defer cancel()
body, err := s.Querier.Query(qctx, "hot", strings.Join(parts, " or "), s.Clock.Now())
if err != nil {
return nil, s.vmErr(qctx, err)
}
rows, err := parseVM(body)
if err != nil {
return nil, fmt.Errorf("bgp rates: %w", err)
}
out := map[string]rateReadings{}
for _, r := range rows {
if r.Value == nil || !r.Value.OK {
continue
}
dev, k := r.Metric["device"], r.Metric["k"]
if out[dev] == nil {
out[dev] = rateReadings{}
}
if out[dev][k] == nil {
out[dev][k] = map[string]float64{}
}
out[dev][k][r.Metric["component"]] = r.Value.V
}
return out, nil
}
// peersOf builds the peers of one device.
func peersOf(d bgpDevice, rd readings, rates rateReadings) []BGPPeer {
seen := map[string]int{}
for _, c := range d.comps {
seen[componentLabel(c)]++
}
out := make([]BGPPeer, 0, len(d.comps))
for _, c := range d.comps {
peer := componentLabel(c)
series := peer // the poller keeps rows with equal labels apart with the index
if seen[peer] > 1 {
series += "#" + c.Idx
}
out = append(out, buildPeer(d, c, peer, series, rd, rates))
}
return out
}
func componentLabel(c store.Component) string {
if c.Label != "" {
return c.Label
}
return c.Idx
}
func buildPeer(d bgpDevice, c store.Component, peer, series string, rd readings, rates rateReadings) BGPPeer {
var attrs map[string]string
_ = json.Unmarshal([]byte(c.Attrs), &attrs) // attrs are written by discovery; unreadable ones are simply absent
p := BGPPeer{
Device: DeviceRef{Slug: d.dev.Slug, Name: d.dev.Name}, Peer: peer, Component: series, Family: peerFamily(peer),
RemoteAS: parseAS(attrs["remote_as"]), LocalAS: parseAS(attrs["local_as"]), RouterID: attrs["router_id"],
Description: attrs["description"], State: stateUnknown, StateClass: stateUnknown, Prefixes: []BGPFamily{},
}
if r := rd.at(bgpState, series); r != nil {
name := d.state.Enum[int(r.v)]
if name == "" {
name = stateUnknown
}
p.State, p.StateClass, p.Established = name, stateOf(d.state, name), name == stateEstablished
t := r.ts
p.Sampled = &t
}
if r := rd.at(bgpUptime, series); r != nil {
v := r.v
p.UptimeS = &v
}
p.InUpdatesPerS, p.OutUpdatesPerS = rateAt(rates, bgpInUpd, series), rateAt(rates, bgpOutUpd, series)
p.Prefixes = prefixFamilies(rd[bgpPrefixes], series)
return p
}
func rateAt(rates rateReadings, metric, comp string) *float64 {
if v, ok := rates[metric][comp]; ok {
return &v
}
return nil
}
// prefixFamilies reads the "<peer>|<family>" components of the prefix metric that belong to the peer.
func prefixFamilies(by map[string]*reading, series string) []BGPFamily {
out := []BGPFamily{}
for comp, r := range by {
if fam, ok := strings.CutPrefix(comp, series+"|"); ok {
v := r.v
out = append(out, BGPFamily{Family: fam, Accepted: &v})
}
}
slices.SortFunc(out, func(a, b BGPFamily) int { return strings.Compare(a.Family, b.Family) })
return out
}
func peerFamily(peer string) string {
if a, err := netip.ParseAddr(peer); err == nil && a.Is6() {
return "ipv6"
}
return "ipv4"
}
func parseAS(s string) *int64 {
if v, err := strconv.ParseInt(s, 10, 64); err == nil && v >= 0 {
return &v
}
return nil
}
func bgpMatches(p BGPPeer, as *int64, family, q string) bool {
if as != nil && (p.RemoteAS == nil || *p.RemoteAS != *as) {
return false
}
if family != "" && p.Family != family {
return false
}
if q == "" {
return true
}
q = strings.ToLower(q)
for _, f := range []string{p.Peer, p.Description, p.Device.Name, p.Device.Slug, p.RouterID} {
if strings.Contains(strings.ToLower(f), q) {
return true
}
}
return false
}
func bgpStateMatches(p BGPPeer, state string) bool {
switch state {
case "":
return true
case "down":
return !p.Established
}
return p.State == state
}
// sortPeers puts the sessions that are down first, then orders by device and peer.
func sortPeers(ps []BGPPeer) {
slices.SortStableFunc(ps, func(a, b BGPPeer) int {
if a.Established != b.Established {
if a.Established {
return 1
}
return -1
}
if c := strings.Compare(a.Device.Slug, b.Device.Slug); c != 0 {
return c
}
return strings.Compare(a.Peer, b.Peer)
})
}

116
internal/api/bgp_test.go Normal file
View File

@@ -0,0 +1,116 @@
package api_test
import (
"context"
"strings"
"testing"
"time"
"github.com/v10networks/nmon/internal/api"
"github.com/v10networks/nmon/internal/store"
)
// bgpRouter seeds an Arista with an IPv4 and an IPv6 peer (the latter down) and the readings the hot store would hold.
func bgpRouter(t *testing.T, e *env, name, profileName string) *store.Device {
t.Helper()
d := e.device(t, name, "yyz", "router", func(d *store.Device) { d.Profile = profileName })
table := "ARISTA-BGP4V2-MIB::aristaBgp4V2PeerTable"
err := e.st.ReplaceComponents(context.Background(), d.ID, table, []store.Component{
{Idx: "1.1.4.192.0.2.1", Label: "192.0.2.1", Attrs: `{"remote_as":"65001","local_as":"65000","description":"transit-a","router_id":"198.51.100.1"}`},
{Idx: "1.2.16.32.1.13.184.0.0.0.0.0.0.0.0.0.0.0.2", Label: "2001:db8::2", Attrs: `{"remote_as":"65002","local_as":"65000"}`},
})
if err != nil {
t.Fatal(err)
}
return d
}
func bgpBody(dev string) func(string, string, time.Time, time.Time, time.Duration) ([]byte, error) {
return func(_, expr string, _, _ time.Time, _ time.Duration) ([]byte, error) {
if strings.Contains(expr, "rate(") {
return vector(
vrow{lbl(dev, "bgp_peer_in_updates", "192.0.2.1"), 0.25}, vrow{lbl(dev, "bgp_peer_out_updates", "192.0.2.1"), 0.5},
), nil
}
var rows []vrow
rows = append(rows, rd(dev, "bgp_peer_state", "192.0.2.1", 6)...)
rows = append(rows, rd(dev, "bgp_peer_state", "2001:db8::2", 3)...) // active
rows = append(rows, rd(dev, "bgp_peer_uptime", "192.0.2.1", 86400)...)
rows = append(rows, rd(dev, "bgp_prefixes_accepted", "192.0.2.1|ipv4-unicast", 540000)...)
rows = append(rows, rd(dev, "bgp_prefixes_accepted", "192.0.2.1|ipv6-unicast", 12)...)
return vector(rows...), nil
}
}
func TestBGPPeersListsSessionsDownFirstWithAttributesFromTheInventory(t *testing.T) {
e := newEnvDeps(t, 0, withShipped(t))
d := bgpRouter(t, e, "edge1", "arista-eos")
e.device(t, "plain-switch", "yyz", "access") // generic: no BGP, never listed
e.q.fn = bgpBody(d.Slug)
var got api.BGPPeerList
e.getJSON(t, "/api/v1/bgp/peers", &got)
if got.Total != 2 || len(got.Items) != 2 || got.Counts.Total != 2 || got.Counts.Established != 1 || got.Counts.Down != 1 {
t.Fatalf("list = %+v", got)
}
down, up := got.Items[0], got.Items[1]
if down.Peer != "2001:db8::2" || down.Family != "ipv6" || down.State != "active" || down.StateClass != "crit" || down.Established {
t.Errorf("the down IPv6 session must come first: %+v", down)
}
if *down.RemoteAS != 65002 || *down.LocalAS != 65000 || down.Description != "" {
t.Errorf("down attrs = %+v", down)
}
if up.Component != "192.0.2.1" {
t.Errorf("component = %q", up.Component)
}
if up.Peer != "192.0.2.1" || up.State != "established" || up.StateClass != "ok" || !up.Established || up.Description != "transit-a" ||
up.RouterID != "198.51.100.1" || *up.RemoteAS != 65001 || *up.UptimeS != 86400 {
t.Errorf("up = %+v", up)
}
if len(up.Prefixes) != 2 || up.Prefixes[0].Family != "ipv4-unicast" || *up.Prefixes[0].Accepted != 540000 || up.Prefixes[1].Family != "ipv6-unicast" {
t.Errorf("prefixes = %+v", up.Prefixes)
}
if up.InUpdatesPerS == nil || *up.InUpdatesPerS != 0.25 || *up.OutUpdatesPerS != 0.5 || down.InUpdatesPerS != nil {
t.Errorf("update rates = %v %v / %v", up.InUpdatesPerS, up.OutUpdatesPerS, down.InUpdatesPerS)
}
if up.Device.Slug != d.Slug || up.Sampled == nil {
t.Errorf("device/sampled = %+v", up)
}
}
func TestBGPPeersFiltersByStateASFamilyDeviceAndText(t *testing.T) {
e := newEnvDeps(t, 0, withShipped(t))
d := bgpRouter(t, e, "edge1", "arista-eos")
e.q.fn = bgpBody(d.Slug)
for path, want := range map[string]int64{
"?state=down": 1, "?state=established": 1, "?state=active": 1, "?state=idle": 0, "?as=65001": 1, "?as=1": 0,
"?family=ipv6": 1, "?family=ipv4": 1, "?device=" + d.Slug: 2, "?device=nope": 0, "?q=transit": 1, "?q=EDGE1": 2, "?q=2001:db8": 1,
} {
var got api.BGPPeerList
e.getJSON(t, "/api/v1/bgp/peers"+path, &got)
if got.Total != want {
t.Errorf("%s: total %d, want %d", path, got.Total, want)
}
}
var got api.BGPPeerList
e.getJSON(t, "/api/v1/bgp/peers?state=down", &got)
if got.Counts.Total != 2 {
t.Errorf("counts must ignore the state filter: %+v", got.Counts)
}
}
func TestBGPPeersWithNoReadingIsUnknownAndCountsAsDown(t *testing.T) {
e := newEnvDeps(t, 0, withShipped(t))
bgpRouter(t, e, "edge1", "juniper-junos") // profile with a different table key: no components match
var got api.BGPPeerList
e.getJSON(t, "/api/v1/bgp/peers", &got)
if got.Total != 0 || got.Items == nil {
t.Errorf("a router whose rows are in another table lists nothing: %+v", got)
}
e2 := newEnvDeps(t, 0, withShipped(t))
bgpRouter(t, e2, "edge2", "arista-eos") // store has no readings at all
e2.getJSON(t, "/api/v1/bgp/peers", &got)
if got.Total != 2 || got.Items[0].State != "unknown" || got.Counts.Down != 2 {
t.Errorf("no readings: %+v", got)
}
}

52
internal/api/bgp_types.go Normal file
View File

@@ -0,0 +1,52 @@
package api
import "time"
// BGPFamily is the prefixes a peer has accepted in one address family.
type BGPFamily struct {
Family string `json:"family" example:"ipv6-unicast" doc:"Address family and subsequent family, for example ipv4-unicast."`
Accepted *float64 `json:"accepted" nullable:"true" doc:"Prefixes accepted from the peer; null when unread."`
}
// BGPPeer is one BGP session of one device.
type BGPPeer struct {
Device DeviceRef `json:"device"`
Peer string `json:"peer" doc:"The peer address, canonical; IPv6 compressed."`
// Component is the label of the peer's metric series: the peer address, with #<index> added when two rows of
// the device share an address (two routing instances). Pass it to /devices/{slug}/metrics/graph.
Component string `json:"component"`
// Family is the address family of the peer address itself: ipv4 or ipv6.
Family string `json:"family" enum:"ipv4,ipv6"`
RemoteAS *int64 `json:"remote_as" nullable:"true"`
LocalAS *int64 `json:"local_as" nullable:"true"`
RouterID string `json:"router_id" doc:"The peer's BGP identifier, when the device reports it."`
Description string `json:"description" doc:"The configured neighbor description, where the device's MIB carries it (Arista)."`
// State is the BGP FSM state: idle, connect, active, opensent, openconfirm or established; "unknown" without a reading.
State string `json:"state"`
StateClass string `json:"state_class" enum:"ok,warn,crit,unknown"`
Established bool `json:"established"`
// UptimeS is how long the session has been established, in seconds.
UptimeS *float64 `json:"uptime_s" nullable:"true"`
Prefixes []BGPFamily `json:"prefixes" doc:"Accepted prefixes per address family, where the device reports them."`
InUpdatesPerS *float64 `json:"in_updates_per_s" nullable:"true" doc:"UPDATE messages received per second over the last 15 minutes."`
OutUpdatesPerS *float64 `json:"out_updates_per_s" nullable:"true"`
// Sampled is when the state was last read.
Sampled *time.Time `json:"sampled" nullable:"true"`
}
// BGPCounts are the session counts of a listing before the state filter.
type BGPCounts struct {
Total int64 `json:"total"`
Established int64 `json:"established"`
Down int64 `json:"down" doc:"Sessions not established, including those with no reading."`
}
// BGPPeerList is a page of BGP peers, sessions that are down first.
type BGPPeerList struct {
Items []BGPPeer `json:"items"`
Page int64 `json:"page"`
PerPage int64 `json:"per_page"`
Total int64 `json:"total"`
// Counts cover the device, AS and q filters but not the state filter, for the page's summary line.
Counts BGPCounts `json:"counts"`
}

View File

@@ -0,0 +1,344 @@
package api
import (
"context"
"encoding/json"
"fmt"
"math"
"net/http"
"sort"
"strconv"
"strings"
"time"
"github.com/danielgtaylor/huma/v2"
"github.com/v10networks/nmon/internal/profile"
)
// latestWindow is how far back the latest reading of a metric may be; the slowest profile interval is 15 minutes.
const latestWindow = "1h"
var categoryLabels = map[string]string{
"system": "System", "health": "Health", "power": "Power", "environment": "Environment", "wireless": "Wireless",
"printer": "Printer", "routing": "Routing", "optics": "Optics", "firewall": "Firewall",
}
func (s *server) registerDeviceMetrics(api huma.API) {
huma.Register(api, huma.Operation{
OperationID: "get-device-metrics", Method: http.MethodGet, Path: "/api/v1/devices/{slug}/metrics", Tags: []string{"devices"},
Summary: "Latest non-interface metrics of a device, by category",
Description: "Driven by the device's profile: each pollable metric names a category, a label and a display. Values come " +
"from the hot store in one batched query. max is set for metrics that are a fraction of another (profile of), percent " +
"is computed here, sentinel values (a printer's -3) come back as text, and enum metrics carry a state. Categories " +
"with no data are left out.",
Errors: []int{http.StatusNotFound, http.StatusTooManyRequests, http.StatusServiceUnavailable},
}, s.deviceMetrics)
huma.Register(api, huma.Operation{
OperationID: "get-device-metric-graph", Method: http.MethodGet, Path: "/api/v1/devices/{slug}/metrics/graph", Tags: []string{"graph"},
Summary: "One metric component over time",
Description: "Gauges raw, counters as a per-second rate, percent displays as percent. The step rules are those of /graph; " +
"only the hot store's retention is searched.",
Errors: []int{http.StatusNotFound, http.StatusUnprocessableEntity, http.StatusTooManyRequests, http.StatusServiceUnavailable},
}, s.metricGraph)
}
// reading is the latest sample of one metric component.
type reading struct {
v float64
ts time.Time
hasTS bool
}
// readings is metric -> component -> latest sample.
type readings map[string]map[string]*reading
func (r readings) at(metric, comp string) *reading { return r[metric][comp] }
// listedMetrics are the metrics a device's tabs can show: polled, not interface metrics, with a category.
func listedMetrics(res *profile.Resolved) []profile.Metric {
var out []profile.Metric
for _, m := range res.Metrics {
if m.NeedsCategory() && m.Category != "" {
out = append(out, m)
}
}
return out
}
// latestExpr is one query yielding, for every metric, its last value (k=name) and the time of that
// sample (k=@name) for the device.
func latestExpr(match string, names []string) string {
parts := make([]string, 0, 2*len(names))
for _, n := range names {
sel := n + `{` + match + `}[` + latestWindow + `]`
parts = append(parts,
`label_set(last_over_time(`+sel+`), "k", `+quote(n)+`)`,
`label_set(tlast_over_time(`+sel+`), "k", `+quote("@"+n)+`)`)
}
return strings.Join(parts, " or ")
}
// queryLatest runs the batched query and indexes the result by device.
func (s *server) queryLatest(ctx context.Context, match string, names []string) (map[string]readings, error) {
if len(names) == 0 {
return map[string]readings{}, nil
}
release, err := s.admit(ctx, false)
if err != nil {
return nil, err
}
defer release()
qctx, cancel := context.WithTimeout(ctx, QueryTimeout)
defer cancel()
body, err := s.Querier.Query(qctx, "hot", latestExpr(match, names), s.Clock.Now())
if err != nil {
return nil, s.vmErr(qctx, err)
}
rows, err := parseVM(body)
if err != nil {
return nil, fmt.Errorf("metrics: %w", err)
}
return indexReadings(rows), nil
}
// indexReadings sorts result rows by device, metric and component.
func indexReadings(rows []vmSeries) map[string]readings {
all := map[string]readings{}
for _, r := range rows {
out := all[r.Metric["device"]]
if out == nil {
out = readings{}
all[r.Metric["device"]] = out
}
if r.Value == nil || !r.Value.OK {
continue
}
k := r.Metric["k"]
isTS := strings.HasPrefix(k, "@")
name := strings.TrimPrefix(k, "@")
if out[name] == nil {
out[name] = map[string]*reading{}
}
rd := out[name][r.Metric["component"]]
if rd == nil {
rd = &reading{}
out[name][r.Metric["component"]] = rd
}
if isTS {
rd.ts, rd.hasTS = time.Unix(int64(r.Value.V), 0).UTC(), true
} else {
rd.v = r.Value.V
if !rd.hasTS {
rd.ts = time.Unix(r.Value.TS, 0).UTC()
}
}
}
return all
}
func (s *server) deviceMetrics(ctx context.Context, in *struct {
Slug string `path:"slug" doc:"Device slug."`
}) (*struct{ Body MetricsResponse }, error) {
d, err := s.Store.GetDeviceBySlug(ctx, in.Slug)
if err != nil {
return nil, s.storeErr(err)
}
if d.PurgedAt != nil {
return nil, huma.Error404NotFound("not found")
}
out := MetricsResponse{Categories: []MetricCategory{}}
if s.Profiles == nil {
return &struct{ Body MetricsResponse }{out}, nil
}
res := s.Profiles.Get(d.Profile)
if res == nil {
return &struct{ Body MetricsResponse }{out}, nil
}
ms := listedMetrics(res)
names := make([]string, len(ms))
for i, m := range ms {
names[i] = m.Name
}
all, err := s.queryLatest(ctx, `device=`+quote(d.Slug), names)
if err != nil {
return nil, err
}
attrs, err := s.componentAttrs(ctx, d.ID)
if err != nil {
return nil, s.storeErr(err)
}
b := &metricsBuilder{res: res, rd: all[d.Slug], attrs: attrs}
out.Categories = b.categories(ms)
return &struct{ Body MetricsResponse }{out}, nil
}
// componentAttrs loads the discovery attributes of a device's rows, keyed by table and label.
func (s *server) componentAttrs(ctx context.Context, deviceID int64) (map[string]map[string]string, error) {
rows, err := s.Store.ListComponents(ctx, deviceID)
if err != nil {
return nil, err
}
out := map[string]map[string]string{}
for _, c := range rows {
if c.Attrs == "" || c.Attrs == "{}" {
continue
}
var m map[string]string
if json.Unmarshal([]byte(c.Attrs), &m) == nil && len(m) > 0 {
out[attrKey(c.TableName, c.Label)] = m
}
}
return out, nil
}
func attrKey(table, label string) string { return table + "\x00" + label }
// metricsBuilder turns readings into the response, using the profile's presentation fields.
type metricsBuilder struct {
res *profile.Resolved
rd readings
attrs map[string]map[string]string
}
func (b *metricsBuilder) categories(ms []profile.Metric) []MetricCategory {
byCat := map[string][]MetricGroup{}
for _, m := range ms {
if b.res.Referenced(m.Name) {
continue // a denominator is shown through the metric that refers to it
}
if g, ok := b.group(m); ok {
byCat[m.Category] = append(byCat[m.Category], g)
}
}
out := []MetricCategory{}
for _, c := range profile.Categories {
if gs := byCat[c]; len(gs) > 0 {
out = append(out, MetricCategory{Name: c, Label: categoryLabels[c], Groups: gs})
}
}
return out
}
func (b *metricsBuilder) group(m profile.Metric) (MetricGroup, bool) {
g := MetricGroup{Metric: m.Name, Label: m.Title(), Unit: b.unitOf(m), Display: m.EffectiveDisplay(), Components: []MetricComponent{}}
for comp, rd := range b.rd[m.Name] {
g.Components = append(g.Components, b.component(m, comp, rd))
}
sort.Slice(g.Components, func(i, j int) bool { return g.Components[i].Component < g.Components[j].Component })
return g, len(g.Components) > 0
}
// unitOf is the metric's unit, or the scale_by metric's when the value is multiplied by it.
func (b *metricsBuilder) unitOf(m profile.Metric) string {
if m.ScaleBy != "" {
if sb, ok := b.res.Metric(m.ScaleBy); ok {
return sb.Unit
}
}
return m.Unit
}
// scaled is the metric's value at comp after scale_by.
func (b *metricsBuilder) scaled(m profile.Metric, comp string) (float64, bool) {
rd := b.rd.at(m.Name, comp)
if rd == nil {
return 0, false
}
if m.ScaleBy == "" {
return rd.v, true
}
if f := b.rd.at(m.ScaleBy, comp); f != nil {
return rd.v * f.v, true
}
return 0, false
}
func (b *metricsBuilder) component(m profile.Metric, comp string, rd *reading) MetricComponent {
c := MetricComponent{Component: comp, TS: rd.ts}
if words, ok := sentinelText(m, rd.v); ok {
c.Text = &words
} else if v, ok := b.scaled(m, comp); ok && !math.IsNaN(v) {
c.Value = &v
}
b.fillMax(&c, m)
b.fillState(&c, m, rd.v)
if m.EffectiveDisplay() == "text" && c.Text == nil && c.Value != nil {
t := strconv.FormatFloat(*c.Value, 'f', -1, 64)
c.Text = &t
}
c.Attrs = b.attrs[attrKey(m.TableKey(), comp)]
return c
}
// sentinelText is the profile's words for a raw value like -3.
func sentinelText(m profile.Metric, v float64) (string, bool) {
if v != math.Trunc(v) {
return "", false
}
t, ok := m.Sentinels[int(v)]
return t, ok
}
// fillMax sets max and percent for a metric that is a fraction of another.
func (b *metricsBuilder) fillMax(c *MetricComponent, m profile.Metric) {
if m.Of == "" {
return
}
om, ok := b.res.Metric(m.Of)
if !ok {
return
}
mx, ok := b.scaled(om, c.Component)
if !ok {
return
}
c.Max = &mx
if c.Value != nil && mx > 0 && *c.Value >= 0 {
p := 100 * *c.Value / mx
c.Percent = &p
}
}
func (b *metricsBuilder) fillState(c *MetricComponent, m profile.Metric, raw float64) {
if m.Type != profile.Status {
return
}
name := m.Enum[int(raw)]
st := stateOf(m, name)
c.State = &st
}
var (
critWords = []string{"crit", "fail", "fault", "depleted", "error", "down"}
warnWords = []string{"warn", "low", "degrad", "bypass", "test"}
unknownWords = []string{"unknown", "other", "notpresent"}
)
// stateOf maps an enum name to ok, warn, crit or unknown: the profile's enum_state, else by name.
func stateOf(m profile.Metric, name string) string {
if st, ok := m.EnumState[name]; ok {
return st
}
l := strings.ToLower(name)
switch {
case name == "":
return "unknown"
case containsAny(l, unknownWords):
return "unknown"
case containsAny(l, critWords):
return "crit"
case containsAny(l, warnWords):
return "warn"
}
return "ok"
}
func containsAny(s string, words []string) bool {
for _, w := range words {
if strings.Contains(s, w) {
return true
}
}
return false
}

View File

@@ -0,0 +1,167 @@
package api
import (
"context"
"time"
"github.com/danielgtaylor/huma/v2"
"github.com/v10networks/nmon/internal/profile"
)
type metricGraphIn struct {
Slug string `path:"slug" doc:"Device slug."`
Metric string `query:"metric" required:"true" minLength:"1" doc:"Profile metric name, as /metrics lists it."`
Component string `query:"component" doc:"Row label; empty for a device-wide scalar."`
From time.Time `query:"from" doc:"Start, RFC 3339. Default one hour before to."`
To time.Time `query:"to" doc:"End, RFC 3339. Default now."`
Step int64 `query:"step" minimum:"1" doc:"Step in seconds. Default max(60, range/1000) rounded up to whole minutes."`
Aggregate string `query:"aggregate" enum:"avg,sum,max" doc:"Without component: one series over all components."`
}
// allComponents, as the component of metricSel, selects every component of the metric.
const allComponents = "\x00all"
// metricSel is the selector of one component of a metric.
func metricSel(name, slug, comp string) string {
if comp == allComponents {
return name + `{device=` + quote(slug) + `}`
}
return name + `{device=` + quote(slug) + `,component=` + quote(comp) + `}`
}
// valueExpr is the metric's value at one component with scale_by applied. A metric with sentinels
// drops its negative values, which are words rather than numbers.
func valueExpr(m profile.Metric, slug, comp string) string {
e := metricSel(m.Name, slug, comp)
if len(m.Sentinels) > 0 {
e = `(` + e + ` >= 0)`
}
if m.ScaleBy != "" {
e = `(` + e + ` * on(device, component) ` + metricSel(m.ScaleBy, slug, comp) + `)`
}
return e
}
// graphExprs returns the series expression and, for a metric with a capacity that is not drawn as
// percent, the capacity expression.
func graphExprs(res *profile.Resolved, m profile.Metric, slug, comp string, step time.Duration) (expr, maxExpr string) {
display := m.EffectiveDisplay()
val := valueExpr(m, slug, comp)
counter := m.Type == profile.Counter32 || m.Type == profile.Counter64
if counter && display == "rate" {
val = `rate(` + metricSel(m.Name, slug, comp) + `[` + secs(max(step, minRateWindow)) + `])`
}
om, hasOf := res.Metric(m.Of)
if !hasOf {
return val, ""
}
den := valueExpr(om, slug, comp)
if display == "percent" {
return `100 * ` + val + ` / on(device, component) (` + den + ` > 0)`, ""
}
return val, den
}
func (s *server) metricGraph(ctx context.Context, in *metricGraphIn) (*struct{ Body MetricGraphResponse }, error) {
d, err := s.Store.GetDeviceBySlug(ctx, in.Slug)
if err != nil {
return nil, s.storeErr(err)
}
if d.PurgedAt != nil || s.Profiles == nil {
return nil, huma.Error404NotFound("not found")
}
res := s.Profiles.Get(d.Profile)
m, ok := metricOf(res, in.Metric)
if !ok {
return nil, huma.Error404NotFound("no such metric in the device's profile")
}
plan, err := s.metricPlan(in)
if err != nil {
return nil, err
}
step := plan.newest().step
comp := in.Component
if in.Aggregate != "" && comp == "" {
comp = allComponents
}
expr, maxExpr := graphExprs(res, m, d.Slug, comp, step)
if comp == allComponents {
expr, maxExpr = in.Aggregate+`(`+expr+`)`, ""
}
resp := MetricGraphResponse{
Unit: unitOfMetric(res, m), Display: m.EffectiveDisplay(), StepSeconds: int64(step / time.Second), Series: []Point{},
}
grid := gridOf(plan)
series, err := s.fetchMetricSeries(ctx, expr, plan)
if err != nil {
return nil, err
}
resp.Series = pointsOf(grid, series)
if maxExpr != "" {
mx, err := s.fetchMetricSeries(ctx, maxExpr, plan)
if err != nil {
return nil, err
}
resp.MaxSeries = pointsOf(grid, mx)
}
return &struct{ Body MetricGraphResponse }{resp}, nil
}
func metricOf(res *profile.Resolved, name string) (profile.Metric, bool) {
if res == nil {
return profile.Metric{}, false
}
m, ok := res.Metric(name)
return m, ok && m.NeedsCategory()
}
func unitOfMetric(res *profile.Resolved, m profile.Metric) string {
return (&metricsBuilder{res: res}).unitOf(m)
}
// metricPlan applies the /graph range defaults and step rules, limited to the hot store.
func (s *server) metricPlan(in *metricGraphIn) (*graphPlan, error) {
to := in.To
if to.IsZero() {
to = s.Clock.Now()
}
from := in.From
if from.IsZero() {
from = to.Add(-time.Hour)
}
hotStart := s.Clock.Now().Add(-s.HotRetention)
from = from.UTC()
if from.Before(hotStart) {
from = hotStart
}
if !from.Before(to) {
return nil, huma.Error422UnprocessableEntity("from must be before to, within the hot retention")
}
plan, err := planGraph(from, to.UTC(), hotStart, time.Duration(in.Step)*time.Second)
if err != nil {
return nil, huma.Error422UnprocessableEntity(err.Error())
}
return plan, nil
}
// fetchMetricSeries runs the expression over the plan's hot windows and returns ts -> value.
func (s *server) fetchMetricSeries(ctx context.Context, expr string, p *graphPlan) (map[int64]float64, error) {
release, err := s.admit(ctx, false)
if err != nil {
return nil, err
}
defer release()
qctx, cancel := context.WithTimeout(ctx, QueryTimeout)
defer cancel()
out := map[int64]float64{}
for _, w := range p.tiers {
if w.tier != tierRaw {
continue // only the hot store is searched
}
if err := s.fetchOne(qctx, "hot", expr, w, func(ts int64, v float64) { out[ts] = v }); err != nil {
return nil, err
}
}
return out, nil
}

View File

@@ -0,0 +1,208 @@
package api_test
import (
"strings"
"testing"
"github.com/v10networks/nmon/internal/api"
"github.com/v10networks/nmon/internal/profile"
"github.com/v10networks/nmon/internal/store"
)
func withShipped(t *testing.T) func(*api.Deps) {
t.Helper()
set, err := profile.Load("")
if err != nil {
t.Fatal(err)
}
return func(d *api.Deps) { d.Profiles = set }
}
func lbl(dev, k, comp string) map[string]string {
m := map[string]string{"device": dev, "k": k}
if comp != "" {
m["component"] = comp
}
return m
}
// rd is a value row plus its timestamp row.
func rd(dev, metric, comp string, v float64) []vrow {
return []vrow{{lbl(dev, metric, comp), v}, {lbl(dev, "@"+metric, comp), 1_790_000_000}}
}
func TestDeviceMetricsByCategory(t *testing.T) {
e := newEnvDeps(t, 0, withShipped(t))
d := e.device(t, "srv1", "yyz", "server")
var rows []vrow
rows = append(rows, rd(d.Slug, "hrProcessorLoad", "CPU 0", 12)...)
rows = append(rows, rd(d.Slug, "hrStorageUsed", "/", 50)...)
rows = append(rows, rd(d.Slug, "hrStorageSize", "/", 100)...)
rows = append(rows, rd(d.Slug, "hrStorageAllocationUnits", "/", 1024)...)
e.q.body = vector(rows...)
var got api.MetricsResponse
e.getJSON(t, "/api/v1/devices/"+d.Slug+"/metrics", &got)
if len(got.Categories) != 1 || got.Categories[0].Name != "health" || got.Categories[0].Label != "Health" {
t.Fatalf("categories = %+v", got.Categories)
}
groups := map[string]api.MetricGroup{}
for _, g := range got.Categories[0].Groups {
groups[g.Metric] = g
}
if _, ok := groups["hrStorageSize"]; ok {
t.Error("a denominator is listed on its own")
}
cpu := groups["hrProcessorLoad"]
if cpu.Unit != "percent" || cpu.Display != "gauge" || len(cpu.Components) != 1 || *cpu.Components[0].Value != 12 {
t.Errorf("cpu = %+v", cpu)
}
st := groups["hrStorageUsed"]
c := st.Components[0]
if st.Unit != "bytes" || st.Display != "percent" || *c.Value != 51200 || *c.Max != 102400 || *c.Percent != 50 {
t.Errorf("storage = %+v %+v", st, c)
}
if c.TS.Unix() != 1_790_000_000 {
t.Errorf("ts = %v", c.TS)
}
qs := e.q.queries()
if len(qs) != 1 || qs[0].store != "hot" || !strings.Contains(qs[0].expr, `device="`+d.Slug+`"`) {
t.Errorf("queries = %+v", qs)
}
}
func TestDeviceMetricsEmptyAndUnknown(t *testing.T) {
e := newEnvDeps(t, 0, withShipped(t))
d := e.device(t, "srv1", "yyz", "server")
var got api.MetricsResponse
e.getJSON(t, "/api/v1/devices/"+d.Slug+"/metrics", &got)
if got.Categories == nil || len(got.Categories) != 0 {
t.Errorf("categories = %#v, want empty list", got.Categories)
}
isProblem(t, e.do("GET", "/api/v1/devices/nope/metrics", ""), 404)
}
func printerRows(dev string) []vrow {
var rows []vrow
rows = append(rows, rd(dev, "prtMarkerSuppliesLevel", "Black", 8)...)
rows = append(rows, rd(dev, "prtMarkerSuppliesMaxCapacity", "Black", 100)...)
rows = append(rows, rd(dev, "prtMarkerSuppliesLevel", "Cyan", -3)...)
rows = append(rows, rd(dev, "prtMarkerSuppliesMaxCapacity", "Cyan", 100)...)
rows = append(rows, rd(dev, "prtMarkerSuppliesLevel", "Magenta", -2)...)
rows = append(rows, rd(dev, "prtAlertSeverityLevel", "1", 3)...)
rows = append(rows, rd(dev, "prtMarkerLifeCount", "1", 4321)...)
return rows
}
func TestPrinterMetricsSentinelsAndState(t *testing.T) {
e := newEnvDeps(t, 0, withShipped(t))
d := e.device(t, "prn1", "yyz", "printer", func(d *store.Device) { d.Profile = "printer" })
e.q.body = vector(printerRows(d.Slug)...)
var got api.MetricsResponse
e.getJSON(t, "/api/v1/devices/"+d.Slug+"/metrics", &got)
if len(got.Categories) != 1 || got.Categories[0].Name != "printer" {
t.Fatalf("categories = %+v", got.Categories)
}
by := map[string]map[string]api.MetricComponent{}
for _, g := range got.Categories[0].Groups {
by[g.Metric] = map[string]api.MetricComponent{}
for _, c := range g.Components {
by[g.Metric][c.Component] = c
}
}
lv := by["prtMarkerSuppliesLevel"]
if p := lv["Black"].Percent; p == nil || *p != 8 {
t.Errorf("black = %+v", lv["Black"])
}
if c := lv["Cyan"]; c.Value != nil || c.Text == nil || *c.Text != "some remaining" || c.Percent != nil {
t.Errorf("cyan = %+v", c)
}
if c := lv["Magenta"]; c.Text == nil || *c.Text != "unknown" {
t.Errorf("magenta = %+v", c)
}
if s := by["prtAlertSeverityLevel"]["1"].State; s == nil || *s != "crit" {
t.Errorf("alert state = %v", s)
}
if _, listed := by["prtMarkerSuppliesMaxCapacity"]; listed {
t.Error("capacity listed on its own")
}
if v := by["prtMarkerLifeCount"]["1"].Value; v == nil || *v != 4321 {
t.Errorf("pages = %v", v)
}
}
func TestPrinterFleetSortedByLowestSupply(t *testing.T) {
e := newEnvDeps(t, 0, withShipped(t))
asPrinter := func(d *store.Device) { d.Profile = "printer" }
low := e.device(t, "lowprn", "yyz", "printer", asPrinter)
full := e.device(t, "fullprn", "yyz", "printer", asPrinter)
e.device(t, "a-switch", "yyz", "switch")
var rows []vrow
rows = append(rows, printerRows(low.Slug)...)
rows = append(rows, rd(full.Slug, "prtMarkerSuppliesLevel", "Black", 90)...)
rows = append(rows, rd(full.Slug, "prtMarkerSuppliesMaxCapacity", "Black", 100)...)
e.q.body = vector(rows...)
var got api.PrinterList
e.getJSON(t, "/api/v1/printers", &got)
if got.Total != 2 || len(got.Items) != 2 || got.Items[0].Device.Slug != low.Slug {
t.Fatalf("fleet = %+v", got)
}
sup := got.Items[0].Supplies
if len(sup) != 3 || sup[0].Component != "Black" || sup[0].State != "warn" || *sup[0].Percent != 8 {
t.Errorf("supplies = %+v", sup)
}
if sup[1].Component != "Cyan" || *sup[1].Text != "some remaining" || sup[1].State != "ok" {
t.Errorf("cyan = %+v", sup[1])
}
if sup[2].State != "unknown" {
t.Errorf("magenta = %+v", sup[2])
}
if p := got.Items[0].PagesTotal; p == nil || *p != 4321 {
t.Errorf("pages = %v", p)
}
if got.Items[1].Supplies[0].State != "ok" || got.Items[0].Reachability == "" {
t.Errorf("second = %+v", got.Items[1])
}
n := 0
for _, q := range e.q.queries() {
if strings.Contains(q.expr, "prtMarkerSuppliesLevel") {
n++
}
}
if n != 1 {
t.Errorf("%d supply queries for the fleet, want 1", n)
}
}
func TestMetricGraphExpressions(t *testing.T) {
e := newEnvDeps(t, 0, withShipped(t))
d := e.device(t, "srv1", "yyz", "server")
now := e.clk.Now().Unix()
e.q.body = matrix(mrow{lbl(d.Slug, "", "/"), map[int64]float64{now - 600: 25, now - 540: 30}})
var got api.MetricGraphResponse
path := "/api/v1/devices/" + d.Slug + "/metrics/graph?metric=hrStorageUsed&component=%2F"
e.getJSON(t, path, &got)
expr := e.q.q
for _, want := range []string{`hrStorageUsed{device="` + d.Slug + `",component="/"}`, `hrStorageSize{`, `hrStorageAllocationUnits{`, `100 *`} {
if !strings.Contains(expr, want) {
t.Errorf("expr %q lacks %q", expr, want)
}
}
if got.Display != "percent" || got.Unit != "bytes" || got.StepSeconds != 60 || len(got.Series) == 0 {
t.Errorf("graph = %+v", got)
}
var withValue int
for _, p := range got.Series {
if p.V != nil {
withValue++
}
}
if withValue != 2 {
t.Errorf("%d points with values, want 2", withValue)
}
// A counter is a rate; a gauge is raw.
e.getJSON(t, "/api/v1/devices/"+d.Slug+"/metrics/graph?metric=hrProcessorLoad&component=CPU+0", &got)
if strings.Contains(e.q.q, "rate(") || !strings.Contains(e.q.q, `hrProcessorLoad{`) {
t.Errorf("gauge expr %q", e.q.q)
}
isProblem(t, e.do("GET", "/api/v1/devices/"+d.Slug+"/metrics/graph?metric=nope", ""), 404)
}

View File

@@ -0,0 +1,79 @@
package api
import "time"
// MetricsResponse is a device's latest non-interface metrics, grouped into the tabs of the device page.
type MetricsResponse struct {
Categories []MetricCategory `json:"categories" doc:"Only categories with data, in the fixed tab order (system first, firewall last)."`
}
// MetricCategory is one tab: the metrics of one category.
type MetricCategory struct {
Name string `json:"name" enum:"system,health,power,environment,wireless,printer,routing,optics,firewall"`
Label string `json:"label"`
Groups []MetricGroup `json:"groups" doc:"One group per metric, in profile order; components are its rows (cores, filesystems, supplies)."`
}
// MetricGroup is one metric with its latest value per component.
type MetricGroup struct {
Metric string `json:"metric" doc:"The profile metric name; pass it to /metrics/graph."`
Label string `json:"label" doc:"The profile's human name for the metric."`
Unit string `json:"unit" doc:"Unit of value and max. With scale_by this is the scaled unit (bytes)."`
Display string `json:"display" enum:"gauge,percent,state,rate,text" doc:"Which renderer to use."`
Components []MetricComponent `json:"components"`
}
// MetricComponent is the latest reading of one row of a metric.
type MetricComponent struct {
Component string `json:"component" doc:"The row label; empty for a device-wide scalar."`
Value *float64 `json:"value" doc:"Value after the profile's scale and scale_by; null for a sentinel or no reading."`
Max *float64 `json:"max" doc:"The capacity this value is a fraction of (profile of), same unit; null otherwise."`
Percent *float64 `json:"percent,omitempty" doc:"100 * value / max, computed at query time, for display percent."`
Text *string `json:"text" doc:"A sentinel's words (for example some remaining), or the number as text for display text."`
State *string `json:"state" enum:"ok,warn,crit,unknown" doc:"For enum (status) metrics."`
TS time.Time `json:"ts" doc:"When the reading was taken."`
Attrs map[string]string `json:"attrs,omitempty" doc:"Discovery attributes of the row (a printer supply's type and color)."`
}
// MetricGraphResponse is a single-series graph of one metric component.
type MetricGraphResponse struct {
Unit string `json:"unit"`
Display string `json:"display" enum:"gauge,percent,state,rate,text"`
StepSeconds int64 `json:"step_seconds"`
Series []Point `json:"series" doc:"[unix_s, value|null] pairs. Counters are a per-second rate; percent displays are percent."`
MaxSeries []Point `json:"max_series,omitempty" doc:"The capacity over time, when the metric has one and the display is not percent."`
}
// PrinterSupply is one supply or paper tray in the fleet view.
type PrinterSupply struct {
Component string `json:"component"`
Type string `json:"type,omitempty" doc:"Supply type (toner, drum, fuser, ...), from discovery."`
Color string `json:"color,omitempty" doc:"Colorant as the printer names it (black, cyan, ...), from discovery."`
Percent *float64 `json:"percent" doc:"Level as a percent of capacity; null for a sentinel or no reading."`
Text *string `json:"text" doc:"A sentinel's words, for example some remaining."`
State string `json:"state" enum:"ok,warn,crit,unknown"`
}
// PrinterRow is one printer of the fleet view.
type PrinterRow struct {
Device DeviceRefSite `json:"device"`
Reachability string `json:"reachability" enum:"up,down,warning,unknown,maintenance"`
PagesTotal *float64 `json:"pages_total" doc:"Lifetime page count; null when unread."`
Supplies []PrinterSupply `json:"supplies"`
Trays []PrinterSupply `json:"trays"`
}
// DeviceRefSite is a device reference with its site.
type DeviceRefSite struct {
Slug string `json:"slug"`
Name string `json:"name"`
Site string `json:"site"`
}
// PrinterList is a page of printers, the one with the lowest supply first.
type PrinterList struct {
Items []PrinterRow `json:"items"`
Page int64 `json:"page"`
PerPage int64 `json:"per_page"`
Total int64 `json:"total"`
}

View File

@@ -19,6 +19,7 @@ type PutDeviceBody struct {
Enabled *bool `json:"enabled,omitempty"`
Parents []string `json:"parents,omitempty" doc:"Parent device slugs, replacing the current parents."`
SNMP *SNMPBody `json:"snmp,omitempty" doc:"Replaces the SNMP credential; secrets are write-only."`
Profile *string `json:"profile,omitempty" doc:"A profile name pins it; an empty string returns the device to automatic matching. A change triggers one discovery."` //nolint:lll // struct tag text is the OpenAPI doc; cannot wrap
}
func isNotFound(err error) bool { return errors.Is(err, store.ErrNotFound) }
@@ -45,6 +46,9 @@ func (s *server) putDevice(ctx context.Context, in *struct {
return nil, s.storeErr(err)
}
s.inventoryChanged(ctx)
if plan.rediscover {
s.rediscoverAfterEdit(ctx, dev.Slug)
}
return s.view(ctx, dev.Slug)
}
@@ -56,6 +60,7 @@ type editPlan struct {
parents []*store.Device
cred *store.SNMPCredential
change map[string]any // what the audit row records; never a secret
rediscover bool // the profile or its pin changed: run one discovery after the write
}
// planEdit validates body against the device without writing anything.
@@ -85,6 +90,11 @@ func (s *server) planEdit(ctx context.Context, dev *store.Device, b PutDeviceBod
p.setParents, p.parents = true, parents
p.change["parents"] = b.Parents
}
if b.Profile != nil {
if err := s.planProfile(p, dev, *b.Profile); err != nil {
return nil, err
}
}
if b.SNMP != nil {
if dev.Kind != store.KindNetwork {
return nil, huma.Error422UnprocessableEntity("snmp applies to network devices only")

View File

@@ -49,34 +49,37 @@ type InterfaceView struct {
OutBPS *float64 `json:"out_bps" nullable:"true"`
InErrorsPS *float64 `json:"in_errors_ps" nullable:"true" doc:"Inbound errors per second over 5 minutes."`
OutErrorsPS *float64 `json:"out_errors_ps" nullable:"true"`
AlertDownState
}
// DeviceView is a device as the API shows it. It carries no numeric ID and no credential.
type DeviceView struct {
Slug string `json:"slug" doc:"Immutable public key."`
Name string `json:"name"`
Kind string `json:"kind"`
Site string `json:"site"`
Role string `json:"role"`
Profile string `json:"profile"`
Enabled bool `json:"enabled"`
MgmtIP string `json:"mgmt_ip"`
Addresses []string `json:"addresses"`
SNMPVersion string `json:"snmp_version,omitempty"`
SysName string `json:"sys_name"`
SysObjectID string `json:"sys_object_id"`
SysDescr string `json:"sys_descr"`
HWSerial string `json:"hw_serial"`
UptimeSeconds *float64 `json:"uptime_seconds" nullable:"true" doc:"sysUpTime from the hot store; null when unknown."`
Reachability string `json:"reachability" enum:"up,down,warning,unknown,maintenance"`
PollState string `json:"poll_state" enum:"ok,degraded,backoff"`
Warnings []string `json:"warnings"`
LastSeen *time.Time `json:"last_seen" nullable:"true"`
CreatedAt time.Time `json:"created_at"`
RetiredAt *time.Time `json:"retired_at,omitempty" nullable:"true"`
Parents []DeviceRef `json:"parents"`
Groups []GroupView `json:"groups"`
Interfaces []InterfaceView `json:"interfaces"`
Slug string `json:"slug" doc:"Immutable public key."`
Name string `json:"name"`
Kind string `json:"kind"`
Site string `json:"site"`
Role string `json:"role"`
Profile string `json:"profile"`
ProfilePinned bool `json:"profile_pinned" doc:"An operator chose the profile; discovery keeps it."`
ProfileMatch *ProfileMatchView `json:"profile_match,omitempty"`
Enabled bool `json:"enabled"`
MgmtIP string `json:"mgmt_ip"`
Addresses []string `json:"addresses"`
SNMPVersion string `json:"snmp_version,omitempty"`
SysName string `json:"sys_name"`
SysObjectID string `json:"sys_object_id"`
SysDescr string `json:"sys_descr"`
HWSerial string `json:"hw_serial"`
UptimeSeconds *float64 `json:"uptime_seconds" nullable:"true" doc:"sysUpTime from the hot store; null when unknown."`
Reachability string `json:"reachability" enum:"up,down,warning,unknown,maintenance"`
PollState string `json:"poll_state" enum:"ok,degraded,backoff"`
Warnings []string `json:"warnings"`
LastSeen *time.Time `json:"last_seen" nullable:"true"`
CreatedAt time.Time `json:"created_at"`
RetiredAt *time.Time `json:"retired_at,omitempty" nullable:"true"`
Parents []DeviceRef `json:"parents"`
Groups []GroupView `json:"groups"`
Interfaces []InterfaceView `json:"interfaces"`
// M5 additions.
HostedOn *DeviceRef `json:"hosted_on,omitempty" doc:"The host node of a VM or container."`
Guests []DeviceRef `json:"guests" doc:"VMs and containers hosted on this device."`
@@ -118,7 +121,8 @@ func (s *server) view(ctx context.Context, slug string) (*deviceOut, error) {
return nil, huma.Error404NotFound("not found")
}
v := DeviceView{
Slug: d.Slug, Name: d.Name, Kind: d.Kind, Site: d.Site, Role: d.Role, Profile: d.Profile, Enabled: d.Enabled,
Slug: d.Slug, Name: d.Name, Kind: d.Kind, Site: d.Site, Role: d.Role, Profile: d.Profile,
ProfilePinned: d.ProfilePinned, ProfileMatch: s.profileMatch(d), Enabled: d.Enabled,
MgmtIP: d.MgmtIP, SysName: d.SysName, SysObjectID: d.SysObjectID, SysDescr: d.SysDescr, HWSerial: d.HWSerial,
PollState: pollStateOf(d), Warnings: s.warningsOf(d), LastSeen: utcp(d.LastSeen), CreatedAt: d.CreatedAt.UTC(),
RetiredAt: utcp(d.RetiredAt), Addresses: []string{}, Parents: []DeviceRef{}, Groups: []GroupView{},
@@ -192,7 +196,7 @@ func (s *server) fillInterfaces(ctx context.Context, d *store.Device, v *DeviceV
for i := range ifs {
v.Interfaces = append(v.Interfaces, interfaceView(&ifs[i], rates[ifs[i].ID]))
}
return nil
return s.withAlertDown(ctx, ifs, v.Interfaces)
}
func interfaceView(i *store.Interface, r *ifRates) InterfaceView {

View File

@@ -15,7 +15,8 @@ import (
// GroupSummary is a group in a list.
type GroupSummary struct {
GroupView
MemberCount int `json:"member_count" doc:"Interfaces in the group now."`
MemberCount int `json:"member_count" doc:"Interfaces in the group now."`
AlertDown bool `json:"alert_down" doc:"Member ports alert on interface_down while they are members."`
}
// GroupList is every group, by name.
@@ -57,6 +58,8 @@ type GroupDetail struct {
type GroupBody struct {
Name string `json:"name" minLength:"1" maxLength:"100"`
Kind string `json:"kind" enum:"site,customer,role,custom" default:"custom"`
// AlertDown is a pointer so a PUT that omits it leaves the flag alone.
AlertDown *bool `json:"alert_down,omitempty" doc:"Opt member ports into interface_down alerts. Omitted: off on create, kept on update."`
}
// GroupMemberBody is the body of POST /groups/{id}/members.
@@ -110,7 +113,8 @@ func summaryOf(now time.Time, g store.Group, members []store.GroupMember) GroupS
n++
}
}
return GroupSummary{GroupView: GroupView{ID: strconv.FormatInt(g.ID, 10), Name: g.Name, Kind: g.Kind}, MemberCount: n}
gv := GroupView{ID: strconv.FormatInt(g.ID, 10), Name: g.Name, Kind: g.Kind}
return GroupSummary{GroupView: gv, MemberCount: n, AlertDown: g.AlertDown}
}
func (s *server) listGroups(ctx context.Context, _ *struct{}) (*struct{ Body GroupList }, error) {
@@ -138,6 +142,7 @@ func groupBody(b GroupBody) (store.Group, error) {
if g.Name == "" {
return g, huma.Error422UnprocessableEntity("name must not be empty")
}
g.AlertDown = b.AlertDown != nil && *b.AlertDown
return g, nil
}
@@ -155,6 +160,7 @@ func (s *server) createGroup(ctx context.Context, in *struct{ Body GroupBody })
if err != nil {
return nil, s.storeErr(err)
}
s.alertDownChanged()
return &struct{ Body GroupSummary }{summaryOf(s.Clock.Now(), g, nil)}, nil
}
@@ -173,7 +179,8 @@ func (s *server) getGroup(ctx context.Context, in *struct {
if err != nil {
return nil, s.storeErr(err)
}
d := GroupDetail{GroupSummary: summaryOf(s.Clock.Now(), store.Group{ID: ref.ID, Name: ref.Name, Kind: ref.Kind}, members),
g := store.Group{ID: ref.ID, Name: ref.Name, Kind: ref.Kind, AlertDown: ref.AlertDown}
d := GroupDetail{GroupSummary: summaryOf(s.Clock.Now(), g, members),
Members: make([]GroupMemberView, 0, len(members)), Flags: []GroupFlag{}}
for _, m := range members {
d.Members = append(d.Members, s.memberView(ctx, m))
@@ -226,6 +233,13 @@ func (s *server) updateGroup(ctx context.Context, in *struct {
return nil, err
}
g.ID = id
if in.Body.AlertDown == nil {
cur, err := s.Store.GetGroup(ctx, id)
if err != nil {
return nil, s.storeErr(err)
}
g.AlertDown = cur.AlertDown
}
err = s.Store.WithTx(ctx, func(tx store.Store) error {
if err := tx.UpdateGroup(ctx, g); err != nil {
return err
@@ -235,6 +249,7 @@ func (s *server) updateGroup(ctx context.Context, in *struct {
if err != nil {
return nil, s.storeErr(err)
}
s.alertDownChanged()
members, err := s.Store.GroupMembers(ctx, id, time.Time{}, time.Time{})
if err != nil {
return nil, s.storeErr(err)
@@ -258,6 +273,7 @@ func (s *server) deleteGroup(ctx context.Context, in *struct {
if err != nil {
return nil, s.storeErr(err)
}
s.alertDownChanged()
return &struct{}{}, nil
}
@@ -287,6 +303,7 @@ func (s *server) addGroupMember(ctx context.Context, in *struct {
if err != nil {
return nil, s.storeErr(err)
}
s.alertDownChanged()
return &struct{ Body GroupMemberView }{s.memberView(ctx, m)}, nil
}
@@ -312,5 +329,6 @@ func (s *server) endGroupMember(ctx context.Context, in *struct {
if err != nil {
return nil, s.storeErr(err)
}
s.alertDownChanged()
return &struct{}{}, nil
}

View File

@@ -39,10 +39,33 @@ type PollerTotals struct {
DevicesDegraded int64 `json:"devices_degraded"`
}
// HealthNote is a condition worth showing on the health page that is not an alert: optional data is missing, or
// a background job is skipping devices.
type HealthNote struct {
Code string `json:"code" example:"oui_unavailable"`
Message string `json:"message"`
}
// L2Health is the MAC/ARP/FDB table job's last cycle.
type L2Health struct {
Running bool `json:"running"`
LastCycle *time.Time `json:"last_cycle" nullable:"true" doc:"When the last cycle ended."`
IntervalS int64 `json:"interval_s"`
OK int64 `json:"devices_ok"`
TimedOut int64 `json:"devices_timed_out" doc:"Devices skipped in the last cycle because they did not answer in time."`
Failed int64 `json:"devices_failed"`
// Skipped names the devices that timed out or failed in the last cycle.
Skipped []string `json:"skipped"`
}
// PollerHealth is the poller health page.
type PollerHealth struct {
Devices []PollerDeviceHealth `json:"devices"`
Totals PollerTotals `json:"totals"`
// L2 is the MAC/ARP/FDB table job; absent when it is not running.
L2 *L2Health `json:"l2,omitempty"`
// Notes are conditions that are not alerts, such as oui_unavailable (no vendor registry loaded).
Notes []HealthNote `json:"notes"`
}
func (s *server) registerHealth(api huma.API) {
@@ -58,7 +81,13 @@ func (s *server) pollerHealth(ctx context.Context, _ *struct{}) (*struct{ Body P
if err != nil {
return nil, s.storeErr(err)
}
out := PollerHealth{Devices: []PollerDeviceHealth{}}
out := PollerHealth{Devices: []PollerDeviceHealth{}, Notes: []HealthNote{}}
if s.L2Health != nil {
out.L2 = s.L2Health()
}
if s.HealthNotes != nil {
out.Notes = append(out.Notes, s.HealthNotes()...)
}
if s.PollerStatus != nil {
for _, d := range s.PollerStatus() {
h := PollerDeviceHealth{

View File

@@ -22,6 +22,7 @@ type InterfaceHit struct {
SpeedBPS int64 `json:"speed_bps" doc:"Link speed in bits per second (ifHighSpeed when set)."`
InBPS *float64 `json:"in_bps" nullable:"true"`
OutBPS *float64 `json:"out_bps" nullable:"true"`
AlertDownState
}
// InterfaceList is a page of InterfaceHit.
@@ -52,11 +53,11 @@ func (s *server) listInterfaces(ctx context.Context, in *struct {
out := InterfaceList{Items: make([]InterfaceHit, 0, len(rows)), Page: in.pageNo(), PerPage: in.limit(), Total: total}
var rates map[string]*ifRates
if len(rows) > 0 {
ids := make([]string, len(rows))
for i, r := range rows {
ids[i] = r.ID
}
rates, _ = s.liveRates(ctx, `iface=~`+quote(regexAlt(ids)), false)
rates, _ = s.liveRates(ctx, `iface=~`+quote(regexAlt(hitIDs(rows))), false)
}
via, err := s.Store.InterfaceAlertDownVia(ctx, hitIDs(rows))
if err != nil {
return nil, s.storeErr(err)
}
for i := range rows {
r := &rows[i]
@@ -64,7 +65,7 @@ func (s *server) listInterfaces(ctx context.Context, in *struct {
out.Items = append(out.Items, InterfaceHit{
ID: r.ID, Device: DeviceRef{Slug: r.DeviceSlug, Name: r.DeviceName}, IfName: r.IfName, IfAlias: r.IfAlias, Description: r.Description,
IfDescr: r.IfDescr, OperStatus: r.OperStatus, AdminStatus: r.AdminStatus, SpeedBPS: speedBPS(&r.Interface),
InBPS: inB, OutBPS: outB,
InBPS: inB, OutBPS: outB, AlertDownState: alertDownState(r.AlertDown, via[r.ID]),
})
}
return &struct{ Body InterfaceList }{out}, nil
@@ -78,3 +79,11 @@ func speedBPS(i *store.Interface) int64 {
}
return i.IfSpeed
}
func hitIDs(rows []store.InterfaceHit) []string {
ids := make([]string, len(rows))
for i := range rows {
ids[i] = rows[i].ID
}
return ids
}

83
internal/api/l2_rank.go Normal file
View File

@@ -0,0 +1,83 @@
package api
import (
"cmp"
"slices"
"github.com/v10networks/nmon/internal/store"
)
// ifTypeLAG is IANAifType ieee8023adLag: a link aggregate carries the MACs of its members, so it is an uplink.
const ifTypeLAG = 161
// Reasons a port is not an edge port.
const (
uplinkMACs = "macs"
uplinkLLDP = "lldp"
uplinkLAG = "lag"
)
// portFacts is what decides whether a port is where a host is plugged in: a MAC is seen on every switch it passes
// through, so only a port with few MACs and no switch behind it is the host's own.
type portFacts struct {
counts map[store.PortKey]int64
lldp map[int64]map[int64]bool // device -> ifIndexes with an LLDP neighbour
edgeMax int64 // l2.edge_max_macs
}
// key is the MAC count key of a row: the port's MACs seen at or after the row's last_seen, which for a current
// row is every MAC of the device's last walk on that port.
func portKeyOf(r *store.FDBRow) store.PortKey {
return store.PortKey{DeviceID: r.DeviceID, InterfaceID: r.InterfaceID, Since: r.LastSeen}
}
// uplink says why the port of r is not an edge port, or "" when it is one.
func (f portFacts) uplink(r *store.FDBRow, count int64) string {
switch {
case count > f.edgeMax:
return uplinkMACs
case r.IfIndex != 0 && f.lldp[r.DeviceID][r.IfIndex]:
return uplinkLLDP
case r.IfType == ifTypeLAG:
return uplinkLAG
}
return ""
}
// location builds the API view of a row.
func (f portFacts) location(r *store.FDBRow) L2Location {
n := f.counts[portKeyOf(r)]
why := f.uplink(r, n)
loc := L2Location{
Device: DeviceRef{Slug: r.DeviceSlug, Name: r.DeviceName}, VLAN: r.VLAN, VLANName: r.VLANName,
Edge: why == "", UplinkReason: why, PortMACCount: n, Current: r.Current, FirstSeen: r.FirstSeen, LastSeen: r.LastSeen,
}
if r.InterfaceID != "" {
loc.Iface = &L2IfaceRef{ID: r.InterfaceID, IfName: r.IfName}
}
return loc
}
// sortLocations puts the host's own port first: edge ports before uplinks, then rows the last walk still sees
// before history, then the port with the fewest MACs, then the most recently seen.
func sortLocations(locs []L2Location) {
slices.SortStableFunc(locs, func(a, b L2Location) int {
if c := cmp.Compare(boolRank(b.Edge), boolRank(a.Edge)); c != 0 {
return c
}
if c := cmp.Compare(boolRank(b.Current), boolRank(a.Current)); c != 0 {
return c
}
if c := cmp.Compare(a.PortMACCount, b.PortMACCount); c != 0 {
return c
}
return b.LastSeen.Compare(a.LastSeen)
})
}
func boolRank(b bool) int {
if b {
return 1
}
return 0
}

200
internal/api/l2_search.go Normal file
View File

@@ -0,0 +1,200 @@
package api
import (
"context"
"net/http"
"slices"
"strconv"
"github.com/danielgtaylor/huma/v2"
"github.com/v10networks/nmon/internal/config"
"github.com/v10networks/nmon/internal/hostaddr"
"github.com/v10networks/nmon/internal/store"
)
func (s *server) registerL2(api huma.API) {
huma.Register(api, huma.Operation{
OperationID: "search-l2", Method: http.MethodGet, Path: "/api/v1/l2/search", Tags: []string{"l2"},
Summary: "Find where a MAC or IP address is plugged in",
Description: "q is a MAC address in any common notation (aabb.ccdd.eeff, aa-bb-..., aabbccddeeff), a MAC prefix of at least " +
"six hex digits, an IPv4 or IPv6 address, a CIDR (matched against ARP and neighbour tables), or vendor:<name> " +
"(the current MACs of a vendor in the IEEE registry). Each hit is a MAC with the addresses it answers to and every " +
"place it was seen, history included: edge ports first (a port that carries few MACs, has no LLDP neighbour and " +
"is not a LAG is where the host is plugged in), then uplinks.",
Errors: []int{http.StatusUnprocessableEntity},
}, s.l2Search)
s.registerL2Tables(api)
}
// maxVendorHits caps how many MACs one vendor search considers before paging.
const maxVendorHits = 20_000
func (s *server) l2Search(ctx context.Context, in *struct {
Q string `query:"q" minLength:"1" maxLength:"200" doc:"MAC, MAC prefix, IP, CIDR or vendor:name."`
VLAN string `query:"vlan" pattern:"^[0-9]{1,4}$" doc:"Only MACs seen in this VLAN."`
PageParams
}) (*struct{ Body L2SearchResult }, error) {
q, err := hostaddr.Classify(in.Q)
if err != nil {
return nil, huma.Error422UnprocessableEntity(err.Error())
}
res := L2SearchResult{Items: []L2Hit{}, Page: in.pageNo(), PerPage: in.limit(), Kind: q.Kind}
res.VendorsAvailable = s.Vendors != nil && s.Vendors.Available()
var hits []store.MACHit
vlan := optInt(in.VLAN)
if q.Kind == hostaddr.QueryVendor {
hits, res.Total, err = s.vendorHits(ctx, q.Vendor, vlan, in.PageParams)
} else {
hits, res.Total, err = s.Store.FindMACs(ctx, finderFor(q, vlan), store.Page{Offset: in.offset(), Limit: in.limit()})
}
if err != nil {
return nil, s.storeErr(err)
}
if res.Items, err = s.hitsFor(ctx, hits); err != nil {
return nil, s.storeErr(err)
}
return &struct{ Body L2SearchResult }{res}, nil
}
// optInt reads an optional non-negative integer query value: nil when empty.
func optInt(s string) *int64 {
if v, err := strconv.ParseInt(s, 10, 64); err == nil {
return &v
}
return nil
}
func finderFor(q hostaddr.Query, vlan *int64) store.MACFinder {
f := store.MACFinder{VLAN: vlan}
switch q.Kind {
case hostaddr.QueryMAC:
f.MAC = q.MAC
case hostaddr.QueryPrefix:
f.MACPrefix = q.MAC
case hostaddr.QueryIP:
f.IP = q.IP
case hostaddr.QueryCIDR:
f.Prefix = q.Prefix
}
return f
}
// vendorHits finds the current MACs whose registered vendor contains name, and pages them. The registry is in
// memory, so the filter runs here on the distinct current MACs the store streams.
func (s *server) vendorHits(ctx context.Context, name string, vlan *int64, p PageParams) ([]store.MACHit, int64, error) {
if s.Vendors == nil {
return nil, 0, nil
}
match := s.Vendors.Match(name)
var macs []string
err := s.Store.WalkCurrentMACs(ctx, vlan, func(mac string) bool {
if match(mac) {
macs = append(macs, mac)
}
return len(macs) < maxVendorHits
})
if err != nil {
return nil, 0, err
}
total := int64(len(macs))
lo := min(p.offset(), total)
hi := min(lo+p.limit(), total)
out := make([]store.MACHit, 0, hi-lo)
for _, m := range macs[lo:hi] {
out = append(out, store.MACHit{Mac: m})
}
return out, total, nil
}
// vendorOf is the vendor of a canonical MAC, "" without a registry.
func (s *server) vendorOf(mac string) string {
if s.Vendors == nil {
return ""
}
return s.Vendors.Vendor(mac)
}
// edgeMax is l2.edge_max_macs.
func (s *server) edgeMax() int64 {
if s.L2EdgeMax != nil {
return int64(s.L2EdgeMax())
}
return config.DefaultL2EdgeMaxMACs
}
// hitsFor loads the FDB and ARP rows of the MACs and builds the grouped hits in the order given.
func (s *server) hitsFor(ctx context.Context, hits []store.MACHit) ([]L2Hit, error) {
macs := make([]string, len(hits))
for i, h := range hits {
macs[i] = h.Mac
}
fdb, err := s.Store.FDBForMACs(ctx, macs)
if err != nil {
return nil, err
}
arp, err := s.Store.ARPForMACs(ctx, macs)
if err != nil {
return nil, err
}
facts, err := s.facts(ctx, fdb)
if err != nil {
return nil, err
}
locs := map[string][]L2Location{}
for i := range fdb {
locs[fdb[i].Mac] = append(locs[fdb[i].Mac], facts.location(&fdb[i]))
}
ips := map[string][]L2IP{}
for _, a := range arp {
ips[a.Mac] = append(ips[a.Mac], L2IP{IP: a.IP, Device: DeviceRef{Slug: a.DeviceSlug, Name: a.DeviceName}, LastSeen: a.LastSeen})
}
out := make([]L2Hit, 0, len(hits))
for _, h := range hits {
l := locs[h.Mac]
sortLocations(l)
out = append(out, L2Hit{
MAC: h.Mac, Vendor: s.vendorOf(h.Mac), MACKind: hostaddr.Kind(h.Mac),
IPs: nonNilIPs(ips[h.Mac]), Locations: nonNilLocs(l),
})
}
return out, nil
}
func nonNilIPs(v []L2IP) []L2IP {
if v == nil {
return []L2IP{}
}
slices.SortStableFunc(v, func(a, b L2IP) int { return b.LastSeen.Compare(a.LastSeen) })
return v
}
func nonNilLocs(v []L2Location) []L2Location {
if v == nil {
return []L2Location{}
}
return v
}
// facts gathers the per-port MAC counts and LLDP links that decide edge versus uplink for the given rows.
func (s *server) facts(ctx context.Context, rows []store.FDBRow) (portFacts, error) {
keys := make([]store.PortKey, 0, len(rows))
devs := map[int64]bool{}
for i := range rows {
keys = append(keys, portKeyOf(&rows[i]))
devs[rows[i].DeviceID] = true
}
counts, err := s.Store.PortMACCounts(ctx, keys)
if err != nil {
return portFacts{}, err
}
ids := make([]int64, 0, len(devs))
for id := range devs {
ids = append(ids, id)
}
lldp, err := s.Store.LLDPIfIndexes(ctx, ids)
if err != nil {
return portFacts{}, err
}
return portFacts{counts: counts, lldp: lldp, edgeMax: s.edgeMax()}, nil
}

283
internal/api/l2_tables.go Normal file
View File

@@ -0,0 +1,283 @@
package api
import (
"context"
"net/http"
"net/netip"
"github.com/danielgtaylor/huma/v2"
"github.com/v10networks/nmon/internal/hostaddr"
"github.com/v10networks/nmon/internal/store"
)
func (s *server) registerL2Tables(api huma.API) {
huma.Register(api, huma.Operation{
OperationID: "get-device-l2", Method: http.MethodGet, Path: "/api/v1/devices/{slug}/l2", Tags: []string{"l2"},
Summary: "What the L2 job knows about a device",
Description: "Row counts, the time and source of the last walk and the VLAN list. The device page shows its L2 tab " +
"when has_data is true.",
Errors: []int{http.StatusNotFound},
}, s.deviceL2)
huma.Register(api, huma.Operation{
OperationID: "list-device-fdb", Method: http.MethodGet, Path: "/api/v1/devices/{slug}/fdb", Tags: []string{"l2"},
Summary: "A device's forwarding table (MACs by port and VLAN)",
Description: "current=true (the default) lists the rows the last walk saw; current=false adds history, each row with its first and " +
"last seen. Ordered by VLAN, then MAC.",
Errors: []int{http.StatusNotFound, http.StatusUnprocessableEntity},
}, s.deviceFDB)
huma.Register(api, huma.Operation{
OperationID: "list-device-arp", Method: http.MethodGet, Path: "/api/v1/devices/{slug}/arp", Tags: []string{"l2"},
Summary: "A device's ARP and IPv6 neighbour table",
Description: "current=true (the default) lists the rows the last walk saw; current=false adds history. ip is an address or a CIDR.",
Errors: []int{http.StatusNotFound, http.StatusUnprocessableEntity},
}, s.deviceARP)
huma.Register(api, huma.Operation{
OperationID: "list-device-l2-ports", Method: http.MethodGet, Path: "/api/v1/devices/{slug}/l2/ports", Tags: []string{"l2"},
Summary: "How many MACs each port of a device carries now",
Description: "For the Ports tab: ports that carry MACs, with whether they look like uplinks " +
"(too many MACs, an LLDP neighbour, or a LAG).",
Errors: []int{http.StatusNotFound},
}, s.deviceL2Ports)
huma.Register(api, huma.Operation{
OperationID: "list-interface-macs", Method: http.MethodGet, Path: "/api/v1/interfaces/{id}/macs", Tags: []string{"l2"},
Summary: "The MACs on one port, with their IP addresses",
Description: "current=true (the default) lists the MACs the last walk saw on the port; current=false adds history.",
Errors: []int{http.StatusNotFound},
}, s.interfaceMACs)
}
// l2Device loads a device for the L2 routes.
func (s *server) l2Device(ctx context.Context, slug string) (*store.Device, error) {
d, err := s.Store.GetDeviceBySlug(ctx, slug)
if err != nil {
return nil, s.storeErr(err)
}
if d.PurgedAt != nil {
return nil, huma.Error404NotFound("not found")
}
return d, nil
}
func (s *server) deviceL2(ctx context.Context, in *struct {
Slug string `path:"slug" doc:"Device slug."`
}) (*struct{ Body L2Summary }, error) {
d, err := s.l2Device(ctx, in.Slug)
if err != nil {
return nil, err
}
c, err := s.Store.L2Counts(ctx, d.ID)
if err != nil {
return nil, s.storeErr(err)
}
vl, err := s.Store.ListVLANs(ctx, d.ID)
if err != nil {
return nil, s.storeErr(err)
}
out := L2Summary{
HasData: c.FDBTotal+c.ARPTotal > 0, FDBCurrent: c.FDBCurrent, FDBTotal: c.FDBTotal, ARPCurrent: c.ARPCurrent, ARPTotal: c.ARPTotal,
VLANs: make([]L2VLAN, 0, len(vl)), EdgeMaxMACs: s.edgeMax(),
}
if c.Walk != nil {
t := c.Walk.WalkedAt.UTC()
out.LastWalk, out.Source = &t, c.Walk.Source
}
for _, v := range vl {
out.VLANs = append(out.VLANs, L2VLAN{VLAN: v.VLAN, Name: v.Name})
}
return &struct{ Body L2Summary }{out}, nil
}
// ifaceRef is the interface reference of a row, nil without an interface.
func ifaceRef(id, name string) *L2IfaceRef {
if id == "" {
return nil
}
return &L2IfaceRef{ID: id, IfName: name}
}
// ipsByMAC maps each MAC to the addresses ARP and neighbour tables give it, most recent first.
func (s *server) ipsByMAC(ctx context.Context, macs []string) (map[string][]string, error) {
arp, err := s.Store.ARPForMACs(ctx, macs)
if err != nil {
return nil, err
}
out := map[string][]string{}
for _, a := range arp {
if !contains(out[a.Mac], a.IP) {
out[a.Mac] = append(out[a.Mac], a.IP)
}
}
return out, nil
}
func contains(s []string, v string) bool {
for _, x := range s {
if x == v {
return true
}
}
return false
}
// fdbRows builds the API rows of a page of FDB rows with their IPs.
func (s *server) fdbRows(ctx context.Context, rows []store.FDBRow) ([]L2FDBRow, error) {
macs := make([]string, len(rows))
for i := range rows {
macs[i] = rows[i].Mac
}
ips, err := s.ipsByMAC(ctx, macs)
if err != nil {
return nil, err
}
out := make([]L2FDBRow, 0, len(rows))
for i := range rows {
r := &rows[i]
out = append(out, L2FDBRow{
MAC: r.Mac, Vendor: s.vendorOf(r.Mac), MACKind: hostaddr.Kind(r.Mac), VLAN: r.VLAN, VLANName: r.VLANName,
Iface: ifaceRef(r.InterfaceID, r.IfName), IPs: nonNilStrings(ips[r.Mac]), Current: r.Current,
FirstSeen: r.FirstSeen, LastSeen: r.LastSeen,
})
}
return out, nil
}
func nonNilStrings(v []string) []string {
if v == nil {
return []string{}
}
return v
}
func (s *server) deviceFDB(ctx context.Context, in *struct {
Slug string `path:"slug" doc:"Device slug."`
VLAN string `query:"vlan" pattern:"^[0-9]{1,4}$" doc:"Only this VLAN."`
Interface string `query:"interface" doc:"Only this interface (ULID)."`
MAC string `query:"mac" doc:"Only MACs starting with this prefix (at least 6 hex digits)."`
Current bool `query:"current" default:"true" doc:"true (default): rows the last walk saw. false: with history."`
PageParams
}) (*struct{ Body L2FDBList }, error) {
d, err := s.l2Device(ctx, in.Slug)
if err != nil {
return nil, err
}
f := store.FDBFilter{VLAN: optInt(in.VLAN), InterfaceID: in.Interface, Current: in.Current}
if in.MAC != "" {
if f.MACPrefix, err = hostaddr.ParsePrefix(in.MAC); err != nil {
return nil, huma.Error422UnprocessableEntity(err.Error())
}
}
rows, total, err := s.Store.ListFDB(ctx, d.ID, f, store.Page{Offset: in.offset(), Limit: in.limit()})
if err != nil {
return nil, s.storeErr(err)
}
items, err := s.fdbRows(ctx, rows)
if err != nil {
return nil, s.storeErr(err)
}
return &struct{ Body L2FDBList }{L2FDBList{Items: items, Page: in.pageNo(), PerPage: in.limit(), Total: total}}, nil
}
func (s *server) deviceARP(ctx context.Context, in *struct {
Slug string `path:"slug" doc:"Device slug."`
IP string `query:"ip" doc:"An address or a CIDR."`
Interface string `query:"interface" doc:"Only this interface (ULID)."`
MAC string `query:"mac" doc:"Only MACs starting with this prefix (at least 6 hex digits)."`
Current bool `query:"current" default:"true" doc:"true (default): rows the last walk saw. false: with history."`
PageParams
}) (*struct{ Body L2ARPList }, error) {
d, err := s.l2Device(ctx, in.Slug)
if err != nil {
return nil, err
}
f := store.ARPFilter{InterfaceID: in.Interface, Current: in.Current}
if in.IP != "" {
if f.IP, err = parseIPOrCIDR(in.IP); err != nil {
return nil, huma.Error422UnprocessableEntity(err.Error())
}
}
if in.MAC != "" {
if f.MACPrefix, err = hostaddr.ParsePrefix(in.MAC); err != nil {
return nil, huma.Error422UnprocessableEntity(err.Error())
}
}
rows, total, err := s.Store.ListARP(ctx, d.ID, f, store.Page{Offset: in.offset(), Limit: in.limit()})
if err != nil {
return nil, s.storeErr(err)
}
items := make([]L2ARPRow, 0, len(rows))
for _, r := range rows {
items = append(items, L2ARPRow{
IP: r.IP, MAC: r.Mac, Vendor: s.vendorOf(r.Mac), MACKind: hostaddr.Kind(r.Mac), Iface: ifaceRef(r.InterfaceID, r.IfName),
Current: r.Current, FirstSeen: r.FirstSeen, LastSeen: r.LastSeen,
})
}
return &struct{ Body L2ARPList }{L2ARPList{Items: items, Page: in.pageNo(), PerPage: in.limit(), Total: total}}, nil
}
// parseIPOrCIDR reads an address as a host prefix or a CIDR as given.
func parseIPOrCIDR(s string) (netip.Prefix, error) {
q, err := hostaddr.Classify(s)
switch {
case err != nil:
return netip.Prefix{}, err
case q.Kind == hostaddr.QueryIP:
return netip.PrefixFrom(q.IP, q.IP.BitLen()), nil
case q.Kind == hostaddr.QueryCIDR:
return q.Prefix, nil
}
return netip.Prefix{}, hostaddr.ErrBadQuery
}
func (s *server) deviceL2Ports(ctx context.Context, in *struct {
Slug string `path:"slug" doc:"Device slug."`
}) (*struct{ Body L2PortList }, error) {
d, err := s.l2Device(ctx, in.Slug)
if err != nil {
return nil, err
}
counts, err := s.Store.CurrentMACCounts(ctx, d.ID)
if err != nil {
return nil, s.storeErr(err)
}
ifs, err := s.Store.ListInterfaces(ctx, d.ID, false)
if err != nil {
return nil, s.storeErr(err)
}
lldp, err := s.Store.LLDPIfIndexes(ctx, []int64{d.ID})
if err != nil {
return nil, s.storeErr(err)
}
f := portFacts{lldp: lldp, edgeMax: s.edgeMax()}
out := L2PortList{Items: []L2Port{}}
for i := range ifs {
n := counts[ifs[i].ID]
if n == 0 {
continue
}
row := store.FDBRow{DeviceID: d.ID, IfIndex: ifs[i].IfIndex, IfType: ifs[i].IfType}
why := f.uplink(&row, n)
ref := L2IfaceRef{ID: ifs[i].ID, IfName: ifs[i].IfName}
out.Items = append(out.Items, L2Port{Iface: ref, MACs: n, Uplink: why != "", UplinkReason: why})
}
return &struct{ Body L2PortList }{out}, nil
}
func (s *server) interfaceMACs(ctx context.Context, in *struct {
ID string `path:"id" doc:"Interface ULID."`
Current bool `query:"current" default:"true" doc:"true (default): MACs the last walk saw. false: with history."`
PageParams
}) (*struct{ Body L2FDBList }, error) {
if _, err := s.Store.GetInterface(ctx, in.ID); err != nil {
return nil, s.storeErr(err)
}
rows, total, err := s.Store.InterfaceMACs(ctx, in.ID, in.Current, store.Page{Offset: in.offset(), Limit: in.limit()})
if err != nil {
return nil, s.storeErr(err)
}
items, err := s.fdbRows(ctx, rows)
if err != nil {
return nil, s.storeErr(err)
}
return &struct{ Body L2FDBList }{L2FDBList{Items: items, Page: in.pageNo(), PerPage: in.limit(), Total: total}}, nil
}

302
internal/api/l2_test.go Normal file
View File

@@ -0,0 +1,302 @@
package api_test
import (
"context"
"fmt"
"net/url"
"strings"
"testing"
"time"
"github.com/v10networks/nmon/internal/api"
"github.com/v10networks/nmon/internal/store"
)
// stubVendors is a registry of two vendors.
type stubVendors struct{}
func (stubVendors) Vendor(mac string) string {
switch {
case strings.HasPrefix(mac, "f0:9f:c2"):
return "Ubiquiti Inc"
case mac[1] == '2' || mac[1] == '6' || mac[1] == 'a' || mac[1] == 'e':
return "Private / randomised"
}
return ""
}
func (stubVendors) Available() bool { return true }
func (stubVendors) Match(sub string) func(string) bool {
return func(mac string) bool { return strings.Contains(strings.ToLower((stubVendors{}).Vendor(mac)), sub) }
}
// seedL2Net seeds three switches that all see the same host: its own access switch, a distribution switch where the
// host's MAC is one of ten behind an uplink, and a core switch whose port toward the distribution layer has an
// LLDP neighbour.
const host = "f0:9f:c2:11:22:33"
func seedL2Net(t *testing.T, e *env) {
t.Helper()
ctx := context.Background()
access := e.device(t, "access1", "yyz", "access")
dist := e.device(t, "dist1", "yyz", "distribution")
core := e.device(t, "core1", "yyz", "core")
aif := e.iface(t, access, 7, "Gi1/0/7", "")
dif := e.iface(t, dist, 48, "Te1/1", "")
cif := e.iface(t, core, 3, "Eth3", "")
now := e.clk.Now()
rec := func(d *store.Device, entries []store.FDBEntry, arp []store.ARPEntry) {
if err := e.st.RecordL2Walk(ctx, d.ID, store.L2WalkResult{At: now, Source: "qbridge", FDB: entries, ARP: arp,
VLANs: []store.VLANName{{VLAN: 20, Name: "cameras"}}}); err != nil {
t.Fatal(err)
}
}
rec(access, []store.FDBEntry{{InterfaceID: aif.ID, VLAN: 20, Mac: host}}, nil)
var many []store.FDBEntry
many = append(many, store.FDBEntry{InterfaceID: dif.ID, VLAN: 20, Mac: host})
for i := 1; i <= 9; i++ {
many = append(many, store.FDBEntry{InterfaceID: dif.ID, VLAN: 20, Mac: fmt.Sprintf("02:00:00:00:00:%02x", i)})
}
rec(dist, many, nil)
rec(core, []store.FDBEntry{{InterfaceID: cif.ID, VLAN: 20, Mac: host}}, []store.ARPEntry{
{IP: "192.0.2.77", Mac: host, InterfaceID: cif.ID}, {IP: "2001:db8::77", Mac: host, InterfaceID: cif.ID},
})
// The core's port toward the distribution layer has an LLDP neighbour (local port number = ifIndex).
if err := e.st.ReplaceComponents(ctx, core.ID, "LLDP-MIB::lldpRemTable", []store.Component{{Idx: "0.3.1", Label: "dist1"}}); err != nil {
t.Fatal(err)
}
}
func searchL2(t *testing.T, e *env, q string) api.L2SearchResult {
t.Helper()
var got api.L2SearchResult
e.getJSON(t, "/api/v1/l2/search?q="+url.QueryEscape(q), &got)
return got
}
func TestAMACSeenOnAnAccessPortAndTwoUplinksResolvesToTheAccessPortFirst(t *testing.T) {
e := newEnvDeps(t, 0, func(d *api.Deps) { d.Vendors = stubVendors{} })
seedL2Net(t, e)
got := searchL2(t, e, host)
if got.Total != 1 || len(got.Items) != 1 {
t.Fatalf("hits = %+v", got)
}
h := got.Items[0]
if h.MAC != host || h.Vendor != "Ubiquiti Inc" || h.MACKind != "global" {
t.Errorf("hit = %+v", h)
}
if len(h.Locations) != 3 {
t.Fatalf("locations = %+v", h.Locations)
}
first := h.Locations[0]
if !first.Edge || first.Device.Slug == "" || first.Iface == nil || first.Iface.IfName != "Gi1/0/7" || first.PortMACCount != 1 ||
first.VLAN != 20 || first.VLANName != "cameras" || !first.Current {
t.Errorf("the access port must come first: %+v", first)
}
reasons := map[string]string{}
for _, l := range h.Locations[1:] {
if l.Edge {
t.Errorf("an uplink was reported as an edge port: %+v", l)
}
reasons[l.Device.Name] = l.UplinkReason
}
if reasons["dist1"] != "macs" || reasons["core1"] != "lldp" {
t.Errorf("uplink reasons = %v", reasons)
}
if len(h.IPs) != 2 {
t.Errorf("ips = %+v", h.IPs)
}
}
func TestSearchAcceptsEveryMACFormatAndPrefixAndRefusesGarbage(t *testing.T) {
e := newEnv(t, 0)
seedL2Net(t, e)
for _, q := range []string{"f09f.c211.2233", "F0-9F-C2-11-22-33", "f09fc2112233", "f0:9f:c2", "f09fc2", "F0:9F:C2:11:2"} {
if got := searchL2(t, e, q); got.Total != 1 || got.Items[0].MAC != host {
t.Errorf("q=%q: %+v", q, got)
}
}
for _, q := range []string{"hello", "f09f", "zz:zz:zz:zz:zz:zz", "1.2.3", "10.0.0.0/99", "vendor:"} {
rec := e.do("GET", "/api/v1/l2/search?q="+url.QueryEscape(q), "")
isProblem(t, rec, 422)
}
}
func TestIPv6AndIPv4AndCIDRSearchFindTheMAC(t *testing.T) {
e := newEnv(t, 0)
seedL2Net(t, e)
for q, kind := range map[string]string{"2001:db8::77": "ip", "192.0.2.77": "ip", "192.0.2.0/24": "cidr", "2001:db8::/32": "cidr"} {
got := searchL2(t, e, q)
if got.Kind != kind || got.Total != 1 || got.Items[0].MAC != host {
t.Errorf("q=%q: %+v", q, got)
}
}
if got := searchL2(t, e, "198.51.100.0/24"); got.Total != 0 || len(got.Items) != 0 {
t.Errorf("a CIDR with no entries matched: %+v", got)
}
}
func TestVendorSearchListsCurrentMACsAndWorksEmptyWithoutARegistry(t *testing.T) {
e := newEnvDeps(t, 0, func(d *api.Deps) { d.Vendors = stubVendors{} })
seedL2Net(t, e)
got := searchL2(t, e, "vendor:ubiquiti")
if got.Kind != "vendor" || !got.VendorsAvailable || got.Total != 1 || got.Items[0].MAC != host {
t.Fatalf("vendor search = %+v", got)
}
var vlan20 api.L2SearchResult
e.getJSON(t, "/api/v1/l2/search?q=vendor:ubiquiti&vlan=30", &vlan20)
if vlan20.Total != 0 {
t.Errorf("a vendor search restricted to another VLAN matched: %+v", vlan20)
}
none := newEnv(t, 0)
seedL2Net(t, none)
got = searchL2(t, none, "vendor:ubiquiti")
if got.VendorsAvailable || got.Total != 0 || len(got.Items) != 0 {
t.Errorf("without a registry the vendor search must find nothing, not fail: %+v", got)
}
// The same lookup by MAC still works, with an empty vendor.
if h := searchL2(t, none, host).Items[0]; h.Vendor != "" || h.MACKind != "global" || len(h.Locations) != 3 {
t.Errorf("hit without a registry = %+v", h)
}
}
func TestLocallyAdministeredMACsAreFlaggedPrivate(t *testing.T) {
e := newEnvDeps(t, 0, func(d *api.Deps) { d.Vendors = stubVendors{} })
seedL2Net(t, e)
h := searchL2(t, e, "02:00:00:00:00:01").Items[0]
if h.MACKind != "local" || h.Vendor != "Private / randomised" {
t.Errorf("hit = %+v", h)
}
}
func TestDeviceTablesFilterByVLANInterfaceAndCurrent(t *testing.T) {
e := newEnv(t, 0)
seedL2Net(t, e)
devs, _, _ := e.st.ListDevices(context.Background(), store.DeviceFilter{}, store.Page{})
var dist, core *store.Device
for i := range devs {
switch devs[i].Name {
case "dist1":
dist = &devs[i]
case "core1":
core = &devs[i]
}
}
var fdb api.L2FDBList
e.getJSON(t, "/api/v1/devices/"+dist.Slug+"/fdb", &fdb)
if fdb.Total != 10 || len(fdb.Items) != 10 || !fdb.Items[0].Current {
t.Fatalf("fdb = %+v", fdb)
}
e.getJSON(t, "/api/v1/devices/"+dist.Slug+"/fdb?vlan=30", &fdb)
if fdb.Total != 0 {
t.Errorf("vlan filter: %+v", fdb)
}
e.getJSON(t, "/api/v1/devices/"+dist.Slug+"/fdb?mac=f09fc2", &fdb)
if fdb.Total != 1 || fdb.Items[0].MAC != host {
t.Errorf("mac prefix filter: %+v", fdb)
}
isProblem(t, e.do("GET", "/api/v1/devices/"+dist.Slug+"/fdb?mac=zz", ""), 422)
var arp api.L2ARPList
e.getJSON(t, "/api/v1/devices/"+core.Slug+"/arp", &arp)
if arp.Total != 2 {
t.Fatalf("arp = %+v", arp)
}
e.getJSON(t, "/api/v1/devices/"+core.Slug+"/arp?ip=2001:db8::/32", &arp)
if arp.Total != 1 || arp.Items[0].IP != "2001:db8::77" {
t.Errorf("arp ip filter: %+v", arp)
}
// History: a later walk that no longer sees the host leaves the row, no longer current.
if err := e.st.RecordL2Walk(context.Background(), core.ID, store.L2WalkResult{At: e.clk.Now().Add(time.Hour)}); err != nil {
t.Fatal(err)
}
e.getJSON(t, "/api/v1/devices/"+core.Slug+"/fdb", &fdb)
if fdb.Total != 0 {
t.Errorf("current=true must hide rows the last walk missed: %+v", fdb)
}
e.getJSON(t, "/api/v1/devices/"+core.Slug+"/fdb?current=false", &fdb)
if fdb.Total != 1 || fdb.Items[0].Current {
t.Errorf("history: %+v", fdb)
}
isProblem(t, e.do("GET", "/api/v1/devices/nope/fdb", ""), 404)
}
func TestInterfaceMACsWithTheirIPsAndPortsSummary(t *testing.T) {
e := newEnv(t, 0)
seedL2Net(t, e)
ifs, _, _ := e.st.SearchInterfaces(context.Background(), "Eth3", false, store.Page{})
var macs api.L2FDBList
e.getJSON(t, "/api/v1/interfaces/"+ifs[0].ID+"/macs", &macs)
if macs.Total != 1 || macs.Items[0].MAC != host || len(macs.Items[0].IPs) != 2 {
t.Fatalf("macs = %+v", macs)
}
isProblem(t, e.do("GET", "/api/v1/interfaces/01ARZ3NDEKTSV4RRFFQ69G5FAV/macs", ""), 404)
devs, _, _ := e.st.ListDevices(context.Background(), store.DeviceFilter{}, store.Page{})
for _, d := range devs {
var sum api.L2Summary
e.getJSON(t, "/api/v1/devices/"+d.Slug+"/l2", &sum)
if !sum.HasData || sum.LastWalk == nil || sum.Source != "qbridge" || len(sum.VLANs) != 1 || sum.EdgeMaxMACs != 8 {
t.Errorf("%s summary = %+v", d.Name, sum)
}
var ports api.L2PortList
e.getJSON(t, "/api/v1/devices/"+d.Slug+"/l2/ports", &ports)
if len(ports.Items) != 1 {
t.Fatalf("%s ports = %+v", d.Name, ports)
}
p := ports.Items[0]
wantUp := map[string]string{"access1": "", "dist1": "macs", "core1": "lldp"}[d.Name]
if p.UplinkReason != wantUp || p.Uplink != (wantUp != "") {
t.Errorf("%s port = %+v", d.Name, p)
}
}
other := e.device(t, "plain", "yyz", "server")
var sum api.L2Summary
e.getJSON(t, "/api/v1/devices/"+other.Slug+"/l2", &sum)
if sum.HasData || sum.LastWalk != nil || sum.VLANs == nil {
t.Errorf("a device the job has not walked: %+v", sum)
}
}
func TestEdgeLimitComesFromTheLiveSetting(t *testing.T) {
limit := 8
e := newEnvDeps(t, 0, func(d *api.Deps) { d.L2EdgeMax = func() int { return limit } })
seedL2Net(t, e)
if h := searchL2(t, e, host).Items[0]; h.Locations[1].UplinkReason == "" || h.Locations[0].Device.Name != "access1" {
t.Fatalf("baseline = %+v", h.Locations)
}
limit = 20 // the distribution port now counts as an edge port: it carries 10 MACs
h := searchL2(t, e, host).Items[0]
edge := 0
for _, l := range h.Locations {
if l.Edge {
edge++
}
}
if edge != 2 || h.Locations[0].Device.Name != "access1" {
t.Errorf("with a limit of 20 the 10-MAC port is an edge port, ranked after the 1-MAC one: %+v", h.Locations)
}
}
func TestPollerHealthCarriesL2StatusAndNotes(t *testing.T) {
e := newEnvDeps(t, 0, func(d *api.Deps) {
d.L2Health = func() *api.L2Health { return &api.L2Health{OK: 3, TimedOut: 1, Skipped: []string{"slow1"}} }
d.HealthNotes = func() []api.HealthNote {
return []api.HealthNote{{Code: "oui_unavailable", Message: "no IEEE vendor registry is loaded"}}
}
})
var h api.PollerHealth
e.getJSON(t, "/api/v1/poller/health", &h)
if h.L2 == nil || h.L2.TimedOut != 1 || len(h.Notes) != 1 || h.Notes[0].Code != "oui_unavailable" {
t.Errorf("health = %+v", h)
}
plain := newEnv(t, 0)
h = api.PollerHealth{}
plain.getJSON(t, "/api/v1/poller/health", &h)
if h.L2 != nil || h.Notes == nil || len(h.Notes) != 0 {
t.Errorf("a server without the L2 job: %+v", h)
}
}

146
internal/api/l2_types.go Normal file
View File

@@ -0,0 +1,146 @@
package api
import "time"
// VendorLookup names the vendor of a MAC from the IEEE registry. It is optional: with none, vendor is empty
// everywhere. *oui.Service implements it.
type VendorLookup interface {
// Vendor is the registered vendor of a canonical MAC, "Private / randomised" for a locally administered
// one, and "" when unknown.
Vendor(mac string) string
// Match returns a predicate for MACs whose vendor contains the (lower-case) substring.
Match(sub string) func(mac string) bool
// Available reports whether a registry is loaded.
Available() bool
}
// L2IfaceRef is the port a MAC was learned on.
type L2IfaceRef struct {
ID string `json:"id" doc:"Interface ULID."`
IfName string `json:"ifname"`
}
// L2IP is an address an ARP or IPv6 neighbour table maps to a MAC.
type L2IP struct {
IP string `json:"ip" doc:"Canonical text; IPv6 compressed."`
Device DeviceRef `json:"device" doc:"The device whose table holds the entry (usually the gateway)."`
LastSeen time.Time `json:"last_seen"`
}
// L2Location is one place a MAC was seen: a port of a device in a VLAN.
type L2Location struct {
Device DeviceRef `json:"device"`
Iface *L2IfaceRef `json:"iface,omitempty" doc:"Absent when the bridge port has no ifIndex."`
VLAN int64 `json:"vlan" doc:"0 when the device's BRIDGE-MIB table does not say."`
// VLANName is empty when the device reports no name for the VLAN.
VLANName string `json:"vlan_name"`
// Edge is true for a port where a host is plugged in: not an uplink. A port is an uplink when it carries more MACs
// than l2.edge_max_macs, has an LLDP neighbour, or is a LAG.
Edge bool `json:"edge"`
// UplinkReason is why the port is not an edge port: macs, lldp or lag; empty for an edge port.
UplinkReason string `json:"uplink_reason"`
// PortMACCount is how many MACs the port carried in the walk this row was last seen in.
PortMACCount int64 `json:"port_mac_count"`
// Current is true when the device's last walk saw this row.
Current bool `json:"current"`
FirstSeen time.Time `json:"first_seen"`
LastSeen time.Time `json:"last_seen"`
}
// L2Hit is one MAC with the addresses it answers to and the places it was seen, edge ports first.
type L2Hit struct {
MAC string `json:"mac" doc:"Lower-case aa:bb:cc:dd:ee:ff."`
// Vendor is the IEEE-registered vendor, "Private / randomised" for a locally administered MAC, empty when
// unknown or when no registry is loaded.
Vendor string `json:"vendor"`
MACKind string `json:"mac_kind" enum:"global,local,multicast"`
IPs []L2IP `json:"ips"`
Locations []L2Location `json:"locations" doc:"Edge ports first (current before history, then fewest MACs, then newest), then uplinks."`
}
// L2SearchResult is a page of MAC hits.
type L2SearchResult struct {
Items []L2Hit `json:"items"`
Page int64 `json:"page"`
PerPage int64 `json:"per_page"`
Total int64 `json:"total"`
// Kind says how q was read: mac, prefix, ip, cidr or vendor.
Kind string `json:"kind" enum:"mac,prefix,ip,cidr,vendor"`
// VendorsAvailable is false when no IEEE registry is loaded, in which case a vendor: search finds nothing.
VendorsAvailable bool `json:"vendors_available"`
}
// L2FDBRow is a row of a device's forwarding table.
type L2FDBRow struct {
MAC string `json:"mac"`
Vendor string `json:"vendor"`
MACKind string `json:"mac_kind" enum:"global,local,multicast"`
VLAN int64 `json:"vlan"`
VLANName string `json:"vlan_name"`
Iface *L2IfaceRef `json:"iface,omitempty"`
IPs []string `json:"ips" doc:"Addresses any ARP or neighbour table maps this MAC to."`
Current bool `json:"current"`
FirstSeen time.Time `json:"first_seen"`
LastSeen time.Time `json:"last_seen"`
}
// L2FDBList is a page of L2FDBRow.
type L2FDBList struct {
Items []L2FDBRow `json:"items"`
Page int64 `json:"page"`
PerPage int64 `json:"per_page"`
Total int64 `json:"total"`
}
// L2ARPRow is a row of a device's ARP or IPv6 neighbour table.
type L2ARPRow struct {
IP string `json:"ip"`
MAC string `json:"mac"`
Vendor string `json:"vendor"`
MACKind string `json:"mac_kind" enum:"global,local,multicast"`
Iface *L2IfaceRef `json:"iface,omitempty"`
Current bool `json:"current"`
FirstSeen time.Time `json:"first_seen"`
LastSeen time.Time `json:"last_seen"`
}
// L2ARPList is a page of L2ARPRow.
type L2ARPList struct {
Items []L2ARPRow `json:"items"`
Page int64 `json:"page"`
PerPage int64 `json:"per_page"`
Total int64 `json:"total"`
}
// L2VLAN is a VLAN of a device with its name.
type L2VLAN struct {
VLAN int64 `json:"vlan"`
Name string `json:"name"`
}
// L2Summary says what the L2 job knows about a device; the device page shows its L2 tab only when there is something.
type L2Summary struct {
HasData bool `json:"has_data" doc:"True when the device has any FDB or ARP row, current or historical."`
LastWalk *time.Time `json:"last_walk" nullable:"true"`
Source string `json:"source" doc:"What supplied the FDB: qbridge, bridge or per-vlan."`
FDBCurrent int64 `json:"fdb_current"`
FDBTotal int64 `json:"fdb_total"`
ARPCurrent int64 `json:"arp_current"`
ARPTotal int64 `json:"arp_total"`
VLANs []L2VLAN `json:"vlans"`
EdgeMaxMACs int64 `json:"edge_max_macs" doc:"l2.edge_max_macs: ports with more MACs are uplinks."`
}
// L2Port is how many MACs a port carries now, for the Ports tab.
type L2Port struct {
Iface L2IfaceRef `json:"iface"`
MACs int64 `json:"macs" doc:"MACs seen on the port by the device's last walk."`
// Uplink is true when the port has more MACs than l2.edge_max_macs, an LLDP neighbour, or is a LAG.
Uplink bool `json:"uplink"`
UplinkReason string `json:"uplink_reason"`
}
// L2PortList lists the ports of a device that carry MACs.
type L2PortList struct {
Items []L2Port `json:"items"`
}

144
internal/api/overview.go Normal file
View File

@@ -0,0 +1,144 @@
package api
import (
"context"
"net/http"
"time"
"github.com/danielgtaylor/huma/v2"
"github.com/v10networks/nmon/internal/store"
)
var overviewRanges = map[string]time.Duration{"24h": 24 * time.Hour, "7d": 7 * 24 * time.Hour, "30d": 30 * 24 * time.Hour}
func (s *server) registerOverview(api huma.API) {
huma.Register(api, huma.Operation{
OperationID: "get-device-overview", Method: http.MethodGet, Path: "/api/v1/devices/{slug}/overview", Tags: []string{"devices"},
Summary: "Data of the device Overview tab",
Description: "Aggregate traffic (the sum over the device's non-excluded, non-retired, reliable ports; one query per direction), " +
"port counts, availability over 24h, 7d, 30d and 1y from device_down alerts, ICMP round-trip time and the last 20 events. " +
"The step rules are those of /graph, limited to the hot store. If the metric store cannot answer, traffic and rtt are " +
"empty and partial says so.",
Errors: []int{http.StatusNotFound, http.StatusUnprocessableEntity},
}, s.overview)
}
type overviewIn struct {
Slug string `path:"slug" doc:"Device slug."`
Range string `query:"range" enum:"24h,7d,30d" default:"24h" doc:"Window ending now."`
}
func (s *server) overview(ctx context.Context, in *overviewIn) (*struct{ Body OverviewResponse }, error) {
d, err := s.Store.GetDeviceBySlug(ctx, in.Slug)
if err != nil {
return nil, s.storeErr(err)
}
if d.PurgedAt != nil {
return nil, huma.Error404NotFound("not found")
}
now := s.Clock.Now()
ifs, err := s.Store.ListInterfaces(ctx, d.ID, false)
if err != nil {
return nil, s.storeErr(err)
}
resp := OverviewResponse{Range: in.Range, Ports: portCounts(ifs), Traffic: OverviewTraffic{In: []Point{}, Out: []Point{}}, RTT: []Point{}}
if err := s.overviewHistory(ctx, &resp, d, ifs, now); err != nil {
return nil, s.storeErr(err)
}
s.overviewMetrics(ctx, &resp, d, ifs, now)
return &struct{ Body OverviewResponse }{resp}, nil
}
// overviewHistory fills availability and events from the inventory.
func (s *server) overviewHistory(ctx context.Context, resp *OverviewResponse, d *store.Device, ifs []store.Interface, now time.Time) error {
rows, err := s.Store.ListDeviceDownAlerts(ctx, d.Slug, now.Add(-year))
if err != nil {
return err
}
resp.Availability = availabilityOf(rows, now, d.CreatedAt)
alerts, _, err := s.Store.ListAlerts(ctx, store.AlertFilter{Device: d.Slug}, store.Page{Limit: overviewEvents})
if err != nil {
return err
}
objects := []string{d.Slug}
for _, i := range ifs {
objects = append(objects, i.ID)
}
audit, err := s.Store.ListAuditByObjects(ctx, objects, overviewEvents)
if err != nil {
return err
}
resp.Events = mergeEvents(alerts, audit)
return nil
}
// overviewMetrics fills the traffic and rtt series; a metric store that cannot answer leaves them empty.
func (s *server) overviewMetrics(ctx context.Context, resp *OverviewResponse, d *store.Device, ifs []store.Interface, now time.Time) {
plan, err := s.metricPlan(&metricGraphIn{From: now.Add(-overviewRanges[resp.Range]), To: now})
if err != nil {
resp.Partial = append(resp.Partial, "range")
return
}
resp.StepSeconds = int64(plan.newest().step / time.Second)
grid := gridOf(plan)
step := plan.newest().step
hc, soft := countedIfaces(ifs, &resp.Traffic)
for _, dir := range []string{dirIn, dirOut} {
if len(hc)+len(soft) == 0 {
break
}
vals, err := s.fetchMetricSeries(ctx, trafficExpr(hc, soft, dir, step), plan)
if err != nil {
resp.Partial = append(resp.Partial, "traffic")
resp.Traffic.In, resp.Traffic.Out = []Point{}, []Point{}
break
}
if dir == dirIn {
resp.Traffic.In = pointsOf(grid, vals)
} else {
resp.Traffic.Out = pointsOf(grid, vals)
}
}
rtt, err := s.fetchMetricSeries(ctx, `nmon_rtt_seconds{device=`+quote(d.Slug)+`}`, plan)
if err != nil {
resp.Partial = append(resp.Partial, "rtt")
return
}
if len(rtt) > 0 {
resp.RTT = pointsOf(grid, rtt)
}
}
// countedIfaces splits the ports that count toward traffic by counter width and counts the unreliable ones left out.
func countedIfaces(ifs []store.Interface, t *OverviewTraffic) (hc, soft []string) {
for _, i := range ifs {
switch {
case i.RetiredAt != nil || i.Excluded:
case i.Unreliable:
t.UnreliableExcluded++
case i.HasHCCounters:
hc = append(hc, i.ID)
default:
soft = append(soft, i.ID)
}
}
return hc, soft
}
// trafficExpr is the summed bits per second of dir; HC counters where a port has them, else the 32-bit ones.
func trafficExpr(hc, soft []string, dir string, step time.Duration) string {
names := map[string][2]string{dirIn: {"ifHCInOctets", "ifInOctets"}, dirOut: {"ifHCOutOctets", "ifOutOctets"}}[dir]
win := secs(max(step, minRateWindow))
var parts []string
for n, ids := range [][]string{hc, soft} {
if len(ids) > 0 {
parts = append(parts, `rate(`+names[n]+`{iface=~`+ifaceMatcher(ids)+`}[`+win+`])*8`)
}
}
switch len(parts) {
case 1:
return `sum(` + parts[0] + `)`
}
return `sum(` + parts[0] + ` or ` + parts[1] + `)`
}

View File

@@ -0,0 +1,145 @@
package api
import (
"fmt"
"sort"
"time"
"github.com/v10networks/nmon/internal/store"
)
const (
adminDown = 2
operUp = 1
overviewEvents = 20
year = 365 * 24 * time.Hour
)
// availabilityPeriods are the keys of the availability block and their lengths.
var availabilityPeriods = []time.Duration{24 * time.Hour, 7 * 24 * time.Hour, 30 * 24 * time.Hour, year}
type span struct{ from, to time.Time }
// downtime sums the time inside [from, to] covered by the incidents; overlaps count once.
func downtime(incidents []span, from, to time.Time) time.Duration {
clipped := make([]span, 0, len(incidents))
for _, s := range incidents {
s.from, s.to = laterOf(s.from, from), earlierOf(s.to, to)
if s.from.Before(s.to) {
clipped = append(clipped, s)
}
}
sort.Slice(clipped, func(i, j int) bool { return clipped[i].from.Before(clipped[j].from) })
var total time.Duration
var end time.Time
for _, s := range clipped {
if s.from.Before(end) {
s.from = end
}
if s.to.After(s.from) {
total += s.to.Sub(s.from)
end = s.to
}
}
return total
}
func laterOf(a, b time.Time) time.Time {
if a.After(b) {
return a
}
return b
}
func earlierOf(a, b time.Time) time.Time {
if a.Before(b) {
return a
}
return b
}
// availabilityPct is 100 x (1 - downtime/period) over the period ending at now, counted from the device's creation
// if that is later; nil when the device has existed for no time.
func availabilityPct(incidents []span, now, created time.Time, period time.Duration) *float64 {
start := laterOf(now.Add(-period), created)
if !start.Before(now) {
return nil
}
v := 100 * (1 - float64(downtime(incidents, start, now))/float64(now.Sub(start)))
return &v
}
// incidentSpans turns device_down rows into spans; an open row lasts until now.
func incidentSpans(rows []store.Alert, now time.Time) []span {
out := make([]span, 0, len(rows))
for _, a := range rows {
if a.FiredAt == nil {
continue
}
end := now
if a.ClearedAt != nil {
end = *a.ClearedAt
}
out = append(out, span{from: *a.FiredAt, to: end})
}
return out
}
func availabilityOf(rows []store.Alert, now, created time.Time) OverviewAvailability {
sp := incidentSpans(rows, now)
at := func(i int) *float64 { return availabilityPct(sp, now, created, availabilityPeriods[i]) }
return OverviewAvailability{Day: at(0), Week: at(1), Month: at(2), Year: at(3)}
}
// portCounts classifies the ports that count: admin down first, then oper up, else down.
func portCounts(ifs []store.Interface) OverviewPorts {
var p OverviewPorts
for _, i := range ifs {
if i.RetiredAt != nil || i.Excluded {
continue
}
p.Total++
switch {
case i.AdminStatus == adminDown:
p.AdminDown++
case i.OperStatus == operUp:
p.Up++
default:
p.Down++
}
}
return p
}
// mergeEvents turns alerts (a fired and a resolved line each) and audit entries into the newest-first list.
func mergeEvents(alerts []store.Alert, audit []store.AuditEntry) []OverviewEvent {
ev := []OverviewEvent{}
for _, a := range alerts {
if a.FiredAt != nil {
ev = append(ev, OverviewEvent{At: a.FiredAt.UTC(), Kind: "alert", Severity: a.Severity, Summary: "Fired: " + a.Summary})
}
if a.ClearedAt != nil {
sum := closedWord(a.State) + ": " + a.Summary
ev = append(ev, OverviewEvent{At: a.ClearedAt.UTC(), Kind: "alert", Severity: a.Severity, Summary: sum})
}
}
for _, e := range audit {
ev = append(ev, OverviewEvent{At: e.At.UTC(), Kind: "audit", Summary: auditSummary(e)})
}
sort.SliceStable(ev, func(i, j int) bool { return ev[i].At.After(ev[j].At) })
return ev[:min(len(ev), overviewEvents)]
}
func closedWord(state string) string {
if state == store.AlertRetired {
return "Retired"
}
return "Resolved"
}
func auditSummary(e store.AuditEntry) string {
return fmt.Sprintf("%s by %s", e.Action, e.Actor)
}
// ifaceMatcher is a regexp alternation of interface ids, for an iface=~ matcher.
func ifaceMatcher(ids []string) string { return quote(regexAlt(ids)) }

View File

@@ -0,0 +1,149 @@
package api_test
import (
"errors"
"strings"
"testing"
"time"
"github.com/v10networks/nmon/internal/api"
"github.com/v10networks/nmon/internal/store"
)
func overviewEnv(t *testing.T) (*env, *store.Device) {
e := newEnvDeps(t, 0, withShipped(t))
d := e.device(t, "sw1", "yyz", "access")
return e, d
}
func hoursAgo(e *env, n float64) *time.Time {
x := e.clk.Now().Add(-time.Duration(n * float64(time.Hour)))
return &x
}
func TestOverviewShape(t *testing.T) {
e, d := overviewEnv(t)
e.iface(t, d, 1, "up1", "")
e.iface(t, d, 2, "dn1", "", func(i *store.Interface) { i.OperStatus = 2 })
e.iface(t, d, 3, "ad1", "", func(i *store.Interface) { i.AdminStatus, i.OperStatus = 2, 2 })
e.iface(t, d, 4, "soft", "", func(i *store.Interface) { i.HasHCCounters = false })
e.iface(t, d, 5, "bad", "", func(i *store.Interface) { i.Unreliable = true })
e.iface(t, d, 6, "ex", "", func(i *store.Interface) { i.Excluded = true })
now := e.clk.Now().Unix()
e.q.fn = func(_, expr string, _, _ time.Time, _ time.Duration) ([]byte, error) {
if strings.HasPrefix(expr, "nmon_rtt") {
return matrix(mrow{map[string]string{}, map[int64]float64{now - now%300: 0.002}}), nil
}
return matrix(mrow{map[string]string{}, map[int64]float64{now - now%300: 5e6}}), nil
}
var got api.OverviewResponse
e.getJSON(t, "/api/v1/devices/"+d.Slug+"/overview?range=7d", &got)
if got.Range != "7d" || got.StepSeconds != 660 || got.Ports != (api.OverviewPorts{Total: 5, Up: 3, Down: 1, AdminDown: 1}) {
t.Fatalf("got %+v", got)
}
if got.Traffic.UnreliableExcluded != 1 || len(got.Traffic.In) == 0 || len(got.Traffic.Out) != len(got.Traffic.In) || len(got.RTT) != len(got.Traffic.In) {
t.Fatalf("traffic %d/%d rtt %d excl %d", len(got.Traffic.In), len(got.Traffic.Out), len(got.RTT), got.Traffic.UnreliableExcluded)
}
var traffic []string
for _, c := range e.q.queries() {
if strings.HasPrefix(c.expr, "sum(") {
traffic = append(traffic, c.expr)
}
}
if len(traffic) != 2 || !strings.Contains(traffic[0], "ifHCInOctets") || !strings.Contains(traffic[0], "ifInOctets{") ||
strings.Contains(traffic[0], "ifOutOctets") || !strings.Contains(traffic[1], "ifHCOutOctets") {
t.Fatalf("traffic queries = %q", traffic)
}
if got.Availability.Day != nil || got.Events == nil {
t.Errorf("availability %+v events %v", got.Availability, got.Events)
}
}
func TestOverviewHotStoreDownIsPartial(t *testing.T) {
e, d := overviewEnv(t)
e.iface(t, d, 1, "up1", "")
e.q.err = errors.New("boom")
e.q.fn = nil
var got api.OverviewResponse
e.getJSON(t, "/api/v1/devices/"+d.Slug+"/overview", &got)
if got.Range != "24h" || got.Ports.Total != 1 || len(got.Traffic.In) != 0 || len(got.RTT) != 0 || len(got.Partial) == 0 {
t.Fatalf("got %+v", got)
}
isProblem(t, e.do("GET", "/api/v1/devices/nope/overview", ""), 404)
isProblem(t, e.do("GET", "/api/v1/devices/"+d.Slug+"/overview?range=9d", ""), 422)
}
func TestOverviewAvailability(t *testing.T) {
e, d := overviewEnv(t)
ctx := t.Context()
e.clk.Advance(48 * time.Hour) // the device is two days old
add := func(i int, fired, cleared *time.Time) {
a := &store.Alert{Rule: "device_down", DeviceSlug: d.Slug, SeriesKey: "device_down|" + d.Slug + "|" + string(rune('a'+i)),
State: store.AlertResolved, BaseState: store.AlertResolved, FiredAt: fired, ClearedAt: cleared, FlapChanges: "[]", Severity: "critical", Summary: "down"}
if err := e.st.UpsertAlertState(ctx, a); err != nil {
t.Fatal(err)
}
}
add(0, hoursAgo(e, 10), hoursAgo(e, 8)) // 2h
add(1, hoursAgo(e, 9), hoursAgo(e, 7)) // overlaps: union 10h..7h = 3h
add(2, hoursAgo(e, 1), nil) // open: 1h
add(3, hoursAgo(e, 300), hoursAgo(e, 100)) // before the device existed: no effect
var got api.OverviewResponse
e.getJSON(t, "/api/v1/devices/"+d.Slug+"/overview", &got)
a := got.Availability
day := 100 * (1 - 4.0/24)
age := 100 * (1 - 4.0/48)
if a.Day == nil || *a.Day < day-0.001 || *a.Day > day+0.001 {
t.Errorf("24h = %v want %v", a.Day, day)
}
for name, v := range map[string]*float64{"7d": a.Week, "30d": a.Month, "1y": a.Year} { // clipped to created_at: 48h
if v == nil || *v < age-0.001 || *v > age+0.001 {
t.Errorf("%s = %v want %v", name, v, age)
}
}
if len(got.Events) != 5 && len(got.Events) != 7 {
t.Errorf("events = %+v", got.Events)
}
}
func TestOverviewEventsMergeNewestFirst(t *testing.T) {
e, d := overviewEnv(t)
i := e.iface(t, d, 1, "up1", "")
ctx := t.Context()
a := &store.Alert{Rule: "device_down", DeviceSlug: d.Slug, SeriesKey: "k", State: store.AlertResolved, BaseState: store.AlertResolved,
FiredAt: hoursAgo(e, 3), ClearedAt: hoursAgo(e, 1), FlapChanges: "[]", Severity: "critical", Summary: "sw1 down"}
if err := e.st.UpsertAlertState(ctx, a); err != nil {
t.Fatal(err)
}
_ = e.st.AppendAudit(ctx, "ann", "interface.alert_down", i.ID, nil, nil)
_ = e.st.AppendAudit(ctx, "ann", "device.edit", "other", nil, nil)
var got api.OverviewResponse
e.getJSON(t, "/api/v1/devices/"+d.Slug+"/overview", &got)
if len(got.Events) != 3 {
t.Fatalf("events = %+v", got.Events)
}
kinds := got.Events[0].Kind + got.Events[1].Kind + got.Events[2].Kind
if kinds != "auditalertalert" || got.Events[1].Severity != "critical" || got.Events[0].Severity != "" ||
!strings.HasPrefix(got.Events[2].Summary, "Fired") || !strings.HasPrefix(got.Events[1].Summary, "Resolved") {
t.Fatalf("events = %+v", got.Events)
}
}
func TestMetricGraphAggregate(t *testing.T) {
e := newEnvDeps(t, 0, withShipped(t))
d := e.device(t, "srv1", "yyz", "server")
now := e.clk.Now().Unix()
e.q.body = matrix(mrow{map[string]string{}, map[int64]float64{now - 600: 40}})
var got api.MetricGraphResponse
e.getJSON(t, "/api/v1/devices/"+d.Slug+"/metrics/graph?metric=hrProcessorLoad&aggregate=avg", &got)
if want := `avg(hrProcessorLoad{device="` + d.Slug + `"})`; e.q.q != want {
t.Errorf("expr %q want %q", e.q.q, want)
}
if len(got.Series) == 0 {
t.Error("no series")
}
e.getJSON(t, "/api/v1/devices/"+d.Slug+"/metrics/graph?metric=hrProcessorLoad&aggregate=max&component=CPU+0", &got)
if !strings.Contains(e.q.q, `component="CPU 0"`) || strings.Contains(e.q.q, "max(") {
t.Errorf("explicit component must win: %q", e.q.q)
}
}

View File

@@ -0,0 +1,46 @@
package api
import "time"
// OverviewTraffic is the device's aggregate traffic over the range.
type OverviewTraffic struct {
In []Point `json:"in" doc:"[unix_s, bits per second|null] pairs: the sum over the device's counted ports."`
Out []Point `json:"out"`
UnreliableExcluded int `json:"unreliable_excluded" doc:"Ports left out of the sums because their counters are unreliable."`
}
// OverviewPorts counts the device's ports by state.
type OverviewPorts struct {
Total int `json:"total"`
Up int `json:"up"`
Down int `json:"down" doc:"Administratively up but not operationally up."`
AdminDown int `json:"admin_down"`
}
// OverviewAvailability is the percent of time the device was not down over each period; null for no time yet.
type OverviewAvailability struct {
Day *float64 `json:"24h" nullable:"true"`
Week *float64 `json:"7d" nullable:"true"`
Month *float64 `json:"30d" nullable:"true"`
Year *float64 `json:"1y" nullable:"true"`
}
// OverviewEvent is one line of the recent-events list.
type OverviewEvent struct {
At time.Time `json:"at"`
Kind string `json:"kind" enum:"alert,audit"`
Severity string `json:"severity,omitempty" enum:"critical,warning,info"`
Summary string `json:"summary"`
}
// OverviewResponse is the data of the device Overview tab.
type OverviewResponse struct {
Range string `json:"range" enum:"24h,7d,30d"`
StepSeconds int64 `json:"step_seconds"`
Traffic OverviewTraffic `json:"traffic"`
Ports OverviewPorts `json:"ports"`
Availability OverviewAvailability `json:"availability"`
RTT []Point `json:"rtt" doc:"[unix_s, seconds|null] pairs of ICMP round-trip time; empty when ICMP is off."`
Events []OverviewEvent `json:"events" doc:"The last 20 alert transitions and audit entries of the device or its ports."`
Partial []string `json:"partial,omitempty" doc:"Parts that could not be read, for example the metric store."`
}

View File

@@ -41,9 +41,8 @@ func Routes() []Route {
// adminReads are GET routes only an admin may call: they reveal who can reach nmon or how it is configured.
var adminReads = []string{"GET /api/v1/tokens", "GET /api/v1/settings", "GET /api/v1/audit"}
// viewerPosts are non-GET routes that only read, so a viewer may call them. None today; a query endpoint taking
// a body would be listed here.
var viewerPosts []string
// viewerPosts are non-GET routes that only read, so a viewer may call them (a query endpoint taking a body).
var viewerPosts = []string{"POST /api/v1/alert-policies/preview"}
// DeclarePolicy states the minimum role of each route that departs from the default (GET and HEAD need viewer,
// everything else admin).

192
internal/api/printers.go Normal file
View File

@@ -0,0 +1,192 @@
package api
import (
"context"
"math"
"net/http"
"sort"
"strings"
"github.com/danielgtaylor/huma/v2"
"github.com/v10networks/nmon/internal/store"
)
const (
supplyTable = "Printer-MIB::prtMarkerSuppliesTable"
// supplyWarnPercent is the level under which a supply reads as low, as the printer_supply_low rule does.
supplyWarnPercent = 10
)
func (s *server) registerPrinters(api huma.API) {
huma.Register(api, huma.Operation{
OperationID: "list-printers", Method: http.MethodGet, Path: "/api/v1/printers", Tags: []string{"devices"},
Summary: "Printer fleet: supplies, trays and page counts",
Description: "Every device whose profile is or extends printer, with its supply and paper tray levels from the hot store. " +
"The printer with the lowest supply comes first; printers with no reading come last.",
Errors: []int{http.StatusTooManyRequests, http.StatusServiceUnavailable},
}, s.listPrinters)
}
// printerNames are the metrics the fleet view reads.
var printerNames = []string{
"prtMarkerSuppliesLevel", "prtMarkerSuppliesMaxCapacity", "prtInputCurrentLevel", "prtInputMaxCapacity", "prtMarkerLifeCount",
}
// printerDevices lists the devices whose profile chain includes printer.
func (s *server) printerDevices(ctx context.Context) ([]store.Device, error) {
if s.Profiles == nil {
return nil, nil
}
var out []store.Device
for off := int64(0); ; off += store.MaxLimit {
rows, n, err := s.Store.ListDevices(ctx, store.DeviceFilter{}, store.Page{Offset: off, Limit: store.MaxLimit})
if err != nil {
return nil, err
}
for _, d := range rows {
if res := s.Profiles.Get(d.Profile); res != nil && containsString(res.Chain, "printer") {
out = append(out, d)
}
}
if off+store.MaxLimit >= n {
return out, nil
}
}
}
func containsString(l []string, s string) bool {
for _, x := range l {
if x == s {
return true
}
}
return false
}
type printerEntry struct {
dev store.Device
row PrinterRow
lowest float64 // lowest supply percent; +Inf with no reading
sentinel map[int]string
}
// supplyState: empty is critical, under supplyWarnPercent a warning, a sentinel or no reading unknown or ok by its words.
func supplyState(pct *float64, text *string) string {
switch {
case pct != nil && *pct <= 0:
return "crit"
case pct != nil && *pct < supplyWarnPercent:
return "warn"
case pct != nil:
return "ok"
case text != nil && *text != "unknown":
return "ok"
}
return "unknown"
}
// levels builds the rows of one level/capacity metric pair, lowest first.
func levels(rd readings, level, capacity string, sent map[int]string) []PrinterSupply {
out := []PrinterSupply{}
for comp, lv := range rd[level] {
sup := PrinterSupply{Component: comp}
if words, ok := sent[int(lv.v)]; ok && lv.v == math.Trunc(lv.v) {
sup.Text = &words
} else if c := rd.at(capacity, comp); c != nil && c.v > 0 && lv.v >= 0 {
p := 100 * lv.v / c.v
sup.Percent = &p
}
sup.State = supplyState(sup.Percent, sup.Text)
out = append(out, sup)
}
sort.Slice(out, func(i, j int) bool {
a, b := out[i].Percent, out[j].Percent
switch {
case a != nil && b != nil && *a != *b:
return *a < *b
case (a == nil) != (b == nil):
return a != nil
}
return out[i].Component < out[j].Component
})
return out
}
func (s *server) printerEntry(d store.Device, rd readings) printerEntry {
sent := map[int]string{}
if res := s.Profiles.Get(d.Profile); res != nil {
if m, ok := res.Metric("prtMarkerSuppliesLevel"); ok {
sent = m.Sentinels
}
}
e := printerEntry{dev: d, lowest: math.Inf(1), sentinel: sent}
e.row = PrinterRow{
Device: DeviceRefSite{Slug: d.Slug, Name: d.Name, Site: d.Site},
Supplies: levels(rd, "prtMarkerSuppliesLevel", "prtMarkerSuppliesMaxCapacity", sent),
Trays: levels(rd, "prtInputCurrentLevel", "prtInputMaxCapacity", sent),
}
for _, sup := range e.row.Supplies {
if sup.Percent != nil {
e.lowest = min(e.lowest, *sup.Percent)
}
}
var pages float64
for _, c := range rd["prtMarkerLifeCount"] {
pages += c.v
e.row.PagesTotal = &pages
}
return e
}
func (s *server) listPrinters(ctx context.Context, in *PageParams) (*struct{ Body PrinterList }, error) {
devs, err := s.printerDevices(ctx)
if err != nil {
return nil, s.storeErr(err)
}
var all map[string]readings
if len(devs) > 0 {
slugs := make([]string, len(devs))
for i, d := range devs {
slugs[i] = d.Slug
}
if all, err = s.queryLatest(ctx, `device=~`+quote(regexAlt(slugs)), printerNames); err != nil {
return nil, err
}
}
entries := make([]printerEntry, len(devs))
reach := s.Reachability.Reachability(ctx, devs)
for i, d := range devs {
entries[i] = s.printerEntry(d, all[d.Slug])
entries[i].row.Reachability = reach[d.Slug]
}
sort.SliceStable(entries, func(i, j int) bool {
if entries[i].lowest != entries[j].lowest {
return entries[i].lowest < entries[j].lowest
}
return strings.ToLower(entries[i].dev.Name) < strings.ToLower(entries[j].dev.Name)
})
lo := min(in.offset(), int64(len(entries)))
hi := min(lo+in.limit(), int64(len(entries)))
out := PrinterList{Items: []PrinterRow{}, Page: in.pageNo(), PerPage: in.limit(), Total: int64(len(entries))}
for _, e := range entries[lo:hi] {
if err := s.addSupplyAttrs(ctx, &e); err != nil {
return nil, s.storeErr(err)
}
out.Items = append(out.Items, e.row)
}
return &struct{ Body PrinterList }{out}, nil
}
// addSupplyAttrs fills each supply's type and color from the discovery attributes of its row.
func (s *server) addSupplyAttrs(ctx context.Context, e *printerEntry) error {
attrs, err := s.componentAttrs(ctx, e.dev.ID)
if err != nil {
return err
}
for i := range e.row.Supplies {
a := attrs[attrKey(supplyTable, e.row.Supplies[i].Component)]
e.row.Supplies[i].Type, e.row.Supplies[i].Color = a["type"], a["color"]
}
return nil
}

143
internal/api/profiles.go Normal file
View File

@@ -0,0 +1,143 @@
package api
import (
"context"
"fmt"
"net/http"
"github.com/danielgtaylor/huma/v2"
"github.com/v10networks/nmon/internal/discovery"
"github.com/v10networks/nmon/internal/profile"
"github.com/v10networks/nmon/internal/store"
)
// ProfileMatchRules are a profile's own match rules (they are not inherited through extends).
type ProfileMatchRules struct {
SysObjectID []string `json:"sysObjectID" doc:"sysObjectID prefixes, matched on a dot boundary."`
SysDescr []string `json:"sysDescr" doc:"Regular expressions tried against sysDescr."`
}
// ProfileView is one profile as GET /profiles lists it.
type ProfileView struct {
Name string `json:"name"`
Priority int `json:"priority" doc:"The highest priority wins when several profiles match a device."`
Extends string `json:"extends,omitempty"`
Source string `json:"source" enum:"shipped,operator" doc:"operator: the file comes from the operator's profiles directory."`
Match ProfileMatchRules `json:"match"`
}
// ProfileMatchView says how a device's profile was chosen.
type ProfileMatchView struct {
MatchedBy string `json:"matched_by" enum:"sysObjectID,sysDescr,default,pinned" doc:"The rule that picks it."`
GenericReason string `json:"generic_reason,omitempty" doc:"Why no vendor profile applies, when the profile is generic."`
}
func (s *server) registerProfiles(api huma.API) {
huma.Register(api, huma.Operation{
OperationID: "list-profiles", Method: http.MethodGet, Path: "/api/v1/profiles", Tags: []string{"profiles"},
Summary: "List the loaded device profiles with their match rules",
}, s.listProfiles)
}
func (s *server) listProfiles(context.Context, *struct{}) (*struct{ Body []ProfileView }, error) {
out := []ProfileView{}
if s.Profiles != nil {
for _, n := range s.Profiles.Names() {
r, _ := s.Profiles.Lookup(n)
v := ProfileView{
Name: r.Name, Priority: r.Priority, Source: "shipped",
Match: ProfileMatchRules{SysObjectID: nonNil(r.Match.SysObjectID), SysDescr: nonNil(r.Match.SysDescr)},
}
if r.Operator {
v.Source = "operator"
}
if len(r.Chain) > 1 {
v.Extends = r.Chain[1]
}
out = append(out, v)
}
}
return &struct{ Body []ProfileView }{out}, nil
}
// profileMatch explains the device's profile choice; nil when no profiles are loaded.
func (s *server) profileMatch(d *store.Device) *ProfileMatchView {
if d.ProfilePinned {
return &ProfileMatchView{MatchedBy: "pinned"}
}
if s.Profiles == nil {
return nil
}
_, by := s.Profiles.MatchWhy(d.SysObjectID, d.SysDescr)
v := &ProfileMatchView{MatchedBy: string(by)}
if by == profile.ByDefault {
if d.SysObjectID == "" {
v.GenericReason = "not discovered yet: no sysObjectID to match"
} else {
v.GenericReason = fmt.Sprintf("no vendor profile matched sysObjectID %s or sysDescr", d.SysObjectID)
}
}
return v
}
// planProfile applies the body's profile to the plan: a name pins it, "" returns the device to automatic
// matching (the profile is re-matched from the stored sysObjectID/sysDescr now and again at each discovery).
func (s *server) planProfile(p *editPlan, dev *store.Device, name string) error {
if name == "" {
p.after.ProfilePinned, p.after.Profile = false, s.autoProfile(dev)
p.change["profile"] = "automatic"
} else {
if err := s.knownProfile(name); err != nil {
return err
}
p.after.Profile, p.after.ProfilePinned = name, true
p.change["profile"] = name
}
p.rediscover = p.after.Profile != dev.Profile || p.after.ProfilePinned != dev.ProfilePinned
if !p.rediscover {
delete(p.change, "profile")
}
return nil
}
func (s *server) knownProfile(name string) error {
if s.Profiles == nil {
return nil
}
if _, ok := s.Profiles.Lookup(name); !ok {
return huma.Error422UnprocessableEntity("unknown profile " + name)
}
return nil
}
// autoProfile is the profile Match picks for the device's stored sysObjectID and sysDescr, or its current one
// when nothing was discovered yet.
func (s *server) autoProfile(dev *store.Device) string {
if s.Profiles == nil || (dev.SysObjectID == "" && dev.SysDescr == "") {
return dev.Profile
}
if m := s.Profiles.Match(dev.SysObjectID, dev.SysDescr); m != nil {
return m.Name
}
return dev.Profile
}
// rediscoverAfterEdit runs one discovery so the new profile's tables are read at once. A device that cannot be
// reached keeps its edit; the next scheduled run applies it.
func (s *server) rediscoverAfterEdit(ctx context.Context, slug string) {
dev, err := s.Store.GetDeviceBySlug(ctx, slug)
if err != nil || dev.RetiredAt != nil || dev.Kind != store.KindNetwork || dev.SNMPCredentialID == nil {
return
}
dctx, cancel := context.WithTimeout(ctx, s.DiscoveryTimeout)
defer cancel()
deps := discovery.Deps{Store: s.Store, Profiles: s.Profiles, Clock: s.Clock, SNMPPort: s.SNMPPort, Logger: s.Logger}
if _, err := discovery.Discover(dctx, deps, dev); err != nil {
s.Logger.Warn("discovery after profile change failed", "device", slug, "err", err)
return
}
if s.AfterDiscover != nil {
s.AfterDiscover(dev.ID)
}
}

View File

@@ -0,0 +1,79 @@
package api_test
import (
"context"
"net/http"
"testing"
"github.com/v10networks/nmon/internal/api"
"github.com/v10networks/nmon/internal/profile"
"github.com/v10networks/nmon/internal/store"
)
func profileEnv(t *testing.T) *env {
t.Helper()
set, err := profile.Load("")
if err != nil {
t.Fatal(err)
}
return newEnvDeps(t, 0, func(d *api.Deps) { d.Profiles = set })
}
func TestListProfiles(t *testing.T) {
e := profileEnv(t)
var got []api.ProfileView
e.getJSON(t, "/api/v1/profiles", &got)
by := map[string]api.ProfileView{}
for _, p := range got {
by[p.Name] = p
}
es, ok := by["ubiquiti-edgeswitch"]
if !ok || es.Source != "shipped" || es.Priority != 30 || len(es.Match.SysObjectID) != 1 || es.Extends != "generic" {
t.Fatalf("edgeswitch = %+v", es)
}
if _, ok := by["generic"]; !ok || len(by) < 15 {
t.Fatalf("%d profiles", len(by))
}
}
func TestPutDeviceProfile(t *testing.T) {
e := profileEnv(t)
d := e.device(t, "sw", "", "", func(d *store.Device) { d.SysObjectID, d.SysDescr = "1.3.6.1.4.1.4413", "EdgeSwitch 12 Fiber" })
var v api.DeviceView
e.getJSON(t, "/api/v1/devices/"+d.Slug, &v)
if v.ProfilePinned || v.ProfileMatch == nil || v.ProfileMatch.MatchedBy != "sysObjectID" {
t.Fatalf("view %+v %+v", v, v.ProfileMatch)
}
if rec := e.do(http.MethodPut, "/api/v1/devices/"+d.Slug, `{"profile":"nope"}`); rec.Code != http.StatusUnprocessableEntity {
t.Fatalf("unknown profile: %d %s", rec.Code, rec.Body)
}
if rec := e.do(http.MethodPut, "/api/v1/devices/"+d.Slug, `{"profile":"vyos"}`); rec.Code != http.StatusOK {
t.Fatalf("pin: %d %s", rec.Code, rec.Body)
}
e.getJSON(t, "/api/v1/devices/"+d.Slug, &v)
if v.Profile != "vyos" || !v.ProfilePinned || v.ProfileMatch.MatchedBy != "pinned" {
t.Fatalf("after pin %+v", v)
}
if rec := e.do(http.MethodPut, "/api/v1/devices/"+d.Slug, `{"profile":""}`); rec.Code != http.StatusOK {
t.Fatalf("auto: %d %s", rec.Code, rec.Body)
}
e.getJSON(t, "/api/v1/devices/"+d.Slug, &v)
if v.Profile != "ubiquiti-edgeswitch" || v.ProfilePinned {
t.Fatalf("after unpin %+v", v)
}
audits, _, err := e.st.ListAudit(context.Background(), store.Page{Limit: 10})
if err != nil || len(audits) < 2 || audits[0].Action != "device.edit" {
t.Fatalf("audit rows %d, %v", len(audits), err)
}
}
func TestGenericDeviceExplainsWhy(t *testing.T) {
e := profileEnv(t)
d := e.device(t, "odd", "", "", func(d *store.Device) { d.SysObjectID, d.SysDescr = "1.3.6.1.4.1.99999.1", "mystery box" })
var v api.DeviceView
e.getJSON(t, "/api/v1/devices/"+d.Slug, &v)
if v.ProfileMatch == nil || v.ProfileMatch.MatchedBy != "default" ||
v.ProfileMatch.GenericReason != "no vendor profile matched sysObjectID 1.3.6.1.4.1.99999.1 or sysDescr" {
t.Fatalf("%+v", v.ProfileMatch)
}
}

41
internal/api/retire.go Normal file
View File

@@ -0,0 +1,41 @@
package api
import (
"context"
"net/http"
"github.com/danielgtaylor/huma/v2"
)
func (s *server) registerRetire(api huma.API) {
huma.Register(api, huma.Operation{
OperationID: "retire-device", Method: http.MethodPost, Path: "/api/v1/devices/{slug}/retire", Tags: []string{"devices"},
Summary: "Retire a device",
Description: "Admin only. Soft-deletes the device and its interfaces: polling stops, history is kept, and adding the " +
"same address again offers to restore it. Retiring a retired device is a 409.",
Errors: []int{http.StatusNotFound, http.StatusConflict},
}, s.retireDevice)
}
func (s *server) retireDevice(ctx context.Context, in *struct {
Slug string `path:"slug" doc:"Device slug."`
}) (*deviceOut, error) {
d, err := s.Store.GetDeviceBySlug(ctx, in.Slug)
if err != nil {
return nil, s.storeErr(err)
}
if d.PurgedAt != nil {
return nil, huma.Error404NotFound("not found")
}
if d.RetiredAt != nil {
return nil, huma.Error409Conflict("device is already retired")
}
if err := s.Store.RetireDevice(ctx, d.ID); err != nil {
return nil, s.storeErr(err)
}
if err := s.Store.AppendAudit(ctx, actorOf(ctx), "device.retire", d.Slug, nil, map[string]any{"name": d.Name}); err != nil {
s.Logger.Warn("audit device.retire", "err", err)
}
s.inventoryChanged(ctx)
return s.view(ctx, d.Slug)
}

View File

@@ -0,0 +1,24 @@
package api_test
import (
"net/http"
"testing"
)
func TestRetireDevice(t *testing.T) {
e := newEnv(t, 0)
d := e.device(t, "Old Box", "", "")
rec := e.do("POST", "/api/v1/devices/"+d.Slug+"/retire", "")
if rec.Code != http.StatusOK {
t.Fatalf("retire: %d %s", rec.Code, rec.Body)
}
got, err := e.st.GetDeviceBySlug(t.Context(), d.Slug)
if err != nil || got.RetiredAt == nil {
t.Fatalf("not retired: %+v %v", got, err)
}
if e.ref.n.Load() == 0 {
t.Error("poller was not refreshed")
}
isProblem(t, e.do("POST", "/api/v1/devices/"+d.Slug+"/retire", ""), http.StatusConflict)
isProblem(t, e.do("POST", "/api/v1/devices/nope/retire", ""), http.StatusNotFound)
}

View File

@@ -0,0 +1,45 @@
package app
import (
"context"
"log/slog"
"sync"
"time"
"github.com/v10networks/nmon/internal/clock"
"github.com/v10networks/nmon/internal/store"
)
// optInSet answers alert.Options.OptedIn from the store's effective opt-in set (a port's own flag, or a group
// holding it now with the flag on). The set is loaded once per entityTTL, and API writes invalidate it.
type optInSet struct {
st store.AlertDown
cache *ttlCache[struct{}, map[string]bool]
log *slog.Logger
mu sync.Mutex
last map[string]bool // the last set that loaded; used while the store fails
}
func newOptInSet(st store.AlertDown, clk clock.Clock, ttl time.Duration, log *slog.Logger) *optInSet {
return &optInSet{st: st, cache: newTTLCache[struct{}, map[string]bool](clk, ttl), log: log}
}
// opted reports whether the interface is opted in to interface_down alerts. While the store fails it keeps
// answering from the last good set, so a database hiccup neither fires nor closes alerts.
func (o *optInSet) opted(iface string) bool {
set, err := o.cache.get(struct{}{}, func() (map[string]bool, error) {
return o.st.InterfacesAlertDown(context.Background())
})
o.mu.Lock()
defer o.mu.Unlock()
if err != nil {
o.log.Warn("load alert-down opt-in set", "err", err)
return o.last[iface]
}
o.last = set
return set[iface]
}
// invalidate drops the cached set; the API calls it after a write that can change it.
func (o *optInSet) invalidate() { o.cache.invalidate() }

View File

@@ -0,0 +1,52 @@
package app
import (
"context"
"errors"
"io"
"log/slog"
"testing"
"time"
"github.com/v10networks/nmon/internal/clock"
"github.com/v10networks/nmon/internal/store"
)
type fakeAlertDown struct {
set map[string]bool
err error
loads int
}
func (f *fakeAlertDown) SetInterfaceAlertDown(context.Context, []string, bool) error { return nil }
func (f *fakeAlertDown) InterfaceAlertDownVia(context.Context, []string) (map[string]string, error) {
return nil, nil
}
func (f *fakeAlertDown) InterfacesAlertDown(context.Context) (map[string]bool, error) {
f.loads++
return f.set, f.err
}
var _ store.AlertDown = (*fakeAlertDown)(nil)
func TestOptInSetCachesInvalidatesAndSurvivesStoreErrors(t *testing.T) {
clk := clock.NewFake(time.Date(2026, 10, 23, 12, 0, 0, 0, time.UTC))
f := &fakeAlertDown{set: map[string]bool{"p1": true}}
o := newOptInSet(f, clk, time.Minute, slog.New(slog.NewTextHandler(io.Discard, nil)))
if !o.opted("p1") || o.opted("p2") || f.loads != 1 {
t.Fatalf("first answers wrong or loaded %d times", f.loads)
}
f.set = map[string]bool{"p2": true}
if !o.opted("p1") {
t.Fatal("answer changed inside the TTL")
}
o.invalidate()
if o.opted("p1") || !o.opted("p2") {
t.Fatal("invalidate did not reload")
}
clk.Advance(2 * time.Minute)
f.err = errors.New("db down")
if !o.opted("p2") {
t.Fatal("a failing store must keep answering from the last good set")
}
}

View File

@@ -26,10 +26,12 @@ const entityTTL = time.Minute
// queue, the reachability evaluator, the metric-rule evaluator and the
// maintenance windows they consult.
type alerting struct {
queue *notify.Queue
reach *reach.Evaluator
rules *alert.Evaluator
maint *maintenanceCache
queue *notify.Queue
reach *reach.Evaluator
rules *alert.Evaluator
maint *maintenanceCache
optIn *optInSet
policies *policyEngine
nt *notifier
sys *sysAlarms
@@ -54,9 +56,12 @@ func (a *App) newAlerting(s *settings) error {
devices := newTTLCache[string, store.Device](clk, entityTTL)
n := newNotifier(q, s.alerting.Receivers, a.st, devices, a.log)
maint := newMaintenanceCache(a.st, clk, a.log)
a.alerting = &alerting{queue: q, maint: maint, devices: devices}
optIn := newOptInSet(a.st, clk, entityTTL, a.log)
pe := newPolicyEngine(a.st, clk, optIn, s.alerting.Rules, a.log)
a.alerting = &alerting{queue: q, maint: maint, devices: devices, optIn: optIn, policies: pe}
a.alerting.reach = reach.New(reach.Options{
Store: a.st, Notifier: reachNotifier{n}, Clock: clk, Log: a.log, PollInterval: s.PollInterval, ICMPOff: s.ICMP == config.ICMPOff,
Store: a.st, Notifier: reachNotifier{n}, Clock: clk, Log: a.log, Wants: pe.Wants,
PollInterval: s.PollInterval, ICMPOff: s.ICMP == config.ICMPOff,
ICMPBlocked: func(d *store.Device) bool { return a.icmpBlocked(d) },
InMaintenance: func(slug string, _ time.Time) bool {
_, ok := maint.Covered(slug)
@@ -64,7 +69,7 @@ func (a *App) newAlerting(s *settings) error {
},
})
a.alerting.nt = n
a.alerting.sys = newSysAlarms(a.st, clk, alertNotifier{n}, a.log)
a.alerting.sys = newSysAlarms(a.st, clk, alertNotifier{n}, pe.SystemWanted, a.log)
a.policy = pol
if s.ICMP == config.ICMPOff {
a.log.Info("ICMP probing is off (--icmp off): SNMP alone decides whether a device is down; " +
@@ -94,7 +99,7 @@ func (a *App) startAlerting(s *settings) error {
var err error
al.rules, err = alert.New(alert.Options{
Store: a.st, Querier: vmsup.NewQuerier(a.sup), Notifier: alertNotifier{al.nt}, Clock: clock.Real(), Rules: s.alerting.Rules,
Groups: groupLookup{a.st}, Unreliable: a.unreliable(), Active: a.entityActive(), Suppressed: a.suppressed(),
Groups: groupLookup{a.st}, Unreliable: a.unreliable(), Policy: al.policies.Metric, Active: a.entityActive(), Suppressed: a.suppressed(),
StoreHealth: storeHealth(), Interval: s.alerting.Interval, Logger: a.log,
})
if err != nil {

View File

@@ -25,14 +25,23 @@ func (a *App) apiDeps(s *settings) api.Deps {
HotRetention: s.cfg.Retention.Hot.D(), FiveMinuteRetention: s.cfg.Retention.FiveMinute.D(), LongRetention: s.cfg.Retention.Long.D(),
Reachability: a.alerting.reach, DeviceWarnings: a.alerting.reach.Warnings,
Dependencies: depGraph{a.alerting.reach}, OnMaintenanceChange: a.alerting.maint.Refresh,
OnAlertDownChange: a.alertDownChanged, AlertPolicy: a.alerting.policies,
Tokens: a.svc.tokens, Secrets: a.st, Rollup: a.svc.rollup, P95: a.svc.p95, Groups: a.svc.groups, Zones: a.svc.zones,
Checks: a.svc.checks, Proxmox: a.svc.proxmox,
APITokens: a.auth.Tokens, Settings: a, Backups: a.backups, Stores: a.sup,
Series: seriesDeleter{a.sup}, PurgeAfter: func() time.Duration { return a.live.get().PurgeAfter() },
OnImported: a.discoverNew,
OnImported: a.discoverNew, Vendors: a.vendors(), L2EdgeMax: a.l2EdgeMax, L2Health: a.l2Health, HealthNotes: a.healthNotes,
}
}
// vendors is the vendor lookup for the API, or nil (an interface holding a nil *oui.Service is not nil).
func (a *App) vendors() api.VendorLookup {
if a.oui == nil {
return nil
}
return a.oui
}
// discoverNew starts discovery for freshly imported devices.
func (a *App) discoverNew(ids []int64) {
for _, id := range ids {
@@ -133,3 +142,9 @@ func (a *App) OnSIGHUP() {
}
}
}
// alertDownChanged drops the caches an alert_down flag, a group flag or a membership change makes stale.
func (a *App) alertDownChanged() {
a.alerting.optIn.invalidate()
a.alerting.policies.Changed()
}

View File

@@ -20,8 +20,10 @@ import (
"github.com/v10networks/nmon/internal/discovery"
"github.com/v10networks/nmon/internal/diskguard"
"github.com/v10networks/nmon/internal/ingest"
"github.com/v10networks/nmon/internal/l2"
"github.com/v10networks/nmon/internal/metrics"
"github.com/v10networks/nmon/internal/netpolicy"
"github.com/v10networks/nmon/internal/oui"
"github.com/v10networks/nmon/internal/poller"
"github.com/v10networks/nmon/internal/profile"
"github.com/v10networks/nmon/internal/store"
@@ -59,6 +61,8 @@ type App struct {
alerting *alerting
policy *netpolicy.Policy
svc services
l2 *l2.Job // the MAC/ARP/FDB table job
oui *oui.Service // the optional IEEE vendor registry
ingester *ingest.Ingester
serveErr chan error // the HTTP server stopped on its own
@@ -92,14 +96,17 @@ func (a *App) onStop(name string, fn func(ctx context.Context) error) {
// Run starts nmon and blocks until ctx is cancelled or the HTTP server fails,
// then shuts everything down gracefully.
func Run(ctx context.Context, f config.Flags) error {
// Catch SIGHUP before starting: Go's default action is to exit, and startup takes
// seconds (metric stores), so a `systemctl reload` then would kill nmon. A SIGHUP
// that arrives during startup is queued and applied once Start returns.
hup := make(chan os.Signal, 1)
signal.Notify(hup, syscall.SIGHUP)
defer signal.Stop(hup)
a, err := Start(ctx, f)
if err != nil {
return err
}
var runErr error
hup := make(chan os.Signal, 1)
signal.Notify(hup, syscall.SIGHUP)
defer signal.Stop(hup)
wait:
for {
select {
@@ -122,7 +129,7 @@ wait:
//
// Startup order: inventory, metric stores, alerting (built), write queues and the
// disk guard, profiles, poller, alerting (started), discovery scheduler, trap receiver,
// rollup, p95 and group jobs, Proxmox sync and checks, push ingest (HTTP and Graphite),
// rollup, p95 and group jobs, the MAC/ARP table job and vendor registry, Proxmox sync and checks, push ingest (HTTP and Graphite),
// authentication, HTTP API, the nightly backup schedule.
// Stop reverses it.
func Start(ctx context.Context, f config.Flags) (_ *App, err error) {
@@ -168,10 +175,7 @@ func Start(ctx context.Context, f config.Flags) (_ *App, err error) {
a.newTraps(s)
a.startScheduler()
a.startTraps(s)
if err = a.startRollup(s); err != nil {
return nil, err
}
if err = a.startCollectors(s); err != nil {
if err = a.startJobs(s); err != nil {
return nil, err
}
if err = a.startIngest(s); err != nil {
@@ -188,6 +192,17 @@ func Start(ctx context.Context, f config.Flags) (_ *App, err error) {
return a, nil
}
// startJobs starts the storage and collection jobs that follow the poller: rollup with the p95 jobs, the MAC/ARP
// table job with the vendor registry, then Proxmox sync and service checks.
func (a *App) startJobs(s *settings) error {
for _, start := range []func(*settings) error{a.startRollup, a.startL2, a.startCollectors} {
if err := start(s); err != nil {
return err
}
}
return nil
}
// Addr is the address the API listens on.
func (a *App) Addr() string { return a.ln.Addr().String() }

59
internal/app/bgp_alert.go Normal file
View File

@@ -0,0 +1,59 @@
package app
import (
"context"
"encoding/json"
"fmt"
"github.com/v10networks/nmon/internal/alert"
"github.com/v10networks/nmon/internal/notify"
)
// ruleBGPPeerDown is the built-in rule whose notifications name the peer's AS and description.
const ruleBGPPeerDown = "bgp_peer_down"
// describePeer adds the peer's AS, router id and description to a bgp_peer_down notification. The SNMP label
// contract allows only device and component on a series, so these are component attributes kept from discovery
// and are joined here, at delivery, instead of being carried in the rule's template.
func (n *notifier) describePeer(ctx context.Context, note *notify.Notification, t alert.Transition) {
if t.Rule != ruleBGPPeerDown || t.Component == "" {
return
}
dev, err := n.st.GetDeviceBySlug(ctx, t.Device)
if err != nil {
return
}
comps, err := n.st.ListComponents(ctx, dev.ID)
if err != nil {
return
}
for _, c := range comps {
if c.Label != t.Component {
continue
}
var attrs map[string]string
if json.Unmarshal([]byte(c.Attrs), &attrs) != nil || (attrs["remote_as"] == "" && attrs["description"] == "") {
continue
}
detail := peerDetail(attrs)
note.Summary += " " + detail
note.Description += " " + detail
for _, k := range []string{"remote_as", "local_as", "router_id", "description"} {
if attrs[k] != "" {
note.Labels[k] = attrs[k]
}
}
return
}
}
// peerDetail renders "(AS 65001, transit-a)".
func peerDetail(attrs map[string]string) string {
switch {
case attrs["remote_as"] != "" && attrs["description"] != "":
return fmt.Sprintf("(AS %s, %s)", attrs["remote_as"], attrs["description"])
case attrs["remote_as"] != "":
return fmt.Sprintf("(AS %s)", attrs["remote_as"])
}
return fmt.Sprintf("(%s)", attrs["description"])
}

94
internal/app/l2wire.go Normal file
View File

@@ -0,0 +1,94 @@
package app
import (
"fmt"
"strings"
"time"
"github.com/v10networks/nmon/internal/api"
"github.com/v10networks/nmon/internal/clock"
"github.com/v10networks/nmon/internal/config"
"github.com/v10networks/nmon/internal/l2"
"github.com/v10networks/nmon/internal/oui"
)
// ouiHTTPTimeout bounds one registry download through the outbound allow-list.
const ouiHTTPTimeout = 2 * time.Minute
// startL2 starts the MAC/ARP/FDB table job and the optional IEEE vendor registry. Both read their settings from the
// live configuration on every pass, so a reload applies without a restart. Neither can stop nmon from starting.
func (a *App) startL2(s *settings) error {
clk := clock.Real()
a.oui = oui.New(oui.Options{
Dir: config.OUIDir(s.DataDir), Clock: clk, Logger: a.log,
Settings: func() oui.Settings {
c := a.live.get().OUI
return oui.Settings{Fetch: c.FetchEnabled(), Refresh: c.RefreshOrDefault()}
},
Client: a.policy.HTTPClient(ouiHTTPTimeout),
})
job, err := l2.New(l2.Options{
Store: a.st, Profiles: a.profiles, Clock: clk, Logger: a.log, Queue: a.hot, Paused: a.guard.PauseRollup,
SNMPPort: s.SNMPPort, Settings: a.l2Settings,
})
if err != nil {
return fmt.Errorf("l2 job: %w", err)
}
a.l2 = job
a.goStop("oui registry", a.oui.Run)
a.goStop("l2 job", job.Run)
return nil
}
// l2Settings is the L2 job's view of the live configuration.
func (a *App) l2Settings() l2.Settings {
c := a.live.get().L2
return l2.Settings{
Interval: c.IntervalOrDefault(), Keep: c.KeepOrDefault(), Concurrency: c.Workers(), DeviceTimeout: c.DeviceTimeoutOrDefault(),
}
}
// l2EdgeMax is l2.edge_max_macs from the live configuration.
func (a *App) l2EdgeMax() int { return a.live.get().L2.EdgeMax() }
// l2Health adapts the job's status for the poller health page.
func (a *App) l2Health() *api.L2Health {
if a.l2 == nil {
return nil
}
st := a.l2.Status()
h := &api.L2Health{
Running: st.Running, IntervalS: int64(st.Interval.Seconds()), OK: int64(st.OK), TimedOut: int64(st.Timeouts),
Failed: int64(st.Errors), Skipped: []string{},
}
if !st.LastEnd.IsZero() {
t := st.LastEnd.UTC()
h.LastCycle = &t
}
for _, d := range st.Devices {
if d.Result != "ok" {
h.Skipped = append(h.Skipped, d.Slug)
}
}
return h
}
// healthNotes lists conditions for the poller health page that are not alerts: no alert is ever raised for them.
func (a *App) healthNotes() []api.HealthNote {
var out []api.HealthNote
if !a.oui.Available() {
why := "no IEEE vendor registry is loaded, so MAC vendors are not shown"
if !a.live.get().OUI.FetchEnabled() {
why += " (oui.fetch is off; load one with `nmon oui import`)"
} else if st := a.oui.Status(); st.LastError != "" {
why += ": " + st.LastError
}
out = append(out, api.HealthNote{Code: "oui_unavailable", Message: why})
}
if h := a.l2Health(); h != nil && len(h.Skipped) > 0 {
out = append(out, api.HealthNote{Code: "l2_devices_skipped",
Message: fmt.Sprintf("the last MAC/ARP table cycle skipped %d device(s) that did not answer in time or failed: %s",
len(h.Skipped), strings.Join(h.Skipped, ", "))})
}
return out
}

View File

@@ -122,6 +122,7 @@ func (a alertNotifier) Notify(ctx context.Context, t alert.Transition) error {
if t.Component != "" {
note.Labels["component"] = t.Component
}
a.describePeer(ctx, &note, t)
a.device(&note, t.Device)
if t.Iface != "" {
note.Interface.ID, note.Interface.Name = t.Iface, t.Iface

277
internal/app/policies.go Normal file
View File

@@ -0,0 +1,277 @@
package app
import (
"context"
"errors"
"fmt"
"log/slog"
"sync"
"time"
"github.com/v10networks/nmon/internal/alert"
"github.com/v10networks/nmon/internal/alertpolicy"
"github.com/v10networks/nmon/internal/clock"
"github.com/v10networks/nmon/internal/store"
)
// policyRefresh is how often the policy table is re-read; an API write invalidates it at once.
const policyRefresh = 30 * time.Second
// policyStore is what the policy engine reads.
type policyStore interface {
store.AlertPolicies
store.Devices
store.Interfaces
store.Groups
store.AlertDown
}
// policyEngine decides where alerts fire. It caches the compiled policy set (30 s, or until an API write) and
// the device, port and group lookups a Subject is built from, and keeps the last good answer while the store
// fails so a database hiccup neither fires nor closes alerts.
type policyEngine struct {
st policyStore
log *slog.Logger
optIn *optInSet
rules []alert.Rule
set *ttlCache[struct{}, *alertpolicy.Set]
devices *ttlCache[string, store.Device]
ifaces *ttlCache[string, store.Interface]
groups *ttlCache[struct{}, map[string][]string]
devGroups *ttlCache[string, []string]
clk clock.Clock
mu sync.Mutex
lastSet *alertpolicy.Set
lastDev map[string]store.Device // last good lookups, used while the store fails
lastIf map[string]store.Interface // (a database hiccup must not close alerts)
warned bool
}
func newPolicyEngine(st policyStore, clk clock.Clock, optIn *optInSet, rules []alert.Rule, log *slog.Logger) *policyEngine {
return &policyEngine{
st: st, log: log, optIn: optIn, rules: rules, clk: clk,
lastDev: map[string]store.Device{}, lastIf: map[string]store.Interface{},
set: newTTLCache[struct{}, *alertpolicy.Set](clk, policyRefresh),
devices: newTTLCache[string, store.Device](clk, entityTTL),
ifaces: newTTLCache[string, store.Interface](clk, entityTTL),
groups: newTTLCache[struct{}, map[string][]string](clk, entityTTL),
devGroups: newTTLCache[string, []string](clk, entityTTL),
}
}
// Changed drops every cache, so a policy, group or flag write applies on the next cycle.
func (p *policyEngine) Changed() {
p.set.invalidate()
p.groups.invalidate()
p.devGroups.invalidate()
p.ifaces.invalidate()
p.devices.invalidate()
}
// current returns the compiled policy set. While the store fails it keeps the last one that loaded.
func (p *policyEngine) current() *alertpolicy.Set {
set, err := p.set.get(struct{}{}, func() (*alertpolicy.Set, error) { return p.load(context.Background()) })
p.mu.Lock()
defer p.mu.Unlock()
if err != nil {
p.log.Warn("load alert policies", "err", err)
if p.lastSet == nil {
p.lastSet, _ = alertpolicy.NewSet(nil)
}
return p.lastSet
}
p.lastSet = set
return set
}
func (p *policyEngine) load(ctx context.Context) (*alertpolicy.Set, error) {
rows, err := p.st.ListAlertPolicies(ctx)
if err != nil {
return nil, err
}
policies := make([]alertpolicy.Policy, 0, len(rows))
for _, r := range rows {
pol, err := policyOfRow(r)
if err != nil {
p.log.Error("alert policy unreadable, skipped", "policy", r.Name, "err", err)
continue
}
policies = append(policies, pol)
}
set, err := alertpolicy.NewSet(policies)
if err != nil {
p.log.Error("alert policy does not compile", "err", err)
}
p.warnIfEmpty(len(rows))
return set, nil
}
// warnIfEmpty says once that an empty policy table means silence.
func (p *policyEngine) warnIfEmpty(n int) {
p.mu.Lock()
defer p.mu.Unlock()
if n == 0 && !p.warned {
p.warned = true
p.log.Warn("no alert policies: nothing will alert; add policies under Alerts > Alert policies")
}
}
func policyOfRow(r store.AlertPolicy) (alertpolicy.Policy, error) {
scope, err := alertpolicy.ParseScope(r.Scope)
if err != nil {
return alertpolicy.Policy{}, err
}
pol := alertpolicy.Policy{ID: r.ID, Name: r.Name, Rule: r.Rule, Action: r.Action, Priority: r.Priority, Enabled: r.Enabled,
Severity: r.Severity, Threshold: r.Threshold, Scope: scope}
if r.ForSeconds != nil {
d := time.Duration(*r.ForSeconds) * time.Second
pol.For = &d
}
return pol, nil
}
// Metric is alert.Options.Policy.
func (p *policyEngine) Metric(e alert.Entity) (alertpolicy.Binding, bool) {
subj, ok := p.subject(e.Device, e.Iface)
if !ok {
return alertpolicy.Binding{}, false
}
return p.current().Decide(e.Rule, subj)
}
// Wants is reach.Options.Wants.
func (p *policyEngine) Wants(rule, slug string) bool {
subj, ok := p.subject(slug, "")
if !ok {
return false
}
_, ok = p.current().Decide(rule, subj)
return ok
}
// SystemWanted reports whether a system policy binds subsystem alarms. They have no device, so only a scope of
// all can match.
func (p *policyEngine) SystemWanted() bool {
_, ok := p.current().Decide(alertpolicy.RuleSystem, alertpolicy.Subject{})
return ok
}
// subject builds what policies match against; ok false means the device or port is unknown or unreadable.
func (p *policyEngine) subject(slug, iface string) (alertpolicy.Subject, bool) {
d, err := p.devices.get(slug, func() (store.Device, error) {
dev, err := p.st.GetDeviceBySlug(context.Background(), slug)
return remember(p, p.lastDev, slug, dev, err)
})
if err != nil {
p.logLookup("device", slug, err)
return alertpolicy.Subject{}, false
}
subj := alertpolicy.Subject{Device: deviceSubject(d)}
if iface == "" {
subj.Groups = p.deviceGroups(d)
return subj, true
}
i, err := p.ifaces.get(iface, func() (store.Interface, error) {
x, err := p.st.GetInterface(context.Background(), iface)
return remember(p, p.lastIf, iface, x, err)
})
if err != nil {
p.logLookup("interface", iface, err)
return alertpolicy.Subject{}, false
}
port := ifaceSubject(i, p.optIn.opted(i.ID))
subj.Iface = &port
subj.Groups = p.ifaceGroups()[i.ID]
return subj, true
}
func (p *policyEngine) logLookup(what, key string, err error) {
if !errors.Is(err, store.ErrNotFound) {
p.log.Warn("alert policy lookup failed", "what", what, "key", key, "err", err)
}
}
func deviceSubject(d store.Device) alertpolicy.Device {
return alertpolicy.Device{Slug: d.Slug, Profile: d.Profile, Role: d.Role, Site: d.Site, Kind: d.Kind}
}
func ifaceSubject(i store.Interface, alertDown bool) alertpolicy.Iface {
speed := i.IfHighSpeed
if speed == 0 {
speed = i.IfSpeed / 1_000_000
}
return alertpolicy.Iface{Type: i.IfType, Name: i.IfName, Alias: i.IfAlias, SpeedMbps: speed, AlertDown: alertDown}
}
// ifaceGroups maps each interface id to the ids of the groups holding it now.
func (p *policyEngine) ifaceGroups() map[string][]string {
m, err := p.groups.get(struct{}{}, func() (map[string][]string, error) { return p.loadGroups(context.Background(), p.clk.Now()) })
if err != nil {
p.log.Warn("load group membership for alert policies", "err", err)
return nil
}
return m
}
func (p *policyEngine) loadGroups(ctx context.Context, now time.Time) (map[string][]string, error) {
members, err := p.st.GroupMembers(ctx, 0, now, time.Time{})
if err != nil {
return nil, err
}
out := map[string][]string{}
for _, m := range members {
if m.Covers(now) {
out[m.InterfaceID] = append(out[m.InterfaceID], fmt.Sprint(m.GroupID))
}
}
return out, nil
}
// deviceGroups is the union of the groups of the device's ports.
func (p *policyEngine) deviceGroups(d store.Device) []string {
g, err := p.devGroups.get(d.Slug, func() ([]string, error) {
ifs, err := p.st.ListInterfaces(context.Background(), d.ID, false)
if err != nil {
return nil, err
}
return unionGroups(ifs, p.ifaceGroups()), nil
})
if err != nil {
p.log.Warn("load device groups for alert policies", "device", d.Slug, "err", err)
return nil
}
return g
}
func unionGroups(ifs []store.Interface, byIface map[string][]string) []string {
seen := map[string]bool{}
var out []string
for _, i := range ifs {
for _, g := range byIface[i.ID] {
if !seen[g] {
seen[g] = true
out = append(out, g)
}
}
}
return out
}
// remember records a successful lookup and, when the store fails with anything but "not found", answers from the
// last good one so a database hiccup neither fires nor closes alerts.
func remember[V any](p *policyEngine, last map[string]V, key string, v *V, err error) (V, error) {
p.mu.Lock()
defer p.mu.Unlock()
if err == nil {
last[key] = *v
return *v, nil
}
if old, ok := last[key]; ok && !errors.Is(err, store.ErrNotFound) {
return old, nil
}
var zero V
return zero, err
}

View File

@@ -0,0 +1,142 @@
package app
import (
"context"
"strconv"
"testing"
"time"
"github.com/v10networks/nmon/internal/alert"
"github.com/v10networks/nmon/internal/alertpolicy"
"github.com/v10networks/nmon/internal/clock"
"github.com/v10networks/nmon/internal/store"
"github.com/v10networks/nmon/internal/testutil"
"github.com/v10networks/nmon/internal/ulid"
)
type policyEnv struct {
t *testing.T
st store.Store
clk *clock.Fake
pe *policyEngine
dev *store.Device
}
func newPolicyEnv(t *testing.T) *policyEnv {
t.Helper()
ctx := context.Background()
clk := clock.NewFake(time.Date(2026, 10, 23, 12, 0, 0, 0, time.UTC))
st := testutil.OpenStore(t, clk)
d := &store.Device{Name: "sw1", Kind: store.KindNetwork, Role: "access", Site: "hq", Enabled: true}
if err := st.CreateDevice(ctx, d); err != nil {
t.Fatal(err)
}
e := &policyEnv{t: t, st: st, clk: clk, dev: d}
e.pe = newPolicyEngine(st, clk, newOptInSet(st, clk, entityTTL, testLogger()), alert.DefaultRules(), testLogger())
return e
}
func (e *policyEnv) iface(idx int64, name, alias string, typ int64) string {
e.t.Helper()
id := ulid.New(e.clk.Now())
i := store.Interface{ID: id, DeviceID: e.dev.ID, IfIndex: idx, IfName: name, IfAlias: alias, IfType: typ, AdminStatus: 1, OperStatus: 1}
if err := e.st.UpsertInterfaces(context.Background(), e.dev.ID, []store.Interface{i}); err != nil {
e.t.Fatal(err)
}
return id
}
func (e *policyEnv) policy(name, rule, action string, prio int, scope alertpolicy.Scope) {
e.t.Helper()
p := &store.AlertPolicy{Name: name, Rule: rule, Action: action, Priority: prio, Enabled: true, Scope: alertpolicy.FormatScope(scope)}
if err := e.st.CreateAlertPolicy(context.Background(), p); err != nil {
e.t.Fatal(err)
}
e.pe.Changed()
}
func TestNoPoliciesMeansNothingAlertsAnywhere(t *testing.T) {
e := newPolicyEnv(t)
id := e.iface(1, "e1", "", 6)
if _, ok := e.pe.Metric(alert.Entity{Rule: "interface_errors", Device: e.dev.Slug, Iface: id}); ok {
t.Fatal("metric rule bound without policies")
}
if e.pe.Wants("device_down", e.dev.Slug) || e.pe.Wants("device_warning", e.dev.Slug) {
t.Fatal("reachability wanted without policies")
}
if e.pe.SystemWanted() {
t.Fatal("system alarms wanted without policies")
}
}
func TestPoliciesDecideByTypeAliasGroupAndBellAndApplyAfterChanged(t *testing.T) {
e := newPolicyEnv(t)
fiber := e.iface(1, "e1", "Core fiber", 6)
wifi := e.iface(2, "w1", "Core radio", 71)
e.policy("fiber", "interface_errors", alertpolicy.ActionAlert, 0,
alertpolicy.Scope{Interfaces: alertpolicy.IfaceMatch{IfTypes: []int64{6}, IfAlias: "(?i)fib|core"}})
e.policy("wifi", "interface_errors", alertpolicy.ActionIgnore, 10, alertpolicy.Scope{Interfaces: alertpolicy.IfaceMatch{IfTypes: []int64{71}}})
ent := func(id string) alert.Entity {
return alert.Entity{Rule: "interface_errors", Device: e.dev.Slug, Iface: id}
}
if _, ok := e.pe.Metric(ent(fiber)); !ok {
t.Fatal("fiber port not bound")
}
if _, ok := e.pe.Metric(ent(wifi)); ok {
t.Fatal("wifi port bound")
}
// A group matcher and the bell matcher follow live changes once Changed is called.
g := &store.Group{Name: "vip", Kind: store.GroupCustom}
if err := e.st.CreateGroup(context.Background(), g); err != nil {
t.Fatal(err)
}
yes := true
e.policy("vip", "interface_down", alertpolicy.ActionAlert, 0, alertpolicy.Scope{Groups: []string{strconv.FormatInt(g.ID, 10)}})
e.policy("bell", "interface_down", alertpolicy.ActionAlert, 0,
alertpolicy.Scope{Interfaces: alertpolicy.IfaceMatch{AlertDown: &yes}})
down := alert.Entity{Rule: "interface_down", Device: e.dev.Slug, Iface: wifi}
if _, ok := e.pe.Metric(down); ok {
t.Fatal("bound before membership or bell")
}
if err := e.st.AddGroupMember(context.Background(), store.GroupMember{GroupID: g.ID, InterfaceID: wifi, Direction: store.DirToward}); err != nil {
t.Fatal(err)
}
e.pe.Changed()
if _, ok := e.pe.Metric(down); !ok {
t.Fatal("group member not bound")
}
if !e.pe.Wants("interface_down", e.dev.Slug) {
t.Fatal("a device with a member port should match a group policy")
}
if err := e.st.SetInterfaceAlertDown(context.Background(), []string{fiber}, true); err != nil {
t.Fatal(err)
}
e.pe.optIn.invalidate()
e.pe.Changed()
if _, ok := e.pe.Metric(alert.Entity{Rule: "interface_down", Device: e.dev.Slug, Iface: fiber}); !ok {
t.Fatal("belled port not bound")
}
}
func TestSystemPolicyAndPreview(t *testing.T) {
e := newPolicyEnv(t)
e.iface(1, "e1", "uplink", 6)
e.iface(2, "w1", "", 71)
e.policy("sys", alertpolicy.RuleSystem, alertpolicy.ActionAlert, 0, alertpolicy.Scope{All: true})
if !e.pe.SystemWanted() {
t.Fatal("system policy ignored")
}
p, err := e.pe.Preview(context.Background(), "interface_errors",
alertpolicy.Scope{Interfaces: alertpolicy.IfaceMatch{IfTypes: []int64{6}}})
if err != nil {
t.Fatal(err)
}
if p.Interfaces.Count != 1 || p.Devices.Count != 1 || p.Interfaces.Samples[0].IfAlias != "uplink" {
t.Fatalf("preview = %+v", p)
}
d, err := e.pe.Preview(context.Background(), "device_down", alertpolicy.Scope{Devices: alertpolicy.DeviceMatch{Sites: []string{"nowhere"}}})
if err != nil || d.Devices.Count != 0 {
t.Fatalf("device preview = %+v, %v", d, err)
}
}

View File

@@ -0,0 +1,145 @@
package app
import (
"context"
"fmt"
"github.com/v10networks/nmon/internal/alertpolicy"
"github.com/v10networks/nmon/internal/api"
"github.com/v10networks/nmon/internal/store"
)
// previewSamples is how many covered devices and ports a preview names.
const previewSamples = 20
// Rules implements api.AlertPolicyHost: the metric rules plus the reachability and system rules.
func (p *policyEngine) Rules() []api.AlertRuleInfo {
out := make([]api.AlertRuleInfo, 0, len(p.rules)+3)
for _, r := range p.rules {
if r.Disabled {
continue
}
thr := r.Threshold
out = append(out, api.AlertRuleInfo{Name: r.Name, Level: r.Level(), Op: r.Op, Threshold: &thr, Severity: r.Severity})
}
return append(out,
api.AlertRuleInfo{Name: alertpolicy.RuleDeviceDown, Level: alertpolicy.LevelDevice, Severity: "critical"},
api.AlertRuleInfo{Name: alertpolicy.RuleDeviceWarning, Level: alertpolicy.LevelDevice, Severity: "warning"},
api.AlertRuleInfo{Name: alertpolicy.RuleSystem, Level: alertpolicy.LevelSystem, Severity: "warning"})
}
func (p *policyEngine) level(rule string) (string, bool) {
for _, r := range p.Rules() {
if r.Name == rule {
return r.Level, true
}
}
return "", false
}
// Preview implements api.AlertPolicyHost. It reads the inventory afresh, so the counts match what the policy
// would cover on the next cycle.
func (p *policyEngine) Preview(ctx context.Context, rule string, scope alertpolicy.Scope) (api.PolicyPreview, error) {
level, ok := p.level(rule)
if !ok {
return api.PolicyPreview{}, fmt.Errorf("%w: rule %q", store.ErrNotFound, rule)
}
m, err := scope.Compile()
if err != nil {
return api.PolicyPreview{}, err
}
out := api.PolicyPreview{
Devices: api.PreviewDevices{Samples: []api.DeviceRef{}},
Interfaces: api.PreviewInterfaces{Samples: []api.PreviewInterface{}},
}
if level == alertpolicy.LevelSystem {
return out, nil
}
byIface, err := p.loadGroups(ctx, p.clk.Now())
if err != nil {
return out, err
}
down, err := p.st.InterfacesAlertDown(ctx)
if err != nil {
return out, err
}
devs, err := p.activeDevices(ctx)
if err != nil {
return out, err
}
for _, d := range devs {
ifs, err := p.st.ListInterfaces(ctx, d.ID, false)
if err != nil {
return out, err
}
pv := previewOf{m: m, level: level, byIface: byIface, down: down}
pv.device(&out, d, ifs)
}
return out, nil
}
func (p *policyEngine) activeDevices(ctx context.Context) ([]store.Device, error) {
var all []store.Device
for off := int64(0); ; off += store.MaxLimit {
page, total, err := p.st.ListDevices(ctx, store.DeviceFilter{}, store.Page{Offset: off, Limit: store.MaxLimit})
if err != nil {
return nil, err
}
for _, d := range page {
if d.RetiredAt == nil && d.PurgedAt == nil && d.Enabled {
all = append(all, d)
}
}
if len(page) == 0 || off+int64(len(page)) >= total {
return all, nil
}
}
}
// previewOf evaluates one scope over devices and ports.
type previewOf struct {
m *alertpolicy.Matcher
level string
byIface map[string][]string
down map[string]bool
}
// device adds a device (and, for an interface rule, its covered ports) to the preview.
func (pv previewOf) device(out *api.PolicyPreview, d store.Device, ifs []store.Interface) {
base := alertpolicy.Subject{Device: deviceSubject(d)}
if pv.level != alertpolicy.LevelInterface {
base.Groups = unionGroups(ifs, pv.byIface)
if pv.m.Match(base) {
addDevice(out, d)
}
return
}
covered := false
for _, i := range ifs {
if i.Excluded {
continue
}
port := ifaceSubject(i, pv.down[i.ID])
s := base
s.Iface, s.Groups = &port, pv.byIface[i.ID]
if !pv.m.Match(s) {
continue
}
covered = true
out.Interfaces.Count++
if len(out.Interfaces.Samples) < previewSamples {
out.Interfaces.Samples = append(out.Interfaces.Samples,
api.PreviewInterface{ID: i.ID, Device: d.Slug, IfName: i.IfName, IfAlias: i.IfAlias})
}
}
if covered {
addDevice(out, d)
}
}
func addDevice(out *api.PolicyPreview, d store.Device) {
out.Devices.Count++
if len(out.Devices.Samples) < previewSamples {
out.Devices.Samples = append(out.Devices.Samples, api.DeviceRef{Slug: d.Slug, Name: d.Name})
}
}

Some files were not shown because too many files have changed in this diff Show More