diff --git a/.gitignore b/.gitignore index c961f86..ddd11d5 100644 --- a/.gitignore +++ b/.gitignore @@ -7,3 +7,4 @@ web/dist/ /tmp/ *.test coverage.out +/nmon-librenms-import diff --git a/Makefile b/Makefile index ce7efcb..2f0e5c1 100644 --- a/Makefile +++ b/Makefile @@ -4,7 +4,7 @@ LDFLAGS := -s -w BIN := nmon PG_DSN ?= postgres://nmon:nmon@localhost:55432/nmon?sslmode=disable -.PHONY: all build build-arm64 build-noembed fetch-vm web test test-integration test-pg test-v6only vet gate-m1 gate-m2 gate-m3 gate-m4 gate-m5 gate-m6 clean +.PHONY: all build importer build-arm64 build-noembed fetch-vm web test test-integration test-pg test-v6only vet gate-m1 gate-m2 gate-m3 gate-m4 gate-m5 gate-m6 clean all: build @@ -21,6 +21,9 @@ build: fetch-vm build-arm64: fetch-vm CGO_ENABLED=0 GOOS=linux GOARCH=arm64 $(GO) build -tags '$(TAGS)' -ldflags '$(LDFLAGS)' -o $(BIN)-arm64 ./cmd/nmon +importer: + CGO_ENABLED=0 $(GO) build -ldflags '$(LDFLAGS)' -o nmon-librenms-import ./cmd/nmon-librenms-import + build-noembed: CGO_ENABLED=0 $(GO) build -ldflags '$(LDFLAGS)' -o $(BIN) ./cmd/nmon @@ -44,4 +47,4 @@ gate-m%: ./scripts/gate-m$*.sh clean: - rm -f $(BIN) $(BIN)-arm64 + rm -f $(BIN) $(BIN)-arm64 nmon-librenms-import diff --git a/cmd/nmon/main.go b/cmd/nmon/main.go index bcce602..cc5f9cd 100644 --- a/cmd/nmon/main.go +++ b/cmd/nmon/main.go @@ -30,7 +30,7 @@ func register(name, summary string, run func(ctx context.Context, args []string) func init() { for name, summary := range map[string]string{ "discover": "run discovery for a device now", - "profile": "test, record, draft and import device profiles", + "profile": "test, record and draft device profiles", "backup": "write a backup of inventory, metrics, config and profiles", "restore": "restore a backup into a clean data dir", "loadtest": "run the scale test against simulated devices", diff --git a/docs/DEVIATIONS.md b/docs/DEVIATIONS.md index c9396ea..3866c86 100644 --- a/docs/DEVIATIONS.md +++ b/docs/DEVIATIONS.md @@ -13,3 +13,4 @@ conscious, reviewed exception. | 6 | "No MIB file is ever shipped in the repo or the binary"; vendor MIBs are the operator's to fetch | MIBs are bundled as a **sidecar** (user decision, 2026-10-22). All 4,865 LibreNMS MIBs are vendored under `mibs/` and shipped in the release tarball (installed to `/usr/share/nmon/mibs`). **None is embedded in the binary**, not even the IETF/IANA set. They are read via `--mibs-dir` (default `/mibs`, then `/usr/share/nmon/mibs`). The poller never reads a MIB. | Operators get `nmon profile new --mib` working out of the box while the binary stays small. Each MIB keeps its publisher's terms (see `mibs/README.md`). | M2 | | 7 | `import-librenms` reads `includes/definitions/*.yaml` and `includes/definitions/discovery/*.yaml` | Reads `resources/definitions/os_detection/*.yaml` and `resources/definitions/os_discovery/*.yaml`, with the old paths as a fallback. | LibreNMS moved them; the spec's paths no longer exist upstream. | M2 | | 8 | Real recordings replace synthetic fixtures when available | LibreNMS's `tests/snmpsim/*.snmprec` (2,054 real device recordings) are used by optional tests when `NMON_LIBRENMS_DIR` points at a checkout. They are never copied into this repo. | Real-device coverage without vendoring GPL test data into an Apache-2.0 repo. | M2 | +| 9 | `nmon profile import-librenms ` is an nmon subcommand | A separate binary, `nmon-librenms-import` (`cmd/nmon-librenms-import`), run offline before nmon. It writes draft profiles that the operator reviews and copies into `/profiles/`. `nmon` neither links nor exposes it. | User decision (2026-10-22). It keeps the importer out of the product binary and makes "the converter never runs at serve time" structural. | M2 | diff --git a/docs/milestones/M2-discovery-and-profiles.md b/docs/milestones/M2-discovery-and-profiles.md index d5e9957..51f590a 100644 --- a/docs/milestones/M2-discovery-and-profiles.md +++ b/docs/milestones/M2-discovery-and-profiles.md @@ -35,7 +35,7 @@ - `nmon profile test [--community|--v3-*]`: discovery plus one poll with the matching profile; prints each metric with its unit, value and PDU count. - `nmon profile record `: read-only walk to snmprec, redacting communities, sysContact and sysLocation. - `nmon profile new --mib `: compiles MIBs in `/mibs/` into draft metric entries (OID, type, textual convention). Uses a small SMIv2 parser in `internal/mib`; no external tools. **Bundled MIBs are a sidecar** (DEVIATIONS #6): nothing is embedded in the binary. MIBs are read from `--mibs-dir` (default `/mibs`, then `/usr/share/nmon/mibs`). The repo's `mibs/` tree ships in the release tarball. The parser must import every file in the bundled `mibs/` tree without error (a test enforces this). -- `nmon profile import-librenms `: the field mapping from Build detail; writes drafts with `imported: librenms`, `type: unverified`. Reads `resources/definitions/os_detection/*.yaml` + `os_discovery/*.yaml` (current LibreNMS layout; falls back to `includes/definitions/`); see `docs/reference/librenms.md`. Contains no LibreNMS code. Tested against the reference checkout when `NMON_LIBRENMS_DIR` is set. +- **LibreNMS importer is a separate tool, run before nmon** (DEVIATIONS #9): `cmd/nmon-librenms-import` builds its own binary, `nmon-librenms-import`. It is not linked into `nmon`, and `nmon` has no import subcommand. Usage: `nmon-librenms-import --librenms --out [--os junos,ios,...] [--mibs-dir ]`. It reads `resources/definitions/os_detection/*.yaml` + `os_discovery/*.yaml` (current LibreNMS layout; falls back to `includes/definitions/`) and writes one draft profile per OS with `imported: librenms`, `type: unverified`, using the field mapping from Build detail. The operator reviews the drafts and copies them into `/profiles/`. Then `nmon profile test` confirms each metric's type against a device or fixture, which promotes it. It contains no LibreNMS code, lives in `internal/librenms` (imported only by that command; a test asserts `cmd/nmon` does not depend on it), and is tested against the reference checkout when `NMON_LIBRENMS_DIR` is set. See `docs/reference/librenms.md`. - Optional real-device tests: when `NMON_LIBRENMS_DIR` is set, discovery and one poll run against matching `tests/snmpsim/*.snmprec` recordings (e.g. junos, ios, nxos, arista_eos, routeros, edgeos, fortigate, pfsense, linux, windows, apc, printers) and must match a profile, find interfaces, and convert every pollable metric without a type mismatch. ### Fixtures (`testdata/`) @@ -51,7 +51,7 @@ | WP2 Fixture generator, behaviour fixtures, expect harness | `testdata/**`, `internal/fixtures/**`, simagent extensions | WP1 schema types | | WP3 Discovery and identity matcher | `internal/discovery/**`, store additions | WP1 | | WP4 Profile-driven poller, adaptation, backoff, reboot, traps | `internal/poller/**`, `internal/snmp/**`, `internal/traps/**` | WP1 | -| WP5 CLI: profile test/record/new/import-librenms, MIB parser | `cmd/nmon/profile*.go`, `internal/mib/**`, `internal/librenms/**` | WP1, WP3, WP4 | +| WP5 CLI: profile test/record/new, MIB parser; separate `nmon-librenms-import` tool | `cmd/nmon/profile*.go`, `internal/mib/**`, `internal/librenms/**`, `cmd/nmon-librenms-import/**` | WP1, WP3, WP4 | ## Acceptance lines closed diff --git a/docs/reference/librenms.md b/docs/reference/librenms.md index 1f94f53..f5c61a6 100644 --- a/docs/reference/librenms.md +++ b/docs/reference/librenms.md @@ -5,7 +5,7 @@ A shallow clone lives at `/home/svcuser/src/librenms` (commit 06080bb3) for refe | Path | What it is | Used for | |---|---|---| | `mibs/` | 4,865 MIBs (standard at the top level, vendors in subdirs) | copied into `nmon/mibs/` (DEVIATIONS #6) | -| `resources/definitions/os_detection/*.yaml` (817) | OS matchers: `sysObjectID`, `sysDescr`, `mib_dir` | `nmon profile import-librenms`, the profile `match:` section | +| `resources/definitions/os_detection/*.yaml` (817) | OS matchers: `sysObjectID`, `sysDescr`, `mib_dir` | `nmon-librenms-import` (separate offline tool), the profile `match:` section | | `resources/definitions/os_discovery/*.yaml` (708) | per-OS processors, mempools, storage, sensors with `oid`, `num_oid`, `divisor`, `skip_values` | the importer, and checking vendor OIDs in shipped profiles | | `tests/snmpsim/*.snmprec` (2,054) | real device recordings | optional real-device fixture tests (DEVIATIONS #8) | | `LibreNMS/OS/*.php`, `includes/polling/` | poller logic | reading only; the converter contains no LibreNMS code | diff --git a/mibs/README.md b/mibs/README.md index 94fd9bf..795e71b 100644 --- a/mibs/README.md +++ b/mibs/README.md @@ -7,7 +7,7 @@ See `docs/DEVIATIONS.md` row 6. - All MIBs are a **sidecar**: none is embedded in the `nmon` binary. That includes the standard IETF/IANA/IEEE set at the top level. - They live in this repo under `mibs/` (standards at the top level, vendors in `mibs//`) and ship in the release tarball next to the binary. They are installed to `/usr/share/nmon/mibs` by default. -- `nmon profile new --mib` and `nmon profile import-librenms` read them from `--mibs-dir`. The default +- `nmon profile new --mib` and the separate `nmon-librenms-import` tool read them from `--mibs-dir`. The default is `/mibs`, falling back to `/usr/share/nmon/mibs`. An operator adds or overrides MIBs by dropping files in `/mibs/`. The poller never reads MIBs. Profiles store numeric OIDs. These files are only for writing profiles.