How Netwarden's Security Wedge Works

How Netwarden's Security Wedge Works

Most monitoring tools don't surface security signals. Most security tools don't surface monitoring signals. We built one tool that does both, because the people we sell to don't want to pay for two — and because, on a small fleet of servers, the line between "this host is unhealthy" and "this host is compromised" is thin enough that having two dashboards open is its own kind of mistake.

This post is the implementation walkthrough. What the agent collects, how the platform turns those snapshots into discrete findings, how the CVE matcher works, what we do (and don't do) with GeoIP, and how the routing and auto-resolve logic keeps the noise tolerable. It's longer than our usual post because the topic deserves the detail. If you only want the marketing version, the security features page has it.

The core idea: a finding is a noun

Before any of the architecture makes sense, the central concept needs a name. Inside the codebase, a finding is a first-class object: a discrete, fingerprinted security observation derived from a snapshot, with a stable identity across cycles.

A finding has:

• A finding_type — ssh_root_login_enabled, cve_match, failed_login_high_risk_country, etc. There are fourteen of them today.
• A severity — critical | high | medium | low.
• A fingerprint — sha1(host_id | finding_type | stable_subset_of_details). Stable across re-evaluations of the same observation; divergent when the underlying thing changes.
• A details blob — the structured payload the UI renders ("PermitRootLogin = yes", "port 5432 bound on 0.0.0.0", "openssl < 3.0.13-0ubuntu0.22.04.1").
• A description and a remediation snippet — the part you copy-paste into a config file to make the finding go away.

The fingerprint matters because of idempotency. The agent re-collects every snapshot every cycle. If we wrote a new database row each cycle, you'd drown in duplicates within an hour. Instead, the upsert path keys on fingerprint: same observation → same row, mutates last_seen_at. Different observation (port changed, package version moved, country newly seen) → different fingerprint → new finding. This is what makes the pipeline economical.

Auto-resolve falls out of the same design. Each snapshot type maps to a set of finding types it's authoritative for (SNAPSHOT_FINDING_COVERAGE in evaluator.ts). When a snapshot ingests successfully, every open finding whose finding_type is in coverage[T] but whose fingerprint was not in the just-emitted set gets auto-resolved. Fix the SSH config, restart sshd, the agent re-collects, the next snapshot omits the bad fingerprint, the finding closes itself. No manual ack. No stale alerts.

The two pieces — fingerprinting on the way in, coverage-map resolution on the way out — are most of why this works.

The four agent collectors

The Linux agent ships with four security collectors. They live in apps/agent/internal/collectors/security/ and they're real, boring, run-on-Linux collectors. Nothing exotic; the work is in being correct, fast, and surviving permission edge cases.

Failed-login collector

Source preference, in order:

1. journalctl _SYSTEMD_UNIT=ssh.service --since=... (Debian/Ubuntu). Falls through to sshd.service for the RHEL family.
2. Tail of /var/log/auth.log, capped at 256KB.
3. Tail of /var/log/secure, same cap.

The agent runs on a 60-second cadence, so the look-back window is now - 60s. We extract three patterns from the log lines: Failed (password|publickey|none) for ... from <ip>, Invalid user <user> from <ip>, plus a few minor variants. From those, the snapshot carries the failed-attempt count, unique-user count, unique-IP count, and a top-IPs list with per-IP counts.

The only thing that's slightly clever is the source-fallback: journalctl is preferred because it's structured and respects unit boundaries, but a meaningful chunk of distros run agents under a user that can't read the journal. The fallback to auth.log keeps the collector working without root, at a small cost in log-format fragility.

The whole thing emits a single metric (security_failed_logins) plus the top_ips payload. That payload is what the platform-side GeoIP enrichment consumes — more on that below.

SSH config collector

Reads /etc/ssh/sshd_config. Then it follows Include directives one level deep (capped at 64 files), because OpenSSH 8.2+ ships drop-in configs in /etc/ssh/sshd_config.d/*.conf and modern distros put their hardening there.

What it parses:

• PermitRootLogin
• PasswordAuthentication
• PermitEmptyPasswords
• Protocol (looking specifically for 1)
• X11Forwarding
• The raw KexAlgorithms, Ciphers, and MACs strings

Posture flags (the booleans) become snapshot fields directly. The crypto strings are passed through verbatim — the matching against the weak-list happens server-side in the evaluator, not in the agent. That separation lets us update the weak-cipher list without redeploying agents.

Listening-ports collector

ss -tulnpH is the primary command (no header, both TCP and UDP, with PID/process info). On distros without -H, it falls back to ss -tlnp and the platform tolerates the header line.

The output rows are parsed into {proto, port, bind_addr, process, pid}. Bind address is the load-bearing field for findings — 0.0.0.0 and :: mean the port is exposed on every interface; 127.0.0.1 and ::1 are loopback-only and uninteresting. We classify a finite list of management ports (SSH, Postgres, MySQL, MongoDB, Redis, Memcached, Elasticsearch, RDP, VNC) and fire public_management_port findings when one of them is bound publicly. There's also a many_public_bindings finding for the case where the count of public binds crosses a threshold — useful for catching the "this host accidentally became a router" failure mode.

Installed-packages collector

Distro detection comes from /etc/os-release — specifically ID and ID_LIKE. From there:

• Debian / Ubuntu / debian-like → dpkg-query -W -f='${Package}\t${Version}\t${Architecture}\n'
• RHEL / Fedora / Rocky / Alma → rpm -qa --queryformat='%{NAME}\t%{VERSION}-%{RELEASE}\t%{ARCH}\n'

Output is parsed into a flat list of {name, version, arch} rows. Source-package mapping lives on the platform side, not the agent — keeps the agent dumb and the matcher in one place.

The snapshot is intentionally not a metric. It's a JSON blob attached to the host's security state. Sending an entire package list as Prometheus-style metrics would explode cardinality; sending it as a snapshot row that gets joined against the advisory tables on a cron is the right shape.

The non-Linux variants of all four collectors are no-ops with a clear log line. macOS and Windows hosts ingest fine; the security findings simply don't fire on them.

The CVE pipeline

This is the part that took the longest to get right. The premise is simple: every host emits a package snapshot; we have a table of advisories with affected-package versions; the matcher does the join and emits cve_match findings when installed_version < fixed_version.

In practice, "the matcher does the join" is the easy part. The hard parts are: getting reliable advisory data from three different feeds with three different formats; comparing versions correctly across distro families; and not creating duplicate findings every cron tick.

Three feeds

1. Ubuntu USN — https://ubuntu.com/security/notices.json. JSON, paginated, well-structured. Each notice carries CVEs, releases, and per-release release_packages with the fixed source-package version. The parser lives in cve-feeds/ubuntu-usn.ts. USN doesn't expose severity; we set it to unknown and let the NVD CVSS lookup (below) sharpen it.

2. Debian DSA — https://salsa.debian.org/security-tracker-team/security-tracker/-/raw/master/data/DSA/list. Plain text, blocks of advisories, one per release-codename per package. Parser is in debian-dsa.ts. The full Debian security tracker JSON is also available, but it's hundreds of MB and contains every CVE keyed by package; the DSA list is a fraction of the size and contains exactly what we need for matching.

3. Red Hat — https://access.redhat.com/hydra/rest/securitydata/cve.json for the list, then per-CVE detail at /securitydata/cve/<id>.json. The list endpoint is filterable by date — we pull the last 45 days every night. The detail call gives affected_release[] rows whose package strings have the format name-EPOCH:VERSION-RELEASE (e.g. rsync-0:3.4.1-2.el10). Parser is redhat-oval.ts (named for the original spec; the underlying source is JSON, not OVAL XML — far easier to deal with).

Each feed parser produces a normalized ParsedAdvisory object with {advisory_id, source, cve_ids[], severity, title, summary, url, affected_packages[]}. The upsert into cve_advisories and cve_affected_packages is uniform across the three sources. One feed failing does not abort the others — failures are logged per-feed and aggregated into the cron summary.

Refresh runs daily.

CVSS enrichment from NVD

Feed-published severity is uneven. Red Hat publishes severity for almost everything. Ubuntu publishes basically nothing. Debian publishes severity inconsistently. So after each daily refresh, we kick off a bounded enrichment pass against the NVD CVE 2.0 API. For each advisory's CVE list, we fetch the NVD record and pull the highest CVSS3 score available. Precedence is cvssMetricV31 → cvssMetricV30 → cvssMetricV2 (legacy fallback only).

The score-to-severity mapping in cvssToSeverity() matches NVD's published bucket boundaries:

9.0 - 10.0 -> critical
7.0 -  8.9 -> high
4.0 -  6.9 -> medium
0.1 -  3.9 -> low

When a CVSS score exists, it wins over the feed's published severity. When NVD has no CVSS yet (common for fresh CVEs), we fall back to the feed value, and finally to a medium default if neither source has anything. The matcher's resolveSeverity() audit-logs which path each finding took, so when a critical lands, you can see whether it came from CVSS, the feed, or the fallback.

NVD outages do not break the pipeline. Feed refresh always succeeds even if enrichment fails — the next day's run will pick up the missing scores.

The matcher

runMatcher() is the daily cron that joins host package snapshots × advisory rows. The interesting bits are:

Version comparison. Different distros have different version-comparison rules. Debian/Ubuntu uses dpkg's algorithm (epoch + upstream-version + revision, with tilde sorting before everything). RPM uses a different segmented comparison with its own twists. We have a single compareVersions() in cve-feeds/version-compare.ts that handles both — see isVulnerable(installed, fixed). The reason this is a custom implementation and not a library call is that the boundary cases between dpkg and rpm are subtle enough that depending on a third-party library means depending on its bug-for-bug compatibility forever.

Fingerprint stability. Per-CVE fingerprint is sha1(host_id | cve_match | advisory_id | package_name). Two USNs covering the same package on the same host produce two findings; the same USN re-emitted next cycle produces one. When the host upgrades the package and the next snapshot shows installed >= fixed, the matcher omits the fingerprint, the resolver closes the finding.

Distro scoping. The match uses (distro_id, distro_version_id) as a join key. A Debian 12 host doesn't get matched against an Ubuntu 22.04 advisory even when the package and version look superficially similar. This sounds obvious; the wrong way to do this would have been to match on package name alone and make every customer's CVE list a noisy mess.

Severity routing happens at upsert. Every cve_match finding is written with the CVSS-resolved severity, which means it routes through the same email + push + webhook channels the SSH-config and listening-port findings do, with the same severity gates.

GeoIP for failed logins

This is the most recent piece of the wedge and the one customers ask about most after they've been running the agent for a few weeks.

The agent ships the top_ips list with each failed_login_geo snapshot. Server-side, before the snapshot is persisted, we enrich each IP with a MaxMind GeoLite2-City lookup. Each row gains {country, region, city, country_iso} fields. The geoip.ts module memory-maps the .mmdb file once at startup and reuses the reader for the process lifetime; per-call overhead is microseconds.

License terms matter here. The MaxMind database is free to download with a license key, but it is not redistributable. We do not bundle it. Customers running self-hosted point GEOLITE2_CITY_PATH at their own .mmdb file (typically maintained by geoipupdate's cron). When the env var is unset or the file is missing, the lookup gracefully no-ops: failed-login data still flows, the country/region fields are simply null. SaaS customers get the lookup automatically; nothing to configure.

From the enriched snapshot, the evaluator emits two finding types:

failed_login_high_risk_country — fires when ≥ 20 failed attempts/minute originate from a country on a deliberately blunt allowlist (RU, CN, KP, IR, BY). Severity: high. Hosts that legitimately operate in those regions get noisy findings — that's an accepted Phase 3 trade-off, and the auto-resolver clears the finding the moment the burst dies down.

failed_login_country_anomaly — fires when ≥ 5 failed attempts/minute come from a country with no failed-login history on this host in the prior 30 days. Severity: medium. Lower threshold than the high-risk rule because any sustained traffic from a brand-new origin is worth a nudge.

The fingerprint for both is sha1(host_id | finding_type | country_iso). Different countries produce different findings; the same country in two consecutive cycles dedupes. The "high-risk" rule and the "anomaly" rule are deliberately exclusive — when both would fire on the same observation, only the more specific high-risk one does, so operators don't get paged twice for the same data.

The country-anomaly rule is the one customers find most useful in practice. SSH brute-force traffic from China and Russia is so consistent it becomes background noise; the first sustained attempt from a country that's never tried before is genuinely interesting, and it's a class of signal you basically can't get without geo data.

Severity, routing, auto-resolve

These three pieces are what turn raw findings into something operators can live with.

Severity flows from CVSS first (when present), then feed severity, then a medium fallback. For findings that aren't CVE-derived (SSH-config, listening-ports, geo), severity is hardcoded per-rule in the evaluator: ssh_protocol_v1_enabled is critical, ssh_x11_forwarding_enabled is low, public_management_port is high, etc. The hardcoding is deliberate — it's a small, opinionated list, and tuning it per-tenant would create more confusion than it would solve.

Routing is per-tenant. The notification preferences let you set, per-severity, which channels fire: email, mobile push, outbound webhook. The default is "all three for high and critical, email-only for medium, dashboard-only for low." That maps to the three-tier alerting model from alerts that actually page you — page-me-now, tell-me-today, show-me-on-the-dashboard.

The weekly digest is the lower-noise option. Monday at 09:00 UTC, every tenant with the digest enabled gets one email with every open finding above a configurable severity floor, grouped by host. The implementation is in lib/security/digest.ts and uses security_weekly_digest_enabled per-user. It's designed for the audience that doesn't want push notifications for medium-severity findings but does want a weekly inventory.

Auto-resolve is the bit that earns trust. Every finding is tied to a snapshot type via SNAPSHOT_FINDING_COVERAGE. When a new snapshot of that type lands, the resolver computes the set difference: open findings whose fingerprint isn't in the new snapshot's emitted set get marked resolved. CVE findings auto-resolve too, but on a different schedule — the daily matcher cron is the authoritative re-emitter, so the resolution pass runs once a day after the matcher finishes.

The combined effect: a finding shows up when something's wrong, persists across re-collection cycles, and disappears when you fix it. There's no acknowledge-but-don't-resolve, no snooze-this-for-a-week, no manual workflow. You either fix it or you don't.

Remediation, in the box

Every finding type ships with a copy-pasteable fix. The remediation isn't a separate doc; it's part of the finding payload.

For example, the ssh_root_login_enabled finding (severity: high) renders this snippet inline in the UI:

# /etc/ssh/sshd_config
PermitRootLogin no
PasswordAuthentication no

Followed by:

sudo systemctl reload ssh   # debian/ubuntu
# or
sudo systemctl reload sshd  # rhel/fedora/rocky

The ssh_weak_cipher finding lists the offending entries from the host's Ciphers line and shows the recommended replacement ([email protected],[email protected],[email protected],aes256-ctr,aes192-ctr,aes128-ctr). The cve_match finding shows the installed version, the fixed version, and the package-manager command (apt-get install --only-upgrade <package> or dnf upgrade <package>). This is a small thing but it's the difference between a finding that takes 30 seconds to fix and one that takes 30 minutes of searching.

We do not ship automated remediation. Pulling the trigger on changes to a customer's sshd_config from a remote agent is not a thing we want to be in the business of; the consequences of a bad rollout are too high. What we do ship is the answer, in the right place, where the operator already is.

What we deliberately don't do

Honest scoping, because this matters when you're evaluating tools:

• Not a full vulnerability scanner. No active probing, no auth-checks against running services, no application-layer scanning. Findings come from snapshots of host state, not from outbound scans.
• Not an EDR. No process behavior analysis, no in-memory inspection, no kernel hooks. The agent reads files and runs a few well-defined shell commands.
• Not a SIEM. No log aggregation, no event correlation across hosts, no query language. Findings are discrete and per-host.
• Not running anomaly detection on host metrics. The geo-anomaly rule is the closest thing, and even that is "have we ever seen this country before, yes/no" — not a statistical baseline.
• No GeoIP without a customer-provided MaxMind file on self-hosted. SaaS users get it transparently.
• No SUSE / Arch / Gentoo / Alpine CVE feeds yet. Today: Ubuntu, Debian, RHEL family. Adding feeds is mechanical work; we'll do it as customer demand justifies.
• No Windows or macOS CVE matching. The package collectors are no-ops outside Linux. Patch-state monitoring on Windows hosts is a different product shape; we don't ship it.

The point of being explicit about the gaps is that it's how you end up with a tool whose findings you can trust. A SIEM that promises everything and false-positives 80% of the time is worse than a small, opinionated set of rules that hold their fingerprints stable across cycles.

Try it on your stack

The free SaaS tier covers 1 host with the security wedge fully enabled. If you'd rather run it on your own box, the self-hosted single binary is in open beta — same wedge, same collectors, same matcher, with SQLite inside. The architecture deep-dive on that binary is in the self-hosting post.

For the operator-side reference on what the findings look like, how to tune severity routing, and how to acknowledge-or-resolve, see the security findings docs. The self-hosting docs cover the GeoIP setup if you want country-level findings on your own infrastructure.

The shortest path to seeing the wedge fire on your own host: install the agent, leave it running for a few hours, then run nc -lvp 5432 to bind a public port (or just check whatever you already have listening on 0.0.0.0). You'll get a public_management_port finding within a minute, with the remediation snippet attached. From there, the rest of the wedge — CVE matches, SSH-config posture, failed-login geography — will surface as the agent's collection cycle finds it.

Tell us what fires and what doesn't. The tuning we do on the rule set is mostly driven by reports of false positives on customer hardware that doesn't look like ours, and that feedback loop is the thing that makes the wedge sharper over time.

---

Keep reading

• Self-Hosting Netwarden: the single-binary preview — how to run the whole stack, including this wedge, on your own box.
• Alerts that actually page you — the severity-tier model the security routing reuses.
• Security features overview — the marketing-page summary of what's covered.
• Security findings reference — the operator reference for working with findings day-to-day.