docs+nix: fix stale certFingerprint/HYPERHIVE_PEERS references (hyperhive#3294)
Follow-up to the plumbing removal: docs/swarm/README.md gets the biggest rewrite (drops the whole "Fingerprint format" section, fixes the runtime-effects list, the WireGuard config example + "what the mesh does" bullet), docs/gateway.md and hive-gateway/options.nix drop 4 "needs no certFingerprint" mentions, swarm-peers-removed.nix's migration-warning text no longer tells an upgrading operator to carry a field over that no longer exists, swarm.nix/swarm-wireguard.nix/ swarm-controller.nix/swarm-controller's main.rs get comment fixes where they described the now-removed HYPERHIVE_PEERS shape. Also caught one more stale "peer hives" mention in docs/web-ui/README.md's SW4RM tab description that the first pass on this issue missed.
This commit is contained in:
parent
c2f8ee225d
commit
c67100588e
8 changed files with 40 additions and 110 deletions
|
|
@ -200,8 +200,6 @@ nginx obtains and auto-renews certs via the ACME HTTP-01 challenge on `port` (de
|
||||||
|
|
||||||
Mutual exclusion: `tls.certDir` set together with `tls.acme.enable = true` fails an assertion — pick one external TLS source (or neither, for the self-signed default).
|
Mutual exclusion: `tls.certDir` set together with `tls.acme.enable = true` fails an assertion — pick one external TLS source (or neither, for the self-signed default).
|
||||||
|
|
||||||
**Swarm peers**: CA-signed certs are trusted by default — this hive's entry in `swarm.hives` needs no `certFingerprint`.
|
|
||||||
|
|
||||||
### Self-signed TLS (default)
|
### Self-signed TLS (default)
|
||||||
|
|
||||||
On by default, and listens on `httpsPort` (default 443) on every vhost beside the plain-http `port` (default 80).
|
On by default, and listens on `httpsPort` (default 443) on every vhost beside the plain-http `port` (default 80).
|
||||||
|
|
@ -252,12 +250,6 @@ security.acme.certs."example.com".group = "nginx";
|
||||||
|
|
||||||
or make the key world-readable (`0644`) if your threat model allows it. nginx errors out at startup on a key it can't read — the error is explicit in the journal, not a silent failure.
|
or make the key world-readable (`0644`) if your threat model allows it. nginx errors out at startup on a key it can't read — the error is explicit in the journal, not a silent failure.
|
||||||
|
|
||||||
**Swarm directory entry**: when using a CA-signed cert, this hive's entry needs no `certFingerprint` — the standard CA bundle validates:
|
|
||||||
|
|
||||||
```nix
|
|
||||||
services.hyperhive.swarm.hives.example = { domain = "example.com"; }; # no certFingerprint needed
|
|
||||||
```
|
|
||||||
|
|
||||||
### Fronting with an external TLS terminator
|
### Fronting with an external TLS terminator
|
||||||
|
|
||||||
There is no http-only mode (see [TLS modes](#tls-modes) above). Two paths
|
There is no http-only mode (see [TLS modes](#tls-modes) above). Two paths
|
||||||
|
|
|
||||||
|
|
@ -107,9 +107,8 @@ swarm service name has to be wired into: [`ui.md`](ui.md).
|
||||||
|
|
||||||
```nix
|
```nix
|
||||||
services.hyperhive.swarm.hives = {
|
services.hyperhive.swarm.hives = {
|
||||||
pr1ma = { domain = "pr1ma.example.com"; }; # this host, per hiveName
|
pr1ma = { domain = "pr1ma.example.com"; }; # this host, per hiveName
|
||||||
lab = { domain = "lab.example.com"; }; # CA-trusted (Let's Encrypt etc.)
|
lab = { domain = "lab.example.com"; }; # a second hive in the swarm
|
||||||
edge = { domain = "edge.corp"; certFingerprint = "sha256:…"; }; # self-signed leaf, pinned
|
|
||||||
};
|
};
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
@ -128,81 +127,30 @@ contain you derives every hive as a peer and you peer with yourself.
|
||||||
conventionally `<name>.<swarm.domain>`, but a wrong domain that
|
conventionally `<name>.<swarm.domain>`, but a wrong domain that
|
||||||
evaluates cleanly points at a real machine that isn't the one you meant.
|
evaluates cleanly points at a real machine that isn't the one you meant.
|
||||||
|
|
||||||
**`certFingerprint`** (`"sha256:…"`, optional) pins that hive's TLS
|
> **There is no per-hive CA field, and no per-hive cert pinning.** Trust
|
||||||
_leaf_. Matrix federation never consults it. Omit it for any hive
|
> inside a swarm comes from the swarm root ([`ca.md`](ca.md)): every
|
||||||
under the swarm root CA or a public CA — which is the normal case.
|
> hive chains to it, so one anchor replaces per-hive pinning entirely.
|
||||||
|
> What that genuinely drops is trusting a hive whose root this swarm
|
||||||
⚠️ **Currently unconsumed by any Rust code.** It was read by the
|
> does *not* own — another swarm's, or one keeping its own CA. That is
|
||||||
per-hive dashboard's now-removed "peer hives" display (see below);
|
> a cross-swarm problem and wants a mechanism designed for it. (An
|
||||||
that was its only consumer. Left in place rather than deleted along
|
> earlier `certFingerprint` field existed for exactly that gap, pinning
|
||||||
with that display — removing a documented, potentially
|
> a peer's TLS leaf for hive-c0re's own peer HTTPS checks — removed
|
||||||
already-set nix option is a bigger, more consequential (breaking)
|
> along with the dashboard feature it existed to serve, since nothing
|
||||||
change than removing dead display code, and wants its own explicit
|
> else ever consumed it.)
|
||||||
decision rather than being swept up here. A future peer-facing
|
|
||||||
feature (e.g. pinning a peer in `swarm-ui`) could pick it back up.
|
|
||||||
|
|
||||||
> **There is no per-hive CA field.** Trust inside a swarm comes from the
|
|
||||||
> swarm root ([`ca.md`](ca.md)): every hive chains to it, so one anchor
|
|
||||||
> replaces the O(n²) pinning. What that genuinely drops is trusting a
|
|
||||||
> hive whose root this swarm does *not* own — another swarm's, or one
|
|
||||||
> keeping its own CA. That is a cross-swarm problem and wants a
|
|
||||||
> mechanism designed for it, not a field that happened to work.
|
|
||||||
|
|
||||||
### Fingerprint format
|
|
||||||
|
|
||||||
The value is the string `sha256:` followed by exactly 64 hexadecimal
|
|
||||||
digits — the SHA-256 digest of the peer's DER-encoded TLS leaf
|
|
||||||
certificate. The hex is case-insensitive (upper or lower both parse),
|
|
||||||
carries no colon separators between bytes, and any value not matching
|
|
||||||
this shape is ignored with a warning rather than weakening trust.
|
|
||||||
|
|
||||||
```
|
|
||||||
sha256:b1946ac92492d2347c6235b4d2611184a3f5b6cae6c19d6e3c2f0a8e7d4c9f12
|
|
||||||
```
|
|
||||||
|
|
||||||
Generate it from the peer's certificate with openssl. The
|
|
||||||
`-fingerprint -sha256` output is uppercase and colon-separated, so
|
|
||||||
strip the colons, lowercase, and prepend the `sha256:` prefix:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
# from a PEM/CRT file
|
|
||||||
openssl x509 -in peer.crt -noout -fingerprint -sha256 \
|
|
||||||
| sed 's/^.*=//; s/://g' | tr 'A-Z' 'a-z' | sed 's/^/sha256:/'
|
|
||||||
|
|
||||||
# straight from the live endpoint (port 443)
|
|
||||||
echo | openssl s_client -connect peer.example.com:443 -servername peer.example.com 2>/dev/null \
|
|
||||||
| openssl x509 -noout -fingerprint -sha256 \
|
|
||||||
| sed 's/^.*=//; s/://g' | tr 'A-Z' 'a-z' | sed 's/^/sha256:/'
|
|
||||||
```
|
|
||||||
|
|
||||||
Pin the leaf certificate, not an intermediate or the CA — the
|
|
||||||
digest must match the exact cert the peer serves on its HTTPS
|
|
||||||
endpoint. When the peer rotates its cert, update the pin to the new
|
|
||||||
fingerprint (or switch the peer to a CA-trusted cert and drop the
|
|
||||||
field).
|
|
||||||
|
|
||||||
The nix module serialises the attrset to a `HYPERHIVE_PEERS` JSON
|
|
||||||
array (`[{ domain, cert_fingerprint }]`) injected into the c0re
|
|
||||||
environment and forwarded to agent containers.
|
|
||||||
|
|
||||||
## What the config does at runtime
|
## What the config does at runtime
|
||||||
|
|
||||||
1. **Agent identity** — the `HYPERHIVE_PEERS` env var is forwarded to
|
1. **Swarm-wide hive roster** — swarm-controller reads this same
|
||||||
agent containers, so agent code can discover peer hives and
|
directory and serves it at `GET /api/hives`; `swarm-ui`'s overview
|
||||||
address them with qualified names (`agent@domain`). See
|
page renders it (`docs/swarm/ui.md`). This is the operator-facing
|
||||||
`hive-agent/src/identity.rs`'s module doc for the label/domain
|
"what hives exist" surface — a per-hive dashboard "peer hives"
|
||||||
helpers. The swarm-wide hive roster used to also be shown as a
|
display existed here once and was removed in favour of it.
|
||||||
"peer hives" list on the per-hive dashboard; that display was
|
|
||||||
removed in favour of `swarm-ui`'s own roster page (`docs/swarm/
|
|
||||||
ui.md`), which reads the same directory from `swarm-controller`
|
|
||||||
instead.
|
|
||||||
|
|
||||||
2. **Matrix federation** — when `matrix.enable` is on, tuwunel
|
2. **Matrix federation** — when `matrix.enable` is on, tuwunel
|
||||||
federates with the peer's matrix server (discovered via the peer's
|
federates with the peer's matrix server (discovered via the peer's
|
||||||
`.well-known/matrix/server` delegation, which the gateway serves).
|
`.well-known/matrix/server` delegation, which the gateway serves).
|
||||||
Federation validates the peer's TLS certificate against the matrix
|
Federation validates the peer's TLS certificate against the matrix
|
||||||
**container's** trust bundle — independently of `certFingerprint`,
|
**container's** trust bundle, independent of this directory.
|
||||||
which it never consults.
|
|
||||||
|
|
||||||
⚠️ **That container currently trusts no swarm-internal CA**, so a
|
⚠️ **That container currently trusts no swarm-internal CA**, so a
|
||||||
self-signed gateway certificate does not federate. The swarm root
|
self-signed gateway certificate does not federate. The swarm root
|
||||||
|
|
@ -213,6 +161,10 @@ environment and forwarded to agent containers.
|
||||||
issue. Until then, federation needs CA-issued certs (ACME). See
|
issue. Until then, federation needs CA-issued certs (ACME). See
|
||||||
`docs/matrix.md` for federation firewall + TLS requirements.
|
`docs/matrix.md` for federation firewall + TLS requirements.
|
||||||
|
|
||||||
|
3. **WireGuard mesh** (optional) — `swarm.wireguard.enable` reads each
|
||||||
|
entry's `wireguardPublicKey`/`wireguardEndpoint`/`wireguardAddress`
|
||||||
|
to configure `wg-hive`. See "WireGuard inter-hive mesh" below.
|
||||||
|
|
||||||
## One directory, not a bilateral declaration
|
## One directory, not a bilateral declaration
|
||||||
|
|
||||||
Both hives hold the **same** `hives` attrset; neither declares the
|
Both hives hold the **same** `hives` attrset; neither declares the
|
||||||
|
|
@ -269,7 +221,6 @@ services.hyperhive = {
|
||||||
};
|
};
|
||||||
edge = {
|
edge = {
|
||||||
domain = "edge.corp";
|
domain = "edge.corp";
|
||||||
certFingerprint = "sha256:…"; # TLS trust (unchanged)
|
|
||||||
wireguardPublicKey = "base64keyB=";
|
wireguardPublicKey = "base64keyB=";
|
||||||
wireguardEndpoint = "203.0.113.42:51820";
|
wireguardEndpoint = "203.0.113.42:51820";
|
||||||
wireguardAddress = "10.100.0.2/32";
|
wireguardAddress = "10.100.0.2/32";
|
||||||
|
|
@ -295,9 +246,10 @@ services.hyperhive = {
|
||||||
(not inside agent containers; containers reach peers via the host's
|
(not inside agent containers; containers reach peers via the host's
|
||||||
routing table).
|
routing table).
|
||||||
- UDP port 51820 (or `listenPort`) is opened on the host firewall.
|
- UDP port 51820 (or `listenPort`) is opened on the host firewall.
|
||||||
- `HYPERHIVE_PEERS` gains a `wireguard_address` field for each mesh
|
- `swarm-wireguard.nix` reads each entry's `wireguardAddress` directly
|
||||||
peer so hive-c0re can reach intra-swarm services without a public
|
from `services.hyperhive.swarm.peerHives` to build `wg-hive`'s
|
||||||
DNS round-trip.
|
`allowedIPs`, so intra-swarm traffic can route over the mesh address
|
||||||
|
rather than the public domain.
|
||||||
- `persistentKeepalive = 25` is set by default; override or null to
|
- `persistentKeepalive = 25` is set by default; override or null to
|
||||||
disable (not needed when both sides have public IPs and no NAT).
|
disable (not needed when both sides have public IPs and no NAT).
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -18,9 +18,7 @@ The **dashboard** itself (`/dashboard.html`) is where you'll spend most
|
||||||
of your time. It's a single page with exactly four tabs:
|
of your time. It's a single page with exactly four tabs:
|
||||||
|
|
||||||
- **SW4RM** — every agent, live. This is the default tab and the one
|
- **SW4RM** — every agent, live. This is the default tab and the one
|
||||||
you'll check most. When the hive has peer hives configured, they show
|
you'll check most.
|
||||||
up here too, as a card list under the main container list — not a
|
|
||||||
separate tab.
|
|
||||||
- **Y3R C4LL** — anything waiting on *you*: pending approvals and
|
- **Y3R C4LL** — anything waiting on *you*: pending approvals and
|
||||||
agent questions. If an agent needs a decision from you, it's here.
|
agent questions. If an agent needs a decision from you, it's here.
|
||||||
- **P3RM1SS10NS** — what tools and system-level access each agent has.
|
- **P3RM1SS10NS** — what tools and system-level access each agent has.
|
||||||
|
|
|
||||||
|
|
@ -242,10 +242,6 @@ in
|
||||||
config.security.acme.certs."example.com".directory;
|
config.security.acme.certs."example.com".directory;
|
||||||
```
|
```
|
||||||
|
|
||||||
When using an external CA cert, other hives can declare this
|
|
||||||
one in `services.hyperhive.swarm.hives` without
|
|
||||||
`certFingerprint` — the standard CA bundle validates.
|
|
||||||
|
|
||||||
Mutual exclusion with `tls.acme.enable` — set one or the other,
|
Mutual exclusion with `tls.acme.enable` — set one or the other,
|
||||||
not both.
|
not both.
|
||||||
'';
|
'';
|
||||||
|
|
@ -301,10 +297,6 @@ in
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
```
|
```
|
||||||
|
|
||||||
After enabling, this hive's entry in `swarm.hives` can omit
|
|
||||||
`certFingerprint` — Let's Encrypt certs are CA-trusted
|
|
||||||
by default.
|
|
||||||
'';
|
'';
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -301,10 +301,9 @@ in
|
||||||
# as one attrset and never assigned individually.
|
# as one attrset and never assigned individually.
|
||||||
environment = {
|
environment = {
|
||||||
SWARM_CONTROLLER_SOCKET = cfg.socketPath;
|
SWARM_CONTROLLER_SOCKET = cfg.socketPath;
|
||||||
# The swarm's hive directory, JSON-encoded — same shape hive-c0re
|
# The swarm's hive directory, JSON-encoded — the full directory
|
||||||
# already builds for HYPERHIVE_PEERS (../hive-c0re/environment.nix),
|
# (this daemon has no "self" hive to exclude, unlike
|
||||||
# just the full directory (this daemon has no "self" hive to
|
# `swarm.peerHives`, `swarm.hives` minus this hive) rather than
|
||||||
# exclude, unlike a per-hive c0re's peer list) rather than
|
|
||||||
# peers-minus-self. Consumed by `GET /api/hives`
|
# peers-minus-self. Consumed by `GET /api/hives`
|
||||||
# (swarm-controller/src/main.rs::load_hives).
|
# (swarm-controller/src/main.rs::load_hives).
|
||||||
SWARM_CONTROLLER_HIVES = builtins.toJSON (
|
SWARM_CONTROLLER_HIVES = builtins.toJSON (
|
||||||
|
|
|
||||||
|
|
@ -48,7 +48,7 @@ in
|
||||||
|
|
||||||
services.hyperhive.swarm.hives.<name> = {
|
services.hyperhive.swarm.hives.<name> = {
|
||||||
domain = "<the old attrset key>";
|
domain = "<the old attrset key>";
|
||||||
# certFingerprint / wireguard* carry over unchanged
|
# wireguard* carries over unchanged
|
||||||
};
|
};
|
||||||
|
|
||||||
Still set: ${lib.concatStringsSep ", " (lib.attrNames peers)}
|
Still set: ${lib.concatStringsSep ", " (lib.attrNames peers)}
|
||||||
|
|
@ -61,9 +61,7 @@ in
|
||||||
(services.hyperhive.swarm.ca — see docs/swarm/ca.md): every hive
|
(services.hyperhive.swarm.ca — see docs/swarm/ca.md): every hive
|
||||||
under it chains to it, so a per-hive CA is dead weight. What this
|
under it chains to it, so a per-hive CA is dead weight. What this
|
||||||
genuinely drops is trusting a hive whose root this swarm does NOT
|
genuinely drops is trusting a hive whose root this swarm does NOT
|
||||||
own — another swarm's, or one keeping its own CA. certFingerprint
|
own — another swarm's, or one keeping its own CA.
|
||||||
does not cover that: it pins a leaf for hive-c0re's own HTTPS
|
|
||||||
checks and does not reach Matrix federation.
|
|
||||||
|
|
||||||
Still set on: ${lib.concatStringsSep ", " withCaCert}
|
Still set on: ${lib.concatStringsSep ", " withCaCert}
|
||||||
'';
|
'';
|
||||||
|
|
|
||||||
|
|
@ -1,7 +1,7 @@
|
||||||
# The WireGuard inter-hive mesh for the local host. Split out of
|
# The WireGuard inter-hive mesh for the local host. Split out of
|
||||||
# ./swarm.nix because the two are different concerns with different
|
# ./swarm.nix because the two are different concerns with different
|
||||||
# audiences: that file declares WHO the peers are (data hive-c0re
|
# audiences: that file declares WHO the peers are (consumed by
|
||||||
# serialises into HYPERHIVE_PEERS for agent identity), while this one
|
# swarm-controller's hive roster and, here, the mesh), while this one
|
||||||
# is plain host networking that a machine which runs no hive at all
|
# is plain host networking that a machine which runs no hive at all
|
||||||
# --- the snapshot store, for one --- still needs.
|
# --- the snapshot store, for one --- still needs.
|
||||||
#
|
#
|
||||||
|
|
@ -80,9 +80,9 @@
|
||||||
# networking, not a c0re feature: a swarm host that runs no hive —
|
# networking, not a c0re feature: a swarm host that runs no hive —
|
||||||
# the snapshot store, for one — still has to join the mesh, and under
|
# the snapshot store, for one — still has to join the mesh, and under
|
||||||
# the old `c0re.enable` gate it silently got no `wg-hive` interface
|
# the old `c0re.enable` gate it silently got no `wg-hive` interface
|
||||||
# at all. Nothing below is c0re-specific; the peer data
|
# at all. Nothing below is c0re-specific; the peer data c0re consumes
|
||||||
# c0re consumes (HYPERHIVE_PEERS / HIVE_PEER_CA_PATHS) is rendered in
|
# (HIVE_PEER_CA_PATHS) is rendered in ./hive-c0re and stays gated
|
||||||
# ./hive-c0re and stays gated there.
|
# there.
|
||||||
config = lib.mkIf config.services.hyperhive.swarm.wireguard.enable {
|
config = lib.mkIf config.services.hyperhive.swarm.wireguard.enable {
|
||||||
assertions = [
|
assertions = [
|
||||||
{
|
{
|
||||||
|
|
|
||||||
|
|
@ -112,10 +112,9 @@ struct AppState {
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Env var the controller's NixOS module sets from
|
/// Env var the controller's NixOS module sets from
|
||||||
/// `services.hyperhive.swarm.hives`, JSON-encoded — same shape hive-c0re
|
/// `services.hyperhive.swarm.hives`, JSON-encoded — the full directory
|
||||||
/// already builds for `HYPERHIVE_PEERS`
|
/// (this daemon has no "self" to exclude) rather than peers-minus-self
|
||||||
/// (nix/host-modules/hive-c0re/environment.nix), just the full directory
|
/// (`services.hyperhive.swarm.peerHives`, which other consumers use).
|
||||||
/// (this daemon has no "self" to exclude) rather than peers-minus-self.
|
|
||||||
const HIVES_ENV: &str = "SWARM_CONTROLLER_HIVES";
|
const HIVES_ENV: &str = "SWARM_CONTROLLER_HIVES";
|
||||||
|
|
||||||
/// Parses [`HIVES_ENV`] into the swarm's hive directory. Unset or
|
/// Parses [`HIVES_ENV`] into the swarm's hive directory. Unset or
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue