| Filename | Latest commit message | Latest commit date |
|---|---|---|
8891b46943
The controller connects to the swarm queue as its own client and serves what each hive last said about itself at GET /api/hives/status. THE QUEUE IS THE STORE. A hive publishes into the `hive-status` JetStream KV bucket (history 1) and the controller reads it per request, keeping no copy. A cache here would be a second answer to the same question, free to disagree with the first, and the disagreement surfaces as a hive reading healthy on a dashboard while the bucket says otherwise. Whichever side arrives first creates the bucket; both want the same shape. Rows come from the roster rather than from the bucket, so an empty bucket renders as a swarm nobody has heard from instead of a healthy one, and `never_reported` stays distinct from `stale` - went quiet is a fault, never spoke is usually a deployment that has not happened. Freshness is derived at read time and never stored as a flag, because a stored `healthy` boolean goes stale silently the moment nothing arrives, which is the failure this endpoint is designed against. The timestamp is the NATS server's, applied when the value landed, so a publisher cannot make itself look fresher than it is. Authentication is per connection attempt, not per process. Authelia issues `client_credentials` tokens that expire in 3599s, and auth happens at CONNECT, so a long-lived connection is fine but a reconnect an hour later needs a token minted an hour later. `with_auth_callback` is re-run by async-nats for each attempt, which handles expiry by construction rather than by a timer - the alternative fails in the way this subsystem exists to prevent, with the controller still serving while its data quietly stops updating. Three failure shapes are deliberate: - A half-set environment is fatal; an absent one is not. Silently behaving like an unconfigured host is how every hive ends up reading `never_reported` with nothing to point at. - The endpoint answers 503 rather than an empty list when the store cannot be read. "I cannot reach the store" and "every hive is silent" are different answers, and rendering the second turns a local fault into an apparent swarm-wide outage. - `retry_on_initial_connect` makes the daemon and the queue bootable in either order, and the status handler refuses when the client is not Connected rather than issuing a request into it - a request made in that window does not fail, it waits, so every poll would hang and learn nothing. `Pending` is the state a never-connected client is in, which is why the test is `!= Connected` and not `== Disconnected`. The rendering rules are a pure function over a map, so the semantics are tested against a table rather than against a running server. The KV read, the credential rotation and the 503 paths are covered behaviourally instead: a real NATS server with a rotating token endpoint, asserting that the controller recovers only when the credential rotates, and mutation-tested by holding the credential wrong for the same window. |
||
| .. | ||
| src | ||
| Cargo.toml | ||
| README.md | ||
swarm-controller
The swarm-level daemon. Where hive-c0re owns the agents on one host, this
owns what is true across hives — so a swarm runs one of them and most hives
leave it off.
Opt-in per host via services.hyperhive.swarm.controller.enable, which is
deliberately not derived from services.hyperhive.enable: turning it on is
a statement about swarm topology, not about whether hyperhive is installed.
What it does today
Serves one /health endpoint and holds no state.
That is the whole intent of the first slice. The point is to make the unit
real — service user, runtime and state directories, socket, nginx
reachability — so the swarm-level surfaces that follow have somewhere to land.
Inventing those surfaces before they are agreed would bake in a shape nobody
chose. See #3066 and the hyperhive.swarm consolidation epic.
Why a unix socket, not a port
The hive-gateway's nginx is the only intended client and reaches the socket through a bind-mount. A listener that is never bound to an address cannot be reached from off-host by mistake.
The socket path is services.hyperhive.swarm.controller.socketPath, default
/run/swarm-controller/controller.sock, exported to the process as
SWARM_CONTROLLER_SOCKET.
⚠️ The socket's directory is its access control
The socket is 0666. It has to be: nginx runs as a different user and
connect(2) needs write. This matches how hive-c0re publishes the per-agent
sockets, and rests on the same argument — "the bind source dir is per-agent on
host so blast radius is unchanged."
What keeps that safe is that the directory holds one socket. So:
Never point
socketPathat a directory that carries anything else./run/hyperhiveabove all — it holdshost.sock, the host admin socket. Pointing nginx at that directory to reach this socket would put the admin socket within its reach too.
nginx is a host service, so nothing narrows what it can reach except the directory itself — that is the whole of the access control. A unit test pins the default path so a tidying edit fails instead of reviewing cleanly.
RuntimeDirectoryPreserve=yes and the daemon's stale-socket unlink on start are
a pair: preserving the directory without the unlink means bind fails with
EADDRINUSE after a restart.
Packaging
Built by the workspace derivation and extracted as its own package
(nix build .#swarm-controller). Deliberately not in nix/packages'
daemonBins — that list is the core stack and drives the bundle
services.hyperhive.c0re.package points at, so folding this in would put a
swarm-scoped service into every hive's closure.