One-time-setup convenience for the inter-hive WireGuard mesh
(services.hyperhive.swarm) so nobody has to remember the wg dance:
- hivectl wg init [--address X] — generate (if absent) the hive's
private key at /etc/wireguard/hive.key (0400, never clobbered),
derive + print the public key, and print the swarm.wireguard nix
snippet to enable the mesh.
- hivectl wg peer <domain> --pubkey --address [--endpoint] — print the
swarm.peers.<domain> nix snippet to add a remote hive.
- hivectl wg status — wrap wg show wg-hive.
Hybrid model per the design: the verb owns the imperative state (the
key file), the operator pastes the printed nix into host config (kept
in git) — nothing mutates declarative config behind their back.
hivectl-only (root host ops, like the gateway htpasswd verbs); no
priv/wire/c0re changes.
flake: wrap hivectl with wireguard-tools on PATH so wg resolves even
before the mesh config (which would otherwise pull it in) exists —
wg init is the first setup step. Add clippy.toml doc-valid-idents for
the WireGuard proper noun. Regenerate hivectl-cli.md.
Add a 'hivectl completions <shell>' subcommand (clap_complete) that
prints a completion script for bash/zsh/fish/elvish/powershell, generated
from hivectl's own clap command tree so it never drifts from the real
verbs/flags. The package build installs the bash/zsh/fish scripts via
installShellFiles, so an operator gets working completion automatically
once hivectl is on their profile with shell completion enabled.
Regenerated docs/tools/hivectl-cli.md for the new verb.
- Replace the #-number tracker references in code comments with prose
(tracker-tag lint; hive convention is prose in source).
- ensure_agent_subvolume now treats a chown failure on the freshly
created subvolume as fatal: it rolls the subvolume back (deletes it)
and returns an error, instead of warning and leaving a root-owned
subvol that hive-c0re can't write into (which would also make the
c0re-side exists-check skip the retry, wedging the agent).
Progressive enhancement: a brand-new agent's state root under
/var/lib/hyperhive/agents is created as a btrfs subvolume when the host
filesystem is btrfs, otherwise it falls back to a plain directory. No
existing agent is auto-migrated — the new path only fires when the root
does not yet exist, so plain-dir agents are left untouched until an
explicit opt-in upgrade.
Two new privileged ops (subvolume create/delete are root-only):
EnsureAgentSubvolume statfs-gates on btrfs, creates the subvolume, and
chowns it to the hive-core user so the normal state/claude/harness
mkdirs succeed inside it; DeleteAgentSubvolume btrfs-subvolume-deletes
the root iff it is actually a subvolume. hive-c0re calls Ensure before
the per-agent dirs are created (spawn/rebuild/InitConfig) and Delete on
the purge path only — destroy keeps the subvolume for revival, matching
plain-dir semantics. btrfs-progs added to the hive-priv unit PATH.
Per-subvolume usage accounting + optional quota is a separate
follow-up.
Drop the enumeration of what the flag does not apply to, per operator
review on the sibling graceful-stop change. Regenerate the CLI doc so
the markdown-docs self-diff check stays in sync.
`hivectl restart [scope]` cycles the scoped containers — composes the
existing stop + start daemon ops client-side (reusing the merged Stop/Start
wire ops + global --socket), so no new wire/c0re surface. Same scope model
as stop/start (--agents/--agent/--ci/--forge/--gateway/--matrix), and
--graceful on the stop half. If the stop phase reports a failure the start
phase is skipped so a half-stopped hive isn't blindly started over.
Regenerated docs/tools/hivectl-cli.md.
The host admin socket path was duplicated on every daemon-assisted verb
(agents restart/restart-all, stop, start). Hoist it to a single global arg
on the top-level Cli (`--socket`, default DEFAULT_HOST_SOCKET, accepted
before or after the subcommand) and thread cli.socket through dispatch.
Verbs that don't talk to the daemon ignore it. Pure CLI-ergonomics change;
no wire/behaviour change. Regenerated docs/tools/hivectl-cli.md.
Per review: RESTARTABLE_INFRA_CONTAINERS and the new CONTROLLABLE_INFRA_CONTAINERS
were near-identical subsets of SIBLING_CONTAINERS. Drop both and validate infra
lifecycle ops against SIBLING_CONTAINERS directly (all four infra containers;
hive-c0re is never in it, so it can't stop itself). This also makes hive-matrix
restartable, including via an infra_admin agent's restart tool.
Collapse the two priv ops too: RestartInfraContainer is gone; ControlInfraContainer
{ action } is the single op (restart = action: Restart). priv_client's
restart_infra_container is now a thin wrapper over control_infra_container.
Per review: c0re expands the LifecycleScope to explicit container-name
lists (scoped_agents / scoped_infra) in the dispatch arm, then hands those
lists to handle_stop / handle_start. The 'all agents' flag no longer flows
past the resolution boundary, so downstream consumers (incl. the future
graceful-stop queue) always operate on concrete names. CLI --agents flag
unchanged.
Until the per-agent quiesce lands, --graceful falls through to a hard
stop. Help-text the limitation so an operator passing the flag isn't
misled into thinking the agent quiesced.
`hivectl stop` brings the whole hive down in one operator action — all
sub-agents plus the ci/forge/gateway/matrix infra containers — and
`hivectl start` brings it back up. Scope flags (--agents, --agent <name>,
--ci, --forge, --gateway, --matrix) narrow the set; a bare invocation
targets everything. hive-c0re never stops itself.
- hive-sh4re: HostRequest::{Stop,Start} + LifecycleScope wire type;
priv_proto InfraAction + ControlInfraContainer + the
CONTROLLABLE_INFRA_CONTAINERS allowlist (adds hive-matrix, excludes
hive-c0re).
- hive-priv: control_infra_container handler (systemctl <verb>
container@<name>, allowlist-validated root-side).
- hive-c0re: handle_stop / handle_start fan out agents via lifecycle and
infra via hive-priv; per-target failures are aggregated. Infra
systemctl routes through hive-priv (the privsep boundary).
- The --graceful flag is threaded through Stop now; the per-agent quiesce
itself lands with the graceful-agent-stop work.
Wire agents to trust the gateway's self-signed leaf at build time. When
the gateway runs self-signed TLS, hive-tls sets HIVE_TLS_CA_PATH in
hive-c0re's service env pointing at the host hive CA cert. The meta flake
renderer reads it and, when present, writes the public CA cert next to
flake.nix as hive-ca.pem and emits security.pki.certificateFiles so every
agent's system trust store includes the hive CA.
Build-time embedding (rather than a runtime bind-mount + bundle service)
keeps trust robust: the CA travels with the flake source, lands in the
standard NixOS trust store, and needs no per-process SSL_CERT_FILE
plumbing. Only the public CA certificate is embedded; the CA private key
never leaves the host. The cert is re-embedded and re-committed on CA
rotation even when the flake is otherwise byte-identical; when self-signed
TLS is off the embedded cert is dropped so the flake stays buildable.
Covers OpenSSL-based tools (git, curl) directly. A follow-up switches the
hive-forge reqwest client to native roots so it picks up the same store.
The forwarded host env vars (HIVE_FORGE_URL, HIVE_FORGE_PUBLIC_URL,
HYPERHIVE_HIVE_DOMAIN, ...) were emitted only into the harness
service's own unit environment. Under network isolation that left the
bash-task runner, the matrix daemon, tea-login and interactive shells
without HIVE_FORGE_URL, so `hive-forge` and friends fell back to the
loopback default (localhost:3000) — unreachable from a private netns,
where the in-cluster gateway URL (forge.<domain>) is the only path.
Emit the forwarded set into systemd.globalEnvironment too, which every
unit + shell in the container inherits, so all in-container surfaces
see the same forge/matrix endpoints. The existing harness-service
forward is kept (harmless redundancy). Shared netns is unaffected (the
localhost default still works there).
Adds a regression test asserting the forwarded vars land inside the
globalEnvironment block, and the generated flake parses.
argus: record() is pub and returns Option<AuditEntry>; the hive rule
requires #[must_use] on public fns returning a non-trivial value. Call
sites already consume the return (handle_restart_infra via if-let, tests
via let _).
Follow-up to the audit-log backend + surface. Emits a DashboardEvent on
each agent-initiated privileged action so the audit view live-appends off
/dashboard/stream instead of polling.
- new DashboardEvent::AuditEntryAdded { seq, <flattened AuditEntry> } —
serde tag `audit_entry_added`; the AuditEntry fields flatten to the top
level so the wire shape matches one /api/audit-log `entries` row exactly.
- Coordinator::emit_audit_entry helper (stamps seq like the others).
- audit_log::record now returns the canonical inserted AuditEntry (id + ts
assigned) so the streamed event is the same row that was stored — no
drift. Best-effort unchanged (None on a sqlite blip).
- handle_restart_infra records + emits for every attempt (ok/err/denied),
threading the coordinator through.
Tests: kind_tag round-trip now covers the new variant; added a flatten
test pinning the top-level wire shape (kind/seq/id/…/detail, no nesting).
Pairs with iris's audit view (the /dashboard/stream listener half).
argus review: the four public Result-returning methods (open, list_recent,
count_total, vacuum) need # Errors sections per the hive Rust doc rules;
open had no doc comment. Added all four.
Per iris's dashboard-side ask: return { entries, total } instead of a
bare array so the UI can show 'latest 500 of N' rather than silently
capping at the clamp. Adds AuditLog::count_total().
Adds a durable, operator-visible audit trail of privileged operations
hive-c0re performs on behalf of an agent — the ones that cross the
agent/operator trust boundary. First entry: infra-container restarts via
the infra_admin-gated `restart` tool, which until now were recorded only
as a hive-priv journal trace.
Backend:
- new `audit_log` module: sqlite-backed store (audit_log.sqlite, same dir
as build_logs) with schema (ts/agent/action/target/outcome/detail),
best-effort `record`, `list_recent` (clamped 500), 90-day `vacuum`, and
a process-singleton handle mirroring build_logs.
- Coordinator opens + installs the handle; main spawns the hourly vacuum.
- agent_server::handle_restart_infra records every attempt (ok, error, and
capability-denied) via the global handle — best-effort, never fails the
underlying action.
- dashboard: `GET /api/audit-log` returns recent entries as JSON.
Scope is deliberately agent-initiated privileged actions only (not every
PrivRequest — token writes + nspawn edits are constant lifecycle noise).
Extensible: future agent-initiated priv ops record via the same handle.
Unit tests cover record/list ordering, the 500 clamp, and retention vacuum.
The dashboard *surface* (an AUDIT view consuming /api/audit-log) is a
frontend follow-up coordinated with iris.
The dashboard LOAD tab always showed "no running agent containers":
container_stats::gather() looked for each agent's cgroup at
machine.slice/machine-h\x2d<name>.scope, but that path never exists.
nixos-container runs `systemd-nspawn --keep-unit` with
`Slice = "machine.slice"` (nixpkgs virtualisation/nixos-containers.nix),
so --keep-unit means no separate machined `machine-<name>.scope` is
created — the container's cgroup IS the launching service unit,
`container@<machine>.service`, under machine.slice. systemd-machined
still logs "New machine <name>" (registration), which is what made the
scope-path assumption look plausible, but the cgroup stays on the unit.
Fix scope_dir to machine.slice/container@<machine>.service and drop the
\x2d escaping (the service unit name is used verbatim in the cgroup dir;
the journal shows the literal `container@h-<agent>.service`). Adds a unit
test pinning the path.
When isolateContainers=true, claude (and all egress) broke in every
container: agents came up with an IP but no way off the bridge subnet.
Two container-side gaps, both confirmed against nixpkgs
nixos-containers.nix:
1. No default route. hive-priv wrote HOST_ADDRESS= empty in the nspawn
conf. nixos-container's container-side setup only installs
`ip route add default via $HOST_ADDRESS` when HOST_ADDRESS is
non-empty, so the container had an address but no gateway -> nothing
off-subnet (incl. api.anthropic.com) was reachable. Fix: write
HOST_ADDRESS=<bridge-ip>. In bridge mode the host-side address/route
setup is skipped, so this only affects the container's default route.
2. No usable resolver. nixos-container copies the host's /etc/resolv.conf
into the container at every start; the host resolver (e.g. 127.0.0.53)
is unreachable from a private netns and isn't authoritative for the
hive's own zones. Fix: hive-priv drops a marker carrying the gateway
IP only when isolated, and a new harness-base oneshot
(hyperhive-isolated-dns) rewrites resolv.conf to point at the bridge
dnsmasq. Inert in shared-netns mode (no marker), so the shared
container toplevel does the right thing in both modes.
The gateway IP is the address part of HIVE_NETWORK_SUBNET (the bridge IP
verbatim, honouring a non-.1 operator override), via a new validated
bridge_gateway_ip() helper with unit tests.
Unblocks defaulting isolation on.
Clean the legacy backlog so the tracker-tag lint can become a required
gate (mara's warn-during-cleanup -> full-tree-enforcement path). Rewrite
the ~33 real `closes/see #NNN` provenance refs in doc-comments to prose
across hive-forge, hive-c0re, hive-ag3nt, hive-matrix-mcp, hive-sh4re,
and add a `lint:allow` line marker to check-issue-refs.sh for genuine
non-tracker `#<digits>` (a hash-digit heading-detection test input).
Tree is now lint-clean; tracker-tag lint ready to promote to required.
