The gateway's nginx + dnsmasq no longer run in their own nspawn container.
`nix/host-modules/hive-gateway/default.nix` loses the
`containers.hive-gateway` wrapper and everything that existed only to punch
holes in it: `privateNetwork = false`, `CAP_NET_ADMIN`, five bind mounts,
its own `stateVersion`, `networking.firewall.enable = false`,
`networking.resolvconf.enable = false`, and the `hive-gateway-resolv`
path+service pair. 465 -> 303 lines.
The container never bought isolation here. It shared the host netns by
necessity — nginx binds the host's :80/:443, dnsmasq answers on the bridge —
so each of those settings was undoing a boundary the gateway could not
afford in the first place.
Four things made it more than a deletion, none of them visible in the nix
diff:
- The self-signed cert service also imports the hive CA leaf, so removing it
with the container would have left nginx naming a missing cert file, which
it refuses to load at all.
- The nginx reload is a hive-priv verb. It still needs root, but no longer
for the reason its doc gave, and `--machine=` was both transport and
scope — so the unit name is now hard-coded in the helper as the
containment.
- The lifecycle verb named a container that stops existing.
- `journalctl -M hive-gateway` had no machine to enter.
Per the operator's ruling, the operator verb keeps working and agents lose
it. `InfraContainer` answered three questions that used to share an answer;
it now splits into `name()` (identity), `target()` (Container vs HostUnit),
`service_unit()` (the systemd unit), and `agent_restartable()`, which the
MCP restart path checks before the capability so the refusal cannot read as
"ask for infra_admin". `SIBLING_CONTAINERS` drops the gateway — it gates the
requests that name a container as a string — while `FromStr` still accepts
it, because that answers what a name is, not who may act on it. The
dashboard's gateway journal reads host journald filtered to `nginx.service`.
Prose was corrected where it only named a location, and re-argued where the
container was doing security work: a `0666` per-agent socket was safe
because only the gateway container had the directory bind-mounted. There is
no mount now, so the directory permissions are the whole of the access
control — the constraint holds, its mechanism doesn't.
Gate: nix fmt / clippy --all-targets -D warnings / cargo test all clean (710
tests); hivectl-cli.md regenerated from the clap tree. The nix eval was run
in both TLS shapes at this commit: every delta in the rendered
virtualHosts is one of the three intended path moves, dnsmasq settings are
byte-identical, and the absence probe flips true -> false with bindMounts
emptied.
One attrset describing every hive in the swarm including this one,
identical on every host, with hiveName selecting which entry is us.
"My peers" is derived (swarm.peerHives) rather than declared.
Every field in the old per-host peer list was intrinsic to the hive it
described, never to the pair -- so the list was a directory each host
kept its own copy of. Beyond the deduplication it removes a bug class:
two hosts could hold different endpoints for the same third hive with
nothing to detect the disagreement.
Drops the per-hive caCert. Trust inside a swarm derives from the swarm
root, which every hive chains to. What that genuinely removes is
trusting a hive whose root this swarm does not own -- a cross-swarm
problem that wants a mechanism of its own, not a field that happened to
work.
The matrix container's certificateFiles block goes with it and could
NOT be migrated: that list is read at build time and the swarm root is
a runtime file (its key must never enter the store), so there is no
build-time name to put there. caCert being a nix path was precisely
what made it the build-time distribution channel. Agents are unaffected
-- hive-tls folds the root into the hive trust bundle and the meta
renderer embeds that one file. Tracked separately.
Migration is an assertion plus warnings, not a rename: hives is peers
union {self}, and the set gains a member no existing config has written
down. A rename migrates a name and a default can re-root a meaning;
neither can conjure a new member. The warning explains, the self-entry
assertion stops the build.
docs/tools/hivectl-cli.md is generated by `hivectl markdown-docs` and
diffed against a fresh run by the hivectl-docs flake check, so adding
the push verb without regenerating it would have failed CI. The
regeneration also retires two copies of a sentence describing the
cross-hive leg as an ssh pipe that "isn't wired up yet" -- ssh was
dropped when the WireGuard mesh became the authentication, and the leg
is wired up now.
snapshot-store.md documented only the receiving host, so a reader had
no way to learn how a pushing hive is told where the store is. It now
covers services.hyperhive.swarm.snapshotStore, including why address
has no default and port does: an address is a deployment fact that
cannot be guessed, a port is a convention both ends read from the same
docs. It also states the namespace rule the two options illustrate --
swarm.* describes the swarm as seen from here, a bare
services.hyperhive.<service> describes a role this host performs.
swarm.md never mentioned the store even though the option lives in its
namespace, so a reader configuring swarm.peers had no signal it exists.
Both groups only ever act on a single managed agent's state dir, so
they belong in the `agents` namespace rather than as top-level verbs
next to `forge` / `matrix` / `wg`.
Renames `quota limit` -> `quota set`: the enclosing group already
carries the noun, so the bare verb matches the flat `set-parent` /
`set-limits` spelling without stuttering, and it removes the
`set-limits` (cpu/mem) vs `quota limit` (disk) ambiguity. Adds a
cross-pointer from `set-limits` to `agents quota`.
Handlers stay in their own modules; `run_agents` gains the reparenting
glue. Regenerates docs/tools/hivectl-cli.md.
Refs #2724
The hive applies one `agentCpuQuota` / `agentMemoryMax` to every
container. That's the right default and the wrong ceiling: a build-heavy
agent needs headroom the other twelve don't, and raising the hive-wide
value to suit it hands that headroom to everyone.
Adds a per-agent override, persisted host-side and resolved per-field
against the hive defaults.
Follows the existing `meta/*.json` pattern (`capabilities.json`,
`tool-groups.json`): a host-side map read by `hive-c0re`, staged and
committed in the meta repo so every change lands in the audit trail.
```json
{ "sock": { "cpu_quota": "400%", "memory_max": "8G" } }
```
Fallback is **per field**, not per agent: an entry with only
`memory_max` leaves that agent on the hive-wide CPU quota. Absent file,
absent agent and absent field all resolve to the hive default, so the
feature is inert until someone opts an agent in.
Unlike the other meta files this one is **not** injected into the
container — a limit is something done *to* an agent, not something it
reads about itself.
```
hivectl agents set-limits sock --cpu-quota 400% --memory-max 8G
hivectl agents set-limits sock --reset
```
Values are validated before they're persisted: they go into a systemd
drop-in verbatim, and a typo there makes the unit fail to *start* —
turning a fat-fingered quota into a container that won't come back.
The command is declarative: each call replaces the agent's whole entry.
That makes a forgotten flag a silent revert, so a bare `set-limits
<name>` is rejected at the clap layer and clearing needs an explicit
`--reset`.
`ContainerView` gains `cpu_quota` / `memory_max`, both always populated:
there's no "unset" state to render, only "same as everyone else". They
reflect what the drop-in *says* — what the next start will enforce — not
a live cgroup reading.
The write goes through `meta::commit_resource_limits` rather than the
bare setter, so it's staged and committed under `META_LOCK`. Writing
without committing would leave the meta working tree dirty for the next
`prepare_deploy` to trip over.
Docs: `persistence.md` (the new meta file, and why it isn't injected),
`tools/hivectl.md` (the prose guide), `tools/hivectl-cli.md`
(regenerated clap dump).
Closes: internal/requests issue 25
- regenerate docs/tools/hivectl-cli.md for the new `subvol snapshot send` verb
- close the TOCTOU on the no-overwrite guard: File::options().create_new(true)
(O_CREAT|O_EXCL) instead of exists()-then-create, so the guarantee is
atomic against a concurrent request racing the same dest filename
- warn (not silently swallow) if cleaning up a partial export after a
failed btrfs send itself fails, so a stuck garbage file masquerading
as a completed export is visible in the log
Per mara's PR review:
- snapshot label is now mandatory (was optional w/ timestamp default)
and must start with "hive-" — hive-priv enforces this as an
allow-list on top of the existing credential-name charset check, so
only hivectl-issued labels can reach the btrfs shellout.
- nest under `subvol snapshot create`/`subvol snapshot delete`
instead of othering delete as a separate top-level `delete-snapshot`
verb.
Per argus's review:
- regenerate docs/tools/hivectl-cli.md (hivectl markdown-docs) to
include the new subcommands — CI's hivectl-docs-fresh check compares
this file against generated output.
every agent lifecycle verb on the admin socket (rebuild / restart /
restart-all / kill / stop / start) now submits job-queue DAGs and
returns their ids; hivectl polls the new HostRequest::QueueDag and
prints a live node-chain progress line per DAG (fan-out children
included), exiting non-zero on failure — --no-wait opts out. DagView
and the queue wire enums move to hive_sh4re::jobs (wire types live in
the shared crate); the last fused rebuild path (lifecycle::rebuild)
is gone. tracker: #2166
coordinator.md rewrites the queue section (node inventory, DAG shapes,
resources, desired-state reconciliation, boot reconcile); approvals.md
+ persistence.md + hivectl --graceful help updated to match. agent_power
lives in broker.sqlite like approvals/questions (own connection + busy
timeout) instead of a separate db file.
Per operator review (#1899): the hive's own domain should never be
typed by hand. Remove the --domain flag from peer-config and wg init
entirely; both now resolve it from the daemon (HostRequest::HiveDomain).
peer-config errors with a clear hint if the daemon can't answer; wg init
prints the peer block best-effort (skips it, still enables the mesh, if
unresolved). Regenerated docs/tools/hivectl-cli.md.
Per operator review: hivectl shouldn't make the operator retype this
hive's own domain. Add a HostRequest::HiveDomain admin-socket query
(c0re answers from HYPERHIVE_HIVE_DOMAIN, already in its service env) and
a domain field on HostResponse.
hivectl peer-config now resolves the domain as: --domain override (kept
for offline/scripted use), else the daemon query; errors with a clear
hint when neither resolves. wg init resolves it best-effort too, so it
prints the hand-over peer-config block without --domain (skipped, not
failed, when the daemon is unreachable).
Regenerated docs/tools/hivectl-cli.md.
Add `hivectl peer-config --domain <this-hive-domain>`: prints the
`services.hyperhive.swarm.peers."<domain>"` nix block a peer operator
pastes to federate with this hive. Emits `caCert = ./<hive>-ca.pem` plus
a `cp /var/lib/hive-tls/ca.pem ./<hive>-ca.pem` line when this hive
serves a self-signed CA (the cert file exists); omits caCert for
ACME/public-CA hives (trusted by the default bundle). Includes the
WireGuard public key when the mesh key exists, and the
wireguardAddress/Endpoint passed via flags.
`wg init` gains an optional --domain; when set it calls peer-config at
the end, so a fresh mesh setup prints the hand-over block in one command.
Pure output — reads local state (TLS CA cert presence, wg key), never
mutates. Regenerated docs/tools/hivectl-cli.md.
Closes#1897.
New agents get a btrfs subvolume state root automatically when the host
FS is btrfs, but agents that predate that migration are left on plain
dirs and miss the subvolume feature set (snapshots, per-subvol
usage/quota, send/receive migration). Add an opt-in operator verb to
convert an existing plain-dir agent in place.
btrfs cannot promote a directory to a subvolume in place, so the new
privileged op stages a sibling subvolume mirroring the dir (create +
`cp -a --reflink=auto` preserving ownership/permissions/xattrs + match
the root's owner and mode), then atomically renames the original aside
and the subvolume into place, then removes the original. Any failure
before the swap leaves the original untouched; idempotent (no-op if
already a subvolume) and btrfs-gated.
The `hivectl subvol upgrade <agent> --yes` verb composes it client-side
like `restart`: stop the agent so its state bind-mount is released, run
the migration via hive-priv, then restart it — the restart is attempted
regardless of the migration outcome so a failed migration never leaves
the agent down.
- hive-sh4re: UpgradeAgentSubvolume priv request variant.
- hive-priv: the migration handler plus stage/cleanup helpers.
- hive-c0re: priv_client wrapper and the hivectl verb; regen CLI docs.
Follow-up to the btrfs-subvolume migration. Operator-opt-in disk
accounting + quotas on agent state subvolumes via btrfs qgroups:
- three privileged ops (qgroup ops need root): EnsureBtrfsQuota
(btrfs quota enable on the agent-state filesystem — statfs-gated,
idempotent, no-op off btrfs), ReadSubvolumeUsage (btrfs qgroup show
-f --raw for one agent), SetSubvolumeQuota (btrfs qgroup limit, or
clear). Reuses the is_on_btrfs helper from the subvolume work.
- priv_client wrappers, incl parse_qgroup_usage -> (referenced,
exclusive) bytes.
- hivectl 'quota' subcommand: enable / show [agent] / limit <agent>
<size|none>, with a K/M/G/T size parser + human-readable output.
Quota is deliberately NOT auto-enabled: btrfs quota enable triggers a
full rescan that is I/O-heavy on a large filesystem, and the operator
should choose when to pay that. 'quota show' on a plain-dir agent (no
subvolume) reports no qgroup data rather than erroring.
One-time-setup convenience for the inter-hive WireGuard mesh
(services.hyperhive.swarm) so nobody has to remember the wg dance:
- hivectl wg init [--address X] — generate (if absent) the hive's
private key at /etc/wireguard/hive.key (0400, never clobbered),
derive + print the public key, and print the swarm.wireguard nix
snippet to enable the mesh.
- hivectl wg peer <domain> --pubkey --address [--endpoint] — print the
swarm.peers.<domain> nix snippet to add a remote hive.
- hivectl wg status — wrap wg show wg-hive.
Hybrid model per the design: the verb owns the imperative state (the
key file), the operator pastes the printed nix into host config (kept
in git) — nothing mutates declarative config behind their back.
hivectl-only (root host ops, like the gateway htpasswd verbs); no
priv/wire/c0re changes.
flake: wrap hivectl with wireguard-tools on PATH so wg resolves even
before the mesh config (which would otherwise pull it in) exists —
wg init is the first setup step. Add clippy.toml doc-valid-idents for
the WireGuard proper noun. Regenerate hivectl-cli.md.
Add a 'hivectl completions <shell>' subcommand (clap_complete) that
prints a completion script for bash/zsh/fish/elvish/powershell, generated
from hivectl's own clap command tree so it never drifts from the real
verbs/flags. The package build installs the bash/zsh/fish scripts via
installShellFiles, so an operator gets working completion automatically
once hivectl is on their profile with shell completion enabled.
Regenerated docs/tools/hivectl-cli.md for the new verb.
Drop the enumeration of what the flag does not apply to, per operator
review on the sibling graceful-stop change. Regenerate the CLI doc so
the markdown-docs self-diff check stays in sync.
`hivectl restart [scope]` cycles the scoped containers — composes the
existing stop + start daemon ops client-side (reusing the merged Stop/Start
wire ops + global --socket), so no new wire/c0re surface. Same scope model
as stop/start (--agents/--agent/--ci/--forge/--gateway/--matrix), and
--graceful on the stop half. If the stop phase reports a failure the start
phase is skipped so a half-stopped hive isn't blindly started over.
Regenerated docs/tools/hivectl-cli.md.
The host admin socket path was duplicated on every daemon-assisted verb
(agents restart/restart-all, stop, start). Hoist it to a single global arg
on the top-level Cli (`--socket`, default DEFAULT_HOST_SOCKET, accepted
before or after the subcommand) and thread cli.socket through dispatch.
Verbs that don't talk to the daemon ignore it. Pure CLI-ergonomics change;
no wire/behaviour change. Regenerated docs/tools/hivectl-cli.md.
The hivectl-docs flake check regenerates docs/tools/hivectl-cli.md from the
clap command tree and asserts it's committed up to date. Adding the stop/start
verbs changed the CLI, so the doc was stale and the check failed. Regenerate it.
