Six places asserted the old design as fact, and none of them mention the
change by name -- the class of doc breakage that is found by asking what
a diff made untrue, not by grepping for a feature:
- swarmctl/README.md and swarm-authelia-bridge/README.md both described
their own private canonical store. The bridge's "known limitation"
section described the seam as unsolved; it is what this fixes, so it
becomes what both writers must uphold instead.
- docs/swarm/{sso,ui,secrets}.md described a rendered artifact.
- The repo CLAUDE.md entry for swarmctl said the same.
- docs/tools/swarmctl-cli.md is regenerated (CI diffs it against the
clap tree), picking up the removed --store flag.
Operator-facing where it is read: the hand-editing consequence (values
survive a rewrite, comments do not) is stated in sso.md, where an
operator is being told to edit the file, rather than only in a module doc.
authelia now watches the users file, so the restart is redundant -- and it
was the wrong shape twice over. It could fail: a login was refused for a
user whose record was already correct on disk, with nothing in either log
implicating the reload. And it only ever worked for this writer --
swarm-authelia-bridge writes the same file and cannot restart anything,
since running unprivileged inside the container is the whole reason it may
write it at all. A reload that depends on which process did the writing is
not a reload.
--machine/--unit and their two env vars existed solely to name a
systemctl -M target, so they go with it. That drops two required settings
from the operator surface.
The three objections previously recorded against watch are all answered
now, and are kept next to the decision rather than deleted: the key is
verified against the pinned build (validate-config accepts it and rejects
a misspelling), the watch is on the directory so a rename is observed, and
partial reads are structurally impossible because every writer of this
file goes through write_atomic.
Mirrors hivectl exactly: a `completions <shell>` verb that walks the
live clap tree, and a package that pipes it into installShellCompletion
for bash/zsh/fish. Generating from the command tree rather than writing a
script by hand is what keeps completions from drifting away from the
verbs they complete — the same reason `markdown-docs` renders the docs
from that tree.
Dispatched before PathArgs::resolve() for the same reason markdown-docs
is: emitting a completion script needs none of the SWARMCTL_AUTHELIA_*
deployment env vars, and requiring them would make the package's own
build-time invocation fail — exactly where it runs.
swarmctl leaves mkBinPackage for its own derivation, since the extractor
installs a binary and nothing else.
`user add` refuses on an existing name, so the `--group` flag it takes at
creation time could not be added afterwards at all: repairing an account
meant hand-editing both users.json and the rendered users.yml as root.
mara, on #3167: "i will not edit those files by hand, we will have the
same issues elsewhere".
The merge rules live in users.rs as a pure function over a UserUpdate, so
they are testable without a command line, a container or a running
authelia — main.rs's arm only loads, applies, publishes and prints.
Removals are strict and everything else is idempotent, which is the one
asymmetry here and is deliberate: a --remove-group naming a group the
user does not have fails, because a revocation that reports success
without revoking is the outcome nobody re-checks; while refusing an
already-satisfied set would make the multi-attribute call this verb
exists for break whenever one of the values was already right.
A command that changes nothing at all still fails — it would otherwise
rewrite both files and restart the SSO provider to no effect.
Passwords are out of scope: regenerating a credential is a different
intent from editing an attribute, and folded together an attribute edit
can invalidate a login by accident.
Extracts publish() from user_add so both verbs share the
render -> store -> users.yml -> restart ordering and the comment that
explains why that order, rather than the second verb copying it.
`agents.conf` and `gateway.htpasswd` move from /var/lib/hyperhive/gateway
to /var/lib/hive-gateway/conf, alongside the `tls/` the gateway already
kept there.
nginx reads both as an unprivileged user. Under c0re's state dir it could
only reach them by traversing a directory systemd re-declares `0750
hive-core` on every c0re start — so nginx was given `SupplementaryGroups
= [ "hive-core" ]`, which also handed it read access to everything else
group-readable in that tree. The tokens are individually 0600, but the
broker sqlite carries no explicit mode: every message between every agent
was readable by the process whose job is parsing untrusted network input.
Moving the files removes the need and the exposure together. The group is
gone, and its absence is now commented as load-bearing so it doesn't come
back as a fix for a symptom it would recreate.
Also drops this module's `/var/lib/hyperhive` tmpfiles rule. It declared
`0755 root root` and could never win against `StateDirectoryMode`, and a
losing declaration still reads as a guarantee — that is what sent the
first diagnosis of the outage looking for who had changed the mode.
Ordering is unchanged and still the thing that makes a fresh boot work:
tmpfiles runs before services and seeds both files empty-but-valid, nginx
names them (an `include` of a missing file is fatal, not empty), and
content arrives when c0re writes and reloads — which it does on every
topology change, so a boot against the empty seed resolves itself.
Folds in the mode fix: `write` now sets 0644 on the tmp file before the
rename, because a rename carries the source's mode and discards the
destination's, and the tmpfiles rule that declares 0644 is
create-if-absent so it never re-applies.
Adds swarmctl markdown-docs (a hidden Verb, same clap-markdown +
hide=true shape as hivectl markdown-docs) and generates
docs/tools/swarmctl-cli.md from it. Wires a swarmctl-docs freshness
check into nix/checks.nix, same shape as hivectl-docs, diffing against
packages.swarmctl.
One real gotcha: PathArgs::resolve() reads required
SWARMCTL_AUTHELIA_* deployment env vars and errors if unset -
swarmctl markdown-docs must not go through that path (it needs none of
those vars, and the docs build runs it outside any real deployment).
Restructured main() so resolve() only runs for the User arm, not
unconditionally before the match.
Also links the new doc from docs/tools/README.md (new 'for the swarm
operator' section), CLAUDE.md's swarmctl bullet, and
docs/conventions.md's flake-check list.
Verified: cargo check/clippy -D warnings/test/fmt -p swarmctl all
clean; swarmctl markdown-docs diffs clean against the committed doc
(checked against both a plain cargo build and the actual nix build.
#swarmctl output); scripts/check-issue-refs.sh clean.
The gateway's nginx + dnsmasq no longer run in their own nspawn container.
`nix/host-modules/hive-gateway/default.nix` loses the
`containers.hive-gateway` wrapper and everything that existed only to punch
holes in it: `privateNetwork = false`, `CAP_NET_ADMIN`, five bind mounts,
its own `stateVersion`, `networking.firewall.enable = false`,
`networking.resolvconf.enable = false`, and the `hive-gateway-resolv`
path+service pair. 465 -> 303 lines.
The container never bought isolation here. It shared the host netns by
necessity — nginx binds the host's :80/:443, dnsmasq answers on the bridge —
so each of those settings was undoing a boundary the gateway could not
afford in the first place.
Four things made it more than a deletion, none of them visible in the nix
diff:
- The self-signed cert service also imports the hive CA leaf, so removing it
with the container would have left nginx naming a missing cert file, which
it refuses to load at all.
- The nginx reload is a hive-priv verb. It still needs root, but no longer
for the reason its doc gave, and `--machine=` was both transport and
scope — so the unit name is now hard-coded in the helper as the
containment.
- The lifecycle verb named a container that stops existing.
- `journalctl -M hive-gateway` had no machine to enter.
Per the operator's ruling, the operator verb keeps working and agents lose
it. `InfraContainer` answered three questions that used to share an answer;
it now splits into `name()` (identity), `target()` (Container vs HostUnit),
`service_unit()` (the systemd unit), and `agent_restartable()`, which the
MCP restart path checks before the capability so the refusal cannot read as
"ask for infra_admin". `SIBLING_CONTAINERS` drops the gateway — it gates the
requests that name a container as a string — while `FromStr` still accepts
it, because that answers what a name is, not who may act on it. The
dashboard's gateway journal reads host journald filtered to `nginx.service`.
Prose was corrected where it only named a location, and re-argued where the
container was doing security work: a `0666` per-agent socket was safe
because only the gateway container had the directory bind-mounted. There is
no mount now, so the directory permissions are the whole of the access
control — the constraint holds, its mechanism doesn't.
Gate: nix fmt / clippy --all-targets -D warnings / cargo test all clean (710
tests); hivectl-cli.md regenerated from the clap tree. The nix eval was run
in both TLS shapes at this commit: every delta in the rendered
virtualHosts is one of the three intended path moves, dnsmasq settings are
byte-identical, and the absence probe flips true -> false with bindMounts
emptied.
A filter value the forge cannot resolve is DISCARDED, not rejected, so a
typo does not narrow the result set -- it returns the unfiltered one.
That does not waste a query, it inverts the answer: "is anything open in
this milestone" comes back as every open issue and reads as yes, and a
duplicate check gets a list that never narrowed.
`list` now resolves both before querying. Labels reuse the write side's
resolver; the ids are discarded because this endpoint filters by name, so
resolution here is a spell-check rather than a lookup -- reusing it keeps
the message identical to the one the write side has always produced.
Milestones accept a title or an id and are checked against the ALL-state
set: filtering on a closed milestone is a normal query, and validating
against open-only would reject exactly the retrospective ones.
Both fetchers paginate. `repo_labels` asked for one page of 100 and
treated it as the population -- the inverse of the trailer bug, same
root: a valid label past the cut fails to resolve, and the error then
prints an "available labels" list that is itself truncated, so the
message argues for the typo.
`--assignee` / `--author` stay unvalidated on purpose: someone who has
left still legitimately appears on old issues, so a login that is not a
current member is not necessarily a typo.
Also drops the docs paragraph claiming unknown labels are silently
dropped on the write side; that has not been true since the resolver
landed.
One attrset describing every hive in the swarm including this one,
identical on every host, with hiveName selecting which entry is us.
"My peers" is derived (swarm.peerHives) rather than declared.
Every field in the old per-host peer list was intrinsic to the hive it
described, never to the pair -- so the list was a directory each host
kept its own copy of. Beyond the deduplication it removes a bug class:
two hosts could hold different endpoints for the same third hive with
nothing to detect the disagreement.
Drops the per-hive caCert. Trust inside a swarm derives from the swarm
root, which every hive chains to. What that genuinely removes is
trusting a hive whose root this swarm does not own -- a cross-swarm
problem that wants a mechanism of its own, not a field that happened to
work.
The matrix container's certificateFiles block goes with it and could
NOT be migrated: that list is read at build time and the swarm root is
a runtime file (its key must never enter the store), so there is no
build-time name to put there. caCert being a nix path was precisely
what made it the build-time distribution channel. Agents are unaffected
-- hive-tls folds the root into the hive trust bundle and the meta
renderer embeds that one file. Tracked separately.
Migration is an assertion plus warnings, not a rename: hives is peers
union {self}, and the set gains a member no existing config has written
down. A rename migrates a name and a default can re-root a meaning;
neither can conjure a new member. The warning explains, the self-entry
assertion stops the build.
`list` already built its query with `q: None, milestones: None` — both
fields were on the request it was sending. So full-text search over
title and body is a flag, not a new verb, and a text match is only
useful composed with the other filters anyway.
The trailer was the real defect. It fired on `count == limit`, but the
forge clamps page size to its own `api.MAX_RESPONSE_ITEMS`: ask for 400,
get a full 50, and `50 != 400` kept it silent — suppressing the warning
in precisely the case where the truncation is invisible. It now reports
the real total from `X-Total-Count`, which the response header struct
already parsed and the call site discarded. The requested limit is not
clamped client-side: that ceiling is the remote's configuration, not
ours.
Every prose and error-message reference to the moved options, including
two in hive-c0re/src/matrix.rs that omitted the services. prefix and so
read as the per-agent option. An error that names an option the operator
cannot find is worse than no hint.
Part of hyperhive#1898 (b), the other missing docs subdir. Unlike
turn-loop/ this one has no single existing file that already covers
the whole directory - hivectl.md/hivectl-cli.md are genuinely
operator-facing (the operator's own host CLI), while bash.md/forge.md/
lifecycle.md/matrix.md/scheduling.md document the agents' own MCP tool
surface (a different audience: what the agent can do, not what the
operator does). Writes a new README.md rather than moving one,
splitting the link list along that line so the operator-relevant half
leads.
Part of hyperhive#1898 (b): every docs subdir should have a top-level
README.md link, achieved by moving/renaming where an existing file
already fits the role.
docs/turn-loop.md already served as the hub + index for the three
sub-pages under turn-loop/ (claude-invocation.md, config.md, mcp.md),
so it moves wholesale rather than leaving a redundant top-level
pointer stub. Fixes every inbound/relative link across the repo
(top-level README.md, CLAUDE.md, docs/persistence.md,
docs/tools/scheduling.md, the sub-pages own back-link, hive-agent
README + doc comments, hive-agent/Cargo.toml, .prettierignore per-file
exemption entry) - grepped the whole tree for both turn-loop.md and
turn-loop/ to find every reference rather than trusting a partial
list.
nix fmt clean, cargo check -p hive-agent clean.
Per mara on hyperhive#1898: the top-level README should not go into
details about impl details or specific subsystems. Trims the
Multi-account Matrix support and GitHub account sections down to a
one-line pointer each.
The matrixAccounts option detail that was only ever documented in the
README (checked: docs/matrix.md is entirely about the host hive-matrix
container, not this per-agent option) moves to docs/tools/matrix.md
instead, next to the account parameter every matrix tool already
takes - the natural home, not a link to a page that never covered it.
docs/tools/hivectl-cli.md is generated by `hivectl markdown-docs` and
diffed against a fresh run by the hivectl-docs flake check, so adding
the push verb without regenerating it would have failed CI. The
regeneration also retires two copies of a sentence describing the
cross-hive leg as an ssh pipe that "isn't wired up yet" -- ssh was
dropped when the WireGuard mesh became the authentication, and the leg
is wired up now.
snapshot-store.md documented only the receiving host, so a reader had
no way to learn how a pushing hive is told where the store is. It now
covers services.hyperhive.swarm.snapshotStore, including why address
has no default and port does: an address is a deployment fact that
cannot be guessed, a port is a convention both ends read from the same
docs. It also states the namespace rule the two options illustrate --
swarm.* describes the swarm as seen from here, a bare
services.hyperhive.<service> describes a role this host performs.
swarm.md never mentioned the store even though the option lives in its
namespace, so a reader configuring swarm.peers had no signal it exists.
The agents root is 0700 and owned by the daemon's user, so hivectl's
client-side existence guard hit EACCES on traversal for anyone not root.
It reported that as "this command needs root; re-run with sudo", which
turned three verbs' pre-flight check into a permission error about the
wrong thing: `choom`, `subvol upgrade` and `subvol snapshot create` all
failed at the guard rather than at whatever they actually needed.
The daemon runs as the owning user and already answers this question for
its own provisioning paths, so expose it on the host socket as
`AgentExists` and have hivectl ask. Operators reach that socket through
the `hive-admin` group, so the guard now works without sudo.
`choom` still needs root for `machinectl shell` — we ship no polkit rule
granting those actions — so it now checks the effective uid and says so
directly instead of failing later inside systemd's authorisation.
Both groups only ever act on a single managed agent's state dir, so
they belong in the `agents` namespace rather than as top-level verbs
next to `forge` / `matrix` / `wg`.
Renames `quota limit` -> `quota set`: the enclosing group already
carries the noun, so the bare verb matches the flat `set-parent` /
`set-limits` spelling without stuttering, and it removes the
`set-limits` (cpu/mem) vs `quota limit` (disk) ambiguity. Adds a
cross-pointer from `set-limits` to `agents quota`.
Handlers stay in their own modules; `run_agents` gains the reparenting
glue. Regenerates docs/tools/hivectl-cli.md.
Refs #2724
The hive applies one `agentCpuQuota` / `agentMemoryMax` to every
container. That's the right default and the wrong ceiling: a build-heavy
agent needs headroom the other twelve don't, and raising the hive-wide
value to suit it hands that headroom to everyone.
Adds a per-agent override, persisted host-side and resolved per-field
against the hive defaults.
Follows the existing `meta/*.json` pattern (`capabilities.json`,
`tool-groups.json`): a host-side map read by `hive-c0re`, staged and
committed in the meta repo so every change lands in the audit trail.
```json
{ "sock": { "cpu_quota": "400%", "memory_max": "8G" } }
```
Fallback is **per field**, not per agent: an entry with only
`memory_max` leaves that agent on the hive-wide CPU quota. Absent file,
absent agent and absent field all resolve to the hive default, so the
feature is inert until someone opts an agent in.
Unlike the other meta files this one is **not** injected into the
container — a limit is something done *to* an agent, not something it
reads about itself.
```
hivectl agents set-limits sock --cpu-quota 400% --memory-max 8G
hivectl agents set-limits sock --reset
```
Values are validated before they're persisted: they go into a systemd
drop-in verbatim, and a typo there makes the unit fail to *start* —
turning a fat-fingered quota into a container that won't come back.
The command is declarative: each call replaces the agent's whole entry.
That makes a forgotten flag a silent revert, so a bare `set-limits
<name>` is rejected at the clap layer and clearing needs an explicit
`--reset`.
`ContainerView` gains `cpu_quota` / `memory_max`, both always populated:
there's no "unset" state to render, only "same as everyone else". They
reflect what the drop-in *says* — what the next start will enforce — not
a live cgroup reading.
The write goes through `meta::commit_resource_limits` rather than the
bare setter, so it's staged and committed under `META_LOCK`. Writing
without committing would leave the meta working tree dirty for the next
`prepare_deploy` to trip over.
Docs: `persistence.md` (the new meta file, and why it isn't injected),
`tools/hivectl.md` (the prose guide), `tools/hivectl-cli.md`
(regenerated clap dump).
Closes: internal/requests issue 25
A paused agent keeps its container, its claude session and its
dashboard/todo servers up, but stops driving turns. Messages queue
unacked and are drained on resume.
The whole protocol is a single marker file, `<harness>/paused`. That
directory is already a bind-mount shared between host and container, so
both sides just stat the same path: the harness reads it to decide
whether to drive a turn, hive-c0re reads it to render the badge and
writes/removes it for `hivectl pause|resume`. No new wire protocol, no
container round-trip, and it is sticky across restarts by construction.
Not calling `recv_next` while paused *is* the queueing semantic, so
there is no fencing to get wrong: reminders buffer in their unbounded
channel, the todo `Notify` permit coalesces, and a `request_next_turn`
that raced the pause survives because the gate sits above
`self_continue.take()`.
Graceful stop is handled host-side rather than in the harness: a paused
agent provably has no turn in flight, so `run_signal` skips the fence
entirely instead of eating the full `GRACEFUL_STOP_TIMEOUT` waiting for
a checkpoint turn that will never run.
`paused` is reported on `ContainerView` / `AgentStatusRow` for the
dashboard, orthogonal to `running` and reported for stopped containers
too.
Closes: hyperhive/hyperhive issue 2271
The host admin socket `/run/hyperhive/host.sock` was `0660 root:root` (no
SocketGroup), so hivectl needed sudo. Group-own it by a new `hive-admin`
group and add a `services.hyperhive.c0re.adminUsers` allowlist: listed users
join `hive-admin` and drive hivectl without root.
- `SocketGroup = "hive-admin"`, `SocketMode = "0660"` on the hive-c0re.socket
unit.
- `/run/hyperhive` -> `0751` (traverse-only, no listing) so the group can reach
the socket path; the socket's own `0660 hive-admin` mode gates the
connection, and the per-agent subdirs keep their own restrictive perms.
- Empty `adminUsers` (the default) leaves `hive-admin` memberless -> root-only,
as before.
The admin socket is full hive control (spawn/kill/destroy/deploy), so
`adminUsers` is an explicit, opt-in trust grant. Documented in
docs/boundary.md (host admin socket access) + docs/tools/hivectl.md.
Adds a real MCP-side path for the workaround #2639 documented: dial the
in-container HIVE_AGENT_SOCKET directly from cancel_loose_end (kind:"todo")
instead of shelling out via a tracked bash task (nc -U ...), which is what
was spawning a fresh completion todo on every clear and cascading forever.
hive-agent-mcp/src/mcp/render.rs: renamed local_todos's socket-dial guts
into a shared dial_agent_socket(req) helper, added mark_local_todo_done(id)
on top of it (MarkTodoDone request already existed server-side, unused
until now). mod.rs wires kind:"todo" into cancel_loose_end ahead of the
question/reminder/approval parse. args.rs + render.rs + docs/tools/bash.md
text updated to point at the new path instead of the old raw nc invocation.
- regenerate docs/tools/hivectl-cli.md for the new `subvol snapshot send` verb
- close the TOCTOU on the no-overwrite guard: File::options().create_new(true)
(O_CREAT|O_EXCL) instead of exists()-then-create, so the guarantee is
atomic against a concurrent request racing the same dest filename
- warn (not silently swallow) if cleaning up a partial export after a
failed btrfs send itself fails, so a stuck garbage file masquerading
as a completed export is visible in the log
Per mara's PR review:
- snapshot label is now mandatory (was optional w/ timestamp default)
and must start with "hive-" — hive-priv enforces this as an
allow-list on top of the existing credential-name charset check, so
only hivectl-issued labels can reach the btrfs shellout.
- nest under `subvol snapshot create`/`subvol snapshot delete`
instead of othering delete as a separate top-level `delete-snapshot`
verb.
Per argus's review:
- regenerate docs/tools/hivectl-cli.md (hivectl markdown-docs) to
include the new subcommands — CI's hivectl-docs-fresh check compares
this file against generated output.
