argus on #3280: the receiver has no auth extension - the nixpkgs module
passes settings straight through and nothing wires one on - so
'unauthenticated to a bridge address' means reachable from an agent
container, not presents a credential. The firewall is the whole access
control.
Consequence, stated because it is a choice rather than an oversight:
any agent can push arbitrary OTLP and the collector forwards it under
the operator's credential, including data smuggled out in resource
attributes. That is a different risk from the one the collector fixes,
and strictly smaller than what preceded it - before, every agent held
the credential itself and could do all of that plus use the token
anywhere else. The collector removes the token and keeps the pipe.
Same principle this PR already applies to the availability trade: state
it where the reader is, rather than let it be discovered.
The headersCredential section still said hive-c0re forwards the file
into each agent container - that is exactly the behaviour this change
removes, so the doc contradicted the code rather than merely lagging
it. Same for the collector section, which presented the two paths the
change collapses.
Also drops the dangling collector.enable reference in the network
section.
argus's non-blocking note on #3278: docs/observability.md is what this
repo's reading-paths index names as 'what OTEL options are available',
and it did not mention collector.enable/port/upstreamHeaderName at all.
The nix docstrings covered it, but not where a reader following the
established path would look.
Carries the two things a docstring is a poor home for: that endpoint
keeps meaning 'where telemetry ultimately goes' (the agent-facing value
is derived, so an existing deployment is unaffected), and the
availability trade the collector makes against the direct-export
property this page already promises.
Per mara: setup should point at it. The link is placed as a precondition
rather than a see-also -- every step below assumes each credential is
generated where it is read, which is only true all-local.
Per argus: 'with a bound' was missing a word; it now names the actual
120s wait instead of gesturing at one.
Per mara on the CA question: outside all-local this is an ops problem,
and what is missing is documentation rather than machinery.
One row per secret, read against three topologies, because the same
credential is generated in place on one and handed over by an operator on
another. sso.md's table is replaced by a pointer -- two tables listing the
same secrets would drift, and its prose about why a secret is generated
in-container is the half worth keeping there.
States the constraint the whole thing rests on: a hive CA is name-
constrained to the hive domain and a swarm service name is a sibling of
it, so a hive CA cannot issue a swarm-service certificate at all. That is
why placing the swarm root is an operations decision.
The registration half is shared; what each service does with the result
is not, so the differences get a table rather than a second page.
States why matrix reads its secret through LoadCredential and why
/_matrix/ is not a forward-auth surface -- both are conclusions a reader
would otherwise have to re-derive from the module.
Mirrors hivectl exactly: a `completions <shell>` verb that walks the
live clap tree, and a package that pipes it into installShellCompletion
for bash/zsh/fish. Generating from the command tree rather than writing a
script by hand is what keeps completions from drifting away from the
verbs they complete — the same reason `markdown-docs` renders the docs
from that tree.
Dispatched before PathArgs::resolve() for the same reason markdown-docs
is: emitting a completion script needs none of the SWARMCTL_AUTHELIA_*
deployment env vars, and requiring them would make the package's own
build-time invocation fail — exactly where it runs.
swarmctl leaves mkBinPackage for its own derivation, since the extractor
installs a binary and nothing else.
The parent's copy is for reading a child's config; a change to it is a PR
on the child's repo, made from a clone and merged after review. A
writable mount is a second path to the same file that skips that review,
which makes the boundary a convention rather than a permission.
Confirmed with ruth before flipping: it clones from the forge and opens a
PR, including for a brand-new child's first config.
The prose was the larger half. docs/approvals.md did not merely describe
the old mount, it *instructed* agents to use it ("can therefore edit,
commit, and submit changes for any of its direct children directly inside
its container"), and the doc comment in host_config.rs asserted a
dependency that never existed: the InitConfig seed runs as hive-c0re
against the host path, and read_only on a bind constrains writers inside
the container only. That comment is what produced issue #3206, now closed
as invalid.
The step-3 warning said to keep `--group admins` and left the impression
that a later re-run would fix a missing one. It would not: `user add`
refuses on an existing name, so until `user update` existed the only
repair was hand-editing two files as root. mara asked the question the
warning should have answered — "will the cmd just add the group or do i
get a whole new user?".
Deliberately in this PR rather than #3199's: the sentence only becomes
true once the verb it names exists.
`user add` refuses on an existing name, so the `--group` flag it takes at
creation time could not be added afterwards at all: repairing an account
meant hand-editing both users.json and the rendered users.yml as root.
mara, on #3167: "i will not edit those files by hand, we will have the
same issues elsewhere".
The merge rules live in users.rs as a pure function over a UserUpdate, so
they are testable without a command line, a container or a running
authelia — main.rs's arm only loads, applies, publishes and prints.
Removals are strict and everything else is idempotent, which is the one
asymmetry here and is deliberate: a --remove-group naming a group the
user does not have fails, because a revocation that reports success
without revoking is the outcome nobody re-checks; while refusing an
already-satisfied set would make the multi-attribute call this verb
exists for break whenever one of the values was already right.
A command that changes nothing at all still fails — it would otherwise
rewrite both files and restart the SSO provider to no effect.
Passwords are out of scope: regenerating a credential is a different
intent from editing an attribute, and folded together an attribute edit
can invalidate a login by accident.
Extracts publish() from user_add so both verbs share the
render -> store -> users.yml -> restart ordering and the comment that
explains why that order, rather than the second verb copying it.
Per mara on the PR: the guide someone follows on a fresh deploy is a
different page from the one they read while debugging, and only the
second existed.
Also fixes a contradiction the request surfaced: setup.md and
swarm/sso.md have told every operator to create '--group admins' since
the bootstrap step existed, while the new rule required 'operators'.
This is the first rule that CONSUMES a group name, so it takes the one
the guide already creates - inventing a second would have meant every
account made by following the guide silently failing the check it was
supposed to pass, and for mara specifically a migration step that now
may not be needed at all.
setup.md's step 3 says why --group is not decoration; the new step 4
says what decides whether the page opens (the group, and the name
resolving) rather than listing a command, because there is nothing to
run. Steps renumbered, including the matrix block's internal 4a-4e.
New docs/swarm/ui.md (split-page shape, per the docs rule), linked from
the swarm README and added to the gateway's vhost map.
Leads with the step that separates 'protected' from 'locked out':
swarmctl user add <you> --group operators. auth_request asks whether
there is a session; the access_control rule is what makes it mean
operator, and an account created before the rule existed has no groups.
Also records the four wiring sites a swarm service name needs, with the
certificate one called out - serviceDomains is both the sub-CA's
nameConstraints set and the leaf's SANs, and the apex is a sibling of
the other three rather than a parent, so nothing issues for it
implicitly.
The generated landing page's own H1 already said "Crate reference" —
the directory name should match. Rename docs/components/ -> docs/crates/
and update the generating derivation (nix/packages/reference-docs.nix),
its default.nix caller comment, and docs/README.md's link.
Fixes#3193
`agents.conf` and `gateway.htpasswd` move from /var/lib/hyperhive/gateway
to /var/lib/hive-gateway/conf, alongside the `tls/` the gateway already
kept there.
nginx reads both as an unprivileged user. Under c0re's state dir it could
only reach them by traversing a directory systemd re-declares `0750
hive-core` on every c0re start — so nginx was given `SupplementaryGroups
= [ "hive-core" ]`, which also handed it read access to everything else
group-readable in that tree. The tokens are individually 0600, but the
broker sqlite carries no explicit mode: every message between every agent
was readable by the process whose job is parsing untrusted network input.
Moving the files removes the need and the exposure together. The group is
gone, and its absence is now commented as load-bearing so it doesn't come
back as a fix for a symptom it would recreate.
Also drops this module's `/var/lib/hyperhive` tmpfiles rule. It declared
`0755 root root` and could never win against `StateDirectoryMode`, and a
losing declaration still reads as a guarantee — that is what sent the
first diagnosis of the outage looking for who had changed the mode.
Ordering is unchanged and still the thing that makes a fresh boot work:
tmpfiles runs before services and seeds both files empty-but-valid, nginx
names them (an `include` of a missing file is fatal, not empty), and
content arrives when c0re writes and reloads — which it does on every
topology change, so a boot against the empty seed resolves itself.
Folds in the mode fix: `write` now sets 0644 on the tmp file before the
rename, because a rename carries the source's mode and discards the
destination's, and the tmpfiles rule that declares 0644 is
create-if-absent so it never re-applies.
Two claims this branch falsified and left standing, both caught in
review:
`vhosts.nix`'s `errorPages` param comment enumerated the set
(`{ notFound, unreachable, unauthorized }`) and adding a fourth member
made the enumeration wrong at the point a reader consults it.
`gateway.md` said extending custom error pages beyond the per-agent
routes was "a separate follow-up" — while this branch is that follow-up,
so the doc contradicted the code sitting next to it.
Rewrites the scope rule as the criterion rather than a list, since a
list is what went stale: a route earns a page when the default status
would point at the wrong component. That covers the per-agent routes and
the sso vhost, and explains why forge/matrix/fluffychat still don't
qualify — their upstreams being down means what the code says.
`setup.md` is the page someone follows on a fresh deploy, and it had no
step for authelia at all — so the operator finished the walkthrough with
a crash-looping container and a 502, having done everything the doc
asked. The warnings added to sso.md and gateway.md only help someone who
is already debugging; this is the page that decides whether they end up
debugging.
New step 3, gated on `swarm.authelia.enable`, placed before matrix
because it is part of getting the front door open. Says plainly that it
is required to finish the install, names the symptom it prevents, and
links sso.md for the detail rather than restating it.
Renumbers matrix (4), spawn (5) and host commands (6), including the
matrix block's own 3a–3e labels, which would otherwise disagree with
their heading.
`swarm/sso.md` described the empty user store as a resting state — a
provider that is reachable but has nobody in it yet. It isn't. Authelia
validates the store at startup and treats zero users as fatal:
error reading the authentication database: could not validate the
schema: users: non zero value required
so it exits 1, systemd restarts it, and an enabled-but-unbootstrapped
swarm presents as a crash-looping container behind a vhost that is
working correctly. The observed symptom is `502 Bad Gateway`, which
reads as a proxy fault and is not one.
Says so, gives the error text to grep for, and marks the `swarmctl user
add` step as required to finish the install rather than as a first-login
convenience. `gateway.md` gains the same warning next to the vhost,
because that is where someone lands when the 502 is what they can see.
The reason the store ships empty is unchanged and still right: seeding
an account means a credential in a config file. What was wrong was
calling the resulting state harmless.
`DNS:forge.<domain>` and `DNS:matrix.<domain>` sat beside
`DNS:*.<domain>`, which already covers them — a wildcard covers exactly
one label and those are one label. Naming them read as policy, and that
is why they outlived the thing they described: a configured service name
no longer has to be under this hive's domain, and when it isn't, this is
the one list it cannot join.
Proved by execution rather than eval, because the SAN list is a printf
argument filled in at runtime and eval can only show the template — the
lesson from the gateway outage. openssl-verified against a CA carrying
the real name constraint: old and new leaves agree on every name the hive
serves, the wildcard genuinely stops at one label, and a sibling name
added to this leaf fails.
That last case came back stronger than expected: a nameConstraints
violation invalidates the certificate, not the offending SAN, so one
foreign name here would break the leaf for the dashboard and every other
vhost sharing it. The comment says so now, and gateway.md's cert-shape
paragraph no longer describes SANs that are gone.
The vhost half of this change is only useful with an account behind it,
and the provider is generated with an empty user set on purpose. Document
the `swarmctl user add` step rather than automating it: bootstrapping an
IdP non-interactively means a secret arriving from a file, an env var or a
nix expression, all worse than one command typed once.
The gateway and network pages gain the rows they would otherwise be
missing — vhost map, local-dev hosts entry, and the resolver's
authoritative-name list.
The question this answers is "what do I have to configure, and where" —
so the table of secrets is the deliverable and the prose is scaffolding
around it.
The organising idea worth keeping: a secret belongs in-container when
nothing outside that container reads it. Every one of authelia's own
secrets passes that test; the client secret's plaintext fails it, which
is what makes delivery a problem at all rather than a detail.
Adds swarmctl markdown-docs (a hidden Verb, same clap-markdown +
hide=true shape as hivectl markdown-docs) and generates
docs/tools/swarmctl-cli.md from it. Wires a swarmctl-docs freshness
check into nix/checks.nix, same shape as hivectl-docs, diffing against
packages.swarmctl.
One real gotcha: PathArgs::resolve() reads required
SWARMCTL_AUTHELIA_* deployment env vars and errors if unset -
swarmctl markdown-docs must not go through that path (it needs none of
those vars, and the docs build runs it outside any real deployment).
Restructured main() so resolve() only runs for the User arm, not
unconditionally before the match.
Also links the new doc from docs/tools/README.md (new 'for the swarm
operator' section), CLAUDE.md's swarmctl bullet, and
docs/conventions.md's flake-check list.
Verified: cargo check/clippy -D warnings/test/fmt -p swarmctl all
clean; swarmctl markdown-docs diffs clean against the committed doc
(checked against both a plain cargo build and the actual nix build.
#swarmctl output); scripts/check-issue-refs.sh clean.
Names the two ways out (install the sub-CA, or pin back under the hive
domain) and says why it is a warning rather than an assertion — the
module can see what it issues, not what the operator installed.
hive-agent's own web_ui module never exposed the agent's own paused
status to its own /api/state — the dashboard's cross-container view
knew it, but a per-agent page had no way to know it's paused. Added
StateSnapshot.paused (a direct stat of the same harness-local pause
marker hive-c0re's Coordinator::is_paused checks).
The per-agent page's ⋯ overflow menu now has a pause/resume item that
POSTs to hive-c0re's existing /api/pause/<name> / /api/resume/<name> —
the same endpoints the dashboard's <hive-agent-menu> already uses,
same cross-origin form-submit pattern the existing rebuild-container
item uses. The item's label tracks state.paused on every /api/state
refresh so a pause/resume triggered from the dashboard while this page
is open doesn't leave a stale action showing.
Each workspace crate's own README.md now also shows up as
docs/components/<crate>.md via nix/packages/reference-docs.nix, so it
renders on the docs site alongside the rest of docs/ without a second
hand-copied file to keep in sync. Relative links that were correct
from the crate's own position in the tree are rewritten for their new
virtual position (../docs/x.md -> ../x.md, ../<sibling-crate> ->
./<sibling-crate>.md). Landing page at docs/components/README.md, nav
bullet added to docs/README.md. No changes needed in the website
repo's docs.nix - it already walks every subdirectory generically.
hyperhive#2922: the inline-answer slot (pendingAskBinds/reconcileAskBinds/
buildAnswerForm) depended on a since-removed /api/loose-ends endpoint
and had been silently non-functional the whole time — nothing ever
reassigned lastLooseEnds, so the reconciler always no-op'd.
Per mara's call on the issue (ask/answer is deprecated per #2850, only
the main dashboard UI needs to work, remove the broken per-agent inline
view): ripped out the dead JS (binding + form + CSS) rather than
restoring the endpoint. An ask tool call now renders like any other
tool call, no inline answer affordance; the operator answers via the
dashboard's own question surfacing.
Also fixed adjacent doc staleness this same removal made newly
contradictory (docs/web-ui/agent.md, docs/terminal-rendering.md):
the doc already described the loose-ends/bash-tasks flyouts and the
ask-binding as recently dead, but the endpoint reference table and a
header-pills bullet list still described them as live.
Per review: docs represent current state. Every "used to" / "no longer"
clause this branch introduced is gone — including the History section in
network.md, which was a whole subsection about a sync mechanism that
doesn't exist.
Where the removed clause was carrying a real constraint, the constraint
stays and is stated in the present tense instead of as a delta: nothing
narrows what the gateway's nginx can reach except the directory
permissions in front of a socket, and nothing bounds `ReloadGatewayNginx`
except the hard-coded unit name. Those read as rules now rather than as
the story of how they came to be rules.
The gateway's nginx + dnsmasq no longer run in their own nspawn container.
`nix/host-modules/hive-gateway/default.nix` loses the
`containers.hive-gateway` wrapper and everything that existed only to punch
holes in it: `privateNetwork = false`, `CAP_NET_ADMIN`, five bind mounts,
its own `stateVersion`, `networking.firewall.enable = false`,
`networking.resolvconf.enable = false`, and the `hive-gateway-resolv`
path+service pair. 465 -> 303 lines.
The container never bought isolation here. It shared the host netns by
necessity — nginx binds the host's :80/:443, dnsmasq answers on the bridge —
so each of those settings was undoing a boundary the gateway could not
afford in the first place.
Four things made it more than a deletion, none of them visible in the nix
diff:
- The self-signed cert service also imports the hive CA leaf, so removing it
with the container would have left nginx naming a missing cert file, which
it refuses to load at all.
- The nginx reload is a hive-priv verb. It still needs root, but no longer
for the reason its doc gave, and `--machine=` was both transport and
scope — so the unit name is now hard-coded in the helper as the
containment.
- The lifecycle verb named a container that stops existing.
- `journalctl -M hive-gateway` had no machine to enter.
Per the operator's ruling, the operator verb keeps working and agents lose
it. `InfraContainer` answered three questions that used to share an answer;
it now splits into `name()` (identity), `target()` (Container vs HostUnit),
`service_unit()` (the systemd unit), and `agent_restartable()`, which the
MCP restart path checks before the capability so the refusal cannot read as
"ask for infra_admin". `SIBLING_CONTAINERS` drops the gateway — it gates the
requests that name a container as a string — while `FromStr` still accepts
it, because that answers what a name is, not who may act on it. The
dashboard's gateway journal reads host journald filtered to `nginx.service`.
Prose was corrected where it only named a location, and re-argued where the
container was doing security work: a `0666` per-agent socket was safe
because only the gateway container had the directory bind-mounted. There is
no mount now, so the directory permissions are the whole of the access
control — the constraint holds, its mechanism doesn't.
Gate: nix fmt / clippy --all-targets -D warnings / cargo test all clean (710
tests); hivectl-cli.md regenerated from the clap tree. The nix eval was run
in both TLS shapes at this commit: every delta in the rendered
virtualHosts is one of the three intended path moves, dnsmasq settings are
byte-identical, and the absence probe flips true -> false with bindMounts
emptied.
Frontend half of the two-PR split on the systemd restart-bound work
(clause 2): once a container's unit hits systemd's bounded restart
limit and stops on its own, that read exactly like a deliberate stop
("not running") — no way to tell "gave up" from "off on purpose".
Renders ContainerView.failed as a distinct red "gave up" badge on the
container row, in place of the muted "not running" badge. Both states
have running: false; failed is the new orthogonal fact that tells them
apart, same "independent flags, no state machine" shape as
paused/needs_update/needs_login. An older backend without the field
serves failed: undefined, which reads falsy, so this degrades cleanly
to the existing single "not running" badge.
Frontend-only — no Rust changes. Safe to merge in either order
relative to the backend PR carrying ContainerView.failed itself.
GET /api/jobq/graph gains a states query param (comma-separated
hive_jobq::State names): narrows the served root groups to the named
states, keeping a group whole (filtering by a root's own state, which
is already its subtree's rolled-up answer). Absent, empty, or fully
unrecognised is the identity filter, matching prior behaviour.
hive-jobq-graph.js gains a row of per-state checkboxes above the tree,
re-fetching the endpoint with the selection on toggle. Default
selection hides Done and Skipped.
Server-side filtering (not client-side hiding) so hive-jobq-graph-update's
node list, and everything downstream of it in builds.js (count pill,
live-log panel), only ever sees what's actually shown.
hive_sh4re::assets::branding_svg() resolved a server-side default
icon at runtime from HIVE_ASSETS_DIR — the only consumer was
serve_icon(), which fell back to it whenever the agent had no
`hyperhive.icon` override. Removed both the fallback and the
function: serve_icon() now 404s when /etc/hyperhive/icon.svg is
absent, and the per-agent web UI (app.js) picks up the existing
dashboard swarm.js pattern — swap the <img> src to the
frontend-bundled /favicon.svg on load failure, guarded against
looping if the fallback itself 404s.
Updated the doc/comment claims that said the server always returns
an image (docs/web-ui/agent.md, nix/agent-modules/default.nix, the
hive-c0re/forge/users.rs comment referencing the old shared-asset
set). forge-avatar-sync and the matrix avatar sync are unaffected —
both are gated on hyperhive.icon != null and never depended on the
removed fallback.
bind_child_agent_dirs looped over state, harness and config alike and
mounted all three read-write, while the doc comment above it defended
only state. The rationale covered one dir, the loop covered three — the
uniformity is what erased the fact that the three have three different
answers.
harness holds the child's own runtime material (bash-tasks, the
turn-stats and event sqlite dbs) and nothing argues for a parent
touching it. The only other reader is stats::hive_stats, which reads the
host path directly and needs no mount into anyone.
config stays read-write here on purpose. The ruling is that it becomes
read-only, but request_init_config still has the manager seed a new
child's config in place, so flipping the mount before relocating that
step breaks agent creation hive-wide. That ordering now lives in the doc
comment, where someone about to finish the job in one line will see it.
docs/persistence.md justified all three dirs as RW; it now states the
boundary as three answers and names the right source file.
`security.pki.certificateFiles` is read when the system is BUILT and the
swarm root is deliberately a runtime file (its key must never enter the
world-readable store), so there was nothing build-time to name and the
container trusted no swarm-internal CA. Federation with a peer whose
cert chains to the swarm root did not validate.
The bridge already existed: `lib/hive-ca-trust.nix`, which `hive-ci` and
`hive-forge` both cross. This wires the matrix container to it -- read-only
bind of the trust bundle, container ordered after `hive-tls-ca.service` --
rather than inventing a second mechanism.
Consumption is the per-runtime half, and it is not what it looks like.
tuwunel links no openssl, which makes `SSL_CERT_FILE` appear inapplicable.
It isn't: the outbound client is `reqwest` with the `rustls` feature,
which builds a `rustls_platform_verifier::Verifier`, and because tuwunel
calls `tls_certs_merge` (additive) rather than `tls_certs_only`, the
platform roots stay alongside its compiled-in webpki set. On Linux that
verifier resolves through `rustls-native-certs` to `openssl-probe`, which
reads `SSL_CERT_FILE`. Established by reading tuwunel 1.8.0 and reqwest
0.13.4 source, not inferred from the feature list -- an earlier reading of
mine concluded the opposite from the absence of a `native-roots` feature
name and was wrong.
The variable therefore names a CONCATENATION (system CAs + the bundle),
never the anchor alone: `openssl-probe` uses it *instead of* the default
store, so pointing it at the bundle would drop every public CA and break
federation with the wider matrix network -- trading a small outage for a
much larger one.
`container@hive-matrix` needed `mkMerge`: it already assigned `after` for
the gateway ordering and the helper contributes its own `after`/`requires`,
so two bare assignments would have conflicted rather than combined.
⚠️ Gate honesty: `state/eval-3093.sh` proves the mount, the ordering, the
bundle service and the env var are RENDERED -- 6 probes, including two
separate absence probes (mount and consumption are wired at different
sites, and the gateway outage happened precisely because two of three
sites tolerated a missing cert and the third did not). It CANNOT prove
tuwunel loads them: a build that ignores the variable evaluates
identically to one that honours it, and the chain above is
version-specific. That half wants an executed check on a real deploy.
