The comment said "It parses; it does not prove a sample arrives",
which understates the gap in the direction that matters: it reads as
if a green build proves the collector *starts* and only runtime
delivery is unverified.
Measured while probing ingest-auth options for #3283: `otelcol
validate` ACCEPTS a receiver naming an auth extension that is absent
from the build, and the collector then dies at startup with `Failed
to start component`. So the check does not prove this config starts
at all.
Comment-only; no evaluated config changes.
Refs #3283.
Retention so a reader can ask what a hive last said without anyone
keeping a second copy. The container is ephemeral = false, so the store
survives a restart with no bind mount, and losing it degrades to the
true answer (nothing known) rather than a stale one.
The per-account grant is the part that is easy to miss: the server
option gives the SERVER JetStream, and an account gets it only from its
own. Measured against a running 2.14.1 with this exact two-account
shape - global-only makes kv operations fail 10039 while the server
starts cleanly and logs Starting JetStream, so no rendering check can
see it. Deliberately not granted to the callout account: the responder
mints credentials and has no business holding stream state.
The headersCredential section still said hive-c0re forwards the file
into each agent container - that is exactly the behaviour this change
removes, so the doc contradicted the code rather than merely lagging
it. Same for the collector section, which presented the two paths the
change collapses.
Also drops the dangling collector.enable reference in the network
section.
mara: 'there should only be the one via otel-collector'. Collapses
collector.enable away - enabling otel runs the collector, and agents
always export to it.
The argument for it is the same one the collector exists for: as long
as the direct path stays selectable, the credential-in-settings.json
hole stays selectable. An option that can reintroduce a hole is a hole
with extra steps. HYPERHIVE_OTEL_HEADERS_CREDENTIAL is now never
forwarded to containers at all rather than conditionally.
Two consequences, both deliberate:
- the enable option's own promise changes. It said each harness exports
directly so telemetry survives anything host-side being down; that is
now false for every hive rather than for opted-in ones. Stated in the
option text rather than left for a reader to discover.
- the collector-requires-headersCredential assertion is gone. It was
correct only under the shape it was written for: when the collector
was opt-in FOR the credential, opting in without one was pure
indirection. With one path it isn't - an upstream needing no auth
header is legitimate, and the assertion would have rejected it. The
upstream header block is now omitted entirely in that case, rather
than rendering an ${env:...} reference nothing sets.
The module hardcoded otlphttp, so a hive with protocol = "grpc" would
have had its agents' protocol respected end to end before the collector
and silently rewritten to HTTP after it. Splitting the path in two makes
the upstream half the one that has to keep honouring the option; the
agent half is pinned to the receiver's protocol and derived, not
configured.
The agent-facing endpoint becomes derived rather than `otel.endpoint`
re-defined: with the collector on, agents and hive-c0re's own exporter
send to the bridge address; with it off the expression is the old value
character for character. `otel.endpoint` keeps meaning "where telemetry
ultimately goes" on every hive that already sets it.
Suppressing HYPERHIVE_OTEL_HEADERS_CREDENTIAL is the half that actually
closes the hole this issue is about. The credential path host -> container
is sound (/run/credentials is dr-x------ root), but the hive-otel-header
oneshot then writes the value into the agent's own settings.json, mode
0600 and owned by the agent — so today every agent can read the hive's
upstream telemetry credential. With a collector there is nothing to
forward.
The protocol is derived for the same reason the endpoint is: the first
hop is now the collector's OTLP/HTTP receiver, while otel.protocol
describes the upstream link.
Renders services.opentelemetry-collector on the host: OTLP/HTTP receiver
on the bridge address, otlphttp exporter to otel.endpoint, and the
upstream credential delivered as EnvironmentFile so the collector
interpolates it at runtime and nix never sees the value.
Three things worth knowing, each measured rather than assumed:
- network.exposeHostPorts already exists and is wired (it opens the port
on the bridge interface only), so bridge reachability costs nothing.
- validateConfigFile defaults to isStorePath configFile, which is null on
the settings path - so upstream's default is OFF for exactly the way
this module configures it. Set true: it runs otelcol validate at build
time. It parses, it does not prove delivery.
- headersCredential's file is already NAME=value, i.e. EnvironmentFile
format, verified against a real settings.json rather than the doc.
An assertion refuses collector.enable with no headersCredential: the
collector exists to be the only holder of that token, and without one it
is indirection that reads as security.
Adds services.hyperhive.otel.collector.{enable,port}. No implementation
yet - this commit only declares the surface, so the option docs and the
default-off invariant can be reviewed on their own.
The reason the collector exists is the credential: without it every agent
needs headersCredential to reach the upstream, and the harness delivers
that token into the agent's own settings.json where the agent can read
it. The option text states the cost too - the collector becomes a
dependency in an export path that is currently direct.
The responder answered granted=true and the server still refused every
client:
Error non operator mode account "AUTH": attempted to use issuer_account
nats_jwt::Token::new_user always sets issuer_account, which is an
operator-mode field naming the account when a signing key rather than the
account identity key signed the token. In server-config mode - what this
module renders - its mere presence is fatal, and the account is named by
the claims' aud instead.
nats-jwt can express neither aud nor the omission, so the user JWT is now
hand-built by the same signer as the response wrapper, and nats-jwt moves
to dev-dependencies as the encoder's test oracle.
Every unit test passed throughout: they assert fields that must be
present, and the defect was a field that must be absent.
Measured against a running server built from this module's own rendered
config, not a hand-written one -- which is the fix to the method as much
as to the file. Two shapes died in a row: a bare name authenticates
anyone who sends it, and a name plus an nkey refuses to start at all
(nkey users do not take usernames or passwords).
A malformed key is fail-closed too, so the only way to a live server
here is a real key whose seed nobody but the responder holds.
argus caught it and reproduced it: a users entry carrying only a name
has no credential, so CONNECT {"user":"auth"} is accepted with no
password. The name is a literal in this module, so on the shared netns
that identity was walk-in-able from every agent container -- the same
hole the module exists to close, moved rather than closed.
An nkey rather than a password: only the public half appears in config,
the seed reaches the responder alone, so a hive with no responder has
nobody who can authenticate as it. Asserted at eval, because the
rendered config is valid and the server starts either way.
A swarm has one message queue; this is the container that runs it, in
the same shape as every other swarm service. Off by default, and off
means absent.
The auth_callout block is the interim state and the final one at once.
Measured on the pinned nats-server rather than reasoned about: an empty
authorization block, and one with an empty users list, both accept an
anonymous client and answer PONG. auth_callout sets auth_required and
refuses everyone until a responder approves them, so a config whose
responder does not exist yet denies by default -- and the responder
lands beside it without any of this being swapped out.
That matters more here than it would elsewhere because the container
shares the host netns, so an unauthenticated interim state would be
reachable from every agent container on the hive.
No package option, deliberately: nixpkgs' nats module resolves
pkgs.nats-server itself and exposes none, so one here would name
something it cannot control.
No gateway vhost either: NATS speaks its own TCP protocol, nginx cannot
front it, and cross-hive reach is the wireguard mesh.
Runs on the host: the two containers share a network namespace but not a
filesystem root, so this is the only place both trees are addressable.
A copy rather than a bindMounts entry. nixos-container refuses to start
when a bind source is missing, and the secret does not exist until
authelia's first boot has minted it. The registration token dodges that
with an activation script that pre-creates the file; that is unavailable
here, because tuwunel requires the secret to exist and be non-empty, so a
placeholder would satisfy the mount and then stop the homeserver.
Bounded wait then fail, never a silent skip: authelia's container can be
up while its generator is still minting.
Stage 1's actual login path. The provider entry is rendered only when
sso.enable is set, via optionalAttrs rather than an empty list, so a hive
that has not opted in renders byte-identical tuwunel settings.
The secret reaches tuwunel through LoadCredential rather than as a direct
path: upstream requires the file to survive systemd sandboxing and
suggests /etc/tuwunel/, which this container has no writable etc for. The
registration token two units below already solves it the same way, and
for the same reason -- DynamicUser and PrivateUsers stay intact.
Three assertions fail at eval instead of at boot. tuwunel reads identity
providers from its config file, so a half-configured one does not hide a
login button, it can stop the homeserver from starting.
Stage 1 of the staged rollout mara laid out on the issue: the options
and nothing that changes a running hive. `enable` defaults false, so
this commit is inert until someone opts in.
Three rules taken from hive-forge/default.nix rather than re-derived,
since it is the same pattern already merged and reviewed:
- `enable` ADDS a login path and leaves password login alone. An
identity provider that can take the homeserver offline when it
hiccups is worse than two ways in. Making authelia the only path is
tuwunel's `login_with_password`, a separate and reversible switch,
deliberately not folded in here.
- the client secret is a PATH, never a value: it has two holders in two
containers (authelia keeps a hash, the homeserver needs plaintext),
and a literal would be rendered into the world-readable store.
- required when enabled, no fallback. Worse here than for the forge:
tuwunel reads OIDC from its config file rather than a database row,
so a malformed block can stop the server outright instead of merely
hiding a button.
Also records the constraint that shapes the whole issue — matrix SSO
lives inside the homeserver, never behind a forward-auth proxy,
because the client-server API is spoken by non-browser clients holding
matrix access tokens plus federation.
Per mara's review: "describe what is, not what is not."
The header said "the swarm services' vhosts are NOT here", which
describes the file by its absences and dates immediately — the list of
what is missing grows every time a service is added, while the list of
what the file holds does not.
Reworded to say what it is: the hive's own gateway surface (the `_`
default server and the vhost named for the hive domain), with the scope
line stating where a swarm service's vhost lives rather than where it
doesn't.
The header still announced "the forge, matrix and authelia sub-domain
vhosts" and an "Accept-header SPA map". None of the three is true:
- the four swarm vhosts moved into their own service modules, so the
header named contents that are no longer in the file;
- `_` is described as serving the dashboard and per-agent routing, but
it is now a bare 444 catch-all — that surface moved to the vhost
named for the hive domain;
- the "Accept-header SPA map" is not merely gone, it is the mechanism
the file's own dashboard comment argues *against* ("routing is by
PATH, never content-type"), 140 lines below the header claiming it.
A file's opening comment is its orientation text — the first thing read
and the last thing checked — so a stale one costs more than a stale
line in the body. Rewritten to describe what the file holds now, and to
name the property the split established: a service owns its whole
gateway surface in its own module, and a service name appearing here is
the signal that the coupling is growing back.
The `_` vhost was serving the hive's own surface, so every dashboard and
agent-UI request matched the default server rather than a named vhost --
and so did a request for any name at all, including a raw IP.
Split it: `_` keeps only `return 444`, and the hive surface moves to a
vhost named for the hive domain. `_` is `mkDefault` so an operator can
claim default_server themselves, plus an assertion for the case where
they add one without turning ours off -- nginx refuses to start on a
duplicate default_server and nixpkgs asserts nothing, so that would
otherwise surface as a gateway outage at rebuild time.
Clause 2 of #3202, reading 1 (mara: "the all local stuff and swarm
services auto conf belong in those mods, not spread all over").
`gateway.localHostsEntry` is the gateway's only local-deployment knob —
`openFirewall` is about EXTERNAL exposure, `tls.acme` needs a public DNS
name, `hsts` is a hardening choice. It is now asserted by the mode in
local-defaults.nix, beside the three swarm toggles, instead of being the
one all-local implication an operator still had to know about.
`mkDefault`, so "all local except this" still needs no new option.
⚠️ The non-obvious half: this does NOT change what CONTAINERS resolve.
dnsmasq sets `no-hosts = true` unconditionally, so agents keep getting
the bridge IP from the authoritative `address=` rules rather than the
host's 127.0.0.1 — which would point every agent at its own netns. That
guard already existing is what makes this safe to default on; without it
this one line would break every agent's access to the forge.
Last of the four. The vhost, its `auth_request` block and the swarm
apex's dns record move into swarm-ui.nix; vhosts.nix drops `uiCfg`,
`controllerCfg` and `autheliaCfg` and is now 259 lines of hive surface
with no swarm service in it.
Also collapses a THIRD copy of the per-service list. `networking.hosts`
restated every service's name with its own copy of that service's guard,
after the vhosts and the dnsmasq records had each done the same. It asks
the same question — which names does this host answer for — so it now
reads the same answer: a service added later lands in /etc/hosts with no
edit, and cannot land there under a different condition than it used for
DNS.
The `forceSSL`-not-`addSSL` comment travels intact: it records that
authelia answers an http auth subrequest with 400 and nginx's
auth_request only understands 2xx/401/403, so the scheme is load-bearing
for this vhost and no other.
Moves the matrix sub-domain vhost out of the gateway's vhosts.nix, its
`address=` rule out of dnsmasq.nix, and the Accept-header
`$matrix_spa_target` map out of the gateway's appendHttpConfig — all
three into hive-matrix.nix.
The map is the one that had no business being where it was: it exists
solely for the SPA fallback in the vhost's `/` location, and
`appendHttpConfig` is a `lines` option, so a module can contribute to
it without the gateway assembling it.
The `.well-known/matrix/*` delegation deliberately stays on the hive's
own vhost. The spec requires it at the SERVER NAME, which is the hive
domain: that is the hive answering "where is my homeserver", not the
homeserver answering for itself. Moving it would have been the obvious
symmetric thing and it would have been wrong.
Moves the authelia vhost out of the gateway's vhosts.nix and its
`address=` rule out of dnsmasq.nix, into swarm-authelia.nix.
Both land inside that module's existing `cfg.enable` guard, which is
the load-bearing part: every hive in a swarm knows `authelia.url`, but
only the host that RUNS the container may claim the name. A client hive
declaring the vhost would answer for a service it does not run, and
publishing the DNS record would point every agent on its bridge at that
wrong answer.
The kit grows a fourth member, `errorPages`, because the vhost aims its
502/503/504 at the gateway's styled sso-unavailable page. Republished
rather than imported per module: a service rendering its own would drift
from the rest of the gateway the first time the theme changed.
Moves `forgeVhost` out of the gateway's vhosts.nix and the forge's
`address=` rule out of dnsmasq.nix, into nix/host-modules/hive-forge —
the module that already owns everything else about the forge.
The gateway keeps what is gateway knowledge (the listen set, which
issuer covers a name, the header block) and loses the last reason it
had to read `swarm.forge` at all: `forgeCfg` is gone from both files
and from the module's `let`.
Both halves stay gated on `behindGateway` — with it off the operator
fronts forgejo themselves, so this hive must neither claim the vhost nor
answer DNS for the name.
Adds `services.hyperhive.gateway.localNames` (internal): hostnames the
hive resolver answers with the bridge IP, contributed by the modules
that own them. The service says which name, the gateway says where it
points — the same split `lib.tlsFor` already makes.
No behaviour change yet: the list is empty until the service modules
contribute in the following commits.
The assertion is not defensive padding. Duplicate `address=` rules do
not make dnsmasq complain; it resolves them by precedence, so a name
claimed twice silently stops being served by one of its claimants. That
failure mode only becomes reachable because contribution is now open, so
it gets closed in the same commit that opens it.
Slice 1 of #3202. The listen set, the per-name TLS attrs and the
security headers move out of vhosts.nix into ./vhost-lib.nix and are
published as `services.hyperhive.gateway.lib` (internal, readOnly).
No behaviour change: vhosts.nix consumes the published value, so the
rendered vhost tree is identical.
The point is the next slice. Today a swarm service's vhost lives in
the gateway because only the gateway knows the port pair, the issuer
for a name, and the header block. Publishing those three is what lets
a service module declare its own vhost without the gateway having to
know that service by name.
Fixes hyperhive#3223.
swarm-controller: GET /api/hives (utoipa-annotated same as /health),
serving the swarm's hive directory (name + domain) loaded once at
startup from a new SWARM_CONTROLLER_HIVES env var. The controller's
NixOS module sets it from services.hyperhive.swarm.hives, JSON-encoded
the same way hive-c0re already builds HYPERHIVE_PEERS for its own peer
list (environment.nix) — the full directory here rather than
peers-minus-self, since a swarm-level daemon has no 'self' hive to
exclude. Unset/malformed both fall back to an empty list with a
warning rather than failing startup, so /health stays answerable even
if this one env var is wrong.
swarm-ui: App.tsx's Home route fetches /api/hives and renders it
through the already-merged <Table>/<StatusChip>/<Panel> primitives —
name, domain (linking out to that hive's own gateway-routed
dashboard), and a static "configured" status chip until a real
online/stale/offline rollup exists server-side. Also gave swarm-ui a
base <a> color (theme's --blue) — base.css covers body/typography but
not links, and this is genuinely page-level rather than any one
component's concern.
Verified end to end, not just source-reading: ran the real
swarm-controller binary with SWARM_CONTROLLER_HIVES set, curled
/api/hives + /health over its actual unix socket; separately served
the real swarm-ui dist against a mock /api/hives and screenshotted the
rendered table. Also re-verified the nginx wiring evaluates (same
throwaway nixosSystem eval technique as #3212) — SWARM_CONTROLLER_HIVES
resolves to the expected JSON shape.
cargo test/clippy -p swarm-controller clean (2 tests, including a new
load_hives one covering missing/malformed/valid env var states). npm
run build + typecheck clean.
Rust half mirrors hive-c0re/src/dashboard/mod.rs's utoipa pattern
exactly: an ApiDoc root, #[utoipa::path(...)] on /health (the one
existing route), and a raw JSON route at /api/openapi.json served
via OpenApiRouter::split_for_parts(). Only annotated routes appear
in the spec.
Gateway wiring extends the swarm-UI vhost (the only vhost swarm-
controller is reachable from) with:
- /api/ — proxied to the controller's unix socket untouched (no URI
segment after the socket path), so a route swarm-controller
registers is the path nginx forwards, no prefix-stripping to keep
in sync by hand.
- /api/docs/ (+ the bare /api/docs redirect) — the same
swagger-ui-theme dist the per-hive dashboard already serves at its
own /api/docs/, reused as-is since it's generic.
Both new locations reuse the same auth_request block the vhost's own
'/' already applies, factored into a shared swarmAuthRequest string —
auth_request does not inherit across sibling nginx locations, so
without this the page itself would be gated while its own API and
API docs sat open.
cargo test -p swarm-controller + cargo clippy --all-targets both
clean. Verified the new nginx wiring evaluates correctly with a
throwaway nixosSystem eval (services.hyperhive.swarm.{controller,ui}
enabled): /api/ proxies to the socket, /api/docs redirects, and both
require auth_request the same as the vhost root.
Fixes hyperhive#3212
Per mara: a general fix, not one name in one container.
Every container inherits a COPY of the host's /etc/resolv.conf at start
(nixos-containers.nix: cp --remove-destination, one shot, not a
bind-mount), so the address written there is the address every container
tries - in its own netns. That makes the value load-bearing:
value host host-netns containers bridged containers
127.0.0.1 ok ok THEIR OWN loopback
bridge IP ok ok ok
dnsmasq binds lo and the bridge, so the bridge IP works for the host
too. It is the only value correct on both sides of a netns boundary.
resolveLocalQueries goes ON for its plumbing, not its address: it points
dnsmasq's own upstreams at a separate resolv-file, without which
dnsmasq reads /etc/resolv.conf and every non-hive query loops the moment
the host is pointed at dnsmasq. Its two loopback-publishing effects
(networking.nameservers and resolvconf.useLocalResolver) are overridden.
Cost: the host's DNS now depends on dnsmasq being up. Every container
already did.
The forge container keeps its hosts entry from the previous commit -
not redundancy, a fallback in a different failure domain: it works with
no DNS at all, so SSO does not ride on a host-wide resolver change.
The login source still failed after the argv fix, with
dial tcp: lookup auth.constellation.darkest.space: no such host
The hive's dnsmasq is authoritative for the swarm service names, but
only containers whose resolv.conf points at the bridge ask it — agent
containers do, via an explicit unit written for that reason
(nix/agent-modules/network.nix). hive-forge resolves through the host's
resolvers instead, and the swarm domain has no public records, so
discovery fails for a name that resolves fine one container over.
Publish it in the container's own hosts file, mapped to 127.0.0.1:
sharing the host netns, loopback is the host, where nginx already serves
that vhost. TLS still validates - the CA trust bundle is bind-mounted
and the leaf covers the name. Gated on authelia being local, since a
remote provider's name belongs to another machine.
Gate (state/eval-3149-hosts.sh) asserts the RENDERED /etc/hosts rather
than the option, plus an absence probe with SSO off and a check that the
discovery URL names the same host the entry publishes.
authelia refuses to authorize a non-https target: the auth subrequest for
`X-Original-URL: http://…` answers 400, and nginx's auth_request only
understands 2xx/401/403, so a plain-http visit died as "auth request
unexpected status: 400" — a bare 400 in the browser with no hint that a
login exists.
The shared vhostListen binds :80 as well as :443 and every vhost used
addSSL, so the door was open on a port the lock cannot work on. That is
harmless for forge and matrix, where http is merely insecure rather than
broken, so the asymmetry stays local to the one vhost whose correctness
depends on the scheme.
removeAttrs because nixos asserts on a vhost declaring both addSSL and
forceSSL.
Measured against the pinned binary rather than reasoned about:
X-Original-URL: http://constellation…/ -> 400 Bad Request
X-Original-URL: https://constellation…/ -> 401 + Location: auth…?rd=…
Per mara on the PR: the guide someone follows on a fresh deploy is a
different page from the one they read while debugging, and only the
second existed.
Also fixes a contradiction the request surfaced: setup.md and
swarm/sso.md have told every operator to create '--group admins' since
the bootstrap step existed, while the new rule required 'operators'.
This is the first rule that CONSUMES a group name, so it takes the one
the guide already creates - inventing a second would have meant every
account made by following the guide silently failing the check it was
supposed to pass, and for mara specifically a migration step that now
may not be needed at all.
setup.md's step 3 says why --group is not decoration; the new step 4
says what decides whether the page opens (the group, and the name
resolving) rather than listing a command, because there is nothing to
run. Steps renumbered, including the matrix block's internal 4a-4e.
The option defaulted to pkgs.swarm-ui, which does not exist: this
project has no overlay - flake.nix's nixosModules.default wires package
options with mkDefault from its own package set, and swarm.controller
does exactly that. The default would have failed to evaluate on any real
deployment, not just in a test harness.
Found by the gate forcing .package: the earlier probes passed because
they only read option values that never touched it.
Four wiring sites, same as forge/matrix/authelia: vhost, cert name, DNS,
local-dev hosts. This commit is the last two.
The apex is a SIBLING of forge./chat./auth., not a child of anything
dnsmasq already answers for, so the /<hive domain>/ rule does not cover
it - without an explicit record the name is unresolvable from every
agent netns while working fine from the host, which is the shape that
nearly shipped when authelia's name moved.
Published to agents deliberately (mara: publishing it is fine).
Reachability is not the access control: the vhost's auth_request plus
authelia's group:operators rule are, and an agent that resolves the name
still cannot open the page.
Serves the static bundle on the swarm apex and gates it with
auth_request - the first one in this gateway, everything else being
auth_basic + htpasswd.
Header set measured against the pinned authelia (4.39.20) rather than
copied from an example: X-Original-URL and X-Original-Method are present
as literals and are what the auth-request implementation reads, while
X-Forwarded-Uri does not appear in that binary at all - sending it would
look like configuration and be dead weight. The endpoint is
/api/authz/auth-request; /api/verify is the legacy path older examples
show.
auth_request_set captures the return URL BEFORE the error_page jump: in
the 401 handler $request_uri is the internal one, so building the link
there sends the operator back to the auth subrequest rather than the
page they asked for.
Authorisation is the access_control rule from the previous commit, not
this subrequest: auth_request answers 'is there a session'.
default_policy = one_factor means 'any authenticated user', which is
authentication and not authorisation. The swarm UI is operator-only and
agents are getting authelia accounts of their own, so a session alone
would be enough to open it the day that lands - the vhost's auth_request
would be a check nobody fails.
Adds an access_control rule for the UI's domain requiring
subject = group:operators, present only when the UI is enabled. The
group name is a constant beside the rule: it is also what an operator
types into 'swarmctl user add --group', and a configurable name is one
more way for the rule and the account to disagree silently.
New swarm-ui module: enable (derived from swarm.controller.enable - the
UI reads that daemon's state over its socket, so the host that runs the
controller is the host that can serve the UI), domain (defaults to the
swarm apex; an option so a hive can pin it like forge/matrix can), and
package.
Adds the name to swarm.serviceDomains, which is both the services
sub-CA's nameConstraints set and the leaf's SAN set. The apex is a
SIBLING of forge./chat./auth., not a parent, so nothing issues for it
implicitly - left out, the vhost falls back to the hive leaf and the
swarm's front page opens with a name mismatch.
Asserts the UI domain differs from the hive domain: the gateway's
default server already answers for the latter, and two vhosts claiming
one server_name resolve to whichever nginx picks rather than erroring.
Moving agents.conf/gateway.htpasswd out of StateDirectory (0e1a975f)
left hive-c0re unable to write them at all: ProtectSystem=strict only
keeps StateDirectory and RuntimeDirectory writable, so every write to
/var/lib/hive-gateway/conf failed with EROFS. agents.conf never got
past the tmpfiles pre-seed, so nginx had no location block for any
agent.
The login source was never registered on any boot since it landed:
forgejo-sso-source[422]: Command error: unexpected arguments: profile, email, groups'
The flags were built as a shell STRING and word-split at the call site.
Splitting happens after quote removal, so the quotes inside the value are
just characters: --scopes 'openid profile email groups' reached forgejo
as four words, three of them unexpected and one carrying an apostrophe.
escapeShellArg interpolated into a string that is later word-split is a
no-op that looks exactly like protection.
An array carries the argument boundaries instead of re-deriving them from
whitespace, and the shellcheck disable goes with it.
Also assert the effect: the unit now fails if the source is absent from
'admin auth list' afterwards. The old failure exited non-zero and was
still invisible to every check that read the rendered script rather than
its result.
Per mara on the PR: how a thing used to be deployed stops being
relevant the moment this is merged and the hives are rebuilt, so the
comments state the current constraint and nothing about the container
that used to hold it.
Removes the two ⚠️ blocks this PR added (hive-tls-ca's ordering, the
resign propagation), the matrix ordering's, dnsmasq's resolv.conf
paragraph, and the pre-existing 'REMOVED WITH THE CONTAINER' inventory
in the gateway module. The 🚨 do-not-simplify warning on the cert-copy
unit keeps both its reasons and loses the container framing.
Kept deliberately: the two operator-facing 'this used to default to
true' option notes (migration information for someone upgrading), and
the SupplementaryGroups block, which documents why an absence is
load-bearing rather than how a deployment used to look.
Prose-only sweep of every remaining claim that nginx or dnsmasq lives
in a container: the port comment (root in a container -> root on the
host), upstreamHost's netns rationale, the ACME state dir, the store
path reachability note, the vhost tree header, dnsmasq's resolv.conf
paragraph (there is no copy and no path unit watching it any more),
the two hive-network bridge comments, and swarm-controller's socket
access-control note, which described a bind-mount that no longer
exists.
No behaviour change; all of it was describing a mechanism that was
deleted.
Removing the gateway container retired two things that were still
depended on: an ordering edge and a delivery path.
- hive-tls-ca ordered itself before/requiredBy
container@hive-gateway.service. That unit no longer exists, so the CA
was sequenced against nothing while its real consumer,
hive-gateway-self-signed-cert (which nginx Requires=), could win the
race and fail its copy under set -eu, blocking nginx.
- hive-tls-resign propagated a rotated leaf with
`systemctl -M hive-gateway … || true`. The machine is gone, so both
calls failed and both failures were swallowed: the unit logged
"propagating" and exited 0 while nginx served the stale copy until it
expired. Host units now, so no -M and no || true — a failed
propagation fails the timer.
- container@hive-matrix ordered after the gateway container to get the
resolver up first. dnsmasq is a host service now, so it orders after
dnsmasq.service, which is what the edge always meant.
`agents.conf` and `gateway.htpasswd` move from /var/lib/hyperhive/gateway
to /var/lib/hive-gateway/conf, alongside the `tls/` the gateway already
kept there.
nginx reads both as an unprivileged user. Under c0re's state dir it could
only reach them by traversing a directory systemd re-declares `0750
hive-core` on every c0re start — so nginx was given `SupplementaryGroups
= [ "hive-core" ]`, which also handed it read access to everything else
group-readable in that tree. The tokens are individually 0600, but the
broker sqlite carries no explicit mode: every message between every agent
was readable by the process whose job is parsing untrusted network input.
Moving the files removes the need and the exposure together. The group is
gone, and its absence is now commented as load-bearing so it doesn't come
back as a fix for a symptom it would recreate.
Also drops this module's `/var/lib/hyperhive` tmpfiles rule. It declared
`0755 root root` and could never win against `StateDirectoryMode`, and a
losing declaration still reads as a guarantee — that is what sent the
first diagnosis of the outage looking for who had changed the mode.
Ordering is unchanged and still the thing that makes a fresh boot work:
tmpfiles runs before services and seeds both files empty-but-valid, nginx
names them (an `include` of a missing file is fatal, not empty), and
content arrives when c0re writes and reloads — which it does on every
topology change, so a boot against the empty seed resolves itself.
Folds in the mode fix: `write` now sets 0644 on the tmp file before the
rename, because a rename carries the source's mode and discards the
destination's, and the tmpfiles rule that declares 0644 is
create-if-absent so it never re-applies.
Two claims this branch falsified and left standing, both caught in
review:
`vhosts.nix`'s `errorPages` param comment enumerated the set
(`{ notFound, unreachable, unauthorized }`) and adding a fourth member
made the enumeration wrong at the point a reader consults it.
`gateway.md` said extending custom error pages beyond the per-agent
routes was "a separate follow-up" — while this branch is that follow-up,
so the doc contradicted the code sitting next to it.
Rewrites the scope rule as the criterion rather than a list, since a
list is what went stale: a route earns a page when the default status
would point at the wrong component. That covers the per-agent routes and
the sso vhost, and explains why forge/matrix/fluffychat still don't
qualify — their upstreams being down means what the code says.
A dead authelia upstream almost always means "no users yet" — authelia
treats an empty user store as a fatal startup error, so an enabled but
unbootstrapped swarm crash-loops behind a vhost that is working
perfectly. nginx's default 502 says the opposite: it points at the
proxy, which is the one component that is fine.
Adds `ssoUnavailable` to the shared error-page set and wires it on the
authelia vhost the same way the per-agent blocks wire
`__hive_agent_unreachable`: `proxy_intercept_errors on` plus an internal
location serving the static page.
The page leads with the bootstrap command rather than burying it under
an explanation, and names the container journal as the fallback for the
cases where users are not the problem. Same Catppuccin template as its
siblings, so this costs no new styling.
`enableAllLocalDefaults` already asserts the swarm's shared services and
its CA; the controller was the one swarm-level thing it left off, so the
default deployment ran authelia, matrix and the forge with nothing
controlling them — and, until the previous commit in this area, without
`swarmctl` either.
The controller's own option stays `default = false`. Running it is a
statement about swarm topology rather than about hyperhive being
installed, and no single host can infer that on its own. But "this box
is the whole deployment" IS that statement, which is why the mode may
assert what `services.hyperhive.enable` never could.
Derived from the mode, not from `enableRequiredServices`: a hive in a
larger swarm can legitimately want the shared services without being the
host that controls them.
`mkDefault`, so `enableAllLocalDefaults = true` with an explicit
`controller.enable = false` still yields a controller-less box — the
mode fills in for an operator who hasn't spoken and never argues with
one who has.
`DNS:forge.<domain>` and `DNS:matrix.<domain>` sat beside
`DNS:*.<domain>`, which already covers them — a wildcard covers exactly
one label and those are one label. Naming them read as policy, and that
is why they outlived the thing they described: a configured service name
no longer has to be under this hive's domain, and when it isn't, this is
the one list it cannot join.
Proved by execution rather than eval, because the SAN list is a printf
argument filled in at runtime and eval can only show the template — the
lesson from the gateway outage. openssl-verified against a CA carrying
the real name constraint: old and new leaves agree on every name the hive
serves, the wildcard genuinely stops at one label, and a sibling name
added to this leaf fails.
That last case came back stronger than expected: a nameConstraints
violation invalidates the certificate, not the offending SAN, so one
foreign name here would break the leaf for the dashboard and every other
vhost sharing it. The comment says so now, and gateway.md's cert-shape
paragraph no longer describes SANs that are gone.
authelia has listened on 127.0.0.1:9091 since it was stood up, with
nothing proxying to it — so `auth.<swarm.domain>` resolved and then
refused the connection. This is the vhost that was never written.
Follows forge and matrix exactly: one `optionalAttrs` attrset merged into
`virtualHosts`, TLS chosen by `vhostTlsFor` (the swarm-services leaf
already names it, since `swarm.serviceDomains` includes
`authelia.domain`), and the same four wiring sites those two occupy —
vhost, dnsmasq address, local-dev `/etc/hosts`, and the arg lists that
feed both files.
Gated on this host running the container, not on authelia being
configured: every hive knows the swarm's `authelia.url`, but only the one
serving it may claim the name. A client hive declaring this vhost would
answer for a service it does not run.
Two things that are deliberate rather than incidental:
`X-Forwarded-{Proto,Host,Uri,For}` are set because authelia decides by
the *original* request — the login redirect and the session cookie's
domain both derive from them. Without them every request looks like it
arrived at 127.0.0.1 over plain http.
And no `auth_basic`. Applying the gateway's basic-auth block to the SSO
provider would put the login page behind the login mechanism it exists to
replace.
localHostsEntry's 127.0.0.1 rows leak into the address= answers dnsmasq
gives agent containers now that it runs on the host, sending every
agent back to itself instead of the bridge IP.
