Drop the H0M3 tile hub and the core page. The dashboard is now the index;
kept state is its fifth tab. A pages menu in the chrome links flow, stats,
API docs and the swarm UI. The swarm apex URL comes from
services.hyperhive.swarm.domain via HIVE_SWARM_PUBLIC_URL and the new
swarm_public_url field of the state snapshot.
The catch-all, the dashboard and the hive-domain vhost were built in
vhosts.nix with the listen set, TLS attrs and headers applied by hand.
They are declared under services.hyperhive.gateway.vhosts now, so every
vhost behind the gateway goes through declared-vhosts.nix.
Two gateway-side rules cover what the catch-all needed: a vhost can set
securityHeaders = false (the catch-all only returns 444), and `default`
is rendered at mkDefault priority so an operator's own default vhost
still wins. The matrix discovery locations are unchanged.
The rendered nginx vhosts are unchanged.
Ten service modules (forge, matrix chat, authelia, grafana, otel, nats,
swarm-ui, victorialogs, victoriametrics, bao UI) wrote
services.nginx.virtualHosts themselves, each applying the gateway's
listen set, per-name TLS attrs and security headers by hand. They now
declare services.hyperhive.gateway.vhosts.<name> with only their own
locations and options; ./declared-vhosts.nix renders it into the nginx
vhost with listen, TLS and headers added.
An eval assertion refuses a declared vhost that sets listen, addSSL,
onlySSL, enableACME, useACMEHost, sslCertificate or sslCertificateKey,
and one declared while the gateway is off. nats' own leaf is the new
`certificate` field. The rendered nginx vhosts are unchanged.
The gateway nginx serves stub_status on a loopback server of its own,
127.0.0.1:8085 (services.hyperhive.gateway.statusPort). The hive-tier
collector reads it with the contrib nginx receiver, which the pinned
otelcol-contrib 0.151.0 ships, so no exporter daemon is added. The
receiver dials http:// only (a unix: or http+unix: endpoint fails with
"unsupported protocol scheme"), so the listener is loopback TCP with
allow 127.0.0.1 / deny all rather than a unix socket.
The new gateway dashboard plots nginx.requests, nginx.connections_current
by state, and nginx.connections_accepted against handled, per hive.
Part of #5342.
The dashboard, its API, the swagger UI and the agent routes move to their
own vhost, `swarm.dashboardDomain` = `<hive>.hives.<swarm.domain>`, under
the swarm domain so authelia's session cookie reaches it. `/`, `/api/` and
`/api/docs/` ask authelia through `auth_request`, and authelia's
`operatorsOnly` rule for each hive's dashboard name lets only the admins
group in. The hive domain keeps matrix discovery and redirects every other
path to the new name.
The name rides as a SAN on the hive's own gateway leaf: the per-hive bao
role allows it, and hive-tls asks for it and checks coverage.
HTTP Basic auth is gone: the htpasswd writer and its test, the
`hivectl gateway` verbs and host-socket requests, the `gateway.auth`
options (now a removed-option error), the unauthorized page and the bcrypt
dependency. tmpfiles deletes a leftover gateway.htpasswd, which held
bcrypt hashes at 0644.
c0re hands out dashboard and agent URLs on the new name
(HYPERHIVE_DASHBOARD_DOMAIN), and the swarm controller's hive roster
carries each hive's dashboard name for the swarm UI's hive list.
Closes#3469
tuwunel bound `0.0.0.0:<httpPort>` in the host netns, so the homeserver
port was open on every host address, and the gateway proxied to it on
host loopback.
The container now sets `privateNetwork = true` and hangs off the hive
bridge with a DHCP lease, the same attach the CI runner, builder and
review classifier use (resolvconf on, `networking.nameservers` the bridge
IP). tuwunel listens only on `/run/hive-matrix/tuwunel.sock`, a host
directory (root:nginx 0770) bind-mounted into the container; its primary
group is nginx's gid and the socket is 0660. The gateway's `/_matrix/`
location proxies to the socket.
The static resolv.conf goes: it existed because resolvconf's host
tracking came up empty in the shared netns.
`swarm.matrix.httpPort` and `{swarm,deploy}.matrix.openFirewall` are
removed with a message: there is no host port to choose or open. The
`.well-known/matrix/*` fallback to `<hive>:<httpPort>` is dropped with
them; it only rendered with `gatewayHost = null`, which the homeserver's
SSO assertion already rejects.
Part of #4815.
The docs, option descriptions and comments that named the swarm root, the
swarm CA or the hive CA as an existing trust anchor now name the secret
store's pki root or the trust bundle that holds it:
- docs/swarm/README.md, secrets.md, bao.md
- deploy.bao.clientCaFile's and gateway.verifiedProxyTo's descriptions
- comments in hive-c0re/src/meta.rs, hive-agent/src/swarm_queue.rs,
swarm-logs/src/auth.rs, swarm-controller/src/vcs_metrics.rs,
hive-forge, swarm-authelia, swarm-ci-runner, glue-bao-tls, swarm-bao,
swarm-grafana, swarm-otel, otel, hive-c0re, lib/hive-ca-trust.nix,
hive-forge/Cargo.toml and the flake-update workflow
Nothing behaves differently: every edit is a nix, Rust or TOML comment, an
option description, or a test assertion message. The Rust edits change the
workspace source, which this PR already rebuilds for hivectl.
Refs #4588
The secret store's pki root issues every hive's gateway leaf and the
swarm's service names, so the root swarm-ca.service generated on the
host anchors nothing and its key is a secret outside the store.
- swarm-ca.nix is gone: swarm-ca.service, root.pem and root-key.pem,
and the swarm.ca.{autoConfigure,stateDir,validityDays} options. A
definition of any of them fails evaluation with a removal message
(swarm-ca-removed.nix), and a tmpfiles rule deletes /var/lib/swarm-ca
on every host. singleHostSwarm no longer sets a swarm.* option.
- hive-tls-ca drops the $root slot: trust-bundle.pem holds the store's
pki root and a hive CA's public ca.pem/ca-previous.pem only.
- hivectl peer-config no longer prints the scp line for the swarm root;
the store's pki root is the one step.
- The gateway's cert import no longer stands a hive leaf or a throwaway
in for a missing swarm-services leaf where it Requires= the issuing
unit: it fails, naming swarm-services-cert.service. The stand-in stays
where the gateway does not wait (no swarm name, or the store's UI
only, whose passthrough must start with the store sealed).
Comments and docs that named swarm-ca.nix, swarm.ca.* or the host root
now name the store's pki root.
Refs #4588
The CI runner was a hive deploy module: hive-ci.nix ran it beside the
forge, and hive-c0re registered it through the forge's admin API with
the `core` token, writing the registration token through hive-priv. It
could only run on the forge's host, and it kept an admin-scoped forge
token on a hive.
Now the runner is a swarm service:
- swarm-ci-runner.nix runs it in a `swarm-ci-runner` container on the
host that sets deploy.ci-runner.enable, reaching the forge by name.
It needs neither the forge nor a hive there.
- swarm-controller reads the forge's runner registration token with its
own admin token and writes it to swarm/services/ci-runner/
registration-token whenever it differs from what it last wrote. Its
grant on that path is create/update only.
- The runner's own leaf (CN swarm-ci-runner, role and policy of the same
name) reads that one path. swarm-bao-ci-runner.service fetches it into
the runner's EnvironmentFile and restarts the runner on a change.
- A pre-start step removes .runner when the daemon's journal since its
last start holds Forgejo's "unregistered runner" refusal (the daemon
exits 1 when its startup Declare gets it), or when .runner names
another forge address; upstream's pre-start step then registers again.
Removed from the hive side: hive-ci.nix and deploy.forgejo.ci.*,
forge/ci_runner.rs, PrivRequest::RegisterCiRunner and its handler and
ReadWritePaths entry, InfraContainer::Ci with hivectl's --ci and the
dashboard's hive-ci journal entry, HYPERHIVE_FORGE_CI_ENABLED, and
read:admin,write:admin from CORE_TOKEN_SCOPES: nothing else in hive-c0re
calls /api/v1/admin/*. The old ci options, under both swarm.forge.ci and
deploy.forgejo.ci, now fail evaluation naming deploy.ci-runner.
The forge's half of the old ci.enable, the actions/checkout mirror and
DEFAULT_ACTIONS_URL = self, is deploy.forgejo.actionsFromSelf.
Closes#4954
Rewrite the remaining narration in files the previous commit touched:
"moved"/"sat"/"was REMOVED"/"old default" comments in swarm-bao,
swarm-otel, glue-bao-tls, nats-authelia, swarm-controller, swarm-ui,
swarm-otel-service, swarm-authelia-service, the deploy.nix rename table,
hive-ci, hive-forge/service, hive-tls, hyperhive, swarm-nats, swarm.nix
and a few module-eval fixtures, so each says where a thing is or why it
is so now.
Revert the comment edits inside shell script strings in hive-tls,
swarm-nats and the forge token mint to their previous text, so those
generated units are unchanged. The only unit still changing is
swarm-bao-pki (two comment lines in its script). Refs #3902
Rewrite comments and option descriptions under nix/ that narrate how the
code got here ("used to", "new name", "sat at the TOP", "no longer",
renamed/moved narration, attributions) to say what the code does now and
why. Comments and description strings only; no logic changes.
Files in open PR #4970's diff are left alone. Refs #3902
The hive-domain vhost renders only where a hive runs. On a host without
one the only certificate is the swarm-services leaf, which does not name
the hive domain, so a homeserver host without a hive serves matrix on the
homeserver's own name only; apex matrix discovery is #4878.
Comments and test names describe the code's current behaviour.
Refs #4588
The hive-domain vhost renders where a hive or the homeserver runs. Its
matrix .well-known JSON and /matrix/ redirect are served wherever the
homeserver runs; the dashboard, swagger UI, agent routes and the
agents.conf include only where a hive runs, since they proxy to hive-c0re.
The gateway doc's vhost map says which rows exist without a hive, and the
trust-bundle text describes ca.pem by what it is rather than as retired.
Refs #4588
A gateway host without a hive whose only swarm name is the store's UI no
longer waits for swarm-services-cert: its nginx carries the stream
passthrough to the store and must come up while the store is sealed. Until
the services leaf lands, hive-gateway-self-signed-cert installs a throwaway
self-signed pair on the default and swarm-services paths there; on a host
without a hive it never references the hive leaf.
A host without a hive also no longer mints the hive-leaf identity or
defaults the options pointing at it, and does not render the hive-domain
vhost, whose upstream is hive-c0re.
Refs #4588
A self-signed gateway on a host with deploy.hive-controller.enable off
(a dedicated swarm-services host) needed the hive leaf for its catch-all,
and that leaf now comes only from hive-tls-resign with a per-hive store
identity. hive-tls-resign and its timer are now defined only where a hive
runs; elsewhere hive-gateway-self-signed-cert installs the swarm-services
leaf as the default cert, and the gateway waits for swarm-services-cert.
Also corrects the hive-gateway comments and option text that still said
the leaf is hive-CA-signed.
Refs #4588
Every hive gets its own store identity (CN
<deploy.bao.hiveLeafCommonNamePrefix>-<hive>) and its own role on the
existing pki mount (<deploy.bao.hiveLeafPkiRolePrefix>-<hive>), whose
allowed_domains is the hive's domain and *.<domain>. The policy grants
update on that one issue path. swarm-bao-hive-leaf-policy writes role,
policy and cert-auth role for every hive in swarm.hives; the store's host
mints its own hive's identity in glue-bao-tls.nix and a new glue file
points deploy.hive-controller.tls.baoHiveIdentity{Cert,Key}File at it.
Hives on other hosts place it by hand. swarm.nix reserves the composed CN.
hive-tls-ca no longer generates a hive CA or signs a leaf: caGenScript,
signLeafScript/signHiveLeaf and the adoption block are gone, ca-key.pem
is deleted where it exists, and the unit only assembles trust-bundle.pem.
The retired ca.pem/ca-previous.pem stay in the bundle so leaves they
signed keep verifying.
hive-tls-resign becomes the bao issuance: at boot, on deploy and from a
daily timer it re-issues when the leaf is missing, past half the role's
lifetime, missing a name, or not chained to the store's pki root. Nothing
orders the gateway after it, so a valid leaf on disk never waits on the
store; a failed renewal leaves the leaf in place. A missing identity
fails the unit naming both options, as does the root fetch on a hive
fronting no swarm name, which now presents this identity instead of the
reader cert.
caValidityDays and leafValidityDays are removed (both spellings), since
nothing reads them.
Refs #4588
An operator's merge on the forge deploys a config PR through
swarm-controller's DeployRequest{rev}. The hive-side path that queued a
MergeConfigPr approval and merged the PR as `core` goes:
- the `/webhook/config-pr` receiver, its HMAC secret, the WebhookRegister
boot node and the org-hook registration; the hive vhost's `/webhook/`
location
- the 5-minute config-PR poll
- ApprovalKind::MergeConfigPr, its dashboard card, and the deploy DAG it
drove (DeployWindow, MergeVerify, DeployApply, FinalizeDeploy,
DeployTail), with verify_commit, the two-phase meta deploy, rollback
refs, the PR-failure comment and forge/pr_merge.rs
- `fetched_sha`, `sha_short`/`pr_number` on approval events, and
`sha`/`tag` on HelperEvent::ApprovalResolved: only the merge path set
them
`config_repo`, `merged_pr_for_commit` and `post_pr_comment` move to
forge/pr_comment.rs for the merged-rev deploy's refusal comment.
Approvals v5 drops stored `merge_config_pr` rows; a test reopens a v4
database holding them.
Closes#4850
Removes services.hyperhive.deploy.matrix.gui.enable and its
swarm.matrix.gui.enable alias; both are mkRemovedOptionModule stubs. A
host running the homeserver serves fluffychat at gatewayHost's vhost,
and the hive's /matrix/ redirect follows the same condition.
The swarm-controller builds the Matrix quick link from
swarm.matrix.gatewayHost, replacing hive-matrix.nix's per-host entry.
HIVE_MATRIX_PUBLIC_URL is set on every hive with a gatewayHost, so
`hivectl open matrix` resolves off the homeserver's host too.
Drops HIVE_MATRIX_GUI_ENABLED and the dashboard's matrix_gui_enabled
field; nothing in the frontend reads it.
Refs #4885
authelia 4.39.20 exits at startup on `users: {}` ("users: non zero value
required"), and the first-boot unit seeded exactly that, so a swarm with
no users crash-looped authelia and answered 502 until `swarmctl user add`
ran.
The first-boot unit now writes one subject, `swarm.placeholder`, when
the users database is absent, empty, or exactly `users: {}`:
- `disabled: true` — authelia returns "user not found" for a disabled
user before any password check (file_user_provider.go,
CheckUserPassword).
- password: an argon2id digest with an all-zero key. It decodes (authelia
rejects a non-digest at startup) and no known password hashes to it.
- the `.` keeps it out of agent names (`[a-z0-9-]`), and `swarmctl user
add` refuses it as already existing. Neither writer removes users, and
both round-trip `disabled`.
A file with any user in it is never touched.
The docs that described the crash-loop (sso.md, gateway.md, setup.md,
the sso-unavailable error page) now describe the placeholder; the
writers' load_store docs and the seed fixtures follow. module-eval
nats-authelia asserts the seed branch.
The swarm collector's journald receiver read only the units listed in
`services.hyperhive.swarm.otel.journaldUnits`. A unit nobody listed
never reached the store, and a misspelt entry shipped nothing without
an error. The list existed to keep an operator's desktop session out of
a store every swarm operator can read, but the receiver can only match
positively, so the only way to express "not user sessions" was to name
every service instead.
The receiver now reads the whole host journal, and a new
`filter/exclude-user-sessions` processor in the `logs/<swarm>` pipeline
drops records whose `_SYSTEMD_SLICE` is `user-<uid>.slice` (session
scopes and `user@<uid>.service`). The per-hive `logs/<hive>` pipelines
carry agent-container journals only and get no filter.
`journaldUnits` is removed with `mkRemovedOptionModule`, together with
its non-empty assertion and the entry each host module added. The four
module-eval membership checks go with it, replaced by one structural
case in swarm-otel-core.
Closes#3646
/etc/tmpfiles.d/hyperhive-agents.conf was a boot-time backstop (#2290)
that pre-created every agent's bind sources. The start preamble already
creates them for every c0re-driven start, and on this host only hive-c0re
starts agent containers. The file was also the reason the socket dir's
owner had to be declared there, which is how it spent its life at
`0777 root root` whenever the uid could not be resolved (#4742).
- hive-priv gains `EnsureAgentSocketDir { name }`, called from
`set_nspawn_flags` in every start path. It creates
`/run/hive-agent/<name>` `0751 root:root` with mkdirat relative to an
O_DIRECTORY|O_NOFOLLOW fd for the parent. An existing entry has to be a
directory (fstatat AT_SYMLINK_NOFOLLOW); anything else is refused, and a
directory is left alone. hive-c0re's own create_dir_all went: its /run
is read-only under ProtectSystem=strict.
- The container's `hive-agent-user-migrate` activation chowns that dir to
the agent user and sets 0751, the same way it already handles state/ and
harness/. It refuses a symlink or non-directory there, since `test -d`
and chmod follow links. No host-side passwd parse, and no window where
the dir is world-writable.
- `/run/hyperhive/agents/<name>` stays created by hive-c0re itself
(`ensure_agent_runtime_dir`). It holds the `mcp.sock` that hive-c0re
binds as hive-core, so it must not become root- or agent-owned.
- The `/run/hive-agent` parent is declared in hive-priv.nix, `0755
root:root`, instead of hive-gateway's hive-core rule. hive-priv is its
only writer now, and hive-priv's ReadWritePaths needs it to exist.
- The manager start in `ensure_root_agent` now goes through
`converge_start_preamble` + `start_with_fallback`. It was a bare start,
so after a reboot the manager's bind sources existed only because of the
tmpfiles file, and its limits drop-in did not exist at all.
- Removed: `sync_tmpfiles`, `agent_uid_gid` / `parse_passwd_uid_gid`,
`priv_client::sync_agent_tmpfiles`, `AgentTmpfilesEntry`, the tmpfiles
body builder and their tests, plus the three call sites.
- Legacy: hive-priv unlinks the file at every start, ignoring ENOENT.
`SyncAgentTmpfiles` stays one release as a payload-ignoring variant that
does the same unlink and returns Ok, for an older hive-c0re.
Salvaged from #4752: the boundary.md correction that nginx only dials,
because ProtectSystem=strict makes its /run read-only.
Behaviour change: a manual `nixos-container start h-<name>` right after a
reboot, before hive-c0re has started that agent, now fails on a missing
bind source instead of starting.
Closes#4742
The units on the path a deploy takes to TLS, the store's grants and the
swarm collector itself were not on the host collector's journald
allowlist, so an ingest outage one of them caused showed in the store
only as every source going quiet at once.
Each module names its own units, per the option's rule:
- hive-tls.nix: hive-tls-ca, swarm-services-cert
- hive-gateway: hive-gateway-self-signed-cert (self-signed mode only)
- swarm-bao.nix: the seven grant units beside
swarm-bao-services-issuer-policy
- swarm-otel.nix: container@<machine>, and
nixos-rebuild-switch-to-configuration, the transient unit nixos-rebuild
runs the activation in and whose syslog lines carry its status
The module-eval arm pins each unit as both listed and defined, since a
listed name that matches nothing is silent.
Fix wording flagged on #4521 review: "Single nginx in front of every
hyperhive web surface" and "Runs on the host" claimed a deployment-wide
topology. There's one nginx per host that has something on it, and
potentially more inside service containers. Reword to describe what
this module builds on this host, with no count claim swarm-wide.
`services.hyperhive.gateway.enable`, `gateway.dns.enable` and
`network.enable` replace the `hyperhive.enable` gate on all three. Each
defaults to false; the modules that need one assert it with `mkDefault
true` from inside the guard their own deployment already carries, and
`swarm-required-services.nix` — the module that owns what the
swarm-services toggle implies — asserts all three explicitly.
hive-c0re asserts all three unconditionally, so an ordinary hive keeps
getting them with no opt-in: it is the host's only knowledge that agent
containers exist.
The resolver moves to its own `hive-gateway/dns.nix` so it can be gated
without reindenting the nginx half of the module.
Reinstates `network.enable`, dropping its `mkRemovedOptionModule` shim.
A config still carrying `network.enable = false` from before the removal
now switches the bridge off instead of failing eval.
Also deletes a duplicate `centralToggleOff` fixture in nix/module-eval.nix.
Two sibling slices added it independently (c5f60fd5, ce3b3d94); the merge
was textually clean and left `main` failing to evaluate at all, so this
file could not be gated without removing one.
`swarm.*` is what a hive needs to be a *client* of the swarm. For the
homeserver that is what it IS from anywhere: its package, the name it
answers to, the ports and URLs it is reached on, and the client id it is
registered under. Whether it is exposed, which peers it trusts, how large
a request it accepts and where its host-local secrets sit are decisions
of the machine running it, so openFirewall, trustedServers,
maxRequestSize, registrationTokenFile, gui.enable and
sso.clientSecretFile move to `deploy.matrix.*`.
Two sub-blocks split rather than moving whole, on their own evidence.
`gui.enable` is whether THIS host serves the web client; `gui.package` is
which client, an artifact identity, and stays. `sso.clientSecretFile` is a
path on one host; `clientId` must match the id in authelia's register, so
it is swarm-wide. Each half now points at the other, because the rendered
docs put them on separate pages.
hive-gateway passed the whole `swarm.matrix` attrset into vhosts.nix, so
that file read a moving option through an argument with no option path
anywhere in it. It now takes `matrixDeployCfg` beside `matrixCfg` — the
only shape that carries a split namespace across that boundary.
While there: vhosts.nix read `matrixCfg.enable`, which has been a rename
alias for `deploy.matrix.enable` since the enable moved. Reading it made
the module system print `Obsolete option services.hyperhive.swarm.matrix.
enable is used` on EVERY evaluation of every host — a deprecation warning
no operator could silence, because the config tripping it was ours. That
shim lives in hive-matrix.nix rather than in this file's table, which is
why deploy.nix's header claim to be their single home is now qualified
in the new block's comment.
glue-matrix-bao-token.nix read the registration token through its own
`matrixCfg` alias; with that read repointed, the binding had no reader
left, so it goes, and the comment naming it is reworded.
module-eval gains a case configuring a hive through all six OLD paths and
asserting two rendered effects — the host firewall's port list and the
container's bind-mount table — because the new paths evaluate fine
without the shims. `gui.enable` is set to the opposite of its default so
the definition has to land rather than agreeing with it by accident.
Filtering the gateway's journal by severity returned nothing while nginx
was logging [error] continuously. Measured against nginx.service:
`-p warning` and `-p err` over 24h both said "No entries", while the
unfiltered query returned a continuous stream of [error] lines.
The failure is inverted rather than merely missing. A severity filter is
how you ask "is anything wrong", and here it answered "no" *because* the
errors were misfiled -- one step from recording "gateway clean, no
warnings in 24h".
Cause: the module default is `error_log stderr`, and systemd files that
whole stream at one priority regardless of what each line says. Routing
to syslog preserves per-line severity, so [error] lands as err and every
existing severity filter starts working without changing a consumer.
Uses the module's own `logError` option rather than appendConfig:
nixpkgs emits `error_log ${cfg.logError};` at exactly one site, in the
main context, so the option guarantees the placement. Transport only --
no level suffix, so nginx keeps its default `error` threshold and this
changes where lines are filed, not which lines exist.
Gated with `nginx -t` on the flake-locked nginx against the exact string
shipped here, with a deliberately invalid syslog target as the control
that the test can fail on this directive. gixy (what the module's own
validateConfigFile actually runs) reports no issues.
What that does NOT prove: that journald files the lines at the right
severity. `nginx -t` is a parse check; only a deploy shows the effect.
Two comment blocks explained things `docs/networking/gateway.md` already
explains, one of them directly above a line pointing at that very doc. A
pointer under a duplicate is not a pointer; it is a duplicate with a
citation.
Measured, not eyeballed: 6-gram intersection between this file's comments
and that doc, 73 -> 31. The residual is mostly the one-line summaries kept
at the call sites, which necessarily share phrasing with the doc they
summarise — that is the floor of the metric, not remaining duplication.
* the self-signed-cert import unit: kept the 🚨 that stops someone
collapsing it into "point nginx at the CA dir", both jobs at one line
each, and the blast radius (whole gateway, not one vhost). Dropped the
`BIO_new_file()` mechanism and the production-outage narrative — the
doc carries both, and the second is history.
* the nginx-reload note: rewritten to say the one thing this file needs
to say, that nothing here reloads nginx and hive-c0re drives it from
the host through hive-priv. The per-unit-state dispatch table is the
doc's, and only the doc has it in full.
Both pointers use the `docs/x.md::Section` form and name headings verified
to exist; the first replaces a parenthetical `("Self-signed TLS")` spelling
that no check can resolve.
Comments only; no behaviour change. Refs #3901.
Per mara's go-ahead on hyperhive#3902 ("getting started is good, but
terminal rendering does not go in there i think"):
Moved 21 top-level docs/*.md files into 7 new topic subdirectories
(existing web-ui/, turn-loop/, swarm/, tools/, crates/ untouched):
getting-started/ setup.md
agent-lifecycle/ agent-hierarchy.md, approvals.md, persistence.md
trust-boundary/ boundary.md, security.md
integrations/ forge.md, matrix.md, github.md, knowledge.md
networking/ gateway.md, network.md, snapshot-store.md
scheduler/ jobq.md, coordinator.md, ci.md, observability.md
process/ conventions.md, gotchas.md, pr-review-gate.md
web-ui/ terminal-rendering.md (moved into the EXISTING dir,
per mara's correction to the original getting-started
guess -- it's UI implementation detail, not onboarding)
The physical layout now matches docs/README.md's own topical headers,
which already amounted to this taxonomy -- see the scoping comment on
the issue for the two findings that motivated this (a genuine
duplication between CLAUDE.md's old "Reading paths" list and
docs/README.md's grouped one, since drifted out of sync with each
other; and the flat layout not matching the grouping we already had).
Fixed every cross-reference this moved across the whole repo (~120
files: docs/ internal links at every depth, Rust doc comments, nix
module option docs, crate READMEs) -- verified two ways: a grep sweep
confirming zero remaining references to any old path, and a script
that resolves every markdown link in docs/**/*.md + CLAUDE.md +
README.md against the filesystem and reports anything that doesn't
exist (zero broken links).
Collapsed CLAUDE.md's "Reading paths" section (the duplicate) down to
a pointer at docs/README.md, now the single index. Rewrote
docs/README.md itself to use the new subdirectory paths and added the
one doc it was missing that CLAUDE.md's old copy had (pr-review-gate.md).
Classified all 22 docs/*.md files first via a haiku subagent (mara's
suggestion) on two axes -- proposed grouping and operator-vs-
implementation focus -- before finalizing the taxonomy; spot-checked
the report and found internal inconsistencies (its classification
table disagreed with its own summary section for a few files), so this
taxonomy is my original proposal + the one correction mara gave
directly, not a blind application of the subagent's table. The
operator-focus data it gathered is still useful for a follow-up
content pass (docs skewing 'mixed' rather than pure operator-facing),
not addressed in this PR -- structure only.
nix fmt clean, both pre-push lints clean.
`services.hyperhive.tls.{stateDir,caValidityDays,leafValidityDays}` sat at
the top of `services.hyperhive`, which is meant to be everything about
hyperhive rather than the settings of one hive. Where the hive CA lives,
how long it lasts and how long the leaves it signs last are decisions of
the host holding the key — `deploy.*`, by the same rule as the switches
that moved before them.
`hive-controller` is hive-c0re's new name (mara on the issue), so the
knobs hang off the daemon that owns the CA rather than off a bare `tls`
at the root. mkRenamedOptionModule entries carry existing configs.
⚠️ Unlike the two switch renames, these names are NOT unique, so this was
swept by ALIAS BINDING rather than by identifier: hive-tls.nix alone holds
two options spelled `stateDir` — its own `cfg.stateDir` and the swarm CA's
`swarmCaCfg.stateDir`, four sites that must not move. Nine files bind an
alias to this config; the rename followed those bindings.
Two sites were invisible to the obvious check, and an unanchored sweep for
`hyperhive\.tls\b` is what found them: the option declaration (`= {` after
the path, so no trailing `.` or `;`) and the alias convention documented in
a comment in lib/hive-ca-trust.nix.
Also renamed the `<tls.stateDir>` shorthand in four docs and two Rust doc
comments, anchored on its delimiters — the new path contains the old one
as a substring, so an unanchored replace would have doubled the prefix.
Same defect as the switch below it, one tier up: it sat at the TOP of
`services.hyperhive`, a namespace that is meant to be everything about
hyperhive rather than the settings of a single hive. Whether this box is
the whole deployment is as per-host as a decision gets.
The name follows mara's sentence for what it means — "everything in the
swarm is running on this host" — rather than naming its mechanism.
"Defaults" was doing no work: it is not a defaults toggle, it is a claim
about where the swarm lives, and the pair now reads as the containment it
already was, singleHostSwarm implying allSwarmServices plus this hive.
One site was a setter rather than a reference: module-eval's `allLocal`
fixture passes an attrset merged into `services.hyperhive`, so its key
carries the path and had to become `deploy.singleHostSwarm`. A rename by
bare identifier is right for the twelve prose mentions and wrong for
exactly this one, which is worth knowing before the next rename.
The comment-block lint (added in 79dc8ca6) now trips on this block —
genuinely pre-existing, unrelated to that change, just newly caught.
Per the lint's own suggested remedy: relocated the full per-directive
reasoning plus both footguns (session-cache keying, the Host-header
clobber that can recurse a subrequest into itself) to a new
"Dialing another vhost by name" section in docs/gateway.md, and left
a short why + pointer comment in the source. No behavior change.
verifiedProxyTo (43ae164d) verified TLS but left Host to nixpkgs'
recommendedProxySettings, which sets Host to the CALLING vhost, not
the target. Since every consumer resolves back to this same gateway,
nginx picks the vhost to answer by Host header (not by the SNI
proxy_ssl_name already sends) — so every auth subrequest looped back
into its own vhost's auth_request, recursing until nginx's subrequest
depth limit turned it into a 500, on every domain gated by SSO.
Pin Host (and the rest of the header set nixpkgs' recommended include
would otherwise still be the one to set) inside verifiedProxyTo, and
set recommendedProxySettings = false on each of the four call sites so
nixpkgs' own copy — appended after a location's extraConfig — can't
clobber it back.
Consumers reached authelia at `127.0.0.1:<port>`, which encoded a
co-location nobody agreed to: the gateway and authelia are not required
to share a host, so the literal is a requirement stated only by being
unwriteable any other way. Moving them to the name is the point of the
issue.
But a name over https is only half of "https and auth". nginx's
`proxy_ssl_verify` is OFF by default and there was no `proxy_ssl_*`
anywhere in the tree, so the obvious repoint would have produced an
encrypted, unauthenticated hop -- which works, and keeps working,
against any certificate at all.
Adds `gateway.lib.verifiedProxyTo <name>` next to the rest of the vhost
kit, so the convention has one definition rather than a copy in each
consuming module, and repoints the four call sites through it.
Each directive was checked against a real nginx with the opposite arm
run as a control:
- the CA *bundle* (root + intermediate) is accepted -- worth checking,
since `hive-ca-trust.nix` warns off consumers that read only one
certificate, and nginx is not one of those
- verification checks the chain: an unrelated CA fails
- and the HOSTNAME: a wrong `proxy_ssl_name` fails even with a good
chain. Chain-only would accept any cert this CA ever signed, which
for an internal CA is every service on the hive
- with verify off, the wrong CA passes -- so the failures above come
from verification, not from the connection
Bind addresses are untouched. This changes what consumers dial, not what
anything listens on.
The journald receiver was configured with a directory and no filter, so
the swarm's log store received every unit on the host that runs the
collector. On a hive whose services live on a workstation that includes
the operator's desktop session, in a store every swarm operator can read.
The receiver has no system-only switch and its `matches` field is an
allowlist too, so what the swarm collects has to be stated rather than
excluded. Each service module names its own units: a service that is not
running contributes nothing, and one added later arrives declared.
An empty list is fail-open — the receiver renders no filter at all and
reads everything — so it is asserted against.
Follow-up to the plumbing removal: docs/swarm/README.md gets the
biggest rewrite (drops the whole "Fingerprint format" section, fixes
the runtime-effects list, the WireGuard config example + "what the
mesh does" bullet), docs/gateway.md and hive-gateway/options.nix drop
4 "needs no certFingerprint" mentions, swarm-peers-removed.nix's
migration-warning text no longer tells an upgrading operator to carry
a field over that no longer exists, swarm.nix/swarm-wireguard.nix/
swarm-controller.nix/swarm-controller's main.rs get comment fixes
where they described the now-removed HYPERHIVE_PEERS shape. Also
caught one more stale "peer hives" mention in docs/web-ui/README.md's
SW4RM tab description that the first pass on this issue missed.
Per mara's review: "describe what is, not what is not."
The header said "the swarm services' vhosts are NOT here", which
describes the file by its absences and dates immediately — the list of
what is missing grows every time a service is added, while the list of
what the file holds does not.
Reworded to say what it is: the hive's own gateway surface (the `_`
default server and the vhost named for the hive domain), with the scope
line stating where a swarm service's vhost lives rather than where it
doesn't.
The header still announced "the forge, matrix and authelia sub-domain
vhosts" and an "Accept-header SPA map". None of the three is true:
- the four swarm vhosts moved into their own service modules, so the
header named contents that are no longer in the file;
- `_` is described as serving the dashboard and per-agent routing, but
it is now a bare 444 catch-all — that surface moved to the vhost
named for the hive domain;
- the "Accept-header SPA map" is not merely gone, it is the mechanism
the file's own dashboard comment argues *against* ("routing is by
PATH, never content-type"), 140 lines below the header claiming it.
A file's opening comment is its orientation text — the first thing read
and the last thing checked — so a stale one costs more than a stale
line in the body. Rewritten to describe what the file holds now, and to
name the property the split established: a service owns its whole
gateway surface in its own module, and a service name appearing here is
the signal that the coupling is growing back.
The `_` vhost was serving the hive's own surface, so every dashboard and
agent-UI request matched the default server rather than a named vhost --
and so did a request for any name at all, including a raw IP.
Split it: `_` keeps only `return 444`, and the hive surface moves to a
vhost named for the hive domain. `_` is `mkDefault` so an operator can
claim default_server themselves, plus an assertion for the case where
they add one without turning ours off -- nginx refuses to start on a
duplicate default_server and nixpkgs asserts nothing, so that would
otherwise surface as a gateway outage at rebuild time.
Clause 2 of #3202, reading 1 (mara: "the all local stuff and swarm
services auto conf belong in those mods, not spread all over").
`gateway.localHostsEntry` is the gateway's only local-deployment knob —
`openFirewall` is about EXTERNAL exposure, `tls.acme` needs a public DNS
name, `hsts` is a hardening choice. It is now asserted by the mode in
local-defaults.nix, beside the three swarm toggles, instead of being the
one all-local implication an operator still had to know about.
`mkDefault`, so "all local except this" still needs no new option.
⚠️ The non-obvious half: this does NOT change what CONTAINERS resolve.
dnsmasq sets `no-hosts = true` unconditionally, so agents keep getting
the bridge IP from the authoritative `address=` rules rather than the
host's 127.0.0.1 — which would point every agent at its own netns. That
guard already existing is what makes this safe to default on; without it
this one line would break every agent's access to the forge.
Last of the four. The vhost, its `auth_request` block and the swarm
apex's dns record move into swarm-ui.nix; vhosts.nix drops `uiCfg`,
`controllerCfg` and `autheliaCfg` and is now 259 lines of hive surface
with no swarm service in it.
Also collapses a THIRD copy of the per-service list. `networking.hosts`
restated every service's name with its own copy of that service's guard,
after the vhosts and the dnsmasq records had each done the same. It asks
the same question — which names does this host answer for — so it now
reads the same answer: a service added later lands in /etc/hosts with no
edit, and cannot land there under a different condition than it used for
DNS.
The `forceSSL`-not-`addSSL` comment travels intact: it records that
authelia answers an http auth subrequest with 400 and nginx's
auth_request only understands 2xx/401/403, so the scheme is load-bearing
for this vhost and no other.
Moves the matrix sub-domain vhost out of the gateway's vhosts.nix, its
`address=` rule out of dnsmasq.nix, and the Accept-header
`$matrix_spa_target` map out of the gateway's appendHttpConfig — all
three into hive-matrix.nix.
The map is the one that had no business being where it was: it exists
solely for the SPA fallback in the vhost's `/` location, and
`appendHttpConfig` is a `lines` option, so a module can contribute to
it without the gateway assembling it.
The `.well-known/matrix/*` delegation deliberately stays on the hive's
own vhost. The spec requires it at the SERVER NAME, which is the hive
domain: that is the hive answering "where is my homeserver", not the
homeserver answering for itself. Moving it would have been the obvious
symmetric thing and it would have been wrong.
Moves the authelia vhost out of the gateway's vhosts.nix and its
`address=` rule out of dnsmasq.nix, into swarm-authelia.nix.
Both land inside that module's existing `cfg.enable` guard, which is
the load-bearing part: every hive in a swarm knows `authelia.url`, but
only the host that RUNS the container may claim the name. A client hive
declaring the vhost would answer for a service it does not run, and
publishing the DNS record would point every agent on its bridge at that
wrong answer.
The kit grows a fourth member, `errorPages`, because the vhost aims its
502/503/504 at the gateway's styled sso-unavailable page. Republished
rather than imported per module: a service rendering its own would drift
from the rest of the gateway the first time the theme changed.
Moves `forgeVhost` out of the gateway's vhosts.nix and the forge's
`address=` rule out of dnsmasq.nix, into nix/host-modules/hive-forge —
the module that already owns everything else about the forge.
The gateway keeps what is gateway knowledge (the listen set, which
issuer covers a name, the header block) and loses the last reason it
had to read `swarm.forge` at all: `forgeCfg` is gone from both files
and from the module's `let`.
Both halves stay gated on `behindGateway` — with it off the operator
fronts forgejo themselves, so this hive must neither claim the vhost nor
answer DNS for the name.
Adds `services.hyperhive.gateway.localNames` (internal): hostnames the
hive resolver answers with the bridge IP, contributed by the modules
that own them. The service says which name, the gateway says where it
points — the same split `lib.tlsFor` already makes.
No behaviour change yet: the list is empty until the service modules
contribute in the following commits.
The assertion is not defensive padding. Duplicate `address=` rules do
not make dnsmasq complain; it resolves them by precedence, so a name
claimed twice silently stops being served by one of its claimants. That
failure mode only becomes reachable because contribution is now open, so
it gets closed in the same commit that opens it.
Slice 1 of #3202. The listen set, the per-name TLS attrs and the
security headers move out of vhosts.nix into ./vhost-lib.nix and are
published as `services.hyperhive.gateway.lib` (internal, readOnly).
No behaviour change: vhosts.nix consumes the published value, so the
rendered vhost tree is identical.
The point is the next slice. Today a swarm service's vhost lives in
the gateway because only the gateway knows the port pair, the issuer
for a name, and the header block. Publishing those three is what lets
a service module declare its own vhost without the gateway having to
know that service by name.
Rust half mirrors hive-c0re/src/dashboard/mod.rs's utoipa pattern
exactly: an ApiDoc root, #[utoipa::path(...)] on /health (the one
existing route), and a raw JSON route at /api/openapi.json served
via OpenApiRouter::split_for_parts(). Only annotated routes appear
in the spec.
Gateway wiring extends the swarm-UI vhost (the only vhost swarm-
controller is reachable from) with:
- /api/ — proxied to the controller's unix socket untouched (no URI
segment after the socket path), so a route swarm-controller
registers is the path nginx forwards, no prefix-stripping to keep
in sync by hand.
- /api/docs/ (+ the bare /api/docs redirect) — the same
swagger-ui-theme dist the per-hive dashboard already serves at its
own /api/docs/, reused as-is since it's generic.
Both new locations reuse the same auth_request block the vhost's own
'/' already applies, factored into a shared swarmAuthRequest string —
auth_request does not inherit across sibling nginx locations, so
without this the page itself would be gated while its own API and
API docs sat open.
cargo test -p swarm-controller + cargo clippy --all-targets both
clean. Verified the new nginx wiring evaluates correctly with a
throwaway nixosSystem eval (services.hyperhive.swarm.{controller,ui}
enabled): /api/ proxies to the socket, /api/docs redirects, and both
require auth_request the same as the vhost root.
Fixes hyperhive#3212
Per mara: a general fix, not one name in one container.
Every container inherits a COPY of the host's /etc/resolv.conf at start
(nixos-containers.nix: cp --remove-destination, one shot, not a
bind-mount), so the address written there is the address every container
tries - in its own netns. That makes the value load-bearing:
value host host-netns containers bridged containers
127.0.0.1 ok ok THEIR OWN loopback
bridge IP ok ok ok
dnsmasq binds lo and the bridge, so the bridge IP works for the host
too. It is the only value correct on both sides of a netns boundary.
resolveLocalQueries goes ON for its plumbing, not its address: it points
dnsmasq's own upstreams at a separate resolv-file, without which
dnsmasq reads /etc/resolv.conf and every non-hive query loops the moment
the host is pointed at dnsmasq. Its two loopback-publishing effects
(networking.nameservers and resolvconf.useLocalResolver) are overridden.
Cost: the host's DNS now depends on dnsmasq being up. Every container
already did.
The forge container keeps its hosts entry from the previous commit -
not redundancy, a fallback in a different failure domain: it works with
no DNS at all, so SSO does not ride on a host-wide resolver change.
