nginx proxied `<hive>/` straight to hive-c0re:7000, and hive-c0re served the
dashboard dist itself via `tower_http::ServeDir` (from `HIVE_STATIC_DIR` baked
into its service env). So a frontend-only change rebuilt the hive-c0re unit and
restarted the core daemon — every operator session dropped its SSE stream for a
pure CSS/JS change.
The gateway nginx now static-serves the dashboard dist directly; hive-c0re's
dashboard router is API-only. The split uses the Accept-header SPA fallback (the
same `map $http_accept` pattern the matrix/agent vhosts already use), so no
backend prefix has to be enumerated: a browser navigation (Accept: text/html)
whose path is not an on-disk asset gets the SPA index.html; everything else
(every /api route, the bare action/mutation routes, the two SSE streams, the
knowledge webhook — all Accept != text/html) falls through `try_files` to the
`@c0re` named location and is reverse-proxied to hive-c0re. A new c0re route
needs no gateway change.
- hive-c0re.nix: expose the themed dist as a new internal read-only option
`services.hyperhive.c0re.servedFrontend`; drop `HIVE_STATIC_DIR` from the
service env (the router no longer serves files).
- hive-gateway.nix: read that option in host-module scope (dashboardDist),
static-serve `dashboard/` with the Accept-header `try_files ... @c0re` split;
`@c0re` carries `proxy_buffering off` + a 1d read timeout for the SSE streams
and a duplicated auth_basic block (named locations do not inherit it). The
dashboard map is unconditional; the matrix map stays gated on the matrix GUI.
- dashboard.rs: drop the ServeDir fallback + the HIVE_STATIC_DIR resolution; the
router 404s unmatched paths (the gateway only proxies non-static requests).
- hive-c0re/Cargo.toml: drop the now-unused tower-http dependency.
- docs/gateway.md: document the dashboard static split + the `@c0re` fall-through.
The store path is reachable inside the gateway nspawn container (shared
/nix/store), mirroring how HIVE_AGENT_FRONTEND_DIR already exposes the per-agent
UIs. The gateway and c0re changes must land together (atomic cutover) or the
dashboard 404s — this needs a watched gateway + c0re rebuild.
The CI container set `nix.settings.sandbox-fallback` but never
`nix.settings.fallback`, so a cache-miss build with an unreachable
remote build machine had no local fallback and hard-failed. A single
down or DNS-unresolvable `nix.buildMachines` host therefore turned every
fresh `nix flake check` red hive-wide, in ~30s, even for diffs that
can't affect the nix build (pure JS/markdown). Add
`nix.settings.fallback = true` so CI degrades to a slower local build
instead of failing.
Progressive enhancement: a brand-new agent's state root under
/var/lib/hyperhive/agents is created as a btrfs subvolume when the host
filesystem is btrfs, otherwise it falls back to a plain directory. No
existing agent is auto-migrated — the new path only fires when the root
does not yet exist, so plain-dir agents are left untouched until an
explicit opt-in upgrade.
Two new privileged ops (subvolume create/delete are root-only):
EnsureAgentSubvolume statfs-gates on btrfs, creates the subvolume, and
chowns it to the hive-core user so the normal state/claude/harness
mkdirs succeed inside it; DeleteAgentSubvolume btrfs-subvolume-deletes
the root iff it is actually a subvolume. hive-c0re calls Ensure before
the per-agent dirs are created (spawn/rebuild/InitConfig) and Delete on
the purge path only — destroy keeps the subvolume for revival, matching
plain-dir semantics. btrfs-progs added to the hive-priv unit PATH.
Per-subvolume usage accounting + optional quota is a separate
follow-up.
A bare `nix store ping` run as root with the daemon socket absent
resolves to a local store (root writes /nix/store directly) and pings
successfully, so the gate could false-pass at cold boot: nix-daemon.socket
carries ConditionPathIsReadWrite=/nix/var/nix/daemon-socket and is
condition-skipped until /nix/var goes read-write. systemd service units
don't source the profile that sets NIX_REMOTE=daemon, so auto-store
fallback to local is the real environment here.
Pin --store daemon so the readiness poll verifies the actual daemon
socket and the gate honours its wait-for-daemon contract instead of
passing against a local fallback while the daemon is still down.
Per review: the `tls.certDir == null && !tls.acme.enable` derivation was
duplicated in hive-gateway, hive-tls, and hive-ci. Expose it once as a
read-only internal option `services.hyperhive.gateway.useSelfSigned` (the
gateway module's single source of truth) and have hive-tls and hive-ci
consume it instead of re-deriving.
Eval-proven: gateway.useSelfSigned is true on the self-signed default /
false with tls.certDir, and the hive-tls (HIVE_TLS_CA_PATH) + hive-ci
(NODE_EXTRA_CA_CERTS) wiring derives correctly from it.
With self-signed TLS the gateway/forge serve a hive-CA-signed leaf and
forgejo's ROOT_URL is https://forge.<domain>. The CI runner's Node-based
actions (e.g. upload-artifact) POST to the ROOT_URL-derived artifact
endpoint and fail with "unable to verify the first certificate": Node uses
its own bundled CA bundle, not the system store, so it rejects the
self-signed chain. checkout etc. are fine — they hit the localhost http
registration URL.
Bind-mount the public hive CA cert (only ca.pem — never the hive-tls state
dir, which holds the CA + leaf private keys) into the hive-ci container and
set NODE_EXTRA_CA_CERTS on the runner service so every Node action trusts
it, hive-wide. Order container@hive-ci after the host hive-tls-ca service so
the cert exists before the bind-mount is set up. All gated on self-signed
mode; with an operator cert / ACME the public CA already validates and the
mount + env var are absent.
Eval-proven: self-signed → /run/hive-ca/ca.pem bind-mount (from
/var/lib/hive-tls/ca.pem), NODE_EXTRA_CA_CERTS=/run/hive-ca/ca.pem, and
container@hive-ci ordered after hive-tls-ca.service; certDir → all absent.
After a hive-ci restart the act_runner re-registers and immediately claims
any queued jobs, which can beat the in-container nix daemon coming up: its
nix-daemon.socket carries ConditionPathIsReadWrite=/nix/var/nix/daemon-socket
and is skipped until /nix/var is read-write, so the first nix-dependent
build dispatches into a cold daemon and hangs/retries (observed: a 55m48s
nix flake check vs the normal ~30s — a build-offload stall, not a code
failure; a non-nix step on the same runner passed in 1s, masking it).
Ordering the runner after/wants nix-daemon.socket does not help — a
condition-skipped unit satisfies systemd ordering immediately. Instead add
a blocking ExecStartPre that polls until the daemon actually answers
(nix store ping), placed with mkBefore so it runs ahead of the upstream
module's runner-registration ExecStartPre. The runner therefore cannot
register or claim jobs until nix is usable. Topology-agnostic (works
whether the daemon is in-container or a shared host socket); ~180s ceiling
then the unit fails cleanly rather than claiming jobs into a dead daemon.
Eval-proven: gitea-runner-hive's ExecStartPre is
[wait-nix-daemon, gitea-register-runner-hive] in that order.
Make self-signed TLS the gateway's default whenever no external TLS source
is configured, and deprecate the explicit selfSignedTls toggle. Self-signed
is now derived as `tls.certDir == null && !tls.acme.enable`, so an operator
selects a TLS mode by setting tls.certDir or tls.acme — or neither, for the
self-signed default. There is no http-only mode: matrix discovery hardcodes
https, so the gateway always terminates TLS.
The selfSignedTls option is kept as a deprecated no-op (warns when set to
false) so existing configs still eval. The two selfSignedTls mutual-
exclusion assertions and the HSTS-requires-TLS assertion are dropped — they
are impossible or vacuous now that self-signed is the floor. The hive-tls
module and the forge ROOT_URL scheme consume the derived value: the gateway
always terminates TLS, so behind the gateway the forge is always advertised
over https.
Updates docs/gateway.md (TLS-modes table, self-signed section, the removed
http-only section, firewall + discovery notes). Eval-proven: default →
self-signed (hive CA active, https ROOT_URL); tls.certDir → CA inactive;
selfSignedTls=false → deprecation warning fires.
Wire agents to trust the gateway's self-signed leaf at build time. When
the gateway runs self-signed TLS, hive-tls sets HIVE_TLS_CA_PATH in
hive-c0re's service env pointing at the host hive CA cert. The meta flake
renderer reads it and, when present, writes the public CA cert next to
flake.nix as hive-ca.pem and emits security.pki.certificateFiles so every
agent's system trust store includes the hive CA.
Build-time embedding (rather than a runtime bind-mount + bundle service)
keeps trust robust: the CA travels with the flake source, lands in the
standard NixOS trust store, and needs no per-process SSL_CERT_FILE
plumbing. Only the public CA certificate is embedded; the CA private key
never leaves the host. The cert is re-embedded and re-committed on CA
rotation even when the flake is otherwise byte-identical; when self-signed
TLS is off the embedded cert is dropped so the flake stays buildable.
Covers OpenSSL-based tools (git, curl) directly. A follow-up switches the
hive-forge reqwest client to native roots so it picks up the same store.
Replace the gateway's bare in-container self-signed leaf with a leaf
signed by a host-held hive CA. A bare self-signed leaf is its own trust
anchor, so every regeneration is a fresh anchor every consumer would have
to re-trust, and a runtime-generated in-container leaf cannot be wired
into an agent's build-time trust store at all. A stable CA fixes both: a
single anchor that agents and federation peers trust once, surviving leaf
rotation.
New hive-tls module: a host oneshot generates a long-lived CA (default
~20y) under services.hyperhive.tls.stateDir and signs a gateway leaf
(default ~10y, SAN covering the bare domain, forge., matrix. and the
wildcard). It is ordered before the gateway container so the leaf exists
when nginx starts. Active only when the gateway uses self-signed TLS
(default) and a domain is set; inert under operator-cert or ACME modes.
Gateway: bind-mount the host CA dir read-only at /run/hive-ca; the
existing in-container cert unit now imports the host leaf into nginx's
state dir (copy as root, key left root:nginx 0640 for the pre-start
config test) instead of generating one. Cert/key paths nginx serves are
unchanged.
Foundational step toward agent + federation trust of self-signed hives;
no behaviour change for agents yet (they still reach the forge over plain
http on port 80). Eval-proven across self-signed, certDir and the inert
default paths.
Fixes#1724. The forgejo ROOT_URL was hardcoded `http://` even when the
gateway terminates TLS, producing broken clone links and mixed-content
redirects for a TLS deployment.
Derive the scheme + port from gateway state: `https` (on httpsPort) when
the gateway terminates TLS (`gateway.selfSignedTls` or `gateway.tls.certDir`
set), `http` (on gateway.port) otherwise — dropping the port suffix on the
canonical port for the scheme (80 / 443). The per-agent in-cluster
HIVE_FORGE_URL is unaffected (it stays http on the gateway's :80 path);
this only changes the public ROOT_URL forgejo advertises.
Refreshes the `rootUrl` option description (the TLS scheme is auto-derived
now, so the manual-override-for-TLS note is gone).
Eval-proven: gateway.selfSignedTls = true → ROOT_URL =
https://forge.<domain>/; false → http://forge.<domain>/.
Per mara's review of #1718: the HIVE_FORGE_URL / HIVE_MATRIX_URL comments
carried "network isolation is now unconditional / shared-netns removed"
backstory that doesn't belong on those keys. Trimmed to describe just the
value (the gateway vhost) + the gatewayHost null-guard. The
isolation-removal narrative stays where it's on-topic — the network
module's own deprecated-option descriptions and warning text.
Per mara's review of #1718: this env block only exists when hyperhive is
enabled (it lives inside `config = mkIf cfg.enable`), so the
`if hyperhive.enable then gateway else loopback` branch was dead — the
loopback fallback could never be reached. Set HIVE_FORGE_URL to the
gateway vhost directly.
HIVE_MATRIX_URL drops the same redundant `hyperhive.enable` term but
keeps the gatewayHost null-guard — that one is a real fallback for a
domain-less config, not the dead enable branch.
Addresses argus's review of #1718: the HIVE_FORGE_URL / HIVE_MATRIX_URL
conditionals still keyed off the now-deprecated `network.enable`, so
setting it to `false` would point agents at host loopback (unreachable
from their private netns) even though isolation stays on — contradicting
the "deprecated and ignored" warning. Guard on the top-level
`hyperhive.enable` instead, so `network.enable` truly has no effect
anywhere and the loopback branch only covers a hyperhive-disabled host.
Eval-proven: with `network.enable = false`, HIVE_FORGE_URL now resolves
to the gateway vhost (`http://forge.<domain>`), and the deprecation
warning still fires.
Network isolation is validated working end to end (forge + matrix reach
the gateway under private netns), so the shared-host-netns mode is
removed — there is now one mode.
- hive-network.nix: the bridge/DNS block and the isolation overlay are
now unconditional whenever hyperhive is enabled (gated on
services.hyperhive.enable, not the per-feature toggles). The redundant
isolateContainers-implies-enable guard is dropped.
- network.enable and isolateContainers are kept as DEPRECATED no-op
options so existing configs that set them to true still evaluate
cleanly; setting either to false now warns (gated on hyperhive being
enabled, so a non-hyperhive host gets no spurious warning) and has no
effect. Both are slated for removal in a future release.
- hive-c0re.nix: the HIVE_FORGE_URL / HIVE_MATRIX_URL conditionals drop
the "&& isolateContainers" clause — agents always reach forge/matrix
through the gateway vhost now.
- hive-matrix.nix: refresh one stale comment.
The Rust lifecycle is env-driven (HIVE_NETWORK_ISOLATION), so no Rust
change is needed: the env is now always set, so the existing
private-netns path always runs.
Supersedes the default-on flip (the option it would have defaulted is
gone).
Forgejo's Actions artifact API hands the runner an upload URL built from
forgejo's ROOT_URL — the public forge domain (forge.<domain>), not the
runner's 127.0.0.1:<httpPort> registration URL. The hive-ci container shares
host netns but has no resolver entry for the hive's own domains (the
gateway's localHostsEntry only touches the host), so actions/upload-artifact
fails with 'getaddrinfo ENOTFOUND forge.<domain>' while normal CI (checkout
via the localhost registration URL) is unaffected. Add a networking.extraHosts
entry mapping the forge domain to loopback so the upload reaches the local
forge via the gateway.
Forgejo's local Actions-artifact storage defaults to
/var/lib/forgejo/data/actions_artifacts but Forgejo does not pre-create it.
The artifact endpoint ingests the chunked upload, then the merge-chunks step
fails with 'lstat .../actions_artifacts/tmpNNN: no such file or directory',
so every upload-artifact step dies after a successful build. Pre-create the
dir (forgejo-owned) via tmpfiles. actions.ENABLED already registers the
endpoints; this gives them somewhere to write.
Forgejo reported "does not have a signing key" on PRs despite the
forgejo-gpg-init service generating one. Two causes, both fixed:
- SIGNING_KEY = "default" resolves through the forgejo process's git
config (user.signingkey), not by scanning GNUPGHOME — and the keygen
never set it, so "default" found no key. forgejo-gpg-init now sets the
forgejo user's git user.signingkey to the generated key (+ commit and
tag gpgsign), with HOME pinned to the state dir so the global config
lands where forgejo reads it.
- The keygen guard was stamp-file based, so a partial state wipe that
lost the key while keeping the stamp would never regenerate. It's now
keyed on the actual secret key (regenerate iff gpg shows none) and
runs idempotently before forgejo on each start.
Also drops the heredoc for the gpg params in favour of printf (avoids
nix-string indentation fragility) and corrects the stale comment that
claimed "default" scans GNUPGHOME.
The runner hardcoded a 3h per-job timeout. Surface it as
services.hyperhive.forge.ci.jobTimeout so a stuck or runaway job is
bounded by a sane default while staying overridable for operators who
legitimately run longer jobs.
Default drops to 1h (comfortably covers a cold-cache nix build while
bounding a hang far sooner than the old 3h); raise it via the option
for genuinely long jobs. Accepts a Go duration string.
Resolves the stuck-runner concern from the CI-outage follow-up with a
runner-enforced job timeout rather than an external watchdog, per
operator direction.
Part of the tracker-tag cleanup: the hive convention is prose, not
issue-tracker tags, in code. Reword the 21 tags in the nix tree
(flake.nix + the hive-c0re/ci/gateway/network modules) to describe
the thing they pointed at, preserving the context without the tag.
Comment-only — no eval or logic change. Validated with nix fmt
(no reformatting) and nix flake check --no-build (all checks
evaluate clean); the full build check was skipped locally because
the shared remote builder is degraded, so CI will exercise the
build derivations once the runner recovers.
The gateway container starts alongside every hyperhive deployment, so
gating it behind a separate enable flag was a footgun: an operator who
set it false lost the only thing exposed to the outside while the
agent containers kept running. Re-gate the gateway config on the
top-level services.hyperhive.enable instead.
- hive-gateway.nix: drop the gateway.enable mkOption; gate the config
block on config.services.hyperhive.enable.
- hive-forge.nix: behindGateway now defaults to services.hyperhive.enable;
remove the behindGateway-requires-gateway assertion (now vacuous).
- hive-network.nix: remove both gateway.enable assertions (vacuous).
- hive-c0re.nix: drop the firewall.allowedTCPPortRanges 8100-8999
fallback that opened agent ports when the gateway was off (the
gateway is now the sole entry point); HIVE_GATEWAY_ENABLED is always
set since the gateway always runs.
- nix/docs/default.nix: remove the gateway.enable = mkForce false stub
(would be an eval error against the removed option; the gateway is
already re-gated on hyperhive.enable, which docs force false).
- hive-matrix.nix, dashboard.rs: comment/prose updates only.
BREAKING: operators relying on services.hyperhive.gateway.enable = false
to suppress the gateway must instead point their own reverse proxy at
the gateway's port. NixOS errors clearly on the now-unknown option.
Builds on the merged dashboard theming: the themed-frontend overlay now
swaps colors.css in both the dashboard/ and agent/ dist subtrees, and the
agent frontend dir (HIVE_AGENT_FRONTEND_DIR, which the gateway serves agent
static files from) points at the themed tree too. So when stylix is enabled
on the host, both the dashboard and the gateway-served agent UIs re-theme
from the host palette — still zero-op auto-detect, still a no-op (both dirs
on the unthemed dist) when stylix isn't imported.
Not covered: an agent reached directly on its own harness web server (no
gateway) serves from its per-agent mergedDist, built in the agent's own
nixosSystem with no host stylix access — that path needs the base16 palette
forwarded host->agent and is tracked separately.
When the operator's host config has stylix enabled, generate a base16
colors.css from its palette and overlay it onto the bundled frontend dist,
then serve that themed tree as the dashboard's HIVE_STATIC_DIR. Zero-op
auto-detect — no flake input, no operator action, no npm/esbuild rebuild
(a pure file-copy over the prebuilt dist). colors.css is the whole swap
contract; theme.css derives the semantic vars from the 16 base16 slots.
Guarded access (config.lib.stylix / config.stylix.enable via 'or' fallbacks)
makes it a clean no-op when stylix isn't imported — HIVE_STATIC_DIR stays
on the unthemed dist, which is what every non-stylix deployment + CI gets.
Dashboard surface only for now; the agent surface needs the base16 palette
forwarded host->agent (the per-agent harness builds its own dist) and is a
separate follow-up.
The systemd ExecStart carried every host-level setting as its own flag —
nine of them, including two escaped JSON blobs (the context-window map and
the model-price table). Collapse them into a single `--config <file>` JSON.
- Reuse the existing HiveEnv as the container-injected config shape (add
Deserialize + Default), and add a ServeConfig wrapper = flattened HiveEnv
plus the hive-c0re-local model_prices table (kept out of HiveEnv since it
is never injected into containers). serde(default) lets any field be
omitted and fall back to its canonical default.
- clap: add --config; the per-setting flags become optional overrides
(config file is the base, explicit flags win — preserves hivectl/debug
ergonomics and bare `hive-c0re serve`).
- Coordinator::open and cmd_serve now take the bundled HiveEnv, which drops
their too_many_arguments clippy allows. cmd_serve keeps a single
too_many_lines allow (inherent daemon-boot orchestration, not arg-driven).
- nix: write the config as JSON to the store + pass --config, so ExecStart
is one short line.
- Add a round-trip test proving the flatten + per-field defaults work.
Closes the ExecStart-length issue.
Two small follow-ups after the modelPrices PR (#1442):
- nix/modules/hive-c0re.nix: the `modelPrices` option `example` still
showed the old 5-minute sonnet `cache_write = 3.75`; the default is now
the 1-hour TTL `6.0`. Bump the example to match so it doesn't mislead.
- frontend/packages/agent/agent.css: remove the dead `.stats-empty-note`
selector (no element uses it — the stats empty state paints on canvas).
Per operator request on the PR: the built-in/default prices were the old
Claude 3 numbers (opus 15/75, etc.). Update opus + haiku to the current
Claude 4.x family list pricing (cache_write = the default 5-minute cache
TTL); sonnet was already correct:
- opus: input 5, output 25, cache_read 0.5, cache_write 6.25
- sonnet: input 3, output 15, cache_read 0.3, cache_write 3.75 (unchanged)
- haiku: input 1, output 5, cache_read 0.1, cache_write 1.25
Updated in both builtin_prices (hive_stats.rs) and the nix modelPrices
default (hive-c0re.nix), with cross-reference "keep in sync" notes on
both sides. Also addresses the earlier reviewer note: dropped the
over-strong "single source of truth" wording in the --model-prices arg
doc (the nix default does mirror the numbers in production).
- modelPrices submodule fields use lib.types.numbers.nonnegative
instead of lib.types.float: accepts bare ints (15) as well as floats
(15.0) and rejects negative prices for free.
- Collapse the triple-sourced default: hive-c0re serve --model-prices
now defaults to "{}" so builtin_prices() is the single in-code
fallback. The nix option default still carries the full
opus/sonnet/haiku table to self-document prices for operators.
The hive-wide cost estimate on the dashboard's ST4TS tab used a
hard-coded model->price table in hive_stats.rs. Anthropic list pricing
drifts, so move the table to a nix option operators can keep current
without a code change.
- New `services.hyperhive.modelPrices` option: attrset of model-family
short name -> { input, output, cache_read, cache_write } USD per
million tokens. Passed to `hive-c0re serve --model-prices <json>`.
- hive_stats: `Prices` is now public + Deserialize; add `PriceTable`
type and `resolve_prices` (longest case-insensitive substring key
wins) with the old hard-coded table preserved as `builtin_prices`
fallback for any model not covered.
- Coordinator holds the parsed table (hive-c0re-local, not injected
into containers, so not part of HiveEnv); `/api/stats-hive` reads it.
- Docs: dashboard.md ST4TS cost note updated; option self-documents
via nixosOptionsDoc.
Closes#1434
- remove the (#1500) issue tag from the resolv.conf source comment
(no-NNN-in-code rule; context lives in the commit/PR/issue link)
- drop networking.nameservers from the network.enable branch: resolvconf
is disabled, so nothing reads it to synthesise resolv.conf — the static
environment.etc."resolv.conf" is the sole source. eval output unchanged
(nameserver <bridgeIp> + options edns0), confirming it was dead config.
#1485's simplified fix turned off useHostResolvConf and trusted resolvconf
to honour networking.nameservers, but that is a runtime resolvconf behaviour
we couldn't verify at eval time — and it STILL came up with an empty
/etc/resolv.conf in practice, so tuwunel kept failing the resolver init and
matrix stayed down (#1500).
Take resolvconf out of the loop entirely: resolvconf.enable = false plus an
explicit environment.etc."resolv.conf" that writes nameserver <bridgeIp>
statically. Nothing regenerates it out from under tuwunel.
Eval-proven (unlike the prior variant): on a host with matrix+network on, the
generated container environment.etc."resolv.conf".text is
"nameserver <bridgeIp>\noptions edns0\n".
Step 3 of the hive-gateway.nix vhost cleanup. Lift the /agent/ catch-all
(+ its two internal error-page targets) and the catch-all dashboard proxy
out of the inline //-chain into agentLocations / dashboardProxyLocation
bindings. The default _ server's locations now read as a flat composition
of named groups — matrixRedirectLocations // wellKnownLocations //
agentLocations // dashboardProxyLocation // <auth optionalAttrs> — instead
of a deep nested literal. The auth-401 group stays inline (a self-contained
lib.optionalAttrs already).
Pure readability refactor, eval-identical: generated virtualHosts toJSON is
byte-identical before/after (8888 bytes, diff empty).
Step 2 of the hive-gateway.nix vhost cleanup (follows the sub-domain
vhost extraction). Lift the two deepest-nested location groups of the
default _ server — the /matrix/ 301 redirect and the
.well-known/matrix/{client,server} discovery JSON, each carrying its own
let block — out of the inline //-chain into matrixRedirectLocations /
wellKnownLocations bindings. The _ vhost locations now open with
matrixRedirectLocations // wellKnownLocations // ... instead of two ~60-line
nested literals. agent/dashboard/auth groups stay inline (a later step).
Pure readability refactor, eval-identical: the generated
services.nginx.virtualHosts toJSON is byte-identical before/after (8888
bytes, diff empty) on a host with matrix+forge+gui+auth+tls enabled.
Step 1 of the hive-gateway.nix vhost-builder cleanup. The two
sub-domain vhosts (forge, matrix) were inline `lib.optionalAttrs`
blocks `//`-appended to the virtualHosts attrset, burying the
top-level structure. Lift them into `forgeVhost` / `matrixVhost`
bindings in the config-closure let so the composition reads as
`{ "_" = ...; } // forgeVhost // matrixVhost` — the three vhosts are
now visible at a glance. Also fixes a garbled merge-mangled comment on
the nginx-reload host-trigger.
Pure readability refactor, eval-identical: verified the generated
`services.nginx.virtualHosts` toJSON is byte-identical before/after
(8888 bytes, diff empty) on a host with matrix+forge+gui+auth+tls all
enabled. The deeper `_` vhost location-group extraction is a follow-up.
The hive-matrix nixos-container came up with an EMPTY /etc/resolv.conf
even with networking.nameservers set, so tuwunel hard-failed at boot
(no nameservers found). The nixos-container default useHostResolvConf=true
puts in-container resolvconf in host-tracking mode: it ignores
networking.nameservers and never receives the host resolv.conf across the
shared-netns boundary, so resolvconf regenerates an empty file.
When the hive network module is on, turn off host-tracking (mkForce, to
beat the module default) so resolvconf honours networking.nameservers,
pointing the resolver at the gateway-container dnsmasq at bridgeIp.
Network module off -> inherit the host resolv.conf.
The runner's prefetch 401/403 diagnostic baked a literal issue tag into
its runtime journal output. Per the hive-wide rule (no issue/PR #NNN tags
in code), strip it — the message already explains the stale-core-token
condition and the operator remedy in full prose, so nothing is lost.
Two bugs that together kept the runner registration token from
refreshing (#1475):
1. Unit name: the prefetch's before/wantedBy/partOf targeted
nixos-container@hive-ci.service, but a declarative containers.<n> is
the host unit container@<n>.service (confirmed against the live
container@hive-matrix.service during the #1465 incident). The wrong
name made all three silent no-ops, so the partOf never bound — the
RemainAfterExit oneshot stayed 'active (exited)' and never re-ran on
nixos-container restart, leaving the stale token in place. Corrected
to container@hive-ci.service.
2. 401-hardening: the registration-token fetch used a bare curl -sf | jq,
so a forge-core-token that is stale/invalid for the current forge
(e.g. after a forge rebuild) 401s and fails silently every attempt for
the full 60s loop, then exits with a misleading 'core token absent or
forge unreachable'. Now capture the HTTP status and fail fast + loudly
on 401/403 with a clear message pointing at re-minting the core token.
The host systemd unit for a declarative nixos-container is
container@<name>.service (confirmed from the live
container@hive-matrix.service status), not nixos-container@. The prior
commit's nixos-container@ ordering targeted a non-existent unit (a
silent no-op). Use container@hive-{matrix,gateway}.service so the
after-ordering actually takes effect.
Per operator review on the PR: when the network module is on, the
matrix container's resolver is the dnsmasq in the gateway container, so
order the matrix container start after the gateway container. This is
robustness for tuwunel's lazy federation lookups, not a boot
requirement — the boot fix is the resolv.conf nameserver line (the
failure was a parse error on an empty resolv.conf, not connectivity).
Soft 'after' (not 'requires') keeps lifecycles decoupled; network.enable
asserts gateway.enable so the gateway container unit always exists.
tuwunel hard-fails to start when /etc/resolv.conf has no nameserver
line (Failed to configure DNS resolver: no nameservers found in
config -> exit 1 -> systemd start-limit). The declarative
containers.hive-matrix generates its own resolv.conf via resolvconf
and, unlike agent containers whose resolv.conf is written by
hive-c0re's lifecycle, has no nameserver source -> it comes up empty
(just 'options edns0'). Defaulting network.enable on surfaced this:
the host DNS moved to the bridge dnsmasq but the container was never
pointed at it, so the homeserver could not boot, taking down matrix
for all agents.
Point the container at the hive resolver (the dnsmasq the network
module runs at bridgeIp) when the network module is enabled; the
container always shares the host netns (privateNetwork = false) so it
reaches bridgeIp whether or not isolateContainers is set. With the
network module off, inherit the host resolv.conf.
The matrix server-server spec resolves a portless m.server delegated
host to the federation default port 8448 (not 443 — the URL convention
that https implies 443 does not apply to the m.server field). The
gateway reused publicPortSuffix, which intentionally drops :443 for the
client base_url, so .well-known/matrix/server advertised the bare
gateway host. Peers then federated to <gatewayHost>:8448, which nothing
listens on, while the federation endpoint actually lives on the gateway
443 vhost (/_matrix/ proxied to tuwunel). Always emit the explicit port
in the federation delegation so peers reach the live 443 endpoint.
