WriteAgentForgeToken and WriteAgentMatrixToken write agent state files
under /var/lib/hyperhive/agents/<name>/state/. The hive-priv service
has ProtectSystem=strict which makes all of /var read-only by default,
but /var/lib/hyperhive was missing from ReadWritePaths, causing EROFS
on every token write.
Closes#1274.
Addresses mara's review: add a NixOS assertion that rejects
hsts.enable = true when no TLS mode is configured. HSTS over
plain HTTP is silently ignored by browsers; the assertion turns
a silent misconfiguration into a build-time error.
HSTS was unconditionally tied to hasTls. This is risky: enabling it on a
deployment that later loses TLS locks browsers out until max-age expires.
Add three options under services.hyperhive.gateway.hsts:
enable — bool, default false
maxAge — seconds, default 31536000 (1 year)
includeSubDomains — bool, default true
HSTS header is now only emitted when hsts.enable = true.
X-Frame-Options, X-Content-Type-Options, Referrer-Policy at server
scope on _, forge, and matrix vhosts. HSTS added when TLS is active.
nginx inheritance rule: locations with their own add_header (CORS API
endpoints like /.well-known/matrix/client, /_matrix/) are unaffected —
they already carry the headers they need. HTML-serving and proxy
locations pick the security headers up automatically.
On first boot (or after a wipe) hive-c0re writes forge-core-token only
after the forge container starts and the admin is provisioned. This
lags hive-c0re.service becoming active. The previous code bailed
immediately with TOKEN=placeholder if the token file was absent,
causing the runner to fail registration with 'token not found'.
Fix: merge both waits (core-token file appearance + forge API ready)
into a single 60s retry loop. The early-bail path is removed; the
script only exits cleanly if .runner is valid (writes placeholder)
or a fresh registration token is obtained.
When .runner exists but core-token is absent after 60s, we keep the
existing credentials (safe — the runner holds valid creds; next boot
will validate properly).
Closes#1224 (which tracks #1221).
The gitea-actions-runner's host-scheme job processes use the
service's environment PATH, not the NixOS login-shell PATH.
Without this, 'nix flake check' and similar steps fail with
'nix: command not found'.
- add pkgs.nix to environment.systemPackages (ensures the binary
is in /run/current-system/sw/bin)
- set explicit PATH on gitea-runner-hive service covering both
/run/current-system/sw/bin and /nix/var/nix/profiles/default/bin
With ProtectSystem=strict (added in the previous commit), / is read-only
inside hive-priv. nix creates a temp result symlink in its cwd; without
WorkingDirectory that cwd is / (systemd default), causing:
error: creating symlink /.tmp.tmp-...: Read-only file system
Point WorkingDirectory at the StateDirectory (/var/lib/hive-priv) which is
always writable under strict. Nix drops its temp symlink there instead.
add services.hyperhive.gateway.tls.certDir option: operators with a
CA-signed cert (Let's Encrypt, corporate CA) point at the ACME output
dir instead of using the auto-generated self-signed cert.
- tls.certDir: host path bind-mounted r/o at /run/hive-tls/ in gateway
- tls.certName / tls.keyName: filenames within certDir (default: cert.pem / key.pem, matches nixpkgs security.acme layout)
- hasTls = selfSignedTls || certDir != null: publicScheme=https in both cases
- assertion: selfSignedTls=true + certDir set together is an error
- openFirewall: httpsPort opened in both TLS modes
- docs/gateway.md: TLS modes table + operator-cert section
- docs updated in swarm.md peer config reference in the cert TLS section
when using operator cert, swarm peers can omit certFingerprint —
standard CA bundle handles trust automatically.
isolated agents reach forge via http://forge.<domain> → nginx.
without the gateway there is nothing on port 80 to serve that
hostname. assert early rather than fail silently at runtime.
addresses argus yellow note on PR #1150.
when isolateContainers=true, isolated agents have dnsmasq as their
resolver — forge.<domain> resolves to bridgeIp. route HIVE_FORGE_URL
through nginx on port 80 instead of exposing the raw forge port.
- HIVE_FORGE_URL: http://<forge.domain> when isolated (nginx proxies)
- bridge firewall: open 80+443 for agents to reach nginx (gateway)
- remove forge-specific httpPort rule (no longer needed)
- update docs/gateway.md + docs/network.md
per mara's review comment on PR #1150.
When containers run in private netns (isolateContainers=true), host
loopback is unreachable so HIVE_FORGE_URL=http://127.0.0.1:3000 breaks.
- nix/modules/hive-network.nix: when isolateContainers is on + forge
is enabled, open forge.httpPort on the bridge interface so agents
can reach forgejo at bridgeIp:httpPort (forgejo binds 0.0.0.0)
- nix/modules/hive-c0re.nix: HIVE_FORGE_URL switches to bridge IP
when network.enable && isolateContainers; loopback path retained
when isolateContainers=false
- docs/network.md: add Forge access + Forge URL rows to effects table
- docs/gateway.md: rewrite HIVE_FORGE_URL section for both modes
Add opt-in WireGuard mesh support to services.hyperhive.swarm:
- swarm.peers.<domain>.wireguardPublicKey — peer's wg public key
- swarm.peers.<domain>.wireguardEndpoint — peer's UDP endpoint (optional)
- swarm.peers.<domain>.wireguardAddress — peer's mesh IP with prefix
- swarm.wireguard.enable — bring up wg-hive interface
- swarm.wireguard.privateKeyFile — path to host's wg private key
- swarm.wireguard.address — this host's mesh IP/prefix
- swarm.wireguard.listenPort — UDP listen port (default 51820)
- swarm.wireguard.persistentKeepalive — keepalive seconds (default 25)
When enabled, generates networking.wireguard.interfaces.wg-hive with
one peer entry per mesh-enabled swarm.peers entry. Opens listenPort
UDP on the host firewall. Adds wireguard_address to HYPERHIVE_PEERS
JSON so hive-c0re can use mesh IPs for intra-swarm routing.
Assertions guard against enable=true without privateKeyFile or address.
Also refactors networking.firewall.allowedTCPPortRanges from the
nested attrset form (which conflicted with the new allowedUDPPorts
line) to the per-attribute form.
docs/swarm.md: adds WireGuard setup section with key generation
commands, two-hive config example, NAT/keepalive notes.
Two argus review fixups:
1. Move isolateContainers assertion to an unconditional `lib.mkIf
cfg.isolateContainers` arm. The prior placement inside
`mkIf (enable && isolateContainers)` was dead code — the assertion
could never fire because both flags were already true by the time the
block activated. Now `isolateContainers=true; enable=false;` raises
a NixOS assertion error at eval time.
2. Add comment on HIVE_NETWORK_SUBNET noting it carries the host-side
bridge IP (e.g. "10.42.0.1/24"), not the canonical network address
("10.42.0.0/24"). Rust consumer must normalize via bitwise AND before
subnet membership checks or address arithmetic.
Adds `services.hyperhive.network.isolateContainers` (bool, default
false). When enabled alongside `network.enable`, activates:
- IP forwarding + NAT masquerade so isolated agents reach the internet
- nftables DROP rule blocking bridge-subnet → loopback (defence-in-depth
against compromised agent reaching the c0re dashboard)
- `HIVE_NETWORK_ISOLATION`, `HIVE_NETWORK_BRIDGE`, `HIVE_NETWORK_SUBNET`
injected into the hive-c0re service env; the Rust lifecycle reads these
to set `PRIVATE_NETWORK`, `LOCAL_ADDRESS`, and `HOST_BRIDGE` in each
agent container's conf
Config block rewritten as `lib.mkMerge [...]` — the prior `lib.mkIf //
lib.mkIf` pattern was invalid nix (mkIf returns a tagged value, not an
attrset; // on it is a type error). See docs/network.md for full design.
Privsep splits repo ownership: hive-priv (root) fetches the hive-core-owned
meta/applied repos via nix, and hive-c0re (hive-core) fetches the agent-owned
proposed-config repos. git/libgit2's dubious-ownership guard rejects both
('repository path is not owned by current user'), failing every rebuild.
Install a root-trusted gitconfig with safe.directory=* in each service's
HOME; both already have read access — this only satisfies the guard.
Two privsep follow-ups, both in the hive-c0re service:
1. The migration 'chown -R hive-core /var/lib/hyperhive' (which only
started running once ExecStartPre stopped failing 203/EXEC) stomped every
agent's bind-mounted creds — agents/<name>/{claude,state,harness,config}
are owned by the per-agent/manager users — logging all agents out with no
way back in. Scope it to everything *except* agents/, plus the agents/
dir node itself so c0re can still create new per-agent subdirs. Each
container's hive-agent-user-migrate activation chowns the contents back.
2. nix (prebuild 'nix build', flake-check, meta eval in c0re; nixos-container
update->nix in priv) writes its cache under $HOME/.cache. Both services
run as users with no home -> HOME=/var/empty (unwritable) -> Lix cache
init fails, rebuilds error out. Set HOME to each service's StateDirectory
(adding one for hive-priv).
hive-priv had no PATH, so the nixos-container it runs for every container
op (incl. lifecycle::list) failed ENOENT. build_all swallows that into an
empty list — the 'no managed containers' symptom. Give the helper the
minimal set nixos-container shells out to (nixos-container, nix, util-linux,
e2fsprogs) on top of the systemd/coreutils/findutils already in the unit
PATH, rather than the whole system profile.
Also route forge/matrix is_present() through priv_client::list_containers
instead of spawning nixos-container directly from unprivileged hive-c0re.
The state-migration chown invoked ${pkgs.coreutils}/bin/sh, which does not
exist (coreutils ships chown, not sh), so ExecStartPre exited 203/EXEC on
every boot. hive-c0re hit its start limit and never came up, so the gateway
returned 502 Bad Gateway. Call chown directly with the +- prefix (run as
root, tolerate failure) instead of going through a shell for '|| true'.
Tuwunel 1.6.1 appends " 💕" to every new user's display name via the
new_user_displayname_suffix config option. Set it to empty string so
agent and operator matrix accounts get clean names.
Existing users (atlas, mara, root) need a one-time display name reset.
Atlas's was already corrected via the client API.
hive-ci-register.service now runs unconditionally on every boot (not
just when .runner is absent). Before fetching a registration token it
validates existing .runner credentials via the forge admin API:
- 200: runner still registered, write dummy token and exit
- 404: runner deleted from forge, purge .runner and re-register
- 000: forge unreachable, keep credentials (runner surfaces the error)
- other non-200 or malformed .runner: purge and re-register
Removes ConditionPathExists so stale credentials from a wiped forge
no longer block the runner indefinitely. Updates docs/ci.md to match.
Remove the custom htpasswdFile option and bind-mount. The htpasswd file
now lives at the fixed path /var/lib/hyperhive/gateway/gateway.htpasswd
on the host, which is already exposed inside the container at
/run/hive-state/gateway.htpasswd via the existing gateway state
bind-mount — no extra bind-mount needed.
A tmpfiles rule pre-creates the file so nginx can open it even before
any users exist (empty file → all requests return 401, which is correct).
hivectl gateway commands default --file to the standard path so
`hivectl gateway create-user alice` just works without any flags.
Per argus review: the hardcoded /etc/hyperhive/gateway.htpasswd
example was wrong for operators with a custom htpasswdFile path.
Move the unauthorized.html from the static agentErrorPagesDir derivation
into a pkgs.writeText inside the lib.optionalAttrs guard where
cfg.auth.htpasswdFile is in scope and statically known non-null.
The rendered page now shows the operator's actual configured path.
When HTTP Basic auth is enabled and credentials are absent or rejected,
nginx serves a Catppuccin-styled 401 page that tells the operator which
hivectl command to run to create a user. Uses error_page 401 =401 so
the browser still receives a 401 status (login dialog fires on first
visit) while getting a human-readable body when the dialog is dismissed.
The exact-match location (= /__hive_auth_unauthorized) beats location /
in nginx's prefix ordering so the internal subrequest does not loop back
through auth_basic.
Per mara's direction: both nixpkgs and nixpkgs-unstable are now
top-level meta flake inputs with explicit store-path URLs. Hyperhive
follows them rather than the other way around:
inputs.nixpkgs.url = "path:${pkgs.path}";
inputs.nixpkgs-unstable.url = "path:${nixpkgs-unstable}";
inputs.hyperhive.url = "...";
inputs.hyperhive.inputs.nixpkgs.follows = "nixpkgs";
inputs.hyperhive.inputs.nixpkgs-unstable.follows = "nixpkgs-unstable";
New NixOS host options (auto-set at build time, overridable):
services.hyperhive.c0re.nixpkgsFlake
default: "path:${pkgs.path}" — host's evaluated nixpkgs.
services.hyperhive.c0re.nixpkgsUnstableFlake
default: "path:${nixpkgs-unstable}" from hyperhive's flake.nix —
the channel that carries claude-code. Operators can override to
track a different unstable snapshot.
Legacy fallback (both args empty) preserved for backward compat.
Two new Rust tests cover the full-URL and fallback paths.
meta flake was using `nixpkgs.follows = "hyperhive/nixpkgs"` but
`hyperhive` is a store-path input, so nix resolves hyperhive's own
pinned lock rather than the host's follows-substituted version.
When an operator sets `inputs.hyperhive.inputs.nixpkgs.follows =
"nixpkgs"` in their host flake, the meta flake was silently ignoring
it and using hyperhive's pinned nixpkgs instead.
Fix: hive-c0re.nix injects `--nixpkgs-flake path:${pkgs.path}` into
the daemon's ExecStart. `pkgs` IS the host's nixpkgs when follows is
set; otherwise it's hyperhive's own pin — so the meta flake gets the
right nixpkgs in both cases. render_flake emits `nixpkgs.url = "..."`
(explicit) when nixpkgs_flake is non-empty, falling back to the old
`follows` form when empty for backward compat.
Add `hivectl gateway {create-user,delete-user,list-users}` subcommands for
managing htpasswd files used by gateway Basic auth. Pure Rust bcrypt
(cost 12, $2y$ prefix nginx accepts). No external htpasswd binary required.
Also fix the NixOS module assertion: `cfg.auth ? htpasswdFile` is always
true in the module system (declared options always exist as keys); switch
to `nullOr path; default = null` + `!= null` check so the assertion
actually fires with a useful error when enable=true but no file is set.
Guard bind-mount and nginx config against null to prevent eval errors.
Update docs/gateway.md to show hivectl commands instead of raw htpasswd.
Replaces the earlier PAM+binary approach with nginx's built-in
`auth_basic` module. No new binary, no new systemd service, no PAM.
New option `services.hyperhive.gateway.auth`:
- `enable` — off by default
- `htpasswdFile` — host path to an htpasswd file (required when enable)
- `realm` — WWW-Authenticate realm string (default "hyperhive");
restricted to `strMatching "[^\"$]*"` to prevent nginx config injection
When enabled:
- the parent directory of `htpasswdFile` is bind-mounted read-only
into the gateway container at `/run/gateway-auth/`
- the `"/"` proxy location gets `auth_basic` + `auth_basic_user_file`
Create credentials: `htpasswd -Bc /path/to/file alice` (BCrypt).
See `docs/gateway.md` ("HTTP Basic auth") for the full setup guide.
Adds opt-in HTTP Basic auth to the hive-gateway backed by the host PAM
stack + group membership check.
New binary `hive-gateway-auth` (hive-c0re workspace):
- Axum HTTP service on 127.0.0.1:7002 (host loopback)
- Decodes Basic credentials, authenticates via pam_unix.so
- Checks membership in `hyperhive-operator` group (or custom)
- Returns 200 / 401 / 403; nginx `auth_request` consumes these
New options under `services.hyperhive.gateway.auth`:
- `enable` — off by default
- `port` — auth service port (default 7002)
- `realm` — WWW-Authenticate realm string (default "hyperhive")
- `group` — required host group (default "hyperhive-operator")
- `pamService` — PAM service name (default "hive-gateway")
Host-side NixOS wiring:
- `users.groups.hyperhive-operator` declared when default group used
- `/etc/pam.d/hive-gateway` emitted via `security.pam.services`
- `systemd.services.hive-gateway-auth` runs the auth binary as root
(needs /etc/shadow access for pam_unix.so)
Gateway container nginx wiring:
- `location = /__hive_gateway_auth` — internal proxy to auth service
- `auth_request /__hive_gateway_auth` on the `"/"` proxy location
- `@hive_auth_required` named location adds WWW-Authenticate: Basic
header on 401 so browsers display a login prompt
Workspace deps: pam = "0.8"; flake.nix: linux-pam added to
nativeBuildInputs so pkg-config can find libpam at build time.
Without [actions] ENABLED = true in forgejo's config, the
GET /api/v1/admin/runners/registration-token endpoint returns
"runner registration token not found" regardless of token scopes,
causing hive-ci-register.service to fail on first boot.
gateway_nginx.rs reads HIVE_AGENT_FRONTEND_DIR (injected by hive-c0re.nix
as ${cfg.frontend}/agent). When set, agents.conf emits per-agent split
blocks instead of the old single proxy_pass:
# Compiled assets — immutable nix store path, cache 1y
location ^~ /agent/<name>/static/ {
alias <frontend>/static/;
expires 1y; add_header Cache-Control "public, immutable, ...";
}
# Static dist + proxy fallback
location /agent/<name>/ {
alias <frontend>/;
try_files $uri $uri.html $uri/index.html @<name>_dynamic;
}
location @<name>_dynamic {
proxy_pass <upstream>; # api, events, icon, login, …
…proxy headers unchanged…
}
try_files path resolution (nginx applies alias mapping first):
$uri — exact file (/static/app.js → static/app.js)
$uri.html — bare-path fallback (/stats → stats.html)
$uri/index.html — directory index (/ → index.html)
@<name>_dynamic — proxy catchall for anything not in the dist
Adding pages to the frontend dist works automatically — no generator
change needed. Per-agent extraFiles (in mergedDist, not in the base
nix-store path) continue to proxy to the agent daemon.
frontend is a nix store path injected at build time — only [a-z0-9/._-],
no shell metacharacters — safe to interpolate without sanitization;
comment added documenting this assumption.
Without HIVE_AGENT_FRONTEND_DIR the existing single-proxy block is
emitted unchanged — backward-compatible for deployments without the env.
render() takes frontend_dir as a parameter so tests exercise both code
paths safely in parallel. 13 tests: 7 legacy, 6 split-mode. No clippy
warnings in changed files.
nix/modules/hive-c0re.nix: inject HIVE_AGENT_FRONTEND_DIR = "${cfg.frontend}/agent".
Nix side of the hive-priv phase 1 work. hive-priv is socket-activated
exclusively: systemd holds /run/hive/priv.sock (mode 0660) and starts
the service on first connection. LISTEN_FDS + LISTEN_PID are passed;
hive-priv reads them in socket_listener() to accept the pre-bound fd.
Phase 2 note (comment in file): when hive-c0re drops to a non-root
user, add SocketGroup = hive-core to the socket unit so the
unprivileged caller can still connect. No code change needed in
hive-priv itself.
