vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix(3179): the gateway's config files get their own state dir

Browse source 
`agents.conf` and `gateway.htpasswd` move from /var/lib/hyperhive/gateway
to /var/lib/hive-gateway/conf, alongside the `tls/` the gateway already
kept there.

nginx reads both as an unprivileged user. Under c0re's state dir it could
only reach them by traversing a directory systemd re-declares `0750
hive-core` on every c0re start — so nginx was given `SupplementaryGroups
= [ "hive-core" ]`, which also handed it read access to everything else
group-readable in that tree. The tokens are individually 0600, but the
broker sqlite carries no explicit mode: every message between every agent
was readable by the process whose job is parsing untrusted network input.

Moving the files removes the need and the exposure together. The group is
gone, and its absence is now commented as load-bearing so it doesn't come
back as a fix for a symptom it would recreate.

Also drops this module's `/var/lib/hyperhive` tmpfiles rule. It declared
`0755 root root` and could never win against `StateDirectoryMode`, and a
losing declaration still reads as a guarantee — that is what sent the
first diagnosis of the outage looking for who had changed the mode.

Ordering is unchanged and still the thing that makes a fresh boot work:
tmpfiles runs before services and seeds both files empty-but-valid, nginx
names them (an `include` of a missing file is fatal, not empty), and
content arrives when c0re writes and reloads — which it does on every
topology change, so a boot against the empty seed resolves itself.

Folds in the mode fix: `write` now sets 0644 on the tmp file before the
rename, because a rename carries the source's mode and discards the
destination's, and the tmpfiles rule that declares 0644 is
create-if-absent so it never re-applies.
This commit is contained in:
atlas 2026-08-12 00:05:53 +02:00 • committed by mara
commit 0e1a975f9f
9 changed files with 98 additions and 50 deletions
Download patch file Download diff file Expand all files Collapse all files

81
nix/host-modules/hive-gateway/default.nix
View file

@ -117,23 +117,44 @@ in
# `create_dir_all(/run/hive-agent/<name>)` itself, so a root-owned
# parent would EACCES on the very first agent create on a fresh host
# (hive-priv only chowns the subdir afterwards, it doesn't make it).
# /var/lib/hyperhive — hyperhive state dir, created by c0re on
# first run. Also pre-seed agents.conf with an empty-but-valid
# header so nginx can start + include the file before c0re writes
# its first real content (f = create-if-absent, no overwrite).
#
# ⚠️ There is deliberately NO rule for /var/lib/hyperhive here. One
# used to declare it `0755 root root` and could never win:
# `hive-c0re.service` sets `StateDirectory = "hyperhive"` with
# `StateDirectoryMode = "0750"`, which systemd re-applies on every
# start. Two mechanisms owning one path, and the loser still read as
# a guarantee — it is why the resulting outage was first misdiagnosed
# as someone having changed the mode. c0re's own unit owns that dir;
# this module no longer has an opinion about it.
systemd.tmpfiles.rules = [
# Must stay in step with the identical rule hive-priv generates into
# /etc/tmpfiles.d/hyperhive-agents.conf — the two used to declare
# different owners for this path.
"d /run/hive-agent 0755 hive-core hive-core - -"
"d /var/lib/hyperhive 0755 root root - -"
"d /var/lib/hyperhive/gateway 0755 root root - -"
"f /var/lib/hyperhive/gateway/agents.conf 0644 root root - # Generated by hive-c0re — do not edit.\n"
# Pre-create the htpasswd file so nginx can open it even before any
# users have been added. An empty file causes all auth checks to
# return 401 (no valid credentials), which is the correct no-users
# behaviour. `f` = create-if-absent, never overwrite.
"f /var/lib/hyperhive/gateway/gateway.htpasswd 0644 root root - -"
# The gateway's own config dir — NOT under /var/lib/hyperhive. c0re
# writes here, nginx reads here, and neither needs any access to the
# other's tree: no shared parent to traverse means no group
# membership handing nginx c0re's broker db and everything else
# beside it. Sibling of `tls/`, which already lived under this root.
#
# Owned by hive-core because c0re is the writer; 0755 so the
# unprivileged nginx user can traverse and read. This is now the
# ONLY declaration of these paths' modes — nothing re-applies a
# different owner on top the way `StateDirectory=` does for
# /var/lib/hyperhive.
"d /var/lib/hive-gateway 0755 root root - -"
"d /var/lib/hive-gateway/conf 0755 hive-core hive-core - -"
"f /var/lib/hive-gateway/conf/agents.conf 0644 hive-core hive-core - # Generated by hive-c0re — do not edit.\n"
# Pre-create both files so nginx's config test can open them before
# c0re has ever written: nginx names them, and an `include` of a
# missing file is a fatal config error, not an empty one. tmpfiles
# runs before services, which is the whole ordering guarantee —
# content arrives when c0re writes and reloads, which it does on
# every topology change. `f` = create-if-absent, never overwrite.
#
# An empty htpasswd causes all auth checks to return 401 (no valid
# credentials), which is the correct no-users behaviour.
"f /var/lib/hive-gateway/conf/gateway.htpasswd 0644 hive-core hive-core - -"
];
# ⚠️ REMOVED WITH THE CONTAINER, and each one was a workaround for the
@ -285,22 +306,26 @@ in
inherit (nginxTree) appendHttpConfig virtualHosts;
};
# nginx now reads `/var/lib/hyperhive/gateway/agents.conf` (+
# gateway.htpasswd) directly off the host filesystem instead of
# through the old container's dedicated `/run/hive-state`
# bind-mount. `hive-c0re.service` declares `StateDirectory =
# "hyperhive"` with `StateDirectoryMode = "0750"` owned by
# `hive-core`, and systemd re-applies that owner/mode to the
# top-level `/var/lib/hyperhive` dir on every c0re start —
# overriding this module's own `0755 root:root` tmpfiles rule
# above. Without group membership, the `nginx` user can't even
# traverse into the directory, so nginx fails its config test and
# never starts (`nginx: [emerg] open() ".../agents.conf" failed
# (13: Permission denied)`) — the whole gateway, and every hive
# domain behind it, goes down. `gateway/` and `agents.conf` are
# already declared world-readable (0755 / 0644), so group
# traversal on the parent is the only thing missing.
systemd.services.nginx.serviceConfig.SupplementaryGroups = [ "hive-core" ];
# ⚠️ NO `SupplementaryGroups = [ "hive-core" ]` on nginx, and its
# absence is load-bearing rather than an omission.
#
# It used to be here, to let nginx traverse `/var/lib/hyperhive` and
# reach the config fragments that lived inside: `hive-c0re.service`
# declares `StateDirectory = "hyperhive"` with `StateDirectoryMode =
# "0750"` owned by `hive-core`, and systemd re-applies that on every
# c0re start — beating this module's own tmpfiles rule for the same
# path. Without the group, nginx failed its config test and never
# started (`nginx: [emerg] open() ".../agents.conf" failed (13:
# Permission denied)`), taking the gateway and every hive domain
# behind it down.
#
# The group fixed the symptom and paid for it: it also gave nginx
# read access to everything ELSE group-readable under that dir,
# including the broker sqlite — i.e. every message between every
# agent, reachable by the process whose entire job is parsing
# untrusted network input. Moving the fragments to the gateway's own
# dir removes the need and the exposure together. Re-adding this line
# would restore both.
# dnsmasq is a host service alongside nginx, so it reads the host's
# /etc/resolv.conf directly and picks up network changes as they

2
nix/host-modules/hive-gateway/options.nix
View file

@ -232,7 +232,7 @@ in
enabled, every request to the gateway's main vhost requires a
valid username and password. nginx's built-in `auth_basic`
module validates credentials against
`/var/lib/hyperhive/gateway/gateway.htpasswd`. Off by default.
`/var/lib/hive-gateway/conf/gateway.htpasswd`. Off by default.
Manage users with `hivectl gateway create-user`, `delete-user`,
and `list-users` — see `hivectl gateway --help` for usage.

8
nix/host-modules/hive-gateway/vhosts.nix
View file

@ -289,7 +289,7 @@ let
# `/agent/` catch-all 404 + the two internal error-page targets it
# points at. Per-agent `location /agent/<name>/` blocks live in the
# runtime-generated `/var/lib/hyperhive/gateway/agents.conf` (included via
# runtime-generated `/var/lib/hive-gateway/conf/agents.conf` (included via
# `extraConfig` on the vhost); nginx longest-prefix-match makes a
# real `/agent/<name>/` beat this catch-all. `internal` keeps the
# error pages reachable only through nginx's error handling.
@ -323,7 +323,7 @@ let
# secret (`X-Hub-Signature-256`) protects those endpoints instead.
dashboardAuth = lib.optionalString cfg.auth.enable ''
auth_basic "${cfg.auth.realm}";
auth_basic_user_file /var/lib/hyperhive/gateway/gateway.htpasswd;
auth_basic_user_file /var/lib/hive-gateway/conf/gateway.htpasswd;
# `=401` keeps the status 401 so the login dialog shows; the
# internal page explains `hivectl gateway create-user`.
error_page 401 =401 /__hive_auth_unauthorized;
@ -444,7 +444,7 @@ in
};
};
# Per-agent location blocks, generated at runtime by
# hive-c0re and written to /var/lib/hyperhive/gateway/agents.conf
# hive-c0re and written to /var/lib/hive-gateway/conf/agents.conf
# on the host — the same machine nginx runs on. nginx parses
# `include` at config-load time so a reload (triggered by c0re
# after each agents.conf write) picks up new or removed
@ -452,7 +452,7 @@ in
# match rule ensures `/agent/<name>/` from this file beats
# the `/agent/` catch-all above.
extraConfig = securityHeaders + ''
include /var/lib/hyperhive/gateway/agents.conf;
include /var/lib/hive-gateway/conf/agents.conf;
'';
};
}