The per-agent and manager sockets ran two parallel dispatchers with
duplicated lifecycle handlers (agent-side topology-gated, manager-side
ungated) plus a manager-only handler set. Collapse to one parameterized
server in socket_server.rs:
- one serve() + dispatch(req, agent, privileged, coord); start() binds
the per-agent sockets (privileged=false), start_manager() binds the
manager socket (privileged=true).
- each lifecycle/config handler (start/restart/kill/update/init_config/
apply_commit) merges its dual: the topology guard (require_child /
require_new_child) runs only on the !privileged path; init_config
records the requester as parent only when !privileged. restart keeps
the orthogonal, capability-gated + audited infra-container branch.
- the agent-state queries (loose-ends / reminder count + rollup) branch
on privileged: privileged keeps any-target + the "*" hive-wide sweep
(query_agent_state-gated), non-privileged keeps the topology/cap gate.
- the privileged-only verbs (schedules / meta-inputs / get_logs) plus
the submit/schedule/watchdog helpers move into socket_server; they are
reached via dispatch_privileged_only(), which rejects the whole group
on a non-privileged socket.
- delete manager_server.rs; repoint refs; merge the test modules.
No behavior change: the topology guard still applies on every
non-privileged lifecycle call, the privileged socket still acts on any
agent, and privileged-only verbs are still rejected on agent sockets.
Pure rename ahead of the agent+manager server consolidation: the
per-agent socket dispatcher already hosts the shared dispatch and all
lifecycle handlers, and will absorb the manager-only handlers next, so
`agent_server` becomes a misnomer. No logic change — git mv plus a
mechanical `agent_server` -> `socket_server` rename across refs.
Adds a durable, operator-visible audit trail of privileged operations
hive-c0re performs on behalf of an agent — the ones that cross the
agent/operator trust boundary. First entry: infra-container restarts via
the infra_admin-gated `restart` tool, which until now were recorded only
as a hive-priv journal trace.
Backend:
- new `audit_log` module: sqlite-backed store (audit_log.sqlite, same dir
as build_logs) with schema (ts/agent/action/target/outcome/detail),
best-effort `record`, `list_recent` (clamped 500), 90-day `vacuum`, and
a process-singleton handle mirroring build_logs.
- Coordinator opens + installs the handle; main spawns the hourly vacuum.
- agent_server::handle_restart_infra records every attempt (ok, error, and
capability-denied) via the global handle — best-effort, never fails the
underlying action.
- dashboard: `GET /api/audit-log` returns recent entries as JSON.
Scope is deliberately agent-initiated privileged actions only (not every
PrivRequest — token writes + nspawn edits are constant lifecycle noise).
Extensible: future agent-initiated priv ops record via the same handle.
Unit tests cover record/list ordering, the 500 clamp, and retention vacuum.
The dashboard *surface* (an AUDIT view consuming /api/audit-log) is a
frontend follow-up coordinated with iris.
Per operator request: instead of a disk-specific alert, surface a generic
server-warnings banner at the very top of every page, so new system
warnings can be added backend-side with no frontend change.
- hive-c0re `host_stats`: `server_warnings() -> Vec<ServerWarning>`
(`{ kind, level, message }`). The threshold logic lives server-side; the
host disk-pressure check (a `statvfs` probe of `/nix`: ≥85% used → warn,
≥95% → crit) is the first and only producer today. No new deps (libc).
- `/api/state` carries `server_warnings` (replaces the disk-specific
field). Empty when all clear.
- frontend: `renderServerWarnings` / `initServerWarnings` in `common.js`
inject a sticky top-of-<body> banner and render the list, coloured by
`level`. Wired on every page — dashboard (live, via refreshState),
FL0W, L0GS, H0M3. No per-warning frontend code; adding a warning kind
is a pure backend change.
cargo check/clippy/fmt + npm run build green. Closes#1518.
C0NT41N3R L04D on the SYST3M tab + GET /api/container-resources.
Backend (hive-c0re/src/container_stats.rs): reads cgroup v2 cpu.stat +
memory.{current,peak,max} for each running agent machine
(machine-h\x2d<name>.scope under machine.slice), read-only/world-
readable so no hive-priv. CPU is a two-sample (~200ms) host-normalised
percentage; one shared sleep covers all agents. Skips agents whose
scope dir is absent (= not running). Network omitted: agents share the
host netns, no per-container counter.
Frontend: a polled C0NT41N3R L04D table on SYST3M (agent / cpu / mem /
peak / limit with meter bars), reusing the ST4TS table style. Polls
/api/container-resources every 5s only while the tab is active.
Backend reviewed-in-principle by damocles (path escaping + cpu delta
math); ping for the on-host sign-off.
#1424 P2 backend. New hive_stats module aggregates every agent's
turn-stats.sqlite read-only (reusing Coordinator::kept_state_names +
agent_harness_dir, skipping missing/unreadable dbs) into swarm totals,
a busiest-first per-agent rollup, swarm model mix, and a labelled USD
cost estimate (rough model->price table; can move to a nix option
later). Exposed as GET /api/stats-hive?window=. Dashboard UI follows.
Add `hivectl gateway {create-user,delete-user,list-users}` subcommands for
managing htpasswd files used by gateway Basic auth. Pure Rust bcrypt
(cost 12, $2y$ prefix nginx accepts). No external htpasswd binary required.
Also fix the NixOS module assertion: `cfg.auth ? htpasswdFile` is always
true in the module system (declared options always exist as keys); switch
to `nullOr path; default = null` + `!= null` check so the assertion
actually fires with a useful error when enable=true but no file is set.
Guard bind-mount and nginx config against null to prevent eval errors.
Update docs/gateway.md to show hivectl commands instead of raw htpasswd.
Part of the docs-migration chore (issue #708). Remove GitHub issue
numbers from inline comments, option descriptions, and rustdoc —
these are contextless noise for anyone reading the code without
access to the original discussions. Replace with prose that captures
the same rationale directly.
No functional change. Build still clean (cargo check passes).
Replace eval-time per-agent nginx location baking with a runtime
include file. c0re writes /var/lib/hyperhive/agents.conf (nginx
location blocks, UDS or TCP per agent) on every topology change and
on the 10s marker poll. The gateway container bind-mounts
/var/lib/hyperhive/ at /run/hive-state/ and nginx includes
/run/hive-state/agents.conf. A systemd path unit inside the container
watches the file for changes and fires `nginx -s reload` on each
atomic rename from c0re — no nixos-rebuild switch needed when agents
start, stop, or flip useUnixSocket.
- new hive-c0re/src/gateway_nginx.rs: write() + render()
- lib.rs + meta.rs + agent_sockets::spawn_poll: hook in write()
- hive-gateway.nix: drop agentPortsTable/agentSocketsTable/
agentUpstreamFor/lib.mapAttrs', add /run/hive-state bind-mount,
include directive, systemd path unit + reload service, tmpfiles
for /var/lib/hyperhive + agents.conf seed
- docs/gateway.md: update vhost table + Per-agent UDS section
Backend half of #726. The 32-line stderr ring buffer in
`lifecycle::run` and `lifecycle::prebuild_toplevel` routinely
truncated the actual eval error — a single 'tried alternatives'
block out of a failing module ref is often 30+ lines on its own,
which pushed the real cause out of the bailout message. With this
patch the full stream lands in sqlite where the dashboard can
surface it; bail-outs now point at the build log id instead of an
arbitrary tail.
### New module: `hive-c0re::build_logs`
`BuildLogs::open(db_path)` creates a sqlite db at
`<db_path>/build_logs.sqlite`. Schema: id, agent, kind, cmdline,
started_at, finished_at, status, stdout, stderr — indexed for
both per-agent latest-N queries and the status-driven retention
sweep. API: `start / append_stdout / append_stderr / finish` for
the streaming writer side (best-effort — every append handles
sqlite errors via tracing::warn so a transient blip never tears
down a rebuild), plus `list_recent_for_agent / get_full` for the
read side (50-row cap clamped server-side).
### Process-singleton handle
`build_logs::install / global()` install the `Arc<BuildLogs>` at
`Coordinator::open` so `lifecycle::run` and
`lifecycle::prebuild_toplevel` can write without us threading
the handle through every `pub async fn` entry point in the
lifecycle surface — there are 10+ call sites and the handle is
the same Arc everywhere anyway. Reads via `global()` return None
in early-startup / standalone-test paths so callers no-op cleanly.
### Lifecycle integration
`run` derives the kind from `args[0]` (the nixos-container verb)
and the agent name from `args[1]` (stripped of the `h-` agent
prefix so dashboard grouping matches the bare agent name). It opens
a row before spawning, pipes stdout/stderr into both tracing AND
the row, then `finish`es with the terminal status. `prebuild_toplevel`
does the same with kind = "prebuild" and the agent name already
in scope from its caller. On failure both bail with "see build log
#<id>" instead of the ring-buffer tail.
### Retention
`spawn_vacuum` mirrors `stats_vacuum`/`events_vacuum` in shape —
hourly tick that calls `BuildLogs::vacuum()`. Rule: failures kept
30d (operators dig into them), successes 24h (mostly noise after
a day), in-flight rows never reaped regardless of age (running
builds shouldn't disappear from their own log viewer mid-stream).
### Out of scope (follow-ups)
- Dashboard endpoints (`GET /api/build-logs/{agent}`,
`GET /api/build-logs/{id}`) — wire layer
- ContainerView.build_logs field — agent-card chip data source
- Side-panel viewer + SSE `build_log_appended` event — UX
- Download-as-text link — operator workflow polish
These all stack cleanly on top of the data layer + writer this
PR ships. Filing as phase 2 PRs.
### Validation
- 5 new unit tests pass (start/append/finish flow, list ordering
+ clamp, get_full miss, vacuum per-status rule, post-finish
append fault tolerance)
- 157 hive-c0re lib tests pass overall
- cargo check workspace clean
Refs #726.
sibling to agent_ports.rs (#748 / #15). Writes
/var/lib/hyperhive/agent-sockets.json mapping agent name → unix
socket path under /run/hive-agent/<name>.sock — the source of truth
for the gateway's eventual unix-socket upstreams once it flips off
TCP per #784's transition plan.
shape mirrors agent_ports.rs:
- BTreeMap for deterministic JSON output
- atomic <path>.tmp + rename for partial-write safety
- idempotent (skip rewrite when content unchanged)
- MANAGER_NAME filter (gateway routes manager via dashboard upstream,
not /agent/<n>/)
socket_path_for(name) is pure-deterministic so a name alone resolves
to the same path the c0re writes + the harness binds via
HIVE_WEB_SOCKET (post-PR #800).
hooked into meta::sync_agents alongside the existing
agent_ports::write call (same best-effort + non-fatal shape).
next: step 2b (lifecycle bind-mount /run/hive-agent/ into agent +
gateway containers, with per-agent isolation per mara's #800 nudge
'agents can only access their own sockets'). then atlas's step 3
(gateway proxy_pass http://unix:...). then step 4 drops TCP.
