The forwarded host env vars (HIVE_FORGE_URL, HIVE_FORGE_PUBLIC_URL,
HYPERHIVE_HIVE_DOMAIN, ...) were emitted only into the harness
service's own unit environment. Under network isolation that left the
bash-task runner, the matrix daemon, tea-login and interactive shells
without HIVE_FORGE_URL, so `hive-forge` and friends fell back to the
loopback default (localhost:3000) — unreachable from a private netns,
where the in-cluster gateway URL (forge.<domain>) is the only path.
Emit the forwarded set into systemd.globalEnvironment too, which every
unit + shell in the container inherits, so all in-container surfaces
see the same forge/matrix endpoints. The existing harness-service
forward is kept (harmless redundancy). Shared netns is unaffected (the
localhost default still works there).
Adds a regression test asserting the forwarded vars land inside the
globalEnvironment block, and the generated flake parses.
argus: record() is pub and returns Option<AuditEntry>; the hive rule
requires #[must_use] on public fns returning a non-trivial value. Call
sites already consume the return (handle_restart_infra via if-let, tests
via let _).
Follow-up to the audit-log backend + surface. Emits a DashboardEvent on
each agent-initiated privileged action so the audit view live-appends off
/dashboard/stream instead of polling.
- new DashboardEvent::AuditEntryAdded { seq, <flattened AuditEntry> } —
serde tag `audit_entry_added`; the AuditEntry fields flatten to the top
level so the wire shape matches one /api/audit-log `entries` row exactly.
- Coordinator::emit_audit_entry helper (stamps seq like the others).
- audit_log::record now returns the canonical inserted AuditEntry (id + ts
assigned) so the streamed event is the same row that was stored — no
drift. Best-effort unchanged (None on a sqlite blip).
- handle_restart_infra records + emits for every attempt (ok/err/denied),
threading the coordinator through.
Tests: kind_tag round-trip now covers the new variant; added a flatten
test pinning the top-level wire shape (kind/seq/id/…/detail, no nesting).
Pairs with iris's audit view (the /dashboard/stream listener half).
argus review: the four public Result-returning methods (open, list_recent,
count_total, vacuum) need # Errors sections per the hive Rust doc rules;
open had no doc comment. Added all four.
Per iris's dashboard-side ask: return { entries, total } instead of a
bare array so the UI can show 'latest 500 of N' rather than silently
capping at the clamp. Adds AuditLog::count_total().
Adds a durable, operator-visible audit trail of privileged operations
hive-c0re performs on behalf of an agent — the ones that cross the
agent/operator trust boundary. First entry: infra-container restarts via
the infra_admin-gated `restart` tool, which until now were recorded only
as a hive-priv journal trace.
Backend:
- new `audit_log` module: sqlite-backed store (audit_log.sqlite, same dir
as build_logs) with schema (ts/agent/action/target/outcome/detail),
best-effort `record`, `list_recent` (clamped 500), 90-day `vacuum`, and
a process-singleton handle mirroring build_logs.
- Coordinator opens + installs the handle; main spawns the hourly vacuum.
- agent_server::handle_restart_infra records every attempt (ok, error, and
capability-denied) via the global handle — best-effort, never fails the
underlying action.
- dashboard: `GET /api/audit-log` returns recent entries as JSON.
Scope is deliberately agent-initiated privileged actions only (not every
PrivRequest — token writes + nspawn edits are constant lifecycle noise).
Extensible: future agent-initiated priv ops record via the same handle.
Unit tests cover record/list ordering, the 500 clamp, and retention vacuum.
The dashboard *surface* (an AUDIT view consuming /api/audit-log) is a
frontend follow-up coordinated with iris.
The dashboard LOAD tab always showed "no running agent containers":
container_stats::gather() looked for each agent's cgroup at
machine.slice/machine-h\x2d<name>.scope, but that path never exists.
nixos-container runs `systemd-nspawn --keep-unit` with
`Slice = "machine.slice"` (nixpkgs virtualisation/nixos-containers.nix),
so --keep-unit means no separate machined `machine-<name>.scope` is
created — the container's cgroup IS the launching service unit,
`container@<machine>.service`, under machine.slice. systemd-machined
still logs "New machine <name>" (registration), which is what made the
scope-path assumption look plausible, but the cgroup stays on the unit.
Fix scope_dir to machine.slice/container@<machine>.service and drop the
\x2d escaping (the service unit name is used verbatim in the cgroup dir;
the journal shows the literal `container@h-<agent>.service`). Adds a unit
test pinning the path.
When isolateContainers=true, claude (and all egress) broke in every
container: agents came up with an IP but no way off the bridge subnet.
Two container-side gaps, both confirmed against nixpkgs
nixos-containers.nix:
1. No default route. hive-priv wrote HOST_ADDRESS= empty in the nspawn
conf. nixos-container's container-side setup only installs
`ip route add default via $HOST_ADDRESS` when HOST_ADDRESS is
non-empty, so the container had an address but no gateway -> nothing
off-subnet (incl. api.anthropic.com) was reachable. Fix: write
HOST_ADDRESS=<bridge-ip>. In bridge mode the host-side address/route
setup is skipped, so this only affects the container's default route.
2. No usable resolver. nixos-container copies the host's /etc/resolv.conf
into the container at every start; the host resolver (e.g. 127.0.0.53)
is unreachable from a private netns and isn't authoritative for the
hive's own zones. Fix: hive-priv drops a marker carrying the gateway
IP only when isolated, and a new harness-base oneshot
(hyperhive-isolated-dns) rewrites resolv.conf to point at the bridge
dnsmasq. Inert in shared-netns mode (no marker), so the shared
container toplevel does the right thing in both modes.
The gateway IP is the address part of HIVE_NETWORK_SUBNET (the bridge IP
verbatim, honouring a non-.1 operator override), via a new validated
bridge_gateway_ip() helper with unit tests.
Unblocks defaulting isolation on.
Clean the legacy backlog so the tracker-tag lint can become a required
gate (mara's warn-during-cleanup -> full-tree-enforcement path). Rewrite
the ~33 real `closes/see #NNN` provenance refs in doc-comments to prose
across hive-forge, hive-c0re, hive-ag3nt, hive-matrix-mcp, hive-sh4re,
and add a `lint:allow` line marker to check-issue-refs.sh for genuine
non-tracker `#<digits>` (a hash-digit heading-detection test input).
Tree is now lint-clean; tracker-tag lint ready to promote to required.
The gateway container starts alongside every hyperhive deployment, so
gating it behind a separate enable flag was a footgun: an operator who
set it false lost the only thing exposed to the outside while the
agent containers kept running. Re-gate the gateway config on the
top-level services.hyperhive.enable instead.
- hive-gateway.nix: drop the gateway.enable mkOption; gate the config
block on config.services.hyperhive.enable.
- hive-forge.nix: behindGateway now defaults to services.hyperhive.enable;
remove the behindGateway-requires-gateway assertion (now vacuous).
- hive-network.nix: remove both gateway.enable assertions (vacuous).
- hive-c0re.nix: drop the firewall.allowedTCPPortRanges 8100-8999
fallback that opened agent ports when the gateway was off (the
gateway is now the sole entry point); HIVE_GATEWAY_ENABLED is always
set since the gateway always runs.
- nix/docs/default.nix: remove the gateway.enable = mkForce false stub
(would be an eval error against the removed option; the gateway is
already re-gated on hyperhive.enable, which docs force false).
- hive-matrix.nix, dashboard.rs: comment/prose updates only.
BREAKING: operators relying on services.hyperhive.gateway.enable = false
to suppress the gateway must instead point their own reverse proxy at
the gateway's port. NixOS errors clearly on the now-unknown option.
The systemd ExecStart carried every host-level setting as its own flag —
nine of them, including two escaped JSON blobs (the context-window map and
the model-price table). Collapse them into a single `--config <file>` JSON.
- Reuse the existing HiveEnv as the container-injected config shape (add
Deserialize + Default), and add a ServeConfig wrapper = flattened HiveEnv
plus the hive-c0re-local model_prices table (kept out of HiveEnv since it
is never injected into containers). serde(default) lets any field be
omitted and fall back to its canonical default.
- clap: add --config; the per-setting flags become optional overrides
(config file is the base, explicit flags win — preserves hivectl/debug
ergonomics and bare `hive-c0re serve`).
- Coordinator::open and cmd_serve now take the bundled HiveEnv, which drops
their too_many_arguments clippy allows. cmd_serve keeps a single
too_many_lines allow (inherent daemon-boot orchestration, not arg-driven).
- nix: write the config as JSON to the store + pass --config, so ExecStart
is one short line.
- Add a round-trip test proving the flatten + per-field defaults work.
Closes the ExecStart-length issue.
Per operator request: instead of a disk-specific alert, surface a generic
server-warnings banner at the very top of every page, so new system
warnings can be added backend-side with no frontend change.
- hive-c0re `host_stats`: `server_warnings() -> Vec<ServerWarning>`
(`{ kind, level, message }`). The threshold logic lives server-side; the
host disk-pressure check (a `statvfs` probe of `/nix`: ≥85% used → warn,
≥95% → crit) is the first and only producer today. No new deps (libc).
- `/api/state` carries `server_warnings` (replaces the disk-specific
field). Empty when all clear.
- frontend: `renderServerWarnings` / `initServerWarnings` in `common.js`
inject a sticky top-of-<body> banner and render the list, coloured by
`level`. Wired on every page — dashboard (live, via refreshState),
FL0W, L0GS, H0M3. No per-warning frontend code; adding a warning kind
is a pure backend change.
cargo check/clippy/fmt + npm run build green. Closes#1518.
Swarm-level companion to the per-agent favorite-tools doughnut (#1433).
Aggregates each agent's bash_commands(ts, head) table (written by
hive-bash-mcp) across the whole hive and surfaces the top-10 most-run
command heads on the dashboard ST4TS tab, alongside the existing model
mix.
- hive_stats.rs: AgentAgg gains a `bash` head→count map, filled by a new
guarded `read_bash_heads()` that reuses read_agent's read-only
connection. A missing `bash_commands` table (capture hasn't run for
that agent) or any read error yields an empty map — isolated from
read_agent's error path so it never drops an agent from the rollup.
HiveStats gains `bash_mix: Vec<KeyCount>` (busiest-first, top 10).
Unit tests cover the per-head tally + window cutoff and the
missing-table degrade-to-empty path (in-memory sqlite).
- dashboard: a "favorite tools (bash commands across the swarm)" CSS-bar
list on the ST4TS pane, mirroring the model-mix bars. Header + list
stay hidden until bash_mix has data, so a fresh hive shows no empty
block. (Dashboard ships no chart lib — bars, not a doughnut.)
- docs: dashboard.md ST4TS section documents the new rollup.
Closes#1449. Inert until the hive-bash-mcp capture (#1448, merged) has
recorded data across agents.
Per operator request on the PR: the built-in/default prices were the old
Claude 3 numbers (opus 15/75, etc.). Update opus + haiku to the current
Claude 4.x family list pricing (cache_write = the default 5-minute cache
TTL); sonnet was already correct:
- opus: input 5, output 25, cache_read 0.5, cache_write 6.25
- sonnet: input 3, output 15, cache_read 0.3, cache_write 3.75 (unchanged)
- haiku: input 1, output 5, cache_read 0.1, cache_write 1.25
Updated in both builtin_prices (hive_stats.rs) and the nix modelPrices
default (hive-c0re.nix), with cross-reference "keep in sync" notes on
both sides. Also addresses the earlier reviewer note: dropped the
over-strong "single source of truth" wording in the --model-prices arg
doc (the nix default does mirror the numbers in production).
- modelPrices submodule fields use lib.types.numbers.nonnegative
instead of lib.types.float: accepts bare ints (15) as well as floats
(15.0) and rejects negative prices for free.
- Collapse the triple-sourced default: hive-c0re serve --model-prices
now defaults to "{}" so builtin_prices() is the single in-code
fallback. The nix option default still carries the full
opus/sonnet/haiku table to self-document prices for operators.
The hive-wide cost estimate on the dashboard's ST4TS tab used a
hard-coded model->price table in hive_stats.rs. Anthropic list pricing
drifts, so move the table to a nix option operators can keep current
without a code change.
- New `services.hyperhive.modelPrices` option: attrset of model-family
short name -> { input, output, cache_read, cache_write } USD per
million tokens. Passed to `hive-c0re serve --model-prices <json>`.
- hive_stats: `Prices` is now public + Deserialize; add `PriceTable`
type and `resolve_prices` (longest case-insensitive substring key
wins) with the old hard-coded table preserved as `builtin_prices`
fallback for any model not covered.
- Coordinator holds the parsed table (hive-c0re-local, not injected
into containers, so not part of HiveEnv); `/api/stats-hive` reads it.
- Docs: dashboard.md ST4TS cost note updated; option self-documents
via nixosOptionsDoc.
Closes#1434
