Add GET /api/permissions/stale endpoint that returns agent names with
explicit capability/tool-group JSON entries but no live container AND
no kept-state tombstone. Ghost detection is now entirely server-side —
one authoritative call, no client-side roster cache, no staleness window.
The previous client-side approach in core.js made three parallel API
calls (GET /api/capabilities, GET /api/tool-groups, GET /api/state) and
filtered the result against a module-level `liveContainerNames` Set
populated only on cold load and form submits. Any container lifecycle
event (spawn, destroy) while core.html was open left `liveContainerNames`
stale, risking a false-positive ghost entry for a live container.
Changes:
- permissions.rs: add `get_stale_permissions` handler + `StalePermsResponse`
struct. Computes live roster (containers_snapshot), tombstone set
(Coordinator::kept_state_names), explicit perm names (capabilities::read
+ tool_groups::read), then returns the difference sorted.
- dashboard.rs: register GET /api/permissions/stale.
- core.js: replace the three-call client-side logic in
`fetchAndRenderStalePerms` with a single fetch to /api/permissions/stale.
Remove `liveContainerNames` state + its syncFromSnapshot population.
Adds pause/resume support for scheduled prompts.
Backend:
- New paused_at_unix column on scheduled_prompts table (added via
ALTER TABLE migration so existing databases are upgraded on first
start). The due-rows index is dropped and recreated to also exclude
paused rows so the worker never fires them while paused.
- Worker's due() query gains AND paused_at_unix IS NULL filter.
- New pause(id) and resume(id) methods on ScheduledPrompts; both are
idempotent and refuse cancelled rows.
- New POST /api/schedules/{id}/pause and /api/schedules/{id}/resume
dashboard endpoints (operator-direct, no approval gate). Both emit
a schedules snapshot on success so the tab updates live.
- WireSchedule gains paused_at_unix: Option<i64> so the frontend can
render the state without an extra fetch.
Frontend:
- Paused rows render with a distinct row class + muted opacity.
- The next-fire cell shows a yellow pause glyph + tooltip with the
paused-since timestamp and the would-have-fired time.
- Actions column: pause/resume toggle button (⏸/▶) beside fire/edit/cancel.
Fire-now is disabled while paused (resume first).
- Sort order: active → paused → cancelled (paused slot keeps schedules
visible without mixing them into the active top section).
- pauseSchedule() / resumeSchedule() async functions POST to the new
endpoints and refresh the table on success.
The P3RM1SS10NS tab showed agents that no longer exist in the live
container roster — e.g. an agent named 'root' that was renamed or
destroyed but still had explicit entries in tool-groups.json and/or
capabilities.json. The roster-union behaviour is intentional for
temporarily-stopped agents, but stale entries from renamed/destroyed
agents are confusing.
Backend (dashboard/permissions.rs):
- New DELETE /api/permissions/{agent} handler that bypasses the live-
roster guard (intentionally — that's the point). Calls
tool_groups::remove_agent + capabilities::remove_agent to clear both
JSON files, then emits live SSE snapshots so the tab updates without
a page reload. Format-checks the agent name but does not require it to
be in the containers snapshot.
Frontend (permissions.js):
- renderCapabilities / renderToolGroups now cross-reference agentNames
against containersState (the live roster, already imported). Agents
not in the live roster get an isStale flag.
- Stale rows get a '(not running)' label and a '✕ remove' button that
calls clearStaleAgent() — a new async helper that DELETEs the stale
entry and re-fetches both perm tables.
- Non-stale agents without explicit assignments still get '(default)'.
CSS (dashboard.css):
- .perm-row-stale (reduced opacity), .perm-stale-label (muted small
text), .perm-remove-btn (small red-bordered button) + disabled state.
Both kinds fell through to the spawn branch in renderApprovals, showing
a misleading 'spawn' chip and agent-spawn body text. Mara saw a meta-input
bump render as a spawn card for agent damocles and denied it.
Backend (dashboard.rs):
- Add commit_ref: None to the MergeConfigPr arm (struct was incomplete).
All arms of ApprovalView now initialise every field.
Frontend (call.js):
- Add isUpdateMeta / isSchedule booleans alongside the existing kind flags.
- Glyph: update_meta_inputs gets ↻, schedule_prompt gets ⏱.
- Kind chip: 'meta-update' / 'schedule' (no kind-spawn class for either).
- Body: update_meta_inputs parses commit_ref as JSON Vec<String> and shows
'bump flake inputs: foo, bar' or 'bump all flake inputs'; schedule_prompt
parses SchedulePromptPayload and shows targets + first-fire time + cadence
+ a truncated body excerpt.
- History row: add 'meta-update' and 'schedule' cases (were both 'spawn').
- Import fmtDuration from util.js (needed for schedule cadence display).
The per-agent and manager sockets ran two parallel dispatchers with
duplicated lifecycle handlers (agent-side topology-gated, manager-side
ungated) plus a manager-only handler set. Collapse to one parameterized
server in socket_server.rs:
- one serve() + dispatch(req, agent, privileged, coord); start() binds
the per-agent sockets (privileged=false), start_manager() binds the
manager socket (privileged=true).
- each lifecycle/config handler (start/restart/kill/update/init_config/
apply_commit) merges its dual: the topology guard (require_child /
require_new_child) runs only on the !privileged path; init_config
records the requester as parent only when !privileged. restart keeps
the orthogonal, capability-gated + audited infra-container branch.
- the agent-state queries (loose-ends / reminder count + rollup) branch
on privileged: privileged keeps any-target + the "*" hive-wide sweep
(query_agent_state-gated), non-privileged keeps the topology/cap gate.
- the privileged-only verbs (schedules / meta-inputs / get_logs) plus
the submit/schedule/watchdog helpers move into socket_server; they are
reached via dispatch_privileged_only(), which rejects the whole group
on a non-privileged socket.
- delete manager_server.rs; repoint refs; merge the test modules.
No behavior change: the topology guard still applies on every
non-privileged lifecycle call, the privileged socket still acts on any
agent, and privileged-only verbs are still rejected on agent sockets.
nginx proxied `<hive>/` straight to hive-c0re:7000, and hive-c0re served the
dashboard dist itself via `tower_http::ServeDir` (from `HIVE_STATIC_DIR` baked
into its service env). So a frontend-only change rebuilt the hive-c0re unit and
restarted the core daemon — every operator session dropped its SSE stream for a
pure CSS/JS change.
The gateway nginx now static-serves the dashboard dist directly; hive-c0re's
dashboard router is API-only. The split uses the Accept-header SPA fallback (the
same `map $http_accept` pattern the matrix/agent vhosts already use), so no
backend prefix has to be enumerated: a browser navigation (Accept: text/html)
whose path is not an on-disk asset gets the SPA index.html; everything else
(every /api route, the bare action/mutation routes, the two SSE streams, the
knowledge webhook — all Accept != text/html) falls through `try_files` to the
`@c0re` named location and is reverse-proxied to hive-c0re. A new c0re route
needs no gateway change.
- hive-c0re.nix: expose the themed dist as a new internal read-only option
`services.hyperhive.c0re.servedFrontend`; drop `HIVE_STATIC_DIR` from the
service env (the router no longer serves files).
- hive-gateway.nix: read that option in host-module scope (dashboardDist),
static-serve `dashboard/` with the Accept-header `try_files ... @c0re` split;
`@c0re` carries `proxy_buffering off` + a 1d read timeout for the SSE streams
and a duplicated auth_basic block (named locations do not inherit it). The
dashboard map is unconditional; the matrix map stays gated on the matrix GUI.
- dashboard.rs: drop the ServeDir fallback + the HIVE_STATIC_DIR resolution; the
router 404s unmatched paths (the gateway only proxies non-static requests).
- hive-c0re/Cargo.toml: drop the now-unused tower-http dependency.
- docs/gateway.md: document the dashboard static split + the `@c0re` fall-through.
The store path is reachable inside the gateway nspawn container (shared
/nix/store), mirroring how HIVE_AGENT_FRONTEND_DIR already exposes the per-agent
UIs. The gateway and c0re changes must land together (atomic cutover) or the
dashboard 404s — this needs a watched gateway + c0re rebuild.
Per iris's dashboard-side ask: return { entries, total } instead of a
bare array so the UI can show 'latest 500 of N' rather than silently
capping at the clamp. Adds AuditLog::count_total().
Adds a durable, operator-visible audit trail of privileged operations
hive-c0re performs on behalf of an agent — the ones that cross the
agent/operator trust boundary. First entry: infra-container restarts via
the infra_admin-gated `restart` tool, which until now were recorded only
as a hive-priv journal trace.
Backend:
- new `audit_log` module: sqlite-backed store (audit_log.sqlite, same dir
as build_logs) with schema (ts/agent/action/target/outcome/detail),
best-effort `record`, `list_recent` (clamped 500), 90-day `vacuum`, and
a process-singleton handle mirroring build_logs.
- Coordinator opens + installs the handle; main spawns the hourly vacuum.
- agent_server::handle_restart_infra records every attempt (ok, error, and
capability-denied) via the global handle — best-effort, never fails the
underlying action.
- dashboard: `GET /api/audit-log` returns recent entries as JSON.
Scope is deliberately agent-initiated privileged actions only (not every
PrivRequest — token writes + nspawn edits are constant lifecycle noise).
Extensible: future agent-initiated priv ops record via the same handle.
Unit tests cover record/list ordering, the 500 clamp, and retention vacuum.
The dashboard *surface* (an AUDIT view consuming /api/audit-log) is a
frontend follow-up coordinated with iris.
Clean the legacy backlog so the tracker-tag lint can become a required
gate (mara's warn-during-cleanup -> full-tree-enforcement path). Rewrite
the ~33 real `closes/see #NNN` provenance refs in doc-comments to prose
across hive-forge, hive-c0re, hive-ag3nt, hive-matrix-mcp, hive-sh4re,
and add a `lint:allow` line marker to check-issue-refs.sh for genuine
non-tracker `#<digits>` (a hash-digit heading-detection test input).
Tree is now lint-clean; tracker-tag lint ready to promote to required.
The gateway container starts alongside every hyperhive deployment, so
gating it behind a separate enable flag was a footgun: an operator who
set it false lost the only thing exposed to the outside while the
agent containers kept running. Re-gate the gateway config on the
top-level services.hyperhive.enable instead.
- hive-gateway.nix: drop the gateway.enable mkOption; gate the config
block on config.services.hyperhive.enable.
- hive-forge.nix: behindGateway now defaults to services.hyperhive.enable;
remove the behindGateway-requires-gateway assertion (now vacuous).
- hive-network.nix: remove both gateway.enable assertions (vacuous).
- hive-c0re.nix: drop the firewall.allowedTCPPortRanges 8100-8999
fallback that opened agent ports when the gateway was off (the
gateway is now the sole entry point); HIVE_GATEWAY_ENABLED is always
set since the gateway always runs.
- nix/docs/default.nix: remove the gateway.enable = mkForce false stub
(would be an eval error against the removed option; the gateway is
already re-gated on hyperhive.enable, which docs force false).
- hive-matrix.nix, dashboard.rs: comment/prose updates only.
BREAKING: operators relying on services.hyperhive.gateway.enable = false
to suppress the gateway must instead point their own reverse proxy at
the gateway's port. NixOS errors clearly on the now-unknown option.
Per operator request: instead of a disk-specific alert, surface a generic
server-warnings banner at the very top of every page, so new system
warnings can be added backend-side with no frontend change.
- hive-c0re `host_stats`: `server_warnings() -> Vec<ServerWarning>`
(`{ kind, level, message }`). The threshold logic lives server-side; the
host disk-pressure check (a `statvfs` probe of `/nix`: ≥85% used → warn,
≥95% → crit) is the first and only producer today. No new deps (libc).
- `/api/state` carries `server_warnings` (replaces the disk-specific
field). Empty when all clear.
- frontend: `renderServerWarnings` / `initServerWarnings` in `common.js`
inject a sticky top-of-<body> banner and render the list, coloured by
`level`. Wired on every page — dashboard (live, via refreshState),
FL0W, L0GS, H0M3. No per-warning frontend code; adding a warning kind
is a pure backend change.
cargo check/clippy/fmt + npm run build green. Closes#1518.
The hive-wide cost estimate on the dashboard's ST4TS tab used a
hard-coded model->price table in hive_stats.rs. Anthropic list pricing
drifts, so move the table to a nix option operators can keep current
without a code change.
- New `services.hyperhive.modelPrices` option: attrset of model-family
short name -> { input, output, cache_read, cache_write } USD per
million tokens. Passed to `hive-c0re serve --model-prices <json>`.
- hive_stats: `Prices` is now public + Deserialize; add `PriceTable`
type and `resolve_prices` (longest case-insensitive substring key
wins) with the old hard-coded table preserved as `builtin_prices`
fallback for any model not covered.
- Coordinator holds the parsed table (hive-c0re-local, not injected
into containers, so not part of HiveEnv); `/api/stats-hive` reads it.
- Docs: dashboard.md ST4TS cost note updated; option self-documents
via nixosOptionsDoc.
Closes#1434
The operator-inbox handler landed with an unformatted long match line
(`match state.coord.broker.unread_for_recipient(...)`) that rustfmt
wants wrapped. It was force-merged during the CI outage so the formatting
gate didn't catch it, and now the treefmt check fails for every PR based
on current main. Pure formatting, no logic change.
Agents that `send(to: "operator")` were easy to miss — they only
surfaced on the FL0W firehose with no read-state (#1469). Surface them
on the Y3R C4LL ("things waiting on you") tab as a proper inbox.
Backend:
- broker: `unread_for_recipient(recipient, limit)` — unacked messages
for a recipient, newest-first. Mirrors `mark_all_read`'s filter
EXACTLY (`recipient = ?1 AND acked_at IS NULL`, no `delivered_at`
condition) so everything listed is exactly what mark-read clears —
operator rows never get `delivered_at` set (no agent-socket recv).
- dashboard: `GET /api/operator-inbox` → `{ messages: [...] }` (id,
from, body, at, in_reply_to, validated file_refs). Mark-read reuses
the existing `POST /api/agent/operator/mark-all-read` (the route
format-validates the name; "operator" passes; `mark_all_read`
already acks `to="operator"` rows).
Frontend (Y3R C4LL):
- New ◆ 1NB0X ◆ section listing unread messages (sender · time · body,
path-linkified) + a "✓ mark all read" button.
- Cold-loaded on page load + on tab activation; appended live from the
broker `sent` stream (deduped on row id); cleared on mark-all-read.
- Unread count folds into the Y3R C4LL tab pill + the browser-title
`(N)` prefix, so messages are visible from any tab.
Removing the now-redundant FL0W operator-inbox UI is a clean follow-up
(deferred to avoid a flow.js conflict with the in-flight #1473).
Backend (broker + route) is host-side — @damocles to review per plan.
Closes#1469.
C0NT41N3R L04D on the SYST3M tab + GET /api/container-resources.
Backend (hive-c0re/src/container_stats.rs): reads cgroup v2 cpu.stat +
memory.{current,peak,max} for each running agent machine
(machine-h\x2d<name>.scope under machine.slice), read-only/world-
readable so no hive-priv. CPU is a two-sample (~200ms) host-normalised
percentage; one shared sleep covers all agents. Skips agents whose
scope dir is absent (= not running). Network omitted: agents share the
host netns, no per-container counter.
Frontend: a polled C0NT41N3R L04D table on SYST3M (agent / cpu / mem /
peak / limit with meter bars), reusing the ST4TS table style. Polls
/api/container-resources every 5s only while the tab is active.
Backend reviewed-in-principle by damocles (path escaping + cpu delta
math); ping for the on-host sign-off.
#1424 P2 backend. New hive_stats module aggregates every agent's
turn-stats.sqlite read-only (reusing Coordinator::kept_state_names +
agent_harness_dir, skipping missing/unreadable dbs) into swarm totals,
a busiest-first per-agent rollup, swarm model mix, and a labelled USD
cost estimate (rough model->price table; can move to a nix option
later). Exposed as GET /api/stats-hive?window=. Dashboard UI follows.
The nixpkgs bump to clippy 0.1.95 / cargo 1.95.0 added + strengthened a
large batch of lints. CI denied ALL warnings (`-D warnings`) against the
`pedantic = warn` workspace lint, so the bump hard-failed `nix flake
check` workspace-wide with zero code changes — and would recur on every
future clippy bump.
Posture fix (the durable part): CI now runs
`-D warnings -A clippy::pedantic`, so the default/correctness/style lints
stay a hard gate while the "extra, opinionated" pedantic group is
advisory only (still `warn` for local `cargo clippy` via the workspace
lints table, just non-blocking in CI). `-A` rather than `-W` so the
group drop doesn't re-enable the specific pedantic lints the workspace
allows (e.g. `must_use_candidate`).
Also fixes the genuine DEFAULT/STYLE lints the bump surfaced across the
workspace (doc_lazy_continuation, collapsible_if, ptr_arg,
match_like_matches_macro, …) via `cargo clippy --fix` + manual stragglers
(`too_many_arguments` #[allow] on the host-config constructors), and
three tests that had rotted while the CI runner was offline (#1221):
- topology::top_level_agents_in_multi_root — hardcoded unsorted expected
- rebuild_queue::depends_on_evicted_dep_counts_as_resolved — needs
MAX_HISTORY_PER_KIND newer terminals to evict, not one
- coordinator::agent_paths doctest — illustrative pseudo-code, now `ignore`
Validated: clippy + formatting + cargo-test checks all pass.
Add RemindersChanged SSE event so the pending-reminders list in the
SYST3M tab updates live without polling.
Backend emission sites (every path that mutates the reminders table):
- agent_server: store_remind (remind MCP call)
- dashboard.rs: post_cancel_reminder, post_retry_reminder
- questions.rs: cancel_loose_end Reminder kind
- reminder_scheduler: after each delivery batch (any_delivered)
Coordinator gets emit_reminders_snapshot() mirroring the existing
emit_schedules_snapshot() pattern: lists PendingReminder rows from the
broker and emits DashboardEvent::RemindersChanged.
Frontend: applyRemindersChanged(ev) calls renderReminders(ev.reminders)
and is registered as reminders_changed in MUTATION_HANDLERS.
Docs: dashboard.md reminders_changed entry; CLAUDE.md file map updated.
