jobs are now DAGs of primitive nodes (prebuild, stop-for-update, swap,
reconcile, signal, drain, ...) driven by one scheduler with N build
slots + per-agent lifecycle leases. per-agent power intent (wanted
up/offline) is durable in agent_power.sqlite; Reconcile nodes converge
observed state to it. kills the graceful-stop watcher thread, the
deferred-start follow-up, and the cascade pre-enqueue (fan-out on
MetaLock completion instead). tracker: #2166
Add GET /api/permissions/stale endpoint that returns agent names with
explicit capability/tool-group JSON entries but no live container AND
no kept-state tombstone. Ghost detection is now entirely server-side —
one authoritative call, no client-side roster cache, no staleness window.
The previous client-side approach in core.js made three parallel API
calls (GET /api/capabilities, GET /api/tool-groups, GET /api/state) and
filtered the result against a module-level `liveContainerNames` Set
populated only on cold load and form submits. Any container lifecycle
event (spawn, destroy) while core.html was open left `liveContainerNames`
stale, risking a false-positive ghost entry for a live container.
Changes:
- permissions.rs: add `get_stale_permissions` handler + `StalePermsResponse`
struct. Computes live roster (containers_snapshot), tombstone set
(Coordinator::kept_state_names), explicit perm names (capabilities::read
+ tool_groups::read), then returns the difference sorted.
- dashboard.rs: register GET /api/permissions/stale.
- core.js: replace the three-call client-side logic in
`fetchAndRenderStalePerms` with a single fetch to /api/permissions/stale.
Remove `liveContainerNames` state + its syncFromSnapshot population.
Adds pause/resume support for scheduled prompts.
Backend:
- New paused_at_unix column on scheduled_prompts table (added via
ALTER TABLE migration so existing databases are upgraded on first
start). The due-rows index is dropped and recreated to also exclude
paused rows so the worker never fires them while paused.
- Worker's due() query gains AND paused_at_unix IS NULL filter.
- New pause(id) and resume(id) methods on ScheduledPrompts; both are
idempotent and refuse cancelled rows.
- New POST /api/schedules/{id}/pause and /api/schedules/{id}/resume
dashboard endpoints (operator-direct, no approval gate). Both emit
a schedules snapshot on success so the tab updates live.
- WireSchedule gains paused_at_unix: Option<i64> so the frontend can
render the state without an extra fetch.
Frontend:
- Paused rows render with a distinct row class + muted opacity.
- The next-fire cell shows a yellow pause glyph + tooltip with the
paused-since timestamp and the would-have-fired time.
- Actions column: pause/resume toggle button (⏸/▶) beside fire/edit/cancel.
Fire-now is disabled while paused (resume first).
- Sort order: active → paused → cancelled (paused slot keeps schedules
visible without mixing them into the active top section).
- pauseSchedule() / resumeSchedule() async functions POST to the new
endpoints and refresh the table on success.
The P3RM1SS10NS tab showed agents that no longer exist in the live
container roster — e.g. an agent named 'root' that was renamed or
destroyed but still had explicit entries in tool-groups.json and/or
capabilities.json. The roster-union behaviour is intentional for
temporarily-stopped agents, but stale entries from renamed/destroyed
agents are confusing.
Backend (dashboard/permissions.rs):
- New DELETE /api/permissions/{agent} handler that bypasses the live-
roster guard (intentionally — that's the point). Calls
tool_groups::remove_agent + capabilities::remove_agent to clear both
JSON files, then emits live SSE snapshots so the tab updates without
a page reload. Format-checks the agent name but does not require it to
be in the containers snapshot.
Frontend (permissions.js):
- renderCapabilities / renderToolGroups now cross-reference agentNames
against containersState (the live roster, already imported). Agents
not in the live roster get an isStale flag.
- Stale rows get a '(not running)' label and a '✕ remove' button that
calls clearStaleAgent() — a new async helper that DELETEs the stale
entry and re-fetches both perm tables.
- Non-stale agents without explicit assignments still get '(default)'.
CSS (dashboard.css):
- .perm-row-stale (reduced opacity), .perm-stale-label (muted small
text), .perm-remove-btn (small red-bordered button) + disabled state.
Both kinds fell through to the spawn branch in renderApprovals, showing
a misleading 'spawn' chip and agent-spawn body text. Mara saw a meta-input
bump render as a spawn card for agent damocles and denied it.
Backend (dashboard.rs):
- Add commit_ref: None to the MergeConfigPr arm (struct was incomplete).
All arms of ApprovalView now initialise every field.
Frontend (call.js):
- Add isUpdateMeta / isSchedule booleans alongside the existing kind flags.
- Glyph: update_meta_inputs gets ↻, schedule_prompt gets ⏱.
- Kind chip: 'meta-update' / 'schedule' (no kind-spawn class for either).
- Body: update_meta_inputs parses commit_ref as JSON Vec<String> and shows
'bump flake inputs: foo, bar' or 'bump all flake inputs'; schedule_prompt
parses SchedulePromptPayload and shows targets + first-fire time + cadence
+ a truncated body excerpt.
- History row: add 'meta-update' and 'schedule' cases (were both 'spawn').
- Import fmtDuration from util.js (needed for schedule cadence display).
The per-agent and manager sockets ran two parallel dispatchers with
duplicated lifecycle handlers (agent-side topology-gated, manager-side
ungated) plus a manager-only handler set. Collapse to one parameterized
server in socket_server.rs:
- one serve() + dispatch(req, agent, privileged, coord); start() binds
the per-agent sockets (privileged=false), start_manager() binds the
manager socket (privileged=true).
- each lifecycle/config handler (start/restart/kill/update/init_config/
apply_commit) merges its dual: the topology guard (require_child /
require_new_child) runs only on the !privileged path; init_config
records the requester as parent only when !privileged. restart keeps
the orthogonal, capability-gated + audited infra-container branch.
- the agent-state queries (loose-ends / reminder count + rollup) branch
on privileged: privileged keeps any-target + the "*" hive-wide sweep
(query_agent_state-gated), non-privileged keeps the topology/cap gate.
- the privileged-only verbs (schedules / meta-inputs / get_logs) plus
the submit/schedule/watchdog helpers move into socket_server; they are
reached via dispatch_privileged_only(), which rejects the whole group
on a non-privileged socket.
- delete manager_server.rs; repoint refs; merge the test modules.
No behavior change: the topology guard still applies on every
non-privileged lifecycle call, the privileged socket still acts on any
agent, and privileged-only verbs are still rejected on agent sockets.
nginx proxied `<hive>/` straight to hive-c0re:7000, and hive-c0re served the
dashboard dist itself via `tower_http::ServeDir` (from `HIVE_STATIC_DIR` baked
into its service env). So a frontend-only change rebuilt the hive-c0re unit and
restarted the core daemon — every operator session dropped its SSE stream for a
pure CSS/JS change.
The gateway nginx now static-serves the dashboard dist directly; hive-c0re's
dashboard router is API-only. The split uses the Accept-header SPA fallback (the
same `map $http_accept` pattern the matrix/agent vhosts already use), so no
backend prefix has to be enumerated: a browser navigation (Accept: text/html)
whose path is not an on-disk asset gets the SPA index.html; everything else
(every /api route, the bare action/mutation routes, the two SSE streams, the
knowledge webhook — all Accept != text/html) falls through `try_files` to the
`@c0re` named location and is reverse-proxied to hive-c0re. A new c0re route
needs no gateway change.
- hive-c0re.nix: expose the themed dist as a new internal read-only option
`services.hyperhive.c0re.servedFrontend`; drop `HIVE_STATIC_DIR` from the
service env (the router no longer serves files).
- hive-gateway.nix: read that option in host-module scope (dashboardDist),
static-serve `dashboard/` with the Accept-header `try_files ... @c0re` split;
`@c0re` carries `proxy_buffering off` + a 1d read timeout for the SSE streams
and a duplicated auth_basic block (named locations do not inherit it). The
dashboard map is unconditional; the matrix map stays gated on the matrix GUI.
- dashboard.rs: drop the ServeDir fallback + the HIVE_STATIC_DIR resolution; the
router 404s unmatched paths (the gateway only proxies non-static requests).
- hive-c0re/Cargo.toml: drop the now-unused tower-http dependency.
- docs/gateway.md: document the dashboard static split + the `@c0re` fall-through.
The store path is reachable inside the gateway nspawn container (shared
/nix/store), mirroring how HIVE_AGENT_FRONTEND_DIR already exposes the per-agent
UIs. The gateway and c0re changes must land together (atomic cutover) or the
dashboard 404s — this needs a watched gateway + c0re rebuild.
Per iris's dashboard-side ask: return { entries, total } instead of a
bare array so the UI can show 'latest 500 of N' rather than silently
capping at the clamp. Adds AuditLog::count_total().
Adds a durable, operator-visible audit trail of privileged operations
hive-c0re performs on behalf of an agent — the ones that cross the
agent/operator trust boundary. First entry: infra-container restarts via
the infra_admin-gated `restart` tool, which until now were recorded only
as a hive-priv journal trace.
Backend:
- new `audit_log` module: sqlite-backed store (audit_log.sqlite, same dir
as build_logs) with schema (ts/agent/action/target/outcome/detail),
best-effort `record`, `list_recent` (clamped 500), 90-day `vacuum`, and
a process-singleton handle mirroring build_logs.
- Coordinator opens + installs the handle; main spawns the hourly vacuum.
- agent_server::handle_restart_infra records every attempt (ok, error, and
capability-denied) via the global handle — best-effort, never fails the
underlying action.
- dashboard: `GET /api/audit-log` returns recent entries as JSON.
Scope is deliberately agent-initiated privileged actions only (not every
PrivRequest — token writes + nspawn edits are constant lifecycle noise).
Extensible: future agent-initiated priv ops record via the same handle.
Unit tests cover record/list ordering, the 500 clamp, and retention vacuum.
The dashboard *surface* (an AUDIT view consuming /api/audit-log) is a
frontend follow-up coordinated with iris.
Clean the legacy backlog so the tracker-tag lint can become a required
gate (mara's warn-during-cleanup -> full-tree-enforcement path). Rewrite
the ~33 real `closes/see #NNN` provenance refs in doc-comments to prose
across hive-forge, hive-c0re, hive-ag3nt, hive-matrix-mcp, hive-sh4re,
and add a `lint:allow` line marker to check-issue-refs.sh for genuine
non-tracker `#<digits>` (a hash-digit heading-detection test input).
Tree is now lint-clean; tracker-tag lint ready to promote to required.
The gateway container starts alongside every hyperhive deployment, so
gating it behind a separate enable flag was a footgun: an operator who
set it false lost the only thing exposed to the outside while the
agent containers kept running. Re-gate the gateway config on the
top-level services.hyperhive.enable instead.
- hive-gateway.nix: drop the gateway.enable mkOption; gate the config
block on config.services.hyperhive.enable.
- hive-forge.nix: behindGateway now defaults to services.hyperhive.enable;
remove the behindGateway-requires-gateway assertion (now vacuous).
- hive-network.nix: remove both gateway.enable assertions (vacuous).
- hive-c0re.nix: drop the firewall.allowedTCPPortRanges 8100-8999
fallback that opened agent ports when the gateway was off (the
gateway is now the sole entry point); HIVE_GATEWAY_ENABLED is always
set since the gateway always runs.
- nix/docs/default.nix: remove the gateway.enable = mkForce false stub
(would be an eval error against the removed option; the gateway is
already re-gated on hyperhive.enable, which docs force false).
- hive-matrix.nix, dashboard.rs: comment/prose updates only.
BREAKING: operators relying on services.hyperhive.gateway.enable = false
to suppress the gateway must instead point their own reverse proxy at
the gateway's port. NixOS errors clearly on the now-unknown option.
