ensure_mirror_repo treated 409|422 as success (copied from ensure_org, where
422 *does* mean 'org exists'). For the migrate endpoint 422 is a validation
error (bad clone_addr/service); the GET-first check is the real idempotency
guard, so 409 stays as a race guard but 422 now falls through to the bail arm
(→ caller warns) instead of silently dropping a misconfigured mirror.
Per mara: the mirror seeding belongs in hive-c0re's forge provisioning
sweep, where the core admin token + org-ensure already live — not a parallel
host-side nix oneshot.
- forge.rs: ensure_mirrors() reads HYPERHIVE_FORGE_MIRRORS (JSON list of
{upstream,dest}), ensures each dest org (reuse ensure_org) + creates the
pull-mirror via the migrate API (reuse forge_http, serde_json::json! body,
409/existing = success). Called in ensure_all() right after the SEEDED_ORGS
loop (token in scope, warn-and-continue like the other ensure_* steps).
- hive-forge.nix: forward effectiveMirrors to c0re via
systemd.services.hive-c0re.environment.HYPERHIVE_FORGE_MIRRORS; drop the
forgejo-seed-mirrors.service + its script + the host-side core-token read.
Keep the forge.mirrors option, DEFAULT_ACTIONS_URL=self (CI-gated), and the
dest-shape / no-c0re-namespace-collision assertions.
Verified locally: nix parse + treefmt (incl rustfmt) clean; serde/serde_json
patterns mirror dashboard.rs. cargo build runs in CI (no cc in my container).
nixfmt wanted the ${...} on a single line (verified locally:
nix build .#checks.x86_64-linux.formatting passes). The earlier CI 'docs'
failure was a transient muede-pc2 build hiccup — the docs check builds clean
locally on the same drv.
- drop the three cross-ref tracker tags from comments/description (prose only)
- build the orgs + migrate JSON bodies with jq -n --arg (an upstream URL
containing a quote no longer corrupts the request)
- don't auto-append the actions/checkout mirror when the operator already
declared that dest (avoids a duplicate effectiveMirrors entry when CI is on)
General-purpose mirror mechanism for the internal forge, per mara's call on
#2074 (real Forgejo pull-mirrors, nix-configured — not a pushed clone).
- services.hyperhive.forge.mirrors: list of { upstream, dest } pull-mirrors,
any repo. Each is created as a real Forgejo pull-mirror (re-syncs from
upstream), dest = <owner>/<repo> in its own org.
- When forge.ci.enable is set: an actions/checkout mirror is auto-appended +
forgejo DEFAULT_ACTIONS_URL is pointed at this instance, so CI
'uses: actions/checkout@vN' resolves on loopback — immune to a host-resolver
blip that previously reded every checkout (the seed/re-sync needs external
DNS, but that's off the CI critical path).
- forgejo-seed-mirrors.service: host-side oneshot (the core admin token never
enters a container), modelled on hive-ci-prefetch — waits <=60s for the core
token, then idempotently ensures each dest org + creates the pull-mirror via
the migrate API. partOf the forge container so it re-ensures on restart.
- assertions: dest must be <owner>/<repo>; mirror orgs can't shadow the
c0re-managed namespaces (config/shared/agents/core) so the seed never races
hive-c0re's own provisioning.
Supersedes #2074 (the raw-clone stopgap) as the durable #2072 fix.
General-purpose Forgejo pull-mirror config (services.hyperhive.forge.mirrors:
list of {upstream, dest}). When CI is enabled, auto-append an actions/checkout
mirror + point forgejo DEFAULT_ACTIONS_URL at this instance so CI's
actions/checkout@vN resolves on loopback (immune to host-resolver blips, #2072).
Seed oneshot (creates the dest orgs + pull-mirrors via the migrate API) is the
next commit.
Remove or shorten comment blocks that:
- explain where code used to live (impl history)
- duplicate rationale already in docs/web-ui.md
- contain speculative/future-work notes
Changes:
- tabs.js: drop "SYST3M tab used to render this" history; keep badge purpose
- schedules.js: drop "used to be a card layout" history + speculative nit note
- stream-worker.js: replace 10-line problem-statement with docs pointer
(rationale already in docs/web-ui.md SSE multiplexing section); trim
future-work parenthetical from subscription-tracking comment
- system-sections.css: drop extraction-history prose; keep what it does
- core.css: drop move-history prose; keep what it imports and why
Part of issue cleanup per operator feedback.
New standalone page /builds.html consolidating rebuild queue, meta
inputs, and build log history into one place, with a new 'Builds' home
tile linking to it. Addresses mara's request: new sub-page with a new
home tile, all three items moved there.
Changes:
- builds.html: new page with three sub-tabs: R3BU1LD QU3U3, M3T4 1NPUTS,
BUILD L0GS. Same minimal-chrome header + createTabStrip pattern as core
and logs pages.
- builds.js: new bundle combining rebuild queue renderer (from core.js),
meta inputs renderer (from core.js), rebuild-live-log renderer (from
core.js), and build log history renderer (from logs.js). Deep-links to
/builds.html?id=N#buildlogs. Count pill id: builds-tab-count-rebuild.
BUILD L0GS tab lazy-loads on first activation.
- builds.css: @imports system-sections.css (rebuild queue + meta inputs +
live-log styles) and logs.css (build-logs-* component styles).
- build.mjs: register builds.js, builds.css, builds.html.
- core.html: remove R3BU1LD QU3U3 + M3T4 1NPUTS tabs (now on builds.html).
Default tab changes to K3PT ST4T3.
- core.js: remove renderMetaInputs, renderRebuildQueue + helpers,
renderRebuildLiveLog + live-log state, elapsed-time tickers,
updateRebuildCount, and the rebuild_queue/meta_inputs SSE handlers.
Remove openBuildLogStream + util imports no longer needed.
- core.css: remove .rebuild-live-log-* rules (moved to system-sections.css
so builds.css can share them via @import).
- system-sections.css: add .rebuild-live-log-* styles (moved from core.css);
update comment to mention builds.html.
- logs.html: remove BUILD tab + pane (moved to builds.html).
- logs.js: remove fetchBuild(), fmtTs, fmtDuration, openBuildLogStream
import, and rebuild_queue_changed SSE debounce. Default tab: 'agent'.
SSE stream retained for audit_entry_added live-appends.
- index.html: add Builds tile (🔨, rebuild queue · meta inputs · build
logs); update Core tile desc to 'kept state · container load'; update
Logs tile desc to remove 'build'.
The variable was misleadingly named `tombstones` and commented as
'any name here is a tombstone', but kept_state_names() returns ALL
agents with a state dir on disk — both live containers and soft-deleted
tombstones. The logic was always correct (union with the live roster
filters both), but the comment would mislead future readers.
Rename the variable to `kept` and update comments to accurately
describe the contents.
Add GET /api/permissions/stale endpoint that returns agent names with
explicit capability/tool-group JSON entries but no live container AND
no kept-state tombstone. Ghost detection is now entirely server-side —
one authoritative call, no client-side roster cache, no staleness window.
The previous client-side approach in core.js made three parallel API
calls (GET /api/capabilities, GET /api/tool-groups, GET /api/state) and
filtered the result against a module-level `liveContainerNames` Set
populated only on cold load and form submits. Any container lifecycle
event (spawn, destroy) while core.html was open left `liveContainerNames`
stale, risking a false-positive ghost entry for a live container.
Changes:
- permissions.rs: add `get_stale_permissions` handler + `StalePermsResponse`
struct. Computes live roster (containers_snapshot), tombstone set
(Coordinator::kept_state_names), explicit perm names (capabilities::read
+ tool_groups::read), then returns the difference sorted.
- dashboard.rs: register GET /api/permissions/stale.
- core.js: replace the three-call client-side logic in
`fetchAndRenderStalePerms` with a single fetch to /api/permissions/stale.
Remove `liveContainerNames` state + its syncFromSnapshot population.
Adds a "stale permission entries" sub-section within the K3PT ST4T3 pane
on /core.html showing agents with explicit capability/tool-group JSON
entries but no live container — typically renamed or manually-deleted agents
whose entries persisted (e.g. the old "root" manager name after rename to
"ruth").
Each ghost agent gets a "✕ clear perms" button that calls
DELETE /api/permissions/{agent} (added in the prior commit). Lazy-loaded
on first K3PT ST4T3 tab activation; auto-refreshes on capabilities_changed
and tool_groups_changed SSE events.
core.js: track liveContainerNames from /api/state.containers; add
renderStalePerms + fetchAndRenderStalePerms; hook tab onShow + SSE handlers.
system-sections.css: new .tombstones-stale-* selectors for the ghost list
rows and error message.
core.html: add #tombstones-stale-perms div inside the K3PT ST4T3 pane;
expand comment to describe both sub-sections.
Part of the #1802 hive-forge audit: the read verbs (view / issue / pr /
comments / comment-show / timeline / diff / pr-status / pr-reviews) look
overlapping but each has a distinct output shape. Add a selection table +
rule-of-thumb so agents pick the right one (esp. 'view first' to clear the
read-before-comment guard) instead of guessing. Zero blast radius — pure
docs, no verb changes (the audit concluded the CLI is structurally clean
and renames/removals aren't worth the caller breakage).
The field was originally backed by harness/hyperhive-model; after the
rework (fab6259d) it reads from state/hyperhive-harness.json. Update
the struct-level doc comment to match.
hive-c0re was reading harness/hyperhive-model directly to surface the
model badge on the dashboard. hyperhive-model is a runtime-override
file (not the resolved priority) and adds to the marker-file count.
Instead: mirror the fully-resolved model into hyperhive-harness.json
(the consolidated state file that already replaced hyperhive-rate-limited
/ hyperhive-needs-login). Written by hive-ag3nt on:
- Bus::new() startup (captures nix config > override > default)
- set_model() runtime change (MCP set-model call)
- emit_status() (keeps model current across rate-limit / auth flips)
hive-c0re reads active_model from hyperhive-harness.json, same dir +
same read path as rate_limited / needs_login. No new files.
Read the persisted model name from each agent's harness state file
(harness/hyperhive-model) and surface it as a small blue badge on
the container row in the SW4RM tab.
- container_view.rs: add `active_model: Option<String>` to
ContainerView; populated by new `read_active_model` helper that
reads harness/hyperhive-model; only set when container is running
(stale model info from a stopped agent is misleading)
- container_view.rs: add active_model to ContainerView literal in
host_stats test helper
- tabs.js: render badge-model chip after needs-update, before
reminders; add active_model to the row fingerprint so re-renders
fire on model change
- common.css: add .badge-model (blue, 80% opacity — informational)
Per mara: a secret in the nix store is not acceptable. The non-secret
OTEL config (telemetry-enable, endpoint, protocol, resource attributes)
stays in the world-readable managed settings json; the auth header is
handled separately at runtime so it never touches the store.
New hive-otel-header oneshot (only when otel.enable && headersCredential
is set): inherits the forwarded otel-headers systemd credential via
LoadCredential, reads it at start, and merges OTEL_EXPORTER_OTLP_HEADERS
into the agent's 0600 ~/.claude/settings.json env block via jq. claude
layers the user env on top of the managed settings, so both the harness
turn-loop and hivectl choom (same agent user) export with auth. The
token is read from disk at start and never copied into the nix store or
the world-readable managed file.
Ordering is best-effort (before=, not a hard dep): a failure leaves the
harness running and telemetry exporting unauthenticated. headersCredential
option description updated to reflect it's now wired.
nix fmt clean.
The option description still claimed the credential is loaded via
systemd LoadCredential, but this PR removed that path. Clarify that the
option is currently inert (only the unauthenticated OTEL export is
implemented) and that runtime header injection is a planned follow-up,
so configuring it doesn't silently no-op without explanation.
Per mara: configure OTEL in the generated claude settings json (what the
Claude Code docs suggest), not a launch wrapper or /etc shell file.
claude-code auto-discovers /etc/claude-code/managed-settings.json in
every context — the harness turn loop AND hivectl choom — so putting the
OTEL env there gives telemetry parity declaratively, with no wrapper and
no --settings plumbing.
- managed-settings.json: was a static shared .source; now, when OTEL is
enabled, a per-agent build-time jq merge of the base asset + an env
block (jq at build, not eval-time readFile, to avoid IFD). OTEL off =
the static asset verbatim.
- otelSettingsEnv carries the static OTEL knobs + OTEL_RESOURCE_ATTRIBUTES
with the agent name (build-time) and the hive/swarm names forwarded by
meta.rs into environment.variables (mara: forward host config into
agent config where needed).
- removed the hive-serve-otel ExecStart wrapper, the per-unit otelEnv,
and the otel-headers LoadCredential from the harness service — the
harness binary emits no OTEL itself; only claude does, and it now reads
the settings json directly.
Known follow-ups (noted in code): the auth header (otel.headersCredential,
opt-in/default-null) is a secret and can't live in the world-readable
settings file — authenticated collectors need a runtime mechanism; this
PR covers the unauthenticated default.
nix fmt clean.
Replace brittle msg.contains("not found") string matching in
post_schedule_pause / post_schedule_resume with a typed
ScheduleNotFoundOrCancelled error that handlers downcast on directly.
pause() and resume() now return Err(ScheduleNotFoundOrCancelled(id).into())
instead of bail!("schedule {id} not found or is cancelled"); handlers call
e.downcast_ref::<ScheduleNotFoundOrCancelled>().is_some() for the 404 branch,
making the discrimination stable even if the error message wording changes.
Adds pause/resume support for scheduled prompts.
Backend:
- New paused_at_unix column on scheduled_prompts table (added via
ALTER TABLE migration so existing databases are upgraded on first
start). The due-rows index is dropped and recreated to also exclude
paused rows so the worker never fires them while paused.
- Worker's due() query gains AND paused_at_unix IS NULL filter.
- New pause(id) and resume(id) methods on ScheduledPrompts; both are
idempotent and refuse cancelled rows.
- New POST /api/schedules/{id}/pause and /api/schedules/{id}/resume
dashboard endpoints (operator-direct, no approval gate). Both emit
a schedules snapshot on success so the tab updates live.
- WireSchedule gains paused_at_unix: Option<i64> so the frontend can
render the state without an extra fetch.
Frontend:
- Paused rows render with a distinct row class + muted opacity.
- The next-fire cell shows a yellow pause glyph + tooltip with the
paused-since timestamp and the would-have-fired time.
- Actions column: pause/resume toggle button (⏸/▶) beside fire/edit/cancel.
Fire-now is disabled while paused (resume first).
- Sort order: active → paused → cancelled (paused slot keeps schedules
visible without mixing them into the active top section).
- pauseSchedule() / resumeSchedule() async functions POST to the new
endpoints and refresh the table on success.
Per review: rather than adding forge_http_full (a near-duplicate of
forge_http), change forge_http itself to return (StatusCode, String).
Status-only callers bind (status, _); the branch-protection verify path
uses the body to log the real Forgejo rejection reason. Updates all call
sites accordingly.
apply_config_repo_branch_protection treated 200/409/422 from the
create-branch-protection POST all as success. But a 422 means Forgejo
*rejected* the request and created no rule — so a rejected POST silently
left the agent's config repo unprotected, with nothing logged (a new
agent's config repo was found with no main-branch protection and no
trace of why).
Don't trust the status code:
- On any non-201, GET the single .../branch_protections/main rule and
only treat it as success if the rule is actually present.
- Otherwise return Err carrying the POST's response body, so the real
Forgejo rejection reason lands in the host journal. (forge_http
discarded the body; added forge_http_full that returns it.)
ensure_config_repo runs on every sync_agent sweep (startup + each
rebuild), so a now-Err result is logged and retried next sweep —
self-healing once a real cause is fixed. Net: the failure is loud +
retried instead of silently swallowed.
nix fmt clean.
Both remove_agent() calls now run unconditionally for maximum partial
cleanup, but any I/O error is returned as HTTP 500 instead of silently
200-ing — so the frontend's !resp.ok path fires and the operator sees a
meaningful error rather than the stale row reappearing unchanged.
Also add a clarifying comment on isStale in permissions.js explaining
that containersState is keyed from nixos-container list (which includes
stopped-but-configured containers), so a temporarily-stopped agent is
not treated as stale — only destroyed/renamed agents are absent.
The P3RM1SS10NS tab showed agents that no longer exist in the live
container roster — e.g. an agent named 'root' that was renamed or
destroyed but still had explicit entries in tool-groups.json and/or
capabilities.json. The roster-union behaviour is intentional for
temporarily-stopped agents, but stale entries from renamed/destroyed
agents are confusing.
Backend (dashboard/permissions.rs):
- New DELETE /api/permissions/{agent} handler that bypasses the live-
roster guard (intentionally — that's the point). Calls
tool_groups::remove_agent + capabilities::remove_agent to clear both
JSON files, then emits live SSE snapshots so the tab updates without
a page reload. Format-checks the agent name but does not require it to
be in the containers snapshot.
Frontend (permissions.js):
- renderCapabilities / renderToolGroups now cross-reference agentNames
against containersState (the live roster, already imported). Agents
not in the live roster get an isStale flag.
- Stale rows get a '(not running)' label and a '✕ remove' button that
calls clearStaleAgent() — a new async helper that DELETEs the stale
entry and re-fetches both perm tables.
- Non-stale agents without explicit assignments still get '(default)'.
CSS (dashboard.css):
- .perm-row-stale (reduced opacity), .perm-stale-label (muted small
text), .perm-remove-btn (small red-bordered button) + disabled state.
The CSS for the rebuild-queue live-log panel targets .rebuild-live-log
(border, border-radius, margin-top, background) but the HTML element only
had id="rebuild-live-log" — no class. As a result the panel box styles
never applied and the live log rendered unstyled (no border, no background,
no visual separation from the queue rows).
Fix: add class="rebuild-live-log" to the element so the CSS selector
matches.
Both kinds fell through to the spawn branch in renderApprovals, showing
a misleading 'spawn' chip and agent-spawn body text. Mara saw a meta-input
bump render as a spawn card for agent damocles and denied it.
Backend (dashboard.rs):
- Add commit_ref: None to the MergeConfigPr arm (struct was incomplete).
All arms of ApprovalView now initialise every field.
Frontend (call.js):
- Add isUpdateMeta / isSchedule booleans alongside the existing kind flags.
- Glyph: update_meta_inputs gets ↻, schedule_prompt gets ⏱.
- Kind chip: 'meta-update' / 'schedule' (no kind-spawn class for either).
- Body: update_meta_inputs parses commit_ref as JSON Vec<String> and shows
'bump flake inputs: foo, bar' or 'bump all flake inputs'; schedule_prompt
parses SchedulePromptPayload and shows targets + first-fire time + cadence
+ a truncated body excerpt.
- History row: add 'meta-update' and 'schedule' cases (were both 'spawn').
- Import fmtDuration from util.js (needed for schedule cadence display).
