Phase 4 (repoint every container onto `meta#<n>`) and phase 5 (rename
the `root` container to `h-root`) were marker-guarded one-shots for
layouts no live hive still has: containers are rendered onto `meta#<n>`
at creation, and the `h-` prefix has been the naming for far longer than
any deployment predates. A one-shot nobody can still trigger is dead
weight, so both are gone along with `repoint_container`,
`rename_manager_container`, `CONTAINER_TIMEOUT` and the two marker paths.
Phase 6 was not obsolete, only misplaced. Ruth's tool groups are now
seeded by `ensure_root_agent` on the one path that creates her, rather
than re-asserted on every hive-c0re boot. The skip-if-already-set guard
survives the move: a destroy+recreate under the same name must not reset
an operator's chosen group set back to MANAGER_DEFAULT.
That also settles a latent bug. Phase 4's marker check was a `return`,
not a skip, so on any hive carrying the marker phases 5 and 6 never ran
at all — the tool-group backfill, whose whole job was preventing a silent
privilege downgrade, has not executed here in a long time. Moving it to
create-time removes the question rather than answering it.
What stays is convergence: three unguarded, idempotent phases that re-run
each boot and no-op once their state is right. The module doc now names
the three categories so the next person can tell which kind they're
adding.
The config-PR deploy's apply node still did the whole container rebuild
inline, through the last surviving `lifecycle::rebuild_no_meta` call. It
now merges, opens the two-phase meta deploy, and returns the ordinary
rebuild chain as a subgraph the scheduler grafts into the live DAG under
it. A new `FinalizeDeploy` node, gated on that graft, plants the deploy
tag and commits the staged lock.
Net effect: "did the agent come back up?" is answered by `Reconcile`
succeeding, the same way it is for every other rebuild, instead of by a
fused inline start — and each deploy phase is its own queue node, so the
dashboard shows which one is running.
The grafted nodes root on the apply node, so they land inside
`DeployWindow`'s subtree and re-enter the meta window and build slot it
already holds rather than deadlocking against them. The new happy-path
test runs on a one-slot queue specifically to pin that down.
`FinalizeDeploy`'s two git writes are fatal, deliberately: they are what
tells `DeployTail` a deploy confirmed good, so a node that merely warned
on them could report success while leaving the tail looking at the git
state of a failure — and the tail would then roll a good deploy back.
The trailing `meta::finalize_deploy` stays warn-only, since by then the
container already runs the new config.
The `failed/<id>` annotated tag moves into the tail, which is now the
only place holding a failed deploy. It reads the reason off the DAG via
a new `JobQueue::first_error`, and is gated on `main` having actually
moved — the rollback ref is parked *before* the merge, so its existence
alone does not mean a merge happened, and a pre-merge rejection must not
tag the previous, innocent head.
Removing the last inline rebuild orphaned a chain of now-dead code:
`rebuild_no_meta`, `container_exists`, `Coordinator::set_queue_build_log`
and `JobQueue::set_build_log_id_running`, all deleted here.
Rewrite the approval flow's step 4 as the three phases, with the reason
the rollback state is a git ref, and refresh the coordinator's node
inventory + dispatch table. Fix four doc comments that still pointed at
the deleted `run_merge_config_pr`.
coordinator.md rewrites the queue section (node inventory, DAG shapes,
resources, desired-state reconciliation, boot reconcile); approvals.md
+ persistence.md + hivectl --graceful help updated to match. agent_power
lives in broker.sqlite like approvals/questions (own connection + busy
timeout) instead of a separate db file.
notify_manager (always-root) has been replaced by notify_submitter
(routes to the approval submitter, with root fallback for legacy rows).
Update three locations:
- Step 6 of end-to-end flow: "root agent's inbox" →
"submitting agent's inbox"; explain notify_submitter + submitter column.
- InitConfig paragraph: remove limitation note; "to the root agent's inbox
(current limitation...)" → "to the submitting agent's inbox via notify_submitter".
- Section header + prose: "Helper events to the root agent" →
"Helper events to the submitting agent"; rewrite lead paragraph around
notify_submitter semantics (approvals.submitter column, legacy fallback).
Remove three bare #NNN tracker-tag references in docs/approvals.md
(hive-rules: no #NNN in code/comments/docs):
- Step 6 of end-to-end flow: "tracked in #1953" → "a known limitation
under active development"
- InitConfig paragraph: same rewrite
- Helper events section: same rewrite
Also add a code pointer for `can_manage_top_level_agents`: the role is
defined as ROLE_CAN_MANAGE_TOP_LEVEL_AGENTS in hive-c0re/src/topology.rs,
now noted inline so reviewers can verify the name is real.
The last docs/ piece of the manager-cleanup. The manager is no longer a
structural role — root-ness is purely topological. Reframe:
- title 'Approvals + manager + helper events' -> 'Approvals + helper events'
- section headers: 'Manager view of applied'/'Manager policy'/'Manager
(ruth) is hive-c0re-managed'/'Helper events to the manager' ->
root-agent / root-bootstrap-container equivalents
- body prose: 'the manager (ruth)' -> 'the root agent' (or 'the submitter'
in the approval-flow steps)
- authority semantics: 'manager-only' -> approvals are submitted by an
agent with the approvals tool group, for its direct children
- dropped the stale 'the manager refuses to destroy itself' line (the
bootstrap container is now destroyable + transient; recreated on startup)
Kept the genuine code/protocol identifiers (nixosConfigurations.manager,
manager_server, role:manager prompt block, notify_manager, the
/run/hyperhive/manager/ socket path) — renaming those would diverge from
the source (de-hardcoding is its own backend cleanup).
coordinator.md: GracefulStop entry referenced the old bare /kill/<agent>
path with the lax graceful=1 bool — update to /api/kill/<agent>?graceful=true.
approvals.md: the auto-update badge description referenced /rebuild/<name>
(bare, no /api/ prefix) — update to /api/rebuild/<name>.
The last docs/ piece of the manager-cleanup. The manager is no longer a
structural role — root-ness is purely topological. Reframe:
- title 'Approvals + manager + helper events' -> 'Approvals + helper events'
- section headers: 'Manager view of applied'/'Manager policy'/'Manager
(ruth) is hive-c0re-managed'/'Helper events to the manager' ->
root-agent / root-bootstrap-container equivalents
- body prose: 'the manager (ruth)' -> 'the root agent' (or 'the submitter'
in the approval-flow steps)
- authority semantics: 'manager-only' -> approvals are submitted by an
agent with the approvals tool group, for its direct children
- dropped the stale 'the manager refuses to destroy itself' line (the
bootstrap container is now destroyable + transient; recreated on startup)
Kept the genuine code/protocol identifiers (nixosConfigurations.manager,
manager_server, role:manager prompt block, notify_manager, the
/run/hyperhive/manager/ socket path) — renaming those would diverge from
the source (de-hardcoding is its own backend cleanup).
The internal forge is the canonical store for the meta flake, every
agent's config repo, and the internal/* repos, so it can no longer be
optional. Remove the services.hyperhive.forge.enable toggle:
- hive-forge.nix: drop the `enable` option; the forge config now
deploys gated on `services.hyperhive.enable` (it ships with hyperhive).
- hive-c0re.nix: HIVE_FORGE_URL env unconditional; forge-public-URL gate
drops the enable check (keeps behindGateway).
- hive-gateway.nix: local /etc/hosts forge entry keyed on behindGateway.
- hive-ci.nix: drop the now-moot `forge.ci.enable requires forge.enable`
assertion (forge is always present); reword the option doc.
- nix/docs/default.nix: drop the `forge.enable = mkForce false` stub
(option gone); the options-doc eval stays light via hyperhive.enable.
- hive-c0re forge.rs / hivectl.rs: reword 'forge.enable = true' error
text to 'wait for hive-c0re to start the container' (the runtime
token-absent path is unchanged — it's a bootstrap-timing check, not
the opt-out).
- docs/approvals.md, docs/ci.md: drop stale forge.enable references.
Migration: configs that set `services.hyperhive.forge.enable = false`
must drop the line — the forge is now mandatory.
Prereq/companion to #1838 (PR-based config flow, which assumes the forge
is always present).
claude.md flips 'in flight' → 'just landed' for the meta
overhaul + extends the file map with meta.rs and migrate.rs.
docs/approvals.md replaces the in-flight callout with a
proper 'Meta flake' section (two-phase deploy walkthrough,
sync_agents semantics, single-phase variants), updates the
two-repo box diagram to include the /var/lib/hyperhive/meta/
tree and tracks flake.nix in applied, rewrites the
container --flake reference to meta#<name>, replaces the
'Manager view of applied' section with a unified
'/agents + /applied + /meta' inventory listing every useful
git incantation, and explains the in-place no-state-loss
migration that now runs on hive-c0re startup.
docs/persistence.md grows entries for the meta repo + the
.meta-migration-done marker. readme box diagram picks up the
/meta RO bind; approval-flow paragraph rewritten end to end
to describe the meta lock dance.
lifecycle::flake_base deleted — the meta render hardcodes
the manager vs agent-base choice as nix expression.
scratchpad in claude.md and an in-flight callout at the top of
docs/approvals.md describe the upcoming overhaul so subsequent
commits can cite the design. covers: module-only agent flake
shape, /var/lib/hyperhive/meta/ as a hive-c0re-owned single
repo, applied remote pre-wired in proposed for manager git
plumbing, /meta RO bind for the system-wide deploy log,
auto-migration on hive-c0re startup with HIVE_SKIP_META_MIGRATION
kill-switch.
manager prompt: explain that arbitrary files now travel with
the proposal, document the /applied/<n>/.git RO mount and the
tag scheme (git show applied/deployed/<id> etc.), call out
that applied/main only advances on deployed so a failed build
isn't terminal. approvals.md: drop the old per-agent
applied.git phrasing in favour of the single /applied RO
bind, mention both manager binds together. claude.md
scratchpad flips from in-flight to just-landed.
scratchpad in claude.md marks this as in-flight; docs/approvals.md
gets the new tag state machine (proposal/approved/building/deployed/
failed/denied) and the manager applied.git read-only mount. todo
picks up the unprivileged-containers git-identity caveat and a web
ui for config repos as a downstream follow-up.
crash_watch grows two more state-axes alongside running/stopped:
- logged-in (claude session dir populated for the agent)
- up-to-date (recorded flake rev matches current)
per-tick transitions emit HelperEvent::NeedsLogin / LoggedIn /
NeedsUpdate. seed-on-first-tick semantics retained — nothing fires
on harness boot for agents that were already in their state. only
needs_update fires the 'stale appeared' direction; the resolved
direction is already covered by Rebuilt.
new mcp__hyperhive__update(name) on the manager surface: idempotent
rebuild via auto_update::rebuild_agent. transient-aware (Rebuilding)
so the dashboard shows the spinner. login intentionally has NO tool
— it's interactive OAuth, only the operator can complete it.
prompts + approvals doc + turn-loop doc updated. todo grows a
'show per-agent applied config in dashboard' entry (separate
follow-up).
revert the earlier 'operator must set allowUnfree' move:
per-agent containers evaluate their own nixpkgs and the operator's
host-level allowUnfree doesn't propagate in. restoring the scoped
allowUnfreePredicate inside both the claude-unstable overlay and
harness-base.nix; documented in README + gotchas as 'nothing to
set on the operator side'.
docs:
- claude.md file map adds crash_watch.rs, kick_agent on coordinator,
/api/model + journald viewer + bind-with-retry references.
- scratchpad rewritten to reflect the recent run.
- web-ui.md: notification row + browser notifications section,
state row (badge + model chip + last-turn chip + cancel button),
per-agent inbox, /model slash, /cancel-question + journald
endpoints, focus-preservation on refresh.
- turn-loop.md: --model is read from Bus::model() per turn (runtime
override via /model); recv(wait_seconds) up to 180s with the
rationale; ask_operator gains ttl_seconds; new TurnState section;
kick_agent inbox-on-startup hint.
- approvals.md: ttl/cancel resolution paths for operator questions.
- persistence.md: /state/hyperhive-model file.
- gotchas.md: web UI port collision policy (rename, don't probe);
bind retry + SO_REUSEADDR shape; auto-unfree restored.
- todo.md: cleaned up empty sections and stale entries; /model
shipped, dropped from the list.
new hive_c0re::crash_watch task polls every 10s, builds the set of
currently-running containers, and on running→stopped transitions
checks the transient snapshot: if no Stopping / Restarting /
Destroying / Rebuilding flag is set, the container exited
unexpectedly and we fire HelperEvent::ContainerCrash into the
manager's inbox so it can react (typically: start it again).
first poll is a seeding pass — no events on harness startup. dbus
subscription would be lower-latency but polling is honest and
debuggable, and a 10s delay on crash detection is fine for our
scale.
manager prompt + approvals doc updated to advertise the new
event variant. todo drops the entry (and the journald-viewer
entry that already shipped).
claude.md was eating 400 lines of subsystem detail that's useful
when you're working on that subsystem and noise the rest of the
time. split into:
- docs/conventions.md naming, identity, async forms, commit style
- docs/gotchas.md nspawn / nixos-container quirks
- docs/web-ui.md dashboard + per-agent layouts and endpoints
- docs/turn-loop.md claude invocation, wake prompt, mcp surface
- docs/approvals.md approval flow, manager policy, helper events
- docs/persistence.md sqlite dbs, retention, state dir layout
claude.md is now the entry point — file map, reading paths
("pick the doc that matches your task"), quick reminders that
fit on one screen, and a small scratchpad section for in-flight
context. references the docs; the docs don't reference claude.md.
no content was lost — the docs/ files cover everything the old
claude.md did, plus things i wrote up better while extracting.
