hyperhive/docs/agent-lifecycle/config-changes.md

Config changes + system messages

Config changes are pull requests on the agent's config repo, which an operator merges on the forge; that merge deploys them. The "System messages" section covers the plain-text system messages hive-c0re sends agents.

For operators

• Config change — an agent proposed a change to another agent's config (or its own, via a sub-manager) as a pull request on agent-configs/<agent>. Review it on the forge like any other PR, and merge it there to deploy it. See Config changes.

Everything below this point is the implementation detail behind that flow.

Config changes

Config changes flow through a forge pull request on the agent's agent-configs/<name> repo — the same surface agents use for code PRs. No MCP tool exists for config changes: opening the PR IS the request.

1. The submitting agent clones agent-configs/<name>, edits it there (any tracked path, but agent.nix is the contract entry point), commits with its own git identity, and pushes a branch + opens a PR with hive-forge — the same way it would change any other repo. The container has no config mount: the clone is how an agent reads its own config as well as how it changes it. Branch protection (the agent isn't on the main push allowlist; merge and approvals allowlist = operators team; see "Forge config repos" below) makes the agent a write collaborator that can't merge its own config PR.
2. An operator reviews the PR on the forge (native diff, threaded comments, CI status) and merges it there. Merging needs membership of the operators team in agent-configs. The team starts empty: add each operator by hand in the forge UI.
3. swarm-controller gets the agent-configs org's pull_request delivery. A closed event with merged: true and base branch main names the commit in merge_commit_sha.
4. The controller looks up which hive's wanted state places the agent and publishes a deploy request carrying that commit on that hive's deploy subject. With no such hive, or more than one, it deploys nothing and logs a warning naming them.
5. If the closure the swarm declares for the agent is the one the hive last applied (the applied/.<n>.closure marker), the hive does nothing. Otherwise it queues a rebuild, which downloads the declared closure. The request's commit plays no part: the hive keeps no copy of the config and reads no forge head. An agent with no container on that hive yet gets its first deploy from the declared closure.

This path runs no eval-verify. A merged config that doesn't evaluate or build fails the rebuild, so the agent's rebuilds keep failing until a fix merges. A failed rebuild shows on the hive like any other. A merge whose webhook delivery never reaches the controller deploys nothing.

Creating a brand-new agent is swarm-level (swarmctl agent create, POST /api/agents on the swarm controller), and no hive can originate an agent. The swarm controller's InitAgentConfigRepo job seeds the agent's config repo with a default agent.nix template, then asks the target hive to deploy it. An agent created from an archetype gets no config repo: the swarm builds it from the archetype's module.

Changing what the template seeded isn't an exception: like every later change, it's a PR on that config repo, made from a clone and merged by an operator on the forge (see Config changes).

Agent lifecycle

Reminder delivery: file-path semantics

A reminder may carry a file_path (the agent-visible path inside its container, for example /agents/<name>/state/foo.md). On delivery hive-c0re:

1. Translates the container path to the host path (/var/lib/hyperhive/agents/<name>/state/foo.md) so c0re can write from outside the container.
2. Validates the path: rejects anything outside the agent's own state subtree, containing .. (path traversal), or with an empty relative tail. On rejection hive-c0re skips the write and delivers the original message inline with a warning — the reminder still fires.
3. Defends against symlink escape: after create_dir_all, hive-c0re canonicalizes the parent dir and re-verifies it lives under the agent's host state root. hive-c0re opens the final file with O_NOFOLLOW | O_CREAT | O_TRUNC so an existing symlink at the basename can't redirect the write to an arbitrary host path.
4. Writes the body to disk and delivers a short pointer message in its place, keeping the agent's inbox / wake-prompt small while the agent reads the bulky payload out of band.

Broker::deliver_reminders_batch handles atomicity of the inbox INSERT + reminders.sent_at UPDATE; the scheduler only computes the body strings before calling it.

Destroy semantics

HostRequest::Destroy { name, purge } is the lifecycle tear-down. Stops + removes the nspawn container and drops the systemd drop-in. Persistent state (claude credentials, /state/ notes) is kept by default — recreating the agent with the same name reuses prior notes + login. With purge = true the agent's /var/lib/hyperhive/agents/<name>/ tree is also wiped (creds + notes gone forever). The root/bootstrap container is destroyable like any other, and stays gone until you create ruth again from the swarm dashboard.

Meta flake

A hive has no meta flake of its own. It builds every agent from the swarm's internal/meta, which swarm-controller commits (below). It declares one flake input per agent (agent-<n>.url = "git+<forge>/agent-configs/<n>.git?rev=<sha>", the head of main on the agent's forge config repo, the one config PRs merge into) and one nixosConfigurations.<n> output per agent. Each output wraps inputs.agent-<n>.nixosModules.default with the identity + HIVE_PORT / HIVE_LABEL injection module. Values that differ per host (the dashboard port, the pinned claude-code, extra OTEL labels, the queue address) aren't in it: hive-priv writes them to the container's /etc/hyperhive-hive-facts at start (see docs/process/conventions.md). A config that sits only in a local checkout, not on forge main, never deploys.

What hive-c0re still keeps under /var/lib/hyperhive/meta/ is plain state, bound read-only into the root agent's container as /meta: topology.json, the swarm resource limits, and the permission seed files of a hive that predates the swarm's permissions. agent_set::refresh(agents: &[AgentSpec]) keeps topology.json, the gateway's agent-socket map and its nginx include in line with the live agents. provision_container, destroy, rebuild and the startup migration call it. It refuses an empty agent list, which means nixos-container list failed, and it fails when HIVE_FORGE_URL is unset, since every agent would come up with no forge.

The swarm's meta flake

swarm-controller commits internal/meta: one flake.nix and flake.lock for every agent any hive places (the pins: docs/swarm/builder.md). The swarm builder builds every agent from it, and a hive deploys that build. While the swarm has recorded no build of an agent, that agent's deploy fails and names the agent. The controller queues its update of the repo before the deploy request for a merged config commit and for a new agent, and sends the deploy only when that update succeeded and the swarm builder built the agent from the updated repo (the deploy gate). A failed build skips the deploy, and the timer builds again.

• Choosing the closure. The hive doesn't choose. It deploys the closure the agent's record in agent-closures declares, whichever internal/meta commit the builder built it from, and reads no forge head. The rebuild's MetaSync node prepares it (the agent's archetype, its runtime directories, the agent set), reads the record once and pins its store path. Prebuild and Swap use the pin, so a record that changes mid-rebuild can't make Swap pull another closure than Prebuild fetched, and the marker names the closure it applied. The next autoupdate pass picks up the newer record. A first deploy has no MetaSync; its Create reads the record once.
• The closure. The hive builds nothing. Prebuild, Swap and Create read the agent's record in the swarm's agent-closures bucket (the controller writes it from the builder's reply) and have hive-priv download the closure it names from the swarm cache (RealiseClosure). Swap and Create then give hive-priv the store path (toplevel on UpdateContainer and CreateContainer), which accepts only the path its last RealiseClosure for that agent returned. On the wire the hive_priv_sock::PrivRequest variants carry a required field: {"op":"update_container","name":"atlas","stream":true,"toplevel":"/nix/store/<hash>-nixos-system-atlas"}. A rebuild with no record, or none the builder pushed, fails with an error that names the agent; see docs/swarm/builder.md.
• Hyperhive. The commit pins one hyperhive rev for the swarm, so an agent runs the hyperhive of the controller's deploy, not its hive's.
• Marker. After a successful swap or create hive-c0re writes the closure's store path to applied/.<n>.closure. The autoupdate pass (below) compares it with the record.

Deploy markers per agent

/var/lib/hyperhive/applied/                 deploy markers - core-only
└── .<name>.closure                         # store path of the last closure applied

/var/lib/hyperhive/meta/                    plain hive state - core
├── topology.json                           # the live agent set
└── swarm-resource-limits.json              # the swarm's per-agent caps

hive-c0re keeps no copy of an agent's config: the forge's agent-configs/<name> main is the only one, and a config change is a PR against it. The marker records the closure last applied, and nothing else. Hives that ran before the marker existed have .<name>.hyperhive-rev, .<name>.config-rev and meta-rev/<name> files instead. Nothing reads them any more, and you can delete them.

Older hives also kept an applied repo at applied/<name> and a proposed clone at agents/<name>/config. Nothing reads either any more, and you can delete them.

The container's --flake ref is internal/meta (see "Meta flake" above). The agent's agent-configs/<n>/flake.nix is a fixed boilerplate that exports nixosModules.default = import ./agent.nix; the meta flake imports that module and wraps it with identity + HIVE_PORT / HIVE_LABEL.

Forge config repos

The bundled hive-forge container runs on the swarm's forge host (deploy.forgejo.enable, see ../swarm/services.md). Every agent created through the swarm has a repo in the private agent-configs Forgejo org, created and seeded by swarm-controller. hive-c0re pushes nothing to it, keeps no clone and reads nothing from it: forge main advances only by an operator's merge.

Each agent is a write collaborator on its own agent-configs/<name> repo — so it can push a branch and open a config PR — but not a member of any other agent's, so it can't reach another agent's config through the forge. Branch protection keeps the agent off the main push allowlist and allowlists merging to the operators team only, so an agent can't fast-forward its own config or self-merge its PR (see Config changes above). hive-c0re passes the credential to each git call inline, never writing it into a repo's config, so no stored token sits in a repo an agent might one day read.

The dashboard deep-links into this org — a config repo link per container row. See docs/web-ui/dashboard.md.

Submitting agent's view of config repos

No agent has a config repo mounted, not even one holding ManageRootAgent: reading another agent's config means cloning its repo from the forge, subject to the forge's own access rules.

An agent submits a change to another agent's config the same way it makes any other change: clone the target agent's config repo from the forge into its own state dir, commit on a branch, open a PR, and let the operator review and merge it. By design, no second, mount-shaped path reaches the same file without the review.

Agents holding the manage_root_agent permission (granted per agent in the swarm's agent-permissions record; see docs/process/conventions.md#permissions) get additional host-side bind mounts via set_nspawn_flags:

• /var/lib/hyperhive/agents/<n>/state → /agents/<n>/state (RW) for every agent. The permission means "may manage any agent," so the mounted set is every agent.
• /var/lib/hyperhive/meta/ → /meta/ (RO) — the hive's plain state (topology, swarm limits).

An agent without the permission only has its direct children's config dirs.

⚠️ nspawn bakes bind flags at container start, so hive-c0re restarts an agent, after its current turn, when its manage_root_agent changes.

No agent holds the permission by default: an agent whose record leaves permissions unset gets the default, which doesn't include manage_root_agent, whatever the agent's name. The swarm UI's setup assistant creates ruth with it. For a ruth that predates that, whose permissions were unset, swarm-controller writes the default plus manage_root_agent into her key once at start, so she keeps her mounts; it never touches a key that's already set.

Startup migrations (older hosts)

hive-c0re runs a couple of idempotent migrations on every startup so a host set up before the current deploy scheme (described above) converges to it automatically. Each phase is a no-op once already applied:

• Agent files: refreshes the files that follow the agent set (agent_set::refresh) from the current agent list. Set HIVE_SKIP_META_MIGRATION=1 on the service to defer this phase.
• Legacy meta repo: deletes the leftover .git in /var/lib/hyperhive/meta/ and the unread /var/lib/hyperhive/forge-core-token. The files beside the .git stay. The forge-side core/meta repo and core account aren't touched.

No state loss in either migration: claude creds, /state/ notes, the events DB all survive. The root agent keeps its session; sub-agents stay logged in.

The root/bootstrap container

The root agent container runs through the same lifecycle as sub-agents. Nothing creates it at hive boot: you create ruth from the swarm dashboard like any agent, and an existing ruth container starts with the hive like every other. The root agent can clone agent-configs/ruth to read its own agent.nix and open a config PR there for operator review, same as any other agent.

Differences from sub-agents:

• Reference docs: her agent.nix turns on services.hyperhive.agent.docs.enable. Her container builds from agent-base like every agent, and the seeded agent.nix is the same for every name, so a newly created ruth has no docs until a config PR adds that line.
• Web UI port via lifecycle::agent_web_port("ruth") — same FNV-1a hash as every other agent (8100..8999 range).
• The root agent holds manage_root_agent (the swarm UI's setup assistant creates her with it), for which set_nspawn_flags adds extra binds: every agent's state dir (RW) under /agents/<n>/state, and the hive's plain state under /meta (RO).
• Its socket comes from the generic per-agent registration, in the same per-agent runtime dir as any other agent (/run/hyperhive/agents/ruth/), not a separate manager-only path.

Migration note (for older hosts): drop any containers.root = { ... } block from your host NixOS config, and any services.hyperhive.ruthless definition, then create ruth from the swarm dashboard.

Root-agent policy

The system prompt (hive-agent/prompts/system.md, rendered by hive-agent/src/prompt.rs) is the same for every agent; what varies is which MCP tools it surfaces (gated by the agent's permissions, which the swarm sets). No role:manager block renders only for the root agent. The root agent's role-specific behaviour comes from its AGENTS.md / agent-specific instructions, not the system prompt template.

System messages

Hive-c0re has no typed event that drives an immediate claude turn. A message from sender system is plain inbox text, and the harness logs it (log_system_event in hive-agent/src/main.rs) without parsing the body. Today hive-c0re sends two: the /knowledge updated broadcast (workers/knowledge.rs) and the wake message for a graceful stop (Coordinator::kick_agent).

A previously running container crashing drives no agent turn. crash_watch only tracing::warn!s "container crash detected"; a swarm-level grafana alert matches that log line directly and pages the operator, with no agent (and no per-hive dashboard banner) in the loop. See docs/scheduler/observability.md.

The remaining lower-urgency lifecycle notices — Rebuilt, Killed, Destroyed, NeedsLogin, LoggedIn — are "FYI, check when convenient" events with no reason to drive an immediate turn, so they go out via swarm_notices::notify instead: subsystem = "core", key = "<event>:<agent>", and a single free-text summary (rebuilt_todo_summary renders Rebuilt's ok/note/sha/tag fields into that string).

To add a new lifecycle notice: if it's "FYI, check when convenient," call swarm_notices::notify — no new wire type needed. If it needs to drive an immediate turn, decide first whether it actually needs an agent in the loop at all (the crash case above didn't, once a swarm-level alert could carry the same signal).

Autoupdate on startup

hive-c0re serve runs auto_update::run in a background task right after opening the coordinator. It enumerates managed containers and rebuilds any whose closure marker differs from the store path the agent's record in agent-closures declares. That's how a hive that missed the swarm's deploy request catches up, and it repeats every five minutes, which covers a request lost while the hive is up. Sub-agents and the root agent go through the same graceful rebuild subgraph.

The comparison reads the record and the marker (/var/lib/hyperhive/applied/.<name>.closure, written after a successful swap or create) and nothing on the forge. Where the marker is missing, as for every agent on the first boot after the marker arrived, the hive reads the closure the container's profile (/nix/var/nix/profiles/per-container/h-<name>/system) points at and writes that as the marker, so the first pass catches an agent that doesn't run its declared closure. Those first-boot rebuilds go through the same job queue as any other: each holds one of the buildSlots, so they don't swap all at once. It rebuilds nothing when the hive can't tell: no connection to the swarm queue, no record, a closure the builder didn't push, or a profile it can't read or that leaves the nix store. The periodic pass attempts a declared store path once per process, and skips an agent while a rebuild of it waits or runs, so a closure that fails to deploy doesn't restart its agent every five minutes. Restarting hive-c0re, or a rebuild from the agent's menu, tries again.

A stopped agent gets its rebuild at its next start: the start path upgrades the start of a stale, stopped agent to a rebuild followed by the start. In the window after a config merge, before the swarm's builder has recorded the new closure, the rebuild deploys the previous one.

The dashboard shows no pending-update state: the hive applies a changed declaration itself. The agent menu's rebuild queues the same job_queue::templates::rebuild DAG, which downloads and applies the declared closure again.