vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

drop the applied repo: the hive keeps no copy of an agent's config, only the deploy markers. Refs #5913

Browse source
This commit is contained in:
damocles 2026-11-01 17:19:41 +01:00
commit 862495530c
10 changed files with 190 additions and 1634 deletions
Download patch file Download diff file Expand all files Collapse all files

106
docs/agent-lifecycle/config-changes.md
View file

@ -41,22 +41,17 @@ No MCP tool exists for config changes: opening the PR IS the request.
and publishes a deploy request carrying that commit on that hive's
deploy subject. With no such hive, or more than one, it deploys
nothing and logs a warning naming them.
5. If the hive's `applied/main` already is that commit, it does nothing.
Otherwise it fetches the config repo's `main`. A commit that
`applied/main` already contains is a no-op too: requests for two merges
can arrive in either order. A commit that descends from `applied/main`
is fast-forwarded to and a rebuild queued. A failure before the rebuild
(the fetch, or a commit that has diverged from `applied/main`) deploys
nothing and logs `swarm events: rev not
applied; nothing queued` at error, which fires the Grafana alert
`config rev refused`. An agent with no container on that hive yet ignores
the commit; its first deploy builds what it seeds.
5. If the commit already is the one the agent's last successful build pinned
(the `applied/.<n>.config-rev` marker), the hive does nothing. Otherwise it
queues a rebuild, which resolves its pins from `internal/meta` like any
rebuild. The request's commit only feeds that check: the hive keeps no copy
of the config. An agent with no container on that hive yet ignores the
commit; its first deploy builds whatever `internal/meta` pins.
This path runs **no eval-verify**. A merged config that doesn't
evaluate or build fails the rebuild, and `applied/main` stays at that
commit, so the agent's rebuilds keep failing until a fix merges. A
failed rebuild shows on the hive like any other. A merge whose webhook delivery never reaches the controller
deploys nothing.
evaluate or build fails the rebuild, so the agent's rebuilds keep failing
until a fix merges. A failed rebuild shows on the hive like any other. A merge
whose webhook delivery never reaches the controller deploys nothing.
Creating a brand-new agent is swarm-level (`swarmctl agent create`,
`POST /api/agents` on the swarm controller), and no hive can originate
@ -100,12 +95,11 @@ body strings before calling it.
`HostRequest::Destroy { name, purge }` is the lifecycle tear-down.
Stops + removes the nspawn container and drops the systemd drop-in.
Persistent state (the applied repo, claude credentials, `/state/`
notes) is
Persistent state (claude credentials, `/state/` notes) is
**kept by default** — recreating the agent with the same name
reuses prior config + login. With `purge = true` the agent's
`/var/lib/hyperhive/{agents,applied}/<name>/` trees are also
wiped (config history + creds + notes gone forever). The
reuses prior notes + login. With `purge = true` the agent's
`/var/lib/hyperhive/agents/<name>/` tree is also
wiped (creds + notes gone forever). The
root/bootstrap container is destroyable like any other, and stays gone
until you create ruth again from the swarm dashboard.
@ -123,8 +117,8 @@ agent's forge config repo, the one config PRs merge into) and one
address) aren't in it:
`hive-priv` writes them to the container's `/etc/hyperhive-hive-facts` at
start (see `docs/process/conventions.md`).
A config that sits only in the hive's `applied/<n>` mirror, not on forge
`main`, never deploys.
A config that sits only in a local checkout, not on forge `main`, never
deploys.
What `hive-c0re` still keeps under `/var/lib/hyperhive/meta/` is plain
state, bound read-only into the root agent's container as `/meta`:
@ -184,59 +178,41 @@ the swarm builder built the agent from the updated repo (the
reconcile ([below](#autoupdate-on-startup)) and the dashboard's update chip
read them.
## The applied repo per agent
## Deploy markers per agent
```
/var/lib/hyperhive/applied/<name>/ applied - core-only
├── .git/ # tag-rich history
├── flake.nix # tracked, fixed
│ # boilerplate exporting
│ # nixosModules.default
├── agent.nix # working tree of main
└── <other committed files> # also tracked
/var/lib/hyperhive/applied/ deploy markers - core-only
├── .<name>.hyperhive-rev # hyperhive commit of the last good build
└── .<name>.config-rev # config commit of the last good build
/var/lib/hyperhive/meta/ plain hive state - core
├── topology.json # the live agent set
└── swarm-resource-limits.json # the swarm's per-agent caps
```
hive-c0re seeds the applied repo at first spawn from the forge's
`agent-configs/<name>` `main`, or from a template when the forge has none yet,
and fast-forwards it to each merged config commit that deploys. An agent never
edits it: it isn't mounted into any container, and a config change is a PR
against the agent's forge repo.
Older hives also kept a *proposed* clone at `agents/<name>/config`. Nothing
reads it any more, and you can delete it.
hive-c0re keeps no copy of an agent's config: the forge's `agent-configs/<name>`
`main` is the only one, and a config change is a PR against it. The two markers
record what the agent's last successful build pinned, and nothing else.
Older hives also kept an *applied* repo at `applied/<name>` and a *proposed*
clone at `agents/<name>/config`. The startup migration removes the applied
repos. Nothing reads the proposed clone any more, and you can delete it.
The container's `--flake` ref is `internal/meta` (see "Meta flake" above).
The agent's own `applied/<n>/flake.nix`
The agent's `agent-configs/<n>/flake.nix`
is a fixed boilerplate that exports `nixosModules.default =
import ./agent.nix`; the meta flake imports that module and
wraps it with identity + `HIVE_PORT` / `HIVE_LABEL`.
### Tag state machine
hive-c0re plants `deployed/0` on the seed commit at first spawn. A
merged config commit that deploys plants no tag: the merged PR on the
forge and `internal/meta`'s commits record it. A config PR nobody merges
carries no extra state on the forge side: the PR stays open, and
the submitter pushes again (or closes it) to retry.
### Forge config repos
The bundled `hive-forge` container runs on the swarm's forge host
(`deploy.forgejo.enable`, see [`../swarm/services.md`](../swarm/services.md)).
Every agent created through the swarm has a repo in the private
`agent-configs` Forgejo org, created and seeded by swarm-controller.
hive-c0re pushes nothing to it: the applied repo's tags (`deployed/0`)
exist only locally, and forge `main` advances only by an operator's merge.
hive-c0re reads from it - reseeding a purged `applied`, catching `applied` up, and
and seeding `applied` at first spawn (`forge::fetch_config_main_into_applied`).
A legacy agent spawned by the hive alone has no swarm-seeded repo; if its
`agent-configs/<name>` has no `main`, a first spawn falls back to the local
template, and reseeding a purged `applied` fails with a `destroy --purge`
and re-spawn instruction.
hive-c0re pushes nothing to it and keeps no clone: forge `main` advances only
by an operator's merge. The hive reads the head of `main`, to resolve the
commit `internal/meta` pins.
Each agent is a **write collaborator on its own** `agent-configs/<name>`
repo — so it can push a branch and open a config PR — but not a member
@ -245,7 +221,7 @@ the forge. Branch protection keeps the agent off the `main` push
allowlist and allowlists merging to the `operators` team only, so an
agent can't fast-forward its own config or self-merge its PR (see
[Config changes](#config-changes) above). hive-c0re passes the credential to
each `git` call inline, never writing it into `applied/<n>/.git/config`, so
each `git` call inline, never writing it into a repo's config, so
no stored token sits in a repo an agent might one day read.
The dashboard deep-links into this org — a `config repo` link
@ -291,15 +267,14 @@ it never touches a key that's already set.
## Startup migrations (older hosts)
hive-c0re runs a couple of idempotent migrations on every startup so a
host set up before the tag-driven-deploy scheme (described above)
host set up before the current deploy scheme (described above)
converges to it automatically. Each phase is a no-op
once already applied:
- **Tags**: hive-c0re tags agents from before the tag-driven scheme
`deployed/0` on `main` once. Non-destructive — it doesn't touch live
containers, state dirs, or claude creds.
- **Agent files**: rewrites each `applied/<n>/flake.nix` to the
module-only boilerplate and refreshes the files that follow the agent set
- **Applied repos**: deletes the per-agent `applied/<n>` repos an earlier hive
kept. The deploy markers beside them stay. Non-destructive for everything
else — it doesn't touch live containers, state dirs, or claude creds.
- **Agent files**: refreshes the files that follow the agent set
(`agent_set::refresh`) from the current agent list. Set
`HIVE_SKIP_META_MIGRATION=1` on the service to defer this phase.
- **Legacy meta repo**: deletes the leftover `.git` in
@ -308,7 +283,7 @@ once already applied:
stay. The forge-side `core/meta` repo and `core` account aren't touched.
No state loss in either migration: claude creds, `/state/` notes, the
events DB, and the applied history all survive. The root
events DB all survive. The root
agent keeps its session; sub-agents stay logged in.
## The root/bootstrap container
@ -316,8 +291,7 @@ agent keeps its session; sub-agents stay logged in.
The root agent container runs through the **same lifecycle as
sub-agents**. Nothing creates it at hive boot: you create ruth from the
swarm dashboard like any agent, and an existing ruth container starts with
the hive like every other. The root agent's flake lives at
`/var/lib/hyperhive/applied/ruth/`. The root agent can clone
the hive like every other. The root agent can clone
`agent-configs/ruth` to read its own `agent.nix` and open a config PR there
for operator review, same as any other agent.
@ -414,8 +388,8 @@ config, that rebuild fails with the error that names the agent, and the
agent stays stopped until the update lands and you ask for the start again.
The dashboard surfaces pending config changes per agent: a clickable
"needs update ↻" badge appears whenever the agent's applied repo has a
commit that its last build didn't pin. The badge POSTs `/api/rebuild/<name>`, which inserts the
"needs update ↻" badge appears whenever the config commit that `internal/meta`
pins for the agent's next build differs from the one its last build pinned. The badge POSTs `/api/rebuild/<name>`, which inserts the
same `job_queue::templates::rebuild` DAG so manual triggers and the
startup scan can't drift. When at least one container is stale, a
top-level `↻ UPD4TE 4LL` button appears that loops over every

18
docs/agent-lifecycle/persistence.md
View file

@ -543,12 +543,12 @@ whenever the forge has moved on.
Isolation still holds by default: a container has its *own* dirs and,
unless it holds the permission, nothing else.
Under `/var/lib/hyperhive/applied/<name>/` — the hive-c0re-only
applied repo. Tracks `flake.nix` (module-only boilerplate; never
edited after first spawn) + `agent.nix` (the actual config; edits land
here from merged config PRs) + any other files those PRs commit.
`.git/` carries the proposal / approved / building / deployed / failed /
denied tag history.
Under `/var/lib/hyperhive/applied/` — the hive-c0re-only deploy markers
`.<name>.hyperhive-rev` and `.<name>.config-rev`: the hyperhive and config
commits the agent's last successful build pinned. No config lives here; the
forge's `agent-configs/<name>` is the only copy. A hive that predates this
finds per-agent `applied/<name>` repos here; `migrate::run` deletes them at
startup.
Under `/var/lib/hyperhive/meta/` — system-level config files, plain state
with no git history. The flake agents build from is the
@ -594,7 +594,7 @@ reads it at every container start for `manage_root_agent`.
`/var/lib/hyperhive/swarm-agent-archetypes.json` holds the archetype each of
this hive's archetype agents runs, as last read from the swarm's
`agent-archetype` bucket at the agent's provision and each rebuild. An agent
with an entry has no `applied/<n>` repo and no `agent-<n>` meta input; see
with an entry has no config repo and no `agent-<n>` meta input; see
[archetypes](../swarm/archetypes.md).
hive-c0re creates containers from `internal/meta` (`nixosConfigurations.<n>`).
@ -607,7 +607,7 @@ an agent's state. The mechanics, for completeness:
- `DESTR0Y` also drops the systemd drop-in; reviving the agent is
`swarmctl agent create` with the same name and hive, which reuses the
kept state.
- `PURG3` wipes `/var/lib/hyperhive/{agents,applied}/<name>/` — the
- `PURG3` wipes `/var/lib/hyperhive/agents/<name>/` — the
union of everything `DESTR0Y` left behind.
`actions::destroy` has no guard for the root/bootstrap agent: it destroys ruth
@ -636,7 +636,7 @@ until an explicit opt-in upgrade.
a subvolume root, so purge first calls hive-priv
(`DeleteAgentSubvolume`) which `btrfs subvolume delete`s it iff it's
actually a subvolume, then the normal `remove_dir_all` sweep covers
plain-dir agents + the applied dir.
plain-dir agents.
Per-subvolume disk-usage accounting and optional quotas have since
landed as the qgroup work: `hivectl quota-enable` turns on btrfs

5
docs/integrations/forge.md
View file

@ -63,10 +63,7 @@ store certificate, every time it reads the forge. A hive whose store holds
none reads nothing. These reads go through it, and none of them needs a local
forge container:
- fetching a merged config commit for a swarm deploy, and
`hivectl forge reconcile-config`
- reseeding a purged `applied` repo
- seeding `applied` from `agent-configs/<agent>` on first deploy
- resolving the config commit `internal/meta` pins for a build
- cloning and pulling `/knowledge`
The hive builds nothing, so no nix process reads the forge as the hive's

2
docs/networking/snapshot-store.md
View file

@ -206,7 +206,7 @@ means anything that has ever leaked into a task's captured output is
in the retained snapshots as well.
<!-- vale write-good.Passive = YES -->
It does **not** cover the agent's applied config (`/applied/<name>/`) or
It does **not** cover the agent's config (the forge's `agent-configs/<name>`) or
its topology entry, both of which live outside the subvolume. A restore
therefore yields an agent's memory without its definition.

4
docs/process/conventions.md
View file

@ -402,8 +402,8 @@ sets `EXTRA_NSPAWN_FLAGS` — plus the systemd resource-limits drop-in)
into the `Swap` node, then runs `nixos-container update` + stop +
start across the `StopForUpdate → Swap → RebuildBookkeeping`
brace and the tail `Reconcile` node. `flake.nix` itself lives in the agent's
the applied repo and rides along on every fetch (see
`docs/agent-lifecycle/config-changes.md::The applied repo per agent`).
`agent-configs/<name>` repo and rides along with its pinned commit (see
`docs/agent-lifecycle/config-changes.md::Deploy markers per agent`).
Anything that changes per-container state on the host should be
re-applied here so a manual `↻ R3BU1LD` from the dashboard is

4
docs/scheduler/coordinator.md
View file

@ -55,7 +55,7 @@ Cheap — no build slot:
| Node | Behavior |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `MetaSync` | the rebuild's preamble — rebuild-dir prep, the agent-set refresh (`agent_set::refresh`), and the `internal/meta` commit `Prebuild` and `Swap` expect the swarm's closure to be built from, resolved once and recorded; fails when the repo doesn't pin the agent's current config. Declares no resources |
| `Provision` | first-spawn pre-create provisioning — the applied repo, state subvolume, agent-set refresh (`agent_set::refresh`); runs ahead of `Create`. Store-only, no container yet |
| `Provision` | first-spawn pre-create provisioning — the state subvolume, agent-set refresh (`agent_set::refresh`); runs ahead of `Create`. Store-only, no container yet |
| `Reconcile` | idempotent power converge: read `wanted` (below) + observed state; start if `Up` & down (cold-start fallback included), stop if `Offline` & up, else noop |
| `Start` | mechanical container start — runtime dir + drop-ins, `start_with_fallback`, MCP listener registration, the manager kick. Fanned out by a `Reconcile` that observed `wanted = Up` and the container down |
| `Stop` | mechanical container stop — `nixos-container` kill, MCP listener unregister, the `Killed` manager notify. Fanned out by a `Reconcile` that observed `wanted = Offline` and up |
@ -67,7 +67,7 @@ Cheap — no build slot:
| `PauseSignal` | write the pause marker + mark `pause_pending`. No kick, unlike `Signal` — the harness's between-turns poll is already responsive enough, and `Signal`'s kick-message body ("you were just (re)started") would be actively misleading here |
| `PauseDrain` | await the harness reporting `PauseAcknowledged`, bounded timeout; best-effort like `Drain` |
| `DestroyContainer` | `nixos-container destroy` + un-registration (drop from the roster, clear the ephemeral runtime dir). Runs downstream of a `Stop`, so deliberately excluded from `takes_container_down` — the container is already down by the time it claims |
| `PurgeState` | the `purge = true` half of a destroy: delete the agent's state subvolume (via hive-priv) plus its state/applied dirs. Own node because it's conditional and the irreversible step |
| `PurgeState` | the `purge = true` half of a destroy: delete the agent's state subvolume (via hive-priv) plus its state dir. Own node because it's conditional and the irreversible step |
| `DestroyBookkeeping` | the post-destroy tail — meta sync, drop the power intent, notify the manager, rescan, re-emit the tombstone. Same split rationale as `RebuildBookkeeping`/`Swap`. The destructive work is `PurgeState`'s |
| `SetWanted` | write the durable power intent (`wanted = Up`/`Offline`) as the head node of a power-op DAG. Takes the agent lease even though it's a store write, so the intent write and the tail `Reconcile` are atomic per-agent — two racing power ops can't clobber each other's intent before either reconciles |
| `EmitRebuilt` | tail of a rebuild — emit the agent's `Rebuilt` manager event (ok/fail per outcome, nothing on cancel). One node per agent _and_ per outcome |

4
docs/swarm/archetypes.md
View file

@ -130,9 +130,7 @@ provisions the agent and on every rebuild, and keeps the last value read in
it has now. When the read fails, a rebuild keeps the stored value. For an
archetype agent the hive also:
- seeds no `proposed` or `applied` repo;
- ignores the `rev` of a swarm deploy request;
- mounts an empty `/agents/<name>/config/`.
- ignores the `rev` of a swarm deploy request.
The agent's system prompt says it has no config repo and names its
archetype.

43
docs/tools/hivectl-cli.md
View file

@ -5,8 +5,6 @@ This document contains the help content for the `hivectl` command-line program.
**Command Overview:**
* [`hivectl`↴](#hivectl)
* [`hivectl forge`↴](#hivectl-forge)
* [`hivectl forge reconcile-config`↴](#hivectl-forge-reconcile-config)
* [`hivectl agent`↴](#hivectl-agent)
* [`hivectl agent restart`↴](#hivectl-agent-restart)
* [`hivectl agent pause`↴](#hivectl-agent-pause)
@ -49,7 +47,6 @@ Sibling to the `hive-c0re` daemon binary. Covers host-side admin operations that
###### **Subcommands:**
* `forge` — Reconcile an agent's config between this hive and the forge
* `agent` — Lifecycle actions on ONE managed agent container. Needs the hive-c0re daemon running
* `list-agents` — Show all managed agents with their status and technical state
* `quota-enable` — Enable btrfs qgroup accounting on the agent-state filesystem
@ -69,46 +66,6 @@ Sibling to the `hive-c0re` daemon binary. Covers host-side admin operations that
## `hivectl forge`
Reconcile an agent's config between this hive and the forge.
A human's first SSO login to the forge makes their forge account.
**Usage:** `hivectl forge <COMMAND>`
###### **Subcommands:**
* `reconcile-config` — Show + reconcile the divergence between an agent's local applied config checkout and its forge `agent-configs/<agent>` main
## `hivectl forge reconcile-config`
Show + reconcile the divergence between an agent's local applied config checkout and its forge `agent-configs/<agent>` main.
Always prints the diff first. `--from forge` resets the local checkout to forge main (effective on the next deploy); `--from local` isn't supported yet. With no `--from`, prompts for the direction.
**Usage:** `hivectl forge reconcile-config [OPTIONS] <AGENT>`
###### **Arguments:**
* `<AGENT>` — Agent whose config branches to reconcile
###### **Options:**
* `--from <FROM>` — Which side to reconcile from. Omit it and hivectl prompts you after the diff
Possible values:
- `forge`:
Reset the local applied checkout to forge main
- `local`:
Advance forge main from local — not supported yet
* `--verbose` — Include the full diff (not just `--stat`) in the report
## `hivectl agent`
Lifecycle actions on ONE managed agent container. Needs the hive-c0re daemon running.

22
docs/tools/hivectl.md
View file

@ -23,24 +23,10 @@ markdown-docs > docs/tools/hivectl-cli.md`.
## Forge
Reconciles an agent's config between this hive and the forge. `hivectl`
makes no forge accounts: a human's first SSO login to the forge makes
theirs, and `swarmctl forge make-admin <name>` makes it a site admin.
swarm-controller makes an agent's, and `swarmctl agent mint-forge-token
<agent>` checks and, if needed, re-mints its token.
```bash
hivectl forge reconcile-config iris # show local-applied <-> forge config divergence, then prompt
hivectl forge reconcile-config iris --from forge # reset local applied checkout to forge main (effective next deploy)
hivectl forge reconcile-config iris --verbose # include the full diff, not just --stat
```
- `reconcile-config <agent>` shows the divergence between the agent's local
applied config checkout and its forge `agent-configs/<agent>` `main`, then
reconciles. `--from forge` resets the local checkout to forge `main` (takes
effect on the next deploy — it doesn't autorebuild). `--from local` isn't
supported yet (forge `main` is branch-protected; resolve via a config
PR). With no `--from` it prompts for the direction after the diff.
`hivectl` has no forge verbs and makes no forge accounts: a human's first SSO
login to the forge makes theirs, and `swarmctl forge make-admin <name>` makes it
a site admin. swarm-controller makes an agent's, and `swarmctl agent
mint-forge-token <agent>` checks and, if needed, re-mints its token.
## Agents

8
docs/web-ui/dashboard.md
View file

@ -81,12 +81,12 @@ same figures go out as OTEL metrics and appear in Grafana (see
caps live per agent in the swarm UI.
**K3PT ST4T3** — agents with kept state and **no container** (size + age +
claude-creds badge). One action: `PURG3` (wipes state + applied
dirs; `POST /api/purge-tombstone/{name}`).
claude-creds badge). One action: `PURG3` (wipes the state
dir; `POST /api/purge-tombstone/{name}`).
⚠️ **Not only *destroyed* agents.** Nothing records a destroy: `Provision`
writes every definition-side artifact (state subvolume, applied
repo, meta registration, topology entry) *before* the container exists,
writes every definition-side artifact (state subvolume,
meta registration, topology entry) *before* the container exists,
and that artifact set survives `lifecycle::destroy`. An agent
part-way through a spawn is byte-identical on disk to a tombstone, and
the pane lists both. The pane carries a standing warning to that effect, and