mara/hyperhive
mara/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

deploy the closure the swarm declares, read no forge head, drop the update badge

Browse source
This commit is contained in:
atlas 2026-11-01 23:56:40 +01:00 • committed by mara
commit 47c2acb640
51 changed files with 647 additions and 2001 deletions
Download patch file Download diff file Expand all files Collapse all files

4
AGENTS.md
View file

@ -95,8 +95,8 @@ Per-hive (host + in-container):
utility crate.
- **`hive-metric/`** — one-shot CLI pushing a labeled metric over OTLP.
- **`hive-meta-flake/`** — the meta flake's `flake.nix` text, a pure
render, and the reads of its pins. swarm-controller renders with it;
hive-c0re reads the pins of the `internal/meta` it builds from.
render, and the reads of its pins. swarm-controller renders and reads
with it; hive-c0re reads none of its pins.
Wire types + socket plumbing:

1
Cargo.lock generated
View file

@ -1775,7 +1775,6 @@ dependencies = [
"clap-markdown",
"clap_complete",
"ed25519-dalek",
"forgejo-api",
"futures-util",
"getrandom 0.3.4",
"hive-core-agent-sock",

108
docs/agent-lifecycle/config-changes.md
View file

@ -41,12 +41,12 @@ No MCP tool exists for config changes: opening the PR IS the request.
and publishes a deploy request carrying that commit on that hive's
deploy subject. With no such hive, or more than one, it deploys
nothing and logs a warning naming them.
5. If the commit already is the one the agent's last successful build pinned
(the `applied/.<n>.config-rev` marker), the hive does nothing. Otherwise it
queues a rebuild, which resolves its pins from `internal/meta` like any
rebuild. The request's commit only feeds that check: the hive keeps no copy
of the config. An agent with no container on that hive yet ignores the
commit; its first deploy builds whatever `internal/meta` pins.
5. If the closure the swarm declares for the agent is the one the hive last
applied (the `applied/.<n>.closure` marker), the hive does nothing.
Otherwise it queues a rebuild, which downloads the declared closure. The
request's commit plays no part: the hive keeps no copy of the config and
reads no forge head. An agent with no container on that hive yet gets its
first deploy from the declared closure.
This path runs **no eval-verify**. A merged config that doesn't
evaluate or build fails the rebuild, so the agent's rebuilds keep failing
@ -130,35 +130,24 @@ nginx include in line with the live agents. `provision_container`, `destroy`,
which means `nixos-container list` failed, and it fails when `HIVE_FORGE_URL`
is unset, since every agent would come up with no forge.
The dashboard's `deployed:<sha12>` chip per container row shows the config
commit the agent's last successful build pinned.
## The swarm's meta flake
swarm-controller commits `internal/meta`: one `flake.nix` and `flake.lock` for
every agent any hive places (the pins:
[`docs/swarm/builder.md`](../swarm/builder.md#meta-flake-lock)). The swarm
builder builds every agent from it, and a hive deploys that build. While the
repo has no commit that pins an agent's current config, that agent's deploy
fails and names the agent. The controller queues
swarm has recorded no build of an agent, that agent's deploy fails and names the
agent. The controller queues
its update of the repo before the deploy request for a merged config commit
and for a new agent, and sends the deploy only when that update succeeded and
the swarm builder built the agent from the updated repo (the
[deploy gate](../swarm/builder.md#deploy-gate)). A failed build skips the deploy, and the timer builds again.
- **Choosing the commit.** The rebuild's `MetaSync` node reads the repo's
default-branch head with the hive's read token (`forge::reader_token()`)
and takes it when its `flake.nix` builds the agent the way the agent needs now:
a custom agent from the config commit at the head of `agent-configs/<n>`
`main`, an archetype agent from the archetype it has now. An agent switched
between custom and archetype, or to another archetype, fails that test.
A repo with no commit, a head that lacks the agent and a head behind the
config each fail the deploy with an error that names the agent. `MetaSync` records the commit,
with the hyperhive and config commits the head pins, in `hive-c0re`'s own state,
`/var/lib/hyperhive/meta-rev/<n>`, which the agent can't write (the container mounts its runtime
directory read-write); `Prebuild` and `Swap` read that
record, so both expect the closure of one commit. A first deploy resolves the commit in its `Create`
node, since it has no `MetaSync`.
- **Choosing the closure.** The hive doesn't choose. It deploys the closure the
agent's record in `agent-closures` declares, whichever `internal/meta` commit
the builder built it from, and reads no forge head. The rebuild's `MetaSync`
node only prepares it (the agent's archetype, its runtime directories, the
agent set).
- **The closure.** The hive builds nothing. `Prebuild`, `Swap` and `Create` read
the agent's record in the swarm's `agent-closures` bucket (the controller
writes it from the builder's reply) and have `hive-priv` download the closure
@ -168,22 +157,20 @@ the swarm builder built the agent from the updated repo (the
that agent returned. On the wire the
`hive_priv_sock::PrivRequest` variants carry a required field:
`{"op":"update_container","name":"atlas","stream":true,"toplevel":"/nix/store/<hash>-nixos-system-atlas"}`.
The three cases when the swarm's closure isn't the one this hive expects
are in [`docs/swarm/builder.md`](../swarm/builder.md#rebuilds-download-the-closure).
A rebuild with no record, or none the builder pushed, fails with an error
that names the agent; see
[`docs/swarm/builder.md`](../swarm/builder.md#rebuilds-download-the-closure).
- **Hyperhive.** The commit pins one hyperhive rev for the swarm, so an agent
runs the hyperhive of the controller's deploy, not its hive's.
- **Markers.** After a successful build `hive-c0re` writes the hyperhive
commit the head pins to `applied/.<n>.hyperhive-rev`, and the config
commit to `applied/.<n>.config-rev` (none for an archetype agent). The boot
reconcile ([below](#autoupdate-on-startup)) and the dashboard's update chip
read them.
- **Marker.** After a successful swap or create `hive-c0re` writes the closure's
store path to `applied/.<n>.closure`. The autoupdate pass
([below](#autoupdate-on-startup)) compares it with the record.
## Deploy markers per agent
```
/var/lib/hyperhive/applied/ deploy markers - core-only
├── .<name>.hyperhive-rev # hyperhive commit of the last good build
└── .<name>.config-rev # config commit of the last good build
└── .<name>.closure # store path of the last closure applied
/var/lib/hyperhive/meta/ plain hive state - core
├── topology.json # the live agent set
@ -191,8 +178,11 @@ the swarm builder built the agent from the updated repo (the
```
hive-c0re keeps no copy of an agent's config: the forge's `agent-configs/<name>`
`main` is the only one, and a config change is a PR against it. The two markers
record what the agent's last successful build pinned, and nothing else.
`main` is the only one, and a config change is a PR against it. The marker
records the closure last applied, and nothing else. Hives that ran before the
marker existed have `.<name>.hyperhive-rev`, `.<name>.config-rev` and
`meta-rev/<name>` files instead. Nothing reads them any more, and you can
delete them.
Older hives also kept an *applied* repo at `applied/<name>` and a *proposed*
clone at `agents/<name>/config`. Nothing reads either any more, and you can
@ -210,9 +200,8 @@ The bundled `hive-forge` container runs on the swarm's forge host
(`deploy.forgejo.enable`, see [`../swarm/services.md`](../swarm/services.md)).
Every agent created through the swarm has a repo in the private
`agent-configs` Forgejo org, created and seeded by swarm-controller.
hive-c0re pushes nothing to it and keeps no clone: forge `main` advances only
by an operator's merge. The hive reads the head of `main`, to resolve the
commit `internal/meta` pins.
hive-c0re pushes nothing to it, keeps no clone and reads nothing from it: forge
`main` advances only by an operator's merge.
Each agent is a **write collaborator on its own** `agent-configs/<name>`
repo — so it can push a branch and open a config PR — but not a member
@ -356,29 +345,30 @@ swarm-level alert could carry the same signal).
`hive-c0re serve` runs `auto_update::run` in a background task right
after opening the coordinator. It enumerates managed containers and
rebuilds any whose recorded hyperhive rev differs from the one
`internal/meta` pins now. That's how a hive that missed the swarm's
fan-out catches up. Sub-agents and the root agent go through the same
graceful rebuild subgraph.
rebuilds any whose closure marker differs from the store path the agent's
record in `agent-closures` declares. That's how a hive that missed the
swarm's deploy request catches up, and it repeats every five minutes, which
covers a request lost while the hive is up. Sub-agents and the root agent go
through the same graceful rebuild subgraph.
"Rev" = the hyperhive commit pinned by the head of `internal/meta`, as
the hive reads it with its forge token. Marker
file: `/var/lib/hyperhive/applied/.<name>.hyperhive-rev`, written after
a successful build. If the hive can't read the pin (no forge token, the
forge down, `internal/meta` not published yet), the scan rebuilds
nothing; a rebuild would have no flake to build from either. A hive deploy
alone doesn't rebuild agents.
The comparison reads the record and the marker
(`/var/lib/hyperhive/applied/.<name>.closure`, written after a successful
swap or create) and nothing on the forge. It rebuilds nothing when the hive
can't tell: no connection to the swarm queue, no record, a closure the builder
didn't push, or an agent with no marker. The last case covers every agent on
the first boot after the marker arrived, so that boot rebuilds nothing; the
agent's next deploy request from the swarm writes the marker. The periodic pass
attempts a declared store path once per process, and skips an agent while a
rebuild of it waits or runs, so a closure that fails to deploy doesn't
restart its agent every five minutes. Restarting `hive-c0re`, or a rebuild from
the agent's menu, tries again.
A stopped agent gets its rebuild at its next start: the start path upgrades
the start of a stale, stopped agent to a rebuild followed by the start. In
the window after a config merge, before the swarm's update has pinned the new
config, that rebuild fails with the error that names the agent, and the
agent stays stopped until the update lands and you ask for the start again.
the window after a config merge, before the swarm's builder has recorded the
new closure, the rebuild deploys the previous one.
The dashboard surfaces pending config changes per agent: a clickable
"needs update ↻" badge appears whenever the config commit that `internal/meta`
pins for the agent's next build differs from the one its last build pinned. The badge POSTs `/api/rebuild/<name>`, which inserts the
same `job_queue::templates::rebuild` DAG so manual triggers and the
startup scan can't drift. When at least one container is stale, a
top-level `↻ UPD4TE 4LL` button appears that loops over every
stale container.
The dashboard shows no pending-update state: the hive applies a changed
declaration itself. The agent menu's rebuild queues the same
`job_queue::templates::rebuild` DAG, which downloads and applies the declared
closure again.

4
docs/agent-lifecycle/persistence.md
View file

@ -544,8 +544,8 @@ Isolation still holds by default: a container has its *own* dirs and,
unless it holds the permission, nothing else.
Under `/var/lib/hyperhive/applied/` — the hive-c0re-only deploy markers
`.<name>.hyperhive-rev` and `.<name>.config-rev`: the hyperhive and config
commits the agent's last successful build pinned. No config lives here; the
`.<name>.closure`: the store path of the closure the agent was last swapped to
or created from. No config lives here; the
forge's `agent-configs/<name>` is the only copy. A hive that predates this
finds per-agent `applied/<name>` repos here. Nothing reads them any more, and
you can delete them.

10
docs/integrations/forge.md
View file

@ -63,7 +63,6 @@ store certificate, every time it reads the forge. A hive whose store holds
none reads nothing. These reads go through it, and none of them needs a local
forge container:
- resolving the config commit `internal/meta` pins for a build
- cloning and pulling `/knowledge`
The hive builds nothing, so no nix process reads the forge as the hive's
@ -107,19 +106,12 @@ Two things live in the `agent-configs` Forgejo organization:
`main` directly nor self-merge. An operator's merge in the Forgejo UI
deploys the merged commit (see
[config-changes.md § Config changes](../agent-lifecycle/config-changes.md#config-changes)).
hive-c0re pushes nothing to this org; it only reads from it.
hive-c0re pushes nothing to this org and reads nothing from it.
Repos stay private, so an agent can't read another
agent's config. The agent's **persistent meta flake input** is this repo's `main` head,
pinned by sha in `internal/meta` (`agent-<n>.url =
git+<forge>/agent-configs/<n>.git?rev=<sha>`; see
[config-changes.md § Meta flake](../agent-lifecycle/config-changes.md#meta-flake)).
hive-c0re's own git reads of this repo, which resolve the sha, authenticate with the hive's read
token through `forge::reader_token()` (see
[How a hive reads the forge](#how-a-hive-reads-the-forge)).
- The dashboard links each container's deployed config commit to this
config repo on the swarm's forge (`swarm.forge.publicUrl`; `null` means no
link), so operators can go straight from the 4G3NTS tab into
the rendered repo without an extra `git` step.
The `hive-forge` CLI (separate workspace crate, see
[`README.md`](../README.md) file map) wraps the Forgejo REST API

50
docs/scheduler/coordinator.md
View file

@ -42,11 +42,11 @@ because there is no malformed spec to reject.
Nix-heavy — hold one of the `buildSlots` permits for the node's duration:
| Node | Wraps |
| ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `Prebuild` | download the swarm's closure for the agent into the store while the container keeps serving (its meta preamble is the upstream `MetaSync` node). With no closure to deploy it fails here, before the stop. Skipped when the container is already down; `Swap` downloads it instead |
| `Swap` | drop-in rewrite + profile-swap onto the swarm's closure (requires the container stopped); the post-swap bookkeeping tail lives in the sibling `RebuildBookkeeping` node |
| `Create` | first-spawn `nixos-container create` proper; assumes the upstream `Provision` node already registered the agent; downloads the closure the swarm built for the agent and creates the container from it (with none to deploy it fails and creates nothing), then marks the agent's pins |
| Node | Wraps |
| ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `Prebuild` | download the swarm's closure for the agent into the store while the container keeps serving (its meta preamble is the upstream `MetaSync` node). With no closure to deploy it fails here, before the stop. Skipped when the container is already down; `Swap` downloads it instead |
| `Swap` | drop-in rewrite + profile-swap onto the swarm's closure (requires the container stopped); the post-swap bookkeeping tail lives in the sibling `RebuildBookkeeping` node |
| `Create` | first-spawn `nixos-container create` proper; assumes the upstream `Provision` node already registered the agent; downloads the closure the swarm built for the agent and creates the container from it (with none to deploy it fails and creates nothing), then marks the closure applied |
Cheap — no build slot:
@ -54,7 +54,7 @@ Cheap — no build slot:
| Node | Behavior |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `MetaSync` | the rebuild's preamble — rebuild-dir prep, the agent-set refresh (`agent_set::refresh`), and the `internal/meta` commit `Prebuild` and `Swap` expect the swarm's closure to be built from, resolved once and recorded; fails when the repo doesn't pin the agent's current config. Declares no resources |
| `MetaSync` | the rebuild's preamble — rebuild-dir prep, the archetype refresh and the agent-set refresh (`agent_set::refresh`); reads nothing from the forge. Declares no resources |
| `Provision` | first-spawn pre-create provisioning — the state subvolume, agent-set refresh (`agent_set::refresh`); runs ahead of `Create`. Store-only, no container yet |
| `Reconcile` | idempotent power converge: read `wanted` (below) + observed state; start if `Up` & down (cold-start fallback included), stop if `Offline` & up, else noop |
| `Start` | mechanical container start — runtime dir + drop-ins, `start_with_fallback`, MCP listener registration, the manager kick. Fanned out by a `Reconcile` that observed `wanted = Up` and the container down |
@ -103,7 +103,7 @@ state-conditional — skipped for a _down_ agent (nothing to quiesce/stop). Keep
closes the TOCTOU window: if an agent flips state between the `is_running`
read and node exec, the tail `Reconcile` still converges it in-DAG (with
`StopForUpdate`-noop as the backstop) — no reliance on an external reconcile
sweep. `start` folds the per-agent stale-rev upgrade in (a _down + stale_
sweep. `start` folds the per-agent declared-closure upgrade in (a _down + stale_
agent's subgraph is a rebuild-then-start).
```text
@ -114,7 +114,7 @@ restart(a..): online a: [Signal→Drain→ if graceful] StopForUpdate(a) → Re
offline a: Reconcile(a) (nothing to stop; Reconcile converges to existing wanted)
start(a..): a: SetWanted(a,Up) → Reconcile(a) (down+stale ⇒ SetWanted(a,Up) → «rebuild subgraph»)
spawn(a): [wanted=Up at deploy] Provision(a) → Create(a) → WriteDropin(a) → Reconcile(a)
boot: «graceful rebuild subgraph» per stale agent (rev marker ≠ the `internal/meta` pin);
boot: «graceful rebuild subgraph» per stale agent (closure marker ≠ the record's store path);
plus Reconcile(a) for every drifted agent (all ONE DAG)
```
@ -285,25 +285,31 @@ render.
## Boot reconcile
On startup, `auto_update::run` classifies every agent by rev freshness (the
per-agent `.{name}.hyperhive-rev` marker under `/var/lib/hyperhive/applied/`
vs the hyperhive commit the head of `internal/meta` pins) and persisted
On startup, `auto_update::run` classifies every agent by closure freshness (the
per-agent `.{name}.closure` marker under `/var/lib/hyperhive/applied/` vs the
store path the agent's `agent-closures` record declares) and persisted
`wanted` intent, then:
1. **Config path** — submit one `Boot` DAG holding a graceful `Rebuild`
subgraph for each stale agent whose `wanted = Up` (name-sorted). This is how
a hive that missed the swarm's fan-out catches up. Stale but wanted-offline
a hive that missed the swarm's deploy request catches up. Stale but wanted-offline
agents get no boot-time nix work — their rebuild happens on their next start
(the start submit path upgrades a stale start to rebuild+start). Each
rebuild's tail `Reconcile` brings the agent (back) up, covering both the
running-stale and stopped-but-wanted-up cases. A hive that can't read the
pin (no forge token, the forge down, no `internal/meta` yet) rebuilds
nothing.
running-stale and stopped-but-wanted-up cases. An agent whose declaration
the hive can't read or compare (no queue connection, no record, a closure the
builder didn't push, no marker) rebuilds nothing, and neither does any
agent on the first boot after the marker arrived.
2. **Power path** — every agent whose observed state drifted from `wanted`
gets a plain `Reconcile` DAG (`kind = reconcile`, source `auto_update`).
Booting with no pin change performs no rebuild — only reconciles.
Booting with every agent on its declared closure performs no rebuild — only
reconciles. After the boot pass, `auto_update::run` repeats the rebuild half
every five minutes. It skips an agent with a rebuild waiting or running
(`JobQueue::rebuild_live`) and one whose declared store path this process
already attempted, so a closure that fails to deploy gets one try per process.
The boot log line records the rebuild / deferred / up-to-date counts so the
operator sees at a glance how much work the boot triggered. `auto_update::run`
seeds agents without an `agent_power` row from observed state during
@ -314,8 +320,9 @@ classification (the one-time migration; thereafter the DB is authoritative).
Agents build from the swarm's `internal/meta` flake, which swarm-controller
commits and every hive reads (see
[Config changes](../agent-lifecycle/config-changes.md#the-swarms-meta-flake)).
The hive keeps no flake of its own. `meta_rev.rs` picks the `internal/meta`
commit an agent builds from and records it; `agent_set.rs` keeps the
The hive keeps no flake of its own and deploys the closure the swarm builds
from it, as the agent's `agent-closures` record declares (`attic/closure.rs`).
`agent_set.rs` keeps the
host-side files that follow the agent set (`topology.json`, the gateway's
agent-socket map and nginx include) in line with the live agents, and
`meta.rs` removes the permission seed files once the swarm has them, and
@ -342,9 +349,8 @@ one go, rather than the double-bounce a live `update` would trigger.
Sequence for a rebuild DAG (each step is its own queue node):
1. `MetaSync` — rebuild-dir prep, the agent-set refresh, and the choice of the
`internal/meta` commit the swarm's closure should come from. Short,
and it declares no resources.
1. `MetaSync` — rebuild-dir prep, the archetype refresh and the agent-set
refresh. Short, and it declares no resources.
2. `Prebuild` — download the swarm's closure for the agent **before** stopping.
The container keeps serving the previous generation while the download
happens out-of-band. A missing or unpullable closure fails the DAG here,
@ -353,7 +359,7 @@ Sequence for a rebuild DAG (each step is its own queue node):
no uptime to preserve, so `Swap` downloads it.
3. `StopForUpdate` — bring the container down (noop when already stopped).
4. `Swap` — profile-swap onto the swarm's closure (near-instant after the
prebuild).
prebuild), then it writes the closure marker.
5. `Reconcile` — boot into the new generation when `wanted = Up`; the
in-container activation script transitions old → new. Holds no build
slot, so the next DAG's `Prebuild` overlaps the container boot.

20
docs/scheduler/observability.md
View file

@ -334,16 +334,16 @@ exists, so off-the-shelf OTEL/Grafana container dashboards work. Resource
`service.name = hyperhive-c0re`; hive-c0re tags each data point `container.name`
(= the `h-<agent>` machine) and the hive `agent` label:
| Metric | Unit | Kind | Source |
| --------------------------------------- | ---- | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `container.cpu.time` | `s` | counter | cumulative `cpu.stat` `usage_usec` → seconds |
| `container.memory.usage` | `By` | gauge | `memory.current` |
| `hyperhive.container.memory.limit` | `By` | gauge | `memory.max` (custom — semconv has no `.limit` metric; omitted when unlimited) |
| `hyperhive.container.memory.peak` | `By` | gauge | `memory.peak` (custom — no semconv metric; omitted if unavailable) |
| `hyperhive.container.storage.usage` | `By` | gauge | `state/` + `work/` + the root template (the live ephemeral layer isn't counted), measured by `hive-priv` every 30 min (custom — semconv only has `disk.io`; omitted while unknown: not measured yet, or the last measurement failed) |
| `hyperhive.container.cpu.percent` | `%` | gauge | host-normalised percent (custom — no `rate()` needed) |
| `hyperhive.agent.unrestricted_internet` | `1` | gauge | the agent's `unrestricted_internet` flag in the swarm permissions this hive last read: `1` on, `0` off; a flag never set reads `1`. Omitted while the stored permissions file is unreadable |
| `hyperhive.agent.closure_pullable` | `1` | gauge | `1` when this hive can fetch the agent's latest closure from the swarm's binary cache and verify its signature, else `0` with a `reason` label (`no-record`, `rev-mismatch`, `not-cached`, `no-credentials`, `fetch-failed`, `bad-signature`; see [the builder docs](../swarm/builder.md#what-hives-do-with-the-cache)). Omitted before the first probe pass, every 5 minutes |
| Metric | Unit | Kind | Source |
| --------------------------------------- | ---- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `container.cpu.time` | `s` | counter | cumulative `cpu.stat` `usage_usec` → seconds |
| `container.memory.usage` | `By` | gauge | `memory.current` |
| `hyperhive.container.memory.limit` | `By` | gauge | `memory.max` (custom — semconv has no `.limit` metric; omitted when unlimited) |
| `hyperhive.container.memory.peak` | `By` | gauge | `memory.peak` (custom — no semconv metric; omitted if unavailable) |
| `hyperhive.container.storage.usage` | `By` | gauge | `state/` + `work/` + the root template (the live ephemeral layer isn't counted), measured by `hive-priv` every 30 min (custom — semconv only has `disk.io`; omitted while unknown: not measured yet, or the last measurement failed) |
| `hyperhive.container.cpu.percent` | `%` | gauge | host-normalised percent (custom — no `rate()` needed) |
| `hyperhive.agent.unrestricted_internet` | `1` | gauge | the agent's `unrestricted_internet` flag in the swarm permissions this hive last read: `1` on, `0` off; a flag never set reads `1`. Omitted while the stored permissions file is unreadable |
| `hyperhive.agent.closure_pullable` | `1` | gauge | `1` when this hive can fetch the agent's latest closure from the swarm's binary cache and verify its signature, else `0` with a `reason` label (`no-record`, `not-cached`, `no-credentials`, `fetch-failed`, `bad-signature`; see [the builder docs](../swarm/builder.md#what-hives-do-with-the-cache)). Omitted before the first probe pass, every 5 minutes |
The `hyperhive.`-prefixed metrics have no semconv equivalent (memory
limit + peak, on-disk footprint, and an instantaneous cpu percent kept

6
docs/swarm/archetypes.md
View file

@ -127,12 +127,10 @@ swarm-controller renders `internal/meta`'s `flake.nix` with the
The hive builds from `internal/meta` alone. It reads the agent's key when it
provisions the agent and on every rebuild, and keeps the last value read in
`/var/lib/hyperhive/swarm-agent-archetypes.json`. It takes the head of
`internal/meta` only when that `flake.nix` lists the agent at the archetype
it has now. When the read fails, a rebuild keeps the stored value. For an
`/var/lib/hyperhive/swarm-agent-archetypes.json`. When the read fails, a rebuild keeps the stored value. For an
archetype agent the hive also:
- ignores the `rev` of a swarm deploy request.
- deploys the closure its record declares, as every agent does.
The agent's system prompt says it has no config repo and names its
archetype.

16
docs/swarm/builder.md
View file

@ -462,12 +462,11 @@ decides the reason:
| ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `no-credentials` | One of the four leaves is missing or unusable (a URL that isn't `https` is unusable). The swarm has no cache, the controller is too old to copy them, or you haven't seeded them yet. |
| `no-record` | `agent-closures` holds no build for the agent. |
| `rev-mismatch` | The record's build came from another `internal/meta` commit than its head that pins the agent's hyperhive or config differently, or the hive can't read the head. |
| `not-cached` | The record says the builder didn't push the closure, or the cache answers 404 for the narinfo. |
| `fetch-failed` | The request failed, or the cache answered with a status other than 200 or 404 (a refused token is a 401 or 403). |
| `bad-signature` | The narinfo describes another path, or no signature by the public key covers it. |
A failed read of the store, the queue or the forge leaves the last result in
A failed read of the store or the queue leaves the last result in
place and raises a warning after three failed passes. A hive reports nothing
before its first pass.
@ -502,15 +501,14 @@ store, and gives `hive-priv` the store path to apply. A stopped agent skips
`Prebuild`, so `Swap` downloads it. A first deploy's `Create` node does the
same and creates the container from that path.
A rebuild or create has three outcomes, by what the lookup and the download find:
A rebuild or create has two outcomes, by what the lookup and the download find. The hive deploys the record's closure whatever `internal/meta` commit the builder built it from, and reads no forge head to choose one:
| Found | What happens |
| ----------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| a record that the builder pushed, for the commit this hive chose | The hive downloads the closure and deploys it. The hive logs `prebuild: substituted <path>` (`create: substituted <path>` for a create) at info level. |
| a record that the builder pushed, for another commit (the hive and the builder disagree) | The hive downloads the builder's closure and deploys it, with a warning that names the agent, both commits and the path. The hive evaluates nothing, and records the pins of the commit the builder built the closure from, so the agent's markers say what runs. |
| no record, or one the hive can't use (not pushed, queue or cache unreachable, download or signature failed) | The rebuild or create fails with an error that names the agent and the reason, logged at error level, and nothing changes. A running container keeps running unchanged, because `Prebuild` fails before `StopForUpdate`. A create makes no container. |
| Found | What happens |
| ----------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| a record that the builder pushed | The hive downloads the closure and deploys it, then writes the store path to the agent's `applied/.<name>.closure` marker. The hive logs `prebuild: substituted <path>` (`create: substituted <path>` for a create) at info level. |
| no record, or one the hive can't use (not pushed, queue or cache unreachable, download or signature failed) | The rebuild or create fails with an error that names the agent and the reason, logged at error level, and nothing changes. A running container keeps running unchanged, because `Prebuild` fails before `StopForUpdate`. A create makes no container. |
The hive never falls back to a local build. The third row has one owner in the
The hive never falls back to a local build. The second row has one owner in the
code, `without_closure` in `hive-c0re/src/job_queue/exec.rs`, so a change to what
an unreachable cache costs is one function.

7
docs/tools/hivectl.md
View file

@ -44,11 +44,10 @@ hivectl agent iris resume # let it drive turns again, draining what que
```
`list-agents` prints a padded table with one row per managed agent —
`NAME STATUS REV REMIND`. STATUS collapses the health flags
(`running` / `stopped`, plus ` paused` / ` needs-login` / ` needs-update`
`NAME STATUS REMIND`. STATUS collapses the health flags
(`running` / `stopped`, plus ` paused` / ` needs-login`
when set — `paused` is orthogonal to running, see below);
REV is the first 12 chars of the agent's locked config sha; REMIND is the
count of pending reminders. It reuses the same per-agent aggregation the dashboard
REMIND is the count of pending reminders. It reuses the same per-agent aggregation the dashboard
renders, so the CLI roster and the web UI never drift. `--json` emits the
raw rows instead of the table.

6
docs/web-ui/README.md
View file

@ -48,9 +48,9 @@ See [`config-changes.md`](../agent-lifecycle/config-changes.md#config-changes).
**Start, stop, restart, or rebuild an agent.** Select one or more
agents on 4G3NTS (select the icon) and use the selection bar, or use the
per-agent `⋮` menu on a single row. Rebuilding re-applies that agent's
current config; use it whenever an agent
shows as "needs update."
per-agent `⋮` menu on a single row. Rebuilding downloads and applies the closure the swarm
declares for the agent again. The hive applies a changed declaration itself,
so use it to retry one that failed.
**Watch a build.** The dashboard's R3BU1LD QU3U3 tab shows the rebuild queue live, plus a
streaming log of whatever's currently building. Useful right after

20
docs/web-ui/dashboard.md
View file

@ -65,9 +65,6 @@ one of them and rebuild. `lifecycle::{spawn,rebuild}` also
preflight this and refuse with a clear error message naming the
conflicting agent.
`↻ UPD4TE 4LL` button appears above the containers list when any
agent is stale.
## K3PT ST4T3 tab
Passive / rare-interaction state. `kept.js` renders it from the
@ -280,8 +277,7 @@ web server is down), so a single badge replaces the badge chain,
the nav strip is empty, and status text / rate-limited / ctx
badges disappear. The agent icon goes straight to the dimmed
`/favicon.svg` fallback instead of attempting a doomed load from the
container's URL. Static fields — `needs_update`, `deployed_sha`,
`pending_reminders`, `parent`, `config` link — remain visible
container's URL. Static fields — `pending_reminders`, `parent`, `config` link — remain visible
regardless of run state.
<!-- vale write-good.Passive = YES -->
@ -296,7 +292,7 @@ below).
restarts and gave up on its own) draws a red `✖ gave up` badge;
otherwise a plain muted `■ not running` — a container an operator
stopped deliberately. Both states read `running: false`; `failed` is
the orthogonal fact (a fifth one alongside `paused`/`needs_update`/
the orthogonal fact (a fourth one alongside `paused`/
`needs_login`, same "independent flags, no state machine" shape —
see `ContainerView`'s own doc comment) that tells them apart. An
older hive-c0re without the field serves `failed: undefined`, which
@ -304,14 +300,14 @@ reads falsy — degrades cleanly to the single `not running` badge.
When the container is running, status badges follow — `⊘ rate
limited` (red, while the harness parks itself after a 429), `needs
login`, `needs update` — plus **one `◐ pending-state…` pill per
login` — plus **one `◐ pending-state…` pill per
active transient** (replaces buttons during operator-initiated
start / stop / restart / rebuild / destroy). An agent can carry
**multiple transients at once** — for example a lease-exempt `prebuild`
running alongside a `stop_for_update` on the same agent — and each
renders as its own independent badge instead of collapsing
into one label, matching the existing multi-badge convention this
line already uses for `paused`/`needs_update`/model/ctx.
line already uses for `paused`/model/ctx.
Any pending badge means the row is actually **running** something
right now (see **Pending-badge derivation** below); every row
@ -377,13 +373,6 @@ per-agent actions and navigation links. Contents:
- `↻ R3BU1LD` — always available; queues a rebuild for this agent.
- `DESTR0Y` / `PURG3` — destructive, each prompts for confirmation
via the themed dialog (see **Themed dialogs** below).
- `deployed:<sha> ↗` — present when the agent has a `deployed_sha`
and the forge is reachable; links the deployed commit on the forge
agent-configs mirror.
`↻ UPD4TE 4LL` button appears above the containers list when any
agent is stale. Banner pulses on each broker SSE event
(`pulseBanner` with a 4s grace timer).
**Build-queue summary banner** — when the job queue has any active
work, a compact amber banner sits above the container list: `◐ build
@ -491,7 +480,6 @@ below — some endpoints aren't in it yet.
- `POST /api/{rebuild,kill,restart,start,destroy}/{name}` — lifecycle.
`destroy` accepts `purge=on` to also wipe state dirs.
- `POST /api/purge-tombstone/{name}` — wipe a tombstone's state dirs.
- `POST /api/update-all` — rebuild every stale container.
- `POST /api/rebuild-queue/{id}/cancel` — drop a `Queued` entry.
Refuses `Running` / terminal-state entries (in-flight
rebuilds can't be safely interrupted). Always 200; body is

20
frontend/packages/dashboard/src/agent-menu/hive-agent-menu.js
View file

@ -72,7 +72,7 @@ class HiveAgentMenu extends HTMLElement {
// first, doubling the dropdown.
if (this._menu) return;
const { c, forgeBase } = this._opts || {};
const { c } = this._opts || {};
const btn = el(
"button",
@ -206,24 +206,6 @@ class HiveAgentMenu extends HTMLElement {
}),
);
if (c.deployed_sha && forgeBase) {
const li = el("li", { role: "presentation" });
const a = el(
"a",
{
class: "agent-menu-item",
href: `${forgeBase}/agent-configs/${encodeURIComponent(c.name)}/commit/${c.deployed_sha}`,
target: "_blank",
rel: "noopener",
role: "menuitem",
title: "deployed config commit on forge",
},
`deployed:${c.deployed_sha} ↗`,
);
li.append(a);
dropdown.append(menuSep(), li);
}
this._menu = document.createElement("hive-menu");
this._menu._opts = { trigger: btn, content: dropdown };
this.append(this._menu);

34
frontend/packages/dashboard/src/agents.js
View file

@ -196,9 +196,9 @@ document.addEventListener("click", (e) => {
// wrapper matching <hive-dialog>'s `._opts`-before-append convention,
// since a custom element created via `document.createElement` can't
// take constructor args.
function buildAgentMenu(c, forgeBase) {
function buildAgentMenu(c) {
const menu = document.createElement("hive-agent-menu");
menu._opts = { c, forgeBase };
menu._opts = { c };
return menu;
}
@ -255,7 +255,6 @@ function containerRowFingerprint(
failed: c.failed,
paused: c.paused,
needs_login: c.needs_login,
needs_update: c.needs_update,
active_model: c.active_model,
port: c.port,
pending,
@ -545,19 +544,6 @@ function buildContainerLi(c, opts) {
),
);
}
if (c.needs_update) {
head.append(
form(
"/api/rebuild/" + c.name,
"hive-pill-sm badge-warn btn-inline",
"needs update ↻",
"rebuild " + c.name + "? hot-reloads the container.",
{},
{ noRefresh: true },
),
);
}
if (c.active_model) {
head.append(
el(
@ -579,7 +565,7 @@ function buildContainerLi(c, opts) {
// chip in the head row stays — it's a state-hint, not an
// action button.
li.append(icon, body, buildAgentMenu(c, forgeBase));
li.append(icon, body, buildAgentMenu(c));
return li;
}
@ -601,7 +587,6 @@ export function renderContainers(s) {
// tree builder handles the ordering — we don't pre-sort here.
const containers = Array.from(containersState.values());
const portConflicts = derivePortConflicts(containers);
const anyStale = containers.some((c) => c.needs_update);
// Preserve the keyed container list across the header-section
// rebuild. We wipe the banners/buttons above the list on every
@ -627,19 +612,6 @@ export function renderContainers(s) {
root.append(banner);
}
if (anyStale) {
root.append(
form(
"/api/update-all",
"btn-rebuild",
"↻ UPD4TE 4LL",
"rebuild every stale container?",
{},
{ noRefresh: true },
),
);
}
// Queue-summary banner: see initJobqRollup, above.
if (!containers.length && !transientsState.size) {

2
hive-agent/src/web_ui/state.rs
View file

@ -82,7 +82,7 @@ pub(super) async fn api_state(State(state): State<AppState>) -> axum::Json<State
/// fetches this once per running agent to get fresh, agent-authoritative
/// values instead of relying on hive-c0re's periodic file-reads.
///
/// Structural fields (running, `needs_update`, `deployed_sha`, parent, …)
/// Structural fields (running, `needs_login`, parent, …)
/// continue to come from hive-c0re's `/api/state`; this endpoint covers
/// only the fields the agent itself is the source of truth for.
pub(super) async fn api_dashboard_state(

1
hive-c0re/Cargo.toml
View file

@ -22,7 +22,6 @@ base64.workspace = true
getrandom.workspace = true
sha-crypt.workspace = true
reqwest.workspace = true
forgejo-api.workspace = true
url.workspace = true
clap.workspace = true
clap_complete.workspace = true

3
hive-c0re/README.md
View file

@ -41,8 +41,7 @@ to track it:
per-agent sockets.
- **`forge/`** — optional Forgejo wiring (`docs/integrations/forge.md`).
- **`coordinator.rs`** — top-level wiring for `serve`.
- **`meta_rev.rs`** — which `internal/meta` commit an agent builds from, and
the markers a successful build writes. **`agent_set.rs`** — the files that
- **`agent_set.rs`** — the files that
follow the agent set (topology, gateway map). **`meta.rs`** — the commit
that removes the legacy permission seed files. **`migrate.rs`** —
schema/state migrations.

9
hive-c0re/src/agent_config/archetypes.rs
View file

@ -33,15 +33,6 @@ pub fn read_at(path: &Path) -> std::io::Result<Stored> {
super::read_map(path)
}
/// `agent`'s archetype per the file at [`path`], `None` for a custom agent.
///
/// # Errors
///
/// As [`read_at`].
pub fn of(agent: &str) -> std::io::Result<Option<ArchetypeRef>> {
Ok(read_at(&path())?.remove(agent))
}
/// Read `agent`'s key and store what it holds. No queue, or a bucket that
/// does not answer, leaves the stored entry as it is.
///

168
hive-c0re/src/attic/closure.rs
View file

@ -1,7 +1,10 @@
//! The closure the swarm built for an agent, as a deploy reads its record
//! before downloading it.
//! The closure the swarm declares for an agent: its record in `agent-closures`
//! read before a deploy downloads it, and the marker of the store path the
//! hive last applied, which a changed declaration is compared with.
use anyhow::{Result, anyhow};
use std::path::Path;
use anyhow::{Context as _, Result, anyhow};
use swarm_wire::agent_closures::AgentClosure;
use super::probe;
@ -17,3 +20,162 @@ pub async fn record(agent: &str) -> Result<Option<AgentClosure>> {
.ok_or_else(|| anyhow!("this hive has no connection to the swarm queue"))?;
probe::read_record(&client, agent).await
}
/// Record that `agent` runs the closure at `store_path`.
///
/// # Errors
/// The marker could not be written.
pub fn mark_deployed(agent: &str, store_path: &str) -> Result<()> {
mark_deployed_in(&crate::paths::applied_closure_marker(agent), store_path)
}
/// The store path `agent` was last swapped to; `None` when the hive has no
/// record of one: it never applied a closure, or last did before the marker
/// existed.
#[must_use]
pub fn deployed(agent: &str) -> Option<String> {
deployed_in(&crate::paths::applied_closure_marker(agent))
}
/// Written whole to a temp file beside the marker and renamed over it, so a
/// reader sees the old marker or the new one.
fn mark_deployed_in(marker: &Path, store_path: &str) -> Result<()> {
let tmp = marker.with_extension("tmp");
if let Some(dir) = marker.parent() {
std::fs::create_dir_all(dir).with_context(|| format!("create {}", dir.display()))?;
}
std::fs::write(&tmp, store_path).with_context(|| format!("write {}", tmp.display()))?;
std::fs::rename(&tmp, marker)
.with_context(|| format!("rename {} to {}", tmp.display(), marker.display()))
}
fn deployed_in(marker: &Path) -> Option<String> {
let stored = std::fs::read_to_string(marker).ok()?;
let stored = stored.trim();
(!stored.is_empty()).then(|| stored.to_owned())
}
/// Whether the swarm's declared closure is one the hive should now deploy: the
/// builder pushed it and it differs from what `deployed` says runs.
///
/// An agent with no `deployed` marker is not pending, since the hive cannot
/// tell what it runs: every agent starts that way when the hive first reads
/// this marker, and deploying all of them at once is worse than waiting for the
/// swarm's next deploy request, which writes the marker.
#[must_use]
pub fn pending(record: &AgentClosure, deployed: Option<&str>) -> bool {
record.cached && deployed.is_some_and(|deployed| deployed != record.store_path)
}
/// Whether `deployed` is the closure `record` declares, so deploying it again
/// would change nothing.
#[must_use]
pub fn up_to_date(record: &AgentClosure, deployed: Option<&str>) -> bool {
deployed == Some(record.store_path.as_str())
}
/// The store path the swarm declares for `agent` when it is [`pending`]. A record
/// that cannot be read, or none, is logged and counts as nothing pending: the
/// hive then deploys nothing it was not told to.
pub async fn pending_path(agent: &str) -> Option<String> {
match record(agent).await {
Ok(Some(record)) => {
pending(&record, deployed(agent).as_deref()).then_some(record.store_path)
}
Ok(None) => None,
Err(e) => {
tracing::warn!(%agent, error = %format!("{e:#}"), "cannot read the agent's declared closure");
None
}
}
}
/// Whether the closure the swarm declares for `agent` is the one it runs; false
/// when that cannot be told, so a deploy request goes ahead.
pub async fn is_up_to_date(agent: &str) -> bool {
match record(agent).await {
Ok(Some(record)) => up_to_date(&record, deployed(agent).as_deref()),
_ => false,
}
}
#[cfg(test)]
mod tests {
use super::*;
const DECLARED: &str = "/nix/store/0123456789abcdfghijklmnpqrsvwxyz-nixos-system-alice";
const RUNNING: &str = "/nix/store/zyxwvsrqpnmlkjihgfdcbadfhijklmn-nixos-system-alice";
fn declared(cached: bool) -> AgentClosure {
AgentClosure {
meta_rev: "a".repeat(40),
store_path: DECLARED.to_owned(),
system: "x86_64-linux".to_owned(),
built_unix: 1,
cached,
private_users: false,
}
}
#[test]
fn a_declaration_other_than_the_running_closure_is_pending() {
assert!(pending(&declared(true), Some(RUNNING)));
}
#[test]
fn the_running_closure_is_not_pending() {
assert!(!pending(&declared(true), Some(DECLARED)));
}
#[test]
fn a_closure_the_builder_did_not_push_is_not_pending() {
assert!(!pending(&declared(false), Some(RUNNING)));
}
#[test]
fn only_the_declared_closure_is_up_to_date() {
assert!(up_to_date(&declared(true), Some(DECLARED)));
assert!(!up_to_date(&declared(true), Some(RUNNING)));
assert!(!up_to_date(&declared(true), None), "unknown is deployed");
}
#[test]
fn an_agent_with_no_marker_is_not_pending() {
assert!(!pending(&declared(true), None));
}
#[test]
fn the_marker_holds_the_store_path_and_is_replaced_whole() {
let dir = tempfile::tempdir().unwrap();
let marker = dir.path().join("applied").join(".alice.closure");
assert_eq!(deployed_in(&marker), None, "no marker yet");
mark_deployed_in(&marker, RUNNING).unwrap();
assert_eq!(deployed_in(&marker).as_deref(), Some(RUNNING));
mark_deployed_in(&marker, DECLARED).unwrap();
assert_eq!(deployed_in(&marker).as_deref(), Some(DECLARED));
}
/// A hive that ran the previous marker format has a config commit in its
/// old files and none at the new path: the agent is not pending, and the
/// old files are not read or touched.
#[test]
fn markers_in_the_old_format_are_neither_read_nor_removed() {
let dir = tempfile::tempdir().unwrap();
let old = dir.path().join(".alice.config-rev");
std::fs::write(&old, "1111111111111111111111111111111111111111").unwrap();
let marker = dir.path().join(".alice.closure");
assert_eq!(deployed_in(&marker), None);
assert!(!pending(&declared(true), deployed_in(&marker).as_deref()));
assert!(old.exists());
mark_deployed_in(&marker, DECLARED).unwrap();
assert!(old.exists(), "a successful deploy leaves old markers alone");
}
#[test]
fn an_empty_marker_is_unknown() {
let dir = tempfile::tempdir().unwrap();
let marker = dir.path().join(".alice.closure");
std::fs::write(&marker, "\n").unwrap();
assert_eq!(deployed_in(&marker), None);
}
}

146
hive-c0re/src/attic/probe.rs
View file

@ -7,9 +7,7 @@
//! [`super::closure`].
//!
//! The verdict is the first of these that fails, in this order:
//! no credentials (`no-credentials`), no record (`no-record`), a record built
//! from another `internal/meta` commit than the head that pins the agent
//! differently (`rev-mismatch`), a record
//! no credentials (`no-credentials`), no record (`no-record`), a record
//! the builder did not push (`not-cached`), the cache not holding the narinfo
//! (`not-cached`), a request that failed or was refused (`fetch-failed`), a
//! narinfo no signature of the key covers (`bad-signature`).
@ -35,7 +33,6 @@ const FAILURES_BEFORE_WARNING: u32 = 3;
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Reason {
NoRecord,
RevMismatch,
NotCached,
NoCredentials,
FetchFailed,
@ -47,7 +44,6 @@ impl Reason {
pub const fn label(self) -> &'static str {
match self {
Reason::NoRecord => "no-record",
Reason::RevMismatch => "rev-mismatch",
Reason::NotCached => "not-cached",
Reason::NoCredentials => "no-credentials",
Reason::FetchFailed => "fetch-failed",
@ -84,26 +80,10 @@ impl Cache for Http {
}
}
/// `record` when it was built from the `internal/meta` commit `head`, or from
/// one that pins the agent as `head` does (`same_pins`), and the builder pushed
/// it to the cache.
pub(crate) fn current_record<'a>(
record: Option<&'a AgentClosure>,
head: Option<&str>,
same_pins: bool,
) -> Result<&'a AgentClosure, Unpullable> {
/// `record` when the builder pushed it to the cache.
pub(crate) fn current_record(record: Option<&AgentClosure>) -> Result<&AgentClosure, Unpullable> {
let record =
record.ok_or_else(|| no(Reason::NoRecord, "no build is recorded for the agent"))?;
if head != Some(record.meta_rev.as_str()) && !same_pins {
return Err(no(
Reason::RevMismatch,
format!(
"built from {}, internal/meta is at {}",
record.meta_rev,
head.unwrap_or("nothing")
),
));
}
if !record.cached {
return Err(no(
Reason::NotCached,
@ -113,14 +93,10 @@ pub(crate) fn current_record<'a>(
Ok(record)
}
/// Whether the closure `record` names can be pulled with `facts`. `head` is the
/// `internal/meta` commit a deploy would build from; `same_pins` is whether the
/// record's commit pins the agent as `head` does.
/// Whether the closure `record` names can be pulled with `facts`.
pub(crate) async fn judge(
facts: Option<&Facts>,
record: Option<&AgentClosure>,
head: Option<&str>,
same_pins: bool,
cache: &impl Cache,
) -> Result<(), Unpullable> {
let facts = facts.ok_or_else(|| {
@ -129,7 +105,7 @@ pub(crate) async fn judge(
"the swarm has given this hive no cache facts",
)
})?;
let record = current_record(record, head, same_pins)?;
let record = current_record(record)?;
let text = match cache.narinfo(facts, &record.store_path).await {
Ok(Some(text)) => text,
Ok(None) => return Err(no(Reason::NotCached, "the cache does not hold the narinfo")),
@ -182,7 +158,7 @@ pub async fn follow(client: async_nats::Client, mut shutdown: tokio::sync::watch
/// One pass: publish the facts, then judge every agent.
///
/// # Errors
/// A read that failed (the store, the queue, the forge), which leaves the last
/// A read that failed (the store, the queue), which leaves the last
/// verdicts as they are: the failure says nothing about any agent.
async fn pass(client: &async_nats::Client) -> Result<()> {
swarm_queue_client::ensure_connected(client)?;
@ -190,10 +166,6 @@ async fn pass(client: &async_nats::Client) -> Result<()> {
let agents = crate::lifecycle::agents_for_meta_listing()
.await
.context("enumerating this hive's agents")?;
let head = match &facts {
Some(_) => meta_head().await?,
None => None,
};
let http = Http(narinfo::client()?);
let mut now = Vec::with_capacity(agents.len());
for spec in agents {
@ -201,18 +173,7 @@ async fn pass(client: &async_nats::Client) -> Result<()> {
Some(_) => read_record(client, &spec.name).await?,
None => None,
};
let same_pins = match (&record, head.as_deref()) {
(Some(r), Some(head)) => pins_unchanged(&spec.name, &r.meta_rev, head).await,
_ => false,
};
let verdict = judge(
facts.as_ref(),
record.as_ref(),
head.as_deref(),
same_pins,
&http,
)
.await;
let verdict = judge(facts.as_ref(), record.as_ref(), &http).await;
now.push((spec.name, verdict.as_ref().err().map(|u| u.reason)));
report(&now[now.len() - 1], verdict.err().map(|u| u.detail));
}
@ -222,26 +183,6 @@ async fn pass(client: &async_nats::Client) -> Result<()> {
Ok(())
}
/// Whether `agent` has the same pins at `built` as at `head`. A commit that
/// cannot be read counts as different, which leaves the verdict to the commit
/// ids.
async fn pins_unchanged(agent: &str, built: &str, head: &str) -> bool {
if built == head {
return true;
}
let read = async {
let archetype = crate::archetypes::of(agent)?.is_some();
let source = crate::forge::internal_meta_source()
.await
.context("no forge token")?;
crate::meta_rev::pins_unchanged(&source, agent, archetype, built, head).await
};
read.await.unwrap_or_else(|e| {
tracing::debug!(%agent, error = %format!("{e:#}"), "attic: cannot compare the agent's pins");
false
})
}
pub(crate) async fn read_record(
client: &async_nats::Client,
agent: &str,
@ -256,19 +197,6 @@ pub(crate) async fn read_record(
}
}
/// The `internal/meta` head, or `None` when the swarm has not published it or
/// this hive has no forge token.
async fn meta_head() -> Result<Option<String>> {
use crate::meta_rev::MetaSource as _;
match crate::forge::internal_meta_source().await {
Some(source) => source
.meta_head()
.await
.context("reading internal/meta's head"),
None => Ok(None),
}
}
/// The log line: at info when the verdict differs from the last pass, else
/// debug.
fn report((agent, verdict): &(String, Option<Reason>), detail: Option<String>) {
@ -341,13 +269,9 @@ mod tests {
async fn reason(
facts: Option<&Facts>,
record: Option<&AgentClosure>,
head: Option<&str>,
cache: Answers,
) -> Option<Reason> {
judge(facts, record, head, false, &cache)
.await
.err()
.map(|u| u.reason)
judge(facts, record, &cache).await.err().map(|u| u.reason)
}
fn good() -> Answers {
@ -355,73 +279,37 @@ mod tests {
}
#[tokio::test]
async fn a_signed_narinfo_for_the_head_build_is_pullable() {
assert_eq!(
reason(Some(&facts()), Some(&record()), Some(REV), good()).await,
None
);
async fn a_signed_narinfo_for_the_recorded_build_is_pullable() {
assert_eq!(reason(Some(&facts()), Some(&record()), good()).await, None);
}
#[tokio::test]
async fn no_facts_is_no_credentials_even_with_a_record() {
let r = reason(None, Some(&record()), Some(REV), good()).await;
let r = reason(None, Some(&record()), good()).await;
assert_eq!(r, Some(Reason::NoCredentials));
}
#[tokio::test]
async fn no_record_is_no_record() {
let r = reason(Some(&facts()), None, Some(REV), good()).await;
let r = reason(Some(&facts()), None, good()).await;
assert_eq!(r, Some(Reason::NoRecord));
}
#[tokio::test]
async fn a_record_from_another_commit_is_a_rev_mismatch() {
let r = reason(Some(&facts()), Some(&record()), Some("other"), good()).await;
assert_eq!(r, Some(Reason::RevMismatch));
let r = reason(Some(&facts()), Some(&record()), None, good()).await;
assert_eq!(r, Some(Reason::RevMismatch), "no head to compare with");
}
#[tokio::test]
async fn a_record_from_another_commit_with_the_same_pins_is_pullable() {
let (facts, record) = (facts(), record());
let judged = judge(Some(&facts), Some(&record), Some("other"), true, &good()).await;
assert!(judged.is_ok());
let unpushed = AgentClosure {
cached: false,
..record
};
let judged = judge(Some(&facts), Some(&unpushed), Some("other"), true, &good()).await;
assert_eq!(judged.err().map(|u| u.reason), Some(Reason::NotCached));
}
#[tokio::test]
async fn an_unpushed_record_or_a_missing_narinfo_is_not_cached() {
let unpushed = AgentClosure {
cached: false,
..record()
};
let r = reason(Some(&facts()), Some(&unpushed), Some(REV), good()).await;
let r = reason(Some(&facts()), Some(&unpushed), good()).await;
assert_eq!(r, Some(Reason::NotCached));
let r = reason(
Some(&facts()),
Some(&record()),
Some(REV),
Answers(Ok(None)),
)
.await;
let r = reason(Some(&facts()), Some(&record()), Answers(Ok(None))).await;
assert_eq!(r, Some(Reason::NotCached));
}
#[tokio::test]
async fn a_failed_request_is_fetch_failed() {
let r = reason(
Some(&facts()),
Some(&record()),
Some(REV),
Answers(Err("401")),
)
.await;
let r = reason(Some(&facts()), Some(&record()), Answers(Err("401"))).await;
assert_eq!(r, Some(Reason::FetchFailed));
}
@ -430,7 +318,7 @@ mod tests {
let forged = Answers(Ok(Some(narinfo_signed_by(&SigningKey::from_bytes(
&[9; 32],
)))));
let r = reason(Some(&facts()), Some(&record()), Some(REV), forged).await;
let r = reason(Some(&facts()), Some(&record()), forged).await;
assert_eq!(r, Some(Reason::BadSignature));
}
@ -438,7 +326,6 @@ mod tests {
fn the_labels_are_the_documented_ones() {
let labels = [
Reason::NoRecord,
Reason::RevMismatch,
Reason::NotCached,
Reason::NoCredentials,
Reason::FetchFailed,
@ -449,7 +336,6 @@ mod tests {
labels,
[
"no-record",
"rev-mismatch",
"not-cached",
"no-credentials",
"fetch-failed",

Some files were not shown because too many files have changed in this diff Show more