hive-c0re: remove the approvals subsystem
Nothing submits an approval since scheduled prompts left hive-c0re, so the
queue only acted on rows written before that.
- hive-c0re: the approvals store and its migrations, approve/deny/
resolve actions, the ResolveApproval tail node (and meta_update's
approval_id), POST /api/approve and /api/deny, the approval views on
/api/state, the approval_resolved dashboard event, the orphan GC,
destroy/purge failing pending rows, and the approval rows in
get_loose_ends. DestroyBookkeeping loses the purge flag, which only
worded the approval-failure note.
- The CancelLooseEnd request goes: its only hive-c0re arm was the
approval one. cancel_loose_end handles reminder and todo in the harness.
- Wire types: hive_sh4re::approvals (ReminderStats moves to inbox),
LooseEnd::Approval, CancelLooseEndKind, HelperEvent::ApprovalResolved,
ToolGroup::Approvals, HostRequest::{Pending,Approve,Deny} and
HostResponse.approvals. HOST_SOCK_VERSION goes to 2 for the removed
field.
- hivectl loses the approvals verbs; docs/tools/hivectl-cli.md is
regenerated.
- The system prompt drops approval_resolved.
- The approvals table stays in broker.sqlite, unread, the same way the
operator_questions table was left when its store was removed.
- An `approvals` entry still in tool-groups.json is skipped with a warning
by ToolGroup::parse_list, like any unknown group.
- docs/agent-lifecycle/approvals.md is renamed to config-changes.md so the
code's doc pointers resolve; its content follows in the docs commit.
This commit is contained in:
atlas
parent
99bd912ae0
commit
58ef299c3c
60 changed files with 153 additions and 1828 deletions
|
|
@ -14,7 +14,7 @@ docs/tools/swarm-logs-cli.md
|
|||
# indentation normalisation, so these files need manual re-wrap before
|
||||
# adding them back to the formatter scope.
|
||||
CLAUDE.md
|
||||
docs/agent-lifecycle/approvals.md
|
||||
docs/agent-lifecycle/config-changes.md
|
||||
docs/process/conventions.md
|
||||
docs/networking/gateway.md
|
||||
docs/integrations/matrix.md
|
||||
|
|
|
|||
502
docs/agent-lifecycle/config-changes.md
Normal file
502
docs/agent-lifecycle/config-changes.md
Normal file
|
|
@ -0,0 +1,502 @@
|
|||
# Approvals + helper events
|
||||
|
||||
The approval queue is where an agent asks the operator for something it
|
||||
can't do itself. Nothing submits a new approval today: the one kind
|
||||
left, a meta-input bump, only reads back from existing rows. Config
|
||||
changes don't go through this queue: they're pull requests
|
||||
on the agent's config repo, which an operator merges on the forge, and
|
||||
that merge deploys them. The submitting agent — any agent with the
|
||||
`approvals` tool group, which manages the config of its **direct
|
||||
children** (the root agent for top-level agents; a sub-manager for its
|
||||
own subtree) — is the policy gate in front of both; helper events are how
|
||||
it stays informed about what happens after a decision lands.
|
||||
|
||||
## For operators
|
||||
|
||||
- **Config change** — an agent proposed a change to another agent's
|
||||
config (or its own, via a sub-manager) as a pull request on
|
||||
`agent-configs/<agent>`. Review it on the forge like any other PR, and
|
||||
merge it there to deploy it. See [Config changes](#config-changes).
|
||||
- **Meta/flake update** (`UpdateMetaInputs`) — an existing row reads
|
||||
back and you can approve it. Approving runs the update and commits
|
||||
the lock change; it doesn't rebuild anything by itself.
|
||||
|
||||
Don't want to approve something? **Deny it** (`DENY` on the dashboard
|
||||
card, or `hivectl approvals deny <id>`) — nothing runs. Either way the
|
||||
submitting agent is always notified that the operator denied their request; what's
|
||||
optional is only the reason text, which you can add on the dashboard's
|
||||
prompt (cancelling that prompt aborts the whole deny, not just the
|
||||
reason) but not from the CLI. Denying is final: a denied approval
|
||||
can't be re-approved later, the agent has to submit a fresh request.
|
||||
|
||||
Everything below this point is the implementation detail behind that
|
||||
flow.
|
||||
|
||||
## Config changes
|
||||
|
||||
Config changes flow through a **forge pull request** on the agent's
|
||||
`agent-configs/<name>` repo — the same surface agents use for code PRs.
|
||||
No MCP tool exists for config changes: opening the PR IS the request.
|
||||
|
||||
1. The submitting agent (the child's parent, holding the `approvals`
|
||||
tool group) **clones** `agent-configs/<name>`, edits it there (any
|
||||
tracked path, but `agent.nix` is the contract entry point), commits
|
||||
with its own git identity, and pushes a branch + opens a PR with
|
||||
`hive-forge` — the same way it would change any other repo.
|
||||
The bind-mounted `/agents/<name>/config/` is a **copy for reading** a
|
||||
config, not the tree to edit: authoring in place there produces no PR.
|
||||
(it's currently mounted read-write, which is a defect tracked
|
||||
separately, not an authoring path.)
|
||||
Branch protection (the agent isn't on the `main` push allowlist;
|
||||
merge and approvals allowlist = `operators` team; see "Forge mirror"
|
||||
below) makes the agent a write collaborator that **can't merge its own
|
||||
config PR**.
|
||||
2. An operator reviews the PR **on the forge** (native diff, threaded
|
||||
comments, CI status) and merges it there. Merging needs membership of
|
||||
the `operators` team in `agent-configs`. The team starts empty: add
|
||||
each operator by hand in the forge UI.
|
||||
3. swarm-controller gets the `agent-configs` org's `pull_request`
|
||||
delivery. A `closed` event with `merged: true` and base branch `main`
|
||||
names the commit in `merge_commit_sha`.
|
||||
4. The controller looks up which hive's wanted state places the agent
|
||||
and publishes a deploy request carrying that commit on that hive's
|
||||
deploy subject. With no such hive, or more than one, it deploys
|
||||
nothing and logs a warning naming them.
|
||||
5. If the hive's `applied/main` already is that commit, it does nothing.
|
||||
Otherwise it fetches the config repo's `main`, requires the commit to
|
||||
descend from `applied/main`, fast-forwards `applied/main` to it, and
|
||||
queues a rebuild. A failure before the rebuild (the fetch, or a commit
|
||||
that doesn't descend) deploys nothing and posts the error as a comment
|
||||
on the merged PR. An agent with no container on that hive yet ignores
|
||||
the commit; its first deploy builds what it seeds.
|
||||
|
||||
This path runs **no eval-verify**. A merged config that doesn't
|
||||
evaluate or build fails the rebuild, and `applied/main` stays at that
|
||||
commit, so the agent's rebuilds keep failing until a fix merges. A
|
||||
failed rebuild shows on the hive like any other and isn't commented on
|
||||
the PR. A merge whose webhook delivery never reaches the controller
|
||||
deploys nothing.
|
||||
|
||||
## Approval queue
|
||||
|
||||
### Withdrawing a pending approval
|
||||
|
||||
The submitting agent can call `cancel_loose_end(kind: "approval", id)` to
|
||||
withdraw an approval the operator hasn't acted on yet.
|
||||
The row transitions to `ApprovalStatus::Cancelled` (distinct from
|
||||
`Denied`/`Failed`), the dashboard pulls the card out of the
|
||||
pending pane, and `ApprovalResolved { status: "cancelled" }` fires
|
||||
on the root agent + dashboard channels. Approvals that the operator
|
||||
(or a lifecycle failure) has already approved, denied, or failed return an error — the resolution is
|
||||
final once acted on.
|
||||
|
||||
The socket refuses the `approval` kind with a clear error for any
|
||||
agent that lacks the `approvals` tool group: only an agent with that
|
||||
group submits approvals (for its direct children), so an agent
|
||||
without it has nothing of its own to withdraw.
|
||||
|
||||
Creating a brand-new agent isn't an approval: it's swarm-level
|
||||
(`swarmctl agent create`, `POST /api/agents` on the swarm controller),
|
||||
and no hive can originate an agent. The swarm controller's
|
||||
`InitAgentConfigRepo` job seeds the agent's config repo with a default
|
||||
`agent.nix` template, then asks the target hive to deploy it.
|
||||
|
||||
Changing what the template seeded isn't a special case: like every
|
||||
later change, it's a PR on that config repo, made from a clone and
|
||||
merged by an operator on the forge (see [Config changes](#config-changes)).
|
||||
|
||||
### Approval kinds (wire shapes)
|
||||
|
||||
`ApprovalKind` carries one variant, and it overloads `commit_ref` as
|
||||
its payload carrier.
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
- `UpdateMetaInputs` — `commit_ref` stores the JSON-encoded inputs
|
||||
array (`"[]"` = all inputs, `"[\"nixpkgs\"]"` = just nixpkgs,
|
||||
etc.). hive-c0re sets the `agent` field to the requesting root agent.
|
||||
On approve hive-c0re runs `nix flake update [inputs...]` on the
|
||||
meta flake and commits the resulting lock changes.
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
### Reminder delivery: file-path semantics
|
||||
|
||||
A reminder may carry a `file_path` (the agent-visible path inside its
|
||||
container, for example `/agents/<name>/state/foo.md`). On delivery hive-c0re:
|
||||
|
||||
1. **Translates** the container path to the host path
|
||||
(`/var/lib/hyperhive/agents/<name>/state/foo.md`) so c0re can write
|
||||
from outside the container.
|
||||
2. **Validates** the path: rejects anything outside the agent's own state
|
||||
subtree, containing `..` (path traversal), or with an empty relative
|
||||
tail. On rejection hive-c0re skips the write and delivers the
|
||||
original message inline with a warning — the reminder still fires.
|
||||
3. **Defends against symlink escape**: after `create_dir_all`, hive-c0re
|
||||
canonicalizes the parent dir and re-verifies it lives under the agent's host
|
||||
state root. hive-c0re opens the final file with
|
||||
`O_NOFOLLOW | O_CREAT | O_TRUNC` so an existing symlink at the
|
||||
basename can't redirect the write to an arbitrary host path.
|
||||
4. **Writes the body to disk** and delivers a short pointer message in its
|
||||
place, keeping the agent's inbox / wake-prompt small while the agent
|
||||
reads the bulky payload out of band.
|
||||
|
||||
`Broker::deliver_reminders_batch` handles atomicity of the inbox INSERT + `reminders.sent_at` UPDATE; the scheduler only computes the
|
||||
body strings before calling it.
|
||||
|
||||
### Destroy semantics
|
||||
|
||||
`HostRequest::Destroy { name, purge }` is the lifecycle tear-down,
|
||||
not an approval. Stops + removes the nspawn container, drops the
|
||||
systemd drop-in, fails any pending approvals. Persistent state
|
||||
(proposed/applied repos, claude credentials, `/state/` notes) is
|
||||
**kept by default** — recreating the agent with the same name
|
||||
reuses prior config + login. With `purge = true` the agent's
|
||||
`/var/lib/hyperhive/{agents,applied}/<name>/` trees are also
|
||||
wiped (config history + creds + notes gone forever). The
|
||||
root/bootstrap container is destroyable like any other — hive-c0re
|
||||
recreates it on the next startup if it's absent, so destroying it's
|
||||
transient.
|
||||
|
||||
## Meta flake
|
||||
|
||||
The hive-c0re-owned repo at `/var/lib/hyperhive/meta/`
|
||||
declares one flake input per agent (`agent-<n>.url =
|
||||
"git+http://<forge>/agent-configs/<n>.git"`) and one
|
||||
`nixosConfigurations.<n>` output per agent. Each output wraps
|
||||
`inputs.agent-<n>.nixosModules.default` with the identity +
|
||||
`HIVE_PORT` / `HIVE_LABEL` / `HIVE_DASHBOARD_PORT` injection module.
|
||||
Containers run against `--flake /var/lib/hyperhive/meta#<n>`.
|
||||
|
||||
The declared input url is the agent's **forge config repo** (the
|
||||
same `agent-configs/<n>` config PRs merge into), so the meta flake
|
||||
references a reviewable, reproducible source rather than a local
|
||||
checkout. hive-c0re authenticates that `git+http` fetch via a git
|
||||
credential helper that reads the live forge-core token — no token in
|
||||
the url or the lock. The rebuild paths, however, do **not** re-lock
|
||||
from the forge: they `--override-input agent-<n>
|
||||
git+file:///var/lib/hyperhive/applied/<n>`, locking the exact config
|
||||
`applied/<n>/main` was fast-forwarded to. That keeps a rebuild
|
||||
reproducible and independent of forge reachability — rebuilds fire on
|
||||
crash-restart and meta bumps, not just config merges — while the
|
||||
declared url stays the forge. `sync_agents` re-renders + re-locks the
|
||||
persistent input; a plain `nix flake lock` leaves an existing applied
|
||||
override in place (it only re-locks when the declared url itself
|
||||
changes), so the forge-declared / applied-deployed split is stable.
|
||||
|
||||
Lock updates are single-phase and commit when the lock changed:
|
||||
`meta::lock_update_for_rebuild(name)` relocks one agent's input for a
|
||||
relocking rebuild (the manual `↻ R3BU1LD` button, and the rebuild a
|
||||
merged config commit queues), and `meta::lock_update_hyperhive()` is the
|
||||
autoupdate flake-rev bump (one shot before per-agent rebuilds).
|
||||
|
||||
`meta::sync_agents(hive: &HiveEnv, agents: &[AgentSpec])` — `hive`
|
||||
carries `hyperhive_flake`, `dashboard_port`, and the rest of the
|
||||
per-hive config — is the idempotent reconciler called by `spawn`,
|
||||
`destroy`, `rebuild`, and the startup migration. Renders `flake.nix`
|
||||
from the agent list; if it differs from disk, runs
|
||||
`nix flake lock` + commits as `regenerate meta flake` (or
|
||||
`seed meta from N agent(s)` on the first call).
|
||||
|
||||
The root agent has `/meta` RO-bound inside its container:
|
||||
`git -C /meta log --oneline` is the swarm-wide deploy log,
|
||||
`cat /meta/flake.lock | jq '.nodes["agent-<n>"].locked'`
|
||||
resolves which sha the flake pins each agent at right now.
|
||||
Dashboard surfaces the same info as a `deployed:<sha12>` chip
|
||||
per container row.
|
||||
|
||||
## Two repos per agent
|
||||
|
||||
```
|
||||
/var/lib/hyperhive/agents/<name>/config/ proposed — parent mount is RO
|
||||
└── <anything> # any files the submitting
|
||||
# agent wants in the commit.
|
||||
# agent.nix is the
|
||||
# convention entry
|
||||
# point; flake.nix is
|
||||
# tracked boilerplate
|
||||
# (submitting agent doesn't
|
||||
# edit it).
|
||||
|
||||
/var/lib/hyperhive/applied/<name>/ applied — core-only
|
||||
├── .git/ # tag-rich history
|
||||
├── flake.nix # tracked, fixed
|
||||
│ # boilerplate exporting
|
||||
│ # nixosModules.default
|
||||
├── agent.nix # working tree of main
|
||||
└── <other committed files> # also tracked
|
||||
|
||||
/var/lib/hyperhive/meta/ swarm-wide flake — core
|
||||
├── .git/ # one commit per lock
|
||||
│ # change
|
||||
├── flake.nix # generated from agent set
|
||||
└── flake.lock # pins each agent's sha
|
||||
```
|
||||
|
||||
Why two physical repos: the submitting agent's `/agents/<n>/config/` is
|
||||
RW — a buggy or hostile agent can `git clean -fdx` its own
|
||||
proposed tree. The applied repo is never bind-mounted (except
|
||||
the read-only `.git` exposure described below) so a destructive
|
||||
move inside the container can't reach it.
|
||||
|
||||
The container's `--flake` ref is `/var/lib/hyperhive/meta#<name>`
|
||||
(see "Meta flake" above). The agent's own `applied/<n>/flake.nix`
|
||||
is a fixed boilerplate that exports `nixosModules.default =
|
||||
import ./agent.nix`; the meta flake imports that module and
|
||||
wraps it with identity + `HIVE_PORT` / `HIVE_LABEL` /
|
||||
`HIVE_DASHBOARD_PORT`.
|
||||
|
||||
### Tag state machine
|
||||
|
||||
hive-c0re plants `deployed/0` on the seed commit at first spawn. A
|
||||
merged config commit that deploys plants no tag: the merged PR on the
|
||||
forge and meta's lock commits record it. A config PR nobody merges
|
||||
carries no extra state on the forge side: the PR stays open, and
|
||||
the submitter pushes again (or closes it) to retry.
|
||||
|
||||
### Dispatch via the job queue
|
||||
|
||||
Long-running approval work — `UpdateMetaInputs` — runs as a DAG on the
|
||||
global job queue (`docs/scheduler/coordinator.md::Job queue`), submitted
|
||||
by the approval handler rather than run inline:
|
||||
|
||||
| `ApprovalKind` | DAG submitted | source |
|
||||
|---|---|---|
|
||||
| `UpdateMetaInputs` | `meta_update` (`MetaLock` + rebuild fan-out) | `approval` |
|
||||
|
||||
The DAG carries the originating `approval_id`. **Every** queued kind
|
||||
resolves through `actions::resolve_approval_dag` when its DAG settles
|
||||
terminal: the DAG's own terminal state is the authoritative outcome.
|
||||
That hook fires the matching `HelperEvent::*` via `finish_approval`.
|
||||
|
||||
Two visible consequences:
|
||||
|
||||
- **Operator dashboard**: after selecting APPR0VE the work-in-progress
|
||||
shows up on the *rebuild queue* card (`GET /api/jobq/graph`, refetched
|
||||
on every `rebuild_queue_changed` tick), not on the approvals panel
|
||||
(which already moved the row to "approved"). A long meta-update
|
||||
cascade renders as a parent DAG with one child rebuild per affected
|
||||
agent — see `docs/web-ui/dashboard.md` for the layout.
|
||||
- **Cancellation**: the dashboard's *× cancel* button on a still-queued
|
||||
DAG calls `POST /api/rebuild-queue/{id}/cancel`, which flips it to
|
||||
`Cancelled` before any node runs (and fails the approval row instead
|
||||
of leaving it dangling). Returns `{"cancelled": true}` on success,
|
||||
`{"cancelled": false}` once any node started — terminal states can't
|
||||
be retroactively rewritten.
|
||||
|
||||
The `approval` source + `approval_id` mean a tail-end build failure
|
||||
surfaces back as a failed approval row, not just a silent queue
|
||||
entry. `manual` (dashboard ↻ R3BU1LD) and `auto_update` (boot
|
||||
reconcile) DAGs use the same queue but skip the approval plumbing.
|
||||
|
||||
### Forge mirror
|
||||
|
||||
The bundled `hive-forge` container runs on the swarm's forge host
|
||||
(`deploy.forgejo.enable`, see [`../swarm/services.md`](../swarm/services.md)),
|
||||
and hive-c0re mirrors every agent's applied repo into a
|
||||
private `agent-configs` Forgejo org. `forge::push_config(<name>)` pushes
|
||||
every tag, then `applied/main`, to `agent-configs/<name>` on every
|
||||
startup sweep and every rebuild. Forge `main` is branch-protected, so
|
||||
the forge routinely refuses a push of an established `main`, which
|
||||
hive-c0re expects. Pushes are best-effort — a missing or stopped forge never
|
||||
blocks a deploy.
|
||||
|
||||
Each agent is a **write collaborator on its own** `agent-configs/<name>`
|
||||
repo — so it can push a branch and open a config PR — but not a member
|
||||
of any other agent's, so it can't reach another agent's config through
|
||||
the forge. Branch protection keeps the agent off the `main` push
|
||||
allowlist and allowlists merging to the `operators` team only, so an
|
||||
agent can't fast-forward its own config or self-merge its PR (see
|
||||
[Config changes](#config-changes) above). hive-c0re passes the tokenised
|
||||
push URL inline to `git push`, never writing it into
|
||||
`applied/<n>/.git/config`; that repo is RO-bind-mounted into the root
|
||||
agent, and a stored token would leak core's admin credential to an
|
||||
agent.
|
||||
|
||||
The dashboard deep-links into this org — a `config repo` link
|
||||
per container row. See `docs/web-ui/dashboard.md`.
|
||||
|
||||
### Submitting agent's view of config repos
|
||||
|
||||
An agent holding `ManageRootAgent` has every other agent's config repo
|
||||
bind-mounted **read-only** (`hive-c0re/src/lifecycle/host_config.rs`
|
||||
calls `bind_child_agent_dirs` for each entry in
|
||||
`topology::all_agents()`). it's a copy to *read* another agent's
|
||||
current config — not an editing surface. An agent without the
|
||||
capability sees no other agent's config at all.
|
||||
|
||||
An agent with the `approvals` tool group submits a change the same way
|
||||
it makes any other change: **clone the target agent's config repo from the
|
||||
forge into its own state dir, commit on a branch, open a PR**, and let
|
||||
the operator review and approve it. By design, no second, mount-shaped
|
||||
path reaches the same file without the review.
|
||||
|
||||
Agents holding the `manage_root_agent` capability (granted per agent in
|
||||
`capabilities.json`; see `hive-sh4re/src/permissions.rs`) get additional
|
||||
host-side bind mounts via `set_nspawn_flags`:
|
||||
|
||||
- `/var/lib/hyperhive/agents/` → `/agents/` (RW) — **every** agent's
|
||||
proposed repo, not just direct children. The capability means "may
|
||||
manage any agent", so the mounted set is every agent.
|
||||
- `/var/lib/hyperhive/applied/` → `/applied/` (RO) — every agent's
|
||||
authoritative applied repo, including `.git`.
|
||||
- `/var/lib/hyperhive/meta/` → `/meta/` (RO) — the swarm-wide
|
||||
deploy flake.
|
||||
|
||||
An agent without the capability only has its direct children's config
|
||||
dirs.
|
||||
|
||||
⚠️ nspawn bakes bind flags at container start, so granting or
|
||||
revoking this capability doesn't change any mount until that agent's
|
||||
container is rebuilt/restarted.
|
||||
|
||||
The root agent gets the capability by default, seeded on its autodeploy
|
||||
path (`workers::auto_update::ensure_root_agent`) so the recovery mounts
|
||||
are there from its first container. That seed only fires while
|
||||
`capabilities.json` doesn't exist yet: any grant or revoke through
|
||||
the dashboard creates the file, so a revoked root-agent grant stays
|
||||
revoked and isn't re-applied on the next hive-c0re restart.
|
||||
|
||||
Each proposed repo (`/agents/<n>/config/`) is pre-configured
|
||||
with `applied` as a git remote pointing at
|
||||
`/applied/<n>/.git`. Useful incantations from inside an agent with
|
||||
the full `/applied` mount:
|
||||
|
||||
```sh
|
||||
git -C /agents/<n>/config fetch applied
|
||||
git -C /agents/<n>/config log applied/main --oneline
|
||||
git -C /agents/<n>/config show applied/refs/tags/deployed/0 # the seed commit
|
||||
git -C /agents/<n>/config show applied/refs/tags/denied/<id> # body = operator note
|
||||
git -C /agents/<n>/config rebase applied/main # base in-flight work on what's deployed
|
||||
|
||||
git -C /meta log --oneline # swarm-wide deploy history
|
||||
cat /meta/flake.lock | jq '.nodes | with_entries(select(.key | startswith("agent-")))'
|
||||
```
|
||||
|
||||
The RO binds block push at the kernel level — git plumbing inside the
|
||||
container can't corrupt either authoritative repo.
|
||||
|
||||
## Startup migrations (older hosts)
|
||||
|
||||
hive-c0re runs a couple of idempotent migrations on every startup so a
|
||||
host set up before the tag-driven-deploy + meta-flake scheme (both
|
||||
described above) converges to it automatically. Each phase is a no-op
|
||||
once already applied:
|
||||
|
||||
- **Tags**: hive-c0re tags agents from before the tag-driven scheme
|
||||
`deployed/0` on `main` once. Non-destructive — it doesn't touch live
|
||||
containers, state dirs, or claude creds.
|
||||
- **Meta flake**: rewrites each `applied/<n>/flake.nix` to the
|
||||
module-only boilerplate, wires the `applied` remote in each proposed
|
||||
repo, and bootstraps the meta repo from the current agent list. Set
|
||||
`HIVE_SKIP_META_MIGRATION=1` on the service to defer this phase.
|
||||
|
||||
No state loss in either migration: claude creds, `/state/` notes, the
|
||||
events DB, and both proposed + applied history all survive. The root
|
||||
agent keeps its session; sub-agents stay logged in.
|
||||
|
||||
## The root/bootstrap container is hive-c0re-managed
|
||||
|
||||
The root agent container runs through the **same lifecycle as
|
||||
sub-agents**. On `hive-c0re serve` startup, if `ruth` is missing,
|
||||
hive-c0re creates it. The root agent's flake lives at
|
||||
`/var/lib/hyperhive/applied/ruth/`; its proposed config at
|
||||
`/var/lib/hyperhive/agents/ruth/config/`. The root agent can edit its own
|
||||
`agent.nix` (visible inside the container at `/agents/ruth/config/`)
|
||||
and open a config PR on `agent-configs/ruth` for operator approval,
|
||||
same as any other agent.
|
||||
|
||||
Differences from sub-agents:
|
||||
|
||||
- `flake.nix` extends `hyperhive.nixosConfigurations.ruth`
|
||||
(vs `agent-base`).
|
||||
- Web UI port via `lifecycle::agent_web_port("ruth")` — same
|
||||
FNV-1a hash as every other agent (8100..8999 range).
|
||||
- `set_nspawn_flags` adds two extra binds: `/var/lib/hyperhive/agents`
|
||||
→ `/agents` (RW) so the root agent can edit per-agent proposed repos,
|
||||
and `/var/lib/hyperhive/applied` → `/applied` (RO) so the root agent
|
||||
can `git fetch` deployed/failed/denied tags from any agent's
|
||||
authoritative applied repo (see "Root-agent view of applied" below).
|
||||
- First-deploy spawn bypasses the approval queue (the root agent is
|
||||
required infrastructure).
|
||||
- `socket_server::start_manager` binds the root agent's socket,
|
||||
pure transport with no dedicated helpers — it uses the same
|
||||
per-agent runtime dir as any other agent (`/run/hyperhive/agents/ruth/`),
|
||||
not a special manager-only path.
|
||||
|
||||
**Migration note** (for older hosts): drop any `containers.root =
|
||||
{ ... }` block from your host NixOS config. hyperhive creates and
|
||||
updates the root agent itself.
|
||||
|
||||
## Root-agent policy
|
||||
|
||||
The system prompt (`hive-agent/prompts/system.md`, rendered by
|
||||
`hive-agent/src/prompt.rs`) is the **same for every agent**; what
|
||||
varies is which MCP tools it surfaces (gated by tool groups and
|
||||
capabilities in `agent.nix`). No `role:manager` block renders only
|
||||
for the root agent. The root agent's approval-gating
|
||||
behaviour comes from its CLAUDE.md / agent-specific instructions, not
|
||||
the system prompt template.
|
||||
|
||||
## Helper events to the submitting agent
|
||||
|
||||
`Coordinator::notify_submitter(approval_id, &HelperEvent)` routes the
|
||||
event to the agent that originally submitted the approval (looked up from
|
||||
the `submitter` column on the `approvals` table). The harness delivers it
|
||||
as a regular `system` inbox message so it drives a normal claude turn.
|
||||
`finish_approval` fires an `ApprovalResolved` HelperEvent this way for
|
||||
**every** approval kind's terminal state. A
|
||||
"FYI, check when convenient" event doesn't need a message — hive-c0re publishes those as swarm
|
||||
lifecycle notices instead (`swarm_notices::notify`). Legacy
|
||||
approval rows that predate the submitter column fall back to the
|
||||
root agent. Variants (`hive_sh4re::manager::HelperEvent`):
|
||||
|
||||
- `ApprovalResolved { id, agent, commit_ref, status, note }` —
|
||||
fired by `actions::approve` + `actions::deny` whenever an
|
||||
approval transitions to its terminal state.
|
||||
- `ContainerCrash { agent, note }` — `crash_watch`: a previously-
|
||||
running container went away with no operator-initiated transient
|
||||
state (Stopping / Restarting / Destroying / Rebuilding) AND nothing
|
||||
cleared that transient in the last 30s (`RECENT_TRANSIENT_GRACE`
|
||||
tombstone, three `POLL_INTERVAL`s — closes the race where a
|
||||
lifecycle op finishes between two crash-watch polls and the
|
||||
container shows briefly as "stopped without transient" before
|
||||
the next start). The root agent escalates to the operator, who
|
||||
starts it again from the dashboard.
|
||||
- `NeedsUpdate { agent }` — sub-agent's recorded flake rev is
|
||||
stale. The operator rebuilds it from the dashboard.
|
||||
|
||||
The remaining lower-urgency lifecycle notices — `Rebuilt`, `Killed`,
|
||||
`Destroyed`, `NeedsLogin`, `LoggedIn` — are "FYI, check
|
||||
when convenient" events with no reason to drive an immediate turn, so
|
||||
they go out via `swarm_notices::notify` (see above) instead
|
||||
of a `HelperEvent`: `subsystem = "core"`, `key = "<event>:<agent>"`,
|
||||
and a single free-text `summary` (`rebuilt_todo_summary` renders
|
||||
`Rebuilt`'s `ok`/`note`/`sha`/`tag` fields into that string).
|
||||
|
||||
To add a new lifecycle notice: if it needs to drive an immediate turn
|
||||
(something genuinely urgent, like `ContainerCrash`), add a
|
||||
`HelperEvent` variant + call sites + update `prompts/system.md`'s
|
||||
message-event list. If it's "FYI, check when convenient," call
|
||||
`swarm_notices::notify` instead — no new wire type
|
||||
needed.
|
||||
|
||||
## Autoupdate on startup
|
||||
|
||||
`hive-c0re serve` runs `auto_update::run` in a background task right
|
||||
after opening the coordinator. It enumerates managed containers and
|
||||
rebuilds any whose recorded hyperhive rev differs from the current
|
||||
one — sub-agents and the root agent go through the same
|
||||
`job_queue::templates::rebuild` DAG.
|
||||
|
||||
"Rev" = canonical filesystem path of
|
||||
`services.hyperhive.c0re.hyperhiveFlake`. Marker
|
||||
file: `/var/lib/hyperhive/applied/.<name>.hyperhive-rev`. If the
|
||||
flake input has no canonical path (for example a `github:` URL),
|
||||
autoupdate is a no-op — rebuild manually.
|
||||
|
||||
The dashboard surfaces pending updates per agent: a clickable
|
||||
"needs update ↻" badge appears whenever the marker differs from
|
||||
current rev. The badge POSTs `/api/rebuild/<name>`, which inserts the
|
||||
same `job_queue::templates::rebuild` DAG so manual triggers and the
|
||||
startup scan can't drift. When at least one container is stale, a
|
||||
top-level `↻ UPD4TE 4LL` button appears that loops over every
|
||||
stale container.
|
||||
|
|
@ -37,10 +37,6 @@ This document contains the help content for the `hivectl` command-line program.
|
|||
* [`hivectl agent subvol snapshot push`↴](#hivectl-agent-subvol-snapshot-push)
|
||||
* [`hivectl list-agents`↴](#hivectl-list-agents)
|
||||
* [`hivectl quota-enable`↴](#hivectl-quota-enable)
|
||||
* [`hivectl approvals`↴](#hivectl-approvals)
|
||||
* [`hivectl approvals pending`↴](#hivectl-approvals-pending)
|
||||
* [`hivectl approvals approve`↴](#hivectl-approvals-approve)
|
||||
* [`hivectl approvals deny`↴](#hivectl-approvals-deny)
|
||||
* [`hivectl wg`↴](#hivectl-wg)
|
||||
* [`hivectl wg init`↴](#hivectl-wg-init)
|
||||
* [`hivectl wg peer`↴](#hivectl-wg-peer)
|
||||
|
|
@ -66,7 +62,6 @@ Sibling to the `hive-c0re` daemon binary. Covers host-side admin operations that
|
|||
* `agent` — Lifecycle actions on ONE managed agent container. Needs the hive-c0re daemon running
|
||||
* `list-agents` — Show all managed agents with their status and technical state
|
||||
* `quota-enable` — Enable btrfs qgroup accounting on the agent-state filesystem
|
||||
* `approvals` — Operator approval queue: list, approve, or deny pending requests
|
||||
* `wg` — WireGuard inter-hive mesh setup helpers
|
||||
* `peer-config` — Generate the federation peer-config block for THIS hive
|
||||
* `stop` — Stop containers hive-wide in one operator action
|
||||
|
|
@ -524,54 +519,6 @@ Global one-shot toggle (not per-agent), hence top-level rather than under `hivec
|
|||
|
||||
|
||||
|
||||
## `hivectl approvals`
|
||||
|
||||
Operator approval queue: list, approve, or deny pending requests.
|
||||
|
||||
Needs the hive-c0re daemon running.
|
||||
|
||||
**Usage:** `hivectl approvals <COMMAND>`
|
||||
|
||||
###### **Subcommands:**
|
||||
|
||||
* `pending` — List pending approval requests submitted by agents
|
||||
* `approve` — Approve a pending request by id; the action runs immediately
|
||||
* `deny` — Deny a pending request by id
|
||||
|
||||
|
||||
|
||||
## `hivectl approvals pending`
|
||||
|
||||
List pending approval requests submitted by agents
|
||||
|
||||
**Usage:** `hivectl approvals pending`
|
||||
|
||||
|
||||
|
||||
## `hivectl approvals approve`
|
||||
|
||||
Approve a pending request by id; the action runs immediately
|
||||
|
||||
**Usage:** `hivectl approvals approve <ID>`
|
||||
|
||||
###### **Arguments:**
|
||||
|
||||
* `<ID>` — Approval id (from `hivectl approvals pending`)
|
||||
|
||||
|
||||
|
||||
## `hivectl approvals deny`
|
||||
|
||||
Deny a pending request by id
|
||||
|
||||
**Usage:** `hivectl approvals deny <ID>`
|
||||
|
||||
###### **Arguments:**
|
||||
|
||||
* `<ID>` — Approval id
|
||||
|
||||
|
||||
|
||||
## `hivectl wg`
|
||||
|
||||
WireGuard inter-hive mesh setup helpers.
|
||||
|
|
|
|||
|
|
@ -94,7 +94,7 @@ pub struct CompactArgs {
|
|||
}
|
||||
|
||||
// -----------------------------------------------------------------------------
|
||||
// Privileged tool arg types (lifecycle, approvals)
|
||||
// Privileged tool arg types (lifecycle)
|
||||
// -----------------------------------------------------------------------------
|
||||
|
||||
#[derive(Debug, serde::Deserialize, schemars::JsonSchema)]
|
||||
|
|
@ -115,8 +115,7 @@ pub struct GetAgentMetaArgs {
|
|||
#[derive(Debug, serde::Deserialize, schemars::JsonSchema)]
|
||||
pub struct CancelLooseEndArgs {
|
||||
/// Which kind of thread to cancel — `"reminder"` for a scheduled
|
||||
/// `remind` that hasn't fired yet, `"approval"` for a pending
|
||||
/// approval you submitted, or `"todo"` for a loose-ends-v2 todo
|
||||
/// `remind` that hasn't fired yet, or `"todo"` for a loose-ends-v2 todo
|
||||
/// (bash/matrix/forge). Use the `kind` field straight off the
|
||||
/// `get_loose_ends` row.
|
||||
pub kind: String,
|
||||
|
|
|
|||
|
|
@ -30,9 +30,8 @@ pub use args::{
|
|||
pub use render::{annotate_retries, format_ack, format_agent_meta, format_recv};
|
||||
|
||||
use render::{
|
||||
dial_agent_socket, format_matrix_summary, local_reminders, local_todos, loose_end_kind_label,
|
||||
mark_local_todo_done, mark_local_todos_done, matrix_unread_summary, parse_loose_end_kind,
|
||||
render_loose_ends, reply_err,
|
||||
dial_agent_socket, format_matrix_summary, local_reminders, local_todos, mark_local_todo_done,
|
||||
mark_local_todos_done, matrix_unread_summary, render_loose_ends, reply_err,
|
||||
};
|
||||
|
||||
/// Write (or remove) the status file in the agent's own `state/` directory.
|
||||
|
|
@ -231,9 +230,7 @@ impl AgentServer {
|
|||
|
||||
#[tool(
|
||||
description = "List loose ends pending against this agent: pending reminders \
|
||||
you scheduled, plus — for the manager only — pending \
|
||||
approvals you submitted that the operator hasn't acted on yet (agents with the \
|
||||
`approvals` tool group also see their own pending approvals). Also lists active \
|
||||
you scheduled. Also lists active \
|
||||
local tasks published by external MCP daemons (e.g. running bash tasks). Cheap sweep, no args. Useful \
|
||||
at turn start to remember what you owe / what's owed to you without scrolling \
|
||||
inbox history. Output is a short bulleted list with ids, ages in seconds, and \
|
||||
|
|
@ -346,9 +343,6 @@ impl AgentServer {
|
|||
you scheduled (hard-deleted before it fires). `kind` is `\"reminder\"`; \
|
||||
`id` is the row id from the matching `get_loose_ends` entry. Auth: you can only \
|
||||
cancel rows where you're the owner. Returns `ok` or an error string.\n\
|
||||
`kind` may also be `\"approval\"` to withdraw a pending approval you submitted \
|
||||
(before the operator acts on it) — root agent (`ruth`) only; the server rejects \
|
||||
`approval` kind for all other callers.\n\
|
||||
`kind` may also be `\"todo\"` to clear one of your own loose-ends-v2 todos (the id \
|
||||
from a `get_loose_ends` `todo #N` line) — dials the in-agent socket directly, safe \
|
||||
to call repeatedly.")]
|
||||
|
|
@ -373,42 +367,26 @@ impl AgentServer {
|
|||
.to_owned(),
|
||||
};
|
||||
}
|
||||
let kind = match parse_loose_end_kind(&args.kind) {
|
||||
Ok(k) => k,
|
||||
Err(e) => return e,
|
||||
};
|
||||
let kind_label = loose_end_kind_label(kind);
|
||||
// Reminders are harness-local — dial the in-agent socket
|
||||
// directly instead of the broker; the other kind (approval)
|
||||
// still lives in c0re.
|
||||
if kind == hive_sh4re::inbox::CancelLooseEndKind::Reminder {
|
||||
return match dial_agent_socket(&hive_agent_sock::Request::CancelReminder { id })
|
||||
.await
|
||||
{
|
||||
Some(hive_agent_sock::Response::Acked { count }) if count > 0 => {
|
||||
format!("cancelled {kind_label} {id}")
|
||||
}
|
||||
Some(hive_agent_sock::Response::Acked { .. }) => {
|
||||
format!("cancel_loose_end failed: no pending {kind_label} {id}")
|
||||
}
|
||||
Some(hive_agent_sock::Response::Err { message }) => {
|
||||
format!("cancel_loose_end failed: {message}")
|
||||
}
|
||||
Some(other) => format!("cancel_loose_end unexpected response: {other:?}"),
|
||||
None => "cancel_loose_end failed: in-agent socket unavailable".to_owned(),
|
||||
};
|
||||
let kind = args.kind.trim().to_ascii_lowercase();
|
||||
if !matches!(kind.as_str(), "reminder" | "r") {
|
||||
return format!(
|
||||
"cancel_loose_end: unknown kind '{kind}' (expected \"reminder\" or \"todo\")"
|
||||
);
|
||||
}
|
||||
// Reminders are harness-local — dial the in-agent socket directly.
|
||||
match dial_agent_socket(&hive_agent_sock::Request::CancelReminder { id }).await {
|
||||
Some(hive_agent_sock::Response::Acked { count }) if count > 0 => {
|
||||
format!("cancelled reminder {id}")
|
||||
}
|
||||
Some(hive_agent_sock::Response::Acked { .. }) => {
|
||||
format!("cancel_loose_end failed: no pending reminder {id}")
|
||||
}
|
||||
Some(hive_agent_sock::Response::Err { message }) => {
|
||||
format!("cancel_loose_end failed: {message}")
|
||||
}
|
||||
Some(other) => format!("cancel_loose_end unexpected response: {other:?}"),
|
||||
None => "cancel_loose_end failed: in-agent socket unavailable".to_owned(),
|
||||
}
|
||||
let (resp, retries) = self
|
||||
.dispatch(hive_core_agent_sock::Request::CancelLooseEnd { kind, id })
|
||||
.await;
|
||||
annotate_retries(
|
||||
format_ack(
|
||||
resp,
|
||||
"cancel_loose_end",
|
||||
format!("cancelled {kind_label} {id}"),
|
||||
),
|
||||
retries,
|
||||
)
|
||||
})
|
||||
.await
|
||||
}
|
||||
|
|
|
|||
|
|
@ -152,8 +152,8 @@ fn msg_id_tag(id: i64) -> String {
|
|||
}
|
||||
|
||||
/// Hard cap on how many individual `Todo` lines `render_loose_ends` will
|
||||
/// emit. Approvals/reminders stay naturally bounded (they're
|
||||
/// triaged interactively and don't self-multiply), but todos are pushed by
|
||||
/// emit. Reminders stay naturally bounded (they're triaged interactively
|
||||
/// and don't self-multiply), but todos are pushed by
|
||||
/// unattended producers — matrix/bash/forge are the built-in ones, but any
|
||||
/// user-configured MCP server can push its own too — an agent that goes a long
|
||||
/// stretch without calling `get_loose_ends`, or whose producers outpace its
|
||||
|
|
@ -178,22 +178,6 @@ const MAX_RENDERED_TODOS: usize = 40;
|
|||
fn render_one_loose_end(out: &mut String, t: &hive_sh4re::inbox::LooseEnd) {
|
||||
use std::fmt::Write as _;
|
||||
match t {
|
||||
hive_sh4re::inbox::LooseEnd::Approval {
|
||||
id,
|
||||
agent,
|
||||
commit_ref,
|
||||
description,
|
||||
age_seconds,
|
||||
} => {
|
||||
let desc = description
|
||||
.as_deref()
|
||||
.map(|d| format!(" — {d}"))
|
||||
.unwrap_or_default();
|
||||
let _ = writeln!(
|
||||
out,
|
||||
"- approval #{id} ({agent} @ {commit_ref}, {age_seconds}s old){desc}"
|
||||
);
|
||||
}
|
||||
hive_sh4re::inbox::LooseEnd::Reminder {
|
||||
id,
|
||||
owner,
|
||||
|
|
@ -438,33 +422,6 @@ pub(super) fn format_matrix_summary(rooms: &[MatrixRoomUnread]) -> String {
|
|||
out
|
||||
}
|
||||
|
||||
/// Parse the user-facing `kind` string for `cancel_loose_end` into the
|
||||
/// wire enum. Accepts a small alias set so claude doesn't have to
|
||||
/// remember the exact spelling (`"q"` / `"r"` shorthand falls out
|
||||
/// for free).
|
||||
pub(super) fn parse_loose_end_kind(
|
||||
raw: &str,
|
||||
) -> Result<hive_sh4re::inbox::CancelLooseEndKind, String> {
|
||||
match raw.trim().to_ascii_lowercase().as_str() {
|
||||
"reminder" | "r" => Ok(hive_sh4re::inbox::CancelLooseEndKind::Reminder),
|
||||
"approval" | "a" => Ok(hive_sh4re::inbox::CancelLooseEndKind::Approval),
|
||||
other => Err(format!(
|
||||
"cancel_loose_end: unknown kind '{other}' \
|
||||
(expected \"reminder\", \"approval\", or \"todo\")"
|
||||
)),
|
||||
}
|
||||
}
|
||||
|
||||
/// Canonical user-facing label for a `CancelLooseEndKind` — used in
|
||||
/// the success ack so the caller always sees `"reminder"` instead of
|
||||
/// whatever alias they passed in (`"r"`).
|
||||
pub(super) fn loose_end_kind_label(kind: hive_sh4re::inbox::CancelLooseEndKind) -> &'static str {
|
||||
match kind {
|
||||
hive_sh4re::inbox::CancelLooseEndKind::Reminder => "reminder",
|
||||
hive_sh4re::inbox::CancelLooseEndKind::Approval => "approval",
|
||||
}
|
||||
}
|
||||
|
||||
/// Format helper for `get_agent_meta`: renders an agent's identity +
|
||||
/// current status as a short human-readable block. `name`,
|
||||
/// `hyperhive_rev`, and `running` are always shown; `status` only
|
||||
|
|
|
|||
|
|
@ -149,7 +149,7 @@ pub enum Response {
|
|||
PendingRemindersCount { count: u64 },
|
||||
/// `ReminderRollup` result.
|
||||
ReminderRollup {
|
||||
stats: hive_sh4re::approvals::ReminderStats,
|
||||
stats: hive_sh4re::inbox::ReminderStats,
|
||||
},
|
||||
/// Op failed; `message` is operator-facing.
|
||||
Err { message: String },
|
||||
|
|
|
|||
|
|
@ -9,13 +9,12 @@ Need new packages, env vars, or other NixOS config for yourself? You can't edit
|
|||
|
||||
Your config repo is mounted **read-only** at `/agents/{label}/config/` — `agent.nix` plus whatever extra files define you (declared packages, env vars, MCP servers). Read it to see exactly what defines you before asking for a change, so you can point at the precise file and line.
|
||||
|
||||
Approval boundary: starting, stopping and rebuilding containers is the operator's — ask them when any agent needs one. _Creating_ a new agent is not something you can do from here — ask the operator, who scaffolds the new agent's config repo and spawns it from the dashboard. _Changing_ any agent's config is not a tool call at all — it's a forge PR on the agent's `agent-configs/<name>` repo, which an operator reviews and merges on the forge; the merge deploys it. The operator only signs off on changes; you run the day-to-day.
|
||||
Operator boundary: starting, stopping and rebuilding containers is the operator's — ask them when any agent needs one. _Creating_ a new agent is not something you can do from here — ask the operator, who scaffolds the new agent's config repo and spawns it from the dashboard. _Changing_ any agent's config is not a tool call at all — it's a forge PR on the agent's `agent-configs/<name>` repo, which an operator reviews and merges on the forge; the merge deploys it. The operator only signs off on changes; you run the day-to-day.
|
||||
|
||||
Messages from sender `system` are hyperhive helper events (JSON body, `event` field discriminates): `approval_resolved`, `container_crash`, `needs_update`. Use these to react to lifecycle changes:
|
||||
Messages from sender `system` are hyperhive helper events (JSON body, `event` field discriminates): `container_crash`, `needs_update`. Use these to react to lifecycle changes:
|
||||
|
||||
- `needs_update` — agent's flake rev is stale. Ask the operator to rebuild it.
|
||||
- `container_crash` — ask the operator to start it again; if it keeps crashing, say so with what you saw.
|
||||
- `approval_resolved` — one of your own submitted approvals was approved, denied, or failed; the body carries the resolution.
|
||||
|
||||
Lifecycle notices that don't need an immediate turn — a new agent spawned, a container rebuilt/killed/destroyed, or its login state changing — surface as todos instead of messages now. Call `get_loose_ends` to see them.
|
||||
|
||||
|
|
|
|||
|
|
@ -18,7 +18,7 @@ use std::sync::Mutex;
|
|||
|
||||
use anyhow::{Context, Result};
|
||||
use chrono::{DateTime, Utc};
|
||||
use hive_sh4re::approvals::ReminderStats;
|
||||
use hive_sh4re::inbox::ReminderStats;
|
||||
use hive_sh4re::wire_time;
|
||||
use rusqlite::{Connection, params};
|
||||
|
||||
|
|
|
|||
|
|
@ -15,7 +15,7 @@ use chrono::Utc;
|
|||
use rusqlite::{Connection, OpenFlags};
|
||||
use serde::Serialize;
|
||||
|
||||
use hive_sh4re::approvals::ReminderStats;
|
||||
use hive_sh4re::inbox::ReminderStats;
|
||||
|
||||
/// Window param accepted by `/api/stats?window=`. Each maps to a
|
||||
/// total span + the bucket width used to roll up trend series.
|
||||
|
|
|
|||
|
|
@ -29,7 +29,7 @@ pub(super) async fn api_stats(
|
|||
/// `HIVE_AGENT_SOCKET` — was a broker RPC before reminders moved
|
||||
/// in-container. Returns `None` on any transport / decode failure or
|
||||
/// when the socket is unset — the stats are decorative, not authoritative.
|
||||
async fn fetch_reminder_stats(window_secs: u64) -> Option<hive_sh4re::approvals::ReminderStats> {
|
||||
async fn fetch_reminder_stats(window_secs: u64) -> Option<hive_sh4re::inbox::ReminderStats> {
|
||||
match crate::todo_server::dial(&hive_agent_sock::Request::ReminderRollup {
|
||||
since_secs: window_secs,
|
||||
})
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
# hive-c0re
|
||||
|
||||
The unprivileged host daemon (runs as `hive-core`). Owns the sqlite
|
||||
broker, the approval/reminder queues, the generic job-DAG
|
||||
broker, the reminder queue, the generic job-DAG
|
||||
queue, container lifecycle, gateway vhost wiring, forge/matrix config
|
||||
reconciliation, per-container stats, and the axum operator dashboard.
|
||||
Largest crate in the workspace — bin-only, no separate lib.
|
||||
|
|
@ -9,7 +9,7 @@ Largest crate in the workspace — bin-only, no separate lib.
|
|||
## When to use it
|
||||
|
||||
Host-level, cross-container orchestration: spawning/rebuilding/
|
||||
destroying agent containers, the approval flow, dashboard-visible
|
||||
destroying agent containers, dashboard-visible
|
||||
state, wiring each agent's gateway vhost and reconciling its forge
|
||||
config-repo access and matrix room membership. The accounts themselves
|
||||
are swarm-controller's (`docs/integrations/forge.md`). Agent-side
|
||||
|
|
@ -25,7 +25,7 @@ subdirectory a module actually lives in). One line each — read the
|
|||
module's own `//!` doc-comment for real detail, don't expect this file
|
||||
to track it:
|
||||
|
||||
- **`dashboard/`** — the operator dashboard (containers, approvals,
|
||||
- **`dashboard/`** — the operator dashboard (containers,
|
||||
schedules, logs, topology).
|
||||
- **`job_queue/`** — the job-DAG queue + desired-state reconciliation
|
||||
(`docs/scheduler/coordinator.md`).
|
||||
|
|
|
|||
|
|
@ -6,53 +6,10 @@
|
|||
use std::sync::Arc;
|
||||
|
||||
use anyhow::{Context as _, Result, bail};
|
||||
use hive_sh4re::approvals::{ApprovalKind, ApprovalStatus};
|
||||
use hive_sh4re::manager::HelperEvent;
|
||||
|
||||
use crate::coordinator::Coordinator;
|
||||
use crate::lifecycle;
|
||||
|
||||
/// Approve a pending request. Marks the approval row durably, then submits
|
||||
/// the work to the job queue so the dashboard POST returns immediately while
|
||||
/// the long-running pipeline runs off-thread.
|
||||
///
|
||||
/// Dispatch:
|
||||
/// - `UpdateMetaInputs` → a `MetaUpdate` DAG (fan-out on completion)
|
||||
///
|
||||
/// Every queued kind resolves its approval row via
|
||||
/// [`resolve_approval_dag`] when the DAG settles terminal.
|
||||
pub fn approve(coord: &Arc<Coordinator>, id: i64) -> Result<()> {
|
||||
let approval = coord.approvals.mark_approved(id)?;
|
||||
tracing::info!(
|
||||
%approval.id,
|
||||
%approval.agent,
|
||||
kind = ?approval.kind,
|
||||
%approval.commit_ref,
|
||||
"approval: dispatching",
|
||||
);
|
||||
match approval.kind {
|
||||
ApprovalKind::UpdateMetaInputs => {
|
||||
// Inputs JSON-encoded into commit_ref by the manager's
|
||||
// submit path — surface them on the DAG so the dashboard
|
||||
// can show *which* inputs are about to bump. The cascade
|
||||
// rebuilds fan out when the lock bump lands (so they build
|
||||
// against the post-bump lock, and a failed bump fans out
|
||||
// nothing).
|
||||
let inputs: Vec<String> =
|
||||
serde_json::from_str(&approval.commit_ref).unwrap_or_default();
|
||||
let inserted = coord.job_queue.insert_job(|b| {
|
||||
crate::job_queue::templates::meta_update(b, inputs, Some(id));
|
||||
Vec::new()
|
||||
});
|
||||
if let Err(e) = inserted {
|
||||
return Err(e.context("insert meta-update dag"));
|
||||
}
|
||||
coord.emit_rebuild_queue_snapshot();
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// `applied/main`, and whether `target` descends from it. Fast-forwarding
|
||||
/// `main` to a `target` that does not is a rewind: it drops every commit on
|
||||
/// `main` that `target` lacks.
|
||||
|
|
@ -149,90 +106,6 @@ pub(crate) async fn post_rev_deploy_failure_to_pr(agent: &str, rev: &str, err: &
|
|||
}
|
||||
}
|
||||
|
||||
/// Resolve an approval row from how its DAG's work ended — the body of the
|
||||
/// [`NodeKind::ResolveApproval`] tail node. Every approval-carrying template
|
||||
/// resolves here: the pipeline is ordinary queue nodes,
|
||||
/// so the work's terminal state is the authoritative outcome and there's no
|
||||
/// in-node resolution to skip around.
|
||||
///
|
||||
/// `outcome` is the one the calling node was built to report — a template emits
|
||||
/// one tail per outcome, so success, failure and cancel each arrive here from
|
||||
/// their own node rather than from one node branching.
|
||||
///
|
||||
/// [`NodeKind::ResolveApproval`]: crate::job_queue::NodeKind::ResolveApproval
|
||||
pub(crate) fn resolve_approval_dag(
|
||||
coord: &Arc<Coordinator>,
|
||||
approval_id: i64,
|
||||
outcome: crate::job_queue::TerminalState,
|
||||
error: Option<&str>,
|
||||
) {
|
||||
use crate::job_queue::TerminalState;
|
||||
let approval = match coord.approvals.get(approval_id) {
|
||||
Ok(Some(a)) => a,
|
||||
Ok(None) => {
|
||||
tracing::warn!(approval_id, "approval dag terminal: row no longer exists");
|
||||
return;
|
||||
}
|
||||
Err(e) => {
|
||||
tracing::warn!(approval_id, error = ?e, "approval dag terminal: row read failed");
|
||||
return;
|
||||
}
|
||||
};
|
||||
let result: Result<()> = match outcome {
|
||||
TerminalState::Done => Ok(()),
|
||||
// `Skipped` never reaches here — no template emits a tail for it — but it
|
||||
// reads the same to an operator either way: the work did not happen.
|
||||
TerminalState::Cancelled | TerminalState::Skipped => {
|
||||
Err(anyhow::anyhow!("cancelled before completion"))
|
||||
}
|
||||
TerminalState::Failed => Err(anyhow::anyhow!("{}", error.unwrap_or("job dag failed"))),
|
||||
};
|
||||
if let Err(e) = finish_approval(coord, &approval, result) {
|
||||
tracing::warn!(approval_id, error = ?e, "approval dag resolved with failure");
|
||||
}
|
||||
}
|
||||
|
||||
fn finish_approval(
|
||||
coord: &Coordinator,
|
||||
approval: &hive_sh4re::approvals::Approval,
|
||||
result: Result<()>,
|
||||
) -> Result<()> {
|
||||
let (status, note, ok) = match &result {
|
||||
Ok(()) => (ApprovalStatus::Approved, None, true),
|
||||
Err(e) => {
|
||||
let note = format!("{e:#}");
|
||||
let _ = coord.approvals.mark_failed(approval.id, ¬e);
|
||||
(ApprovalStatus::Failed, Some(note), false)
|
||||
}
|
||||
};
|
||||
coord.notify_submitter(
|
||||
approval.id,
|
||||
&HelperEvent::ApprovalResolved {
|
||||
id: approval.id,
|
||||
agent: approval.agent.to_string(),
|
||||
commit_ref: approval.commit_ref.clone(),
|
||||
status,
|
||||
note: note.clone(),
|
||||
},
|
||||
);
|
||||
// Phase 5b: also fire on the dashboard event channel so the
|
||||
// browser moves the row out of pending into history without a
|
||||
// snapshot refetch. `approved` rows that succeed get the
|
||||
// approval's logged resolved_at indirectly via `Utc::now()`;
|
||||
// failures already wrote it via mark_failed above.
|
||||
let approval_kind = <&str>::from(approval.kind);
|
||||
let status_str = if ok { "approved" } else { "failed" };
|
||||
coord.emit_approval_resolved(crate::coordinator::ApprovalResolved {
|
||||
id: approval.id,
|
||||
agent: approval.agent.as_str(),
|
||||
approval_kind,
|
||||
status: status_str,
|
||||
note: note.clone(),
|
||||
description: approval.description.clone(),
|
||||
});
|
||||
result
|
||||
}
|
||||
|
||||
/// Fast-forward `applied/main` to `target` and sync the working tree.
|
||||
///
|
||||
/// Compare-and-swap, not a bare set: `main` must still be `expected_main`, the
|
||||
|
|
@ -284,36 +157,6 @@ pub fn destroy(coord: &Arc<Coordinator>, name: &str, purge: bool) {
|
|||
coord.emit_rebuild_queue_snapshot();
|
||||
}
|
||||
|
||||
pub fn deny(coord: &Coordinator, id: i64, note: Option<&str>) -> Result<()> {
|
||||
let approval = coord.approvals.get(id)?;
|
||||
coord.approvals.mark_denied(id, note)?;
|
||||
tracing::info!(%id, note, "approval denied");
|
||||
if let Some(a) = approval {
|
||||
let approval_kind = <&str>::from(a.kind);
|
||||
let description = a.description.clone();
|
||||
let agent_owned = a.agent.clone();
|
||||
coord.notify_submitter(
|
||||
a.id,
|
||||
&HelperEvent::ApprovalResolved {
|
||||
id: a.id,
|
||||
agent: a.agent.to_string(),
|
||||
commit_ref: a.commit_ref,
|
||||
status: ApprovalStatus::Denied,
|
||||
note: note.map(String::from),
|
||||
},
|
||||
);
|
||||
coord.emit_approval_resolved(crate::coordinator::ApprovalResolved {
|
||||
id,
|
||||
agent: agent_owned.as_str(),
|
||||
approval_kind,
|
||||
status: "denied",
|
||||
note: note.map(String::from),
|
||||
description,
|
||||
});
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::{RevAdvance, advance_applied_main};
|
||||
|
|
|
|||
|
|
@ -10,7 +10,6 @@ use std::sync::{Arc, Mutex};
|
|||
use anyhow::{Context, Result};
|
||||
use tokio::sync::{broadcast, watch};
|
||||
|
||||
use crate::approvals::Approvals;
|
||||
use crate::broker::Broker;
|
||||
use crate::container_view::{self, ContainerView};
|
||||
use crate::dashboard_events::DashboardEvent;
|
||||
|
|
@ -30,7 +29,6 @@ const LAST_STOPPED_RUNNING_KEY: &str = "last_stopped_running";
|
|||
|
||||
pub struct Coordinator {
|
||||
pub broker: Arc<Broker>,
|
||||
pub approvals: Arc<Approvals>,
|
||||
/// Scheduled-prompt rows, read-only: the dashboard lists them so the
|
||||
/// operator can recreate them with `swarmctl schedule`. Nothing on
|
||||
/// the hive writes or fires them.
|
||||
|
|
@ -170,7 +168,7 @@ pub struct Coordinator {
|
|||
/// Unified wire-facing event channel feeding the dashboard SSE
|
||||
/// stream. Carries broker messages (mirrored from `broker.subscribe`
|
||||
/// by the forwarder task in `main.rs`) and dashboard-only mutation
|
||||
/// events (approval added/resolved, etc.).
|
||||
/// events (container state, rebuild queue, etc.).
|
||||
/// Snapshot endpoints capture `event_seq` before reading state so
|
||||
/// the client can dedupe its buffered live traffic against the
|
||||
/// snapshot.
|
||||
|
|
@ -413,19 +411,6 @@ impl Drop for MetaUpdateGuard {
|
|||
}
|
||||
}
|
||||
|
||||
/// Field-named payload for [`Coordinator::emit_approval_resolved`].
|
||||
/// Mirrors the `ApprovalResolved` dashboard-event fields. `agent`
|
||||
/// borrows from the caller; `approval_kind` / `status` are
|
||||
/// compile-time constants.
|
||||
pub struct ApprovalResolved<'a> {
|
||||
pub id: i64,
|
||||
pub agent: &'a str,
|
||||
pub approval_kind: &'static str,
|
||||
pub status: &'static str,
|
||||
pub note: Option<String>,
|
||||
pub description: Option<String>,
|
||||
}
|
||||
|
||||
/// The two ways [`Coordinator::set_paused_by_name`] can fail. See its own
|
||||
/// doc comment for why this is a distinct type rather than one
|
||||
/// `anyhow::Error` (`Write` already wraps one, `BadName` never needs to).
|
||||
|
|
@ -467,7 +452,6 @@ impl Coordinator {
|
|||
agent_io_weight,
|
||||
} = env;
|
||||
let broker = Broker::open(db_path).context("open broker")?;
|
||||
let approvals = Approvals::open(db_path).context("open approvals")?;
|
||||
let scheduled_prompts = crate::scheduled_prompts::ScheduledPrompts::open(db_path)
|
||||
.context("open scheduled_prompts")?;
|
||||
// BuildLogs wants a directory (it picks its own `build_logs.sqlite`
|
||||
|
|
@ -488,7 +472,6 @@ impl Coordinator {
|
|||
let (shutdown_tx, _) = watch::channel(false);
|
||||
Ok(Self {
|
||||
broker: Arc::new(broker),
|
||||
approvals: Arc::new(approvals),
|
||||
scheduled_prompts: Arc::new(scheduled_prompts),
|
||||
build_logs,
|
||||
hyperhive_flake,
|
||||
|
|
@ -730,41 +713,6 @@ impl Coordinator {
|
|||
self.meta_updates_active.load(Ordering::SeqCst) > 0
|
||||
}
|
||||
|
||||
/// Emit `ApprovalResolved` after `mark_approved` / `mark_denied` /
|
||||
/// `mark_failed` lands. `resolved_at` is stamped from the system
|
||||
/// clock here so call sites don't repeat the conversion; if you
|
||||
/// already have an authoritative timestamp from the db update,
|
||||
/// the tiny skew between "row updated" and "event emitted" is
|
||||
/// presentation-only and doesn't matter to clients.
|
||||
///
|
||||
/// Takes [`ApprovalResolved`] rather than a positional arg list so
|
||||
/// the seven fields are named at every call site.
|
||||
pub fn emit_approval_resolved(&self, ev: ApprovalResolved<'_>) {
|
||||
let ApprovalResolved {
|
||||
id,
|
||||
agent,
|
||||
approval_kind,
|
||||
status,
|
||||
note,
|
||||
description,
|
||||
} = ev;
|
||||
let resolved_at = std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.ok()
|
||||
.and_then(|d| i64::try_from(d.as_secs()).ok())
|
||||
.unwrap_or(0);
|
||||
self.emit_dashboard_event(DashboardEvent::ApprovalResolved {
|
||||
seq: self.next_seq(),
|
||||
id,
|
||||
agent: agent.to_owned(),
|
||||
approval_kind,
|
||||
status,
|
||||
resolved_at: hive_sh4re::wire_time::from_secs(resolved_at),
|
||||
note,
|
||||
description,
|
||||
});
|
||||
}
|
||||
|
||||
/// Rebuild the per-container snapshot, diff it against the last
|
||||
/// one cached on `self`, and emit one
|
||||
/// `DashboardEvent::ContainerStateChanged` per added/changed row
|
||||
|
|
@ -844,7 +792,7 @@ impl Coordinator {
|
|||
.with_context(|| format!("create agent dir {}", agent_dir.display()))?;
|
||||
let socket_path = Self::socket_path(name);
|
||||
// Hand the full Coordinator to the per-agent socket — it needs
|
||||
// more than just the broker (approvals, ...).
|
||||
// more than just the broker.
|
||||
let socket = socket_server::start(name, &socket_path, self.clone())?;
|
||||
self.agents.lock().unwrap().insert(name.to_owned(), socket);
|
||||
Ok(agent_dir)
|
||||
|
|
@ -1094,28 +1042,6 @@ impl Coordinator {
|
|||
self.notify_agent(hive_sh4re::manager::MANAGER_AGENT, event);
|
||||
}
|
||||
|
||||
/// Route an approval-scoped helper event to the agent that submitted
|
||||
/// approval `approval_id` (the authenticated socket caller at submit
|
||||
/// time). Legacy rows with no recorded submitter — and any lookup
|
||||
/// failure — fall back to the root agent, preserving the prior
|
||||
/// always-root behaviour.
|
||||
pub fn notify_submitter(&self, approval_id: i64, event: &hive_sh4re::manager::HelperEvent) {
|
||||
let target = self.submitter_or_manager(approval_id);
|
||||
self.notify_agent(&target, event);
|
||||
}
|
||||
|
||||
/// Shared resolution for "which agent should hear about approval
|
||||
/// `approval_id`" — the authenticated socket caller at submit time,
|
||||
/// falling back to the manager for legacy rows with no recorded
|
||||
/// submitter (or any lookup failure). Used by both `notify_submitter`
|
||||
/// and `push_todo_submitter` so the fallback rule lives in one place.
|
||||
fn submitter_or_manager(&self, approval_id: i64) -> String {
|
||||
self.approvals
|
||||
.submitter_of(approval_id)
|
||||
.unwrap_or_default()
|
||||
.unwrap_or_else(|| hive_sh4re::manager::MANAGER_AGENT.to_owned())
|
||||
}
|
||||
|
||||
/// Push a `HelperEvent` into an arbitrary agent's inbox. Encoded
|
||||
/// the same way as `notify_manager` (sender = `SYSTEM_SENDER`,
|
||||
/// body = JSON-encoded event) — e.g. `ContainerCrash`, `NeedsUpdate`.
|
||||
|
|
|
|||
|
|
@ -1,104 +0,0 @@
|
|||
//! Approval endpoints for the dashboard.
|
||||
//!
|
||||
//! Approve/deny actions plus the orphan-approval GC sweep used by the
|
||||
//! `/api/state` builder.
|
||||
|
||||
use axum::{
|
||||
extract::{Form, Path as AxumPath, State},
|
||||
http::StatusCode,
|
||||
response::{IntoResponse, Response},
|
||||
};
|
||||
use hive_sh4re::approvals::Approval;
|
||||
use serde::Deserialize;
|
||||
use utoipa::ToSchema;
|
||||
|
||||
use super::{AppState, error_response};
|
||||
use crate::actions;
|
||||
use crate::coordinator::Coordinator;
|
||||
|
||||
/// Approve a pending approval row.
|
||||
#[utoipa::path(
|
||||
post,
|
||||
path = "/api/approve/{id}",
|
||||
params(("id" = i64, Path, description = "approval row id")),
|
||||
responses(
|
||||
(status = 200, description = "approved", body = String),
|
||||
(status = 500, description = "approve failed"),
|
||||
),
|
||||
tag = "approvals"
|
||||
)]
|
||||
pub(super) async fn post_approve(
|
||||
State(state): State<AppState>,
|
||||
AxumPath(id): AxumPath<i64>,
|
||||
) -> Response {
|
||||
match actions::approve(&state.coord, id) {
|
||||
// 200 instead of 303 — `actions::approve` fires
|
||||
// `ApprovalResolved` (success path) or the eventual failure
|
||||
// event, both of which the dashboard's derived store applies
|
||||
// live. The matching form carries `data-no-refresh`.
|
||||
Ok(()) => (StatusCode::OK, "ok").into_response(),
|
||||
Err(e) => error_response(&format!("approve {id} failed: {e:#}")),
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Deserialize, Default, ToSchema)]
|
||||
pub(super) struct DenyForm {
|
||||
#[serde(default)]
|
||||
note: Option<String>,
|
||||
}
|
||||
|
||||
/// Deny a pending approval row, with an optional
|
||||
/// note (form field `note`).
|
||||
#[utoipa::path(
|
||||
post,
|
||||
path = "/api/deny/{id}",
|
||||
params(("id" = i64, Path, description = "approval row id")),
|
||||
request_body(content = DenyForm, content_type = "application/x-www-form-urlencoded"),
|
||||
responses(
|
||||
(status = 200, description = "denied", body = String),
|
||||
(status = 500, description = "deny failed"),
|
||||
),
|
||||
tag = "approvals"
|
||||
)]
|
||||
pub(super) async fn post_deny(
|
||||
State(state): State<AppState>,
|
||||
AxumPath(id): AxumPath<i64>,
|
||||
Form(form): Form<DenyForm>,
|
||||
) -> Response {
|
||||
let note = form
|
||||
.note
|
||||
.as_deref()
|
||||
.map(str::trim)
|
||||
.filter(|s| !s.is_empty());
|
||||
match actions::deny(&state.coord, id, note) {
|
||||
Ok(()) => (StatusCode::OK, "ok").into_response(),
|
||||
Err(e) => error_response(&format!("deny {id} failed: {e:#}")),
|
||||
}
|
||||
}
|
||||
|
||||
/// Filter out approvals whose agent state dir was wiped out from under us
|
||||
/// (e.g. by a test script's cleanup). Marks them failed so they fall out of
|
||||
/// `pending` on next render.
|
||||
pub(super) fn gc_orphans(coord: &Coordinator, approvals: Vec<Approval>) -> Vec<Approval> {
|
||||
approvals
|
||||
.into_iter()
|
||||
.filter(|a| {
|
||||
if Coordinator::agent_proposed_dir(&a.agent).exists() {
|
||||
true
|
||||
} else {
|
||||
let note = "agent state dir missing";
|
||||
let _ = coord.approvals.mark_failed(a.id, note);
|
||||
tracing::info!(id = a.id, agent = %a.agent, "auto-failed orphan approval");
|
||||
coord.emit_approval_resolved(crate::coordinator::ApprovalResolved {
|
||||
id: a.id,
|
||||
agent: a.agent.as_str(),
|
||||
approval_kind: <&str>::from(a.kind),
|
||||
status: "failed",
|
||||
note: Some(note.to_owned()),
|
||||
description: a.description.clone(),
|
||||
});
|
||||
false
|
||||
}
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
|
@ -103,11 +103,8 @@ pub(super) async fn post_kill(
|
|||
}
|
||||
// Manager is stoppable from the dashboard like any other
|
||||
// agent. The host's dashboard server keeps running (it's
|
||||
// hive-c0re, not the manager container), per-agent approvals
|
||||
// submitted by other sub-agents still process through the
|
||||
// host-side approval queue without the manager up, and
|
||||
// operator-driven meta-input updates work from the dashboard
|
||||
// either way.
|
||||
// hive-c0re, not the manager container), and operator-driven
|
||||
// meta-input updates work from the dashboard either way.
|
||||
if let Err(e) =
|
||||
crate::job_queue::power::stop_many(&state.coord, std::slice::from_ref(&logical), false)
|
||||
.await
|
||||
|
|
|
|||
|
|
@ -215,7 +215,7 @@ pub(super) async fn post_meta_update(
|
|||
.coord
|
||||
.job_queue
|
||||
.insert_job(|b| {
|
||||
crate::job_queue::templates::meta_update(b, inputs, None);
|
||||
crate::job_queue::templates::meta_update(b, inputs);
|
||||
Vec::new()
|
||||
})
|
||||
.expect("template-declared shapes are acyclic");
|
||||
|
|
|
|||
|
|
@ -1,6 +1,5 @@
|
|||
//! Hyperhive dashboard. Lists managed containers (with deep-links to each
|
||||
//! container's web UI), pending approvals (with unified diff vs the applied
|
||||
//! repo, plus approve/deny buttons), and the manager.
|
||||
//! container's web UI) and the manager.
|
||||
|
||||
use std::net::SocketAddr;
|
||||
use std::sync::Arc;
|
||||
|
|
@ -35,7 +34,6 @@ use crate::lifecycle;
|
|||
tags(
|
||||
(name = "health", description = "hive-wide liveness/readiness probes"),
|
||||
(name = "journal", description = "container + host journal reads"),
|
||||
(name = "approvals", description = "approve/deny pending approval rows"),
|
||||
(name = "build_logs", description = "build log headers, full rows, and raw text downloads"),
|
||||
(name = "lifecycle_ops", description = "agent container lifecycle: rebuild/restart/start/stop/pause/limits"),
|
||||
(name = "meta_inputs", description = "bulk flake-input update for the meta flake"),
|
||||
|
|
@ -49,7 +47,6 @@ use crate::lifecycle;
|
|||
)]
|
||||
struct ApiDoc;
|
||||
|
||||
mod approvals;
|
||||
mod build_logs;
|
||||
// The single validated identifier type — homed in `hive-host-sock` (the crate
|
||||
// owning agent-path facts) so every dashboard path-param validates through the
|
||||
|
|
@ -142,8 +139,6 @@ pub async fn serve(port: u16, coord: Arc<Coordinator>) -> Result<()> {
|
|||
.routes(routes!(permissions::delete_agent_permissions))
|
||||
.routes(routes!(schedules::api_schedules))
|
||||
.routes(routes!(schedules::post_rebuild_queue_cancel))
|
||||
.routes(routes!(approvals::post_approve))
|
||||
.routes(routes!(approvals::post_deny))
|
||||
.routes(routes!(lifecycle_ops::post_destroy))
|
||||
.routes(routes!(lifecycle_ops::post_kill))
|
||||
.routes(routes!(lifecycle_ops::post_restart))
|
||||
|
|
@ -360,8 +355,6 @@ mod router_build_probe {
|
|||
.routes(routes!(permissions::delete_agent_permissions))
|
||||
.routes(routes!(schedules::api_schedules))
|
||||
.routes(routes!(schedules::post_rebuild_queue_cancel))
|
||||
.routes(routes!(approvals::post_approve))
|
||||
.routes(routes!(approvals::post_deny))
|
||||
.routes(routes!(lifecycle_ops::post_destroy))
|
||||
.routes(routes!(lifecycle_ops::post_kill))
|
||||
.routes(routes!(lifecycle_ops::post_restart))
|
||||
|
|
|
|||
|
|
@ -13,8 +13,6 @@ use axum::{
|
|||
sse::{Event, KeepAlive, Sse},
|
||||
},
|
||||
};
|
||||
use chrono::{DateTime, Utc};
|
||||
use hive_sh4re::approvals::Approval;
|
||||
use serde::{Deserialize, Serialize};
|
||||
use tokio_stream::wrappers::BroadcastStream;
|
||||
use tokio_stream::{Stream, StreamExt};
|
||||
|
|
@ -24,7 +22,7 @@ use crate::container_view::ContainerView;
|
|||
|
||||
use super::meta_inputs::{MetaInputView, read_meta_inputs};
|
||||
use super::tombstones::{TombstoneView, build_tombstone_views};
|
||||
use super::{AppState, approvals, error_response, scan_validated_paths};
|
||||
use super::{AppState, error_response, scan_validated_paths};
|
||||
|
||||
#[allow(clippy::struct_excessive_bools)]
|
||||
#[derive(Serialize)]
|
||||
|
|
@ -48,10 +46,6 @@ pub(super) struct StateSnapshot {
|
|||
any_stale: bool,
|
||||
containers: Vec<ContainerView>,
|
||||
transients: Vec<TransientView>,
|
||||
approvals: Vec<ApprovalView>,
|
||||
/// Last 30 resolved approvals (approved / denied / failed), newest-
|
||||
/// first. Drives the "history" tab on the approvals section.
|
||||
approval_history: Vec<ApprovalHistoryView>,
|
||||
/// State dirs (config history + claude creds + /state/ notes) that
|
||||
/// survive after a destroy-without-purge. The operator can re-spawn
|
||||
/// with the same name to resume, or PURG3 to wipe them.
|
||||
|
|
@ -70,8 +64,8 @@ pub(super) struct StateSnapshot {
|
|||
/// `MetaUpdateRunning` event.
|
||||
meta_update_running: bool,
|
||||
/// Whether the hive-forge container is up. When true the dashboard
|
||||
/// links each container's config + each approval's commit into the
|
||||
/// forge's `agent-configs` repos.
|
||||
/// links each container's config into the forge's `agent-configs`
|
||||
/// repos.
|
||||
forge_present: bool,
|
||||
/// Whether `hive-gateway` is in front of this dashboard. Sourced
|
||||
/// from the `HIVE_GATEWAY_ENABLED` env var, which the c0re NixOS
|
||||
|
|
@ -126,60 +120,6 @@ struct TransientView {
|
|||
secs: u64,
|
||||
}
|
||||
|
||||
#[derive(Serialize)]
|
||||
struct ApprovalHistoryView {
|
||||
id: i64,
|
||||
agent: String,
|
||||
kind: &'static str,
|
||||
/// First 12 chars of the manager-supplied ref. None when it is empty.
|
||||
sha_short: Option<String>,
|
||||
/// `approved` / `denied` / `failed`.
|
||||
status: &'static str,
|
||||
/// RFC 3339 UTC. Renders as a relative time on the dashboard.
|
||||
resolved_at: DateTime<Utc>,
|
||||
/// Operator-supplied deny reason (for `denied`) or build error
|
||||
/// (for `failed`). None on `approved`.
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
note: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Serialize)]
|
||||
struct ApprovalView {
|
||||
id: i64,
|
||||
agent: String,
|
||||
kind: &'static str,
|
||||
/// Manager-supplied description shown on the approval card.
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
description: Option<String>,
|
||||
/// Raw `commit_ref` payload for `UpdateMetaInputs` (JSON-encoded
|
||||
/// `Vec<String>` of input names; `"[]"` = all inputs). The
|
||||
/// frontend parses this to render a human-readable card body.
|
||||
commit_ref: String,
|
||||
/// RFC 3339 UTC time the approval was queued. Rendered as a
|
||||
/// relative time on the card so the operator can spot a stale
|
||||
/// request.
|
||||
requested_at: DateTime<Utc>,
|
||||
}
|
||||
|
||||
/// Replace silent `.unwrap_or_default()` on the data sources behind
|
||||
/// `/api/state` so that whichever query degrades surfaces in journald
|
||||
/// instead of leaving the operator staring at an empty list. The
|
||||
/// dashboard still degrades to a sensible default value; the warn
|
||||
/// is just the diagnostic breadcrumb the old code swallowed.
|
||||
fn log_default<T, E>(what: &str, result: std::result::Result<T, E>) -> T
|
||||
where
|
||||
T: Default,
|
||||
E: std::fmt::Debug,
|
||||
{
|
||||
match result {
|
||||
Ok(v) => v,
|
||||
Err(e) => {
|
||||
tracing::warn!(target: "api_state", source = %what, error = ?e, "snapshot source failed; using default");
|
||||
T::default()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Window over which container crashes count toward the `agents_crashing`
|
||||
/// banner warning. Wide enough that a crash-looping container (restarted
|
||||
/// by `Restart=on-failure` every few seconds) keeps the warning lit
|
||||
|
|
@ -189,12 +129,11 @@ const CRASH_WARNING_WINDOW: std::time::Duration = std::time::Duration::from_mins
|
|||
|
||||
/// Cold-load snapshot of the whole dashboard.
|
||||
///
|
||||
/// Includes the roster, approvals (+ history),
|
||||
/// tombstones, job queue, meta inputs, and more. Live clients then
|
||||
/// follow `/api/dashboard/stream` (SSE) for incremental updates keyed
|
||||
/// off `seq`.
|
||||
/// Includes the roster, tombstones, job queue, meta inputs, and more. Live
|
||||
/// clients then follow `/api/dashboard/stream` (SSE) for incremental updates
|
||||
/// keyed off `seq`.
|
||||
// `StateSnapshot` is a large tree of nested view types (`ContainerView`,
|
||||
// `ApprovalView`, ...) with no `ToSchema` anywhere in that
|
||||
// `TransientView`, ...) with no `ToSchema` anywhere in that
|
||||
// graph; wiring it up is a schema-modelling project of its own, well past
|
||||
// "annotate what's reachable". `serde_json::Value` placeholder for now —
|
||||
// see the batch report.
|
||||
|
|
@ -252,19 +191,7 @@ pub(super) async fn api_state(
|
|||
let containers = state.coord.containers_snapshot().await;
|
||||
let any_stale = containers.iter().any(|c| c.needs_update);
|
||||
let transient_snapshot = state.coord.transient_snapshot();
|
||||
let pending_approvals = approvals::gc_orphans(
|
||||
&state.coord,
|
||||
log_default("approvals.pending", state.coord.approvals.pending()),
|
||||
);
|
||||
let transients = build_transient_views(&containers, &transient_snapshot);
|
||||
let approvals = build_approval_views(pending_approvals);
|
||||
let approval_history = log_default(
|
||||
"approvals.recent_resolved",
|
||||
state.coord.approvals.recent_resolved(30),
|
||||
)
|
||||
.into_iter()
|
||||
.map(history_view)
|
||||
.collect();
|
||||
let tombstones = build_tombstone_views(&state.coord, &containers);
|
||||
let port_conflicts = build_port_conflicts(&containers);
|
||||
|
||||
|
|
@ -287,8 +214,6 @@ pub(super) async fn api_state(
|
|||
any_stale,
|
||||
containers,
|
||||
transients,
|
||||
approvals,
|
||||
approval_history,
|
||||
meta_inputs: read_meta_inputs(),
|
||||
meta_update_running: state.coord.meta_update_in_progress(),
|
||||
tombstones,
|
||||
|
|
@ -375,50 +300,6 @@ fn build_transient_views(
|
|||
out
|
||||
}
|
||||
|
||||
/// Project a resolved sqlite row into the lean shape the dashboard
|
||||
/// history tab consumes — no `diff_html` (rendering 30 of them
|
||||
/// per /api/state poll would mean 30 git diffs per refresh).
|
||||
fn history_view(a: Approval) -> ApprovalHistoryView {
|
||||
let displayed = a.commit_ref.as_str();
|
||||
let sha_short = if displayed.is_empty() {
|
||||
None
|
||||
} else {
|
||||
Some(displayed[..displayed.len().min(12)].to_owned())
|
||||
};
|
||||
let status = match a.status {
|
||||
hive_sh4re::approvals::ApprovalStatus::Approved => "approved",
|
||||
hive_sh4re::approvals::ApprovalStatus::Denied => "denied",
|
||||
hive_sh4re::approvals::ApprovalStatus::Failed => "failed",
|
||||
hive_sh4re::approvals::ApprovalStatus::Cancelled => "cancelled",
|
||||
// Pending shouldn't appear in recent_resolved, but be defensive.
|
||||
hive_sh4re::approvals::ApprovalStatus::Pending => "pending",
|
||||
};
|
||||
let kind = <&str>::from(a.kind);
|
||||
ApprovalHistoryView {
|
||||
id: a.id,
|
||||
agent: a.agent.to_string(),
|
||||
kind,
|
||||
sha_short,
|
||||
status,
|
||||
resolved_at: a.resolved_at.unwrap_or_default(),
|
||||
note: a.note,
|
||||
}
|
||||
}
|
||||
|
||||
fn build_approval_views(approvals: Vec<Approval>) -> Vec<ApprovalView> {
|
||||
approvals
|
||||
.into_iter()
|
||||
.map(|a| ApprovalView {
|
||||
id: a.id,
|
||||
agent: a.agent.to_string(),
|
||||
kind: <&str>::from(a.kind),
|
||||
description: a.description,
|
||||
commit_ref: a.commit_ref,
|
||||
requested_at: a.requested_at,
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// `/api/jobq/graph` query string — the generic jobq/wire query shape (any
|
||||
/// host serving a `GraphWire` projection over HTTP takes the same one; see
|
||||
/// `hive_jobq_wire::parse_states`, which does the actual parsing this side
|
||||
|
|
|
|||
|
|
@ -189,13 +189,6 @@ pub(super) async fn post_purge_tombstone(
|
|||
errors.push(format!("{}: {e}", dir.display()));
|
||||
}
|
||||
}
|
||||
if let Err(e) = state
|
||||
.coord
|
||||
.approvals
|
||||
.fail_pending_for_agent(name.as_str(), "agent state purged")
|
||||
{
|
||||
tracing::warn!(%name, error = ?e, "approvals: failing pending on tombstone purge failed");
|
||||
}
|
||||
if errors.is_empty() {
|
||||
tracing::info!(%name, "tombstone purged");
|
||||
// Fire the post-purge tombstones snapshot so dashboards
|
||||
|
|
|
|||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Reference in a new issue