Compare commits
40 changed files with 78 additions and 377 deletions
|
|
@ -11,11 +11,6 @@ Packages = Microsoft, alex, write-good
|
|||
|
||||
[*.md]
|
||||
BasedOnStyles = Microsoft, alex, write-good
|
||||
# Gated on CI (hyperhive#4546) once the existing backlog cleared to zero
|
||||
# (hyperhive#4548, hyperhive#4549) — every remaining hit at the time of this
|
||||
# flip is a reviewed false positive, individually wrapped in a scoped
|
||||
# `<!-- vale write-good.Passive = NO/YES -->` pair rather than bulk-ignored.
|
||||
write-good.Passive = error
|
||||
|
||||
# --- Rules disabled against evidence from our own docs/ tree, not guessed ---
|
||||
#
|
||||
|
|
|
|||
|
|
@ -1,15 +1,11 @@
|
|||
# Agent hierarchy & privileges
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
Every agent has a place in an operator-editable parent/child tree, used
|
||||
to scope which agents can manage which others. This doc covers how
|
||||
hive-c0re stores and edits the tree today, the rules that are meant to run on top
|
||||
of it once enforcement is finished, and where the manager still gets
|
||||
special-cased in the meantime, as a tracked cleanup.
|
||||
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
## Where the tree lives
|
||||
|
||||
Topology lives in the hive-c0re-owned **meta repo**, alongside
|
||||
|
|
@ -35,7 +31,7 @@ not its tree position (see _Manager special-casing today_ below).
|
|||
### Reparenting
|
||||
|
||||
- CLI: `hivectl agent <child> set-parent --parent <new>` (or `--root`
|
||||
to promote). Pass exactly one of `--parent` / `--root`.
|
||||
to promote). Exactly one of `--parent` / `--root` is required.
|
||||
- Dashboard: `POST /api/topology/set-parent` (form fields `child`,
|
||||
optional `new_parent` — absent / empty ⇒ promote to root).
|
||||
- Wire: `HostRequest::SetParent { child, new_parent: Option<String> }`.
|
||||
|
|
@ -108,8 +104,6 @@ Enforcement of the ancestor rules above isn't fully wired yet, so the
|
|||
**manager (`ruth`) still gets some hard-coded special treatment**
|
||||
other agents don't:
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
- **Naming/bootstrap** — the manager's broker recipient name, state-dir
|
||||
key, and nixos-container name are all `ruth` (container `h-ruth`).
|
||||
`hive-c0re` spawns it directly at boot if missing, with no operator
|
||||
|
|
@ -149,18 +143,17 @@ Manager}` switch picks the MCP tool allow-list claude sees. Both are
|
|||
container including the manager, so all token/state paths resolve
|
||||
through it the same way everywhere.
|
||||
- **Scattered ownership checks** — a handful of independent
|
||||
manager-only overrides exist across `hive-c0re` today: loose-ends
|
||||
visibility (manager sees hive-wide, sub-agents only their own),
|
||||
`destroy` refusing to act on the manager, and crash-watch skipping
|
||||
the manager (it autorestarts via systemd instead of going through
|
||||
the crash-watch loop). Each is planned to become an
|
||||
ancestor/descendant check instead of a manager-name check — see the
|
||||
module docs for `loose_ends.rs`, `stores/broker.rs`, `actions.rs`,
|
||||
and `workers/crash_watch.rs` for the current owner-check logic in
|
||||
each. (The harness handles reminder cancellation fully in-agent — see
|
||||
the note on `CancelLooseEndKind::Reminder` in
|
||||
`hive-c0re/src/socket_server/mod.rs`.)
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
manager-only overrides exist across `hive-c0re` today: loose-ends
|
||||
visibility (manager sees hive-wide, sub-agents only their own),
|
||||
`destroy` refusing to act on the manager, and crash-watch skipping
|
||||
the manager (it autorestarts via systemd instead of going through
|
||||
the crash-watch loop). Each is planned to become an
|
||||
ancestor/descendant check instead of a manager-name check — see the
|
||||
module docs for `loose_ends.rs`, `stores/broker.rs`, `actions.rs`,
|
||||
and `workers/crash_watch.rs` for the current owner-check logic in
|
||||
each. (The harness handles reminder cancellation fully in-agent — see
|
||||
the note on `CancelLooseEndKind::Reminder` in
|
||||
`hive-c0re/src/socket_server/mod.rs`.)
|
||||
|
||||
None of the above is a stable interface — treat the module doc
|
||||
comments as the source of truth for exactly which checks exist today.
|
||||
|
|
@ -211,8 +204,6 @@ direct mentions, reviews, and assignments.
|
|||
path = [ "/run/wrappers" "/run/current-system/sw" ];
|
||||
```
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
`/run/wrappers` (not `/run/wrappers/bin`) comes first so setuid
|
||||
wrappers — notably `sudo` — resolve before bare nix-store binaries; see
|
||||
[`docs/process/gotchas.md`](../process/gotchas.md) ("`systemd.services.*.path` appends
|
||||
|
|
@ -223,8 +214,6 @@ the non-setuid nix-store binary and every
|
|||
`services.hyperhive.agent.user.passwordlessSudo` grant fails with "must be owned by
|
||||
uid 0 and have the setuid bit set."
|
||||
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
### `serviceConfig` highlights
|
||||
|
||||
- `ExecStart = pkgs.hyperhive/bin/hive-agent` — same binary for every
|
||||
|
|
|
|||
|
|
@ -148,10 +148,9 @@ through the web UI or the forge.
|
|||
### Approval kinds (wire shapes)
|
||||
|
||||
`ApprovalKind` carries four variants; each maps to a different
|
||||
`commit_ref` encoding because `ApprovalKind` overloads that field as
|
||||
the kind-specific payload carrier.
|
||||
`commit_ref` encoding because that field is overloaded as the
|
||||
kind-specific payload carrier.
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
- `MergeConfigPr` — the config-change flow. Triggered automatically:
|
||||
when an agent opens (or force-pushes) a PR on its
|
||||
`agent-configs/<agent>` forge repo, hive-c0re's `/webhook/config-pr`
|
||||
|
|
@ -188,7 +187,6 @@ the kind-specific payload carrier.
|
|||
`source = approval:<id>`; the worker fans the body out as
|
||||
inbox messages to each target at the scheduled time, recurring
|
||||
when `interval_seconds` is set.
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
### Scheduled prompts (submit paths)
|
||||
|
||||
|
|
|
|||
|
|
@ -108,7 +108,6 @@ One table:
|
|||
- `events(id, ts, kind, payload_json)` — every `LiveEvent` the
|
||||
harness emits during turn loop execution.
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
The harness both writes and vacuums it — this used to be a host-side
|
||||
sweep, but hive-c0re runs as the unprivileged `hive-core` user under
|
||||
privsep and can't delete agent-owned files (host-side deletes hit
|
||||
|
|
@ -129,7 +128,6 @@ host-side sweep was meant to prevent — but a compromised harness is
|
|||
already inside the container trust boundary
|
||||
([`docs/trust-boundary/security.md`](../trust-boundary/security.md)), and these are ephemeral local
|
||||
artifacts, so cleaning them up where they live is the honest fix.
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
Path overridable via `HYPERHIVE_EVENTS_DB` (for dev / no-`/harness`
|
||||
setups). On open failure the `Bus` falls back to no-store mode
|
||||
|
|
@ -240,7 +238,6 @@ harness keeps serving its web UI and MCP daemons but drives no turns,
|
|||
and inbox messages queue unacked until it's removed (see
|
||||
[turn loop](../turn-loop/README.md#the-loop)).
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
Unusually, it's read and written from **both** sides of the harness
|
||||
bind-mount, and that's the whole design: the harness stats it
|
||||
in-container via `hive-agent`'s `paths::paused_marker`, while hive-c0re
|
||||
|
|
@ -251,7 +248,6 @@ dashboard toggle. Because the file itself is the only shared state
|
|||
there's no protocol between them, no round-trip into the container, and
|
||||
pause keeps working when the harness is wedged or the container is
|
||||
stopped.
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
It lives in `/harness/` rather than `/state/` deliberately: `/state/`
|
||||
is the agent's own space to fill, and this is harness control state.
|
||||
|
|
@ -350,23 +346,19 @@ nothing argues for a parent reading it, let alone writing it. hive-c0re
|
|||
reads a child's harness dir **directly on the host** when it wants
|
||||
those stats, which needs no mount into the parent.
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
**`config` is read-only, including for the parent.** A config change is
|
||||
a PR on the child's config repo, made from a clone and merged after
|
||||
review — so the bind-mounted `config` dir is a *copy to read*, never a
|
||||
tree anyone edits in place. Mounting it writable would leave a second
|
||||
path to the same file that skips the review entirely, which makes the
|
||||
boundary a convention rather than a permission.
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
⚠️ Don't confuse it with the config-repo seeding hive-c0re does at
|
||||
spawn (`lifecycle::setup_proposed`): that writes the child's initial
|
||||
config repo as **hive-c0re, against the host path**, and `read_only` on a bind
|
||||
constrains writers *inside* a container only. The two are unrelated —
|
||||
conflating them can lead you to reason your way into thinking this
|
||||
mount should be writable when it shouldn't.
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
Per-child isolation still holds: a container only ever has its *own*
|
||||
dirs plus its direct children's bind-mounted, never a sibling's.
|
||||
|
|
@ -489,12 +481,10 @@ so it survives restarts and redeploys.
|
|||
verification until the hook is re-registered with the new value. It's
|
||||
generated automatically on first start; there's nothing to configure.
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
If the file is unreadable at startup the daemon still starts and logs
|
||||
`webhook secret unavailable`; the webhook endpoint then answers 503
|
||||
rather than accepting deliveries it can't verify. Everything else the
|
||||
controller serves is unaffected.
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
## Run-time dirs
|
||||
|
||||
|
|
@ -597,7 +587,6 @@ librsvg) and forwards its store path as `HIVE_ICON_PNG` on the daemon
|
|||
unit, gated on `services.hyperhive.agent.icon != null`. No icon configured → the env is
|
||||
unset → `sync_avatar` returns early and sets no avatar.
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
Idempotency is **per-account**: an `avatar-icon-hash` file in each
|
||||
account's matrix-sdk `state_dir`. The daemon hashes the PNG bytes and
|
||||
skips the upload when unchanged, because every upload mints a fresh
|
||||
|
|
@ -607,5 +596,4 @@ account gets its avatar when the `systemd.paths.hive-matrix-daemon` token
|
|||
watcher restarts the daemon (which re-runs the per-account bring-up), so
|
||||
no separate avatar trigger is needed. The daemon swallows avatar failures
|
||||
(logged, non-fatal) so they never break account bring-up or sync.
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
|
|
|
|||
|
|
@ -10,16 +10,12 @@ sequence. All `hivectl` commands below run as **root on the host** (not
|
|||
inside an agent container); the `request_*` steps run from ruth's own
|
||||
turn via the MCP tools.
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
**Bringing up a hive that doesn't host its own swarm services?** Read
|
||||
[`swarm/secrets.md`](../swarm/secrets.md) first. Everything below assumes
|
||||
each credential is generated where it's read, which is true on an
|
||||
all-local deploy and not otherwise — that page says which files an
|
||||
operator has to place, and where.
|
||||
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
## Step-by-step
|
||||
|
||||
### 1 · Forge
|
||||
|
|
@ -161,7 +157,7 @@ leaves it alone.
|
|||
|
||||
</details>
|
||||
|
||||
What else you need depends on
|
||||
Whether anything more is needed depends on
|
||||
`services.hyperhive.deploy.bao.seal`:
|
||||
|
||||
- **`pkcs11`** (the default) — pkcs11 binds the key to the host's TPM, and
|
||||
|
|
@ -184,6 +180,8 @@ the store can't hand you, being what opens it. ⚠️ Not the gateway's HTTPS ce
|
|||
store that took its identity from an authority it will itself distribute could
|
||||
never come up before that authority.
|
||||
|
||||
Making even the `init` unnecessary is tracked as a follow-up.
|
||||
|
||||
### 4 · Swarm SSO (only when `deploy.authelia`)
|
||||
|
||||
⚠️ **Required to finish the install, not optional.** Authelia treats an
|
||||
|
|
@ -289,20 +287,17 @@ See [`tools/hivectl.md`](../tools/hivectl.md) for every `hivectl` verb.
|
|||
|
||||
## Security notes
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
- **No forge admin token is stored in any agent state dir.** Agents
|
||||
hold a regular agent token in their `forge-token` file; sensitive
|
||||
creds (the core token, the matrix admin token) live on the host.
|
||||
- All config changes (forge PRs on `agent-configs/<name>`) go through
|
||||
operator approval — agents can't unilaterally rebuild containers, by design.
|
||||
See [`boundary.md`](../trust-boundary/boundary.md) and [`security.md`](../trust-boundary/security.md).
|
||||
- **Each hive authenticates its own telemetry ingest**, and the `hive` label comes
|
||||
from which hive authenticated rather than from the payload — so no hive can
|
||||
report metrics as another. A first-run all-local hive gets this with nothing
|
||||
to configure; joining a swarm you don't host needs one secret copied across.
|
||||
See [`observability.md`](../scheduler/observability.md#authenticated-ingest).
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
- **Telemetry ingest is authenticated per hive**, and the `hive` label comes
|
||||
from which hive authenticated rather than from the payload — so no hive can
|
||||
report metrics as another. A first-run all-local hive gets this with nothing
|
||||
to configure; joining a swarm you don't host needs one secret copied across.
|
||||
See [`observability.md`](../scheduler/observability.md#authenticated-ingest).
|
||||
|
||||
Once the hive is running, ruth records anything it needs to remember
|
||||
across restarts in `/agents/ruth/state/notes.md`.
|
||||
|
|
|
|||
|
|
@ -12,15 +12,11 @@ injects the token into the agent's state dir out of band.
|
|||
|
||||
## Enabling
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
The integration is **on by default** for every agent (`services.hyperhive.agent.github.enable
|
||||
= true`), inert until the operator provisions a PAT. No per-agent declaration is
|
||||
needed — an agent gains GitHub by having a PAT written to its token
|
||||
file.
|
||||
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
To turn it off for the whole hive, set the host option:
|
||||
|
||||
```nix
|
||||
|
|
|
|||
|
|
@ -14,8 +14,6 @@ Inside any agent container:
|
|||
/knowledge/README.md # table of contents (seeded on first use)
|
||||
```
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
Agents read documents directly from that path. The mount is
|
||||
read-only — agents never write through it. To contribute, use the
|
||||
`hive-forge` AGit flow (no fork needed — see
|
||||
|
|
@ -23,8 +21,6 @@ read-only — agents never write through it. To contribute, use the
|
|||
the local clone updates automatically (see
|
||||
[Sync mechanism](#sync-mechanism) below).
|
||||
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
## Repository layout
|
||||
|
||||
Canonical forge location: `internal/knowledge` (org `internal`,
|
||||
|
|
@ -57,8 +53,8 @@ pull`, so agents see the new content on their next turn.
|
|||
URL, so a second registration against the same repo doesn't add a
|
||||
recipient — it takes delivery away from whoever registered first.
|
||||
Earlier versions had each hive register its own; hive-c0re now
|
||||
removes its own leftover at startup, so migrating needs no operator
|
||||
step.
|
||||
removes its own leftover at startup, so no operator step is needed
|
||||
to migrate.
|
||||
|
||||
2. **Periodic pull** — a background task in `hive-c0re::main`
|
||||
pulls on a fixed cadence as a fallback (webhook missed, c0re
|
||||
|
|
@ -90,15 +86,12 @@ before returning.
|
|||
|
||||
## State
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
- **Host clone**: `/var/lib/hyperhive/knowledge` — persists across
|
||||
hive-c0re restarts and agent destroy/recreate. Deleted only by
|
||||
manual operator action.
|
||||
- **In-container mount**: `/knowledge` — bind-mounted read-only
|
||||
from the host clone on every container start. Gone when container
|
||||
is stopped; reappears on next start with the current clone state.
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
from the host clone on every container start. Gone when container
|
||||
is stopped; reappears on next start with the current clone state.
|
||||
|
||||
The mount deliberately **excludes `.git`**: the host clone embeds the `core`
|
||||
token in `.git/config` (it rides the clone URL), so hive-priv overlays an empty
|
||||
|
|
|
|||
|
|
@ -92,11 +92,9 @@ delegation (the latter lives in `gateway.md::Discovery flow`).
|
|||
|
||||
## Provisioning flow (appservice)
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
Registration is closed. The hive's own **appservice** creates accounts:
|
||||
hive-c0re holds the appservice token, agents never see
|
||||
it, and an agent only ever receives its own `access_token`.
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
The appservice has no URL (`url: null` in its registration), so the
|
||||
homeserver never calls out to it and there is no service to run. What the
|
||||
|
|
@ -177,7 +175,6 @@ Nothing to do, and nothing to time. The activation script mints the
|
|||
appservice token and renders the registration before the homeserver
|
||||
restarts, so the first boot after the switch already has both halves.
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
- **Existing accounts keep working.** An access token lives on the
|
||||
device that minted it; removing the registration token touches no
|
||||
device, no account and no session. `login_with_password` stays on, so
|
||||
|
|
@ -201,7 +198,6 @@ restarts, so the first boot after the switch already has both halves.
|
|||
token only until the first successful read. See
|
||||
[`../swarm/secrets.md`](../swarm/secrets.md) for how that mint stays
|
||||
idempotent across runs.
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
</details>
|
||||
|
||||
|
|
@ -243,7 +239,6 @@ into it. This gives the operator a single Space in FluffyChat or any
|
|||
Matrix client that groups all agent-to-agent + operator rooms in one
|
||||
place.
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
The sweep also provisions a default **`hive-chat` room** as an
|
||||
`m.space.child` of the Space. Joining a Space doesn't autojoin
|
||||
child rooms — the explicit room entry ensures the operator and every
|
||||
|
|
@ -251,7 +246,6 @@ agent can find a common chat room without manual setup. Room join is
|
|||
restricted (any Space member including the operator can join; agents
|
||||
are explicitly invited). Room version pinned to 10 for the restricted
|
||||
join floor.
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
**State**: hive-c0re persists both room IDs to `/var/lib/hyperhive/matrix/`
|
||||
(mode `0600`, owned by the hive-c0re service user):
|
||||
|
|
@ -276,9 +270,9 @@ services.hyperhive.swarm.matrix = {
|
|||
```
|
||||
|
||||
**`trustedServers`** (default `[]`) — list of peer homeserver names
|
||||
whose signing keys tuwunel will fetch and trust. tuwunel enables federation
|
||||
whose signing keys tuwunel will fetch and trust. Federation is enabled
|
||||
at the protocol level from first boot (`allow_federation = true`) but
|
||||
trusts no remote homeserver until you list it here. For a closed
|
||||
tuwunel trusts no remote homeserver until you list it here. For a closed
|
||||
single-hive deployment the default empty list is correct — add peer
|
||||
hive domains here when connecting hives into a swarm (see
|
||||
[`docs/swarm/`](../swarm/README.md)).
|
||||
|
|
@ -302,14 +296,14 @@ surprising behaviour:
|
|||
SSO is unconditional, so the three below are requirements of running a
|
||||
homeserver at all rather than of a setting:
|
||||
|
||||
- **Set `sso.clientSecretFile`** — fails at eval, not at boot:
|
||||
- **`sso.clientSecretFile` is required** — fails at eval, not at boot:
|
||||
tuwunel reads its identity providers from the config file, so a
|
||||
half-configured one can stop the homeserver from starting outright
|
||||
rather than merely hiding a login button. On a host that also runs
|
||||
the swarm's authelia it's wired up for you.
|
||||
- **Set `swarm.authelia.url`** — without a provider URL there
|
||||
- **`swarm.authelia.url` is required** — without a provider URL there
|
||||
is nothing to discover against.
|
||||
- **Set `gatewayHost != null`** — the SSO callback URL is
|
||||
- **`gatewayHost != null` is required** — the SSO callback URL is
|
||||
format-locked to `<homeserver>/_matrix/client/unstable/login/sso/callback/<client_id>`,
|
||||
and the identity provider needs a public name to redirect the
|
||||
browser to.
|
||||
|
|
|
|||
|
|
@ -19,9 +19,7 @@ This host's nginx fronts the hyperhive web surfaces running on it — next to hi
|
|||
|
||||
Only the host that **runs** authelia declares the authelia vhost, not every hive that uses it — a client hive knows the swarm's `authelia.url` but must not answer for a name it doesn't serve. Its server name is exactly `swarm.authelia.domain`: authelia validates `authelia_url ⊂ session cookie domain` at startup, so a near-miss is a container that refuses to boot. It carries no `auth_basic` — the login page must not sit behind the login mechanism it replaces — and sets the four `X-Forwarded-{Proto,Host,Uri,For}` headers, since authelia decides by the *original* request rather than the hop it sees.
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
⚠️ **A `502` from this vhost typically means authelia has no users yet, not that the proxy is misconfigured.** Authelia treats an empty user store as a fatal startup error, so an enabled-but-unbootstrapped swarm crash-loops the container while the vhost in front of it works perfectly. Check `journalctl -M swarm-authelia -u authelia-swarm` before suspecting anything here; the bootstrap step is in [`swarm/sso.md`](../swarm/sso.md).
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
Per-agent UIs stay sub-path, forge and matrix get sub-domains — see
|
||||
[Sub-domain shape (rationale)](#sub-domain-shape-rationale) below for why.
|
||||
|
|
@ -36,9 +34,7 @@ Operator points client at `<hive>`. Sequence:
|
|||
|
||||
matrix-dart-sdk (FluffyChat etc.) hardcodes `https` for the well-known fetch regardless of input scheme, so the discovery endpoint MUST be https — see "Self-signed TLS" below for the cert generation that backs the default-on path.
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
Federation peers fetch `.well-known/matrix/server` → `{"m.server":"chat.<swarm>:<httpsPort>"}` (the federation delegation always carries an explicit port, even the HTTPS default 443 — the https-implies-443 elision only applies to the client base_url above). Gateway only listens on configured `port` (+ `httpsPort` when TLS on); cross-hive federation needs either an SRV record (`_matrix._tcp.chat.<swarm>` → port 80 / 443) OR `matrix.openFirewall = true` so peers reach tuwunel's federation port directly. Hyperhive is closed/internal in most deployments, so this rarely bites.
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
## SPA fallback (Accept-header pattern)
|
||||
|
||||
|
|
@ -138,14 +134,12 @@ now set unconditionally for every agent. The mechanism:
|
|||
retry and report on (`RELOAD_PENDING`), not two units racing on an
|
||||
inotify event.
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
c0re regenerates `agents.conf` (and triggers a reload) on two
|
||||
triggers: every topology change (new/removed agents) and every 10s
|
||||
marker poll tick (`agent_sockets::spawn_poll`). `write()` is
|
||||
idempotent — skips the rename when content is unchanged.
|
||||
`gateway_nginx::reload_if_pending` automatically retries failed reloads
|
||||
on subsequent poll ticks.
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
`agents.conf` uses atomic `<path>.tmp` + `rename()` writes so a crashing
|
||||
c0re process never leaves a partial or unparseable file behind.
|
||||
|
|
@ -235,9 +229,7 @@ swarm-services fallback below is for.
|
|||
|
||||
**Cert shape**: leaf subject CN = bare hive domain; subjectAltName is `<hive>` plus wildcard `*.<hive>`, so all current and future sub-domain vhosts validate under the same leaf + the hive CA. You can't add a swarm service whose name is *not* under this hive's domain here — the hive CA is name-constrained to `<hive>`, and a violating SAN invalidates the whole leaf, not just that name. Those names get the swarm-services leaf instead ([`swarm/ca.md`](../swarm/ca.md)).
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
**Rotation**: `hive-tls-ca.service` is idempotent — it re-signs the leaf when it's missing or within 30 days of expiry, always under the same CA (so consumer trust is undisturbed). It regenerates the CA itself only if missing or already expired. To force a leaf rotation, delete `gateway.pem` under the state dir and restart the unit, then reload `nginx`.
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
**Cert prompts**: browsers still warn once per host until the operator adds the hive's `trust-bundle.pem` to the browser/OS trust store (an anchor, not the leaf, is the thing to trust). A separate mechanism wires agent trust (see the agent-trust work for `/run/hive-ca`).
|
||||
|
||||
|
|
@ -300,7 +292,6 @@ TLS (see [TLS modes](#tls-modes) above).
|
|||
Every agent hashes into the same port range (no special case), so
|
||||
one range opening covers every container.
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
The dashboard port (`services.hyperhive.c0re.dashboardPort`, default 7000) is *not*
|
||||
listed in either case — it binds `127.0.0.1` only, so a firewall
|
||||
hole would be a no-op. Remote dashboard access flows through the
|
||||
|
|
@ -308,7 +299,6 @@ gateway. Operators who opt out of the gateway lose external
|
|||
dashboard reach by design — the surface is privileged (approve /
|
||||
deny / destroy), and operators must not expose it without a real reverse
|
||||
proxy in front.
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
## `HIVE_FORGE_URL`: agents reach the forge via the gateway by domain
|
||||
|
||||
|
|
@ -318,7 +308,7 @@ shared-netns mode no longer exists), so agents run in a private netns and
|
|||
can never reach the host's loopback.
|
||||
`nix/host-modules/hive-c0re/environment.nix` sets `HIVE_FORGE_URL` to
|
||||
`http://<forge.domain>` (default `forge.<swarm-domain>` — a swarm runs
|
||||
one forge; you must set `services.hyperhive.domain`). Agents
|
||||
one forge; `services.hyperhive.domain` is required). Agents
|
||||
get the bridge dnsmasq as their resolver, resolve the hostname →
|
||||
bridge IP, then reach nginx on port 80 (the bridge firewall opens
|
||||
80+443). nginx proxies to forgejo — the same path an operator browser
|
||||
|
|
@ -367,7 +357,6 @@ only if you already have another process bound to 3000.
|
|||
uses for `git clone/push/pull` over SSH (`git@<domain>:owner/repo.git`
|
||||
via `-p 2222`). Port 22 stays alone on the host for openssh.
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
`openFirewall` (default **false**) controls whether the host firewall
|
||||
opens `httpPort` and `sshPort`. Off by default (secure by
|
||||
default): agents reach Forgejo through the gateway (`forge.<swarm-domain>` on
|
||||
|
|
@ -376,7 +365,6 @@ the bridge), not the raw port, so no firewall hole is needed. Flip to
|
|||
- The operator's browser to reach `http://<host>:<httpPort>/` directly
|
||||
(not behind the gateway).
|
||||
- External git clients that push/pull via SSH directly to the host.
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
Forgejo served through the gateway (`deploy.forgejo.behindGateway = true`) does
|
||||
not need `openFirewall` — the gateway's own `openFirewall` option covers
|
||||
|
|
@ -507,14 +495,12 @@ palette (`#1e1e2e` bg, `#cdd6f4` text, `#cba6f7` heading). No
|
|||
dependencies on the frontend dist — these pages render even when
|
||||
hive-c0re itself is down.
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
Scope is intentionally narrow: a route earns a custom page when the
|
||||
default status code would point at the wrong component. The per-agent
|
||||
routes qualify (a 502 there means the harness is restarting, not that
|
||||
the gateway is broken), and so does `auth.<swarm>` — a dead authelia
|
||||
upstream almost always means the user store was never bootstrapped, and
|
||||
a bare 502 blames the proxy, which is the one part that's working.
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
Forge / matrix / fluffychat still get nginx defaults: their upstreams
|
||||
being down means what the status code says, so a themed page would add
|
||||
|
|
@ -522,12 +508,10 @@ styling and no information.
|
|||
|
||||
## HTTP Basic auth
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
`services.hyperhive.gateway.auth.enable = true` gates every request to
|
||||
the main vhost (`_`) behind HTTP Basic auth. nginx's built-in `auth_basic`
|
||||
module validates credentials; no extra service or host-side daemon is
|
||||
required.
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
**Setup:**
|
||||
|
||||
|
|
@ -538,12 +522,10 @@ services.hyperhive.gateway.auth = {
|
|||
};
|
||||
```
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
The credential store lives at the fixed path
|
||||
`/var/lib/hive-gateway/conf/gateway.htpasswd` on the host. A tmpfiles
|
||||
rule pre-creates the file on first boot; no manual path configuration
|
||||
is required. nginx reads it at that path directly.
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
Manage users with `hivectl gateway`. `hivectl` sends the request over the
|
||||
host admin socket and the `hive-c0re` daemon performs the write at its
|
||||
|
|
@ -563,11 +545,9 @@ hivectl gateway delete-user bob
|
|||
hivectl gateway list-users
|
||||
```
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
The daemon hashes passwords with BCrypt (cost 12) and writes
|
||||
`$2y$`-prefixed hashes that nginx accepts natively. No external
|
||||
`htpasswd` binary is required.
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
**What's not gated:** per-agent UI routes emitted into `agents.conf`
|
||||
(served under `/agent/<name>/`) inherit no auth from `/` — nginx
|
||||
|
|
@ -632,7 +612,6 @@ TLS-less mode that could violate it.
|
|||
|
||||
## Dialing another vhost by name (`verifiedProxyTo`)
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
`vhost-lib.nix`'s `verifiedProxyTo` builds the `proxy_ssl_*` /
|
||||
`proxy_set_header` block a module uses to dial another service on this
|
||||
same gateway BY NAME over https, verified. One definition rather than a
|
||||
|
|
@ -640,7 +619,6 @@ copy per module: nginx verifies nothing by default
|
|||
(`proxy_ssl_verify` is off), so a `proxy_pass https://…` without these
|
||||
lines is encrypted and unauthenticated. That failure is invisible — it
|
||||
works, and keeps working, against any certificate at all.
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
Every line earns its place, each confirmed against a real nginx with
|
||||
the opposite arm run as a control:
|
||||
|
|
|
|||
|
|
@ -15,8 +15,8 @@ Configured via `services.hyperhive.network.*`.
|
|||
One picture of the whole hive — two planes: **infra
|
||||
containers share the host netns** and bind host ports directly;
|
||||
**compute containers (agents + CI) each get a private netns** behind
|
||||
the bridge. The unix-socket control plane rides the VFS, and none of
|
||||
that touches it.
|
||||
the bridge. The unix-socket control plane rides the VFS and is
|
||||
untouched by any of it.
|
||||
|
||||
```
|
||||
internet
|
||||
|
|
@ -192,16 +192,12 @@ listed TCP port `P` on the bridge-interface `allowedTCPPorts`, so an
|
|||
agent can connect to `<bridgeIp>:P` (point the collector endpoint at
|
||||
`http://<bridgeIp>:4318`, default `http://10.42.0.1:4318`).
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
This is **firewall-only**: the host service must bind an address
|
||||
reachable from the bridge — `0.0.0.0` or the bridge IP — not loopback
|
||||
only. The bridge→`127.0.0.0/8` DROP rule (below) is unchanged, so a
|
||||
service bound to `127.0.0.1` only stays unreachable; rebind it to
|
||||
`0.0.0.0`.
|
||||
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
The port is reachable by **every** agent on the bridge subnet (like
|
||||
DNS/gateway), so only expose services safe for any agent to reach.
|
||||
|
||||
|
|
|
|||
|
|
@ -188,31 +188,25 @@ of `state/`, `claude/` and `harness/` (see
|
|||
section](../agent-lifecycle/persistence.md#btrfs-subvolumes-for-varlibhyperhiveagentsname)
|
||||
for how and when hive-c0re creates that subvolume). Consequences:
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
- The Claude session (`claude/`) travels, so a restored agent keeps its
|
||||
live `--continue` session rather than needing to log in again.
|
||||
- `harness/` travels too, including `harness/bash-tasks/`. Task output
|
||||
is part of an agent's working continuity, so this is wanted --- but it
|
||||
means anything that has ever leaked into a task's captured output is
|
||||
in the retained snapshots as well.
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
is part of an agent's working continuity, so this is wanted --- but it
|
||||
means anything that has ever leaked into a task's captured output is
|
||||
in the retained snapshots as well.
|
||||
|
||||
It does **not** cover the agent's applied config (`/applied/<name>/`) or
|
||||
its topology entry, both of which live outside the subvolume. A restore
|
||||
therefore yields an agent's memory without its definition.
|
||||
therefore yields an agent's memory without its definition; closing that
|
||||
gap is tracked separately.
|
||||
|
||||
### Retention
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
Retention lives on the _sending_ side (last-N by count, swept
|
||||
periodically), not here. Count rather than age is deliberate: a count
|
||||
is bounded by construction, whereas an age policy silently scales disk
|
||||
usage with how hot a hive runs.
|
||||
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
Per-agent or per-hive `btrfs qgroup` quotas aren't configured yet.
|
||||
Without them one runaway hive can fill the store and take out every
|
||||
other hive's snapshots.
|
||||
|
|
|
|||
|
|
@ -132,14 +132,12 @@ always passes `wait_seconds: None`, an immediate peek.
|
|||
|
||||
Per-row bookkeeping inside the broker:
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
- `delivered_at = NOW` set on every popped row.
|
||||
- Each recipient has an in-memory `unacked_ids` list of every row
|
||||
delivered since the last `AckTurn`.
|
||||
- `redelivered = true` on a row if `RequeueInflight` resurfaced it
|
||||
(the harness prepends a "may already be handled" hint when this
|
||||
flag is set so the per-message warning is visible).
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
`AgentRequest::AckTurn` closes out the in-memory list — the harness
|
||||
fires it after `TurnOutcome::Ok`, marking every message popped since
|
||||
|
|
@ -271,7 +269,7 @@ changes, and deserialization leniently accepts both the string form
|
|||
and the legacy bare integer (rolling-deploy skew, persisted blobs).
|
||||
**Input-direction** fields agents compute as epoch (`first_fire_at_unix`,
|
||||
schedule-edit `next_fire_at_unix`, `Wakeup::At`) stay integers. The
|
||||
`*_unix` field *names* stay for now — renaming is the wire-types
|
||||
`*_unix` field *names* are kept for now — renaming is the wire-types
|
||||
refactor's concern. The dashboard frontend parses via
|
||||
`util.js::epochSec` wherever it needs arithmetic and feeds the string
|
||||
straight to `new Date(s)` for display.
|
||||
|
|
|
|||
|
|
@ -93,8 +93,6 @@ Two things to get right when a daemon needs outbound TCP:
|
|||
enumerate local addresses before it returns any, so name resolution
|
||||
fails without it even when the unit allows `AF_INET`.
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
**The directive is a claim about what the program does, and nothing
|
||||
re-checks it when the program changes.** A unit that only served a unix
|
||||
socket when it was written is correct at `[ "AF_UNIX" ]` and silently wrong
|
||||
|
|
@ -103,8 +101,6 @@ the client — and when narrowing it, prefer a test that derives the required
|
|||
families from the code (which fails on the _next_ client too) over one that
|
||||
asserts today's list.
|
||||
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
### `register_agent` is idempotent
|
||||
|
||||
Drops any prior socket task before rebinding. Required so a
|
||||
|
|
@ -118,7 +114,7 @@ socket without needing a clean reinstall.
|
|||
`claude-code` comes from the flake's main `nixpkgs` (nixos-26.05).
|
||||
It's unfree, so the agent modules set `config.allowUnfreePredicate`
|
||||
at the container level to allowlist `claude-code` specifically —
|
||||
scoped, only this one package, because each per-agent
|
||||
scoped, only this one package. This is needed because each per-agent
|
||||
`nixosConfiguration` evaluates its own nixpkgs instance and the
|
||||
operator's host-level `allowUnfree` does **not** propagate in.
|
||||
Operators don't need to set anything on their side.
|
||||
|
|
@ -142,8 +138,6 @@ reference-less `-source` (so the runtime closure never arrives), and
|
|||
pure evaluation rejects. `services.hyperhive.agent.docs.source` gets away with being
|
||||
an input only because a docs tree has no runtime dependencies.
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
The `storePath` trap is worth spelling out, because it's not confined
|
||||
to options the operator writes: **any** option of type `package` fed a
|
||||
store-path _string_ coerces through `lib.toDerivation`, i.e.
|
||||
|
|
@ -155,10 +149,6 @@ container as text in a real derivation — a symlink farm built from
|
|||
the interpolated string — before it can go anywhere a package is
|
||||
expected.
|
||||
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
The catch is that a path written into a generated flake is text, not a
|
||||
reference — the container's closure doesn't keep the binary alive.
|
||||
The **host** does: it interpolates the package into
|
||||
|
|
@ -170,8 +160,6 @@ the hive's `claude` out from under it. The price of the root is that
|
|||
`nix-collect-garbage` can't reclaim an old `claude-code` until every
|
||||
agent has rebuilt past it and the old generations are gone.
|
||||
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
### Claude credentials are per-agent
|
||||
|
||||
`/var/lib/hyperhive/agents/<name>/claude/` bind-mounts to
|
||||
|
|
@ -327,8 +315,6 @@ CI on drift).
|
|||
|
||||
### Split asset derivations away from the rust workspace
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
`nix/packages/assets.nix` builds the branding SVG/PNG family + claude
|
||||
system-prompt template + claude-settings JSON as its own derivation,
|
||||
separate from the hive-ag3nt / hive-c0re crates. Reason: when the
|
||||
|
|
@ -341,8 +327,6 @@ its own — the split breaks the coupling at the derivation boundary.
|
|||
build time; librsvg dependency lives here, not in the rust
|
||||
derivation's `nativeBuildInputs`.
|
||||
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
### `nix fmt` fails in a git worktree with "object not found"
|
||||
|
||||
`nix fmt` (and any `nix` command that fetches a `git+file://` flake
|
||||
|
|
|
|||
|
|
@ -38,8 +38,6 @@ independently.
|
|||
|
||||
## What arming automerge actually means
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
Automerge isn't "no human ever looked at this." Whoever arms it has
|
||||
already judged the PR sound at a coarse level — the signal it sends is
|
||||
roughly _"apart from maybe minor tweaks a reviewer can still catch,
|
||||
|
|
@ -48,8 +46,6 @@ already happened. No large changes are expected to surface after
|
|||
that point — a reviewer's job past that point is to flag it if one
|
||||
does, not to assume none ever will.
|
||||
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
The practical consequence for a reviewer: on a repo where someone with
|
||||
write access may already have armed automerge before your review
|
||||
lands, a plain `APPROVED` can
|
||||
|
|
|
|||
|
|
@ -31,8 +31,6 @@ writeup.
|
|||
|
||||
## CI checks
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
Several jobs run on every PR (and on `workflow_dispatch` for manual
|
||||
re-triggers), defined in
|
||||
[`.forgejo/workflows/ci.yml`](../../.forgejo/workflows/ci.yml). The first four
|
||||
|
|
@ -43,8 +41,6 @@ gating on the full job's standing warning/suggestion backlog, but flipping
|
|||
that switch on the forge is a separate operator action, not something this
|
||||
table can assert ahead of it happening.
|
||||
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
| Job | What it runs | Required? |
|
||||
| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------- | ------------------------------- |
|
||||
| **nix flake check** | treefmt formatting, `cargo clippy -D warnings`, `cargo test`, module evaluation | yes |
|
||||
|
|
@ -58,8 +54,6 @@ table can assert ahead of it happening.
|
|||
`hive-forge ci-rerun --pr N` dispatches a `workflow_dispatch` retrigger
|
||||
without an empty commit.
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
**`ci-rerun --pr` verifies the code but doesn't reliably move the PR's own
|
||||
status checks.** Measured directly (raw `GET /repos/.../actions/tasks`
|
||||
JSON): a genuine PR-triggered run carries `event: pull_request` with the
|
||||
|
|
@ -76,8 +70,6 @@ ci-log --run <n> --job 0` shows the code green (verdict is the
|
|||
last line) but the status is stuck, ask the operator to select "rerun" in
|
||||
the forge web UI — it's CSRF-gated, so only they can do it.
|
||||
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
### Running checks locally
|
||||
|
||||
Don't run `nix flake check` directly — it dispatches to the shared build farm and
|
||||
|
|
@ -152,16 +144,12 @@ Gated on `HYPERHIVE_FORGE_CI_ENABLED` (the nix module sets it on `hive-c0re.serv
|
|||
|
||||
## Actions checkout mirror
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
When `deploy.forgejo.ci.enable` is set, hive-c0re autoseeds an
|
||||
`actions/checkout` pull-mirror on the local forge and sets Forgejo's
|
||||
`DEFAULT_ACTIONS_URL` to point at the local instance. This means CI
|
||||
`uses: actions/checkout@vN` steps resolve entirely on loopback — no
|
||||
external DNS on the CI critical path.
|
||||
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
**hive-c0re** itself seeds the mirror during its forge
|
||||
provisioning sweep (`forge/repos.rs::ensure_mirrors`). The nix module
|
||||
forwards the effective mirror list as `HYPERHIVE_FORGE_MIRRORS` in the
|
||||
|
|
|
|||
|
|
@ -53,8 +53,6 @@ Nix-heavy — hold one of the `buildSlots` permits for the node's duration:
|
|||
|
||||
Cheap — no build slot:
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
| Node | Behavior |
|
||||
| -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `MergeVerify` | the deploy's pre-merge gate — PR-head drift check, fetch, `verify_commit` eval. Mutates nothing, so a rejection here needs no compensation |
|
||||
|
|
@ -87,8 +85,6 @@ Cheap — no build slot:
|
|||
| `KnowledgePull` | one-shot boot-time `/knowledge` pull (`knowledge::pull`), reconciling commits that landed while `hive-c0re` was down. Same rationale as `MatrixSweep`: the periodic hourly re-pull stays a background loop |
|
||||
| `WantedPull` | one-shot boot-time pull of the agent set the swarm controller declares for this hive (`wanted::pull`), converging the agents it names. No background loop behind this one — boot is the whole cadence; the deploy event (`swarm_status`) is the fast path, this repairs a missed one. Agentless |
|
||||
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
Deliberately, **no `GitCommit` node** exists: `meta.rs` fuses each mutation
|
||||
with its commit under its internal `META_LOCK` mutex, so a standalone commit
|
||||
node would open a dirty-working-tree window between nodes.
|
||||
|
|
@ -210,13 +206,9 @@ they surface under that DAG's id in the same loop — no separate child DAGs.
|
|||
|
||||
### Scheduler semantics
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
A node is **ready** when it's `Queued`, every dep is satisfied, and its
|
||||
resources are free. Resources:
|
||||
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
1. **Build slots** — `services.hyperhive.c0re.buildSlots` permits (default 1),
|
||||
held by nix-heavy nodes for the node's duration.
|
||||
2. **Per-agent lifecycle lease** — keyed on the **node's** agent (agent is
|
||||
|
|
@ -259,21 +251,18 @@ node, with the omission below it documented on the brace itself.
|
|||
|
||||
Two consequences worth knowing:
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
- **Flattening a chain under a brace is safe.** The stop chain used to nest
|
||||
`Signal` over `Drain` over `StopForUpdate` specifically so the lease stayed
|
||||
continuous — as independent siblings each would acquire it separately and
|
||||
leave a gap another DAG could claim the agent in, mid-bounce. A brace supplies
|
||||
that continuity directly, so the nesting is no longer load-bearing.
|
||||
- **Observability is unaffected.** `running_transients` keys off a node's
|
||||
_payload_ agent, not off a declared lease edge, so every child still lights its
|
||||
own dashboard pill and still reports its own `takes_container_down` to the
|
||||
crash watcher. A brace itself reports `false`: it parents the stopping nodes
|
||||
but doesn't stop anything, and claiming otherwise would widen crash
|
||||
suppression across the build and tail, where a vanished container is still a
|
||||
real crash.
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
_payload_ agent, not off a declared lease edge, so every child still lights its
|
||||
own dashboard pill and still reports its own `takes_container_down` to the
|
||||
crash watcher. A brace itself reports `false`: it parents the stopping nodes
|
||||
but doesn't stop anything, and claiming otherwise would widen crash
|
||||
suppression across the build and tail, where a vanished container is still a
|
||||
real crash.
|
||||
|
||||
Among simultaneously ready nodes competing for a resource, DAG-submit order
|
||||
wins (FIFO) so bulk operations drain predictably. The scheduler also owns the
|
||||
|
|
@ -287,7 +276,8 @@ reconcile_), so there is no durable-recovery machinery to go wrong.
|
|||
### Cancel, history
|
||||
|
||||
The agent-per-node move removed submit-time dedup (a multi-agent DAG
|
||||
has no single agent to key a dedup on), so every submit enqueues a fresh DAG.
|
||||
has no single agent to key a dedup on), so every submit enqueues a fresh DAG;
|
||||
whether any dedup needs reintroducing is tracked as a follow-up.
|
||||
|
||||
Cancel only applies to DAGs that are still fully queued (an in-flight nix build isn't
|
||||
interruptible) — each op is one DAG now, so there are no child DAGs to cascade to.
|
||||
|
|
|
|||
|
|
@ -37,8 +37,6 @@ Each **row** you see in a queue view (the **BU1LDS** page's R3BU1LD QU3U3
|
|||
nested under it are that job's steps, in order (occasionally a couple run
|
||||
side by side). A step shows one of:
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
| Glyph | Meaning |
|
||||
| ----- | ------------------------------------------------------- |
|
||||
| `⏸` | queued, waiting its turn |
|
||||
|
|
@ -49,8 +47,6 @@ side by side). A step shows one of:
|
|||
| `⊘` | cancelled |
|
||||
| `·` | skipped (not needed for this run) |
|
||||
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
A step that isn't needed for a given run stays in the graph as `·` rather
|
||||
than being absent from it, so the same kind of operation keeps a
|
||||
recognizable shape run to run, whichever steps it actually needed.
|
||||
|
|
|
|||
|
|
@ -68,8 +68,6 @@ doesn't change that.** Treat it as a convenience for grouping dashboards, never
|
|||
as evidence of which container produced a sample: any agent that can reach this
|
||||
hive's collector can label its data as any other agent.
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
**Logs ride the same hop, and a journal carries more than a counter does.** Each
|
||||
agent container forwards its own journal through this port — every unit in it at
|
||||
`info` and above, not an allowlist. That's the harness, the MCP daemons and
|
||||
|
|
@ -78,8 +76,6 @@ container where before only counts did. The trust boundary is unchanged (same
|
|||
destination, same credential, and an agent could already send arbitrary OTLP);
|
||||
what changes is how much detail leaves by default.
|
||||
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
Worth spelling out, because two different hops are in play and only one of them
|
||||
carries a credential:
|
||||
|
||||
|
|
@ -194,13 +190,9 @@ the `hive-br0` bridge. The service must also bind an address the bridge can
|
|||
reach — a `127.0.0.1`-only listener stays unreachable no matter what the
|
||||
firewall allows. See `docs/networking/network.md::Reaching host services` for details.
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
⚠️ **None of this is needed for hyperhive's own telemetry** — `otel.enable`
|
||||
contributes the collector's port and derives the agent-facing endpoint itself.
|
||||
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
## Built-in resource labels
|
||||
|
||||
The harness sets the OTLP variables (`OTEL_EXPORTER_OTLP_ENDPOINT`, `_PROTOCOL`,
|
||||
|
|
@ -258,16 +250,12 @@ while lines that never had one signal nothing of the sort.
|
|||
|
||||
## Host-emitted container-resource metrics (hive-c0re)
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
When OTEL is enabled, **hive-c0re itself** also exports each agent
|
||||
container's resource load — the same cgroup gauges shown on the dashboard
|
||||
LOAD tab — to this hive's own collector, exactly like an agent does and with
|
||||
no separate toggle. These come from the host, not the in-container Claude SDK,
|
||||
so they cover containers even when their agent is idle.
|
||||
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
Emitted via the OpenTelemetry Rust SDK, using the
|
||||
[semconv `container.*`](https://opentelemetry.io/docs/specs/semconv/system/container-metrics/)
|
||||
metric names + the standard `container.name` attribute where a spec metric
|
||||
|
|
@ -290,19 +278,13 @@ alongside the spec `container.cpu.time` counter for convenience). Hive
|
|||
labels (`hive`, `swarm`, …) ride on the resource via
|
||||
`extraResourceAttributes`.
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
Cadence follows `metricIntervalMs` (default 60s). Transport is OTLP/HTTP
|
||||
(JSON) to the hive collector's bridge address, with no auth header — that
|
||||
first hop is unauthenticated for every producer on this host, and the upstream
|
||||
credential stays on the swarm tier.
|
||||
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
## Agent-emitted per-turn metrics (`hive-agent`)
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
When OTEL is enabled, the harness itself (`hive-agent`) exports one small set
|
||||
of metrics per claude turn, recorded the moment the turn ends (not polled).
|
||||
These are deliberately the fields Claude Code's own built-in export (see
|
||||
|
|
@ -311,8 +293,6 @@ turn, its own outcome classification, the loose-ends backlog, and session
|
|||
boundaries. Token usage, cost, and tool-call counts are **not** duplicated
|
||||
here; that's already covered by Claude's own export.
|
||||
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
| Metric | Unit | Kind | Attributes |
|
||||
| -------------------------------------- | ---- | --------- | ----------------------------------------------------------------------------------------------------------- |
|
||||
| `hyperhive.agent.turn.duration` | `ms` | histogram | `wake_from`, `result_kind`, `model` |
|
||||
|
|
@ -332,8 +312,6 @@ records them (every turn, always).
|
|||
|
||||
## Hive-scoped metrics (hive-c0re)
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
hive-c0re measures everything above **per agent**, tagged with the hive it
|
||||
runs in. It measures these three per **hive** instead, and they carry no
|
||||
`agent` label — so a hive
|
||||
|
|
@ -341,8 +319,6 @@ that hosts no agents still reports, and "this hive is quiet" is
|
|||
distinguishable from "this hive is gone." Select them with
|
||||
`{hive!="",agent=""}`.
|
||||
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
| Metric | Unit | Kind | Meaning |
|
||||
| ------------------------- | ---- | ----- | -------------------------------------------------------------------------------------------------------------------- |
|
||||
| `process.uptime` | `s` | gauge | seconds since this hive's `hive-c0re` started exporting; a restart reads as a drop to ~0 |
|
||||
|
|
@ -444,8 +420,6 @@ overriding Claude Code's default of DELTA. This avoids silent metric drops in
|
|||
Prometheus-family backends (including Grafana LGTM / Mimir) that don't ship a
|
||||
delta-to-cumulative processor.
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
**`hive-metric` counters are the one exception**, reporting delta by default
|
||||
(see above) — programmatically set on the exporter, which overrides this
|
||||
container-wide env var for that tool specifically. `--type gauge` is
|
||||
|
|
@ -454,5 +428,3 @@ runs a `deltatocumulative` processor ahead of export, so a delta
|
|||
`hive-metric` counter still lands in VictoriaMetrics as a cumulative
|
||||
series — the standard `rate()`/`increase()` idioms work on it exactly like
|
||||
any other counter in this system, no special query needed.
|
||||
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
|
|
|||
|
|
@ -40,8 +40,6 @@ services.hyperhive = {
|
|||
};
|
||||
```
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
`swarm.domain` and `hiveName` are **required** whenever hyperhive is
|
||||
enabled; eval fails with a hint naming each. Neither defaults,
|
||||
because a guessed value here is a wrong hostname that evaluates cleanly
|
||||
|
|
@ -49,9 +47,7 @@ and deploys — an eval failure asking the operator to write the address
|
|||
down is the cheaper outcome. **Upgrading past this release means setting
|
||||
both once.**
|
||||
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
You must still set `domain` too, but you no longer _write_ it: it's read from
|
||||
`domain` is required too, but you no longer _write_ it: it's read from
|
||||
this hive's own entry in the directory, whose `domain` defaults to
|
||||
`<name>.<swarm.domain>`. A conventional swarm states no addresses at
|
||||
all, and a hive addressed by something else states it in the one place
|
||||
|
|
@ -280,8 +276,8 @@ If one host is behind NAT and can't accept incoming connections, only
|
|||
that host needs a null `wireguardEndpoint` on the peer config — the
|
||||
other side initiates. With keepalive on, the NAT hole stays open.
|
||||
|
||||
If both hosts are behind NAT, you need a STUN relay or a third host
|
||||
(exit node). Out of scope for v0.
|
||||
If both hosts are behind NAT, a STUN relay or a third host (exit node)
|
||||
is required. Out of scope for v0.
|
||||
|
||||
## Snapshot store
|
||||
|
||||
|
|
@ -462,16 +458,12 @@ leftover at startup. A webhook has exactly one target URL, so per-hive
|
|||
registration never added a recipient — it took delivery away from
|
||||
whichever hive registered before it.
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
**The `agent-configs` org isn't yet.** Each hive still registers its own
|
||||
`pull_request` hook there, so that repo has two — the hive's and the
|
||||
controller's — and **both are expected; don't delete either.** Removing
|
||||
a hive's stops it acting on config PRs; removing the controller's just
|
||||
gets recreated on its next start.
|
||||
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
Nothing to configure. The controller registers the hooks only when this
|
||||
host also serves the swarm UI vhost — that's what publishes the
|
||||
endpoint, and a hook the forge can't reach would collect failed
|
||||
|
|
|
|||
|
|
@ -71,13 +71,9 @@ its own subdomain — enforced by every verifier rather than by
|
|||
convention. The constraint excludes both IP families as well, since a
|
||||
permitted-DNS-only constraint says nothing about IP SANs.
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
The root is issued with `pathlen:1`: it may sign hive CAs, and those may
|
||||
sign leaves, and the chain stops there.
|
||||
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
## What to hand a peer
|
||||
|
||||
`hivectl peer-config` prints the `services.hyperhive.swarm.hives."<name>"`
|
||||
|
|
@ -107,8 +103,6 @@ per-hive file.
|
|||
|
||||
## Adopting the hierarchy on an existing hive
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
A hive that predates the swarm root carries a self-signed `ca.pem`, and
|
||||
adopting the hierarchy means replacing it. That invalidates an anchor
|
||||
consumers already trust, and they refresh on their own schedule — agents
|
||||
|
|
@ -116,8 +110,6 @@ only pick up new trust when their container restarts, peers only on
|
|||
their own rebuild. **Who is allowed to decide that's what splits the
|
||||
two cases.**
|
||||
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
### Where this host owns the root (`autoConfigure`)
|
||||
|
||||
Adoption happens by itself, once. `hive-tls-ca.service` notices that
|
||||
|
|
@ -149,14 +141,10 @@ rm <deploy.hive-controller.tls.stateDir>/ca.pem <deploy.hive-controller.tls.stat
|
|||
systemctl restart hive-tls-ca.service
|
||||
```
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
Failing rather than warning is deliberate: a hive whose CA doesn't
|
||||
chain to the root it received is misconfigured, and a warning in a
|
||||
build log isn't something anyone reads twice.
|
||||
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
To keep the current CA on purpose — a hive that deliberately stays
|
||||
outside the hierarchy, or one mid-migration — `touch` the marker file
|
||||
named in the message. That's a decision, and it's recorded as one.
|
||||
|
|
|
|||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Reference in a new issue