Compare commits
32 changed files with 1256 additions and 1340 deletions
|
|
@ -110,10 +110,10 @@ sudo hivectl matrix create-user mara # provisions a matrix
|
||||||
sudo hivectl matrix create-user mara --password-stdin # … reading one line from stdin
|
sudo hivectl matrix create-user mara --password-stdin # … reading one line from stdin
|
||||||
```
|
```
|
||||||
|
|
||||||
For a name that's a managed agent, `hivectl` persists the resulting token
|
For agent names (i.e., a `Coordinator::agent_state_root(name)` exists),
|
||||||
to that agent's state dir, the same as the boot sweep does. For a
|
`hivectl` persists the resulting token to the agent's state dir like the
|
||||||
non-agent name (e.g. the operator's own forge/matrix account), it prints
|
boot sweep does. For non-agent names (e.g. the operator's own forge/matrix
|
||||||
the token to stdout and writes nothing.
|
account), it prints the token to stdout and writes nothing.
|
||||||
|
|
||||||
## Build / deploy
|
## Build / deploy
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,13 +1,11 @@
|
||||||
# Agent hierarchy & privileges
|
# Agent hierarchy & privileges
|
||||||
|
|
||||||
Every agent has a place in an operator-editable parent/child tree, used
|
Design + audit doc for the agent-privileges + tree-shape milestone
|
||||||
to scope which agents can manage which others. This doc covers how the
|
(the [issue tree](http://localhost:3000/hyperhive/hyperhive/issues/361)).
|
||||||
tree is stored and edited today, the rules that are meant to run on top
|
The implementation lands in pieces; this doc tracks what's done, what's
|
||||||
of it once enforcement is finished, and where the manager still gets
|
planned, and what currently special-cases the manager.
|
||||||
special-cased in the meantime. Tracking issue: hyperhive#361
|
|
||||||
(`$HIVE_FORGE_URL/hyperhive/hyperhive/issues/361`).
|
|
||||||
|
|
||||||
## Where the tree lives
|
## Current state (as of this PR)
|
||||||
|
|
||||||
Topology lives in the hive-c0re-owned **meta repo**, alongside
|
Topology lives in the hive-c0re-owned **meta repo**, alongside
|
||||||
`flake.nix`, at `/var/lib/hyperhive/meta/topology.json`:
|
`flake.nix`, at `/var/lib/hyperhive/meta/topology.json`:
|
||||||
|
|
@ -20,35 +18,45 @@ Topology lives in the hive-c0re-owned **meta repo**, alongside
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
`null` = root-level agent. New agents **default to root** — there is no
|
`null` = root-level agent. New agents **default to root** (`null` parent) —
|
||||||
structural manager that everything hangs under. Hierarchy is built
|
there is no structural manager that everything hangs under. Hierarchy is
|
||||||
explicitly: an agent that requests a sub-agent gets a
|
built explicitly: an agent that requests a sub-agent gets a
|
||||||
requester-as-parent edge written at its `init_config` approval (so
|
requester-as-parent edge written at its `init_config` approval (so `bob`
|
||||||
`bob` above was spawned by `alice`), and the operator can reparent any
|
above was spawned by `alice`), and the operator can reparent any agent. The
|
||||||
agent, including the bootstrap container (`ruth`) — it's just another
|
bootstrap container (`ruth`) is just another root. Re-parenting is
|
||||||
root. The manager is reparentable like any other agent; there's no
|
operator-driven:
|
||||||
"structurally root" carve-out. Its privileges live on its MCP socket,
|
|
||||||
not its tree position (see *Manager special-casing today* below).
|
|
||||||
|
|
||||||
### Reparenting
|
- CLI: `hivectl agent <child> set-parent --parent <new>` (or `--root` to
|
||||||
|
promote). Exactly one of `--parent` / `--root` is required.
|
||||||
- CLI: `hivectl agent <child> set-parent --parent <new>` (or `--root`
|
|
||||||
to promote). Exactly one of `--parent` / `--root` is required.
|
|
||||||
- Dashboard: `POST /api/topology/set-parent` (form fields `child`,
|
- Dashboard: `POST /api/topology/set-parent` (form fields `child`,
|
||||||
optional `new_parent` — absent / empty ⇒ promote to root).
|
optional `new_parent` — absent / empty ⇒ promote to root).
|
||||||
- Wire: `HostRequest::SetParent { child, new_parent: Option<String> }`.
|
- Wire: `HostRequest::SetParent { child, new_parent: Option<String> }`.
|
||||||
|
|
||||||
All three go through the same validation, which refuses:
|
All three converge on `topology::set_parent`, which delegates the
|
||||||
|
validation rules to a pure `apply_set_parent` helper. Refuses:
|
||||||
|
|
||||||
- unknown `child` / `new_parent` (typo guard),
|
- unknown `child` / `new_parent` (typo guard),
|
||||||
- self-parenting,
|
- self-parenting,
|
||||||
- cycles (a bounded ancestor walk — moving the manager under one of
|
- cycles (32-hop ancestor walk, mirroring `is_descendant_of`).
|
||||||
its own descendants is the only real safety concern here, and it's
|
|
||||||
caught the same way as any other agent).
|
|
||||||
|
|
||||||
Setting a parent to its current value is a no-op (no disk write). A
|
The manager is reparentable like any other agent — there's no
|
||||||
successful change triggers an immediate rescan, so connected dashboard
|
"structurally root" carve-out; the manager's privileges live on its
|
||||||
viewers see the tree repaint without polling.
|
MCP socket, not its tree position, and the cycle walk above catches
|
||||||
|
the only real safety concern (moving the manager under one of its
|
||||||
|
own descendants).
|
||||||
|
|
||||||
|
Idempotent no-op fast path skips the disk write when the parent is
|
||||||
|
already what's requested. After a successful write the surfaces call
|
||||||
|
`Coordinator::rescan_containers_and_emit` so connected dashboard
|
||||||
|
viewers see the tree repaint without polling
|
||||||
|
(`ContainerView.parent` is sourced from `topology.json`).
|
||||||
|
|
||||||
|
**Today's caveat:** the move is purely a JSON edit. Only the
|
||||||
|
top-level manager (`root`) gets `/var/lib/hyperhive/agents`
|
||||||
|
bind-mounted at `/agents` in its container, so sub-agents don't yet
|
||||||
|
see their would-be children's state. Once sub-manager bind mounts
|
||||||
|
land alongside cap enforcement, `set_parent` grows a companion
|
||||||
|
umount-old / mount-new / restart-cascade step.
|
||||||
|
|
||||||
### Why meta, not per-agent `agent.nix`
|
### Why meta, not per-agent `agent.nix`
|
||||||
|
|
||||||
|
|
@ -57,39 +65,37 @@ consent, and operator-driven re-parenting shouldn't require touching
|
||||||
the moved agent's config. Topology IS a system-level concern; meta is
|
the moved agent's config. Topology IS a system-level concern; meta is
|
||||||
where system-level facts live.
|
where system-level facts live.
|
||||||
|
|
||||||
### How `topology.json` gets updated
|
### Flow
|
||||||
|
|
||||||
- **Read** — parsed into an agent→parent map; a missing or unparsable
|
1. **Read**: `topology::read()` parses `topology.json` into a
|
||||||
file degrades safely to "every agent is root" (covers a fresh
|
`BTreeMap<String, Option<String>>`. Missing / unparsable file →
|
||||||
install that hasn't synced yet).
|
empty map → every agent treated as root (safe degradation for
|
||||||
- **Reconcile** — runs alongside the periodic meta/flake regeneration.
|
fresh installs that haven't run `meta::sync_agents` yet).
|
||||||
New agents default to root unless they already carry an explicit
|
2. **Reconcile**: `meta::sync_agents` calls `topology::reconcile`
|
||||||
parent edge from an `init_config` approval; existing entries
|
alongside its `flake.nix` regeneration. New agents default to root
|
||||||
(including operator overrides) are preserved; removed agents drop.
|
(null parent) — an agent-requested sub-agent already carries an
|
||||||
Agents that are approved but not yet spawned keep their edge too, so
|
explicit requester-as-parent edge from its `init_config` approval, so
|
||||||
it survives the gap until the container actually appears.
|
only user/operator-initiated spawns hit this default, and those are
|
||||||
- **Inject** — each container's parent (if any) is exposed to its own
|
roots; removed agents drop. Existing entries are preserved as-is so
|
||||||
environment as `HIVE_PARENT`, so the harness / system-prompt
|
operator overrides stick across regenerations. Pending-init agents
|
||||||
renderer can see it.
|
(an operator-approved proposed config repo but no container yet —
|
||||||
- **Surface** — every rescan re-reads `topology.json` and populates
|
`Coordinator::pending_init_names`) are kept too, so the
|
||||||
`ContainerView.parent`, which the dashboard renders as a tree.
|
`child -> parent` edge written when `request_init_config` is approved
|
||||||
|
survives the gap until the first apply-commit spawns the container.
|
||||||
|
3. **Inject**: `meta::render_flake` looks up each agent's parent and
|
||||||
|
passes it to `mkAgent`. When non-null, the mkAgent body sets
|
||||||
|
`HIVE_PARENT = parent` in the agent's systemd service environment
|
||||||
|
so the harness / claude prompts can see it.
|
||||||
|
4. **Surface**: `container_view::build_all` reads `topology.json` and
|
||||||
|
populates `ContainerView.parent: Option<String>` on every rescan.
|
||||||
|
The dashboard renders the field as a tree.
|
||||||
|
|
||||||
See `hive-c0re/src/agent_config/topology.rs` and `hive-c0re/src/meta.rs`'s
|
## Target topology semantics
|
||||||
module docs for the exact call chain.
|
|
||||||
|
|
||||||
### Current limitation: state-dir visibility lags topology
|
Once enforcement lands the rules collapse into:
|
||||||
|
|
||||||
Reparenting today is purely a JSON edit. Only the top-level manager
|
|
||||||
(`root`) gets `/var/lib/hyperhive/agents` bind-mounted at `/agents` in
|
|
||||||
its container, so sub-agents don't yet see their would-be children's
|
|
||||||
state dirs. Once sub-manager bind mounts land alongside capability
|
|
||||||
enforcement, reparenting will grow a companion
|
|
||||||
umount-old / mount-new / restart-cascade step.
|
|
||||||
|
|
||||||
## Planned topology semantics (once ancestor-based enforcement lands)
|
|
||||||
|
|
||||||
| operation | who can do it |
|
| operation | who can do it |
|
||||||
| ----------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
|
| ----------------------------------------------------------------------- | --------------------------------------------------------------------------------------- |
|
||||||
| `kill` / `start` / `restart` / `update` (any descendant) | any ancestor |
|
| `kill` / `start` / `restart` / `update` (any descendant) | any ancestor |
|
||||||
| `request_init_config` (spawn a new child) | any agent, child added under self |
|
| `request_init_config` (spawn a new child) | any agent, child added under self |
|
||||||
| config change via forge PR (any descendant's config) | any ancestor |
|
| config change via forge PR (any descendant's config) | any ancestor |
|
||||||
|
|
@ -99,74 +105,106 @@ umount-old / mount-new / restart-cascade step.
|
||||||
| `request_update_meta_inputs` (bump meta lock) | root agents only (today: just `manager`) |
|
| `request_update_meta_inputs` (bump meta lock) | root agents only (today: just `manager`) |
|
||||||
|
|
||||||
"Ancestor" walks `ContainerView.parent` chains; cycles are guarded by a
|
"Ancestor" walks `ContainerView.parent` chains; cycles are guarded by a
|
||||||
visited-set at dispatch time (a malformed `topology.json` can't lock
|
visited-set at dispatch time (a malformed topology.json can't lock the
|
||||||
the dispatcher into a loop).
|
dispatcher into a loop).
|
||||||
|
|
||||||
## Manager special-casing today
|
## Current manager special-casings — the audit
|
||||||
|
|
||||||
Enforcement of the ancestor rules above isn't fully wired yet, so the
|
What currently makes the manager different from every other agent, and
|
||||||
**manager (`ruth`) still gets some hard-coded special treatment**
|
which axis the post-milestone version reads each special-case along:
|
||||||
other agents don't:
|
|
||||||
|
|
||||||
- **Naming/bootstrap** — the manager's broker recipient name, state-dir
|
### A — naming + bootstrap
|
||||||
key, and nixos-container name are all `ruth` (container `h-ruth`).
|
|
||||||
`hive-c0re` spawns it directly at boot if missing, with no operator
|
|
||||||
approval step — every other agent goes through `request_init_config`
|
|
||||||
→ approval. Topology-wise, `ruth` is still just another root agent.
|
|
||||||
- **Wire-protocol** — the privileged `Request` variants
|
|
||||||
(`RequestInitConfig`; `Kill` / `Start` / `Restart` / `Update`;
|
|
||||||
`GetLogs`; `RequestUpdateMetaInputs`) — marked `*(privileged)*` in
|
|
||||||
`hive-core-agent-sock`'s unified `Request` enum — are reachable only
|
|
||||||
from the manager's socket flavour today. Planned rule for each is in the
|
|
||||||
table above ("any agent, child added under self" for init-config,
|
|
||||||
"any ancestor" for lifecycle/logs); `RequestUpdateMetaInputs` stays
|
|
||||||
a root-only capability even post-milestone, not a topology rule.
|
|
||||||
One exception: `Wake` (inject a `from: <X>` message into the
|
|
||||||
caller's own inbox) isn't really privileged — every per-agent daemon
|
|
||||||
(e.g. `hive-forge-notify`) needs it, and sub-agents already have the
|
|
||||||
equivalent on their own socket.
|
|
||||||
- **Storage/mounts** — only the manager container gets
|
|
||||||
`/var/lib/hyperhive/agents` bind-mounted RW at `/agents` (so it can
|
|
||||||
manage any agent's state dir — config is not authored there, since a
|
|
||||||
real config change is a PR from a clone), plus RO mounts for
|
|
||||||
`/applied` (diff against what's deployed) and `/meta` (system-wide
|
|
||||||
deploy log). Planned: each agent gets RW to `/agents/<descendant>/`
|
|
||||||
for just its own subtree — the manager's full-forest RW becomes the
|
|
||||||
"root's subtree is everything" case of that same rule. RO `/meta`
|
|
||||||
access will be gated on a "meta read" capability; only
|
|
||||||
`request_update_meta_inputs` writes `flake.lock`, gated by its own
|
|
||||||
capability.
|
|
||||||
- **Prompt/tools** — the system prompt uses `<!-- role:agent -->` /
|
|
||||||
`<!-- role:manager -->` marker blocks, and a `Flavor::{Agent,
|
|
||||||
Manager}` switch picks the MCP tool allow-list claude sees. Both are
|
|
||||||
already parametrised on a single flavour value, so the planned
|
|
||||||
per-capability-group version (`cap:<group>` prompt blocks + a
|
|
||||||
matching tool allow-list) is additive rather than a rewrite.
|
|
||||||
- **State dirs** — *not* special-cased: `HYPERHIVE_STATE_DIR` is
|
|
||||||
injected uniformly via `systemd.globalEnvironment` for every
|
|
||||||
container including the manager, so all token/state paths resolve
|
|
||||||
through it the same way everywhere.
|
|
||||||
- **Scattered ownership checks** — a handful of independent
|
|
||||||
manager-only overrides exist across `hive-c0re` today: loose-ends
|
|
||||||
visibility (manager sees hive-wide, sub-agents only their own),
|
|
||||||
"manager can cancel any question/reminder" overrides on the owner
|
|
||||||
check, `destroy` refusing to act on the manager, and crash-watch
|
|
||||||
skipping the manager (it auto-restarts via systemd instead of going
|
|
||||||
through the crash-watch loop). Each is planned to become an
|
|
||||||
ancestor/descendant check instead of a manager-name check — see the
|
|
||||||
module docs for `loose_ends.rs`, `stores/operator_questions.rs`,
|
|
||||||
`stores/broker.rs`, `actions.rs`, and `workers/crash_watch.rs` for
|
|
||||||
the current owner-check logic in each. (Reminder cancellation has
|
|
||||||
since moved fully in-agent — see the note on
|
|
||||||
`CancelLooseEndKind::Reminder` in `hive-c0re/src/questions.rs`.)
|
|
||||||
|
|
||||||
None of the above is a stable interface — treat the module doc
|
- `MANAGER_AGENT = "ruth"` (broker recipient name),
|
||||||
comments as the source of truth for exactly which checks exist today.
|
`MANAGER_NAME = "ruth"` (logical name, state-dir key), and
|
||||||
|
`MANAGER_CONTAINER = "h-ruth"` (nixos-container name); the `h-`
|
||||||
|
prefix lets `lifecycle::list()` use a single `starts_with("h-")`
|
||||||
|
filter.
|
||||||
|
- `auto_update::ensure_manager` runs at hive-c0re boot and spawns
|
||||||
|
`h-ruth` if missing. **Topology**: ruth defaults to root-level (no
|
||||||
|
parent); hive-c0re handles the bootstrap lifecycle directly.
|
||||||
|
|
||||||
## Future work: sub-agents inside the same container
|
### B — wire-protocol privileges
|
||||||
|
|
||||||
When enabled for an agent, it will be able to spawn temporary
|
The `ManagerRequest::*` variants in `hive-sh4re/src/lib.rs` are
|
||||||
"sub-agents" that run inside its own container — lighter than a full
|
operations the manager flavour socket can make that sub-agent sockets
|
||||||
|
can't:
|
||||||
|
|
||||||
|
| variant | semantic | post-milestone |
|
||||||
|
| --------------------------------------- | ---------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||||
|
| `RequestInitConfig` | seed an agent's proposed config repo | **topology** — existing direct child (re-init) or a brand-new name (child added under self on approval); a name owned by a different parent is refused |
|
||||||
|
| `Kill` / `Start` / `Restart` / `Update` | container lifecycle on an existing agent | **topology** — descendants only |
|
||||||
|
| `RequestUpdateMetaInputs` | bump meta `flake.lock` | **per-agent cap** (root-only today; a future "let coder bump its own input" might grant it) |
|
||||||
|
| `GetLogs` | journalctl scrape of a sub-agent | **topology** — descendants only |
|
||||||
|
| `Wake` | inject a `from: <X>` message into self's inbox | **not really privileged** — the wire surface exists because the per-agent daemons (e.g. `hive-forge-notify`) need it. Sub-agents have the same via their own socket. |
|
||||||
|
|
||||||
|
### C — storage / mounts (`hive-c0re::lifecycle`)
|
||||||
|
|
||||||
|
The manager container's nspawn bind set:
|
||||||
|
|
||||||
|
- `HOST_AGENTS_ROOT (/var/lib/hyperhive/agents) → /agents` RW — so the
|
||||||
|
manager can manage any agent's state dir. Config is **not** authored
|
||||||
|
here: a config change is a PR from a clone, and `<agent>/config/` is
|
||||||
|
a copy for reading (its write access is a defect tracked separately)
|
||||||
|
- `HOST_APPLIED_ROOT (/var/lib/hyperhive/applied) → /applied` RO — so
|
||||||
|
the manager can diff against what's deployed
|
||||||
|
- `HOST_META_ROOT (/var/lib/hyperhive/meta) → /meta` RO — so the
|
||||||
|
manager can read the system-wide deploy log
|
||||||
|
|
||||||
|
Tree-shape version:
|
||||||
|
|
||||||
|
- Each agent gets RW to `/agents/<descendant>/` for every descendant in
|
||||||
|
its subtree. The root agent (today: manager) gets RW to the full
|
||||||
|
forest as a special case of "the root has every other agent as a
|
||||||
|
descendant".
|
||||||
|
- RO `/meta` access if the agent holds a "meta read" cap.
|
||||||
|
- `request_update_meta_inputs` is the only path that actually writes
|
||||||
|
`flake.lock`, gated by the cap; everyone else stays RO.
|
||||||
|
|
||||||
|
### D — drop legacy `/state` for manager ✓ done
|
||||||
|
|
||||||
|
`lifecycle.rs` no longer binds `/state` for the manager.
|
||||||
|
`HYPERHIVE_STATE_DIR` is now injected uniformly via
|
||||||
|
`systemd.globalEnvironment` in `meta.rs` for every container
|
||||||
|
(manager included), so all token/state paths resolve through
|
||||||
|
`$HYPERHIVE_STATE_DIR`. The agent-module shell scripts
|
||||||
|
(tea-login, forge-avatar-sync) simplified from glob+for loops to a
|
||||||
|
direct `$HYPERHIVE_STATE_DIR/<token>` read.
|
||||||
|
|
||||||
|
### E — prompt + tools
|
||||||
|
|
||||||
|
- `prompts/system.md` with `<!-- role:agent -->` / `<!-- role:manager -->`
|
||||||
|
marker blocks, assembled by `hive_ag3nt::prompt::render` based on
|
||||||
|
flavor. **Per-agent cap list** of what the agent can do — already
|
||||||
|
a single parametrised prompt; once per-agent cap groups land the
|
||||||
|
marker grammar grows `cap:<group>` blocks the renderer reads from
|
||||||
|
the per-agent ToolGroup set.
|
||||||
|
- `mcp.rs::Flavor::{Agent, Manager}` controls which MCP tools claude
|
||||||
|
sees. Already structured this way internally — the per-flavour
|
||||||
|
allow-list becomes a per-cap-set lookup.
|
||||||
|
|
||||||
|
### F — drive-by checks across c0re
|
||||||
|
|
||||||
|
- `loose_ends.rs`: manager sees hive-wide loose-ends, sub-agents only
|
||||||
|
their own. **Topology** — every agent sees its own + its
|
||||||
|
descendants'.
|
||||||
|
- `operator_questions.rs` + `broker.rs`: "manager can cancel any
|
||||||
|
question" override on the owner check. **Topology** — agents can
|
||||||
|
moderate threads of their descendants.
|
||||||
|
- `reminder_scheduler.rs`: same override pattern for reminder cancel.
|
||||||
|
**Topology** — descendants only.
|
||||||
|
- `actions.rs`: `destroy` refuses to act on `MANAGER_NAME` (no
|
||||||
|
foot-shooting). **Topology** — agents can destroy descendants but
|
||||||
|
never themselves or ancestors.
|
||||||
|
- `crash_watch.rs`: skips `ContainerCrash` for the manager (it
|
||||||
|
auto-restarts via systemd). **Topology** — the root container has
|
||||||
|
different recovery semantics, every other agent falls into the same
|
||||||
|
watch loop.
|
||||||
|
|
||||||
|
### G — sub-agents inside the same container
|
||||||
|
|
||||||
|
Future work: when enabled for an agent, it can spawn temporary
|
||||||
|
"sub-agents" that run inside its own container. Lighter than a full
|
||||||
nspawn agent. Open questions, not yet wired:
|
nspawn agent. Open questions, not yet wired:
|
||||||
|
|
||||||
- Inherit caps from parent, or take an explicit narrower set?
|
- Inherit caps from parent, or take an explicit narrower set?
|
||||||
|
|
@ -178,16 +216,17 @@ nspawn agent. Open questions, not yet wired:
|
||||||
## Harness systemd unit shape
|
## Harness systemd unit shape
|
||||||
|
|
||||||
One harness serve binary (`hive-agent`, with its `hive-agent-mcp`
|
One harness serve binary (`hive-agent`, with its `hive-agent-mcp`
|
||||||
sibling), one shared `nix/agent-modules/` tree, one service unit
|
sibling), one shared `nix/agent-modules/` tree, one
|
||||||
(`systemd.services.hive-agent`) for all agents. There is no separate
|
service unit (`systemd.services.hive-agent`) for all agents. There
|
||||||
manager service name or role distinction in the harness — privilege
|
is no longer a separate manager service name or role distinction in
|
||||||
differences live server-side in the broker socket (which tool groups
|
the harness — privilege differences live server-side in the broker
|
||||||
and manager-surface calls each agent receives).
|
socket (which tool groups and manager-surface calls each agent
|
||||||
|
receives).
|
||||||
|
|
||||||
`agent.nix` and `ruth.nix` both import the shared `nix/agent-modules/`.
|
`agent.nix` and `ruth.nix` both import the shared `nix/agent-modules/`.
|
||||||
`ruth.nix` additionally sets forge defaults to suppress the
|
`ruth.nix` additionally sets forge defaults to suppress the
|
||||||
subscription/participation firehose so ruth's inbox stays focused on
|
subscription/participation firehose so ruth's inbox stays focused
|
||||||
direct mentions, reviews, and assignments.
|
on direct mentions, reviews, and assignments.
|
||||||
|
|
||||||
### Environment variables set on the unit
|
### Environment variables set on the unit
|
||||||
|
|
||||||
|
|
@ -209,37 +248,36 @@ direct mentions, reviews, and assignments.
|
||||||
path = [ "/run/wrappers" "/run/current-system/sw" ];
|
path = [ "/run/wrappers" "/run/current-system/sw" ];
|
||||||
```
|
```
|
||||||
|
|
||||||
`/run/wrappers` (not `/run/wrappers/bin`) comes first so setuid
|
`/run/wrappers` comes first so setuid wrappers (notably `sudo`)
|
||||||
wrappers — notably `sudo` — resolve before bare nix-store binaries; see
|
resolve before bare nix-store binaries. NixOS's
|
||||||
[`docs/gotchas.md`](gotchas.md) ("`systemd.services.*.path` appends
|
`systemd.services.<unit>.path` appends `/bin` to every entry via
|
||||||
`/bin` to every entry") for why the trailing `/bin` matters in
|
`lib.makeBinPath`; passing `/run/wrappers/bin` directly produces
|
||||||
general. It's load-bearing here because the harness runs as the
|
`/run/wrappers/bin/bin` which doesn't exist (`docs/gotchas.md::
|
||||||
per-agent user: without the wrapper dir on `PATH`, `sudo` resolves to
|
systemd.services.*.path appends /bin to every entry`). With the
|
||||||
the non-setuid nix-store binary and every
|
harness running as the per-agent user this matters: without the
|
||||||
`hyperhive.user.passwordlessSudo` grant fails with "must be owned by
|
wrapper dir on PATH, `sudo` resolves to the un-setuid nix-store
|
||||||
uid 0 and have the setuid bit set."
|
binary and rejects with `must be owned by uid 0 and have the setuid
|
||||||
|
bit set` regardless of `hyperhive.user.passwordlessSudo`.
|
||||||
|
|
||||||
### `serviceConfig` highlights
|
### `serviceConfig` highlights
|
||||||
|
|
||||||
- `ExecStart = pkgs.hyperhive/bin/hive-agent` — same binary for every
|
- `ExecStart = pkgs.hyperhive/bin/hive-agent` — same binary for
|
||||||
agent.
|
every agent.
|
||||||
- `Restart = on-failure`, `RestartSec = 2` — keeps the harness
|
- `Restart = on-failure`, `RestartSec = 2` — keeps the harness
|
||||||
resilient across transient crashes without thundering retries.
|
resilient across transient crashes without thundering retries.
|
||||||
- `RuntimeDirectory = "hive-config"` → `/run/hive-config/` owned by
|
- `RuntimeDirectory = "hive-config"` → `/run/hive-config/` owned by
|
||||||
`User=`, auto-cleared on stop. The harness writes regenerated
|
`User=`, auto-cleared on stop. The harness writes regenerated
|
||||||
`claude-{mcp-config,settings,system-prompt}` files there
|
`claude-{mcp-config,settings,system-prompt}` files there
|
||||||
(`paths::config_dir`). Deliberately separate from `/run/hive`, which
|
(`paths::config_dir`). Deliberately separate from `/run/hive`,
|
||||||
the host bind-mounts in root-owned and which holds hive-c0re's
|
which the host bind-mounts in root-owned and which holds
|
||||||
`mcp.sock`.
|
hive-c0re's `mcp.sock`.
|
||||||
- `User = Group = userName` — drops root inside the container; sudo is
|
- `User = Group = userName` — drops root inside the container; sudo
|
||||||
the explicit escalation surface (`hyperhive.user.passwordlessSudo`).
|
is the explicit escalation surface
|
||||||
|
(`hyperhive.user.passwordlessSudo`).
|
||||||
|
|
||||||
## Cross-references
|
## Cross-references
|
||||||
|
|
||||||
- Milestone: "Agent privileges and sub-agents"
|
- Milestone: ["Agent privileges and sub-agents"](http://localhost:3000/hyperhive/hyperhive/issues/361)
|
||||||
(`$HIVE_FORGE_URL/hyperhive/hyperhive/issues/361`)
|
- Dashboard render: ["show agent topology in container list"](http://localhost:3000/hyperhive/hyperhive/issues/363)
|
||||||
- Dashboard render: "show agent topology in container list"
|
- Audit table source: [milestone comment](http://localhost:3000/hyperhive/hyperhive/issues/361#issuecomment-3335)
|
||||||
(`$HIVE_FORGE_URL/hyperhive/hyperhive/issues/363`)
|
|
||||||
- Audit table source: milestone comment
|
|
||||||
(`$HIVE_FORGE_URL/hyperhive/hyperhive/issues/361#issuecomment-3335`)
|
|
||||||
- Operator/agent trust boundary (orthogonal axis): [`boundary.md`](boundary.md)
|
- Operator/agent trust boundary (orthogonal axis): [`boundary.md`](boundary.md)
|
||||||
|
|
|
||||||
|
|
@ -212,9 +212,10 @@ An agent target's delivery is `push_todo` (`Coordinator::push_todo`,
|
||||||
`docs/coordinator.md` covers the mechanism generally), not a broker
|
`docs/coordinator.md` covers the mechanism generally), not a broker
|
||||||
`Message` — a scheduled prompt wakes its target with a todo instead of
|
`Message` — a scheduled prompt wakes its target with a todo instead of
|
||||||
driving an immediate turn, by design. `key = "schedule:<id>"` per
|
driving an immediate turn, by design. `key = "schedule:<id>"` per
|
||||||
target drives `push_todo`'s own upsert-by-key dedup: a re-fire of the
|
target gives `push_todo`'s own upsert-by-key dedup the job a
|
||||||
*same schedule* against a target that hasn't reviewed the last one
|
now-removed `has_pending_with_body` broker check used to do: a re-fire
|
||||||
collapses into that one todo instead of stacking up.
|
of the *same schedule* against a target that hasn't reviewed the last
|
||||||
|
one collapses into that one todo instead of stacking up.
|
||||||
|
|
||||||
**`operator` is the one exception** — it's a valid schedule target but
|
**`operator` is the one exception** — it's a valid schedule target but
|
||||||
has no in-container todo inbox, so it keeps the original broker
|
has no in-container todo inbox, so it keeps the original broker
|
||||||
|
|
@ -289,7 +290,8 @@ declares one flake input per agent (`agent-<n>.url =
|
||||||
"git+http://<forge>/agent-configs/<n>.git"`) and one
|
"git+http://<forge>/agent-configs/<n>.git"`) and one
|
||||||
`nixosConfigurations.<n>` output per agent. Each output wraps
|
`nixosConfigurations.<n>` output per agent. Each output wraps
|
||||||
`inputs.agent-<n>.nixosModules.default` with the identity +
|
`inputs.agent-<n>.nixosModules.default` with the identity +
|
||||||
`HIVE_PORT` / `HIVE_LABEL` / `HIVE_DASHBOARD_PORT` injection module.
|
`HIVE_PORT` / `HIVE_LABEL` / `HIVE_DASHBOARD_PORT` injection
|
||||||
|
module that `setup_applied` used to generate inline.
|
||||||
Containers run against `--flake /var/lib/hyperhive/meta#<n>`.
|
Containers run against `--flake /var/lib/hyperhive/meta#<n>`.
|
||||||
|
|
||||||
The declared input url is the agent's **forge config repo** (the
|
The declared input url is the agent's **forge config repo** (the
|
||||||
|
|
@ -343,10 +345,9 @@ and `meta::lock_update_hyperhive()` for the
|
||||||
auto-update flake-rev bump (one shot before per-agent
|
auto-update flake-rev bump (one shot before per-agent
|
||||||
rebuilds, commits if the lock changed).
|
rebuilds, commits if the lock changed).
|
||||||
|
|
||||||
`meta::sync_agents(hive: &HiveEnv, agents: &[AgentSpec])` — `hive`
|
`meta::sync_agents(hyperhive_flake, dashboard_port, &agents)`
|
||||||
carries `hyperhive_flake`, `dashboard_port`, and the rest of the
|
is the idempotent reconciler called by `spawn`, `destroy`,
|
||||||
per-hive config — is the idempotent reconciler called by `spawn`,
|
`rebuild`, and the startup migration. Renders `flake.nix`
|
||||||
`destroy`, `rebuild`, and the startup migration. Renders `flake.nix`
|
|
||||||
from the agent list; if it differs from disk, runs
|
from the agent list; if it differs from disk, runs
|
||||||
`nix flake lock` + commits as `regenerate meta flake` (or
|
`nix flake lock` + commits as `regenerate meta flake` (or
|
||||||
`seed meta from N agent(s)` on the very first call).
|
`seed meta from N agent(s)` on the very first call).
|
||||||
|
|
@ -418,9 +419,9 @@ submitter pushes again (or closes it) to retry.
|
||||||
### Dispatch via the job queue
|
### Dispatch via the job queue
|
||||||
|
|
||||||
Long-running approval work — `MergeConfigPr`, `UpdateMetaInputs`,
|
Long-running approval work — `MergeConfigPr`, `UpdateMetaInputs`,
|
||||||
`Spawn` — runs as a DAG on the global job queue
|
`Spawn` — no longer runs inline inside `actions::approve`. Instead
|
||||||
(`docs/coordinator.md::Job queue`), submitted by the approval handler
|
the approval handler submits a DAG to the global job queue
|
||||||
rather than run inline:
|
(`docs/coordinator.md::Job queue`):
|
||||||
|
|
||||||
| `ApprovalKind` | DAG submitted | source |
|
| `ApprovalKind` | DAG submitted | source |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
|
|
@ -490,9 +491,8 @@ approval card. See `docs/web-ui.md`.
|
||||||
### Submitting agent's view of config repos
|
### Submitting agent's view of config repos
|
||||||
|
|
||||||
Every parent agent's container has its **direct children's** config
|
Every parent agent's container has its **direct children's** config
|
||||||
repos bind-mounted **read-only** (topology-driven:
|
repos bind-mounted **read-only** (topology-driven: `lifecycle.rs` calls
|
||||||
`hive-c0re/src/lifecycle/host_config.rs` calls `bind_child_agent_dirs`
|
`bind_child_agent_dirs` for each entry in
|
||||||
for each entry in
|
|
||||||
`topology::children_of(agent_name)`). It is a copy to *read* a child's
|
`topology::children_of(agent_name)`). It is a copy to *read* a child's
|
||||||
current config — not an editing surface.
|
current config — not an editing surface.
|
||||||
|
|
||||||
|
|
@ -502,9 +502,9 @@ forge into its own state dir, commit on a branch, open a PR**, and let
|
||||||
the operator review and approve it. There is deliberately no second,
|
the operator review and approve it. There is deliberately no second,
|
||||||
mount-shaped path that reaches the same file without the review.
|
mount-shaped path that reaches the same file without the review.
|
||||||
|
|
||||||
Agents holding the `can_manage_top_level_agents` topology role (see
|
Agents holding the `can_manage_top_level_agents` topology role
|
||||||
`hive-c0re/src/agent_config/topology.rs`) get additional host-side
|
(defined as `ROLE_CAN_MANAGE_TOP_LEVEL_AGENTS` in `hive-c0re/src/agent_config/topology.rs`)
|
||||||
bind mounts via `set_nspawn_flags`:
|
get additional host-side bind mounts via `set_nspawn_flags`:
|
||||||
|
|
||||||
- `/var/lib/hyperhive/agents/` → `/agents/` (RW) — all top-level
|
- `/var/lib/hyperhive/agents/` → `/agents/` (RW) — all top-level
|
||||||
agents' proposed repos (not just direct children).
|
agents' proposed repos (not just direct children).
|
||||||
|
|
@ -536,24 +536,34 @@ cat /meta/flake.lock | jq '.nodes | with_entries(select(.key | startswith("agent
|
||||||
The RO binds block push at the kernel level — git plumbing inside the
|
The RO binds block push at the kernel level — git plumbing inside the
|
||||||
container cannot corrupt either authoritative repo.
|
container cannot corrupt either authoritative repo.
|
||||||
|
|
||||||
## Startup migrations (older hosts)
|
## Migration from the pre-tag / pre-meta schemes
|
||||||
|
|
||||||
hive-c0re runs a couple of idempotent migrations on every startup so a
|
Both overhauls (tag-driven flow + meta flake) ship in-place
|
||||||
host set up before the tag-driven-deploy + meta-flake scheme (both
|
migrations that run on every hive-c0re startup. Idempotent;
|
||||||
described above) converges to it automatically. Each phase is a no-op
|
each phase is a no-op once already applied. Behaviour:
|
||||||
once already applied:
|
|
||||||
|
|
||||||
- **Tags**: agents from before the tag-driven scheme are tagged
|
- Tag-driven phase: assumes the operator ran the one-shot
|
||||||
`deployed/0` on `main` once. Non-destructive — it doesn't touch live
|
`git tag deployed/0 main` script (see commit history /
|
||||||
containers, state dirs, or claude creds.
|
earlier docs revisions) once per agent. Tagging is
|
||||||
- **Meta flake**: rewrites each `applied/<n>/flake.nix` to the
|
non-destructive: it doesn't touch live containers, state
|
||||||
module-only boilerplate, wires the `applied` remote in each proposed
|
dirs, or claude creds.
|
||||||
repo, and bootstraps the meta repo from the current agent list. Set
|
- Meta-flake phase: rewrites each `applied/<n>/flake.nix` to
|
||||||
`HIVE_SKIP_META_MIGRATION=1` on the service to defer this phase.
|
the module-only boilerplate, wires the `applied` remote in
|
||||||
|
each proposed repo, and bootstraps the meta repo from the
|
||||||
|
current agent list. Set `HIVE_SKIP_META_MIGRATION=1` on the
|
||||||
|
service to defer.
|
||||||
|
|
||||||
No state loss in either migration: claude creds, `/state/` notes, the
|
A further step used to `nixos-container update` every
|
||||||
events DB, and both proposed + applied history all survive. The root
|
container onto `meta#<n>`, guarded by a marker file so it
|
||||||
agent keeps its session; sub-agents stay logged in.
|
ran once per hive. It is gone: containers have been rendered
|
||||||
|
onto `meta#<n>` at creation for long enough that no live hive
|
||||||
|
needs the repoint, and a one-shot nobody can still trigger is
|
||||||
|
dead weight. Same for the `root` → `h-root` container rename.
|
||||||
|
|
||||||
|
No state loss in either migration. claude creds, /state/
|
||||||
|
notes, the events DB, proposed history, and applied history
|
||||||
|
all survive. The root agent keeps its session; sub-agents stay
|
||||||
|
logged in.
|
||||||
|
|
||||||
## The root/bootstrap container is hive-c0re-managed
|
## The root/bootstrap container is hive-c0re-managed
|
||||||
|
|
||||||
|
|
@ -568,7 +578,7 @@ same as any other agent.
|
||||||
|
|
||||||
Differences from sub-agents:
|
Differences from sub-agents:
|
||||||
|
|
||||||
- `flake.nix` extends `hyperhive.nixosConfigurations.ruth`
|
- `flake.nix` extends `hyperhive.nixosConfigurations.manager`
|
||||||
(vs `agent-base`).
|
(vs `agent-base`).
|
||||||
- Web UI port via `lifecycle::agent_web_port("ruth")` — same
|
- Web UI port via `lifecycle::agent_web_port("ruth")` — same
|
||||||
FNV-1a hash as every other agent (8100..8999 range).
|
FNV-1a hash as every other agent (8100..8999 range).
|
||||||
|
|
@ -579,10 +589,8 @@ Differences from sub-agents:
|
||||||
authoritative applied repo (see "Root-agent view of applied" below).
|
authoritative applied repo (see "Root-agent view of applied" below).
|
||||||
- First-deploy spawn bypasses the approval queue (the root agent is
|
- First-deploy spawn bypasses the approval queue (the root agent is
|
||||||
required infrastructure).
|
required infrastructure).
|
||||||
- The root agent's socket is bound by `socket_server::start_manager`,
|
- Per-agent socket lives at `/run/hyperhive/manager/`, owned by
|
||||||
pure transport with no dedicated helpers — it uses the same
|
`manager_server::start`.
|
||||||
per-agent runtime dir as any other agent (`/run/hyperhive/agents/ruth/`),
|
|
||||||
not a special manager-only path.
|
|
||||||
|
|
||||||
**Migration note** (for older hosts): drop any `containers.root =
|
**Migration note** (for older hosts): drop any `containers.root =
|
||||||
{ ... }` block from your host NixOS config. hyperhive creates and
|
{ ... }` block from your host NixOS config. hyperhive creates and
|
||||||
|
|
@ -590,35 +598,56 @@ updates the root agent itself.
|
||||||
|
|
||||||
## Root-agent policy
|
## Root-agent policy
|
||||||
|
|
||||||
The system prompt (`hive-agent/prompts/system.md`, rendered by
|
The system prompt (`hive-ag3nt/prompts/system.md`, rendered via
|
||||||
`hive-agent/src/prompt.rs`) is the **same for every agent**; what
|
`hive_ag3nt::prompt::render`) is the **same for every agent**; what
|
||||||
varies is which MCP tools are surfaced (gated by tool groups and
|
varies is which MCP tools are surfaced (gated by tool groups and
|
||||||
capabilities in `agent.nix`). There is no `role:manager` block that
|
capabilities in `agent.nix`). There is no `role:manager` block that
|
||||||
renders only for the root agent. The root agent's approval-gating
|
renders only for the root agent. The root agent's approval-gating
|
||||||
behaviour comes from its CLAUDE.md / agent-specific instructions, not
|
behaviour comes from its CLAUDE.md / agent-specific instructions, not
|
||||||
the system prompt template.
|
the system prompt template.
|
||||||
|
|
||||||
Any agent (root or not) can also ask a structured question of the
|
`ask(question, options?, multi?, ttl_seconds?, to?)` is available to
|
||||||
operator or a peer agent via the `ask`/`answer` MCP tools, independent
|
**any agent** — it queues a question and returns the id immediately.
|
||||||
of the approval flow above — see
|
When `to` is omitted (or `"operator"`) the question shows up on the
|
||||||
`docs/conventions.md#question-routing-ask--answer` for the routing
|
dashboard; when `to` is another agent's name, the recipient receives a
|
||||||
rules, `ttl_seconds` expiry, and cancellation.
|
`HelperEvent::QuestionAsked` and answers via their own `answer`
|
||||||
|
tool. Either way the answer arrives back as
|
||||||
|
`HelperEvent::QuestionAnswered { id, question, answer, answerer }`
|
||||||
|
in the asker's inbox. Storage is `hive-c0re::operator_questions`
|
||||||
|
(sqlite) — same table, with a nullable `target` column
|
||||||
|
(NULL = operator). Dispatch goes through
|
||||||
|
`hive-c0re/src/questions.rs::{handle_ask, handle_answer}`. The answer flow is:
|
||||||
|
|
||||||
|
```
|
||||||
|
POST /answer-question/{id} agent: Answer { id, answer }
|
||||||
|
→ OperatorQuestions::answer(_, _, "operator") → questions::handle_answer
|
||||||
|
→ notify_agent(asker, QuestionAnswered { → OperatorQuestions::answer(_, _, agent)
|
||||||
|
answerer: "operator", ... }) → notify_agent(asker, QuestionAnswered {
|
||||||
|
answerer: agent, ... })
|
||||||
|
```
|
||||||
|
|
||||||
|
Two more paths resolve a pending question with a sentinel answer:
|
||||||
|
|
||||||
|
- `POST /cancel-question/{id}` (✗ CANC3L button on the dashboard)
|
||||||
|
resolves with `[cancelled]`. The asking agent sees a terminal state
|
||||||
|
and can fall back.
|
||||||
|
- `ttl_seconds` deadline: a tokio watchdog spawned at submit time
|
||||||
|
fires `answer(id, "[expired]")` once the ttl runs out. Already-
|
||||||
|
resolved races no-op. The dashboard surfaces a `⏳ MM:SS` chip
|
||||||
|
on each pending question with a deadline.
|
||||||
|
|
||||||
## Helper events to the submitting agent
|
## Helper events to the submitting agent
|
||||||
|
|
||||||
`Coordinator::notify_submitter(approval_id, &HelperEvent)` routes the
|
`Coordinator::notify_submitter(approval_id, &HelperEvent)` routes the
|
||||||
event to the agent that originally submitted the approval (looked up from
|
event to the agent that originally submitted the approval (looked up from
|
||||||
the `submitter` column on the `approvals` table). The harness delivers it
|
the `submitter` column on the `approvals` table). The harness delivers it
|
||||||
as a regular `system` inbox message so it drives a normal claude turn.
|
as a regular `system` inbox message so it drives a normal claude turn. A
|
||||||
`finish_approval` fires an `ApprovalResolved` HelperEvent this way for
|
|
||||||
**every** approval kind's terminal state, `Spawn` included. A
|
|
||||||
"FYI, check when convenient" event doesn't need a message — those go
|
"FYI, check when convenient" event doesn't need a message — those go
|
||||||
through `Coordinator::push_todo`/`push_todo_submitter` instead, a direct
|
through `Coordinator::push_todo`/`push_todo_submitter` instead, a direct
|
||||||
live dial of the target agent's in-container todo socket (same
|
live dial of the target agent's in-container todo socket (same
|
||||||
`UpsertTodo` request in-container producers use); `finish_approval` fires
|
`UpsertTodo` request in-container producers use); the `Spawn` approval
|
||||||
one of these too for `InitConfig`/`Spawn`/`MergeConfigPr`, *in addition to*
|
uses this path, not a `HelperEvent`. Legacy approval rows that predate the
|
||||||
the `ApprovalResolved` HelperEvent above, not instead of it. Legacy
|
submitter column fall back to the
|
||||||
approval rows that predate the submitter column fall back to the
|
|
||||||
root agent. Variants (`hive_sh4re::manager::HelperEvent`):
|
root agent. Variants (`hive_sh4re::manager::HelperEvent`):
|
||||||
|
|
||||||
- `ApprovalResolved { id, agent, commit_ref, status, note }` —
|
- `ApprovalResolved { id, agent, commit_ref, status, note }` —
|
||||||
|
|
@ -644,24 +673,25 @@ root agent. Variants (`hive_sh4re::manager::HelperEvent`):
|
||||||
The recipient responds via `Answer { id, answer }` and the
|
The recipient responds via `Answer { id, answer }` and the
|
||||||
asker sees the matching `QuestionAnswered`.
|
asker sees the matching `QuestionAnswered`.
|
||||||
|
|
||||||
The remaining lower-urgency lifecycle notices — `Rebuilt`, `Killed`,
|
The rest of the original lifecycle notices — `Rebuilt`, `Killed`,
|
||||||
`Destroyed`, `NeedsLogin`, `LoggedIn`, `ConfigReady` — are "FYI, check
|
`Destroyed`, `NeedsLogin`, `LoggedIn`, `ConfigReady` — were pure "FYI,
|
||||||
when convenient" events with no reason to drive an immediate turn, so
|
check when convenient" events with no reason to drive an immediate
|
||||||
they deliver via `push_todo`/`push_todo_submitter` (see above) instead
|
turn, so they've been migrated off `HelperEvent` onto
|
||||||
of `HelperEvent`: an `agent_todo_socket` push instead of a broker
|
`push_todo`/`push_todo_submitter` (see above): `agent_todo_socket`
|
||||||
message, `subsystem = "core"`, `key = "<event>:<agent>"` for dedup,
|
push instead of a broker message, `subsystem = "core"`,
|
||||||
and a single free-text `summary` (`rebuilt_todo_summary` renders
|
`key = "<event>:<agent>"` for dedup, one free-text `summary` in place
|
||||||
`Rebuilt`'s `ok`/`note`/`sha`/`tag` fields into that string).
|
of the old structured fields (`rebuilt_todo_summary` renders
|
||||||
|
`Rebuilt`'s former `ok`/`note`/`sha`/`tag` into that string).
|
||||||
|
|
||||||
Optional `sha` field on `ApprovalResolved` carries the canonical
|
Optional `sha` field on `ApprovalResolved` carries the canonical
|
||||||
hive-c0re-vouched commit sha. Optional `tag` carries the deploy
|
hive-c0re-vouched commit sha. Optional `tag` carries the deploy
|
||||||
bookkeeping tag — `deployed/<id>` on a successful build or
|
bookkeeping tag — `deployed/<id>` on a successful build or
|
||||||
`failed/<id>` on a failed one, planted by the `MergeConfigPr` deploy.
|
`failed/<id>` on a failed one, planted by the `MergeConfigPr` deploy.
|
||||||
Both fields are `Option`: `None` on the paths that don't deploy a new
|
Both fields are `Option`: `None` on the paths that don't deploy a new
|
||||||
commit (spawn / init_config / meta-update / deny, and the auto-update
|
commit (spawn / init_config / meta-update / deny, and
|
||||||
sweep's `job_queue::templates::rebuild` reapplying the existing main,
|
`auto_update::rebuild_agent` reapplying the existing main, or the
|
||||||
or the dashboard `↻ R3BU1LD` button when the lock didn't move). When set,
|
dashboard `↻ R3BU1LD` button when the lock didn't move). When set,
|
||||||
`git show <sha>` against `/applied/<n>/.git` inside the
|
`git show <sha>` against `/agents/<n>/applied.git` inside the
|
||||||
bootstrap container yields the exact tree that was referenced.
|
bootstrap container yields the exact tree that was referenced.
|
||||||
|
|
||||||
To add a new lifecycle notice: if it needs to drive an immediate turn
|
To add a new lifecycle notice: if it needs to drive an immediate turn
|
||||||
|
|
@ -676,8 +706,8 @@ no new wire type needed.
|
||||||
`hive-c0re serve` runs `auto_update::run` in a background task right
|
`hive-c0re serve` runs `auto_update::run` in a background task right
|
||||||
after opening the coordinator. It enumerates managed containers and
|
after opening the coordinator. It enumerates managed containers and
|
||||||
rebuilds any whose recorded hyperhive rev differs from the current
|
rebuilds any whose recorded hyperhive rev differs from the current
|
||||||
one — sub-agents and the root agent go through the same
|
one — sub-agents and the root agent go through the same `lifecycle::rebuild`
|
||||||
`job_queue::templates::rebuild` DAG.
|
path.
|
||||||
|
|
||||||
"Rev" = canonical filesystem path of `cfg.hyperhiveFlake`. Marker
|
"Rev" = canonical filesystem path of `cfg.hyperhiveFlake`. Marker
|
||||||
file: `/var/lib/hyperhive/applied/.<name>.hyperhive-rev`. If the
|
file: `/var/lib/hyperhive/applied/.<name>.hyperhive-rev`. If the
|
||||||
|
|
@ -686,8 +716,8 @@ auto-update is a no-op — rebuild manually.
|
||||||
|
|
||||||
The dashboard surfaces pending updates per agent: a clickable
|
The dashboard surfaces pending updates per agent: a clickable
|
||||||
"needs update ↻" badge appears whenever the marker differs from
|
"needs update ↻" badge appears whenever the marker differs from
|
||||||
current rev. The badge POSTs `/api/rebuild/<name>`, which inserts the
|
current rev. The badge POSTs `/api/rebuild/<name>`, calling the same
|
||||||
same `job_queue::templates::rebuild` DAG so manual triggers and the
|
`auto_update::rebuild_agent` path so manual triggers and the
|
||||||
startup scan can't drift. When at least one container is stale, a
|
startup scan can't drift. When at least one container is stale, a
|
||||||
top-level `↻ UPD4TE 4LL` button appears that loops over every
|
top-level `↻ UPD4TE 4LL` button appears that loops over every
|
||||||
stale container.
|
stale container.
|
||||||
|
|
|
||||||
|
|
@ -5,10 +5,12 @@ _implementation_ work — container network isolation, the unifying
|
||||||
gateway, core-daemon privsep — is tracked as `area:ops` issues on
|
gateway, core-daemon privsep — is tracked as `area:ops` issues on
|
||||||
the forge.
|
the forge.
|
||||||
|
|
||||||
The operator/agent boundary is technically enforced, not just a
|
The operator/agent boundary is now technically enforced, not just a
|
||||||
convention: containers run in private netns (network isolation is
|
convention. Containers run in private netns (network isolation is
|
||||||
always on), the gateway proxies all operator-facing traffic, and
|
always on), the gateway proxies all operator-facing traffic, and
|
||||||
`hive-c0re` runs as the unprivileged `hive-core` user.
|
`hive-c0re` runs as the unprivileged `hive-core` user. All three
|
||||||
|
`area:ops` pillars — network isolation, the gateway, and privsep —
|
||||||
|
are complete and active.
|
||||||
|
|
||||||
## Two principals, two paths
|
## Two principals, two paths
|
||||||
|
|
||||||
|
|
@ -29,10 +31,9 @@ always on), the gateway proxies all operator-facing traffic, and
|
||||||
point.** They live on the core backend.
|
point.** They live on the core backend.
|
||||||
|
|
||||||
Worked example — answering an operator-targeted question is a
|
Worked example — answering an operator-targeted question is a
|
||||||
`POST /api/answer-question/{id}` on the core dashboard, _never_ a
|
`POST /answer-question/{id}` on the core dashboard, _never_ an
|
||||||
per-agent-socket `Request` variant. If it were a per-agent-socket
|
`AgentRequest` variant. If it were a per-agent-socket request, an
|
||||||
request, an agent could `curl` its own socket and spoof an operator
|
agent could `curl` its own socket and spoof an operator answer.
|
||||||
answer.
|
|
||||||
The per-agent web UI POSTs cross-origin to the core for these
|
The per-agent web UI POSTs cross-origin to the core for these
|
||||||
(see the inline-answer feature — the loose-ends section on each
|
(see the inline-answer feature — the loose-ends section on each
|
||||||
agent page).
|
agent page).
|
||||||
|
|
@ -46,9 +47,9 @@ every boundary claim above is aspirational. Network isolation is what
|
||||||
makes the boundary _real_; the gateway and privsep are ergonomics and
|
makes the boundary _real_; the gateway and privsep are ergonomics and
|
||||||
defence-in-depth layered on top.
|
defence-in-depth layered on top.
|
||||||
|
|
||||||
Network isolation is complete and always on: every agent container
|
Network isolation is now complete and always on: every agent container
|
||||||
runs in a private netns behind the hive bridge, and there is no
|
runs in a private netns behind the hive bridge. The shared-netns mode
|
||||||
shared-netns mode. See `docs/network.md`.
|
was removed. See `docs/network.md`.
|
||||||
|
|
||||||
Concretely, the core daemon's dashboard `/api` carries **no
|
Concretely, the core daemon's dashboard `/api` carries **no
|
||||||
application-layer authentication** — operator-authority routes are served
|
application-layer authentication** — operator-authority routes are served
|
||||||
|
|
@ -63,18 +64,15 @@ operator-authority route inherits that assumption. `hive-ci` is treated like an
|
||||||
agent for this purpose — it runs untrusted PR code and is netns-isolated for
|
agent for this purpose — it runs untrusted PR code and is netns-isolated for
|
||||||
the same reason.
|
the same reason.
|
||||||
|
|
||||||
The boundary rests on three layers:
|
The `area:ops` issues followed this sequencing:
|
||||||
|
|
||||||
1. **Gateway** — fronts all surfaces (dashboard + every per-agent UI)
|
1. **Gateway** — pure ergonomics win, unblocks same-origin (lets the
|
||||||
on one origin. An nginx nixos-container proxies per-agent UIs under
|
cross-origin CORS shim on `/answer-question/{id}` go away), no
|
||||||
`/agent/<name>/`, which is what lets the inline-answer POST to
|
behavioural risk. An nginx nixos-container now sits in front of all
|
||||||
`/answer-question/{id}` go same-origin instead of needing a
|
surfaces; per-agent UIs are proxied under `/agent/<name>/`.
|
||||||
cross-origin CORS shim. Pure ergonomics — no behavioural risk on
|
2. **Network isolation** — the load-bearing step that turns the
|
||||||
its own.
|
honour-system split into an enforced boundary. **Complete** —
|
||||||
2. **Network isolation** — the load-bearing layer: every agent
|
always-on, unconditional; the shared-netns mode was removed.
|
||||||
container runs in a private netns behind the hive bridge, always
|
|
||||||
on and unconditional. This is what turns the operator/agent split
|
|
||||||
from an honour-system convention into an enforced boundary.
|
|
||||||
3. **Privsep** — defence in depth on the core process; `hive-c0re`
|
3. **Privsep** — defence in depth on the core process; `hive-c0re`
|
||||||
runs as the unprivileged `hive-core` user and delegates root
|
runs as the unprivileged `hive-core` user and delegates root
|
||||||
operations to `hive-priv`, a narrow socket-activated helper. See
|
operations to `hive-priv`, a narrow socket-activated helper. See
|
||||||
|
|
@ -86,14 +84,15 @@ The boundary rests on three layers:
|
||||||
systemd unit. The unit binds `/run/hive/priv.sock` with
|
systemd unit. The unit binds `/run/hive/priv.sock` with
|
||||||
`SocketGroup=hive-core` and mode `0660` and passes the ready listener
|
`SocketGroup=hive-core` and mode `0660` and passes the ready listener
|
||||||
to the helper as fd 3 (`LISTEN_FDS`). The helper requires this and
|
to the helper as fd 3 (`LISTEN_FDS`). The helper requires this and
|
||||||
bails if it isn't socket-activated.
|
bails if it isn't socket-activated — there is intentionally no
|
||||||
|
self-bind fallback.
|
||||||
|
|
||||||
⚠️ There is intentionally no self-bind fallback: if `hive-priv` bound
|
Dropping the old fallback removed a dev/prod divergence: when
|
||||||
the socket itself, it would create the file owned by root's primary
|
`hive-priv` bound the socket itself it created the file owned by
|
||||||
group rather than `hive-core`, and a `hive-core` client couldn't
|
root's primary group rather than `hive-core`, so a `hive-core` client
|
||||||
connect the way the socket unit's `SocketGroup` grant intends.
|
couldn't connect the way the socket unit's `SocketGroup` grant
|
||||||
Requiring socket activation everywhere keeps dev and prod on the
|
intends. Requiring socket activation everywhere means dev and prod
|
||||||
exact same path, so the group grant always holds.
|
take the exact same path and the group grant always holds.
|
||||||
|
|
||||||
### the per-agent socket dir
|
### the per-agent socket dir
|
||||||
|
|
||||||
|
|
@ -130,9 +129,10 @@ nginx reaches all of `/run/hive-agent` as a plain host path. Dropping
|
||||||
them and the rest of the host. That costs no network isolation: nginx
|
them and the rest of the host. That costs no network isolation: nginx
|
||||||
binds the host's `:80`/`:443` and reaches `localhost` upstreams, which a
|
binds the host's `:80`/`:443` and reaches `localhost` upstreams, which a
|
||||||
netns would have to be opened up for anyway.
|
netns would have to be opened up for anyway.
|
||||||
🔑 It does mean nothing *implicitly* scopes the privileged reload verb —
|
🔑 It does mean nothing *implicitly* scopes the privileged reload verb,
|
||||||
see [`docs/security.md`](security.md#hive-c0re-privilege-separation) for
|
so the scope is explicit: the unit name is hard-coded in `hive-priv` —
|
||||||
how `PrivRequest::ReloadGatewayNginx`'s containment works.
|
see `PrivRequest::ReloadGatewayNginx`. **A caller cannot name the unit,
|
||||||
|
so the verb cannot be steered at another service.**
|
||||||
|
|
||||||
⚠️ Contrast `/shared`, which *is* sticky world-writable (`1777`): it has
|
⚠️ Contrast `/shared`, which *is* sticky world-writable (`1777`): it has
|
||||||
many legitimate writers, so sticky is the best available answer there.
|
many legitimate writers, so sticky is the best available answer there.
|
||||||
|
|
|
||||||
59
docs/ci.md
59
docs/ci.md
|
|
@ -5,7 +5,7 @@ executing CI jobs from `.forgejo/workflows/ci.yml` on every PR.
|
||||||
|
|
||||||
## For operators
|
## For operators
|
||||||
|
|
||||||
**Enabling it is one line**: `services.hyperhive.swarm.forge.ci.enable = true`
|
**Enabling it is one line**: `services.hyperhive.forge.ci.enable = true`
|
||||||
in the host NixOS config. No manual token provisioning — hive-c0re
|
in the host NixOS config. No manual token provisioning — hive-c0re
|
||||||
registers the runner with the forge automatically.
|
registers the runner with the forge automatically.
|
||||||
|
|
||||||
|
|
@ -31,19 +31,17 @@ writeup.
|
||||||
|
|
||||||
## CI checks
|
## CI checks
|
||||||
|
|
||||||
Three jobs run on every PR (and on `workflow_dispatch` for manual re-triggers),
|
Three jobs run on every PR (and on `workflow_dispatch` for manual re-triggers):
|
||||||
defined in [`.forgejo/workflows/ci.yml`](../.forgejo/workflows/ci.yml). All
|
|
||||||
three are required checks (forge branch protection) — a hit on any of them
|
|
||||||
blocks merge.
|
|
||||||
|
|
||||||
| Job | What it runs |
|
| Job | What it runs | Currently required |
|
||||||
| --- | --- |
|
| --- | --- | --- |
|
||||||
| **nix flake check** | treefmt + rustfmt formatting, `cargo clippy -D warnings`, `cargo test`, module evaluation |
|
| **nix flake check** | treefmt + rustfmt formatting, `cargo clippy -D warnings`, `cargo test`, module evaluation | yes |
|
||||||
| **tracker-tag lint** | flags `#NNN` issue tags in source and comments (`scripts/check-issue-refs.sh`) |
|
| **tracker-tag lint** | flags `#NNN` issue tags in source and comments (`scripts/check-issue-refs.sh`) | no (red, non-blocking) |
|
||||||
| **comment-block lint** | flags contiguous comment blocks over 30 lines (`scripts/check-comment-blocks.sh`) |
|
| **comment-block lint** | flags contiguous comment blocks over 30 lines (`scripts/check-comment-blocks.sh`) | no (red, non-blocking) |
|
||||||
|
|
||||||
`hive-forge ci-rerun --pr N` dispatches a `workflow_dispatch` retrigger
|
The tracker-tag and comment-block checks are non-blocking today (a hit fails the
|
||||||
without an empty commit.
|
check but does not prevent merge) while the legacy backlog is cleaned up. They are
|
||||||
|
expected to become required checks once the tree is clean.
|
||||||
|
|
||||||
### Running checks locally
|
### Running checks locally
|
||||||
|
|
||||||
|
|
@ -70,17 +68,14 @@ diagnostic if either fails — catching the issue locally before CI sees it.
|
||||||
Note that the hook does **not** run `cargo clippy` or `cargo test` (those are
|
Note that the hook does **not** run `cargo clippy` or `cargo test` (those are
|
||||||
slow); run those manually before pushing Rust changes.
|
slow); run those manually before pushing Rust changes.
|
||||||
|
|
||||||
## Configuration reference
|
## Operator bootstrap
|
||||||
|
|
||||||
The internal forge is always present (mandatory), so the runner always has a
|
Set `services.hyperhive.forge.ci.enable = true` in the host NixOS config. That's it — no manual token provisioning.
|
||||||
hive-forge instance to register against — nothing extra to enable beyond
|
|
||||||
`services.hyperhive.swarm.forge.ci.enable = true` (see *For operators* above).
|
|
||||||
|
|
||||||
Optional tuning: `services.hyperhive.swarm.forge.ci.name` (runner name in forge
|
**Requirements:**
|
||||||
admin panel), `concurrency` (parallel job capacity), `labels` (workflow
|
|
||||||
targeting), `jobTimeout` (per-job wall-clock cap, default `"1h"`, Go duration
|
- The internal forge is always present (mandatory), so the runner always has a hive-forge instance to register against — nothing extra to enable.
|
||||||
string e.g. `"3h"` — a job that exceeds it is killed so a hung or runaway
|
- Optional: tune `services.hyperhive.forge.ci.name` (runner name in forge admin panel), `concurrency` (parallel job capacity), `labels` (workflow targeting), `jobTimeout` (per-job wall-clock cap, default `"1h"`, Go duration string e.g. `"3h"` — a job that exceeds it is killed so a hung or runaway build can't hold the runner's single slot indefinitely).
|
||||||
build can't hold the runner's single slot indefinitely).
|
|
||||||
|
|
||||||
## Container design
|
## Container design
|
||||||
|
|
||||||
|
|
@ -91,14 +86,7 @@ build can't hold the runner's single slot indefinitely).
|
||||||
|
|
||||||
## Auto-registration flow
|
## Auto-registration flow
|
||||||
|
|
||||||
Registration is **off the container's boot-critical path** — hive-c0re owns
|
Registration is **off the container's boot-critical path** — hive-c0re owns it and runs it out of band, so a slow forge or core-token never delays the container's start. (The earlier design ran a host-side `hive-ci-prefetch.service` that gated `container@hive-ci` start on a forge round-trip, which could exceed the nspawn start timeout and trip a restart loop; moving registration into hive-c0re removed that.) The core admin token is held only by hive-c0re on the host; only the runner registration token reaches the container.
|
||||||
it and runs it out of band, so a slow forge or core-token never delays the
|
|
||||||
container's start. Gotcha: don't gate the container's own start on a forge
|
|
||||||
round-trip (a host-side unit that did this could exceed the nspawn start
|
|
||||||
timeout and trip a restart loop) — registration must stay something
|
|
||||||
hive-c0re drives after the container is already up. The core admin token is
|
|
||||||
held only by hive-c0re on the host; only the runner registration token
|
|
||||||
reaches the container.
|
|
||||||
|
|
||||||
### hive-c0re side (`forge/ci_runner.rs`, run during the startup sweep)
|
### hive-c0re side (`forge/ci_runner.rs`, run during the startup sweep)
|
||||||
|
|
||||||
|
|
@ -126,7 +114,7 @@ When `forge.ci.enable` is set, hive-c0re auto-seeds an
|
||||||
external DNS on the CI critical path.
|
external DNS on the CI critical path.
|
||||||
|
|
||||||
The mirror is seeded by **hive-c0re** itself during its forge
|
The mirror is seeded by **hive-c0re** itself during its forge
|
||||||
provisioning sweep (`forge/repos.rs::ensure_mirrors`). The nix module
|
provisioning sweep (`forge.rs::ensure_mirrors`). The nix module
|
||||||
forwards the effective mirror list as `HYPERHIVE_FORGE_MIRRORS` in the
|
forwards the effective mirror list as `HYPERHIVE_FORGE_MIRRORS` in the
|
||||||
`hive-c0re` service environment (JSON-encoded `[{upstream, dest}]`
|
`hive-c0re` service environment (JSON-encoded `[{upstream, dest}]`
|
||||||
list). hive-c0re already holds the admin token for the rest of the
|
list). hive-c0re already holds the admin token for the rest of the
|
||||||
|
|
@ -134,10 +122,10 @@ forge provisioning sweep (orgs, agent accounts, etc.), so mirror
|
||||||
seeding lives in the same place rather than a separate host-side unit.
|
seeding lives in the same place rather than a separate host-side unit.
|
||||||
|
|
||||||
**General-purpose mirrors**: you can pre-seed any external repo as a
|
**General-purpose mirrors**: you can pre-seed any external repo as a
|
||||||
pull-mirror via `services.hyperhive.swarm.forge.mirrors`:
|
pull-mirror via `services.hyperhive.forge.mirrors`:
|
||||||
|
|
||||||
```nix
|
```nix
|
||||||
services.hyperhive.swarm.forge.mirrors = [
|
services.hyperhive.forge.mirrors = [
|
||||||
{ upstream = "https://github.com/actions/checkout"; dest = "actions/checkout"; }
|
{ upstream = "https://github.com/actions/checkout"; dest = "actions/checkout"; }
|
||||||
{ upstream = "https://github.com/example/tool"; dest = "mirrors/tool"; }
|
{ upstream = "https://github.com/example/tool"; dest = "mirrors/tool"; }
|
||||||
];
|
];
|
||||||
|
|
@ -151,6 +139,13 @@ runner as a hard `git clone` failure. The
|
||||||
hive-c0re-managed namespaces (`config/`, `shared/`, `agents/`, `core/`)
|
hive-c0re-managed namespaces (`config/`, `shared/`, `agents/`, `core/`)
|
||||||
to avoid provisioning collisions.
|
to avoid provisioning collisions.
|
||||||
|
|
||||||
|
## CI workflow
|
||||||
|
|
||||||
|
Three jobs are defined in [`.forgejo/workflows/ci.yml`](../.forgejo/workflows/ci.yml):
|
||||||
|
`nix flake check`, `tracker-tag lint`, and `comment-block lint`. All three are
|
||||||
|
required — a lint failure blocks merge. `hive-forge ci-rerun --pr N` dispatches
|
||||||
|
a `workflow_dispatch` retrigger without an empty commit.
|
||||||
|
|
||||||
## Security: unsandboxed builds and trusted contributors
|
## Security: unsandboxed builds and trusted contributors
|
||||||
|
|
||||||
**hive-ci should only run CI for trusted contributors.** The security boundary is weaker than it looks:
|
**hive-ci should only run CI for trusted contributors.** The security boundary is weaker than it looks:
|
||||||
|
|
|
||||||
|
|
@ -8,7 +8,7 @@ exist because something already went wrong without them.
|
||||||
- Containers are length-bounded by `nixos-container` (≤ 11 chars).
|
- Containers are length-bounded by `nixos-container` (≤ 11 chars).
|
||||||
- Sub-agents are `h-<name>` with `<name>` ≤ 9 chars.
|
- Sub-agents are `h-<name>` with `<name>` ≤ 9 chars.
|
||||||
- One agent is the bootstrap/root container, with a fixed name (`ruth` today).
|
- One agent is the bootstrap/root container, with a fixed name (`ruth` today).
|
||||||
- `MAX_AGENT_NAME` in `hive-c0re/src/lifecycle/mod.rs` enforces the cap.
|
- `MAX_AGENT_NAME` in `lifecycle.rs` enforces the cap.
|
||||||
- Per-agent web UI port = `WEB_PORT_BASE + FNV1a(name) % WEB_PORT_RANGE`
|
- Per-agent web UI port = `WEB_PORT_BASE + FNV1a(name) % WEB_PORT_RANGE`
|
||||||
(8100..8999) for every agent; dashboard
|
(8100..8999) for every agent; dashboard
|
||||||
`cfg.dashboardPort` (default 7000).
|
`cfg.dashboardPort` (default 7000).
|
||||||
|
|
@ -16,7 +16,7 @@ exist because something already went wrong without them.
|
||||||
## Hive identity (label + domain + display names)
|
## Hive identity (label + domain + display names)
|
||||||
|
|
||||||
Four env vars cover the identity surface, read by
|
Four env vars cover the identity surface, read by
|
||||||
`hive-agent/src/identity.rs`:
|
`hive_ag3nt::identity`:
|
||||||
|
|
||||||
- `HIVE_LABEL` — short, hive-local agent label (`iris`,
|
- `HIVE_LABEL` — short, hive-local agent label (`iris`,
|
||||||
`damocles`). `label()` returns it; falls back to empty string if
|
`damocles`). `label()` returns it; falls back to empty string if
|
||||||
|
|
@ -24,8 +24,8 @@ Four env vars cover the identity surface, read by
|
||||||
surface "unknown agent" rather than getting a panic from this
|
surface "unknown agent" rather than getting a panic from this
|
||||||
module.
|
module.
|
||||||
- `HYPERHIVE_HIVE_DOMAIN` — the hive's canonical DNS domain (e.g.
|
- `HYPERHIVE_HIVE_DOMAIN` — the hive's canonical DNS domain (e.g.
|
||||||
`darkest.space`), set by `nix/host-modules/hive-c0re/environment.nix`
|
`darkest.space`), set by `hive-c0re.nix` from
|
||||||
from `services.hyperhive.domain`. When configured, `qualified_label()`
|
`services.hyperhive.domain`. When configured, `qualified_label()`
|
||||||
returns `${label}@${domain}` (e.g. `iris@darkest.space`); when
|
returns `${label}@${domain}` (e.g. `iris@darkest.space`); when
|
||||||
unset (single-hive deployments, dev/test) it degrades to just
|
unset (single-hive deployments, dev/test) it degrades to just
|
||||||
the short label so existing callers see no change. The
|
the short label so existing callers see no change. The
|
||||||
|
|
@ -83,7 +83,7 @@ name validation rejects any character outside `[a-z0-9_-]`, so the
|
||||||
angle-bracket and asterisk shapes below are structurally safe.
|
angle-bracket and asterisk shapes below are structurally safe.
|
||||||
|
|
||||||
- `*` — broadcast: deliver to every running agent except the sender
|
- `*` — broadcast: deliver to every running agent except the sender
|
||||||
(`socket_server::handle_send` fans out via `Coordinator::broadcast_send`).
|
(`agent_server::handle_send` fans out via `Coordinator::broadcast_send`).
|
||||||
- `operator` — the human at the dashboard. Messages accumulate in the
|
- `operator` — the human at the dashboard. Messages accumulate in the
|
||||||
inbox view; no agent ever `recv`'s them.
|
inbox view; no agent ever `recv`'s them.
|
||||||
- `<parent>` — the sender's parent per `topology.json`. Rewritten at
|
- `<parent>` — the sender's parent per `topology.json`. Rewritten at
|
||||||
|
|
@ -93,7 +93,7 @@ angle-bracket and asterisk shapes below are structurally safe.
|
||||||
their parent without learning the label, so runtime reparenting
|
their parent without learning the label, so runtime reparenting
|
||||||
propagates with zero agent-side restart.
|
propagates with zero agent-side restart.
|
||||||
- `<children>` — fan-out to every direct descendant of the sender per
|
- `<children>` — fan-out to every direct descendant of the sender per
|
||||||
`topology.json`. Resolved in `socket_server::handle_send` via
|
`topology.json`. Resolved in `agent_server::handle_send` via
|
||||||
`topology::children_of(sender)`: one message is delivered to each
|
`topology::children_of(sender)`: one message is delivered to each
|
||||||
child, bypassing the allow-list check (structural fan-out targets are
|
child, bypassing the allow-list check (structural fan-out targets are
|
||||||
never user-listed peers). No-op for leaf agents (returns `Ok` when the
|
never user-listed peers). No-op for leaf agents (returns `Ok` when the
|
||||||
|
|
@ -279,7 +279,7 @@ an agent. Self-introspection when `name = None` (replaces the older
|
||||||
`Whoami` request); target query when `name = Some`.
|
`Whoami` request); target query when `name = Some`.
|
||||||
|
|
||||||
Response is `AgentMeta { name, running, hyperhive_rev,
|
Response is `AgentMeta { name, running, hyperhive_rev,
|
||||||
status_text, status_set_at, hive_name, swarm_name, matrix_accounts }`:
|
status_text, status_set_at, hive_name, swarm_name }`:
|
||||||
|
|
||||||
- `hyperhive_rev`: `None` only when the configured flake URL has
|
- `hyperhive_rev`: `None` only when the configured flake URL has
|
||||||
no canonical path. Otherwise carries the rev the target is
|
no canonical path. Otherwise carries the rev the target is
|
||||||
|
|
@ -299,9 +299,6 @@ status_text, status_set_at, hive_name, swarm_name, matrix_accounts }`:
|
||||||
`HYPERHIVE_HIVE_NAME` / `HYPERHIVE_SWARM_NAME` env (sourced from
|
`HYPERHIVE_HIVE_NAME` / `HYPERHIVE_SWARM_NAME` env (sourced from
|
||||||
`services.hyperhive.hiveName` / `services.hyperhive.swarm.name`).
|
`services.hyperhive.hiveName` / `services.hyperhive.swarm.name`).
|
||||||
Both `None` when the options aren't configured.
|
Both `None` when the options aren't configured.
|
||||||
- `matrix_accounts`: one `MatrixIdentity` per configured + live matrix
|
|
||||||
account the agent can act as. Empty for agents with no matrix
|
|
||||||
provisioning.
|
|
||||||
|
|
||||||
### Timestamps on the wire
|
### Timestamps on the wire
|
||||||
|
|
||||||
|
|
@ -326,16 +323,14 @@ binary flavor.
|
||||||
|
|
||||||
| Group | Tools |
|
| Group | Tools |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `messaging` | `send`, `recv`, `ack_until`, `ask`, `answer` |
|
| `messaging` | `send`, `recv`, `ask`, `answer` |
|
||||||
| `meta` | `get_agent_meta` (`set_status` is always-on, see below) |
|
| `meta` | `get_agent_meta` (`set_status` is always-on, see below) |
|
||||||
| `inbox` | `get_loose_ends`, `cancel_loose_end`, `remind` |
|
| `inbox` | `get_loose_ends`, `cancel_loose_end`, `remind` |
|
||||||
| `execution` | vestigial — `mcp__bash__run` / `mcp__bash__status` are always available unconditionally via `extraMcpServers`; this group's entries expand to non-existent `mcp__hyperhive__run` / `mcp__hyperhive__status` and have no effect. See `docs/tools/bash.md`. |
|
| `execution` | vestigial — `mcp__bash__run` / `mcp__bash__status` are always available unconditionally via `extraMcpServers`; this group's entries expand to non-existent `mcp__hyperhive__run` / `mcp__hyperhive__status` and have no effect. See `docs/tools/bash.md`. |
|
||||||
| `lifecycle` | `kill`, `start`, `restart`, `update`, `list_containers` *(privileged)* |
|
| `lifecycle` | `kill`, `start`, `restart`, `update` *(privileged)* |
|
||||||
| `approvals` | `request_init_config`, `request_update_meta_inputs` *(privileged)* |
|
| `approvals` | `request_init_config`, `request_update_meta_inputs` *(privileged)* |
|
||||||
| `scheduling` | `request_schedule_prompt`, `fire_schedule_now`, `cancel_schedule`, `edit_schedule`, `list_schedules` *(privileged)* |
|
| `scheduling` | `request_schedule_prompt`, `fire_schedule_now`, `cancel_schedule`, `edit_schedule`, `list_schedules` *(privileged)* |
|
||||||
| `diagnostics` | `get_logs` *(privileged)* |
|
| `diagnostics` | `get_logs` *(privileged)* |
|
||||||
| `forge` | `create_repo` — create git repos through hive-c0re (operator-gated merge) |
|
|
||||||
| `web_tools` | none (gates the Claude built-ins `WebFetch`/`WebSearch`, not an MCP tool) |
|
|
||||||
|
|
||||||
**Always-on tools** — `set_status`, `compact`, and `mark_todos_done` are
|
**Always-on tools** — `set_status`, `compact`, and `mark_todos_done` are
|
||||||
exposed to every agent regardless of which groups it holds
|
exposed to every agent regardless of which groups it holds
|
||||||
|
|
@ -368,10 +363,10 @@ from `tool-groups.json`). Unrecognised tokens are logged and skipped. Falls back
|
||||||
to `ToolGroup::AGENT_DEFAULT` (`messaging`, `meta`, `inbox`, `execution`) when
|
to `ToolGroup::AGENT_DEFAULT` (`messaging`, `meta`, `inbox`, `execution`) when
|
||||||
the var is absent or empty.
|
the var is absent or empty.
|
||||||
|
|
||||||
**Updating the surface** — when a new `#[tool]` fn is added to `AgentServer`
|
**Updating the surface** — when a new `#[tool]` fn is added to `HiveServer`
|
||||||
in `hive-agent-mcp/src/mcp/mod.rs`, add its name to the matching `ToolGroup::tools()`
|
in `hive-ag3nt/src/mcp.rs`, add its name to the matching `ToolGroup::tools()`
|
||||||
slice in `hive-sh4re/src/permissions.rs`. That's the single source of truth;
|
slice in `hive-sh4re/src/lib.rs`. That's the single source of truth;
|
||||||
`mcp_config::allowed_mcp_tools` (in `hive-agent/src/mcp_config.rs`) reads it at
|
`mcp_config::allowed_mcp_tools` (in `hive-ag3nt/src/mcp_config.rs`) reads it at
|
||||||
session start.
|
session start.
|
||||||
|
|
||||||
## Capabilities
|
## Capabilities
|
||||||
|
|
@ -411,18 +406,14 @@ groups: an agent that could grant its own capabilities via a config commit would
|
||||||
bypass the operator approval gate.
|
bypass the operator approval gate.
|
||||||
|
|
||||||
**Adding a new capability** — add a variant to `Capability` in
|
**Adding a new capability** — add a variant to `Capability` in
|
||||||
`hive-sh4re/src/permissions.rs` + an arm to `as_str`. Add it to `Capability::ALL`
|
`hive-sh4re/src/lib.rs` + an arm to `as_str`. Add it to `Capability::ALL` (the
|
||||||
(the source of truth for the permissions UI columns). Implement the access
|
source of truth for the permissions UI columns). Implement the access check in
|
||||||
check in the relevant handler (`hive-c0re/src/socket_server/mod.rs`,
|
the relevant handler (`agent_server.rs`, `mcp.rs`, or `dashboard.rs`).
|
||||||
`hive-c0re/src/socket_server/lifecycle_handlers.rs`, `coordinator.rs`, or a
|
|
||||||
handler under `hive-c0re/src/dashboard/`).
|
|
||||||
|
|
||||||
## Async forms
|
## Async forms
|
||||||
|
|
||||||
Dashboard + per-agent mutating forms carry `data-async`; the shared
|
Dashboard + per-agent mutating forms carry `data-async`; a delegated
|
||||||
`bindAsyncForms` `submit` listener (`frontend/packages/shared/src/forms.js`,
|
`submit` listener in `assets/tabs.js` (+ `assets/app.js` for the per-agent UI) intercepts, shows a spinner,
|
||||||
imported as `@hive/shared/forms.js` and wired up from `tabs.js` on the
|
|
||||||
dashboard and `app.js` on the per-agent UI) intercepts, shows a spinner,
|
|
||||||
POSTs `application/x-www-form-urlencoded` (axum's `Form` extractor
|
POSTs `application/x-www-form-urlencoded` (axum's `Form` extractor
|
||||||
rejects multipart), calls `refreshState()` on success. New mutating
|
rejects multipart), calls `refreshState()` on success. New mutating
|
||||||
forms should add `data-async` and optionally `data-confirm` (for a
|
forms should add `data-async` and optionally `data-confirm` (for a
|
||||||
|
|
@ -437,16 +428,11 @@ via `snapshotOpenDetails` / `restoreOpenDetails`.
|
||||||
|
|
||||||
## `rebuild` is the reconcile verb
|
## `rebuild` is the reconcile verb
|
||||||
|
|
||||||
`job_queue::templates::rebuild` builds the DAG that reconciles a
|
`lifecycle::rebuild` idempotently rewrites
|
||||||
container to its wanted state: `write_dropins` (the nspawn-conf
|
`/etc/nixos-containers/<C>.conf` (`PRIVATE_NETWORK=0`, clears
|
||||||
rewrite — `PRIVATE_NETWORK=0`, clears `HOST_ADDRESS` / `LOCAL_ADDRESS`,
|
`HOST_ADDRESS` / `LOCAL_ADDRESS`, sets `EXTRA_NSPAWN_FLAGS`),
|
||||||
sets `EXTRA_NSPAWN_FLAGS` — plus the systemd resource-limits drop-in)
|
regenerates `applied/<name>/flake.nix`, writes the systemd limits
|
||||||
is folded into the `Swap` node, then `nixos-container update` + stop +
|
drop-in, then `nixos-container update` + stop + start.
|
||||||
start runs across the `StopForUpdate → Swap → RebuildBookkeeping`
|
|
||||||
brace and the tail `Reconcile` node. `flake.nix` itself is no longer
|
|
||||||
regenerated host-side on rebuild — it's tracked in the agent's
|
|
||||||
proposed/applied repos and rides along on every fetch (see
|
|
||||||
`docs/approvals.md::Two repos per agent`).
|
|
||||||
|
|
||||||
Anything that changes per-container state on the host should be
|
Anything that changes per-container state on the host should be
|
||||||
re-applied here so a manual `↻ R3BU1LD` from the dashboard is
|
re-applied here so a manual `↻ R3BU1LD` from the dashboard is
|
||||||
|
|
@ -455,7 +441,7 @@ sufficient to recover.
|
||||||
## Actions are factored
|
## Actions are factored
|
||||||
|
|
||||||
`approve` / `deny` / `destroy` (and the lifecycle helper) live in
|
`approve` / `deny` / `destroy` (and the lifecycle helper) live in
|
||||||
`actions.rs` / `hive-c0re/src/dashboard/`. The admin socket and the dashboard
|
`actions.rs` / `dashboard.rs`. The admin socket and the dashboard
|
||||||
POST handlers both call into them so the two surfaces never drift.
|
POST handlers both call into them so the two surfaces never drift.
|
||||||
|
|
||||||
## Commit messages
|
## Commit messages
|
||||||
|
|
|
||||||
|
|
@ -12,9 +12,9 @@ Every container/meta operation (rebuild, meta-update, first-spawn, power
|
||||||
changes) is submitted to the global job-DAG queue (`hive-c0re/src/job_queue/`)
|
changes) is submitted to the global job-DAG queue (`hive-c0re/src/job_queue/`)
|
||||||
as a **DAG of primitive nodes**. One scheduler task drives all DAGs;
|
as a **DAG of primitive nodes**. One scheduler task drives all DAGs;
|
||||||
concurrency comes from the resource classes below, not from multiple workers.
|
concurrency comes from the resource classes below, not from multiple workers.
|
||||||
Special cases like graceful stop, deferred starts, and the meta-update
|
The old special cases — the graceful-stop watcher thread, the deferred-start
|
||||||
cascade need no bespoke code paths — each is expressed as a DAG *shape*
|
fast-lane follow-up, the meta-update cascade pre-enqueue — are all just DAG
|
||||||
built from the same primitive nodes.
|
*shapes* now.
|
||||||
|
|
||||||
### Two levels: DAG and node
|
### Two levels: DAG and node
|
||||||
|
|
||||||
|
|
@ -30,7 +30,8 @@ A DAG is **declared, not described**: a template builds it through
|
||||||
`b.node(kind)` handed back, and the builder inserts the nodes itself. A handle
|
`b.node(kind)` handed back, and the builder inserts the nodes itself. A handle
|
||||||
only exists for a node already declared, so every edge points backwards and a
|
only exists for a node already declared, so every edge points backwards and a
|
||||||
cycle cannot be written down — there is no submit-time validation pass, because
|
cycle cannot be written down — there is no submit-time validation pass, because
|
||||||
there is no malformed spec to reject.
|
there is no malformed spec to reject. (The old queue had a petgraph `toposort`
|
||||||
|
here, guarding against the positional indices that used to express edges.)
|
||||||
|
|
||||||
### Node inventory (primitives)
|
### Node inventory (primitives)
|
||||||
|
|
||||||
|
|
@ -60,7 +61,7 @@ Cheap — no build slot:
|
||||||
| `Drain` | await the harness clearing the fence, bounded by the 3-min graceful-stop timeout; resolves ok either way |
|
| `Drain` | await the harness clearing the fence, bounded by the 3-min graceful-stop timeout; resolves ok either way |
|
||||||
| `WriteDropin` | `set_nspawn_flags` + `set_resource_limits` + daemon-reload |
|
| `WriteDropin` | `set_nspawn_flags` + `set_resource_limits` + daemon-reload |
|
||||||
| `WritePermFile` | commit `tool-groups.json` / `capabilities.json` (single git commit under `META_LOCK`) + emit the P3RM1SS10NS snapshots |
|
| `WritePermFile` | commit `tool-groups.json` / `capabilities.json` (single git commit under `META_LOCK`) + emit the P3RM1SS10NS snapshots |
|
||||||
| `Reparent` | `set-parent` / `set-parent-bulk`: apply every `(child, new_parent)` move under one `META_LOCK` commit (`meta::bulk_commit_topology`), send the per-agent move notifications, rescan + diff-emit. Agentless like `MetaLock` — a bulk move can span multiple agents, and a reparent touches the meta repo, not any one container. `moves` is typed `(Ident, Option<Ident>)` pairs, not raw strings. Rides the existing `Template::MetaUpdate` variant rather than a dedicated one — it's internal-only (never reaches the graph wire), so the stand-in only affects `terminal_hook` dispatch (resolves to no hook either way) and history-retention bucketing |
|
| `Reparent` | `set-parent` / `set-parent-bulk`: apply every `(child, new_parent)` move under one `META_LOCK` commit (`meta::bulk_commit_topology`), send the per-agent move notifications, rescan + diff-emit. Agentless like `MetaLock` — a bulk move can span multiple agents, and a reparent touches the meta repo, not any one container. `moves` is `(Ident, Option<Ident>)` pairs, not raw strings — mara: "use Ident type instead of string" (#2719, issuecomment 42691). Rides `Template::MetaUpdate` rather than a dedicated `Template` variant — that enum is on its way out (see `#2665`, still open/blocked on a scope question) and is already internal-only (it never reaches the graph wire), so the stand-in only affects `terminal_hook` dispatch (resolves to no hook either way) and history-retention bucketing |
|
||||||
|
|
||||||
There is deliberately **no `GitCommit` node**: `meta.rs` fuses each mutation
|
There is deliberately **no `GitCommit` node**: `meta.rs` fuses each mutation
|
||||||
with its commit under its internal `META_LOCK` mutex, so a standalone commit
|
with its commit under its internal `META_LOCK` mutex, so a standalone commit
|
||||||
|
|
@ -174,7 +175,7 @@ without a row are seeded from observed state on first touch (running ⇒
|
||||||
`Up`); destroy removes the row.
|
`Up`); destroy removes the row.
|
||||||
|
|
||||||
The admin-socket responses carry the submitted DAG ids; `hivectl` polls
|
The admin-socket responses carry the submitted DAG ids; `hivectl` polls
|
||||||
`HostRequest::QueueNodes` (~1s) and prints a progress line per DAG — roll-up
|
`HostRequest::QueueDag` (~1s) and prints a progress line per DAG — roll-up
|
||||||
glyph, template, agent, node chain — so
|
glyph, template, agent, node chain — so
|
||||||
CLI verbs block until their jobs finish (`--no-wait` opts out; failures exit
|
CLI verbs block until their jobs finish (`--no-wait` opts out; failures exit
|
||||||
non-zero). Nodes appended in-DAG (a `MetaLock` growing per-agent rebuild
|
non-zero). Nodes appended in-DAG (a `MetaLock` growing per-agent rebuild
|
||||||
|
|
@ -259,9 +260,8 @@ whether any dedup needs reintroducing is tracked as a follow-up.
|
||||||
Cancel only applies to still-fully-queued DAGs (an in-flight nix build isn't
|
Cancel only applies to still-fully-queued DAGs (an in-flight nix build isn't
|
||||||
interruptible) — each op is one DAG now, so there are no child DAGs to cascade to.
|
interruptible) — each op is one DAG now, so there are no child DAGs to cascade to.
|
||||||
Roll-up state: `Failed` if any node failed, else `Running` / `Queued` /
|
Roll-up state: `Failed` if any node failed, else `Running` / `Queued` /
|
||||||
`Cancelled` / `Done`. The snapshot retains the 50 most recent terminal
|
`Cancelled` / `Done`. The snapshot retains the 5 most recent terminal DAGs
|
||||||
DAGs — a flat cap over the whole sorted list, not per template, since
|
per template.
|
||||||
the dashboard renders one recent-builds list and one number bounds it.
|
|
||||||
|
|
||||||
### Approvals
|
### Approvals
|
||||||
|
|
||||||
|
|
@ -308,11 +308,10 @@ span agents; consumers derive a group's agent(s) from its nodes. Build logs
|
||||||
are likewise **per-node**: the dashboard renders the node tree and keys the
|
are likewise **per-node**: the dashboard renders the node tree and keys the
|
||||||
live-log panel off the running node.
|
live-log panel off the running node.
|
||||||
|
|
||||||
The event carries no payload by design: shipping a typed projection of
|
The event used to ship the whole queue as a typed `DagView`/`NodeView`
|
||||||
the whole queue in the event itself would be a second rendering of the
|
projection. That was a second rendering of the same graph, kept in
|
||||||
same graph that has to be kept in agreement by hand with the endpoint
|
agreement by hand with the endpoint every consumer actually read; it is
|
||||||
every consumer actually reads. Telling a client *when* to refetch is
|
gone, and the event's whole job is now telling a client *when* to refetch.
|
||||||
the event's whole job.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
@ -412,7 +411,8 @@ Sequence for a rebuild DAG (each step is its own queue node):
|
||||||
(near-instant after the prebuild).
|
(near-instant after the prebuild).
|
||||||
5. `Reconcile` — boot into the new generation when `wanted = Up`; the
|
5. `Reconcile` — boot into the new generation when `wanted = Up`; the
|
||||||
in-container activation script transitions old → new. Holds no build
|
in-container activation script transitions old → new. Holds no build
|
||||||
slot, so the next DAG's `Prebuild` overlaps the container boot.
|
slot, so the next DAG's `Prebuild` overlaps the container boot — the old
|
||||||
|
"deferred start" split, now structural.
|
||||||
|
|
||||||
The approval deploy uses this same chain rather than a rebuild path of its
|
The approval deploy uses this same chain rather than a rebuild path of its
|
||||||
own. Its `DeployApply` node does not build: it merges, opens the two-phase
|
own. Its `DeployApply` node does not build: it merges, opens the two-phase
|
||||||
|
|
@ -501,8 +501,8 @@ Two things to know about the weights:
|
||||||
same value and the weight does *not* rank agents against each other.
|
same value and the weight does *not* rank agents against each other.
|
||||||
What `80` buys is that agents yield to everything **not** on this
|
What `80` buys is that agents yield to everything **not** on this
|
||||||
drop-in path: host services (nginx and dnsmasq among them) and the
|
drop-in path: host services (nginx and dnsmasq among them) and the
|
||||||
infra containers (`hive-ci`, `hive-forge`, `hive-gateway`,
|
infra containers (`hive-ci`, `hive-forge`, `hive-matrix`), which stay
|
||||||
`hive-matrix`), which stay at the kernel default of `100`.
|
at the kernel default of `100`.
|
||||||
- `IOWeight=` is only honoured when the backing device runs the BFQ
|
- `IOWeight=` is only honoured when the backing device runs the BFQ
|
||||||
scheduler or has blk-iocost QoS enabled. On a host using
|
scheduler or has blk-iocost QoS enabled. On a host using
|
||||||
`none`/`mq-deadline`/`kyber` without iocost, systemd writes the value
|
`none`/`mq-deadline`/`kyber` without iocost, systemd writes the value
|
||||||
|
|
|
||||||
|
|
@ -2,7 +2,7 @@
|
||||||
|
|
||||||
Private Forgejo instance running in a nixos-container, used as the
|
Private Forgejo instance running in a nixos-container, used as the
|
||||||
swarm's persistent code-collaboration surface (issues, PRs, reviews,
|
swarm's persistent code-collaboration surface (issues, PRs, reviews,
|
||||||
attachments). Configured via `services.hyperhive.swarm.forge.*`. Container
|
attachments). Configured via `services.hyperhive.forge.*`. Container
|
||||||
shape, ROOT_URL / sub-domain routing, and operator-vs-in-cluster URL
|
shape, ROOT_URL / sub-domain routing, and operator-vs-in-cluster URL
|
||||||
handling live in [`docs/gateway.md`](gateway.md); this file owns the
|
handling live in [`docs/gateway.md`](gateway.md); this file owns the
|
||||||
per-agent integration story and the notification pump that wakes
|
per-agent integration story and the notification pump that wakes
|
||||||
|
|
@ -51,12 +51,9 @@ read it without touching c0re's host-side credential store.
|
||||||
|
|
||||||
Two things live in the `agent-configs` Forgejo organization:
|
Two things live in the `agent-configs` Forgejo organization:
|
||||||
|
|
||||||
- A config repo per agent (`agent-configs/<name>`). The
|
- A config repo per agent (`agent-configs/<name>`). As of #1787 the
|
||||||
agent is a **write collaborator on its own** repo — it can push
|
agent is a **write collaborator on its own** repo — it can push
|
||||||
config-change branches and open config PRs (Forgejo `pull_request`
|
config-change branches and (once #1838 P2 lands) open config PRs — but
|
||||||
webhook at `/webhook/config-pr` queues a `MergeConfigPr` approval;
|
|
||||||
`hive-c0re/src/forge/config_pr_poll.rs` re-scans every 5 minutes as a
|
|
||||||
fault-tolerance backstop) — but
|
|
||||||
`main` is branch-protected core-only: only hive-c0re's verify-and-ff-push
|
`main` is branch-protected core-only: only hive-c0re's verify-and-ff-push
|
||||||
merge handler lands on `main`, an operator-team approval is required, and
|
merge handler lands on `main`, an operator-team approval is required, and
|
||||||
the agent can neither push `main` directly nor self-merge. `main` is
|
the agent can neither push `main` directly nor self-merge. `main` is
|
||||||
|
|
@ -121,20 +118,19 @@ That size property is the whole point. A container rebuild starts the
|
||||||
poller with no memory of what it delivered, re-scans `?all=false`, and
|
poller with no memory of what it delivered, re-scans `?all=false`, and
|
||||||
finds nothing stale — the delivered threads are already read on forge.
|
finds nothing stale — the delivered threads are already read on forge.
|
||||||
Forge's own read-state is thus the durable, cross-rebuild record of
|
Forge's own read-state is thus the durable, cross-rebuild record of
|
||||||
what's been delivered; there is **no persisted cursor**.
|
what's been delivered; there is **no persisted cursor**. (This
|
||||||
|
replaced an earlier design that left threads unread and leaned on a
|
||||||
|
persisted dedup cursor: a rebuild that lost the cursor re-delivered the
|
||||||
|
entire still-unread backlog as fresh wakes — the notification flood of
|
||||||
|
#2593 / #2106.)
|
||||||
|
|
||||||
**Gotcha:** don't reintroduce a persisted dedup cursor here. A design
|
**Read-before-comment coupling, dropped on purpose.** The old design
|
||||||
that leaves threads unread and tracks delivery via a separately-persisted
|
left threads unread so the hive-forge read-before-comment guard (which
|
||||||
cursor is fragile — losing that cursor across a rebuild re-delivers the
|
keys off forge unread-state) would force the agent to view a thread
|
||||||
agent's entire still-unread backlog as a flood of fresh wakes. Forge's
|
before commenting. That coupling is gone: the broker wake already
|
||||||
own read-state is the only durable record this design needs.
|
carries the notification body, so *delivery is the read*. An agent that
|
||||||
|
wants the full thread still runs `hive-forge comments` / `view`; the
|
||||||
**Read-before-comment guard doesn't block a fresh wake.** hive-forge's
|
guard no longer blocks a first comment on a freshly-delivered thread.
|
||||||
read-before-comment guard (which keys off forge unread-state) does not
|
|
||||||
force the agent to view a thread before commenting on it: the broker
|
|
||||||
wake already carries the notification body, so *delivery is the read*.
|
|
||||||
An agent that wants the full thread still runs `hive-forge comments` /
|
|
||||||
`view`.
|
|
||||||
|
|
||||||
**In-process dedupe (tiny, ephemeral).** A single-process map (thread
|
**In-process dedupe (tiny, ephemeral).** A single-process map (thread
|
||||||
id → last-delivered `updated_at`) guards the narrow window where a
|
id → last-delivered `updated_at`) guards the narrow window where a
|
||||||
|
|
@ -248,8 +244,8 @@ A notification carrying a `latest_comment_url` normally takes the comment
|
||||||
path. But a merged/closed subject **keeps** its `latest_comment_url` set,
|
path. But a merged/closed subject **keeps** its `latest_comment_url` set,
|
||||||
so a just-merged PR that had any prior discussion would route to the
|
so a just-merged PR that had any prior discussion would route to the
|
||||||
comment path and render `[comment on PR]` (with a stale pre-merge comment
|
comment path and render `[comment on PR]` (with a stale pre-merge comment
|
||||||
body) instead of `[PR merged]` — the agent never learns its PR merged.
|
body) instead of `[PR merged]` — the agent never learns its PR merged
|
||||||
So when the notification IS the merge/close transition — its
|
(#2495). So when the notification IS the merge/close transition — its
|
||||||
event time (`updated_at`) is within `NEW_ITEM_TOLERANCE_SECS` of the
|
event time (`updated_at`) is within `NEW_ITEM_TOLERANCE_SECS` of the
|
||||||
subject's `closed_at` (set for both `merged` and `closed`) — the
|
subject's `closed_at` (set for both `merged` and `closed`) — the
|
||||||
state-change path wins even with a comment url present
|
state-change path wins even with a comment url present
|
||||||
|
|
@ -284,7 +280,7 @@ A review submitted with **no body** carries no `latest_comment_url`,
|
||||||
so it misses the comment path and lands on the state-change path with
|
so it misses the comment path and lands on the state-change path with
|
||||||
`state == "open"` — exactly like a freshly opened PR. Labeling that
|
`state == "open"` — exactly like a freshly opened PR. Labeling that
|
||||||
`new PR` is misleading: agents dismiss it as a duplicate of the
|
`new PR` is misleading: agents dismiss it as a duplicate of the
|
||||||
original open notification and miss the review. So the `open`
|
original open notification and miss the review (#1637). So the `open`
|
||||||
state only earns the `new <kind>` label when the notification's event
|
state only earns the `new <kind>` label when the notification's event
|
||||||
time (`updated_at`) is within `NEW_ITEM_TOLERANCE_SECS` (120s) of the
|
time (`updated_at`) is within `NEW_ITEM_TOLERANCE_SECS` (120s) of the
|
||||||
subject's `created_at`. Anything later is labeled `activity on <kind>`
|
subject's `created_at`. Anything later is labeled `activity on <kind>`
|
||||||
|
|
@ -292,7 +288,7 @@ subject's `created_at`. Anything later is labeled `activity on <kind>`
|
||||||
activity was without an extra reviews fetch. Missing/unparseable
|
activity was without an extra reviews fetch. Missing/unparseable
|
||||||
timestamps default to `new` (preserve prior behavior rather than mask a
|
timestamps default to `new` (preserve prior behavior rather than mask a
|
||||||
genuine new item). Timestamps are parsed by a small dependency-free
|
genuine new item). Timestamps are parsed by a small dependency-free
|
||||||
RFC 3339 helper (`parse_rfc3339`).
|
RFC 3339 → epoch-seconds helper (`parse_rfc3339_secs`).
|
||||||
|
|
||||||
Number is extracted from `subject.html_url`'s last path segment
|
Number is extracted from `subject.html_url`'s last path segment
|
||||||
(strips `#anchor` first); repo slug from `repository.full_name`.
|
(strips `#anchor` first); repo slug from `repository.full_name`.
|
||||||
|
|
|
||||||
114
docs/gateway.md
114
docs/gateway.md
|
|
@ -1,6 +1,6 @@
|
||||||
# hive-gateway
|
# hive-gateway
|
||||||
|
|
||||||
Single nginx in front of every hyperhive web surface. Runs on the **host**, next to hive-c0re, rather than in its own container: it shares the host netns anyway (see [Vhost map](#vhost-map) below), so containerizing it would buy no network isolation while costing a resolv.conf sync, a machine-bus reload, and three bind mounts. System-config (not meta-flake managed). Configured via `services.hyperhive.gateway.*` + per-subsystem opt-in flags in `services.hyperhive.{forge,matrix,...}`.
|
Single nginx in front of every hyperhive web surface. Runs on the **host**, next to hive-c0re; system-config (not meta-flake managed). Configured via `services.hyperhive.gateway.*` + per-subsystem opt-in flags in `services.hyperhive.{forge,matrix,...}`. (It lived in a `hive-gateway` container until #3088 — one that shared the host netns anyway, so the boundary gave no network isolation while costing a resolv.conf sync, a machine-bus reload and three bind mounts.)
|
||||||
|
|
||||||
## Vhost map
|
## Vhost map
|
||||||
|
|
||||||
|
|
@ -9,11 +9,11 @@ Single nginx in front of every hyperhive web surface. Runs on the **host**, next
|
||||||
| `<hive>/` | `_` (catch-all) | dashboard dist (static, from `servedFrontend`); `/api/` + `/webhook/` → hive-c0re (`7000`) | always |
|
| `<hive>/` | `_` (catch-all) | dashboard dist (static, from `servedFrontend`); `/api/` + `/webhook/` → hive-c0re (`7000`) | always |
|
||||||
| `<hive>/agent/<name>/` | `_` | per-agent harness (UDS or TCP) | `agents.conf` (runtime-generated) |
|
| `<hive>/agent/<name>/` | `_` | per-agent harness (UDS or TCP) | `agents.conf` (runtime-generated) |
|
||||||
| `<hive>/.well-known/matrix/{client,server}` | `_` | inline JSON (no upstream) | `matrix.enable && domain != null` |
|
| `<hive>/.well-known/matrix/{client,server}` | `_` | inline JSON (no upstream) | `matrix.enable && domain != null` |
|
||||||
| `<hive>/matrix/` (deprecated) | `_` | 301 → `chat.<swarm>/` | `matrix.gui.enable` |
|
| `<hive>/matrix/` (deprecated) | `_` | 301 → `matrix.<hive>/` | `matrix.gui.enable` |
|
||||||
| `forge.<swarm>/` | `forge.<swarm>` | forgejo (`3000`) | `forge.behindGateway` |
|
| `forge.<hive>/` | `forge.<hive>` | forgejo (`3000`) | `forge.behindGateway` |
|
||||||
| `chat.<swarm>/_matrix/*` | `chat.<swarm>` | tuwunel (`8008`) | `matrix.gatewayHost != null` |
|
| `matrix.<hive>/_matrix/*` | `matrix.<hive>` | tuwunel (`8008`) | `matrix.gatewayHost != null` |
|
||||||
| `chat.<swarm>/` | `chat.<swarm>` | fluffychat-web static | `matrix.gui.enable` |
|
| `matrix.<hive>/` | `matrix.<hive>` | fluffychat-web static | `matrix.gui.enable` |
|
||||||
| `chat.<swarm>/config.json` | `chat.<swarm>` | inline JSON (FluffyChat boot config) | `matrix.gui.enable && domain != null` |
|
| `matrix.<hive>/config.json` | `matrix.<hive>` | inline JSON (FluffyChat boot config) | `matrix.gui.enable && domain != null` |
|
||||||
| `auth.<swarm>/` | `auth.<swarm>` | authelia (`9091`) | `swarm.authelia.enable` |
|
| `auth.<swarm>/` | `auth.<swarm>` | authelia (`9091`) | `swarm.authelia.enable` |
|
||||||
| `<swarm>/` | `<swarm>` | swarm-ui dist (static), behind an authelia subrequest | `swarm.ui.enable` |
|
| `<swarm>/` | `<swarm>` | swarm-ui dist (static), behind an authelia subrequest | `swarm.ui.enable` |
|
||||||
|
|
||||||
|
|
@ -21,24 +21,23 @@ The authelia vhost is declared only by the host that **runs** authelia, not by e
|
||||||
|
|
||||||
⚠️ **A `502` from this vhost usually means authelia has no users yet, not that the proxy is misconfigured.** Authelia treats an empty user store as a fatal startup error, so an enabled-but-unbootstrapped swarm crash-loops the container while the vhost in front of it works perfectly. Check `journalctl -M swarm-authelia -u authelia-swarm` before suspecting anything here; the bootstrap step is in [`swarm/sso.md`](swarm/sso.md).
|
⚠️ **A `502` from this vhost usually means authelia has no users yet, not that the proxy is misconfigured.** Authelia treats an empty user store as a fatal startup error, so an enabled-but-unbootstrapped swarm crash-loops the container while the vhost in front of it works perfectly. Check `journalctl -M swarm-authelia -u authelia-swarm` before suspecting anything here; the bootstrap step is in [`swarm/sso.md`](swarm/sso.md).
|
||||||
|
|
||||||
Per-agent UIs stay sub-path, forge and matrix get sub-domains — see
|
Per-agent UIs stay sub-path because they're hyperhive-internal and base-path-aware. External standard apps (forge / matrix) get sub-domains because their defaults work cleanly at sub-domain root + per-origin cookies / storage isolation matters.
|
||||||
[Sub-domain shape (rationale)](#sub-domain-shape-rationale) below for why.
|
|
||||||
|
|
||||||
## Discovery flow (matrix)
|
## Discovery flow (matrix)
|
||||||
|
|
||||||
Operator points client at `<hive>`. Sequence:
|
Operator points client at `<hive>`. Sequence:
|
||||||
|
|
||||||
1. Client fetches `https://<hive>/.well-known/matrix/client` → `{"m.homeserver":{"base_url":"https://chat.<swarm>"}}` (no port suffix when gateway listens on 443). The gateway always terminates TLS, so the scheme is always `https`; a non-default `httpsPort` is reflected as the port suffix.
|
1. Client fetches `https://<hive>/.well-known/matrix/client` → `{"m.homeserver":{"base_url":"https://matrix.<hive>"}}` (no port suffix when gateway listens on 443). The gateway always terminates TLS, so the scheme is always `https`; a non-default `httpsPort` is reflected as the port suffix.
|
||||||
2. Client connects to `chat.<swarm>/_matrix/client/...`.
|
2. Client connects to `matrix.<hive>/_matrix/client/...`.
|
||||||
3. Gateway routes `/_matrix/*` → tuwunel at `127.0.0.1:8008`.
|
3. Gateway routes `/_matrix/*` → tuwunel at `127.0.0.1:8008`.
|
||||||
|
|
||||||
matrix-dart-sdk (FluffyChat etc.) hardcodes `https` for the well-known fetch regardless of input scheme, so the discovery endpoint MUST be https — see "Self-signed TLS" below for the cert generation that backs the default-on path.
|
matrix-dart-sdk (FluffyChat etc.) hardcodes `https` for the well-known fetch regardless of input scheme, so the discovery endpoint MUST be https — see "Self-signed TLS" below for the cert generation that backs the default-on path.
|
||||||
|
|
||||||
Federation peers fetch `.well-known/matrix/server` → `{"m.server":"chat.<swarm>:<httpsPort>"}` (the federation delegation always carries an explicit port, even the HTTPS default 443 — the https-implies-443 elision only applies to the client base_url above). Gateway only listens on configured `port` (+ `httpsPort` when TLS on); cross-hive federation needs either an SRV record (`_matrix._tcp.chat.<swarm>` → port 80 / 443) OR `matrix.openFirewall = true` so peers reach tuwunel's federation port directly. Hyperhive is mostly closed/internal, so this rarely bites.
|
Federation peers fetch `.well-known/matrix/server` → `{"m.server":"matrix.<hive>"}` and connect to `matrix.<hive>:8448` per spec default. Gateway only listens on configured `port` (+ `httpsPort` when TLS on); cross-hive federation needs either an SRV record (`_matrix._tcp.matrix.<hive>` → port 80 / 443) OR `matrix.openFirewall = true` so peers reach tuwunel's federation port directly. Hyperhive is mostly closed/internal, so this rarely bites.
|
||||||
|
|
||||||
## SPA fallback (Accept-header pattern)
|
## SPA fallback (Accept-header pattern)
|
||||||
|
|
||||||
The per-agent UIs and the `chat.<swarm>` vhost serve a flutter/SPA bundle via the Accept-header pattern below. The dashboard vhost instead routes by **path** — see [Dashboard: path-based routing](#dashboard-path-based-routing-not-accept-header) below. Two requirements collide:
|
The per-agent UIs and the `matrix.<hive>` vhost serve a flutter/SPA bundle via the Accept-header pattern below. (The `<hive>` dashboard catch-all used this too but now routes by **path** — see the dashboard note after.) Two requirements collide:
|
||||||
|
|
||||||
- hard-refresh on a sub-route must serve `index.html` (SPA's client-side router takes over after JS bootstrap)
|
- hard-refresh on a sub-route must serve `index.html` (SPA's client-side router takes over after JS bootstrap)
|
||||||
- a non-navigation request that isn't an on-disk asset must NOT get HTML with the wrong content-type
|
- a non-navigation request that isn't an on-disk asset must NOT get HTML with the wrong content-type
|
||||||
|
|
@ -49,21 +48,21 @@ For matrix / per-agent static assets, `<final>` is `=404` (a missing asset is ju
|
||||||
|
|
||||||
### Dashboard: path-based routing (not Accept-header)
|
### Dashboard: path-based routing (not Accept-header)
|
||||||
|
|
||||||
Every hive-c0re backend route lives under `/api/` plus the single `/webhook/knowledge` endpoint, so the dashboard vhost routes by **path**, not Accept header — deterministic, unlike a content-type split where the same URL could resolve differently depending on the caller's `Accept` header:
|
Now that every hive-c0re backend route lives under `/api/` plus the single `/webhook/knowledge` endpoint, the dashboard vhost routes by **path**, not Accept header:
|
||||||
|
|
||||||
- `location /api/` → hive-c0re (`7000`): all dashboard data, actions/mutations, and the two SSE streams (`/api/dashboard/stream`, `/api/build-logs/id/{id}/stream`). Carries `proxy_buffering off` + a 1d read timeout for the streams.
|
- `location /api/` → hive-c0re (`7000`): all dashboard data, actions/mutations, and the two SSE streams (`/api/dashboard/stream`, `/api/build-logs/id/{id}/stream`). Carries `proxy_buffering off` + a 1d read timeout for the streams.
|
||||||
- `location /webhook/` → hive-c0re: the knowledge webhook.
|
- `location /webhook/` → hive-c0re: the knowledge webhook.
|
||||||
- `location /` → the dashboard dist (from the `servedFrontend` nix-store path) with `try_files $uri /index.html` (SPA fallback).
|
- `location /` → the dashboard dist (from the `servedFrontend` nix-store path) with `try_files $uri /index.html` (SPA fallback).
|
||||||
|
|
||||||
Each location carries a duplicated `auth_basic` block (separate locations don't inherit it). This keeps the gateway static-serving the dashboard dist while hive-c0re stays API-only — a frontend-only change doesn't rebuild or restart the core daemon. A new top-level c0re route prefix (beyond `/api` + `/webhook`) needs a matching `location` added to the dashboard vhost.
|
Each location carries a duplicated `auth_basic` block (separate locations don't inherit it). This keeps the gateway static-serving the dashboard dist while hive-c0re stays API-only — a frontend-only change no longer rebuilds + restarts the core daemon. The earlier `map $http_accept` Accept-header split was replaced because it made the *same* URL behave differently by content-type (e.g. `/api/state` fetched with `Accept: text/html` wrongly returned `index.html`); path routing is deterministic. A new top-level c0re route prefix (beyond `/api` + `/webhook`) needs a matching `location` added to the dashboard vhost.
|
||||||
|
|
||||||
## Local dev (`localHostsEntry`)
|
## Local dev (`localHostsEntry`)
|
||||||
|
|
||||||
`services.hyperhive.gateway.localHostsEntry = true` adds entries to the host's `/etc/hosts`:
|
`services.hyperhive.gateway.localHostsEntry = true` adds entries to the host's `/etc/hosts`:
|
||||||
|
|
||||||
- `<hive-domain>` → `127.0.0.1`
|
- `<hive-domain>` → `127.0.0.1`
|
||||||
- `forge.<swarm>` → `127.0.0.1` (when forge.behindGateway)
|
- `forge.<hive>` → `127.0.0.1` (when forge.behindGateway)
|
||||||
- `chat.<swarm>` → `127.0.0.1` (when matrix.gatewayHost set)
|
- `matrix.<hive>` → `127.0.0.1` (when matrix.gatewayHost set)
|
||||||
- `auth.<swarm>` → `127.0.0.1` (when swarm.authelia.enable)
|
- `auth.<swarm>` → `127.0.0.1` (when swarm.authelia.enable)
|
||||||
|
|
||||||
`lib.unique` de-dupes if any sub-domain happens to equal another entry. Operators with real DNS leave it off.
|
`lib.unique` de-dupes if any sub-domain happens to equal another entry. Operators with real DNS leave it off.
|
||||||
|
|
@ -150,11 +149,8 @@ When the gateway is in front, the SW4RM tab builds per-agent links
|
||||||
as same-origin `/agent/<name>/…` URLs instead of the legacy direct
|
as same-origin `/agent/<name>/…` URLs instead of the legacy direct
|
||||||
`http://<host>:<container.port>/` TCP shape. The signal comes from
|
`http://<host>:<container.port>/` TCP shape. The signal comes from
|
||||||
`StateSnapshot.gateway_enabled`, sourced from the
|
`StateSnapshot.gateway_enabled`, sourced from the
|
||||||
`HIVE_GATEWAY_ENABLED` env the c0re NixOS module now always sets
|
`HIVE_GATEWAY_ENABLED` env the c0re NixOS module sets when
|
||||||
(`services.hyperhive.gateway.enable` was removed — the gateway runs
|
`services.hyperhive.gateway.enable = true`. Three render sites
|
||||||
unconditionally alongside hyperhive), so this is effectively always
|
|
||||||
true; the `false` branch is retained as a defensive fallback for the
|
|
||||||
env being unset. Three render sites
|
|
||||||
flip together: the primary agent-name link, the favicon fetch
|
flip together: the primary agent-name link, the favicon fetch
|
||||||
(`<url>/icon`), and the nav-strip `container`-kind links from
|
(`<url>/icon`), and the nav-strip `container`-kind links from
|
||||||
`DashboardState.links` (`GET /api/dashboard-state`). `forge`-kind nav-strip links still
|
`DashboardState.links` (`GET /api/dashboard-state`). `forge`-kind nav-strip links still
|
||||||
|
|
@ -175,10 +171,10 @@ selected by which (if any) external TLS source is set:
|
||||||
| ACME (Let's Encrypt) | `tls.acme.enable = true` | nginx via HTTP-01 | `https` |
|
| ACME (Let's Encrypt) | `tls.acme.enable = true` | nginx via HTTP-01 | `https` |
|
||||||
| operator cert | `tls.certDir` set | read from the operator's dir | `https` |
|
| operator cert | `tls.certDir` set | read from the operator's dir | `https` |
|
||||||
|
|
||||||
The `gateway.selfSignedTls` option has been **removed** — self-signed
|
The `gateway.selfSignedTls` option is **deprecated and ignored** — self-signed
|
||||||
is now derived from the absence of `tls.certDir` / `tls.acme`. A config
|
is now derived from the absence of `tls.certDir` / `tls.acme`. Setting it to
|
||||||
that still sets it fails eval with a removal message; use `tls.certDir`
|
`false` (which used to select http-only or force an external cert) warns and
|
||||||
/ `tls.acme` to override the default.
|
has no effect; use `tls.certDir` / `tls.acme` to override the default.
|
||||||
|
|
||||||
### ACME / Let's Encrypt (`tls.acme`)
|
### ACME / Let's Encrypt (`tls.acme`)
|
||||||
|
|
||||||
|
|
@ -208,13 +204,7 @@ On by default, and listens on `httpsPort` (default 443) on every vhost beside th
|
||||||
|
|
||||||
The issuer is a **host-held hive CA**, not a bare self-signed leaf. A host service (`hive-tls-ca.service`, from the `hive-tls` module) generates a long-lived CA (`services.hyperhive.tls.caValidityDays`, default ~20y) under `services.hyperhive.tls.stateDir` (default `/var/lib/hive-tls`), then signs a gateway **leaf** (`leafValidityDays`, default 30d) with it. `hive-gateway-self-signed-cert` then imports the leaf into nginx's state dir (`/var/lib/hive-gateway/tls/{cert,key}.pem`).
|
The issuer is a **host-held hive CA**, not a bare self-signed leaf. A host service (`hive-tls-ca.service`, from the `hive-tls` module) generates a long-lived CA (`services.hyperhive.tls.caValidityDays`, default ~20y) under `services.hyperhive.tls.stateDir` (default `/var/lib/hive-tls`), then signs a gateway **leaf** (`leafValidityDays`, default 30d) with it. `hive-gateway-self-signed-cert` then imports the leaf into nginx's state dir (`/var/lib/hive-gateway/tls/{cert,key}.pem`).
|
||||||
|
|
||||||
⚠️ **Do not collapse that import unit into pointing nginx at the CA dir.**
|
⚠️ **Do not collapse that import unit into pointing nginx at the CA dir.** It does two jobs. It re-modes the leaf (`hive-tls-ca` writes the key `0600 root:root`; nginx's pre-start `nginx -t` runs as the *nginx user*, so a `0600` key fails the config test and blocks the unit), and it guarantees **every cert path the nginx config names exists** — which is what the swarm-services fallback below is for. Removing it re-creates the #3097 outage.
|
||||||
It does two jobs, and skipping it has taken the gateway down in production
|
|
||||||
before. It re-modes the leaf (`hive-tls-ca` writes the key `0600
|
|
||||||
root:root`; nginx's pre-start `nginx -t` runs as the *nginx user*, so a
|
|
||||||
`0600` key fails the config test and blocks the unit), and it guarantees
|
|
||||||
**every cert path the nginx config names exists** — which is what the
|
|
||||||
swarm-services fallback below is for.
|
|
||||||
|
|
||||||
**Why a CA, not a bare leaf**: a bare self-signed leaf is its own trust anchor, so every regeneration is a new anchor every consumer must re-trust — and a runtime-generated leaf can't be wired into an agent's build-time trust store at all. With a stable CA, agents and federation peers trust it *once*; leaf rotation never re-breaks them.
|
**Why a CA, not a bare leaf**: a bare self-signed leaf is its own trust anchor, so every regeneration is a new anchor every consumer must re-trust — and a runtime-generated leaf can't be wired into an agent's build-time trust store at all. With a stable CA, agents and federation peers trust it *once*; leaf rotation never re-breaks them.
|
||||||
|
|
||||||
|
|
@ -260,8 +250,8 @@ services.hyperhive.swarm.hives.example = { domain = "example.com"; }; # no cert
|
||||||
|
|
||||||
### Fronting with an external TLS terminator
|
### Fronting with an external TLS terminator
|
||||||
|
|
||||||
There is no http-only mode (see [TLS modes](#tls-modes) above). Two paths
|
There is no http-only mode: the gateway always terminates TLS (self-signed
|
||||||
for an operator who wants their own TLS terminator:
|
floor). Two paths for an operator who wants their own TLS terminator:
|
||||||
|
|
||||||
- give the gateway the real cert via `tls.certDir` (or `tls.acme`) so it
|
- give the gateway the real cert via `tls.certDir` (or `tls.acme`) so it
|
||||||
serves proper TLS directly — no separate proxy needed; or
|
serves proper TLS directly — no separate proxy needed; or
|
||||||
|
|
@ -269,26 +259,25 @@ for an operator who wants their own TLS terminator:
|
||||||
intended direction for "bring your own proxy" — the gateway is not meant
|
intended direction for "bring your own proxy" — the gateway is not meant
|
||||||
to expose an unencrypted TCP upstream).
|
to expose an unencrypted TCP upstream).
|
||||||
|
|
||||||
Because of this, `.well-known/matrix/{client,server}` discovery responses
|
**`.well-known/matrix/{client,server}` scheme** is always `https` now — the
|
||||||
always advertise `https` (see [Discovery flow](#discovery-flow-matrix) above).
|
gateway always terminates TLS, so discovery responses always advertise https.
|
||||||
|
|
||||||
## Firewall posture (host-level)
|
## Firewall posture (host-level)
|
||||||
|
|
||||||
The gateway is unconditional — `services.hyperhive.gateway.enable` was
|
`hive-c0re.nix` opens the per-agent web-port range
|
||||||
removed, there is no gateway-off mode. nginx is always the sole
|
`8100..8999` in the host firewall **only when
|
||||||
external entry point and routes to agents over the UDS upstream
|
`services.hyperhive.gateway.enable = false`**. With the gateway on
|
||||||
described above (see [Per-agent unix-socket
|
(default) it's the sole external entry point and routes to agents over
|
||||||
upstream](#per-agent-unix-socket-upstream)), so the per-agent web-port
|
the UDS upstream described above (see [Per-agent unix-socket
|
||||||
range `8100..8999` stays closed on the host firewall
|
upstream](#per-agent-unix-socket-upstream)) — leaving the per-agent
|
||||||
unconditionally — opening it would defeat the single-front-door story.
|
ports firewall-open would defeat the single-front-door story. The
|
||||||
The hashed TCP port (`lifecycle::agent_web_port`) still exists as a
|
hashed TCP port (`lifecycle::agent_web_port`) still exists as a direct
|
||||||
fallback bind for an agent whose `HIVE_WEB_SOCKET` env somehow ends up
|
host-loopback fallback for the pre-UDS/gateway-disabled case, but isn't
|
||||||
unset, but nothing opens a matching firewall hole for it and the
|
what the gateway itself proxies through.
|
||||||
gateway itself never proxies through it.
|
|
||||||
|
|
||||||
`services.hyperhive.gateway.openFirewall = true` opens both `port` and
|
`services.hyperhive.gateway.openFirewall = true` opens both `port` and
|
||||||
`httpsPort` — both are always served, since the gateway always terminates
|
`httpsPort` — the gateway always terminates TLS (self-signed floor), so
|
||||||
TLS (see [TLS modes](#tls-modes) above).
|
both are always served.
|
||||||
|
|
||||||
Every agent hashes into the same port range (no special case), so
|
Every agent hashes into the same port range (no special case), so
|
||||||
one range opening covers every container.
|
one range opening covers every container.
|
||||||
|
|
@ -306,10 +295,9 @@ proxy in front.
|
||||||
Agents poll `HIVE_FORGE_URL` for Forgejo notifications + run all
|
Agents poll `HIVE_FORGE_URL` for Forgejo notifications + run all
|
||||||
`hive-forge` calls against it. Network isolation is always on (the
|
`hive-forge` calls against it. Network isolation is always on (the
|
||||||
shared-netns mode was removed), so agents run in a private netns and
|
shared-netns mode was removed), so agents run in a private netns and
|
||||||
can never reach the host's loopback.
|
can never reach the host's loopback. `hive-c0re.nix` sets
|
||||||
`nix/host-modules/hive-c0re/environment.nix` sets `HIVE_FORGE_URL` to
|
`HIVE_FORGE_URL` to `http://<forge.domain>` (default
|
||||||
`http://<forge.domain>` (default `forge.<swarm-domain>` — a swarm runs
|
`forge.<hive-domain>`; `services.hyperhive.domain` is required). Agents
|
||||||
one forge; `services.hyperhive.domain` is required). Agents
|
|
||||||
get the bridge dnsmasq as their resolver, resolve the hostname →
|
get the bridge dnsmasq as their resolver, resolve the hostname →
|
||||||
bridge IP, then reach nginx on port 80 (the bridge firewall opens
|
bridge IP, then reach nginx on port 80 (the bridge firewall opens
|
||||||
80+443). nginx proxies to forgejo — the same path an operator browser
|
80+443). nginx proxies to forgejo — the same path an operator browser
|
||||||
|
|
@ -381,16 +369,14 @@ covers most cases:
|
||||||
|
|
||||||
| Shape | Auto-derived `ROOT_URL` |
|
| Shape | Auto-derived `ROOT_URL` |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `behindGateway = true` | `https://<forge.domain>/` (port suffix omitted when `gateway.httpsPort == 443`) |
|
| `behindGateway = true` | `http://<forge.domain>/` (port suffix omitted when `gateway.port == 80`) |
|
||||||
| `behindGateway = false` | `http://<forge.domain>:<httpPort>/` |
|
| `behindGateway = false` | `http://<forge.domain>:<httpPort>/` |
|
||||||
|
|
||||||
The gateway always terminates TLS, so the `behindGateway = true` case is
|
The auto-derivation always uses `http://`. Set `rootUrl` explicitly when
|
||||||
always advertised over `https://`; only the direct (`behindGateway =
|
you need `https://` (e.g. behind a TLS-terminating reverse proxy, or when
|
||||||
false`) shape stays `http://`. Set `rootUrl` explicitly when
|
clone URLs must carry `https://` because the gateway terminates TLS), or
|
||||||
`forge.domain` resolves differently from the public URL, or for a
|
when `forge.domain` resolves differently from the public URL. Must end with
|
||||||
genuinely bespoke shape (e.g. an external reverse proxy on a different
|
`/` (Forgejo requirement; an assertion enforces this).
|
||||||
host/path). Must end with `/` (Forgejo requirement; an assertion
|
|
||||||
enforces this).
|
|
||||||
|
|
||||||
## Per-agent static frontend split
|
## Per-agent static frontend split
|
||||||
|
|
||||||
|
|
@ -603,7 +589,7 @@ header is added alongside the other security headers.
|
||||||
enabling it on a deployment that later loses TLS locks browsers out
|
enabling it on a deployment that later loses TLS locks browsers out
|
||||||
until `max-age` expires. Only enable when TLS is permanent.
|
until `max-age` expires. Only enable when TLS is permanent.
|
||||||
|
|
||||||
Since the gateway always terminates TLS (see [TLS modes](#tls-modes)
|
The gateway always terminates TLS now (self-signed floor), so HSTS is
|
||||||
above), an enabled HSTS header is always served over https — there is no
|
always served over https when enabled — the old "HSTS requires a TLS mode"
|
||||||
TLS-less mode that could violate it.
|
assertion is gone (it can no longer be violated).
|
||||||
|
|
||||||
|
|
|
||||||
170
docs/gotchas.md
170
docs/gotchas.md
|
|
@ -2,31 +2,27 @@
|
||||||
|
|
||||||
NixOS + nspawn quirks and lessons we hit the hard way. If something
|
NixOS + nspawn quirks and lessons we hit the hard way. If something
|
||||||
here looks unmotivated in the code, there's usually a story underneath.
|
here looks unmotivated in the code, there's usually a story underneath.
|
||||||
Grouped by area — jump to the section that matches what you're
|
|
||||||
touching.
|
|
||||||
|
|
||||||
## NixOS / nspawn containers
|
## `nixos-container` doesn't expose `--bind` on the CLI
|
||||||
|
|
||||||
### `nixos-container` doesn't expose `--bind` on the CLI
|
|
||||||
|
|
||||||
The CLI doesn't accept `--bind`. Path is via `EXTRA_NSPAWN_FLAGS` in
|
The CLI doesn't accept `--bind`. Path is via `EXTRA_NSPAWN_FLAGS` in
|
||||||
`/etc/nixos-containers/<NAME>.conf` — the start script
|
`/etc/nixos-containers/<NAME>.conf` — the start script
|
||||||
(`/nix/store/.../container_-start`) expands it unquoted into the
|
(`/nix/store/.../container_-start`) expands it unquoted into the
|
||||||
`systemd-nspawn` invocation. `lifecycle::host_config::set_nspawn_flags()`
|
`systemd-nspawn` invocation. `lifecycle::set_nspawn_flags()` rewrites
|
||||||
rewrites this line.
|
this line.
|
||||||
|
|
||||||
### `/run/systemd/nspawn/*.nspawn` overrides are ignored
|
## `/run/systemd/nspawn/*.nspawn` overrides are ignored
|
||||||
|
|
||||||
`nixos-container`'s start script builds the nspawn command line
|
`nixos-container`'s start script builds the nspawn command line
|
||||||
directly. Dropping a `.nspawn` file under `/run/systemd/nspawn/`
|
directly. Dropping a `.nspawn` file under `/run/systemd/nspawn/`
|
||||||
looks like the obvious extension point and does nothing. Use
|
looks like the obvious extension point and does nothing. Use
|
||||||
`EXTRA_NSPAWN_FLAGS` (above).
|
`EXTRA_NSPAWN_FLAGS` (above).
|
||||||
|
|
||||||
### `boot.isNspawnContainer = true`
|
## `boot.isNspawnContainer = true`
|
||||||
|
|
||||||
Not `boot.isContainer = true`. Renamed in nixos-25.11+.
|
Not `boot.isContainer = true`. Renamed in nixos-25.11+.
|
||||||
|
|
||||||
### `nixos-container create` auto-assigns `HOST_ADDRESS` / `LOCAL_ADDRESS`
|
## `nixos-container create` auto-assigns `HOST_ADDRESS` / `LOCAL_ADDRESS`
|
||||||
|
|
||||||
…in the `.conf`. The start script's `if HOST_ADDRESS set →
|
…in the `.conf`. The start script's `if HOST_ADDRESS set →
|
||||||
--network-veth` branch then forces a private netns — silently fatal
|
--network-veth` branch then forces a private netns — silently fatal
|
||||||
|
|
@ -34,7 +30,7 @@ for our web UIs (the bind is invisible from the host). We
|
||||||
force-clear `HOST_ADDRESS` / `LOCAL_ADDRESS` / `HOST_ADDRESS6` /
|
force-clear `HOST_ADDRESS` / `LOCAL_ADDRESS` / `HOST_ADDRESS6` /
|
||||||
`LOCAL_ADDRESS6` / `HOST_BRIDGE` and set `PRIVATE_NETWORK=0`.
|
`LOCAL_ADDRESS6` / `HOST_BRIDGE` and set `PRIVATE_NETWORK=0`.
|
||||||
|
|
||||||
### systemd service PATH ≠ host PATH
|
## systemd service PATH ≠ host PATH
|
||||||
|
|
||||||
The hive-c0re service sets `path = [ pkgs.git "/run/current-system/sw" ]`.
|
The hive-c0re service sets `path = [ pkgs.git "/run/current-system/sw" ]`.
|
||||||
In-container harness services do the same so anything an agent adds
|
In-container harness services do the same so anything an agent adds
|
||||||
|
|
@ -44,7 +40,7 @@ editing the service definition.
|
||||||
`environment.HYPERHIVE_GIT` bakes git's absolute path in (read by
|
`environment.HYPERHIVE_GIT` bakes git's absolute path in (read by
|
||||||
`lifecycle::git_command()`) for the host.
|
`lifecycle::git_command()`) for the host.
|
||||||
|
|
||||||
### `systemd.services.*.path` appends `/bin` to every entry
|
## `systemd.services.*.path` appends `/bin` to every entry
|
||||||
|
|
||||||
NixOS's `systemd.services.<unit>.path` list feeds every entry through
|
NixOS's `systemd.services.<unit>.path` list feeds every entry through
|
||||||
`lib.makeBinPath`, which **appends `/bin` unconditionally**. That's
|
`lib.makeBinPath`, which **appends `/bin` unconditionally**. That's
|
||||||
|
|
@ -66,21 +62,19 @@ contains a non-existent directory. The first symptom is usually
|
||||||
the setuid sudo wrapper lives at `/run/wrappers/bin/sudo` and
|
the setuid sudo wrapper lives at `/run/wrappers/bin/sudo` and
|
||||||
the path entry resolves to `/run/wrappers/bin/bin` instead.
|
the path entry resolves to `/run/wrappers/bin/bin` instead.
|
||||||
|
|
||||||
### `RuntimeDirectoryPreserve = "yes"`
|
## `RuntimeDirectoryPreserve = "yes"`
|
||||||
|
|
||||||
…keeps `/run/hyperhive/` (and the per-agent sub-dirs) across
|
…keeps `/run/hyperhive/` (and the per-agent sub-dirs) across
|
||||||
hive-c0re restarts. Without it, every restart wipes bind sources and
|
hive-c0re restarts. Without it, every restart wipes bind sources and
|
||||||
existing containers can't be started.
|
existing containers can't be started.
|
||||||
|
|
||||||
### `register_agent` is idempotent
|
## `register_agent` is idempotent
|
||||||
|
|
||||||
Drops any prior socket task before rebinding. Required so a
|
Drops any prior socket task before rebinding. Required so a
|
||||||
hive-c0re restart followed by `rebuild alice` recreates the agent's
|
hive-c0re restart followed by `rebuild alice` recreates the agent's
|
||||||
socket without needing a clean reinstall.
|
socket without needing a clean reinstall.
|
||||||
|
|
||||||
## Claude Code packaging & credentials
|
## `claude-code` is unfree
|
||||||
|
|
||||||
### `claude-code` is unfree
|
|
||||||
|
|
||||||
`claude-code` comes from the flake's main `nixpkgs` (nixos-26.05).
|
`claude-code` comes from the flake's main `nixpkgs` (nixos-26.05).
|
||||||
It's unfree, so the agent modules set `config.allowUnfreePredicate`
|
It's unfree, so the agent modules set `config.allowUnfreePredicate`
|
||||||
|
|
@ -131,7 +125,7 @@ the hive's `claude` out from under it. The price of the root is that an
|
||||||
old `claude-code` can't be reclaimed until every agent has rebuilt past
|
old `claude-code` can't be reclaimed until every agent has rebuilt past
|
||||||
it and the old generations are gone.
|
it and the old generations are gone.
|
||||||
|
|
||||||
### Claude credentials are per-agent
|
## Claude credentials are per-agent
|
||||||
|
|
||||||
`/var/lib/hyperhive/agents/<name>/claude/` bind-mounts to
|
`/var/lib/hyperhive/agents/<name>/claude/` bind-mounts to
|
||||||
`/home/<name>/.claude` (RW). Sharing one dir across agents is NOT viable —
|
`/home/<name>/.claude` (RW). Sharing one dir across agents is NOT viable —
|
||||||
|
|
@ -139,22 +133,17 @@ OAuth refresh tokens rotate, so any sibling refresh invalidates all
|
||||||
the others. Login flow runs from the per-agent web UI; creds persist
|
the others. Login flow runs from the per-agent web UI; creds persist
|
||||||
across `destroy`/recreate (`--purge` wipes them).
|
across `destroy`/recreate (`--purge` wipes them).
|
||||||
|
|
||||||
### Persistent notes dir per agent
|
## Persistent notes dir per agent
|
||||||
|
|
||||||
`/var/lib/hyperhive/agents/<name>/state/` bind-mounts to
|
`/var/lib/hyperhive/agents/<name>/state/` bind-mounts to
|
||||||
`/agents/<name>/state` (RW; uniform for all agents).
|
`/agents/<name>/state` (RW; uniform for all agents).
|
||||||
The harness exposes the same path
|
The harness exposes the same path
|
||||||
via `$HYPERHIVE_STATE_DIR`. System prompts tell agents to keep
|
via `$HYPERHIVE_STATE_DIR`. System prompts tell agents to keep
|
||||||
durable knowledge here (`notes.md`, anything else) — the harness's own
|
durable knowledge here (`notes.md`, anything else). The harness also
|
||||||
internal files (`hyperhive-events.sqlite`, `hyperhive-turn-stats.sqlite`,
|
writes its events log here (`hyperhive-events.sqlite`).
|
||||||
`hyperhive-model`) live in the separate `harness` dir instead, so they
|
Survives `destroy`/recreate alongside the claude dir.
|
||||||
don't clutter what claude sees as "my notes dir" (see
|
|
||||||
[`docs/persistence.md`](persistence.md)). Survives `destroy`/recreate
|
|
||||||
alongside the claude dir.
|
|
||||||
|
|
||||||
## Networking & ports
|
## Web UI ports collide on hash
|
||||||
|
|
||||||
### Web UI ports collide on hash
|
|
||||||
|
|
||||||
Sub-agent web UI ports are deterministic FNV-1a of the agent name
|
Sub-agent web UI ports are deterministic FNV-1a of the agent name
|
||||||
modulo 900 (range 8100..8999). With ~30 agents the birthday-paradox
|
modulo 900 (range 8100..8999). With ~30 agents the birthday-paradox
|
||||||
|
|
@ -166,7 +155,7 @@ reproducible from just the name. Every agent hashes into
|
||||||
8100..8999 via the same FNV-1a; dashboard
|
8100..8999 via the same FNV-1a; dashboard
|
||||||
at `cfg.dashboardPort` (default 7000).
|
at `cfg.dashboardPort` (default 7000).
|
||||||
|
|
||||||
### Restart races on TCP bind
|
## Restart races on TCP bind
|
||||||
|
|
||||||
Both the dashboard and per-agent web UI use `tokio::net::TcpSocket`
|
Both the dashboard and per-agent web UI use `tokio::net::TcpSocket`
|
||||||
with `SO_REUSEADDR` plus a retry-on-`AddrInUse` loop (12 tries,
|
with `SO_REUSEADDR` plus a retry-on-`AddrInUse` loop (12 tries,
|
||||||
|
|
@ -177,35 +166,38 @@ overlap" case. REUSEADDR does **not** allow two simultaneous
|
||||||
`LISTEN` sockets on the same port (that would be `SO_REUSEPORT`,
|
`LISTEN` sockets on the same port (that would be `SO_REUSEPORT`,
|
||||||
which we don't use) — exclusivity is preserved.
|
which we don't use) — exclusivity is preserved.
|
||||||
|
|
||||||
## Approvals
|
## Orphan approvals
|
||||||
|
|
||||||
### Orphan approvals
|
|
||||||
|
|
||||||
If state dirs are wiped out from under a pending approval (test
|
If state dirs are wiped out from under a pending approval (test
|
||||||
scripts, manual `rm -rf`), the dashboard's next render marks them
|
scripts, manual `rm -rf`), the dashboard's next render marks them
|
||||||
`failed` with note `"agent state dir missing"` so they fall out of
|
`failed` with note `"agent state dir missing"` so they fall out of
|
||||||
`pending`. They stay in sqlite for audit.
|
`pending`. They stay in sqlite for audit.
|
||||||
|
|
||||||
## Gateway / SPA serving
|
## Nix store `cp -r` preserves read-only bits
|
||||||
|
|
||||||
### SPA fallback: use `Accept` header map, not `try_files ... /index.html`
|
Copying a nix store path with `cp -r src/. $out/` inside a
|
||||||
|
`pkgs.runCommand` derivation preserves the read-only permissions of
|
||||||
|
store files. Any subsequent write into the copied tree (adding new
|
||||||
|
files in subdirectories) fails with `EPERM`. Fix: pass
|
||||||
|
`--no-preserve=mode,ownership` so the output tree is writable.
|
||||||
|
|
||||||
The naive nginx pattern for an SPA (`try_files $uri $uri/
|
## SPA fallback: use `Accept` header map, not `try_files ... /index.html`
|
||||||
/index.html`) silently swallows asset 404s — a missing JS file
|
|
||||||
|
The naive nginx pattern for a path-prefix SPA (`try_files $uri $uri/
|
||||||
|
/matrix/index.html`) silently swallows asset 404s — a missing JS file
|
||||||
returns `index.html` with a 200, so the JS runtime never loads and the
|
returns `index.html` with a 200, so the JS runtime never loads and the
|
||||||
page renders blank with no visible error. Extension allowlists (tried
|
page renders blank with no visible error. Extension allowlists (tried
|
||||||
as an alternative) have the same maintenance problem: any new file
|
as an alternative) have the same maintenance problem: any new file
|
||||||
extension the SPA ships breaks silently.
|
extension the SPA ships breaks silently.
|
||||||
|
|
||||||
The pattern that works (`nix/host-modules/hive-matrix.nix`, serving
|
The pattern that works (`hive-gateway.nix`) keys the fallback on the
|
||||||
fluffychat at the matrix gateway vhost's root) keys the fallback on the
|
|
||||||
HTTP `Accept` header:
|
HTTP `Accept` header:
|
||||||
|
|
||||||
```nginx
|
```nginx
|
||||||
# Outside the server block (appendHttpConfig):
|
# Outside the server block (appendHttpConfig):
|
||||||
map $http_accept $matrix_spa_target {
|
map $http_accept $matrix_spa_target {
|
||||||
default "/__matrix_spa_no_html_fallback";
|
default "/__matrix_spa_no_html_fallback";
|
||||||
"~*text/html" "/index.html";
|
"~*text/html" "/matrix/index.html";
|
||||||
}
|
}
|
||||||
|
|
||||||
# Inside the location:
|
# Inside the location:
|
||||||
|
|
@ -218,17 +210,7 @@ firefox / safari are consistent). Asset fetches (`image/*`,
|
||||||
fall through to the trailing `=404`. No extension list to maintain;
|
fall through to the trailing `=404`. No extension list to maintain;
|
||||||
no named-location indirection needed.
|
no named-location indirection needed.
|
||||||
|
|
||||||
## Build & dev workflow
|
## `nix build flake#name` does not walk into `nixosConfigurations`
|
||||||
|
|
||||||
### Nix store `cp -r` preserves read-only bits
|
|
||||||
|
|
||||||
Copying a nix store path with `cp -r src/. $out/` inside a
|
|
||||||
`pkgs.runCommand` derivation preserves the read-only permissions of
|
|
||||||
store files. Any subsequent write into the copied tree (adding new
|
|
||||||
files in subdirectories) fails with `EPERM`. Fix: pass
|
|
||||||
`--no-preserve=mode,ownership` so the output tree is writable.
|
|
||||||
|
|
||||||
### `nix build flake#name` does not walk into `nixosConfigurations`
|
|
||||||
|
|
||||||
`nix build` resolves the fragment (`#name`) against the flake's
|
`nix build` resolves the fragment (`#name`) against the flake's
|
||||||
**top-level output attrs** — not against `nixosConfigurations`
|
**top-level output attrs** — not against `nixosConfigurations`
|
||||||
|
|
@ -251,7 +233,12 @@ instead of `meta#nixosConfigurations.argus.config…`. The fix:
|
||||||
`split_once('#')` to separate flake path from name, then template
|
`split_once('#')` to separate flake path from name, then template
|
||||||
`{path}#nixosConfigurations.{name}.config.system.build.toplevel`.
|
`{path}#nixosConfigurations.{name}.config.system.build.toplevel`.
|
||||||
|
|
||||||
### Containerized nix-daemon needs `sandbox-fallback = true`
|
## `hive-forge`: prefer over raw curl pipelines
|
||||||
|
|
||||||
|
Full CLI reference: [`docs/tools/forge.md`](tools/forge.md).
|
||||||
|
Never use raw `curl` for forge access.
|
||||||
|
|
||||||
|
## Containerized nix-daemon needs `sandbox-fallback = true`
|
||||||
|
|
||||||
Agent containers bind-mount the host's nix-daemon socket. nspawn
|
Agent containers bind-mount the host's nix-daemon socket. nspawn
|
||||||
containers don't get user-namespaces by default, so `nix build`
|
containers don't get user-namespaces by default, so `nix build`
|
||||||
|
|
@ -262,7 +249,7 @@ and fail outright if the host daemon's
|
||||||
fall back to unsandboxed local builds rather than failing. Security
|
fall back to unsandboxed local builds rather than failing. Security
|
||||||
implications: `docs/security.md`.
|
implications: `docs/security.md`.
|
||||||
|
|
||||||
### Linking workspace binaries locally needs `nix develop`
|
## Linking workspace binaries locally needs `nix develop`
|
||||||
|
|
||||||
The Rust workspace links `libsqlite3-sys` (rusqlite) against the
|
The Rust workspace links `libsqlite3-sys` (rusqlite) against the
|
||||||
system `libsqlite3`. Agent containers carry no system libsqlite3 on
|
system `libsqlite3`. Agent containers carry no system libsqlite3 on
|
||||||
|
|
@ -284,7 +271,7 @@ e.g. `docs/tools/hivectl-cli.md` via the `hivectl markdown-docs`
|
||||||
subcommand (its `hivectl-docs` flake check otherwise only fails in
|
subcommand (its `hivectl-docs` flake check otherwise only fails in
|
||||||
CI on drift).
|
CI on drift).
|
||||||
|
|
||||||
### Split asset derivations away from the rust workspace
|
## Split asset derivations away from the rust workspace
|
||||||
|
|
||||||
`nix/packages/assets.nix` builds the branding SVG/PNG family + claude
|
`nix/packages/assets.nix` builds the branding SVG/PNG family + claude
|
||||||
system-prompt template + claude-settings JSON as its own derivation,
|
system-prompt template + claude-settings JSON as its own derivation,
|
||||||
|
|
@ -298,46 +285,7 @@ The agent-configs PNG is rendered from the SVG via `rsvg-convert` at
|
||||||
build time; librsvg dependency lives here, not in the rust
|
build time; librsvg dependency lives here, not in the rust
|
||||||
derivation's `nativeBuildInputs`.
|
derivation's `nativeBuildInputs`.
|
||||||
|
|
||||||
### `nix fmt` fails in a git worktree with "object not found"
|
## Weston VNC compositor (per-agent `hyperhive.gui.enable`)
|
||||||
|
|
||||||
`nix fmt` (and any `nix` command that fetches a `git+file://` flake
|
|
||||||
URL) uses libgit2 internally to compute `revCount` — the number of
|
|
||||||
commits reachable from HEAD. This walk fails with:
|
|
||||||
|
|
||||||
```
|
|
||||||
error: getting Git object '<hash>': object not found (libgit2 error code = 9)
|
|
||||||
```
|
|
||||||
|
|
||||||
when a commit that was reachable at some earlier evaluation is now gone
|
|
||||||
(GC'd, rebased away, or pruned). The failure is persistent: clearing
|
|
||||||
`~/.cache/nix/{eval-cache-v6,gitv3,fetcher-cache-v4.sqlite}` does not
|
|
||||||
help because the missing object is a structural gap in the git object
|
|
||||||
graph itself, not in nix's caches.
|
|
||||||
|
|
||||||
**Workaround: use a plain clone, not a git worktree.**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git clone http://<forge>/hyperhive/hyperhive.git ~/hh-work
|
|
||||||
cd ~/hh-work && nix fmt
|
|
||||||
```
|
|
||||||
|
|
||||||
The root cause is specific to worktrees: a worktree shares the object
|
|
||||||
store with its parent repo. If the parent repo's history was rewritten
|
|
||||||
(rebase, force-push, `git gc --prune`) while the worktree was checked
|
|
||||||
out at a branch tip that references the pruned commits via its reflog or
|
|
||||||
history, libgit2's rev-walk encounters the gap. A plain clone has its
|
|
||||||
own self-consistent object store and is immune to the issue.
|
|
||||||
|
|
||||||
## Tooling
|
|
||||||
|
|
||||||
### `hive-forge`: prefer over raw curl pipelines
|
|
||||||
|
|
||||||
Full CLI reference: [`docs/tools/forge.md`](tools/forge.md).
|
|
||||||
Never use raw `curl` for forge access.
|
|
||||||
|
|
||||||
## GUI (weston/VNC)
|
|
||||||
|
|
||||||
### Weston VNC compositor (per-agent `hyperhive.gui.enable`)
|
|
||||||
|
|
||||||
`nix/agent-modules/weston-vnc.nix` adds an optional Weston Wayland
|
`nix/agent-modules/weston-vnc.nix` adds an optional Weston Wayland
|
||||||
compositor with the VNC backend, surfaced as
|
compositor with the VNC backend, surfaced as
|
||||||
|
|
@ -413,9 +361,7 @@ connects to the compositor at `127.0.0.1:<vnc_port>`.
|
||||||
`wl_event_source_timer_update` treats as "disarm", so the
|
`wl_event_source_timer_update` treats as "disarm", so the
|
||||||
compositor never goes idle and never locks.
|
compositor never goes idle and never locks.
|
||||||
|
|
||||||
## Nix docs pipeline
|
## Nix options reference (`nix/docs/default.nix`)
|
||||||
|
|
||||||
### Nix options reference (`nix/docs/default.nix`)
|
|
||||||
|
|
||||||
`pkgs.nixosOptionsDoc` over two evaluated module trees:
|
`pkgs.nixosOptionsDoc` over two evaluated module trees:
|
||||||
`hostEval` (a stub NixOS system loading the `nix/host-modules/` aggregator with every
|
`hostEval` (a stub NixOS system loading the `nix/host-modules/` aggregator with every
|
||||||
|
|
@ -451,7 +397,7 @@ options tree picks up everything under that root — picking against
|
||||||
stray roots produces an empty tree and renders the host page as
|
stray roots produces an empty tree and renders the host page as
|
||||||
template chrome with no `<h2>` headers.
|
template chrome with no `<h2>` headers.
|
||||||
|
|
||||||
#### Docs drv stability: `nixSrc`
|
### Docs drv stability: `nixSrc`
|
||||||
|
|
||||||
Naively, the docs evaluation depends on `self` (the flake's store path),
|
Naively, the docs evaluation depends on `self` (the flake's store path),
|
||||||
so every commit — even Rust-only or frontend-only changes — produces new
|
so every commit — even Rust-only or frontend-only changes — produces new
|
||||||
|
|
@ -480,3 +426,33 @@ Why `builtins.unsafeDiscardStringContext`? The path string
|
||||||
make `builtins.path` include `self` as a build dependency even after
|
make `builtins.path` include `self` as a build dependency even after
|
||||||
content-addressing the directory. Discarding the context makes the
|
content-addressing the directory. Discarding the context makes the
|
||||||
resulting `nixSrc` truly independent of `self`'s store path.
|
resulting `nixSrc` truly independent of `self`'s store path.
|
||||||
|
|
||||||
|
### `nix fmt` fails in a git worktree with "object not found"
|
||||||
|
|
||||||
|
`nix fmt` (and any `nix` command that fetches a `git+file://` flake
|
||||||
|
URL) uses libgit2 internally to compute `revCount` — the number of
|
||||||
|
commits reachable from HEAD. This walk fails with:
|
||||||
|
|
||||||
|
```
|
||||||
|
error: getting Git object '<hash>': object not found (libgit2 error code = 9)
|
||||||
|
```
|
||||||
|
|
||||||
|
when a commit that was reachable at some earlier evaluation is now gone
|
||||||
|
(GC'd, rebased away, or pruned). The failure is persistent: clearing
|
||||||
|
`~/.cache/nix/{eval-cache-v6,gitv3,fetcher-cache-v4.sqlite}` does not
|
||||||
|
help because the missing object is a structural gap in the git object
|
||||||
|
graph itself, not in nix's caches.
|
||||||
|
|
||||||
|
**Workaround: use a plain clone, not a git worktree.**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone http://<forge>/hyperhive/hyperhive.git ~/hh-work
|
||||||
|
cd ~/hh-work && nix fmt
|
||||||
|
```
|
||||||
|
|
||||||
|
The root cause is specific to worktrees: a worktree shares the object
|
||||||
|
store with its parent repo. If the parent repo's history was rewritten
|
||||||
|
(rebase, force-push, `git gc --prune`) while the worktree was checked
|
||||||
|
out at a branch tip that references the pruned commits via its reflog or
|
||||||
|
history, libgit2's rev-walk encounters the gap. A plain clone has its
|
||||||
|
own self-consistent object store and is immune to the issue.
|
||||||
|
|
|
||||||
|
|
@ -24,9 +24,8 @@ the local clone updates automatically (see
|
||||||
## Repository layout
|
## Repository layout
|
||||||
|
|
||||||
Canonical forge location: `internal/knowledge` (org `internal`,
|
Canonical forge location: `internal/knowledge` (org `internal`,
|
||||||
repo `knowledge`). The repo is public, so every agent's forge account
|
repo `knowledge`). Every agent's forge account is a read-only
|
||||||
has read access without an explicit per-agent collaborator grant;
|
collaborator; the `core` account has push access for auto-seeding.
|
||||||
only the `core` account has push access, for auto-seeding.
|
|
||||||
|
|
||||||
The repo is auto-created at hive-c0re startup if it doesn't exist,
|
The repo is auto-created at hive-c0re startup if it doesn't exist,
|
||||||
seeded with a `README.md` containing a contribution guide and a
|
seeded with a `README.md` containing a contribution guide and a
|
||||||
|
|
@ -40,12 +39,11 @@ hive-c0re maintains the local clone at
|
||||||
|
|
||||||
1. **Forgejo push webhook** — `ensure_webhook` registers a push
|
1. **Forgejo push webhook** — `ensure_webhook` registers a push
|
||||||
hook on `internal/knowledge` at startup pointing at
|
hook on `internal/knowledge` at startup pointing at
|
||||||
`https://<hive_domain>/webhook/knowledge` (routed through the
|
`http://127.0.0.1:<dashboard_port>/webhook/knowledge`. On any
|
||||||
gateway, avoiding the Forgejo SSRF guard that blocks loopback
|
push to main (including merge commits) hive-c0re runs
|
||||||
delivery). On any push to main (including merge commits) hive-c0re
|
`git pull` so agents see the new content on their next turn.
|
||||||
runs `git pull` so agents see the new content on their next turn.
|
The endpoint is loopback-only; no signature verification is
|
||||||
The endpoint is protected by an auto-generated HMAC secret that
|
needed.
|
||||||
hive-c0re verifies on every delivery.
|
|
||||||
|
|
||||||
2. **Periodic pull** — a background task in `hive-c0re::main`
|
2. **Periodic pull** — a background task in `hive-c0re::main`
|
||||||
pulls on a fixed cadence as a fallback (webhook missed, c0re
|
pulls on a fixed cadence as a fallback (webhook missed, c0re
|
||||||
|
|
@ -91,10 +89,9 @@ token.
|
||||||
|
|
||||||
## Contributing
|
## Contributing
|
||||||
|
|
||||||
Agents have read access to `internal/knowledge` (it's public) but no
|
Agents are read-only collaborators on `internal/knowledge`, so they
|
||||||
write access, so they can't push a branch directly. The supported path
|
can't push a branch directly. The supported path is Forgejo's **AGit
|
||||||
is Forgejo's **AGit flow** through the `hive-forge` CLI — no fork
|
flow** through the `hive-forge` CLI — no fork required.
|
||||||
required.
|
|
||||||
|
|
||||||
1. Clone the repo (credentials are injected automatically; the `-r`
|
1. Clone the repo (credentials are injected automatically; the `-r`
|
||||||
flag selects the repo, the clone lands in `./knowledge`):
|
flag selects the repo, the clone lands in `./knowledge`):
|
||||||
|
|
@ -128,6 +125,6 @@ required.
|
||||||
running container sees the updated content within seconds (see
|
running container sees the updated content within seconds (see
|
||||||
[Sync mechanism](#sync-mechanism)).
|
[Sync mechanism](#sync-mechanism)).
|
||||||
|
|
||||||
Do **not** try to `git push` a branch directly — lacking write access,
|
Do **not** try to `git push` a branch directly — read-only
|
||||||
it's rejected. The `--agit` flow above is the no-fork path that works
|
collaborator access rejects it. The `--agit` flow above is the
|
||||||
from any agent.
|
no-fork path that works from any agent.
|
||||||
|
|
|
||||||
|
|
@ -2,7 +2,7 @@
|
||||||
|
|
||||||
Private Matrix homeserver (matrix-tuwunel — the conduwuit
|
Private Matrix homeserver (matrix-tuwunel — the conduwuit
|
||||||
successor) wrapped in a nixos-container, plus optional fluffychat-web
|
successor) wrapped in a nixos-container, plus optional fluffychat-web
|
||||||
client at `chat.<swarm-domain>/` (the `gatewayHost` vhost). Configured via
|
client at `matrix.<hive>/`. Configured via
|
||||||
`services.hyperhive.swarm.matrix.*`; vhost routing lives in
|
`services.hyperhive.swarm.matrix.*`; vhost routing lives in
|
||||||
[`gateway.md`](gateway.md).
|
[`gateway.md`](gateway.md).
|
||||||
|
|
||||||
|
|
@ -208,28 +208,18 @@ resource-constrained hosts where a 20 MB request is unexpectedly large.
|
||||||
|
|
||||||
## Assertion rationale
|
## Assertion rationale
|
||||||
|
|
||||||
`config.assertions` in this module fail eval early rather than ship
|
Two `config.assertions` entries fail eval early rather than ship
|
||||||
surprising behaviour:
|
surprising behaviour:
|
||||||
|
|
||||||
|
- **`hyperhiveDomain != null || cfg.serverName != null`** —
|
||||||
|
`server_name` is embedded into every user / room ID irrevocably;
|
||||||
|
we refuse to spawn the homeserver with a bogus `server_name` we
|
||||||
|
can never change later.
|
||||||
- **`cfg.gatewayHost != ""`** — same footgun as `forge.domain`:
|
- **`cfg.gatewayHost != ""`** — same footgun as `forge.domain`:
|
||||||
empty string renders `.<hive>`-shaped garbage in both nginx
|
empty string renders `.<hive>`-shaped garbage in both nginx
|
||||||
`server_name` (treated as wildcard catch-all, surprising) and
|
`server_name` (treated as wildcard catch-all, surprising) and
|
||||||
`/etc/hosts` (invalid entry). `null` is the right opt-out shape;
|
`/etc/hosts` (invalid entry). `null` is the right opt-out shape;
|
||||||
empty string is rejected explicitly.
|
empty string is rejected explicitly.
|
||||||
- **`sso.enable` requires `sso.clientSecretFile`** — fails at eval,
|
|
||||||
not at boot: tuwunel reads its identity providers from the config
|
|
||||||
file, so a half-configured one can stop the homeserver from
|
|
||||||
starting outright rather than merely hiding a login button.
|
|
||||||
- **`sso.enable` requires `swarm.authelia.url`** — without a
|
|
||||||
provider URL there is nothing to discover against.
|
|
||||||
- **`sso.enable` requires `gatewayHost != null`** — the SSO callback
|
|
||||||
URL is format-locked to `<homeserver>/_matrix/client/unstable/login/sso/callback/<client_id>`,
|
|
||||||
and the identity provider needs a public name to redirect the
|
|
||||||
browser to.
|
|
||||||
|
|
||||||
`server_name`'s own bogus-value guard lives in `hive-network.nix`
|
|
||||||
(`services.hyperhive.domain != null`), not here — see
|
|
||||||
[`docs/network.md`](network.md).
|
|
||||||
|
|
||||||
## fluffychat-web build fixes
|
## fluffychat-web build fixes
|
||||||
|
|
||||||
|
|
@ -260,7 +250,7 @@ Both fixed in `nix/host-modules/hive-matrix.nix` via two derivations:
|
||||||
incremental cost) and (b) installs `fluffychat-web-imaging`'s
|
incremental cost) and (b) installs `fluffychat-web-imaging`'s
|
||||||
outputs into `$out`.
|
outputs into `$out`.
|
||||||
|
|
||||||
Two non-obvious details worth knowing before touching either derivation:
|
Two non-obvious fixes from review history:
|
||||||
|
|
||||||
- **`make -C js`** instead of `cd js; make` — keeps the build-phase
|
- **`make -C js`** instead of `cd js; make` — keeps the build-phase
|
||||||
pwd at the source root so `installPhase` doesn't have to know
|
pwd at the source root so `installPhase` doesn't have to know
|
||||||
|
|
@ -275,5 +265,5 @@ Two non-obvious details worth knowing before touching either derivation:
|
||||||
Drop both derivations when nixpkgs's flutter builder grows worker
|
Drop both derivations when nixpkgs's flutter builder grows worker
|
||||||
+ emcc support upstream.
|
+ emcc support upstream.
|
||||||
|
|
||||||
Mount point is `chat.<swarm-domain>/` (the `gatewayHost` vhost);
|
Mount point is `matrix.<hive>/`; upstream `--base-href "/"` is
|
||||||
upstream `--base-href "/"` is correct at sub-domain root, no override.
|
correct at sub-domain root, no override.
|
||||||
|
|
|
||||||
|
|
@ -112,19 +112,14 @@ schemes pick their own.
|
||||||
|
|
||||||
## Resolver behaviour
|
## Resolver behaviour
|
||||||
|
|
||||||
dnsmasq is **authoritative** for the hive's own zone (`<hive-domain>`)
|
dnsmasq is **authoritative** for the hive's own zones — answers
|
||||||
plus whatever swarm-service names this host contributes via
|
`<hive-domain>`, `forge.<hive-domain>`, `matrix.<hive-domain>` and —
|
||||||
`gateway.localNames` — `forge.<swarm-domain>` and `chat.<swarm-domain>`
|
on the host running it — the swarm's `auth.<swarm-domain>`
|
||||||
(matrix) when this host runs those services, and `auth.<swarm-domain>`
|
queries with the bridge IP (where nginx is reachable). Everything
|
||||||
when it runs authelia — answering each with the bridge IP (where nginx
|
else is forwarded to the host's own resolvers: dnsmasq runs on the host
|
||||||
is reachable). Note forge and matrix are swarm-domain names, not
|
and reads the host's `/etc/resolv.conf` directly. Containers don't need
|
||||||
sub-domains of `<hive-domain>`: a swarm runs one forge and one
|
to know the upstream — they query the bridge IP and dnsmasq does the
|
||||||
homeserver, so their names belong to the swarm rather than to whichever
|
right thing per-name.
|
||||||
hive happens to host them. Everything else is forwarded to the host's
|
|
||||||
own resolvers: dnsmasq runs on the host and reads the host's
|
|
||||||
`/etc/resolv.conf` directly. Containers don't need to know the
|
|
||||||
upstream — they query the bridge IP and dnsmasq does the right thing
|
|
||||||
per-name.
|
|
||||||
|
|
||||||
There is deliberately no fallback `server=`: dnsmasq queries all known
|
There is deliberately no fallback `server=`: dnsmasq queries all known
|
||||||
upstreams in parallel, so a hardcoded public resolver would take a share
|
upstreams in parallel, so a hardcoded public resolver would take a share
|
||||||
|
|
@ -250,11 +245,12 @@ wiring is runtime:
|
||||||
It is ordered `before` the harness (`hive-ag3nt`), the matrix daemon, and
|
It is ordered `before` the harness (`hive-ag3nt`), the matrix daemon, and
|
||||||
`tea-login` so the resolver is correct before the first DNS lookup.
|
`tea-login` so the resolver is correct before the first DNS lookup.
|
||||||
|
|
||||||
**Why isolation is safe**: hive-c0re's control-plane sockets are unix
|
**Why isolation is safe**: all hive-c0re communication goes
|
||||||
domain sockets bind-mounted into containers, not network listeners — see
|
through unix domain sockets (`/run/hive/mcp.sock` for agent requests,
|
||||||
the *Control plane (no network)* bullet under [Network
|
`/run/hive/priv.sock` for privileged ops).
|
||||||
map](#network-map) above. `PRIVATE_NETWORK=1` has no effect on a path
|
These are bind-mounted into containers via the nspawn conf. UDS paths
|
||||||
that never touches the network stack.
|
traverse the VFS, not the network stack, so `PRIVATE_NETWORK=1` does not
|
||||||
|
affect them.
|
||||||
|
|
||||||
The nix side also enables IP forwarding + NAT (agents reach the internet
|
The nix side also enables IP forwarding + NAT (agents reach the internet
|
||||||
through the host) and drops bridge-subnet → loopback traffic (defence-in-depth
|
through the host) and drops bridge-subnet → loopback traffic (defence-in-depth
|
||||||
|
|
|
||||||
|
|
@ -40,23 +40,20 @@ schemas, file layouts, and internal migration mechanics.
|
||||||
|
|
||||||
### `/var/lib/hyperhive/db/broker.sqlite` (host)
|
### `/var/lib/hyperhive/db/broker.sqlite` (host)
|
||||||
|
|
||||||
Seven tables, all in one file — three queues, a small key/value
|
Seven tables, all in one file — four queues, the schedule
|
||||||
table, the schedule header/targets split, and the per-agent
|
header/targets split, and the per-agent power-intent registry:
|
||||||
power-intent registry:
|
|
||||||
|
|
||||||
- `messages` — every inter-agent / operator-bound message.
|
- `messages` — every inter-agent / operator-bound message.
|
||||||
`sender / recipient / body / sent_at / delivered_at / acked_at /
|
`sender / recipient / body / sent_at / delivered_at / acked_at /
|
||||||
in_reply_to / priority`. `in_reply_to` links a reply to its parent
|
in_reply_to`. `in_reply_to` links a reply to its parent row id;
|
||||||
row id; the dashboard and per-agent inbox render these as threaded
|
the dashboard and per-agent inbox render these as threaded rows.
|
||||||
rows.
|
- `reminders` — `mcp__hyperhive__remind` queue.
|
||||||
- `kv` — small persistent key/value store (`key PK / value`) for
|
`agent / message / file_path / due_at / created_at / sent_at /
|
||||||
host-side bookkeeping that doesn't warrant its own table.
|
attempt_count / last_error`. `file_path` set when a body
|
||||||
|
exceeded the inline soft-cap and got auto-spilled to a file
|
||||||
⚠️ The `mcp__hyperhive__remind` queue is **not** here any more: it
|
under the agent's state dir; the worker delivers a short
|
||||||
moved to a harness-local, per-agent store as part of the
|
pointer instead. `attempt_count` / `last_error` accumulate
|
||||||
loose-ends-v2 migration — see [`/harness/` contents
|
on delivery-failed retries.
|
||||||
below](#state-dirs-per-agent) for where reminders (and todos, and
|
|
||||||
the questions mirror) actually live now.
|
|
||||||
- `approvals` — the queue. `agent / kind (merge_config_pr | spawn |
|
- `approvals` — the queue. `agent / kind (merge_config_pr | spawn |
|
||||||
init_config | update_meta_inputs | schedule_prompt) /
|
init_config | update_meta_inputs | schedule_prompt) /
|
||||||
commit_ref / requested_at / status / resolved_at / note`.
|
commit_ref / requested_at / status / resolved_at / note`.
|
||||||
|
|
@ -82,13 +79,16 @@ power-intent registry:
|
||||||
`scheduled_prompts(id)` — requires `PRAGMA foreign_keys = ON`
|
`scheduled_prompts(id)` — requires `PRAGMA foreign_keys = ON`
|
||||||
per connection (set at open).
|
per connection (set at open).
|
||||||
- `agent_power` — one tiny row per agent: `agent PK / wanted (up |
|
- `agent_power` — one tiny row per agent: `agent PK / wanted (up |
|
||||||
offline) / updated_at`, owned by `hive-c0re/src/stores/power.rs`.
|
offline) / updated_at` — the durable power *intent* behind the job
|
||||||
This is the durable power *intent* the job queue reconciles the
|
queue's desired-state reconciliation
|
||||||
observed container state against; intent survives hive-c0re
|
(`docs/coordinator.md::Job queue`; owner: `hive-c0re/src/stores/power.rs`).
|
||||||
restarts even though in-flight queue work doesn't. See
|
Written synchronously by every operator/agent power action
|
||||||
[`docs/coordinator.md`'s Desired-state
|
(dashboard start/stop, `hivectl stop`, the MCP kill/start tools,
|
||||||
section](coordinator.md#desired-state-spec-vs-status) for who
|
spawn approval); read by `Reconcile` nodes and the boot reconcile.
|
||||||
writes and reads it and how reconciliation works.
|
Intent survives hive-c0re restarts — in-flight queue work
|
||||||
|
deliberately does not. Agents without a row are seeded from
|
||||||
|
observed state on first touch (running ⇒ `up`); destroy deletes
|
||||||
|
the row.
|
||||||
|
|
||||||
Retention:
|
Retention:
|
||||||
|
|
||||||
|
|
@ -101,6 +101,8 @@ Retention:
|
||||||
- Approvals and questions are kept indefinitely — both are
|
- Approvals and questions are kept indefinitely — both are
|
||||||
audit trails. `actions::destroy` and answered questions stay
|
audit trails. `actions::destroy` and answered questions stay
|
||||||
visible to anything that queries by id.
|
visible to anything that queries by id.
|
||||||
|
- Reminder rows are kept after `sent_at` is set (audit trail);
|
||||||
|
no automatic vacuum today.
|
||||||
- Scheduled prompts: one-shot rows are deleted on fire by the
|
- Scheduled prompts: one-shot rows are deleted on fire by the
|
||||||
worker; recurring rows live until the operator cancels them
|
worker; recurring rows live until the operator cancels them
|
||||||
(`cancel_schedule` MCP / dashboard ✗) which tombstones via
|
(`cancel_schedule` MCP / dashboard ✗) which tombstones via
|
||||||
|
|
@ -118,26 +120,20 @@ One table:
|
||||||
- `events(id, ts, kind, payload_json)` — every `LiveEvent` the
|
- `events(id, ts, kind, payload_json)` — every `LiveEvent` the
|
||||||
harness emits during turn loop execution.
|
harness emits during turn loop execution.
|
||||||
|
|
||||||
The harness both writes and vacuums it — this used to be a host-side
|
The harness writes; the host vacuums. `hive-c0re::events_vacuum`
|
||||||
sweep, but hive-c0re runs as the unprivileged `hive-core` user under
|
runs hourly and sweeps every existing agent harness dir. Retention
|
||||||
privsep and can't delete agent-owned files (host-side deletes hit
|
is **type-scoped**: it deletes only the verbose `stream` rows (the
|
||||||
`PermissionDenied` on the bash-task trio and a readonly-database error
|
raw claude `stream-json` deltas — one per text chunk / tool use, the
|
||||||
here), so cleanup moved in-container. `hive-agent`'s `vacuum::run`
|
bulk of the file's size) older than 14 days, and keeps every other
|
||||||
(`hive-agent/src/vacuum.rs`) sweeps hourly. Retention is
|
kind (`turn_start`, `turn_end`, `note`, `status_changed`,
|
||||||
**type-scoped**: it deletes only the verbose `stream` rows (the raw
|
`model_changed`, `token_usage_changed`, `turn_state_changed`)
|
||||||
claude `stream-json` deltas — one per text chunk / tool use, the bulk
|
indefinitely — those are small and carry the semantic per-turn
|
||||||
of the file's size) older than 14 days, and keeps every other kind
|
history the operator scrolls back through when debugging a
|
||||||
(`turn_start`, `turn_end`, `note`, `status_changed`, `model_changed`,
|
regression. Age-only within the `stream` kind — no row cap — so a
|
||||||
`token_usage_changed`, `turn_state_changed`) indefinitely — those are
|
chatty turn doesn't lose its stream history sooner than a quiet one.
|
||||||
small and carry the semantic per-turn history the operator scrolls
|
Centralising retention on the host means a misbehaving harness can't
|
||||||
back through when debugging a regression. Age-only within the
|
disable its own vacuum and agents don't need any cleanup wiring of
|
||||||
`stream` kind — no row cap — so a chatty turn doesn't lose its stream
|
their own.
|
||||||
history sooner than a quiet one. The trade-off (accepted): a
|
|
||||||
misbehaving harness could now skip its own cleanup, which the old
|
|
||||||
host-side sweep was meant to prevent — but a compromised harness is
|
|
||||||
already inside the container trust boundary
|
|
||||||
([`docs/security.md`](security.md)), and these are ephemeral local
|
|
||||||
artifacts, so cleaning them up where they live is the honest fix.
|
|
||||||
|
|
||||||
Path overridable via `HYPERHIVE_EVENTS_DB` (for dev / no-`/harness`
|
Path overridable via `HYPERHIVE_EVENTS_DB` (for dev / no-`/harness`
|
||||||
setups). On open failure the `Bus` falls back to no-store mode
|
setups). On open failure the `Bus` falls back to no-store mode
|
||||||
|
|
@ -194,10 +190,12 @@ Shape:
|
||||||
The turn loop is the only writer today, but it still goes
|
The turn loop is the only writer today, but it still goes
|
||||||
read-modify-write under a shared in-process lock and merges into the
|
read-modify-write under a shared in-process lock and merges into the
|
||||||
existing object rather than reconstructing it — so a second writer
|
existing object rather than reconstructing it — so a second writer
|
||||||
would preserve fields it doesn't own, and the lock closes the
|
preserves fields it doesn't own, and the lock closes the lost-update
|
||||||
lost-update window between a writer's read and its rename. The lock
|
window between a writer's read and its rename. (The forge notification
|
||||||
is in-process only, so it wouldn't serialise a writer running as a
|
poller used to be that second writer, for a delivery-dedupe cursor. It
|
||||||
separate process; none of today's writers are.
|
persists nothing now — forge's own read-state is the durable record of
|
||||||
|
what has been delivered — and it is a separate process, which an
|
||||||
|
in-process lock could not have serialised anyway.)
|
||||||
|
|
||||||
hive-c0re reads this file on each `build_all` sweep (~10s) via
|
hive-c0re reads this file on each `build_all` sweep (~10s) via
|
||||||
`container_view::read_harness_flags`. Falls back to the legacy individual
|
`container_view::read_harness_flags`. Falls back to the legacy individual
|
||||||
|
|
@ -211,22 +209,17 @@ Full stdout + stderr capture for every `nixos-container` / `nix
|
||||||
build` invocation the lifecycle layer fires. One row per invocation;
|
build` invocation the lifecycle layer fires. One row per invocation;
|
||||||
the row accumulates lines as the child runs.
|
the row accumulates lines as the child runs.
|
||||||
|
|
||||||
Capturing the full stream (rather than a short tail buffer) matters
|
Replaces the legacy 32-line stderr ring buffer that `lifecycle::run`
|
||||||
because real eval errors routinely run long — "tried alternatives"
|
kept. The ring tail routinely truncated real eval errors ("tried
|
||||||
blocks alone are often 30+ lines — so a truncated tail would cut off
|
alternatives" blocks alone are often 30+ lines), so failures bailed
|
||||||
the actual failure and leave only the host journal holding the
|
with an arbitrary tail whose full stream only lived in the host
|
||||||
complete output. With this table the dashboard can surface the entire
|
journal. With this table the dashboard can surface the entire log.
|
||||||
log.
|
|
||||||
|
|
||||||
Three indices:
|
Two indices:
|
||||||
- `(agent, started_at)` — backs the per-agent latest-N lookup used
|
- `(agent, started_at)` — backs the per-agent latest-N lookup used
|
||||||
by the agent card chip.
|
by the agent card chip.
|
||||||
- `(status, finished_at)` — backs the retention sweep that runs
|
- `(status, finished_at)` — backs the retention sweep that runs
|
||||||
as part of the existing hourly vacuum.
|
as part of the existing hourly vacuum.
|
||||||
- `(node_id)` — added by a later migration so a build log row can be
|
|
||||||
looked up by the job-queue node it belongs to (a `hive_jobq` node is
|
|
||||||
immutable after insert, so the link is recorded on the log row
|
|
||||||
instead); legacy rows predating the column keep `node_id IS NULL`.
|
|
||||||
|
|
||||||
Writes are best-effort: `append_stdout` / `append_stderr` / `finish`
|
Writes are best-effort: `append_stdout` / `append_stderr` / `finish`
|
||||||
log a warning on sqlite error and let the build continue. A failed
|
log a warning on sqlite error and let the build continue. A failed
|
||||||
|
|
@ -250,7 +243,7 @@ and inbox messages queue unacked until it's removed (see
|
||||||
|
|
||||||
Unusually, it's read and written from **both** sides of the harness
|
Unusually, it's read and written from **both** sides of the harness
|
||||||
bind-mount, and that's the whole design: the harness stats it
|
bind-mount, and that's the whole design: the harness stats it
|
||||||
in-container via `hive-agent`'s `paths::paused_marker`, while hive-c0re
|
in-container via `hive_agent::paths::paused_marker`, while hive-c0re
|
||||||
stats it on the host (`Coordinator::is_paused`) to populate the
|
stats it on the host (`Coordinator::is_paused`) to populate the
|
||||||
`paused` field on the agent card, and creates/removes it
|
`paused` field on the agent card, and creates/removes it
|
||||||
(`Coordinator::set_paused`) for `hivectl agent <name> pause|resume` and the
|
(`Coordinator::set_paused`) for `hivectl agent <name> pause|resume` and the
|
||||||
|
|
@ -295,26 +288,15 @@ Under `/var/lib/hyperhive/agents/<name>/`:
|
||||||
- `bash-tasks/` — task JSON + stdout/stderr files for
|
- `bash-tasks/` — task JSON + stdout/stderr files for
|
||||||
background `mcp__bash__run` jobs. JSON files are
|
background `mcp__bash__run` jobs. JSON files are
|
||||||
`<id>.json` (status + tails), `<id>.out` / `<id>.err`
|
`<id>.json` (status + tails), `<id>.out` / `<id>.err`
|
||||||
(full captured output). The harness's own hourly sweep
|
(full captured output). `hive-c0re::bash_tasks_vacuum` runs
|
||||||
(`hive-agent`'s `vacuum::run`, same one that ages out `stream`
|
hourly and deletes terminal task trios older than 48 hours;
|
||||||
event rows above) deletes terminal task trios older than 48
|
non-terminal (still-running) tasks are never deleted by vacuum.
|
||||||
hours; non-terminal (still-running) tasks are never deleted. This
|
- `hyperhive-todos.sqlite` — loose-ends-v2 todo store. In-container
|
||||||
used to be a host-side `hive-c0re` vacuum, moved in-container for
|
daemons (`hive-bash-daemon`, `hive-matrix-daemon`, `hive-forge-notify`)
|
||||||
the same privsep-ownership reason as the events vacuum above.
|
upsert keyed todos here over the harness's in-agent socket
|
||||||
- `hyperhive-state.sqlite` — consolidated loose-ends-v2 store: todos,
|
(`HIVE_AGENT_SOCKET`); the harness merges them into `get_loose_ends`
|
||||||
reminders, and a questions mirror, one small table each in a single
|
output and clears a row on `mark_todo_done`. Replaced the old
|
||||||
file (in-container daemons — `hive-bash-daemon`, `hive-matrix-daemon`,
|
file-based `mcp-loose-ends/` scanner.
|
||||||
`hive-forge-notify` — upsert keyed todos here over the harness's
|
|
||||||
in-agent socket, `HIVE_AGENT_SOCKET`; the harness merges them into
|
|
||||||
`get_loose_ends` output and clears a row on `mark_todo_done`).
|
|
||||||
Replaces three formerly-separate files
|
|
||||||
(`hyperhive-todos.sqlite`, `hyperhive-reminders.sqlite`, and the
|
|
||||||
old file-based `mcp-loose-ends/` scanner before that) — a one-time
|
|
||||||
boot migration (`db_migrate::run`) folds the legacy files into this
|
|
||||||
path the first time a harness boots after the upgrade. Also backs
|
|
||||||
the `mcp__hyperhive__remind` queue, which moved from a host-side
|
|
||||||
`broker.sqlite` table to this per-agent store as part of the same
|
|
||||||
migration.
|
|
||||||
|
|
||||||
The harness itself is also a producer, not just the socket server:
|
The harness itself is also a producer, not just the socket server:
|
||||||
boot wiring's `spawn_todo_socket` starts `todo_server::run` (the
|
boot wiring's `spawn_todo_socket` starts `todo_server::run` (the
|
||||||
|
|
@ -341,9 +323,11 @@ notes, clearing a stuck sentinel) as well as reading it.
|
||||||
|
|
||||||
**`harness` is not mounted at all.** It holds the child's own runtime
|
**`harness` is not mounted at all.** It holds the child's own runtime
|
||||||
material — `bash-tasks/`, the turn-stats and event sqlite dbs — and
|
material — `bash-tasks/`, the turn-stats and event sqlite dbs — and
|
||||||
nothing argues for a parent reading it, let alone writing it. hive-c0re
|
nothing argues for a parent reading it, let alone writing it. It used to
|
||||||
reads a child's harness dir **directly on the host** when it wants
|
be mounted RW for "the same management reasons" as `state`, which was
|
||||||
those stats, which needs no mount into the parent.
|
never an argument so much as the side-effect of one loop treating all
|
||||||
|
three dirs alike. hive-c0re reads a child's harness dir **directly on the
|
||||||
|
host** when it wants those stats, which needs no mount into the parent.
|
||||||
|
|
||||||
**`config` is read-only, including for the parent.** A config change is
|
**`config` is read-only, including for the parent.** A config change is
|
||||||
a PR on the child's config repo, made from a clone and merged after
|
a PR on the child's config repo, made from a clone and merged after
|
||||||
|
|
@ -355,9 +339,8 @@ boundary a convention rather than a permission.
|
||||||
⚠️ Not to be confused with the seeding done when an `InitConfig`
|
⚠️ Not to be confused with the seeding done when an `InitConfig`
|
||||||
approval resolves: that writes the child's initial config repo as
|
approval resolves: that writes the child's initial config repo as
|
||||||
**hive-c0re, against the host path**, and `read_only` on a bind
|
**hive-c0re, against the host path**, and `read_only` on a bind
|
||||||
constrains writers *inside* a container only. The two are unrelated —
|
constrains writers *inside* a container only. The two are unrelated, and
|
||||||
conflating them is an easy way to reason your way into thinking this
|
reading them as the same thing is what kept this mount writable.
|
||||||
mount should be writable when it shouldn't.
|
|
||||||
|
|
||||||
Per-child isolation still holds: a container only ever has its *own*
|
Per-child isolation still holds: a container only ever has its *own*
|
||||||
dirs plus its direct children's bind-mounted, never a sibling's.
|
dirs plus its direct children's bind-mounted, never a sibling's.
|
||||||
|
|
@ -380,11 +363,8 @@ Contents:
|
||||||
audit trail (one commit per successful deploy or hyperhive bump).
|
audit trail (one commit per successful deploy or hyperhive bump).
|
||||||
- `topology.json` — parent/child agent graph
|
- `topology.json` — parent/child agent graph
|
||||||
(`{ "alice": "root", "bob": "alice", "root": null }`).
|
(`{ "alice": "root", "bob": "alice", "root": null }`).
|
||||||
Written by `topology::apply_set_parent` (the pure move-validating
|
Written by `topology::set_parent`; read by the dashboard, the
|
||||||
transform) via `meta::bulk_commit_topology` (the committer — see the
|
renderer, and `<parent>` / `<children>` recipient resolution.
|
||||||
`Reparent` node in [`docs/coordinator.md`](coordinator.md)); read by
|
|
||||||
the dashboard, the renderer, and `<parent>` / `<children>` recipient
|
|
||||||
resolution.
|
|
||||||
- `tool-groups.json` — per-agent MCP tool group grants
|
- `tool-groups.json` — per-agent MCP tool group grants
|
||||||
(`{ "alice": ["messaging", "inbox", "execution"] }`). Written by
|
(`{ "alice": ["messaging", "inbox", "execution"] }`). Written by
|
||||||
`tool_groups::set_groups`; injected as `HIVE_TOOL_GROUPS` env
|
`tool_groups::set_groups`; injected as `HIVE_TOOL_GROUPS` env
|
||||||
|
|
@ -416,21 +396,25 @@ deleted.
|
||||||
|
|
||||||
## Destroy vs purge
|
## Destroy vs purge
|
||||||
|
|
||||||
See [For operators](#for-operators) above for what each action does to
|
- `DESTR0Y` (default) — stops + removes the nspawn container,
|
||||||
an agent's state. The mechanics, for completeness:
|
drops the systemd drop-in, fails any pending approvals. State
|
||||||
|
dirs stay put; the agent appears in the dashboard's K3PT ST4T3
|
||||||
|
section as a tombstone with `⊕ R3V1V3` and `PURG3` actions.
|
||||||
|
`R3V1V3` queues a Spawn approval that reuses the kept state on
|
||||||
|
approve (no re-login).
|
||||||
|
- `PURG3` (opt-in via the dashboard button or
|
||||||
|
`hivectl agent <name> destroy --purge`) — DESTR0Y plus wipes
|
||||||
|
`/var/lib/hyperhive/{agents,applied}/<name>/`. Config history,
|
||||||
|
claude creds, /state/ notes, and the harness dir are all gone.
|
||||||
|
No undo.
|
||||||
|
|
||||||
- `DESTR0Y` also drops the systemd drop-in and fails any pending
|
The root/bootstrap container is **imperative** infrastructure — managed
|
||||||
approvals; the tombstone's `⊕ R3V1V3` button queues a Spawn approval
|
end-to-end by hive-c0re, not declared in the host's NixOS config.
|
||||||
that reuses the kept state on approve.
|
`auto_update::ensure_root_agent` recreates it on the next hive-c0re
|
||||||
- `PURG3` wipes `/var/lib/hyperhive/{agents,applied}/<name>/` — the
|
startup if it's absent (bypassing the approval queue, as required
|
||||||
union of everything `DESTR0Y` left behind.
|
infrastructure). A soft policy guard in `actions::destroy` currently
|
||||||
|
refuses to destroy it; even without that guard, destroying it would only
|
||||||
The root/bootstrap agent's specialness is implemented as a soft policy
|
be transient — hive-c0re brings it back on the next startup.
|
||||||
guard in `actions::destroy` that refuses to destroy it, backstopped by
|
|
||||||
`auto_update::ensure_root_agent`, which recreates it on the next
|
|
||||||
hive-c0re startup if it's ever absent (bypassing the approval queue,
|
|
||||||
as required infrastructure) — so even without the guard, destroying it
|
|
||||||
would only be transient.
|
|
||||||
|
|
||||||
### btrfs subvolumes for `/var/lib/hyperhive/agents/<name>`
|
### btrfs subvolumes for `/var/lib/hyperhive/agents/<name>`
|
||||||
|
|
||||||
|
|
@ -456,17 +440,8 @@ until an explicit opt-in upgrade.
|
||||||
actually a subvolume, then the normal `remove_dir_all` sweep covers
|
actually a subvolume, then the normal `remove_dir_all` sweep covers
|
||||||
plain-dir agents + the applied dir.
|
plain-dir agents + the applied dir.
|
||||||
|
|
||||||
Per-subvolume disk-usage accounting and optional quotas have since
|
Per-subvolume disk-usage accounting and optional quotas are a
|
||||||
landed as the qgroup work: `hivectl quota-enable` turns on btrfs
|
follow-up (the qgroup work), not part of the base migration.
|
||||||
qgroup accounting hive-wide (opt-in, no-op on non-btrfs hosts), and
|
|
||||||
`hivectl agent <name> quota show|set` reads/limits one agent's
|
|
||||||
subvolume usage through the same hive-priv-mediated path as
|
|
||||||
subvolume creation/deletion above.
|
|
||||||
|
|
||||||
This is the same subvolume `hivectl agent <name> subvol snapshot push`
|
|
||||||
sends to the swarm's snapshot store — see
|
|
||||||
[`docs/snapshot-store.md`](snapshot-store.md) for what a pushed
|
|
||||||
snapshot contains and how the store authenticates a sender.
|
|
||||||
|
|
||||||
## Run-time dirs
|
## Run-time dirs
|
||||||
|
|
||||||
|
|
@ -535,12 +510,10 @@ auto-injected `extraMcpServers.matrix` entry read).
|
||||||
**First-boot ordering**: hive-c0re provisions the matrix token AFTER
|
**First-boot ordering**: hive-c0re provisions the matrix token AFTER
|
||||||
agent containers come up. Without the path-trigger sibling
|
agent containers come up. Without the path-trigger sibling
|
||||||
(`systemd.paths.hive-matrix-daemon`, `PathExistsGlob =
|
(`systemd.paths.hive-matrix-daemon`, `PathExistsGlob =
|
||||||
/agents/*/state/matrix-token*` — the trailing `*` also catches a
|
/agents/*/state/matrix-token`), the daemon would exit 0 quietly the
|
||||||
secondary multi-account token like `matrix-token-ccc`), the daemon
|
first time it ran and the MCP would have no backend until the next
|
||||||
would exit 0 quietly the first time it ran and the MCP would have no
|
restart. The `.path` unit makes the appearance of the token re-fire
|
||||||
backend until the next restart. The `.path` unit makes the appearance
|
the service so the daemon comes alive in the same boot cycle as
|
||||||
of the token re-fire the service so the daemon comes alive in the
|
|
||||||
same boot cycle as
|
|
||||||
provisioning. The same token watcher also drives avatar setting: on a
|
provisioning. The same token watcher also drives avatar setting: on a
|
||||||
restart the daemon re-runs each account's bring-up, which sets the
|
restart the daemon re-runs each account's bring-up, which sets the
|
||||||
avatar (see below).
|
avatar (see below).
|
||||||
|
|
|
||||||
|
|
@ -5,8 +5,9 @@
|
||||||
The sections below document specific mechanisms (the state-file endpoint,
|
The sections below document specific mechanisms (the state-file endpoint,
|
||||||
nixbld isolation, privilege separation). This section frames the model they
|
nixbld isolation, privilege separation). This section frames the model they
|
||||||
serve: **what hyperhive defends, what it deliberately does not, and where the
|
serve: **what hyperhive defends, what it deliberately does not, and where the
|
||||||
operator is accepting risk.** It is the reference for "is it safe to give an
|
operator is accepting risk.** It emerged from a security discussion on
|
||||||
agent capability X?".
|
2026-06-24 (prompted by the `gh` CLI helper work) and is the reference for
|
||||||
|
"is it safe to give an agent capability X?".
|
||||||
|
|
||||||
### The trust boundary is the container, not credential storage
|
### The trust boundary is the container, not credential storage
|
||||||
|
|
||||||
|
|
@ -134,9 +135,9 @@ dashboard renders anchors only for tokens that passed the same checks the
|
||||||
read endpoint enforces.
|
read endpoint enforces.
|
||||||
|
|
||||||
The same invariant holds wherever an agent-supplied name reaches a filesystem
|
The same invariant holds wherever an agent-supplied name reaches a filesystem
|
||||||
path: the agent socket's `GetAgentMeta` takes `name` as a serde-validated
|
path: the agent socket's `GetAgentMeta` validates `name` with
|
||||||
`hive_types::Ident` (or falls back to `Ident::parse` for the "self" case)
|
`validate_agent_name` before building `agent_notes_dir(name)`, so a `..`
|
||||||
before building `agent_notes_dir(name)`, so a `..` component can't traverse.
|
component can't traverse.
|
||||||
|
|
||||||
## Nix builds and credential isolation
|
## Nix builds and credential isolation
|
||||||
|
|
||||||
|
|
@ -166,8 +167,8 @@ token policy bounds file reads; network isolation bounds network reach.
|
||||||
- `/home/<name>/.claude/` — mode `0700`, owned by the per-agent
|
- `/home/<name>/.claude/` — mode `0700`, owned by the per-agent
|
||||||
user `<name>`. nixbld users cannot read it.
|
user `<name>`. nixbld users cannot read it.
|
||||||
- `$HYPERHIVE_STATE_DIR/forge-token` (= `/agents/<name>/state/forge-token`)
|
- `$HYPERHIVE_STATE_DIR/forge-token` (= `/agents/<name>/state/forge-token`)
|
||||||
— written at mode `0600` and chowned to the per-agent uid:gid (see
|
— written at mode `0600` by `hive-c0re/src/forge.rs` and chowned to the
|
||||||
`hive-c0re/src/forge/mod.rs`'s module doc for exactly where). nixbld users
|
per-agent uid:gid by `lifecycle::chown_to_agent`. nixbld users
|
||||||
cannot read it.
|
cannot read it.
|
||||||
|
|
||||||
**Policy**: all credential files written to agent state directories MUST be mode
|
**Policy**: all credential files written to agent state directories MUST be mode
|
||||||
|
|
@ -209,8 +210,7 @@ known operations; there is no arbitrary command pass-through:
|
||||||
|
|
||||||
| Operation | What it runs |
|
| Operation | What it runs |
|
||||||
| ---------------------------------------------------- | ----------------------------------------------------------------------------------------- |
|
| ---------------------------------------------------- | ----------------------------------------------------------------------------------------- |
|
||||||
| `StartContainer` / `StopContainer` | `nixos-container start/stop <name>` |
|
| `StartContainer` / `StopContainer` / `KillContainer` | `nixos-container start/stop/kill <name>` |
|
||||||
| `KillContainer` | `machinectl kill <machine> --signal=SIGKILL` (`nixos-container` has no kill verb) |
|
|
||||||
| `CreateContainer` / `UpdateContainer` | `nixos-container create/update <name> --flake <ref>` |
|
| `CreateContainer` / `UpdateContainer` | `nixos-container create/update <name> --flake <ref>` |
|
||||||
| `DestroyContainer` | `nixos-container destroy <name>` |
|
| `DestroyContainer` | `nixos-container destroy <name>` |
|
||||||
| `ListContainers` | `nixos-container list` |
|
| `ListContainers` | `nixos-container list` |
|
||||||
|
|
@ -227,9 +227,7 @@ known operations; there is no arbitrary command pass-through:
|
||||||
**Container allowlist** — every request is validated against an
|
**Container allowlist** — every request is validated against an
|
||||||
allowlist before any operation: only names matching the agent-name
|
allowlist before any operation: only names matching the agent-name
|
||||||
convention (char-validated) or the known sibling service containers
|
convention (char-validated) or the known sibling service containers
|
||||||
(`hive-forge`, `hive-matrix`, `hive-ci`) are accepted. `hive-gateway` is
|
(`hive-gateway`, `hive-forge`, `hive-matrix`, `hive-ci`) are accepted.
|
||||||
a host unit, not a container, so it is not in this list — see
|
|
||||||
`ReloadGatewayNginx` above for how its access is scoped instead.
|
|
||||||
Arbitrary container names are rejected.
|
Arbitrary container names are rejected.
|
||||||
|
|
||||||
**Socket-activated** — systemd starts `hive-priv` on the first
|
**Socket-activated** — systemd starts `hive-priv` on the first
|
||||||
|
|
|
||||||
|
|
@ -183,10 +183,7 @@ the access-review list.
|
||||||
### What a snapshot contains
|
### What a snapshot contains
|
||||||
|
|
||||||
The snapshot covers an agent's **state subvolume**, which is the parent
|
The snapshot covers an agent's **state subvolume**, which is the parent
|
||||||
of `state/`, `claude/` and `harness/` (see
|
of `state/`, `claude/` and `harness/`. Consequences:
|
||||||
[`docs/persistence.md`'s btrfs subvolume
|
|
||||||
section](persistence.md#btrfs-subvolumes-for-varlibhyperhiveagentsname)
|
|
||||||
for how and when that subvolume is created). Consequences:
|
|
||||||
|
|
||||||
- The Claude session (`claude/`) travels, so a restored agent keeps its
|
- The Claude session (`claude/`) travels, so a restored agent keeps its
|
||||||
live `--continue` session rather than needing to log in again.
|
live `--continue` session rather than needing to log in again.
|
||||||
|
|
|
||||||
|
|
@ -73,11 +73,14 @@ and `qualify()` / `qualified_label()` semantics.
|
||||||
|
|
||||||
## Swarm CA
|
## Swarm CA
|
||||||
|
|
||||||
A hive's internal TLS chains to a **swarm root CA**, so a peer that
|
A hive's internal TLS chains to a **swarm root CA**: the root signs each
|
||||||
trusts the root validates every hive in the swarm rather than being
|
hive's own CA, and that hive CA signs the gateway leaf, so a peer that
|
||||||
pinned to each one by hand. Provisioning modes, what to hand a peer
|
trusts the root once validates every hive in the swarm rather than being
|
||||||
(`trust-bundle.pem`, never `ca.pem`), the name constraints on a hive
|
pinned to each one by hand.
|
||||||
CA, and how an existing hive adopts the hierarchy: [`ca.md`](ca.md).
|
|
||||||
|
Provisioning modes, what to hand a peer (`trust-bundle.pem`, never
|
||||||
|
`ca.pem`), the name constraints on a hive CA, and how an existing hive
|
||||||
|
adopts the hierarchy: [`ca.md`](ca.md).
|
||||||
|
|
||||||
## Running the swarm's shared services
|
## Running the swarm's shared services
|
||||||
|
|
||||||
|
|
@ -180,17 +183,17 @@ environment and forwarded to agent containers.
|
||||||
|
|
||||||
## What the config does at runtime
|
## What the config does at runtime
|
||||||
|
|
||||||
1. **Dashboard P33RS tab** — hive-c0re reads `HYPERHIVE_PEERS` and
|
1. **Dashboard P33RS tab** — `parse_peer_hives()` in `dashboard.rs`
|
||||||
surfaces it as the peer list in the dashboard's state API. The
|
reads `HYPERHIVE_PEERS` and includes
|
||||||
dashboard shows a P33RS tab (hidden when the list is empty) with a
|
`peer_hives: Vec<{ name, url }>` in `/api/state`. The dashboard
|
||||||
card per peer linking to `https://{domain}/`. Wire format + module
|
shows a P33RS tab (hidden when the list is empty) with a card per
|
||||||
pointer: `docs/web-ui/dashboard.md` § P33RS tab.
|
peer linking to `https://{domain}/`. See
|
||||||
|
`docs/web-ui/dashboard.md` § P33RS tab.
|
||||||
|
|
||||||
2. **Agent identity** — the same `HYPERHIVE_PEERS` env var is
|
2. **Agent identity** — the same `HYPERHIVE_PEERS` env var is
|
||||||
forwarded to agent containers, so agent code can discover peer
|
forwarded to agent containers by `meta.rs`; agent code can call
|
||||||
hives and address them with qualified names (`agent@domain`). See
|
`identity::peers()` to discover peer hives and address them with
|
||||||
`hive-agent/src/identity.rs`'s module doc for the label/domain
|
qualified names (`agent@domain`).
|
||||||
helpers.
|
|
||||||
|
|
||||||
3. **Matrix federation** — when `matrix.enable` is on, tuwunel
|
3. **Matrix federation** — when `matrix.enable` is on, tuwunel
|
||||||
federates with the peer's matrix server (discovered via the peer's
|
federates with the peer's matrix server (discovered via the peer's
|
||||||
|
|
|
||||||
|
|
@ -76,29 +76,26 @@ sign leaves, and the chain stops there.
|
||||||
|
|
||||||
## What to hand a peer
|
## What to hand a peer
|
||||||
|
|
||||||
`hivectl peer-config` prints the `services.hyperhive.swarm.hives."<name>"`
|
`hivectl peer-config` prints the `cp` line. The file is
|
||||||
block a peer operator pastes into their own config. When this hive's
|
`<tls.stateDir>/trust-bundle.pem` — the hive CA plus the swarm root —
|
||||||
gateway serves a self-signed leaf under the hierarchy (detected by the
|
**not `ca.pem`**.
|
||||||
presence of `<tls.stateDir>/trust-bundle.pem`), it also prints a one-time
|
|
||||||
`scp` line installing the **swarm root** —
|
|
||||||
`<swarm.ca.stateDir>/root.pem`, not this hive's own CA — on the peer's
|
|
||||||
host:
|
|
||||||
|
|
||||||
```
|
The distinction is load-bearing rather than cosmetic: once a hive CA is
|
||||||
scp /var/lib/swarm-ca/root.pem <peer-host>:/var/lib/swarm-ca/root.pem
|
an intermediate, it is no longer something a verifier can build a chain
|
||||||
```
|
*to*. OpenSSL will not terminate a chain at a trusted non-self-signed
|
||||||
|
certificate without `-partial_chain`, so handing a peer the bare
|
||||||
|
intermediate produces a verification failure that reads like a bad cert
|
||||||
|
rather than like a missing anchor. The bundle carries both, so the same
|
||||||
|
file works whichever mode issued it.
|
||||||
|
|
||||||
That is the point of the hierarchy: the root is installed **once per
|
On a hive that predates the swarm root the bundle is just that hive's
|
||||||
swarm host**, not once per peer, so a hive joining later needs no edit on
|
self-signed CA, so the recipe does not change.
|
||||||
the hives already running. A hive whose cert already chains to a public
|
|
||||||
CA has nothing to install — `peer-config` says so instead.
|
|
||||||
|
|
||||||
Handing a peer this hive's own `ca.pem` would not work even as a
|
⚠️ Consumers must read `trust-bundle.pem`, never `ca.pem` directly. A
|
||||||
one-off: once a hive CA is an intermediate under the swarm root, it is
|
consumer that reads `ca.pem` works fine on a hive that has always been
|
||||||
no longer something a verifier can build a chain *to* — OpenSSL will not
|
self-signed and breaks the moment that hive adopts the hierarchy — so
|
||||||
terminate a chain at a trusted non-self-signed certificate without
|
the failure is invisible on the deployment you are most likely to test
|
||||||
`-partial_chain`. That is why the tool distributes the root, not a
|
on.
|
||||||
per-hive file.
|
|
||||||
|
|
||||||
## Adopting the hierarchy on an existing hive
|
## Adopting the hierarchy on an existing hive
|
||||||
|
|
||||||
|
|
@ -179,10 +176,12 @@ Two consumers, and only one of them is fine:
|
||||||
|
|
||||||
The consumption differs per runtime and is the part worth knowing.
|
The consumption differs per runtime and is the part worth knowing.
|
||||||
tuwunel links no openssl, which makes `SSL_CERT_FILE` look inapplicable
|
tuwunel links no openssl, which makes `SSL_CERT_FILE` look inapplicable
|
||||||
— it isn't: its rustls-based TLS stack still resolves trust through the
|
— it isn't. Its outbound client is `reqwest` with the `rustls` feature,
|
||||||
platform certificate store on Linux, and that store honors
|
which builds a `rustls_platform_verifier::Verifier`; because tuwunel
|
||||||
`SSL_CERT_FILE`, so the env var takes effect the same way it would for
|
calls `tls_certs_merge` (additive) rather than `tls_certs_only`, the
|
||||||
an OpenSSL-linked binary.
|
platform roots stay alongside its compiled-in webpki set. On Linux that
|
||||||
|
verifier resolves through `rustls-native-certs` → `openssl-probe`,
|
||||||
|
which reads `SSL_CERT_FILE`.
|
||||||
|
|
||||||
> ⚠️ **Concatenate; never point `SSL_CERT_FILE` at the anchor alone.**
|
> ⚠️ **Concatenate; never point `SSL_CERT_FILE` at the anchor alone.**
|
||||||
> `openssl-probe` uses it *instead of* the default store, so naming
|
> `openssl-probe` uses it *instead of* the default store, so naming
|
||||||
|
|
|
||||||
|
|
@ -30,7 +30,7 @@ and has no `enable` to derive from anything.
|
||||||
### SSO (authelia)
|
### SSO (authelia)
|
||||||
|
|
||||||
One authelia per swarm, in a `swarm-authelia` container, at
|
One authelia per swarm, in a `swarm-authelia` container, at
|
||||||
`auth.<swarm-domain>`. Operator and agents are both subjects of the same
|
`auth.<hive-domain>`. Operator and agents are both subjects of the same
|
||||||
provider, differentiated by roles and claims rather than by mechanism —
|
provider, differentiated by roles and claims rather than by mechanism —
|
||||||
there is one IdP and one auth path.
|
there is one IdP and one auth path.
|
||||||
|
|
||||||
|
|
@ -57,7 +57,3 @@ small-deployment choices, and the scope is the justification: redis
|
||||||
buys shared session state across replicas and there is one instance;
|
buys shared session state across replicas and there is one instance;
|
||||||
SMTP exists to mail humans, and provisioning here is programmatic.
|
SMTP exists to mail humans, and provisioning here is programmatic.
|
||||||
|
|
||||||
See [`sso.md`](sso.md) for bootstrapping the first user and the OIDC
|
|
||||||
relying-party flow, and [`secrets.md`](secrets.md) for where each of
|
|
||||||
authelia's keys is generated and read.
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -91,12 +91,22 @@ an attribute edit can invalidate a login by accident.
|
||||||
## What secrets exist, and where each one lives
|
## What secrets exist, and where each one lives
|
||||||
|
|
||||||
Every secret in the swarm, with its generator and its path, is tabulated
|
Every secret in the swarm, with its generator and its path, is tabulated
|
||||||
in one place: [`secrets.md`](secrets.md), including authelia's own keys
|
in one place: [`secrets.md`](secrets.md). The rows relevant here are
|
||||||
(session, JWT, storage-encryption, OIDC HMAC, OIDC issuer) and the two
|
authelia's own keys (session, JWT, storage-encryption, OIDC HMAC, OIDC
|
||||||
halves of each client secret. That page's two rules — a secret is always
|
issuer) plus the two halves of each client secret.
|
||||||
a path, never a value, and the generator and the reader are usually in
|
|
||||||
different containers — are why the client secret's plaintext half needs
|
What matters for this page is the shape rather than the paths. Authelia's
|
||||||
the delivery step below and the rest of authelia's keys don't.
|
own keys are generated **in-container**, because nothing outside that
|
||||||
|
container ever reads them — that is the test worth applying to any secret
|
||||||
|
added here. The plaintext half of a client secret is the one that fails
|
||||||
|
it: its reader lives in a different container, and that is the entire
|
||||||
|
reason a delivery step exists.
|
||||||
|
|
||||||
|
**None of it is ever written into a nix expression.** authelia's
|
||||||
|
`settings` are rendered into the nix store, which is world-readable and
|
||||||
|
permanent, so the client digest reaches authelia through `settingsFiles`
|
||||||
|
(merged at runtime) and every other secret through a `*File` option
|
||||||
|
carrying a path rather than a value.
|
||||||
|
|
||||||
## Getting the plaintext to the relying party
|
## Getting the plaintext to the relying party
|
||||||
|
|
||||||
|
|
|
||||||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Reference in a new issue