vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

archetypes: build an archetype agent from its archetype's module on the hive

Browse source 
The hive reads the agent-archetype key when it provisions and rebuilds an
agent and keeps it in swarm-agent-archetypes.json. For an archetype agent
the meta flake has no agent-<name> input and mkAgent imports
hyperhive.hyperhiveArchetypes.<name>.module; provisioning, rebuilds, the
meta-input cascade, a deploy's rev and the startup migration skip the
config repos. The harness gets HIVE_ARCHETYPE and the system prompt says
the agent has no config repo. The create form sends the picked archetype.
Adds the android-dev built-in.

Refs #5386
This commit is contained in:
atlas 2026-10-17 08:49:03 +02:00 • committed by mara
commit 553ed8376c
24 changed files with 557 additions and 92 deletions
Download patch file Download diff file Expand all files Collapse all files

7
docs/agent-lifecycle/agent-roster.md
View file

@ -106,13 +106,16 @@ state dirs work as for every other agent.
deploy log). Any agent holding it gets the same mounts.
The operator dashboard's `POST /api/meta-update` is the only path
that writes `flake.lock`.
- **Prompt** — identical for every container: `prompts/system.md`
- **Prompt** — identical for every container but one block (below):
`prompts/system.md`
carries `<!-- role:agent -->` / `<!-- role:manager -->` marker
blocks, and `prompt::render` filters for `"agent"` unconditionally,
manager included ("always `agent` role — there is only one role",
`hive-agent/src/prompt.rs`'s own module doc). The `role:manager`
blocks are dead in production, exercised only by a unit test
(`filter_role_blocks(SAMPLE, "manager")`).
(`filter_blocks(SAMPLE, "role", "manager")`). The one block that differs
is `<!-- config:repo -->` vs `<!-- config:archetype -->`: an agent built
from an [archetype](../swarm/archetypes.md) is told it has no config repo.
- **Tool allow-list** — the MCP tools claude sees come from the agent's
permissions alone, the same mechanism for every agent
([`docs/turn-loop/mcp.md`](../turn-loop/mcp.md)). Ruth's permissions

2
docs/agent-lifecycle/config-changes.md
View file

@ -65,6 +65,8 @@ Creating a brand-new agent is swarm-level (`swarmctl agent create`,
an agent. The swarm controller's `InitAgentConfigRepo` job seeds the
agent's config repo with a default `agent.nix` template, then asks the
target hive to deploy it.
An agent created from an [archetype](../swarm/archetypes.md) gets no
config repo: its hive builds it from the archetype's module.
Changing what the template seeded isn't a special case: like every
later change, it's a PR on that config repo, made from a clone and

6
docs/agent-lifecycle/persistence.md
View file

@ -420,6 +420,12 @@ reads it at every container start for `manage_root_agent`. Each agent gets
its own entry from it as `swarm-permissions.json` in its runtime dir, which
it holds until its own first read of its key succeeds.
`/var/lib/hyperhive/swarm-agent-archetypes.json` holds the archetype each of
this hive's archetype agents runs, as last read from the swarm's
`agent-archetype` bucket at the agent's provision and each rebuild. An agent
with an entry has no `applied/<n>` repo and no `agent-<n>` meta input; see
[archetypes](../swarm/archetypes.md).
hive-c0re renders containers onto `meta#<n>` at creation.
## Destroy vs purge

62
docs/swarm/archetypes.md
View file

@ -1,21 +1,24 @@
# Agent archetypes
An archetype is a named set of settings to create an agent from. The swarm
UI's create form lists the archetypes the swarm knows. Picking one fills the
form's permissions, resource caps and inference fields from it, and you can
still change each field before you create the agent.
An archetype is a named agent setup: settings to create the agent with, and
a NixOS module its container runs. The swarm UI's create form lists the
archetypes the swarm knows. Picking one fills the form's permissions,
resource caps and inference fields from it, and you can still change each
field before you create the agent.
The agent you create this way is an ordinary agent: it gets its own
`agent-configs/<name>` repo like any other, and the request carries only the
fields as the form shows them. Nothing records which archetype you picked.
An agent created from an archetype has no `agent-configs/<name>` repo of its
own. Its hive builds it from the archetype's module, so every agent of that
archetype runs the same configuration. To change what it runs, change the
archetype. Picking "none" creates a custom agent, with its own config repo.
## The built-in archetypes
The hyperhive flake ships these:
| archetype | what it sets |
| ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
| ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
| `reviewer` | permissions `messaging`, `inbox`, `execution`: the default set without `meta`. With `execution` it can run `hive-forge`, `git` and other commands. |
| `android-dev` | installs `android-tools` (`adb`, `fastboot`), Gradle and a JDK; default permissions. |
## Where an archetype comes from
@ -38,8 +41,8 @@ hyperhiveArchetypes.reviewer = {
- Every archetype needs a `description`. `permissions`, `limits` and
`inference` are optional and take the shapes `POST /api/agents` takes. The swarm ignores
other keys in `meta`.
- `module` is a NixOS module for the agent's container. The swarm doesn't
apply it yet.
- `module` is a NixOS module for the agent's container. A hive imports it
into each agent created from the archetype.
Today the swarm reads one flake: the hyperhive source the
[builder](builder.md)'s host runs, so only the built-ins appear.
@ -85,3 +88,42 @@ The stored value:
"refused": { "broken": "limits: …" }
}
```
## Creating an agent from an archetype
The create form sends the picked archetype as `archetype` on
`POST /api/agents`, `{ "source": "hyperhive", "name": "reviewer" }`. The
controller refuses with 400 an archetype the stored list doesn't hold, then:
1. stores the choice in the `agent-archetype` KV bucket, keyed by agent name,
next to the agent's permissions, caps and inference;
2. queues the creation graph without `CreateRepo`, `AddRepoMember` and
`InitAgentConfigRepo`. Identity, forge user, docs read, the mints and the
wanted state stay, and the deploy waits on those alone.
Destroying the agent deletes its key.
Every hive may read any key of the bucket
(`$JS.API.DIRECT.GET.KV_agent-archetype.>`) and write none; only the
controller writes it.
The hive reads the agent's key when it provisions the agent and on every
rebuild, and keeps the last value read in
`/var/lib/hyperhive/swarm-agent-archetypes.json`, so the meta flake renders
without the queue. When the read fails, a rebuild keeps the value
stored there. For an archetype agent the hive:
- seeds no `proposed` or `applied` repo, and gives the meta flake no
`agent-<name>` input. `mkAgent` imports
`hyperhive.hyperhiveArchetypes.<name>.module` in place of the agent's own
flake, and sets `HIVE_ARCHETYPE` in the harness environment;
- relocks no agent input on a rebuild or a meta-input bump, and ignores the
`rev` of a swarm deploy request;
- mounts an empty `/agents/<name>/config/`.
The agent's system prompt says it has no config repo and names its
archetype.
The hive takes the module from its own `hyperhive` input. When that input
predates the archetype, the agent's build fails with
`this hive's hyperhive input has no archetype <name>`. Update the hive's
`hyperhive` input to fix it.

20
docs/turn-loop/claude-invocation.md
View file

@ -255,7 +255,12 @@ needs to `chown` a bind mount), once at startup:
`hive-agent/prompts/system.md` by `hive-agent`'s `prompt::render`:
HTML-comment markers (`<!-- role:agent -->...<!-- /role:agent -->`,
same for `role:manager`) gate the role-specific blocks; both flavors
share everything else. Five placeholders are then
share everything else. `<!-- config:repo -->` and
`<!-- config:archetype -->` blocks gate the config-repo paragraphs: an
agent with `HIVE_ARCHETYPE` set (the meta flake sets it on an agent built
from an [archetype](../swarm/archetypes.md)) gets the archetype block and
every other agent the repo block, with `{archetype}` filled in with the
archetype's name. Five placeholders are then
substituted: `{label}` (short agent name), `{qualified_label}`
(hive-qualified `name@domain` form),
`{hive_identity}` (for example `` on hive `pr1ma` ``; empty when
@ -275,14 +280,17 @@ needs to `chown` a bind mount), once at startup:
telling the agent where the docs live.
Passed via `--system-prompt-file`.
**Marker grammar.** `<!-- role:X -->` opens a block; any
`<!-- /role:X -->` closes the current block. The renderer always uses
role `agent`, so it elides blocks with other role tags. Nesting **isn't**
**Marker grammar.** `<!-- KIND:X -->` opens a block; any
`<!-- /KIND:X -->` closes the current block of that kind. The renderer
filters the two kinds one after the other. The renderer always uses
role `agent`, so it elides blocks with other role tags, and config `repo`
or `archetype` as above. Nesting **isn't**
supported — a stray opener with no closer runs until end of file.
The renderer tolerates whitespace inside markers (`<!--role:foo-->` parses the
same as `<!-- role:foo -->`). Content outside any marker is always
included. Today's `system.md` carries no markers (single agent role) —
the grammar stays wired for a future manager / multi-role prompt.
included. Today's `system.md` carries no role markers (single agent
role), and the role grammar stays wired for a future manager / multi-role
prompt; it carries one pair of config blocks.
**`hive_identity` / `swarm_identity` shape.** Each carries a
leading space + backticked name (` on hive \`pr1ma\``,

10
frontend/packages/swarm-ui/src/pages/CreateAgentArchetype.ts
View file

@ -61,3 +61,13 @@ export function prefill(
},
};
}
/**
* The `archetype` field of `POST /api/agents` for `archetype`, or nothing for
* a custom agent.
*/
export function requestArchetype(
archetype: Archetype | undefined,
): { source: string; name: string } | undefined {
return archetype && { source: archetype.source, name: archetype.name };
}

10
frontend/packages/swarm-ui/src/pages/CreateAgentForm.tsx
View file

@ -57,6 +57,7 @@ import {
import {
archetypeKey,
prefill,
requestArchetype,
type ArchetypeList,
} from "./CreateAgentArchetype.js";
import "./CreateAgentForm.css";
@ -230,6 +231,7 @@ export function CreateAgentForm({
permissions: catalogue
? requestPermissions(selected, catalogue.default)
: undefined,
archetype: requestArchetype(picked),
}),
});
if (!r.ok) {
@ -321,7 +323,9 @@ export function CreateAgentForm({
/>
{picked && (
<p class="create-agent-archetype-description">
{picked.description}
{picked.description} An agent created from an archetype has
no config repo: its hive builds it from the archetype's
module.
</p>
)}
</>
@ -396,7 +400,9 @@ export function CreateAgentForm({
Submitting this queues everything a new agent needs: a swarm-level
identity, a config repo on the forge with the operator added as a
collaborator, and a container running the agent on the hive you pick
above. Watch it all settle on <Link href="/jobs">jobs</Link>.
above. An agent created from an archetype gets no config repo: its
container runs the archetype's module. Watch it all settle on{" "}
<Link href="/jobs">jobs</Link>.
</p>
</Panel>
</div>

20
frontend/packages/swarm-ui/test/create-agent-archetype.test.mjs
View file

@ -2,7 +2,11 @@
// Run by `npm test` with Node's type stripping.
import assert from "node:assert/strict";
import { test } from "node:test";
import { archetypeKey, prefill } from "../src/pages/CreateAgentArchetype.ts";
import {
archetypeKey,
prefill,
requestArchetype,
} from "../src/pages/CreateAgentArchetype.ts";
const defaults = ["messaging", "meta", "inbox", "execution"];
@ -65,3 +69,17 @@ test("the key tells sources apart", () => {
archetypeKey({ source: "b", name: "x", description: "" }),
);
});
test("a picked archetype is sent by source and name, none is left out", () => {
const reviewer = {
source: "hyperhive",
name: "reviewer",
description: "reviews",
permissions: ["messaging"],
};
assert.deepEqual(requestArchetype(reviewer), {
source: "hyperhive",
name: "reviewer",
});
assert.equal(requestArchetype(undefined), undefined);
});

9
hive-agent/prompts/system.md
View file

@ -5,10 +5,19 @@ Tools (hyperhive surface). Full signature + behavior for each comes from the too
- **Inbox / messaging** (always available): `mcp__hyperhive__recv`, `mcp__hyperhive__ack_until`, `mcp__hyperhive__send`, `mcp__hyperhive__get_loose_ends`, `mcp__hyperhive__cancel_loose_end`, `mcp__hyperhive__mark_todos_done`, `mcp__hyperhive__remind`, `mcp__hyperhive__set_status`, `mcp__hyperhive__get_agent_meta`. `send` reaches peers on this hive only; **reach the operator over matrix** if you have the matrix MCP tools (`mcp__matrix__send_dm` or `mcp__matrix__send_message`); without them the operator reads your `set_status` line and turn output on the dashboard, and your logs. `send(to: "operator")` is refused. One habit worth internalizing beyond the tool descriptions themselves: prefer ending the turn over repeatedly polling `recv` when idle (only turn-boundaries observe in-container todo wakes — bash-task completions, matrix unread, forge activity — and ending the turn is also your checkpoint). For a large todo backlog (`get_loose_ends` caps at 40 rows), clear reviewed ids in bulk with `mark_todos_done` rather than cancelling one at a time — there's no blind range-clear, only ids you've actually looked at.
- **Extra MCP tools** (some agents only): `mcp__<server>__<tool>` — agent-specific (matrix client, scraper, db connector, etc.) declared in your `agent.nix` under `services.hyperhive.agent.extraMcpServers`. First-class tools, already operator-approved at deploy time.
<!-- config:repo -->
Need new packages, env vars, or other NixOS config for yourself? You can't edit your own config directly — open a PR on your `agent-configs/<name>` repo, or contact the operator directly. Config repos live at `/agents/{label}/config/` (read-only inside your container). All changes flow through operator-approved commits.
Your config repo is mounted **read-only** at `/agents/{label}/config/` — `agent.nix` plus whatever extra files define you (declared packages, env vars, MCP servers). Read it to see exactly what defines you before asking for a change, so you can point at the precise file and line.
<!-- /config:repo -->
<!-- config:archetype -->
You are built from the `{archetype}` archetype and have no config repo of your own: there is no `agent-configs/<name>` repo for you, and `/agents/{label}/config/` is empty. Need new packages, env vars, or other NixOS config for yourself? Ask the operator: what defines you is the archetype, shared with every agent built from it.
<!-- /config:archetype -->
Operator boundary: starting, stopping and rebuilding containers is the operator's — ask them when any agent needs one. _Creating_ a new agent is not something you can do from here — ask the operator, who scaffolds the new agent's config repo and spawns it from the dashboard. _Changing_ any agent's config is not a tool call at all — it's a forge PR on the agent's `agent-configs/<name>` repo, which an operator reviews and merges on the forge; the merge deploys it. The operator only signs off on changes; you run the day-to-day.
Messages from sender `system` are hyperhive helper events (JSON body, `event` field discriminates): `container_crash`, `needs_update`. Use these to react to lifecycle changes:

161
hive-agent/src/prompt.rs
View file

@ -1,6 +1,6 @@
//! System-prompt renderer. Single `prompts/system.md` with
//! HTML-comment markers gating role-specific blocks; this module
//! assembles the final prompt (always "agent" role — there is only one
//! HTML-comment markers gating role-specific and config-repo-specific
//! blocks; this module assembles the final prompt (always "agent" role — there is only one
//! role). Marker grammar + placeholder substitution rules in
//! `docs/turn-loop/claude-invocation.md::On-boot files` (`claude-system-prompt.md`).
@ -32,7 +32,7 @@ pub fn render(
docs_dir: Option<&str>,
operators: &[OperatorProfile],
) -> String {
let body = filter_role_blocks(template, "agent");
let body = filter_blocks(template, "role", "agent");
let qualified = crate::identity::qualify(label);
let hive_identity = hive_name
.filter(|n| !n.is_empty())
@ -118,27 +118,27 @@ async fn within_timeout<E: std::fmt::Display>(
}
}
/// Walk `template` line-by-line, stripping `<!-- role:X -->` /
/// `<!-- /role:X -->` marker lines and suppressing the lines inside a block
/// unless `X == target`. A close marker ends the current block. Production
/// `system.md` carries no markers today (single agent role — see the module
/// doc), so this is effectively a passthrough; the marker grammar stays wired
/// for a future manager / multi-role prompt.
fn filter_role_blocks(template: &str, target: &str) -> String {
/// Walk `template` line-by-line, stripping `<!-- KIND:X -->` /
/// `<!-- /KIND:X -->` marker lines and suppressing the lines inside a block
/// unless `X == target`. A close marker ends the current block. Markers of
/// any other kind are left in place, for the pass that filters that kind.
/// Two kinds exist: `role` (always `agent`; the grammar stays wired for a
/// future manager / multi-role prompt) and `config` ([`config_blocks`]).
fn filter_blocks(template: &str, kind: &str, target: &str) -> String {
let mut out = String::with_capacity(template.len());
// None = outside any block; Some(role) = inside role-tagged block.
let mut active_role: Option<&str> = None;
// None = outside any block; Some(value) = inside a block of this kind.
let mut active: Option<&str> = None;
for line in template.lines() {
let trimmed = line.trim();
if let Some(role) = parse_open_marker(trimmed) {
active_role = Some(role);
if let Some(value) = parse_open_marker(trimmed, kind) {
active = Some(value);
continue;
}
if parse_close_marker(trimmed).is_some() {
active_role = None;
if parse_close_marker(trimmed, kind).is_some() {
active = None;
continue;
}
if active_role.is_none_or(|role| role == target) {
if active.is_none_or(|value| value == target) {
out.push_str(line);
out.push('\n');
}
@ -146,23 +146,37 @@ fn filter_role_blocks(template: &str, target: &str) -> String {
out
}
/// `<!-- role:agent -->` → `Some("agent")`. Anything else returns
/// None. Whitespace inside the marker is tolerated so a future
/// author's `<!--role:foo-->` (no spaces) still parses; the dashboard
/// markdown renderer is equally lenient. Close tags (`/role:...`)
/// can't accidentally match — the `strip_prefix("role:")` rejects
/// the leading slash before we'd ever see it.
fn parse_open_marker(line: &str) -> Option<&str> {
let inside = line.strip_prefix("<!--")?.strip_suffix("-->")?.trim();
let role = inside.strip_prefix("role:")?.trim();
Some(role)
/// `template` with its `config:repo` blocks for a custom agent, or its
/// `config:archetype` blocks and `{archetype}` filled in for an agent built
/// from `archetype`. An empty name is a custom agent.
fn config_blocks(template: &str, archetype: Option<&str>) -> String {
match archetype.filter(|a| !a.is_empty()) {
Some(name) => filter_blocks(template, "config", "archetype").replace("{archetype}", name),
None => filter_blocks(template, "config", "repo"),
}
}
/// `<!-- /role:agent -->` → `Some("agent")`. Mirror of
/// [`parse_open_marker`] for the closing tag.
fn parse_close_marker(line: &str) -> Option<&str> {
/// `<!-- role:agent -->` with `kind = "role"` → `Some("agent")`. Anything
/// else returns None. Whitespace inside the marker is tolerated so a future
/// author's `<!--role:foo-->` (no spaces) still parses; the dashboard
/// markdown renderer is equally lenient. Close tags (`/role:...`) can't
/// accidentally match — the `strip_prefix(kind)` rejects the leading slash
/// before we'd ever see it.
fn parse_open_marker<'a>(line: &'a str, kind: &str) -> Option<&'a str> {
let inside = line.strip_prefix("<!--")?.strip_suffix("-->")?.trim();
inside.strip_prefix("/role:").map(str::trim)
let value = inside.strip_prefix(kind)?.strip_prefix(':')?.trim();
Some(value)
}
/// `<!-- /role:agent -->` with `kind = "role"` → `Some("agent")`. Mirror of
/// [`parse_open_marker`] for the closing tag.
fn parse_close_marker<'a>(line: &'a str, kind: &str) -> Option<&'a str> {
let inside = line.strip_prefix("<!--")?.strip_suffix("-->")?.trim();
inside
.strip_prefix('/')?
.strip_prefix(kind)?
.strip_prefix(':')
.map(str::trim)
}
/// Write the assembled prompt to a stable path next to the harness
@ -196,6 +210,9 @@ pub async fn write_system_prompt(_socket: &Path, label: &str) -> Result<PathBuf>
let docs_dir = std::env::var("HIVE_DOCS_DIR")
.ok()
.filter(|d| !d.is_empty());
// The meta flake sets HIVE_ARCHETYPE on an agent built from an archetype,
// which has no config repo of its own.
let template = config_blocks(&template, std::env::var("HIVE_ARCHETYPE").ok().as_deref());
let body = render(
&template,
label,
@ -257,7 +274,7 @@ shared closer
#[test]
fn filter_keeps_shared_and_target_role() {
let agent = filter_role_blocks(SAMPLE, "agent");
let agent = filter_blocks(SAMPLE, "role", "agent");
assert!(agent.contains("shared opener"));
assert!(agent.contains("agent-only line"));
assert!(!agent.contains("manager-only line"));
@ -268,7 +285,7 @@ shared closer
#[test]
fn filter_for_manager_picks_manager_block() {
let manager = filter_role_blocks(SAMPLE, "manager");
let manager = filter_blocks(SAMPLE, "role", "manager");
assert!(manager.contains("shared opener"));
assert!(!manager.contains("agent-only line"));
assert!(manager.contains("manager-only line"));
@ -278,23 +295,38 @@ shared closer
#[test]
fn parse_open_marker_handles_whitespace_variants() {
assert_eq!(parse_open_marker("<!-- role:agent -->"), Some("agent"));
assert_eq!(parse_open_marker("<!--role:agent-->"), Some("agent"));
assert_eq!(parse_open_marker("<!-- role:manager -->"), Some("manager"));
assert_eq!(
parse_open_marker("<!-- role:agent -->", "role"),
Some("agent")
);
assert_eq!(
parse_open_marker("<!--role:agent-->", "role"),
Some("agent")
);
assert_eq!(
parse_open_marker("<!-- role:manager -->", "role"),
Some("manager")
);
// Close tags must NOT match open-tag parser.
assert_eq!(parse_open_marker("<!-- /role:agent -->"), None);
assert_eq!(parse_open_marker("<!-- /role:agent -->", "role"), None);
// Non-markers pass through (return None).
assert_eq!(parse_open_marker("just text"), None);
assert_eq!(parse_open_marker("<!-- not a role -->"), None);
assert_eq!(parse_open_marker("just text", "role"), None);
assert_eq!(parse_open_marker("<!-- not a role -->", "role"), None);
}
#[test]
fn parse_close_marker_handles_whitespace_variants() {
assert_eq!(parse_close_marker("<!-- /role:agent -->"), Some("agent"));
assert_eq!(parse_close_marker("<!--/role:manager-->"), Some("manager"));
assert_eq!(
parse_close_marker("<!-- /role:agent -->", "role"),
Some("agent")
);
assert_eq!(
parse_close_marker("<!--/role:manager-->", "role"),
Some("manager")
);
// Open tags must NOT match close-tag parser.
assert_eq!(parse_close_marker("<!-- role:agent -->"), None);
assert_eq!(parse_close_marker("just text"), None);
assert_eq!(parse_close_marker("<!-- role:agent -->", "role"), None);
assert_eq!(parse_close_marker("just text", "role"), None);
}
#[test]
@ -331,6 +363,49 @@ shared closer
assert!(rendered.contains("`/agents/alice/work/`"), "{rendered}");
}
const CONFIG_FIXTURE: &str = "\
shared
<!-- config:repo -->
open a PR on agent-configs
<!-- /config:repo -->
<!-- config:archetype -->
built from `{archetype}`, no config repo
<!-- /config:archetype -->
";
/// A custom agent gets the config-repo lines, an archetype agent the
/// archetype lines with its name, and an empty name is a custom agent.
#[test]
fn config_blocks_pick_the_block_by_archetype() {
for custom in [None, Some("")] {
let picked = config_blocks(CONFIG_FIXTURE, custom);
assert!(picked.contains("open a PR on agent-configs"), "{picked}");
assert!(!picked.contains("no config repo"), "{picked}");
}
let picked = config_blocks(CONFIG_FIXTURE, Some("android-dev"));
assert!(picked.contains("built from `android-dev`, no config repo"));
assert!(!picked.contains("agent-configs"), "{picked}");
assert!(picked.contains("shared"));
assert!(!picked.contains("<!--"), "{picked}");
}
#[test]
fn the_production_prompt_keeps_no_config_markers() {
for archetype in [None, Some("reviewer")] {
let rendered = render(
&config_blocks(&PRODUCTION_TEMPLATE, archetype),
"alice",
None,
None,
None,
&[],
);
assert!(!rendered.contains("<!-- config:"), "{rendered}");
assert!(!rendered.contains("<!-- /config:"), "{rendered}");
assert!(!rendered.contains("{archetype}"), "{rendered}");
}
}
// Inline fixture for the {hive_identity} / {swarm_identity}
// placeholders. Cargo's `cargo test` resolves `PRODUCTION_TEMPLATE`
// against `$HIVE_ASSETS_DIR/prompts/system.md`, which the flake

1
hive-c0re/Cargo.toml
View file

@ -59,6 +59,7 @@ swarm-queue-client = { workspace = true, features = [
"notices",
"agent-limits",
"agent-permissions",
"agent-archetype",
] }
swarm-secret-client.workspace = true
tokio.workspace = true

143
hive-c0re/src/agent_config/archetypes.rs Normal file
View file

@ -0,0 +1,143 @@
//! The archetype each of this hive's archetype agents is built from, as last
//! read from the swarm's `agent-archetype` bucket
//! ([`swarm_queue_client::agent_archetype`]). Stored at
//! `/var/lib/hyperhive/swarm-agent-archetypes.json`, outside the meta repo:
//! the meta render runs at boot, possibly before the queue answers, so it
//! reads this file rather than the bucket.
//!
//! Format: agent name → [`ArchetypeRef`]. An agent with an entry has no config
//! repo: the meta flake builds it from the archetype's module, and the paths
//! that seed, relock or advance `applied/<name>` skip it. An agent with no
//! entry is a custom agent.
//!
//! [`refresh`] reads one agent's key; provisioning and every rebuild's meta
//! sync call it before they render the meta flake.
use std::collections::BTreeMap;
use std::path::{Path, PathBuf};
use anyhow::{Context, Result};
use swarm_queue_client::agent_archetype::{self, ArchetypeRef, Read};
const FILE: &str = "swarm-agent-archetypes.json";
pub type Stored = BTreeMap<String, ArchetypeRef>;
#[must_use]
pub fn path() -> PathBuf {
crate::paths::state_root().join(FILE)
}
/// Read the stored archetypes. An absent file is the empty map: every agent
/// custom. A file that exists but can't be read or parsed is an error.
pub fn read_at(path: &Path) -> std::io::Result<Stored> {
super::read_map(path)
}
/// `agent`'s archetype per the file at [`path`], `None` for a custom agent.
///
/// # Errors
///
/// As [`read_at`].
pub fn of(agent: &str) -> std::io::Result<Option<ArchetypeRef>> {
Ok(read_at(&path())?.remove(agent))
}
/// Read `agent`'s key and store what it holds. No queue, or a bucket that
/// does not answer, leaves the stored entry as it is.
///
/// # Errors
///
/// The queue is configured but not connected, the read failed, the value
/// names a source or archetype that is not an identifier, or the file could
/// not be read or written.
pub async fn refresh(agent: &str) -> Result<()> {
let Some(client) = crate::swarm_queue::client().await else {
return Ok(());
};
// An unconnected client does not fail a JetStream request, it hangs on it.
swarm_queue_client::ensure_connected(&client)?;
let read = agent_archetype::read(&client, agent)
.await
.map_err(|e| anyhow::anyhow!(e))
.with_context(|| format!("reading the swarm's archetype for {agent}"))?;
store_read(&path(), agent, read)
}
/// Store `read`, the value of `agent`'s key, in the file at `path`. Written
/// only when it changes the entry.
fn store_read(path: &Path, agent: &str, read: Read) -> Result<()> {
let value = match read {
Read::NoBucket => {
tracing::debug!(%agent, "archetype: the agent-archetype bucket does not answer");
return Ok(());
}
Read::Current(value) => value,
};
// Both are rendered into the meta flake as nix strings.
if let Some(ArchetypeRef { source, name }) = &value {
for part in [source, name] {
hive_types::Ident::parse(part)
.map_err(|why| anyhow::anyhow!("archetype {part:?} of {agent}: {why}"))?;
}
}
let mut stored = read_at(path).context("reading the stored archetypes")?;
let changed = match value {
Some(value) => stored.insert(agent.to_owned(), value.clone()) != Some(value),
None => stored.remove(agent).is_some(),
};
if changed {
super::write_map(path, &stored).context("storing the archetypes")?;
tracing::info!(%agent, archetype = ?stored.get(agent), "archetype: stored the swarm's archetype");
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
fn reviewer() -> ArchetypeRef {
ArchetypeRef {
source: "hyperhive".to_owned(),
name: "reviewer".to_owned(),
}
}
#[test]
fn a_read_sets_or_clears_the_entry_and_no_bucket_keeps_it() {
let dir = tempfile::tempdir().expect("tempdir");
let path = dir.path().join(FILE);
assert!(read_at(&path).expect("missing is empty").is_empty());
store_read(&path, "atlas", Read::Current(Some(reviewer()))).expect("store");
store_read(&path, "iris", Read::Current(None)).expect("store");
assert_eq!(
read_at(&path).expect("read"),
Stored::from([("atlas".to_owned(), reviewer())])
);
store_read(&path, "atlas", Read::NoBucket).expect("store");
assert_eq!(
read_at(&path).expect("read").get("atlas"),
Some(&reviewer())
);
store_read(&path, "atlas", Read::Current(None)).expect("store");
assert!(read_at(&path).expect("read").is_empty());
}
/// The value is rendered into nix, so a name that is not an identifier is
/// refused and nothing is stored.
#[test]
fn a_name_that_is_not_an_identifier_is_refused() {
let dir = tempfile::tempdir().expect("tempdir");
let path = dir.path().join(FILE);
let bad = ArchetypeRef {
source: "hyperhive".to_owned(),
name: "x\"; y".to_owned(),
};
store_read(&path, "atlas", Read::Current(Some(bad))).expect_err("refused");
assert!(read_at(&path).expect("read").is_empty());
}
}

9
hive-c0re/src/agent_config/mod.rs
View file

@ -1,13 +1,14 @@
//! Per-agent configuration registries: resource limits, topology (JSON files
//! under `/var/lib/hyperhive/meta/`), the permissions the swarm grants
//! (outside the meta repo) and the shared wire-protocol size limits. Each
//! submodule is re-exported at the crate root, so `crate::topology::…` etc.
//! keep working unchanged.
//! under `/var/lib/hyperhive/meta/`), the permissions and archetypes the
//! swarm sets (outside the meta repo) and the shared wire-protocol size
//! limits. Each submodule is re-exported at the crate root, so
//! `crate::topology::…` etc. keep working unchanged.
//!
//! Note the two similarly-named modules: [`limits`] caps inline
//! *message body* sizes on the sockets, while [`resource_limits`] holds
//! per-agent CPU/memory caps for the container drop-in.
pub mod archetypes;
pub mod limits;
pub mod permissions;
pub mod resource_limits;

18
hive-c0re/src/job_queue/exec.rs
View file

@ -396,6 +396,11 @@ async fn run_meta_sync(coord: &Arc<Coordinator>, name: &str, relock: bool) -> Re
let agent_dir = crate::paths::agent_runtime_dir(name);
let hive = coord.hive_env();
let paths = Coordinator::agent_paths(name, agent_dir)?;
// A failed read keeps the archetype last stored, so the agent is built as
// it was.
if let Err(e) = crate::archetypes::refresh(name).await {
tracing::warn!(%name, error = ?e, "rebuild: reading the swarm's archetype failed; keeping the stored one");
}
crate::lifecycle::prepare_rebuild_dirs(name, &paths).await?;
// Idempotent meta sync so a manual rebuild can also recover from a
// divergent meta repo; then bump just this agent's input. `relock =
@ -403,7 +408,8 @@ async fn run_meta_sync(coord: &Arc<Coordinator>, name: &str, relock: bool) -> Re
// would revert the bump the cascade just committed.
let agents = crate::lifecycle::agents_for_meta_listing().await?;
crate::meta::sync_agents(&hive, &agents).await?;
if relock {
// An archetype agent has no `agent-<name>` input to relock.
if relock && crate::archetypes::of(name)?.is_none() {
crate::meta::lock_update_for_rebuild(name).await?;
}
Ok(())
@ -553,8 +559,16 @@ async fn run_meta_lock(
// against whatever `applied/<name>` already happened to be locked to —
// normally current, but silently stale forever if a past deploy failed
// and nothing since retried it.
// An archetype agent has no config repo and no `agent-<name>` input, so
// it is left out here; it is still rebuilt with the cascade.
let archetypes = crate::archetypes::read_at(&crate::archetypes::path())?;
let custom: Vec<String> = cascade
.iter()
.filter(|name| !archetypes.contains_key(*name))
.cloned()
.collect();
catch_up_cascade_agents(
&cascade,
&custom,
crate::forge::fast_forward_applied_main,
|agent_inputs| async move { crate::meta::lock_update(&agent_inputs).await },
)

11
hive-c0re/src/lifecycle/mod.rs
View file

@ -230,8 +230,13 @@ pub async fn provision_container(name: &str, hive: &HiveEnv, paths: &AgentPaths)
agent_web_port(name)
);
}
// Read before anything is seeded: an archetype agent gets no config
// repo, and the meta render below builds it from the archetype.
crate::archetypes::refresh(name).await?;
if crate::archetypes::of(name)?.is_none() {
setup_proposed(&paths.proposed, name).await?;
setup_applied(&paths.applied, Some(&paths.proposed), name).await?;
}
ensure_agent_state_subvolume(name).await?;
ensure_claude_dir(&paths.claude)?;
ensure_state_dir(&paths.notes)?;
@ -273,6 +278,8 @@ pub async fn prepare_rebuild_dirs(name: &str, paths: &AgentPaths) -> Result<()>
agent_web_port(name)
);
}
// An archetype agent has no applied repo to check or recover.
if crate::archetypes::of(name)?.is_none() {
if !paths.applied.join(".git").exists()
&& !crate::forge::fetch_config_main_into_applied(name).await
{
@ -284,6 +291,7 @@ pub async fn prepare_rebuild_dirs(name: &str, paths: &AgentPaths) -> Result<()>
);
}
setup_applied(&paths.applied, None, name).await?;
}
ensure_agent_state_subvolume(name).await?;
ensure_claude_dir(&paths.claude)?;
ensure_state_dir(&paths.notes)?;
@ -319,12 +327,14 @@ pub async fn swap_update(
/// handle the `Err` themselves with `.unwrap_or_default()`.
async fn agents_for_meta(name_to_add: Option<&str>) -> Result<Vec<crate::meta::AgentSpec>> {
let containers = list().await?;
let archetypes = crate::archetypes::read_at(&crate::archetypes::path())?;
let mut out: Vec<crate::meta::AgentSpec> = containers
.into_iter()
.filter_map(|c| {
let name = c.strip_prefix(AGENT_PREFIX)?.to_owned();
Some(crate::meta::AgentSpec {
port: agent_web_port(&name),
archetype: archetypes.get(&name).cloned(),
name,
})
})
@ -334,6 +344,7 @@ async fn agents_for_meta(name_to_add: Option<&str>) -> Result<Vec<crate::meta::A
{
out.push(crate::meta::AgentSpec {
port: agent_web_port(extra),
archetype: archetypes.get(extra).cloned(),
name: extra.to_owned(),
});
}

2
hive-c0re/src/main.rs
View file

@ -43,7 +43,7 @@ mod swarm_status;
mod test_env;
mod workers;
pub(crate) use agent_config::{limits, permissions, resource_limits, topology};
pub(crate) use agent_config::{archetypes, limits, permissions, resource_limits, topology};
pub(crate) use stats::{container_stats, hive_stats, host_stats, otel_metrics, warnings};
pub(crate) use stores::{broker, build_logs, db, power};
pub(crate) use workers::{agent_sockets, auto_update, crash_watch, knowledge, mcp_sockets};

85
hive-c0re/src/meta.rs
View file

@ -42,6 +42,9 @@ pub const CONTAINER_MANAGER_META_MOUNT: &str = "/meta";
pub struct AgentSpec {
pub name: String,
pub port: u16,
/// The archetype the agent is built from (`crate::archetypes`), `None`
/// for a custom agent built from its own config repo.
pub archetype: Option<swarm_queue_client::agent_archetype::ArchetypeRef>,
}
/// Stage topology.json if it exists: it is regenerated by `reconcile`
@ -786,6 +789,11 @@ where
// (config PRs land there) but are not the flake's build input. deploy +
// rebuild re-lock this input to `applied/<name>`'s current `main` head.
for spec in agents {
// An archetype agent has no config repo, so no input: `mkAgent`
// takes its module from the archetype.
if spec.archetype.is_some() {
continue;
}
let _ = writeln!(
out,
" agent-{name}.url = \"git+file://{applied}\";",
@ -811,17 +819,26 @@ where
out.push_str(" };\n outputs =\n { self, hyperhive, ... }@inputs:\n let\n");
let _ = writeln!(
out,
" dashboardPort = {dashboard_port};\n mkAgent = {{ name, port }}:"
" dashboardPort = {dashboard_port};\n mkAgent = {{ name, port, archetype ? null }}:"
);
out.push_str(
r#" let
base = hyperhive.nixosConfigurations.agent-base;
input = inputs."agent-${name}";
# Every flake an archetype can come from, by its source name.
archetypeSources = { inherit hyperhive; };
agentModule =
if archetype == null then
inputs."agent-${name}".nixosModules.default
else
(archetypeSources.${archetype.source}
or (throw "agent ${name}: archetype source ${archetype.source} is not an input of this hive's meta flake")
).hyperhiveArchetypes.${archetype.name}.module
or (throw "agent ${name}: this hive's ${archetype.source} input has no archetype ${archetype.name}; update that input");
service = "hive-agent";
in
base.extendModules {
modules = [
input.nixosModules.default
agentModule
{
"#,
);
@ -1008,6 +1025,10 @@ where
HYPERHIVE_HARNESS_DIR = "/agents/${name}/harness";
};
}
# Tells the harness the agent has no config repo of its own.
(if archetype == null then { } else {
systemd.services.${service}.environment.HIVE_ARCHETYPE = archetype.name;
})
];
};
in
@ -1016,9 +1037,17 @@ where
"#,
);
for spec in agents {
// `source` and `name` are identifiers (`archetypes::store_read`), so
// they need no escaping.
let archetype = spec.archetype.as_ref().map_or(String::new(), |a| {
format!(
" archetype = {{ source = \"{}\"; name = \"{}\"; }};",
a.source, a.name
)
});
let _ = writeln!(
out,
" {} = mkAgent {{ name = \"{}\"; port = {}; }};",
" {} = mkAgent {{ name = \"{}\"; port = {};{archetype} }};",
spec.name, spec.name, spec.port,
);
}
@ -1235,6 +1264,7 @@ mod tests {
AgentSpec {
name: name.to_owned(),
port,
archetype: None,
}
}
@ -1501,6 +1531,53 @@ mod tests {
);
}
/// An archetype agent has no `agent-<name>` input, and `mkAgent` is
/// handed its archetype; a custom agent beside it keeps its input.
#[test]
fn render_flake_builds_an_archetype_agent_from_its_archetype() {
let android = AgentSpec {
archetype: Some(swarm_queue_client::agent_archetype::ArchetypeRef {
source: "hyperhive".to_owned(),
name: "android-dev".to_owned(),
}),
..sample_spec("droid", 9002)
};
let out = render_flake_with_lookup(
"github:example/hyperhive",
"path:/nix/store/bbbb-hyperhive-docs-source",
"path:/nix/store/aaaa-nixpkgs-source",
None,
8000,
&[sample_spec("alice", 9001), android],
|_| vec!["nixpkgs"],
);
assert!(
!out.contains("agent-droid"),
"an archetype agent has no config input:\n{out}"
);
assert!(
out.contains(
"droid = mkAgent { name = \"droid\"; port = 9002; \
archetype = { source = \"hyperhive\"; name = \"android-dev\"; }; };"
),
"{out}"
);
assert!(out.contains("agent-alice.url = \"git+file://"), "{out}");
assert!(out.contains("agent-alice.inputs.nixpkgs.follows"), "{out}");
assert!(
out.contains("alice = mkAgent { name = \"alice\"; port = 9001; };"),
"{out}"
);
assert!(
out.contains("hyperhiveArchetypes.${archetype.name}.module"),
"{out}"
);
assert!(
out.contains("environment.HIVE_ARCHETYPE = archetype.name"),
"{out}"
);
}
#[test]
fn render_flake_forwards_env_into_global_environment() {
// Regression for the isolation breakage where forwarded vars

9
hive-c0re/src/migrate.rs
View file

@ -82,7 +82,16 @@ pub async fn run(coord: &Arc<Coordinator>) -> Result<()> {
// Phase 1 + 2: per-agent applied + proposed.
tracing::debug!("migration: phase 1+2 (applied + proposed repos)");
// An archetype agent has neither repo, and seeding one here would hand it
// a config it is not built from.
let archetypes = crate::archetypes::read_at(&crate::archetypes::path()).unwrap_or_else(|e| {
tracing::warn!(error = ?e, "migration: stored archetypes unreadable; treating every agent as custom");
crate::archetypes::Stored::new()
});
for name in &names {
if archetypes.contains_key(name.as_str()) {
continue;
}
tracing::debug!(%name, "migration: applied+proposed");
if let Err(e) = migrate_applied_repo(name.as_str()).await {
tracing::warn!(%name, error = ?e, "migration: applied repo rewrite failed");

9
hive-c0re/src/swarm_status.rs
View file

@ -298,7 +298,16 @@ async fn handle_deploy_request(
// Only a rebuild applies `rev`. A first deploy ignores it and builds the
// `applied` repo it seeds from the forge's `main`, or the one it finds.
// An archetype agent has no `applied` repo, so its rebuild ignores it too.
let archetype = match crate::archetypes::of(&agent) {
Ok(archetype) => archetype,
Err(e) => {
tracing::warn!(%agent, error = ?e, "swarm events: cannot read the stored archetypes; deploy dropped");
return;
}
};
if known
&& archetype.is_none()
&& let Some(rev) = rev.as_deref()
&& !apply_rev(&agent, rev).await
{

4
hive-sh4re/src/assets.rs
View file

@ -43,8 +43,8 @@ fn dir() -> PathBuf {
}
/// `$HIVE_ASSETS_DIR/prompts/system.md` — the claude system prompt
/// template. `hive-agent::prompt::render` filters the role markers
/// inside it per agent / manager flavor.
/// template. `hive-agent::prompt::render` filters the role and config markers
/// inside it per agent / manager flavor and per custom / archetype agent.
#[must_use]
pub fn prompt_template() -> PathBuf {
dir().join("prompts/system.md")

14
nix/archetypes/default.nix
View file

@ -17,4 +17,18 @@
];
};
};
android-dev = {
module =
{ pkgs, ... }:
{
environment.systemPackages = [
pkgs.android-tools
pkgs.gradle
pkgs.jdk
];
};
meta = {
description = "Builds Android apps: adb and fastboot (android-tools), Gradle and a JDK are installed. It holds the default permissions.";
};
};
}

7
nix/module-eval/builder.nix
View file

@ -151,7 +151,7 @@ let
# output and prints it as JSON. A derivation or a path in `meta` would
# make that evaluation build or copy something; a function would not
# print at all.
name = "the built-in archetypes carry a module and a meta of plain data, and reviewer can run commands";
name = "the built-in archetypes carry a module and a meta of plain data, and reviewer can run commands and android-dev installs its toolchain";
ok =
let
plain =
@ -173,7 +173,10 @@ let
&& lib.elem "messaging" reviewer
&& lib.elem "inbox" reviewer
&& lib.elem "execution" reviewer
&& !(lib.elem "meta" reviewer);
&& !(lib.elem "meta" reviewer)
&&
lib.elem pkgs.android-tools
(archetypes.android-dev.module { inherit pkgs; }).environment.systemPackages;
}
{
# `docs.enable` reads `docs.source`, whose default builds from the

3
swarm-controller/src/agent_archetype.rs
View file

@ -41,8 +41,7 @@ pub(crate) async fn check(
}
None => SourceRecord::default(),
};
listed(&record, choice)
.map_err(|why| error_problem(axum::http::StatusCode::BAD_REQUEST, &why))
listed(&record, choice).map_err(|why| error_problem(axum::http::StatusCode::BAD_REQUEST, &why))
}
/// Whether `record`, the stored record of `choice.source`, lists `choice`.

9
swarm-controller/src/main.rs
View file

@ -2043,7 +2043,9 @@ impl CreateSettings {
/// Whether any setting is given.
fn any(&self) -> bool {
self.limits.is_some() || self.inference.is_some() || self.permissions.is_some()
self.limits.is_some()
|| self.inference.is_some()
|| self.permissions.is_some()
|| self.archetype.is_some()
}
@ -5527,7 +5529,10 @@ mod tests {
"mint_agent_matrix_account",
"set_agent_wanted",
] {
assert!(archetype.iter().any(|l| l == kept), "archetype lacks {kept}");
assert!(
archetype.iter().any(|l| l == kept),
"archetype lacks {kept}"
);
assert!(custom.iter().any(|l| l == kept), "custom lacks {kept}");
}
assert!(archetype_waits.iter().any(|l| l == "set_agent_wanted"));