Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
af914eb875 | ||
|
|
553ed8376c | ||
|
|
b199c83899 |
30 changed files with 1211 additions and 130 deletions
|
|
@ -106,13 +106,16 @@ state dirs work as for every other agent.
|
|||
deploy log). Any agent holding it gets the same mounts.
|
||||
The operator dashboard's `POST /api/meta-update` is the only path
|
||||
that writes `flake.lock`.
|
||||
- **Prompt** — identical for every container: `prompts/system.md`
|
||||
- **Prompt** — identical for every container but one block (below):
|
||||
`prompts/system.md`
|
||||
carries `<!-- role:agent -->` / `<!-- role:manager -->` marker
|
||||
blocks, and `prompt::render` filters for `"agent"` unconditionally,
|
||||
manager included ("always `agent` role — there is only one role",
|
||||
`hive-agent/src/prompt.rs`'s own module doc). The `role:manager`
|
||||
blocks are dead in production, exercised only by a unit test
|
||||
(`filter_role_blocks(SAMPLE, "manager")`).
|
||||
(`filter_blocks(SAMPLE, "role", "manager")`). The one block that differs
|
||||
is `<!-- config:repo -->` vs `<!-- config:archetype -->`: an agent built
|
||||
from an [archetype](../swarm/archetypes.md) is told it has no config repo.
|
||||
- **Tool allow-list** — the MCP tools claude sees come from the agent's
|
||||
permissions alone, the same mechanism for every agent
|
||||
([`docs/turn-loop/mcp.md`](../turn-loop/mcp.md)). Ruth's permissions
|
||||
|
|
|
|||
|
|
@ -65,6 +65,8 @@ Creating a brand-new agent is swarm-level (`swarmctl agent create`,
|
|||
an agent. The swarm controller's `InitAgentConfigRepo` job seeds the
|
||||
agent's config repo with a default `agent.nix` template, then asks the
|
||||
target hive to deploy it.
|
||||
An agent created from an [archetype](../swarm/archetypes.md) gets no
|
||||
config repo: its hive builds it from the archetype's module.
|
||||
|
||||
Changing what the template seeded isn't a special case: like every
|
||||
later change, it's a PR on that config repo, made from a clone and
|
||||
|
|
|
|||
|
|
@ -420,6 +420,12 @@ reads it at every container start for `manage_root_agent`. Each agent gets
|
|||
its own entry from it as `swarm-permissions.json` in its runtime dir, which
|
||||
it holds until its own first read of its key succeeds.
|
||||
|
||||
`/var/lib/hyperhive/swarm-agent-archetypes.json` holds the archetype each of
|
||||
this hive's archetype agents runs, as last read from the swarm's
|
||||
`agent-archetype` bucket at the agent's provision and each rebuild. An agent
|
||||
with an entry has no `applied/<n>` repo and no `agent-<n>` meta input; see
|
||||
[archetypes](../swarm/archetypes.md).
|
||||
|
||||
hive-c0re renders containers onto `meta#<n>` at creation.
|
||||
|
||||
## Destroy vs purge
|
||||
|
|
|
|||
|
|
@ -1,21 +1,24 @@
|
|||
# Agent archetypes
|
||||
|
||||
An archetype is a named set of settings to create an agent from. The swarm
|
||||
UI's create form lists the archetypes the swarm knows. Picking one fills the
|
||||
form's permissions, resource caps and inference fields from it, and you can
|
||||
still change each field before you create the agent.
|
||||
An archetype is a named agent setup: settings to create the agent with, and
|
||||
a NixOS module its container runs. The swarm UI's create form lists the
|
||||
archetypes the swarm knows. Picking one fills the form's permissions,
|
||||
resource caps and inference fields from it, and you can still change each
|
||||
field before you create the agent.
|
||||
|
||||
The agent you create this way is an ordinary agent: it gets its own
|
||||
`agent-configs/<name>` repo like any other, and the request carries only the
|
||||
fields as the form shows them. Nothing records which archetype you picked.
|
||||
An agent created from an archetype has no `agent-configs/<name>` repo of its
|
||||
own. Its hive builds it from the archetype's module, so every agent of that
|
||||
archetype runs the same configuration. To change what it runs, change the
|
||||
archetype. Picking "none" creates a custom agent, with its own config repo.
|
||||
|
||||
## The built-in archetypes
|
||||
|
||||
The hyperhive flake ships these:
|
||||
|
||||
| archetype | what it sets |
|
||||
| ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `reviewer` | permissions `messaging`, `inbox`, `execution`: the default set without `meta`. With `execution` it can run `hive-forge`, `git` and other commands. |
|
||||
| archetype | what it sets |
|
||||
| ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `reviewer` | permissions `messaging`, `inbox`, `execution`: the default set without `meta`. With `execution` it can run `hive-forge`, `git` and other commands. |
|
||||
| `android-dev` | installs `android-tools` (`adb`, `fastboot`), Gradle and a JDK; default permissions. |
|
||||
|
||||
## Where an archetype comes from
|
||||
|
||||
|
|
@ -38,8 +41,8 @@ hyperhiveArchetypes.reviewer = {
|
|||
- Every archetype needs a `description`. `permissions`, `limits` and
|
||||
`inference` are optional and take the shapes `POST /api/agents` takes. The swarm ignores
|
||||
other keys in `meta`.
|
||||
- `module` is a NixOS module for the agent's container. The swarm doesn't
|
||||
apply it yet.
|
||||
- `module` is a NixOS module for the agent's container. A hive imports it
|
||||
into each agent created from the archetype.
|
||||
|
||||
Today the swarm reads one flake: the hyperhive source the
|
||||
[builder](builder.md)'s host runs, so only the built-ins appear.
|
||||
|
|
@ -85,3 +88,46 @@ The stored value:
|
|||
"refused": { "broken": "limits: …" }
|
||||
}
|
||||
```
|
||||
|
||||
## Creating an agent from an archetype
|
||||
|
||||
The create form sends the picked archetype as `archetype` on
|
||||
`POST /api/agents`, `{ "source": "hyperhive", "name": "reviewer" }`. The
|
||||
controller refuses with 400 an archetype the stored list doesn't hold, then:
|
||||
|
||||
1. stores the choice in the `agent-archetype` KV bucket, keyed by agent name,
|
||||
next to the agent's permissions, caps and inference;
|
||||
2. queues the creation graph without `CreateRepo`, `AddRepoMember` and
|
||||
`InitAgentConfigRepo`. Identity, forge user, docs read, the mints and the
|
||||
wanted state stay, and the deploy waits on those alone.
|
||||
|
||||
Destroying the agent deletes its key.
|
||||
Every hive may read any key of the bucket
|
||||
(`$JS.API.DIRECT.GET.KV_agent-archetype.>`) and write none; only the
|
||||
controller writes it.
|
||||
|
||||
Deploy `swarm-nats-auth` before the hives: every provision reads the
|
||||
agent's key, and a hive whose queue connection predates the grant gets a
|
||||
refused read, so its `Provision` nodes fail until it reconnects.
|
||||
|
||||
The hive reads the agent's key when it provisions the agent and on every
|
||||
rebuild, and keeps the last value read in
|
||||
`/var/lib/hyperhive/swarm-agent-archetypes.json`, so the meta flake renders
|
||||
without the queue. When the read fails, a rebuild keeps the value
|
||||
stored there. For an archetype agent the hive:
|
||||
|
||||
- seeds no `proposed` or `applied` repo, and gives the meta flake no
|
||||
`agent-<name>` input. `mkAgent` imports
|
||||
`hyperhive.hyperhiveArchetypes.<name>.module` in place of the agent's own
|
||||
flake, and sets `HIVE_ARCHETYPE` in the harness environment;
|
||||
- relocks no agent input on a rebuild or a meta-input bump, and ignores the
|
||||
`rev` of a swarm deploy request;
|
||||
- mounts an empty `/agents/<name>/config/`.
|
||||
|
||||
The agent's system prompt says it has no config repo and names its
|
||||
archetype.
|
||||
|
||||
The hive takes the module from its own `hyperhive` input. When that input
|
||||
predates the archetype, the agent's build fails with
|
||||
`this hive's hyperhive input has no archetype <name>`. Update the hive's
|
||||
`hyperhive` input to fix it.
|
||||
|
|
|
|||
|
|
@ -255,7 +255,12 @@ needs to `chown` a bind mount), once at startup:
|
|||
`hive-agent/prompts/system.md` by `hive-agent`'s `prompt::render`:
|
||||
HTML-comment markers (`<!-- role:agent -->...<!-- /role:agent -->`,
|
||||
same for `role:manager`) gate the role-specific blocks; both flavors
|
||||
share everything else. Five placeholders are then
|
||||
share everything else. `<!-- config:repo -->` and
|
||||
`<!-- config:archetype -->` blocks gate the config-repo paragraphs: an
|
||||
agent with `HIVE_ARCHETYPE` set (the meta flake sets it on an agent built
|
||||
from an [archetype](../swarm/archetypes.md)) gets the archetype block and
|
||||
every other agent the repo block, with `{archetype}` filled in with the
|
||||
archetype's name. Five placeholders are then
|
||||
substituted: `{label}` (short agent name), `{qualified_label}`
|
||||
(hive-qualified `name@domain` form),
|
||||
`{hive_identity}` (for example `` on hive `pr1ma` ``; empty when
|
||||
|
|
@ -275,14 +280,17 @@ needs to `chown` a bind mount), once at startup:
|
|||
telling the agent where the docs live.
|
||||
Passed via `--system-prompt-file`.
|
||||
|
||||
**Marker grammar.** `<!-- role:X -->` opens a block; any
|
||||
`<!-- /role:X -->` closes the current block. The renderer always uses
|
||||
role `agent`, so it elides blocks with other role tags. Nesting **isn't**
|
||||
**Marker grammar.** `<!-- KIND:X -->` opens a block; any
|
||||
`<!-- /KIND:X -->` closes the current block of that kind. The renderer
|
||||
filters the two kinds one after the other. The renderer always uses
|
||||
role `agent`, so it elides blocks with other role tags, and config `repo`
|
||||
or `archetype` as above. Nesting **isn't**
|
||||
supported — a stray opener with no closer runs until end of file.
|
||||
The renderer tolerates whitespace inside markers (`<!--role:foo-->` parses the
|
||||
same as `<!-- role:foo -->`). Content outside any marker is always
|
||||
included. Today's `system.md` carries no markers (single agent role) —
|
||||
the grammar stays wired for a future manager / multi-role prompt.
|
||||
included. Today's `system.md` carries no role markers (single agent
|
||||
role), and the role grammar stays wired for a future manager / multi-role
|
||||
prompt; it carries one pair of config blocks.
|
||||
|
||||
**`hive_identity` / `swarm_identity` shape.** Each carries a
|
||||
leading space + backticked name (` on hive \`pr1ma\``,
|
||||
|
|
|
|||
|
|
@ -61,3 +61,13 @@ export function prefill(
|
|||
},
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* The `archetype` field of `POST /api/agents` for `archetype`, or nothing for
|
||||
* a custom agent.
|
||||
*/
|
||||
export function requestArchetype(
|
||||
archetype: Archetype | undefined,
|
||||
): { source: string; name: string } | undefined {
|
||||
return archetype && { source: archetype.source, name: archetype.name };
|
||||
}
|
||||
|
|
|
|||
|
|
@ -57,6 +57,7 @@ import {
|
|||
import {
|
||||
archetypeKey,
|
||||
prefill,
|
||||
requestArchetype,
|
||||
type ArchetypeList,
|
||||
} from "./CreateAgentArchetype.js";
|
||||
import "./CreateAgentForm.css";
|
||||
|
|
@ -230,6 +231,7 @@ export function CreateAgentForm({
|
|||
permissions: catalogue
|
||||
? requestPermissions(selected, catalogue.default)
|
||||
: undefined,
|
||||
archetype: requestArchetype(picked),
|
||||
}),
|
||||
});
|
||||
if (!r.ok) {
|
||||
|
|
@ -321,7 +323,9 @@ export function CreateAgentForm({
|
|||
/>
|
||||
{picked && (
|
||||
<p class="create-agent-archetype-description">
|
||||
{picked.description}
|
||||
{picked.description} An agent created from an archetype has
|
||||
no config repo: its hive builds it from the archetype's
|
||||
module.
|
||||
</p>
|
||||
)}
|
||||
</>
|
||||
|
|
@ -396,7 +400,9 @@ export function CreateAgentForm({
|
|||
Submitting this queues everything a new agent needs: a swarm-level
|
||||
identity, a config repo on the forge with the operator added as a
|
||||
collaborator, and a container running the agent on the hive you pick
|
||||
above. Watch it all settle on <Link href="/jobs">jobs</Link>.
|
||||
above. An agent created from an archetype gets no config repo: its
|
||||
container runs the archetype's module. Watch it all settle on{" "}
|
||||
<Link href="/jobs">jobs</Link>.
|
||||
</p>
|
||||
</Panel>
|
||||
</div>
|
||||
|
|
|
|||
|
|
@ -2,7 +2,11 @@
|
|||
// Run by `npm test` with Node's type stripping.
|
||||
import assert from "node:assert/strict";
|
||||
import { test } from "node:test";
|
||||
import { archetypeKey, prefill } from "../src/pages/CreateAgentArchetype.ts";
|
||||
import {
|
||||
archetypeKey,
|
||||
prefill,
|
||||
requestArchetype,
|
||||
} from "../src/pages/CreateAgentArchetype.ts";
|
||||
|
||||
const defaults = ["messaging", "meta", "inbox", "execution"];
|
||||
|
||||
|
|
@ -65,3 +69,17 @@ test("the key tells sources apart", () => {
|
|||
archetypeKey({ source: "b", name: "x", description: "" }),
|
||||
);
|
||||
});
|
||||
|
||||
test("a picked archetype is sent by source and name, none is left out", () => {
|
||||
const reviewer = {
|
||||
source: "hyperhive",
|
||||
name: "reviewer",
|
||||
description: "reviews",
|
||||
permissions: ["messaging"],
|
||||
};
|
||||
assert.deepEqual(requestArchetype(reviewer), {
|
||||
source: "hyperhive",
|
||||
name: "reviewer",
|
||||
});
|
||||
assert.equal(requestArchetype(undefined), undefined);
|
||||
});
|
||||
|
|
|
|||
|
|
@ -3,13 +3,24 @@ You are hyperhive agent `{label}` (qualified: `{qualified_label}`){hive_identity
|
|||
Tools (hyperhive surface). Full signature + behavior for each comes from the tool's own MCP description (you already received it via the MCP tool schema) — this is just the map of what exists and which ones are gated, so you know where to look:
|
||||
|
||||
- **Inbox / messaging** (always available): `mcp__hyperhive__recv`, `mcp__hyperhive__ack_until`, `mcp__hyperhive__send`, `mcp__hyperhive__get_loose_ends`, `mcp__hyperhive__cancel_loose_end`, `mcp__hyperhive__mark_todos_done`, `mcp__hyperhive__remind`, `mcp__hyperhive__set_status`, `mcp__hyperhive__get_agent_meta`. `send` reaches peers on this hive only; **reach the operator over matrix** if you have the matrix MCP tools (`mcp__matrix__send_dm` or `mcp__matrix__send_message`); without them the operator reads your `set_status` line and turn output on the dashboard, and your logs. `send(to: "operator")` is refused. One habit worth internalizing beyond the tool descriptions themselves: prefer ending the turn over repeatedly polling `recv` when idle (only turn-boundaries observe in-container todo wakes — bash-task completions, matrix unread, forge activity — and ending the turn is also your checkpoint). For a large todo backlog (`get_loose_ends` caps at 40 rows), clear reviewed ids in bulk with `mark_todos_done` rather than cancelling one at a time — there's no blind range-clear, only ids you've actually looked at.
|
||||
- **Extra MCP tools** (some agents only): `mcp__<server>__<tool>` — agent-specific (matrix client, scraper, db connector, etc.) declared in your `agent.nix` under `services.hyperhive.agent.extraMcpServers`. First-class tools, already operator-approved at deploy time.
|
||||
- **Extra MCP tools** (some agents only): `mcp__<server>__<tool>` — agent-specific (matrix client, scraper, db connector, etc.) declared in your NixOS config under `services.hyperhive.agent.extraMcpServers`. First-class tools, already operator-approved at deploy time.
|
||||
|
||||
<!-- config:repo -->
|
||||
|
||||
Need new packages, env vars, or other NixOS config for yourself? You can't edit your own config directly — open a PR on your `agent-configs/<name>` repo, or contact the operator directly. Config repos live at `/agents/{label}/config/` (read-only inside your container). All changes flow through operator-approved commits.
|
||||
|
||||
Your config repo is mounted **read-only** at `/agents/{label}/config/` — `agent.nix` plus whatever extra files define you (declared packages, env vars, MCP servers). Read it to see exactly what defines you before asking for a change, so you can point at the precise file and line.
|
||||
|
||||
Operator boundary: starting, stopping and rebuilding containers is the operator's — ask them when any agent needs one. _Creating_ a new agent is not something you can do from here — ask the operator, who scaffolds the new agent's config repo and spawns it from the dashboard. _Changing_ any agent's config is not a tool call at all — it's a forge PR on the agent's `agent-configs/<name>` repo, which an operator reviews and merges on the forge; the merge deploys it. The operator only signs off on changes; you run the day-to-day.
|
||||
_Changing_ any agent's config is not a tool call at all — it's a forge PR on the agent's `agent-configs/<name>` repo, which an operator reviews and merges on the forge; the merge deploys it.
|
||||
|
||||
<!-- /config:repo -->
|
||||
<!-- config:archetype -->
|
||||
|
||||
You are built from the `{archetype}` archetype and have no config repo of your own: there is no `agent-configs/<name>` repo for you, and `/agents/{label}/config/` is empty. Need new packages, env vars, or other NixOS config for yourself? Ask the operator: what defines you is the archetype, shared with every agent built from it.
|
||||
|
||||
<!-- /config:archetype -->
|
||||
|
||||
Operator boundary: starting, stopping and rebuilding containers is the operator's — ask them when any agent needs one. _Creating_ a new agent is not something you can do from here — ask the operator, who creates agents at swarm level. The operator only signs off on changes; you run the day-to-day.
|
||||
|
||||
Messages from sender `system` are hyperhive helper events (JSON body, `event` field discriminates): `container_crash`, `needs_update`. Use these to react to lifecycle changes:
|
||||
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
//! System-prompt renderer. Single `prompts/system.md` with
|
||||
//! HTML-comment markers gating role-specific blocks; this module
|
||||
//! assembles the final prompt (always "agent" role — there is only one
|
||||
//! HTML-comment markers gating role-specific and config-repo-specific
|
||||
//! blocks; this module assembles the final prompt (always "agent" role — there is only one
|
||||
//! role). Marker grammar + placeholder substitution rules in
|
||||
//! `docs/turn-loop/claude-invocation.md::On-boot files` (`claude-system-prompt.md`).
|
||||
|
||||
|
|
@ -32,7 +32,7 @@ pub fn render(
|
|||
docs_dir: Option<&str>,
|
||||
operators: &[OperatorProfile],
|
||||
) -> String {
|
||||
let body = filter_role_blocks(template, "agent");
|
||||
let body = filter_blocks(template, "role", "agent");
|
||||
let qualified = crate::identity::qualify(label);
|
||||
let hive_identity = hive_name
|
||||
.filter(|n| !n.is_empty())
|
||||
|
|
@ -118,27 +118,27 @@ async fn within_timeout<E: std::fmt::Display>(
|
|||
}
|
||||
}
|
||||
|
||||
/// Walk `template` line-by-line, stripping `<!-- role:X -->` /
|
||||
/// `<!-- /role:X -->` marker lines and suppressing the lines inside a block
|
||||
/// unless `X == target`. A close marker ends the current block. Production
|
||||
/// `system.md` carries no markers today (single agent role — see the module
|
||||
/// doc), so this is effectively a passthrough; the marker grammar stays wired
|
||||
/// for a future manager / multi-role prompt.
|
||||
fn filter_role_blocks(template: &str, target: &str) -> String {
|
||||
/// Walk `template` line-by-line, stripping `<!-- KIND:X -->` /
|
||||
/// `<!-- /KIND:X -->` marker lines and suppressing the lines inside a block
|
||||
/// unless `X == target`. A close marker ends the current block. Markers of
|
||||
/// any other kind are left in place, for the pass that filters that kind.
|
||||
/// Two kinds exist: `role` (always `agent`; the grammar stays wired for a
|
||||
/// future manager / multi-role prompt) and `config` ([`config_blocks`]).
|
||||
fn filter_blocks(template: &str, kind: &str, target: &str) -> String {
|
||||
let mut out = String::with_capacity(template.len());
|
||||
// None = outside any block; Some(role) = inside role-tagged block.
|
||||
let mut active_role: Option<&str> = None;
|
||||
// None = outside any block; Some(value) = inside a block of this kind.
|
||||
let mut active: Option<&str> = None;
|
||||
for line in template.lines() {
|
||||
let trimmed = line.trim();
|
||||
if let Some(role) = parse_open_marker(trimmed) {
|
||||
active_role = Some(role);
|
||||
if let Some(value) = parse_open_marker(trimmed, kind) {
|
||||
active = Some(value);
|
||||
continue;
|
||||
}
|
||||
if parse_close_marker(trimmed).is_some() {
|
||||
active_role = None;
|
||||
if parse_close_marker(trimmed, kind).is_some() {
|
||||
active = None;
|
||||
continue;
|
||||
}
|
||||
if active_role.is_none_or(|role| role == target) {
|
||||
if active.is_none_or(|value| value == target) {
|
||||
out.push_str(line);
|
||||
out.push('\n');
|
||||
}
|
||||
|
|
@ -146,23 +146,37 @@ fn filter_role_blocks(template: &str, target: &str) -> String {
|
|||
out
|
||||
}
|
||||
|
||||
/// `<!-- role:agent -->` → `Some("agent")`. Anything else returns
|
||||
/// None. Whitespace inside the marker is tolerated so a future
|
||||
/// author's `<!--role:foo-->` (no spaces) still parses; the dashboard
|
||||
/// markdown renderer is equally lenient. Close tags (`/role:...`)
|
||||
/// can't accidentally match — the `strip_prefix("role:")` rejects
|
||||
/// the leading slash before we'd ever see it.
|
||||
fn parse_open_marker(line: &str) -> Option<&str> {
|
||||
let inside = line.strip_prefix("<!--")?.strip_suffix("-->")?.trim();
|
||||
let role = inside.strip_prefix("role:")?.trim();
|
||||
Some(role)
|
||||
/// `template` with its `config:repo` blocks for a custom agent, or its
|
||||
/// `config:archetype` blocks and `{archetype}` filled in for an agent built
|
||||
/// from `archetype`. An empty name is a custom agent.
|
||||
fn config_blocks(template: &str, archetype: Option<&str>) -> String {
|
||||
match archetype.filter(|a| !a.is_empty()) {
|
||||
Some(name) => filter_blocks(template, "config", "archetype").replace("{archetype}", name),
|
||||
None => filter_blocks(template, "config", "repo"),
|
||||
}
|
||||
}
|
||||
|
||||
/// `<!-- /role:agent -->` → `Some("agent")`. Mirror of
|
||||
/// [`parse_open_marker`] for the closing tag.
|
||||
fn parse_close_marker(line: &str) -> Option<&str> {
|
||||
/// `<!-- role:agent -->` with `kind = "role"` → `Some("agent")`. Anything
|
||||
/// else returns None. Whitespace inside the marker is tolerated so a future
|
||||
/// author's `<!--role:foo-->` (no spaces) still parses; the dashboard
|
||||
/// markdown renderer is equally lenient. Close tags (`/role:...`) can't
|
||||
/// accidentally match — the `strip_prefix(kind)` rejects the leading slash
|
||||
/// before we'd ever see it.
|
||||
fn parse_open_marker<'a>(line: &'a str, kind: &str) -> Option<&'a str> {
|
||||
let inside = line.strip_prefix("<!--")?.strip_suffix("-->")?.trim();
|
||||
inside.strip_prefix("/role:").map(str::trim)
|
||||
let value = inside.strip_prefix(kind)?.strip_prefix(':')?.trim();
|
||||
Some(value)
|
||||
}
|
||||
|
||||
/// `<!-- /role:agent -->` with `kind = "role"` → `Some("agent")`. Mirror of
|
||||
/// [`parse_open_marker`] for the closing tag.
|
||||
fn parse_close_marker<'a>(line: &'a str, kind: &str) -> Option<&'a str> {
|
||||
let inside = line.strip_prefix("<!--")?.strip_suffix("-->")?.trim();
|
||||
inside
|
||||
.strip_prefix('/')?
|
||||
.strip_prefix(kind)?
|
||||
.strip_prefix(':')
|
||||
.map(str::trim)
|
||||
}
|
||||
|
||||
/// Write the assembled prompt to a stable path next to the harness
|
||||
|
|
@ -196,6 +210,9 @@ pub async fn write_system_prompt(_socket: &Path, label: &str) -> Result<PathBuf>
|
|||
let docs_dir = std::env::var("HIVE_DOCS_DIR")
|
||||
.ok()
|
||||
.filter(|d| !d.is_empty());
|
||||
// The meta flake sets HIVE_ARCHETYPE on an agent built from an archetype,
|
||||
// which has no config repo of its own.
|
||||
let template = config_blocks(&template, std::env::var("HIVE_ARCHETYPE").ok().as_deref());
|
||||
let body = render(
|
||||
&template,
|
||||
label,
|
||||
|
|
@ -257,7 +274,7 @@ shared closer
|
|||
|
||||
#[test]
|
||||
fn filter_keeps_shared_and_target_role() {
|
||||
let agent = filter_role_blocks(SAMPLE, "agent");
|
||||
let agent = filter_blocks(SAMPLE, "role", "agent");
|
||||
assert!(agent.contains("shared opener"));
|
||||
assert!(agent.contains("agent-only line"));
|
||||
assert!(!agent.contains("manager-only line"));
|
||||
|
|
@ -268,7 +285,7 @@ shared closer
|
|||
|
||||
#[test]
|
||||
fn filter_for_manager_picks_manager_block() {
|
||||
let manager = filter_role_blocks(SAMPLE, "manager");
|
||||
let manager = filter_blocks(SAMPLE, "role", "manager");
|
||||
assert!(manager.contains("shared opener"));
|
||||
assert!(!manager.contains("agent-only line"));
|
||||
assert!(manager.contains("manager-only line"));
|
||||
|
|
@ -278,23 +295,38 @@ shared closer
|
|||
|
||||
#[test]
|
||||
fn parse_open_marker_handles_whitespace_variants() {
|
||||
assert_eq!(parse_open_marker("<!-- role:agent -->"), Some("agent"));
|
||||
assert_eq!(parse_open_marker("<!--role:agent-->"), Some("agent"));
|
||||
assert_eq!(parse_open_marker("<!-- role:manager -->"), Some("manager"));
|
||||
assert_eq!(
|
||||
parse_open_marker("<!-- role:agent -->", "role"),
|
||||
Some("agent")
|
||||
);
|
||||
assert_eq!(
|
||||
parse_open_marker("<!--role:agent-->", "role"),
|
||||
Some("agent")
|
||||
);
|
||||
assert_eq!(
|
||||
parse_open_marker("<!-- role:manager -->", "role"),
|
||||
Some("manager")
|
||||
);
|
||||
// Close tags must NOT match open-tag parser.
|
||||
assert_eq!(parse_open_marker("<!-- /role:agent -->"), None);
|
||||
assert_eq!(parse_open_marker("<!-- /role:agent -->", "role"), None);
|
||||
// Non-markers pass through (return None).
|
||||
assert_eq!(parse_open_marker("just text"), None);
|
||||
assert_eq!(parse_open_marker("<!-- not a role -->"), None);
|
||||
assert_eq!(parse_open_marker("just text", "role"), None);
|
||||
assert_eq!(parse_open_marker("<!-- not a role -->", "role"), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parse_close_marker_handles_whitespace_variants() {
|
||||
assert_eq!(parse_close_marker("<!-- /role:agent -->"), Some("agent"));
|
||||
assert_eq!(parse_close_marker("<!--/role:manager-->"), Some("manager"));
|
||||
assert_eq!(
|
||||
parse_close_marker("<!-- /role:agent -->", "role"),
|
||||
Some("agent")
|
||||
);
|
||||
assert_eq!(
|
||||
parse_close_marker("<!--/role:manager-->", "role"),
|
||||
Some("manager")
|
||||
);
|
||||
// Open tags must NOT match close-tag parser.
|
||||
assert_eq!(parse_close_marker("<!-- role:agent -->"), None);
|
||||
assert_eq!(parse_close_marker("just text"), None);
|
||||
assert_eq!(parse_close_marker("<!-- role:agent -->", "role"), None);
|
||||
assert_eq!(parse_close_marker("just text", "role"), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
|
@ -331,6 +363,49 @@ shared closer
|
|||
assert!(rendered.contains("`/agents/alice/work/`"), "{rendered}");
|
||||
}
|
||||
|
||||
const CONFIG_FIXTURE: &str = "\
|
||||
shared
|
||||
<!-- config:repo -->
|
||||
open a PR on agent-configs
|
||||
<!-- /config:repo -->
|
||||
<!-- config:archetype -->
|
||||
built from `{archetype}`, no config repo
|
||||
<!-- /config:archetype -->
|
||||
";
|
||||
|
||||
/// A custom agent gets the config-repo lines, an archetype agent the
|
||||
/// archetype lines with its name, and an empty name is a custom agent.
|
||||
#[test]
|
||||
fn config_blocks_pick_the_block_by_archetype() {
|
||||
for custom in [None, Some("")] {
|
||||
let picked = config_blocks(CONFIG_FIXTURE, custom);
|
||||
assert!(picked.contains("open a PR on agent-configs"), "{picked}");
|
||||
assert!(!picked.contains("no config repo"), "{picked}");
|
||||
}
|
||||
let picked = config_blocks(CONFIG_FIXTURE, Some("android-dev"));
|
||||
assert!(picked.contains("built from `android-dev`, no config repo"));
|
||||
assert!(!picked.contains("agent-configs"), "{picked}");
|
||||
assert!(picked.contains("shared"));
|
||||
assert!(!picked.contains("<!--"), "{picked}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_production_prompt_keeps_no_config_markers() {
|
||||
for archetype in [None, Some("reviewer")] {
|
||||
let rendered = render(
|
||||
&config_blocks(&PRODUCTION_TEMPLATE, archetype),
|
||||
"alice",
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
&[],
|
||||
);
|
||||
assert!(!rendered.contains("<!-- config:"), "{rendered}");
|
||||
assert!(!rendered.contains("<!-- /config:"), "{rendered}");
|
||||
assert!(!rendered.contains("{archetype}"), "{rendered}");
|
||||
}
|
||||
}
|
||||
|
||||
// Inline fixture for the {hive_identity} / {swarm_identity}
|
||||
// placeholders. Cargo's `cargo test` resolves `PRODUCTION_TEMPLATE`
|
||||
// against `$HIVE_ASSETS_DIR/prompts/system.md`, which the flake
|
||||
|
|
|
|||
|
|
@ -59,6 +59,7 @@ swarm-queue-client = { workspace = true, features = [
|
|||
"notices",
|
||||
"agent-limits",
|
||||
"agent-permissions",
|
||||
"agent-archetype",
|
||||
] }
|
||||
swarm-secret-client.workspace = true
|
||||
tokio.workspace = true
|
||||
|
|
|
|||
143
hive-c0re/src/agent_config/archetypes.rs
Normal file
143
hive-c0re/src/agent_config/archetypes.rs
Normal file
|
|
@ -0,0 +1,143 @@
|
|||
//! The archetype each of this hive's archetype agents is built from, as last
|
||||
//! read from the swarm's `agent-archetype` bucket
|
||||
//! ([`swarm_queue_client::agent_archetype`]). Stored at
|
||||
//! `/var/lib/hyperhive/swarm-agent-archetypes.json`, outside the meta repo:
|
||||
//! the meta render runs at boot, possibly before the queue answers, so it
|
||||
//! reads this file rather than the bucket.
|
||||
//!
|
||||
//! Format: agent name → [`ArchetypeRef`]. An agent with an entry has no config
|
||||
//! repo: the meta flake builds it from the archetype's module, and the paths
|
||||
//! that seed, relock or advance `applied/<name>` skip it. An agent with no
|
||||
//! entry is a custom agent.
|
||||
//!
|
||||
//! [`refresh`] reads one agent's key; provisioning and every rebuild's meta
|
||||
//! sync call it before they render the meta flake.
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use anyhow::{Context, Result};
|
||||
use swarm_queue_client::agent_archetype::{self, ArchetypeRef, Read};
|
||||
|
||||
const FILE: &str = "swarm-agent-archetypes.json";
|
||||
|
||||
pub type Stored = BTreeMap<String, ArchetypeRef>;
|
||||
|
||||
#[must_use]
|
||||
pub fn path() -> PathBuf {
|
||||
crate::paths::state_root().join(FILE)
|
||||
}
|
||||
|
||||
/// Read the stored archetypes. An absent file is the empty map: every agent
|
||||
/// custom. A file that exists but can't be read or parsed is an error.
|
||||
pub fn read_at(path: &Path) -> std::io::Result<Stored> {
|
||||
super::read_map(path)
|
||||
}
|
||||
|
||||
/// `agent`'s archetype per the file at [`path`], `None` for a custom agent.
|
||||
///
|
||||
/// # Errors
|
||||
///
|
||||
/// As [`read_at`].
|
||||
pub fn of(agent: &str) -> std::io::Result<Option<ArchetypeRef>> {
|
||||
Ok(read_at(&path())?.remove(agent))
|
||||
}
|
||||
|
||||
/// Read `agent`'s key and store what it holds. No queue, or a bucket that
|
||||
/// does not answer, leaves the stored entry as it is.
|
||||
///
|
||||
/// # Errors
|
||||
///
|
||||
/// The queue is configured but not connected, the read failed, the value
|
||||
/// names a source or archetype that is not an identifier, or the file could
|
||||
/// not be read or written.
|
||||
pub async fn refresh(agent: &str) -> Result<()> {
|
||||
let Some(client) = crate::swarm_queue::client().await else {
|
||||
return Ok(());
|
||||
};
|
||||
// An unconnected client does not fail a JetStream request, it hangs on it.
|
||||
swarm_queue_client::ensure_connected(&client)?;
|
||||
let read = agent_archetype::read(&client, agent)
|
||||
.await
|
||||
.map_err(|e| anyhow::anyhow!(e))
|
||||
.with_context(|| format!("reading the swarm's archetype for {agent}"))?;
|
||||
store_read(&path(), agent, read)
|
||||
}
|
||||
|
||||
/// Store `read`, the value of `agent`'s key, in the file at `path`. Written
|
||||
/// only when it changes the entry.
|
||||
fn store_read(path: &Path, agent: &str, read: Read) -> Result<()> {
|
||||
let value = match read {
|
||||
Read::NoBucket => {
|
||||
tracing::debug!(%agent, "archetype: the agent-archetype bucket does not answer");
|
||||
return Ok(());
|
||||
}
|
||||
Read::Current(value) => value,
|
||||
};
|
||||
// Both are rendered into the meta flake as nix strings.
|
||||
if let Some(ArchetypeRef { source, name }) = &value {
|
||||
for part in [source, name] {
|
||||
hive_types::Ident::parse(part)
|
||||
.map_err(|why| anyhow::anyhow!("archetype {part:?} of {agent}: {why}"))?;
|
||||
}
|
||||
}
|
||||
let mut stored = read_at(path).context("reading the stored archetypes")?;
|
||||
let changed = match value {
|
||||
Some(value) => stored.insert(agent.to_owned(), value.clone()) != Some(value),
|
||||
None => stored.remove(agent).is_some(),
|
||||
};
|
||||
if changed {
|
||||
super::write_map(path, &stored).context("storing the archetypes")?;
|
||||
tracing::info!(%agent, archetype = ?stored.get(agent), "archetype: stored the swarm's archetype");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn reviewer() -> ArchetypeRef {
|
||||
ArchetypeRef {
|
||||
source: "hyperhive".to_owned(),
|
||||
name: "reviewer".to_owned(),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_read_sets_or_clears_the_entry_and_no_bucket_keeps_it() {
|
||||
let dir = tempfile::tempdir().expect("tempdir");
|
||||
let path = dir.path().join(FILE);
|
||||
assert!(read_at(&path).expect("missing is empty").is_empty());
|
||||
|
||||
store_read(&path, "atlas", Read::Current(Some(reviewer()))).expect("store");
|
||||
store_read(&path, "iris", Read::Current(None)).expect("store");
|
||||
assert_eq!(
|
||||
read_at(&path).expect("read"),
|
||||
Stored::from([("atlas".to_owned(), reviewer())])
|
||||
);
|
||||
|
||||
store_read(&path, "atlas", Read::NoBucket).expect("store");
|
||||
assert_eq!(
|
||||
read_at(&path).expect("read").get("atlas"),
|
||||
Some(&reviewer())
|
||||
);
|
||||
|
||||
store_read(&path, "atlas", Read::Current(None)).expect("store");
|
||||
assert!(read_at(&path).expect("read").is_empty());
|
||||
}
|
||||
|
||||
/// The value is rendered into nix, so a name that is not an identifier is
|
||||
/// refused and nothing is stored.
|
||||
#[test]
|
||||
fn a_name_that_is_not_an_identifier_is_refused() {
|
||||
let dir = tempfile::tempdir().expect("tempdir");
|
||||
let path = dir.path().join(FILE);
|
||||
let bad = ArchetypeRef {
|
||||
source: "hyperhive".to_owned(),
|
||||
name: "x\"; y".to_owned(),
|
||||
};
|
||||
store_read(&path, "atlas", Read::Current(Some(bad))).expect_err("refused");
|
||||
assert!(read_at(&path).expect("read").is_empty());
|
||||
}
|
||||
}
|
||||
|
|
@ -1,13 +1,14 @@
|
|||
//! Per-agent configuration registries: resource limits, topology (JSON files
|
||||
//! under `/var/lib/hyperhive/meta/`), the permissions the swarm grants
|
||||
//! (outside the meta repo) and the shared wire-protocol size limits. Each
|
||||
//! submodule is re-exported at the crate root, so `crate::topology::…` etc.
|
||||
//! keep working unchanged.
|
||||
//! under `/var/lib/hyperhive/meta/`), the permissions and archetypes the
|
||||
//! swarm sets (outside the meta repo) and the shared wire-protocol size
|
||||
//! limits. Each submodule is re-exported at the crate root, so
|
||||
//! `crate::topology::…` etc. keep working unchanged.
|
||||
//!
|
||||
//! Note the two similarly-named modules: [`limits`] caps inline
|
||||
//! *message body* sizes on the sockets, while [`resource_limits`] holds
|
||||
//! per-agent CPU/memory caps for the container drop-in.
|
||||
|
||||
pub mod archetypes;
|
||||
pub mod limits;
|
||||
pub mod permissions;
|
||||
pub mod resource_limits;
|
||||
|
|
|
|||
|
|
@ -396,6 +396,11 @@ async fn run_meta_sync(coord: &Arc<Coordinator>, name: &str, relock: bool) -> Re
|
|||
let agent_dir = crate::paths::agent_runtime_dir(name);
|
||||
let hive = coord.hive_env();
|
||||
let paths = Coordinator::agent_paths(name, agent_dir)?;
|
||||
// A failed read keeps the archetype last stored, so the agent is built as
|
||||
// it was.
|
||||
if let Err(e) = crate::archetypes::refresh(name).await {
|
||||
tracing::warn!(%name, error = ?e, "rebuild: reading the swarm's archetype failed; keeping the stored one");
|
||||
}
|
||||
crate::lifecycle::prepare_rebuild_dirs(name, &paths).await?;
|
||||
// Idempotent meta sync so a manual rebuild can also recover from a
|
||||
// divergent meta repo; then bump just this agent's input. `relock =
|
||||
|
|
@ -403,7 +408,8 @@ async fn run_meta_sync(coord: &Arc<Coordinator>, name: &str, relock: bool) -> Re
|
|||
// would revert the bump the cascade just committed.
|
||||
let agents = crate::lifecycle::agents_for_meta_listing().await?;
|
||||
crate::meta::sync_agents(&hive, &agents).await?;
|
||||
if relock {
|
||||
// An archetype agent has no `agent-<name>` input to relock.
|
||||
if relock && crate::archetypes::of(name)?.is_none() {
|
||||
crate::meta::lock_update_for_rebuild(name).await?;
|
||||
}
|
||||
Ok(())
|
||||
|
|
@ -553,8 +559,16 @@ async fn run_meta_lock(
|
|||
// against whatever `applied/<name>` already happened to be locked to —
|
||||
// normally current, but silently stale forever if a past deploy failed
|
||||
// and nothing since retried it.
|
||||
// An archetype agent has no config repo and no `agent-<name>` input, so
|
||||
// it is left out here; it is still rebuilt with the cascade.
|
||||
let archetypes = crate::archetypes::read_at(&crate::archetypes::path())?;
|
||||
let custom: Vec<String> = cascade
|
||||
.iter()
|
||||
.filter(|name| !archetypes.contains_key(*name))
|
||||
.cloned()
|
||||
.collect();
|
||||
catch_up_cascade_agents(
|
||||
&cascade,
|
||||
&custom,
|
||||
crate::forge::fast_forward_applied_main,
|
||||
|agent_inputs| async move { crate::meta::lock_update(&agent_inputs).await },
|
||||
)
|
||||
|
|
|
|||
|
|
@ -305,8 +305,10 @@ pub const CONTAINER_MANAGER_APPLIED_MOUNT: &str = "/applied";
|
|||
/// and a `ManageRootAgent` holder's view of another agent's) go through
|
||||
/// here so they cannot drift.
|
||||
///
|
||||
/// Never empty under a live container: `provision_container` runs
|
||||
/// `setup_applied` before `create_only` makes the container at all.
|
||||
/// For a custom agent it is never empty under a live container:
|
||||
/// `provision_container` runs `setup_applied` before `create_only` makes the
|
||||
/// container at all. An archetype agent (`crate::archetypes`) has no applied
|
||||
/// repo, so its mount is the empty directory the caller creates.
|
||||
fn config_bind_source(name: &str) -> PathBuf {
|
||||
crate::paths::applied_dir(name)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -230,8 +230,13 @@ pub async fn provision_container(name: &str, hive: &HiveEnv, paths: &AgentPaths)
|
|||
agent_web_port(name)
|
||||
);
|
||||
}
|
||||
setup_proposed(&paths.proposed, name).await?;
|
||||
setup_applied(&paths.applied, Some(&paths.proposed), name).await?;
|
||||
// Read before anything is seeded: an archetype agent gets no config
|
||||
// repo, and the meta render below builds it from the archetype.
|
||||
crate::archetypes::refresh(name).await?;
|
||||
if crate::archetypes::of(name)?.is_none() {
|
||||
setup_proposed(&paths.proposed, name).await?;
|
||||
setup_applied(&paths.applied, Some(&paths.proposed), name).await?;
|
||||
}
|
||||
ensure_agent_state_subvolume(name).await?;
|
||||
ensure_claude_dir(&paths.claude)?;
|
||||
ensure_state_dir(&paths.notes)?;
|
||||
|
|
@ -273,17 +278,20 @@ pub async fn prepare_rebuild_dirs(name: &str, paths: &AgentPaths) -> Result<()>
|
|||
agent_web_port(name)
|
||||
);
|
||||
}
|
||||
if !paths.applied.join(".git").exists()
|
||||
&& !crate::forge::fetch_config_main_into_applied(name).await
|
||||
{
|
||||
bail!(
|
||||
"applied repo at {} is missing its .git directory and could not be reseeded from \
|
||||
// An archetype agent has no applied repo to check or recover.
|
||||
if crate::archetypes::of(name)?.is_none() {
|
||||
if !paths.applied.join(".git").exists()
|
||||
&& !crate::forge::fetch_config_main_into_applied(name).await
|
||||
{
|
||||
bail!(
|
||||
"applied repo at {} is missing its .git directory and could not be reseeded from \
|
||||
the forge (agent-configs/{name} unreachable, has no `main` — never merged or \
|
||||
never seeded — or absent); destroy --purge and re-spawn this agent.",
|
||||
paths.applied.display()
|
||||
);
|
||||
paths.applied.display()
|
||||
);
|
||||
}
|
||||
setup_applied(&paths.applied, None, name).await?;
|
||||
}
|
||||
setup_applied(&paths.applied, None, name).await?;
|
||||
ensure_agent_state_subvolume(name).await?;
|
||||
ensure_claude_dir(&paths.claude)?;
|
||||
ensure_state_dir(&paths.notes)?;
|
||||
|
|
@ -319,12 +327,14 @@ pub async fn swap_update(
|
|||
/// handle the `Err` themselves with `.unwrap_or_default()`.
|
||||
async fn agents_for_meta(name_to_add: Option<&str>) -> Result<Vec<crate::meta::AgentSpec>> {
|
||||
let containers = list().await?;
|
||||
let archetypes = crate::archetypes::read_at(&crate::archetypes::path())?;
|
||||
let mut out: Vec<crate::meta::AgentSpec> = containers
|
||||
.into_iter()
|
||||
.filter_map(|c| {
|
||||
let name = c.strip_prefix(AGENT_PREFIX)?.to_owned();
|
||||
Some(crate::meta::AgentSpec {
|
||||
port: agent_web_port(&name),
|
||||
archetype: archetypes.get(&name).cloned(),
|
||||
name,
|
||||
})
|
||||
})
|
||||
|
|
@ -334,6 +344,7 @@ async fn agents_for_meta(name_to_add: Option<&str>) -> Result<Vec<crate::meta::A
|
|||
{
|
||||
out.push(crate::meta::AgentSpec {
|
||||
port: agent_web_port(extra),
|
||||
archetype: archetypes.get(extra).cloned(),
|
||||
name: extra.to_owned(),
|
||||
});
|
||||
}
|
||||
|
|
|
|||
|
|
@ -43,7 +43,7 @@ mod swarm_status;
|
|||
mod test_env;
|
||||
mod workers;
|
||||
|
||||
pub(crate) use agent_config::{limits, permissions, resource_limits, topology};
|
||||
pub(crate) use agent_config::{archetypes, limits, permissions, resource_limits, topology};
|
||||
pub(crate) use stats::{container_stats, hive_stats, host_stats, otel_metrics, warnings};
|
||||
pub(crate) use stores::{broker, build_logs, db, power};
|
||||
pub(crate) use workers::{agent_sockets, auto_update, crash_watch, knowledge, mcp_sockets};
|
||||
|
|
|
|||
|
|
@ -42,6 +42,9 @@ pub const CONTAINER_MANAGER_META_MOUNT: &str = "/meta";
|
|||
pub struct AgentSpec {
|
||||
pub name: String,
|
||||
pub port: u16,
|
||||
/// The archetype the agent is built from (`crate::archetypes`), `None`
|
||||
/// for a custom agent built from its own config repo.
|
||||
pub archetype: Option<swarm_queue_client::agent_archetype::ArchetypeRef>,
|
||||
}
|
||||
|
||||
/// Stage topology.json if it exists: it is regenerated by `reconcile`
|
||||
|
|
@ -786,6 +789,11 @@ where
|
|||
// (config PRs land there) but are not the flake's build input. deploy +
|
||||
// rebuild re-lock this input to `applied/<name>`'s current `main` head.
|
||||
for spec in agents {
|
||||
// An archetype agent has no config repo, so no input: `mkAgent`
|
||||
// takes its module from the archetype.
|
||||
if spec.archetype.is_some() {
|
||||
continue;
|
||||
}
|
||||
let _ = writeln!(
|
||||
out,
|
||||
" agent-{name}.url = \"git+file://{applied}\";",
|
||||
|
|
@ -811,17 +819,26 @@ where
|
|||
out.push_str(" };\n outputs =\n { self, hyperhive, ... }@inputs:\n let\n");
|
||||
let _ = writeln!(
|
||||
out,
|
||||
" dashboardPort = {dashboard_port};\n mkAgent = {{ name, port }}:"
|
||||
" dashboardPort = {dashboard_port};\n mkAgent = {{ name, port, archetype ? null }}:"
|
||||
);
|
||||
out.push_str(
|
||||
r#" let
|
||||
base = hyperhive.nixosConfigurations.agent-base;
|
||||
input = inputs."agent-${name}";
|
||||
# Every flake an archetype can come from, by its source name.
|
||||
archetypeSources = { inherit hyperhive; };
|
||||
agentModule =
|
||||
if archetype == null then
|
||||
inputs."agent-${name}".nixosModules.default
|
||||
else
|
||||
(archetypeSources.${archetype.source}
|
||||
or (throw "agent ${name}: archetype source ${archetype.source} is not an input of this hive's meta flake")
|
||||
).hyperhiveArchetypes.${archetype.name}.module
|
||||
or (throw "agent ${name}: this hive's ${archetype.source} input has no archetype ${archetype.name}; update that input");
|
||||
service = "hive-agent";
|
||||
in
|
||||
base.extendModules {
|
||||
modules = [
|
||||
input.nixosModules.default
|
||||
agentModule
|
||||
{
|
||||
"#,
|
||||
);
|
||||
|
|
@ -1008,6 +1025,10 @@ where
|
|||
HYPERHIVE_HARNESS_DIR = "/agents/${name}/harness";
|
||||
};
|
||||
}
|
||||
# Tells the harness the agent has no config repo of its own.
|
||||
(if archetype == null then { } else {
|
||||
systemd.services.${service}.environment.HIVE_ARCHETYPE = archetype.name;
|
||||
})
|
||||
];
|
||||
};
|
||||
in
|
||||
|
|
@ -1016,9 +1037,17 @@ where
|
|||
"#,
|
||||
);
|
||||
for spec in agents {
|
||||
// `source` and `name` are identifiers (`archetypes::store_read`), so
|
||||
// they need no escaping.
|
||||
let archetype = spec.archetype.as_ref().map_or(String::new(), |a| {
|
||||
format!(
|
||||
" archetype = {{ source = \"{}\"; name = \"{}\"; }};",
|
||||
a.source, a.name
|
||||
)
|
||||
});
|
||||
let _ = writeln!(
|
||||
out,
|
||||
" {} = mkAgent {{ name = \"{}\"; port = {}; }};",
|
||||
" {} = mkAgent {{ name = \"{}\"; port = {};{archetype} }};",
|
||||
spec.name, spec.name, spec.port,
|
||||
);
|
||||
}
|
||||
|
|
@ -1235,6 +1264,7 @@ mod tests {
|
|||
AgentSpec {
|
||||
name: name.to_owned(),
|
||||
port,
|
||||
archetype: None,
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -1501,6 +1531,53 @@ mod tests {
|
|||
);
|
||||
}
|
||||
|
||||
/// An archetype agent has no `agent-<name>` input, and `mkAgent` is
|
||||
/// handed its archetype; a custom agent beside it keeps its input.
|
||||
#[test]
|
||||
fn render_flake_builds_an_archetype_agent_from_its_archetype() {
|
||||
let android = AgentSpec {
|
||||
archetype: Some(swarm_queue_client::agent_archetype::ArchetypeRef {
|
||||
source: "hyperhive".to_owned(),
|
||||
name: "android-dev".to_owned(),
|
||||
}),
|
||||
..sample_spec("droid", 9002)
|
||||
};
|
||||
let out = render_flake_with_lookup(
|
||||
"github:example/hyperhive",
|
||||
"path:/nix/store/bbbb-hyperhive-docs-source",
|
||||
"path:/nix/store/aaaa-nixpkgs-source",
|
||||
None,
|
||||
8000,
|
||||
&[sample_spec("alice", 9001), android],
|
||||
|_| vec!["nixpkgs"],
|
||||
);
|
||||
assert!(
|
||||
!out.contains("agent-droid"),
|
||||
"an archetype agent has no config input:\n{out}"
|
||||
);
|
||||
assert!(
|
||||
out.contains(
|
||||
"droid = mkAgent { name = \"droid\"; port = 9002; \
|
||||
archetype = { source = \"hyperhive\"; name = \"android-dev\"; }; };"
|
||||
),
|
||||
"{out}"
|
||||
);
|
||||
assert!(out.contains("agent-alice.url = \"git+file://"), "{out}");
|
||||
assert!(out.contains("agent-alice.inputs.nixpkgs.follows"), "{out}");
|
||||
assert!(
|
||||
out.contains("alice = mkAgent { name = \"alice\"; port = 9001; };"),
|
||||
"{out}"
|
||||
);
|
||||
assert!(
|
||||
out.contains("hyperhiveArchetypes.${archetype.name}.module"),
|
||||
"{out}"
|
||||
);
|
||||
assert!(
|
||||
out.contains("environment.HIVE_ARCHETYPE = archetype.name"),
|
||||
"{out}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn render_flake_forwards_env_into_global_environment() {
|
||||
// Regression for the isolation breakage where forwarded vars
|
||||
|
|
|
|||
|
|
@ -82,7 +82,16 @@ pub async fn run(coord: &Arc<Coordinator>) -> Result<()> {
|
|||
|
||||
// Phase 1 + 2: per-agent applied + proposed.
|
||||
tracing::debug!("migration: phase 1+2 (applied + proposed repos)");
|
||||
// An archetype agent has neither repo, and seeding one here would hand it
|
||||
// a config it is not built from.
|
||||
let archetypes = crate::archetypes::read_at(&crate::archetypes::path()).unwrap_or_else(|e| {
|
||||
tracing::warn!(error = ?e, "migration: stored archetypes unreadable; treating every agent as custom");
|
||||
crate::archetypes::Stored::new()
|
||||
});
|
||||
for name in &names {
|
||||
if archetypes.contains_key(name.as_str()) {
|
||||
continue;
|
||||
}
|
||||
tracing::debug!(%name, "migration: applied+proposed");
|
||||
if let Err(e) = migrate_applied_repo(name.as_str()).await {
|
||||
tracing::warn!(%name, error = ?e, "migration: applied repo rewrite failed");
|
||||
|
|
|
|||
|
|
@ -298,7 +298,16 @@ async fn handle_deploy_request(
|
|||
|
||||
// Only a rebuild applies `rev`. A first deploy ignores it and builds the
|
||||
// `applied` repo it seeds from the forge's `main`, or the one it finds.
|
||||
// An archetype agent has no `applied` repo, so its rebuild ignores it too.
|
||||
let archetype = match crate::archetypes::of(&agent) {
|
||||
Ok(archetype) => archetype,
|
||||
Err(e) => {
|
||||
tracing::warn!(%agent, error = ?e, "swarm events: cannot read the stored archetypes; deploy dropped");
|
||||
return;
|
||||
}
|
||||
};
|
||||
if known
|
||||
&& archetype.is_none()
|
||||
&& let Some(rev) = rev.as_deref()
|
||||
&& !apply_rev(&agent, rev).await
|
||||
{
|
||||
|
|
|
|||
|
|
@ -43,8 +43,8 @@ fn dir() -> PathBuf {
|
|||
}
|
||||
|
||||
/// `$HIVE_ASSETS_DIR/prompts/system.md` — the claude system prompt
|
||||
/// template. `hive-agent::prompt::render` filters the role markers
|
||||
/// inside it per agent / manager flavor.
|
||||
/// template. `hive-agent::prompt::render` filters the role and config markers
|
||||
/// inside it per agent / manager flavor and per custom / archetype agent.
|
||||
#[must_use]
|
||||
pub fn prompt_template() -> PathBuf {
|
||||
dir().join("prompts/system.md")
|
||||
|
|
|
|||
|
|
@ -17,4 +17,18 @@
|
|||
];
|
||||
};
|
||||
};
|
||||
android-dev = {
|
||||
module =
|
||||
{ pkgs, ... }:
|
||||
{
|
||||
environment.systemPackages = [
|
||||
pkgs.android-tools
|
||||
pkgs.gradle
|
||||
pkgs.jdk
|
||||
];
|
||||
};
|
||||
meta = {
|
||||
description = "Builds Android apps: adb and fastboot (android-tools), Gradle and a JDK are installed. It holds the default permissions.";
|
||||
};
|
||||
};
|
||||
}
|
||||
|
|
|
|||
|
|
@ -151,7 +151,7 @@ let
|
|||
# output and prints it as JSON. A derivation or a path in `meta` would
|
||||
# make that evaluation build or copy something; a function would not
|
||||
# print at all.
|
||||
name = "the built-in archetypes carry a module and a meta of plain data, and reviewer can run commands";
|
||||
name = "the built-in archetypes carry a module and a meta of plain data, and reviewer can run commands and android-dev installs its toolchain";
|
||||
ok =
|
||||
let
|
||||
plain =
|
||||
|
|
@ -173,7 +173,10 @@ let
|
|||
&& lib.elem "messaging" reviewer
|
||||
&& lib.elem "inbox" reviewer
|
||||
&& lib.elem "execution" reviewer
|
||||
&& !(lib.elem "meta" reviewer);
|
||||
&& !(lib.elem "meta" reviewer)
|
||||
&&
|
||||
lib.elem pkgs.android-tools
|
||||
(archetypes.android-dev.module { inherit pkgs; }).environment.systemPackages;
|
||||
}
|
||||
{
|
||||
# `docs.enable` reads `docs.source`, whose default builds from the
|
||||
|
|
|
|||
170
swarm-controller/src/agent_archetype.rs
Normal file
170
swarm-controller/src/agent_archetype.rs
Normal file
|
|
@ -0,0 +1,170 @@
|
|||
//! The archetype an agent was created from: one key per archetype agent in
|
||||
//! `agent-archetype` (`swarm_queue_client::agent_archetype`), which this
|
||||
//! daemon alone writes. `POST /api/agents` checks the choice against the
|
||||
//! `agent-archetypes` record and stores it before it queues the creation; a
|
||||
//! destroy deletes it. The hive reads it to build the agent from the
|
||||
//! archetype's module, with no config repo.
|
||||
|
||||
use problem_details::ProblemDetails;
|
||||
use swarm_queue_client::agent_archetype::{self, ArchetypeRef};
|
||||
use swarm_queue_client::archetypes::SourceRecord;
|
||||
|
||||
use crate::schedules::{queue, read_or_default, write_revision};
|
||||
use crate::{AppState, error_problem};
|
||||
|
||||
/// How many times a losing writer re-reads and re-applies, as in
|
||||
/// `wanted::MAX_ATTEMPTS`.
|
||||
const MAX_ATTEMPTS: usize = 5;
|
||||
|
||||
fn queue_problem(e: &anyhow::Error) -> ProblemDetails {
|
||||
tracing::warn!(error = %format!("{e:#}"), "agent archetype: queue operation failed");
|
||||
error_problem(
|
||||
axum::http::StatusCode::INTERNAL_SERVER_ERROR,
|
||||
&format!("{e:#}"),
|
||||
)
|
||||
}
|
||||
|
||||
/// Refuse `choice` unless the stored `agent-archetypes` record lists it.
|
||||
pub(crate) async fn check(
|
||||
client: &async_nats::Client,
|
||||
choice: &ArchetypeRef,
|
||||
) -> Result<(), ProblemDetails> {
|
||||
let record = match swarm_queue_client::archetypes::open(client)
|
||||
.await
|
||||
.map_err(|e| queue_problem(&e.into()))?
|
||||
{
|
||||
Some(store) => {
|
||||
read_or_default::<SourceRecord>(&store, &choice.source)
|
||||
.await
|
||||
.map_err(|e| queue_problem(&e))?
|
||||
.0
|
||||
}
|
||||
None => SourceRecord::default(),
|
||||
};
|
||||
listed(&record, choice).map_err(|why| error_problem(axum::http::StatusCode::BAD_REQUEST, &why))
|
||||
}
|
||||
|
||||
/// Whether `record`, the stored record of `choice.source`, lists `choice`.
|
||||
fn listed(record: &SourceRecord, choice: &ArchetypeRef) -> Result<(), String> {
|
||||
if record.archetypes.contains_key(&choice.name) {
|
||||
return Ok(());
|
||||
}
|
||||
let ArchetypeRef { source, name } = choice;
|
||||
Err(match record.refused.get(name) {
|
||||
Some(why) => format!("archetype {name:?} of {source:?} was refused: {why}"),
|
||||
None => format!("{source:?} has no archetype {name:?}; GET /api/archetypes lists them"),
|
||||
})
|
||||
}
|
||||
|
||||
/// What [`store`] replaced, for [`restore`]; `None` when it wrote nothing.
|
||||
pub(crate) type Replaced = Option<crate::schedules::Replaced<Option<ArchetypeRef>>>;
|
||||
|
||||
/// Record that `agent` is built from `choice`.
|
||||
pub(crate) async fn store(
|
||||
client: &async_nats::Client,
|
||||
agent: &str,
|
||||
choice: &ArchetypeRef,
|
||||
) -> Result<Replaced, ProblemDetails> {
|
||||
let store = agent_archetype::open_or_create(client)
|
||||
.await
|
||||
.map_err(|e| queue_problem(&e.into()))?;
|
||||
let value = Some(choice.clone());
|
||||
for _ in 0..MAX_ATTEMPTS {
|
||||
let (current, revision, undecodable) =
|
||||
read_or_default::<Option<ArchetypeRef>>(&store, agent)
|
||||
.await
|
||||
.map_err(|e| queue_problem(&e))?;
|
||||
if current == value && !undecodable {
|
||||
tracing::info!(%agent, ?choice, "set agent archetype at create");
|
||||
return Ok(None);
|
||||
}
|
||||
if let Some(written) = write_revision(&store, agent, &value, revision)
|
||||
.await
|
||||
.map_err(|e| queue_problem(&e))?
|
||||
{
|
||||
tracing::info!(%agent, ?choice, "set agent archetype at create");
|
||||
return Ok(Some(crate::schedules::Replaced::new(
|
||||
current.map(Some),
|
||||
written,
|
||||
)));
|
||||
}
|
||||
}
|
||||
Err(queue_problem(&anyhow::anyhow!(
|
||||
"gave up writing the archetype of {agent} after {MAX_ATTEMPTS} conflicting writes"
|
||||
)))
|
||||
}
|
||||
|
||||
/// Undo what [`store`] did, unless another write has landed since. Best
|
||||
/// effort: a failure is logged, since the caller is already on its way to an
|
||||
/// error of its own.
|
||||
pub(crate) async fn restore(client: &async_nats::Client, agent: &str, replaced: Replaced) {
|
||||
let Some(replaced) = replaced else { return };
|
||||
let undone = match agent_archetype::open_or_create(client).await {
|
||||
Ok(store) => replaced.undo(&store, agent).await,
|
||||
Err(e) => Err(anyhow::anyhow!("{e:?}")),
|
||||
};
|
||||
match undone {
|
||||
Ok(true) => {}
|
||||
Ok(false) => {
|
||||
tracing::info!(%agent, "left the agent's archetype: another write landed since");
|
||||
}
|
||||
Err(e) => {
|
||||
tracing::warn!(%agent, error = %format!("{e:#}"), "could not put back the agent's archetype");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Delete a destroyed agent's key, so the next agent of that name is not
|
||||
/// built from this one's archetype. A failure is logged and does not stop the
|
||||
/// destroy, as in `agent_permissions::clear_on_destroy`.
|
||||
pub(crate) async fn clear_on_destroy(state: &AppState, hive: &str, agent: &str) {
|
||||
let cleared = async {
|
||||
let client = queue(state).map_err(|(_, why)| anyhow::anyhow!(why))?;
|
||||
agent_archetype::open_or_create(&client)
|
||||
.await?
|
||||
.delete(agent)
|
||||
.await?;
|
||||
anyhow::Ok(())
|
||||
}
|
||||
.await;
|
||||
match cleared {
|
||||
Ok(()) => tracing::info!(hive, agent, "destroyed agent's archetype cleared"),
|
||||
Err(e) => tracing::error!(
|
||||
hive,
|
||||
agent,
|
||||
error = %format!("{e:#}"),
|
||||
"clearing this agent's archetype FAILED; it was destroyed anyway, so a new agent \
|
||||
of the same name is built from this archetype. Declaring the agent destroyed \
|
||||
again re-runs this."
|
||||
),
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use std::collections::BTreeMap;
|
||||
use swarm_queue_client::archetypes::ArchetypeMeta;
|
||||
|
||||
fn choice(name: &str) -> ArchetypeRef {
|
||||
ArchetypeRef {
|
||||
source: "hyperhive".to_owned(),
|
||||
name: name.to_owned(),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn only_a_listed_archetype_passes() {
|
||||
let record = SourceRecord {
|
||||
source: "/nix/store/h".to_owned(),
|
||||
archetypes: BTreeMap::from([("reviewer".to_owned(), ArchetypeMeta::default())]),
|
||||
refused: BTreeMap::from([("broken".to_owned(), "limits: bad".to_owned())]),
|
||||
};
|
||||
assert_eq!(listed(&record, &choice("reviewer")), Ok(()));
|
||||
let refused = listed(&record, &choice("broken")).unwrap_err();
|
||||
assert!(refused.contains("limits: bad"), "{refused}");
|
||||
let unknown = listed(&record, &choice("nope")).unwrap_err();
|
||||
assert!(unknown.contains("no archetype \"nope\""), "{unknown}");
|
||||
listed(&SourceRecord::default(), &choice("reviewer")).unwrap_err();
|
||||
}
|
||||
}
|
||||
|
|
@ -797,6 +797,7 @@ mod tests {
|
|||
limits: None,
|
||||
inference: None,
|
||||
permissions,
|
||||
archetype: None,
|
||||
}),
|
||||
)
|
||||
};
|
||||
|
|
@ -857,6 +858,7 @@ mod tests {
|
|||
limits: None,
|
||||
inference: None,
|
||||
permissions: None,
|
||||
archetype: None,
|
||||
}),
|
||||
)
|
||||
.await
|
||||
|
|
|
|||
|
|
@ -42,6 +42,7 @@ use swarm_authelia_bridge_sock::BridgeResponse;
|
|||
use utoipa::{OpenApi, ToSchema};
|
||||
use utoipa_axum::{router::OpenApiRouter, routes};
|
||||
|
||||
mod agent_archetype;
|
||||
mod agent_control;
|
||||
mod agent_icon;
|
||||
mod agent_identity;
|
||||
|
|
@ -1455,12 +1456,13 @@ async fn set_agent_state(
|
|||
|
||||
// After the declaration and never before it: the write above is the
|
||||
// destroy order, and until it lands the agent is not being torn down, so
|
||||
// its credential and permissions have to keep working. Sequenced rather
|
||||
// than spawned so both attempts are finished — and their outcomes
|
||||
// its credential, permissions and archetype have to keep working. Sequenced
|
||||
// rather than spawned so every attempt is finished — and their outcomes
|
||||
// logged — by the time the operator's call returns.
|
||||
if revokes_queue_credential(req.state) {
|
||||
revoke_or_complain(&hive, &agent).await;
|
||||
agent_permissions::clear_on_destroy(&state, &hive, &agent).await;
|
||||
agent_archetype::clear_on_destroy(&state, &hive, &agent).await;
|
||||
}
|
||||
Ok(Json(render(&declaration)))
|
||||
}
|
||||
|
|
@ -1793,6 +1795,12 @@ struct CreateAgentRequest {
|
|||
#[serde(default)]
|
||||
#[schema(value_type = Option<Vec<String>>)]
|
||||
permissions: Option<Vec<hive_sh4re::permissions::Permission>>,
|
||||
/// The archetype to build the agent from, one `GET /api/archetypes`
|
||||
/// lists. The agent then gets no `agent-configs/<name>` repo: its hive
|
||||
/// builds it from the archetype's module. Absent is a custom agent.
|
||||
#[serde(default)]
|
||||
#[schema(value_type = Option<Object>, example = json!({"source": "hyperhive", "name": "reviewer"}))]
|
||||
archetype: Option<swarm_queue_client::agent_archetype::ArchetypeRef>,
|
||||
}
|
||||
|
||||
/// Where the queued job landed — a caller polls `/api/jobq/graph` (or
|
||||
|
|
@ -1827,9 +1835,11 @@ struct CreateAgentResponse {
|
|||
/// validation error, not an idempotent no-op. `InitAgentConfigRepo` has no
|
||||
/// such second dependency: seeding files uses this daemon's own forge
|
||||
/// token, never the agent's, so it only ever needed the repo (mara, design
|
||||
/// review: it's a jobq graph, not a linear chain). Returns as soon as the
|
||||
/// graph is inserted — **not** once any of it has run; `run_swarm_node`
|
||||
/// does that asynchronously off the scheduler loop already running
|
||||
/// review: it's a jobq graph, not a linear chain). An agent created from an
|
||||
/// `archetype` has no config repo, so its graph has none of the three repo
|
||||
/// nodes. Returns as soon as the graph is inserted — **not** once any of it
|
||||
/// has run; `run_swarm_node` does that asynchronously off the scheduler loop
|
||||
/// already running
|
||||
/// (`spawn_jobq_worker`). The response reports `CreateIdentity`'s id, one
|
||||
/// of the graph's two roots — a caller watches the whole thing settle via
|
||||
/// `/api/jobq/graph`, which serves every root, not just this one.
|
||||
|
|
@ -1846,10 +1856,10 @@ struct CreateAgentResponse {
|
|||
request_body = CreateAgentRequest,
|
||||
responses(
|
||||
(status = 200, description = "job chain queued", body = CreateAgentResponse),
|
||||
(status = 400, description = "`name` or `hive` is not a valid identifier, `name` is new and reserved or refused by the forge, `hive` is not in this swarm, `limits` or `inference` holds a value the matching `PUT` refuses, or a permission is listed twice (problem+json)", body = String),
|
||||
(status = 400, description = "`name` or `hive` is not a valid identifier, `name` is new and reserved or refused by the forge, `hive` is not in this swarm, `limits` or `inference` holds a value the matching `PUT` refuses, a permission is listed twice, or `archetype` is not one the stored archetype list holds (problem+json)", body = String),
|
||||
(status = 409, description = "the swarm has already placed `name` on a different hive (problem+json)", body = String),
|
||||
(status = 422, description = "an unknown field in the body, in `limits`, or in `inference`, an `effort` that is not one of the five levels, or a `permissions` entry that is not a permission name"),
|
||||
(status = 503, description = "the swarm queue is not connected, so whether `name` is placed on another hive is unknown, or `limits`, `inference` or `permissions` is set and cannot be stored (problem+json)", body = String),
|
||||
(status = 503, description = "the swarm queue is not connected, so whether `name` is placed on another hive is unknown, or `limits`, `inference`, `permissions` or `archetype` is set and cannot be stored (problem+json)", body = String),
|
||||
(status = 500, description = "the job chain could not be queued, or another hive's wanted state could not be read (problem+json)", body = String),
|
||||
),
|
||||
tag = "agents"
|
||||
|
|
@ -1864,7 +1874,7 @@ async fn create_agent(
|
|||
|
||||
// Settings are checked before anything is created, so a bad value stops
|
||||
// the request rather than leaving an agent without them.
|
||||
let settings = CreateSettings::parse(req.limits, req.inference, req.permissions)
|
||||
let settings = CreateSettings::parse(req.limits, req.inference, req.permissions, req.archetype)
|
||||
.map_err(|why| error_problem(axum::http::StatusCode::BAD_REQUEST, &why))?;
|
||||
|
||||
// Two checks, and the second is the one that makes the field worth
|
||||
|
|
@ -1928,11 +1938,7 @@ async fn create_agent(
|
|||
}
|
||||
// Checked after the roster, so a mistyped hive stays a 400 while the queue
|
||||
// is down.
|
||||
let settings_queue = if settings.any() {
|
||||
Some(schedules::queue(&state).map_err(schedules::refused)?)
|
||||
} else {
|
||||
None
|
||||
};
|
||||
let settings_queue = settings.queue(&state).await?;
|
||||
|
||||
// Agent names are one swarm-wide namespace: identity, forge user, store
|
||||
// path and matrix id all carry the bare name. The same name on the same
|
||||
|
|
@ -1965,6 +1971,7 @@ async fn create_agent(
|
|||
settings.limits.as_ref(),
|
||||
settings.inference.as_ref(),
|
||||
settings.permissions.as_ref(),
|
||||
settings.archetype.as_ref(),
|
||||
)
|
||||
.await?
|
||||
}
|
||||
|
|
@ -1975,7 +1982,9 @@ async fn create_agent(
|
|||
.jobq
|
||||
.lock()
|
||||
.unwrap_or_else(std::sync::PoisonError::into_inner);
|
||||
sched.insert_job(None, |b| declare_agent_job(b, &agent, &hive))
|
||||
sched.insert_job(None, |b| {
|
||||
declare_agent_job(b, &agent, &hive, settings.archetype.is_some())
|
||||
})
|
||||
};
|
||||
let ids = match queued {
|
||||
Ok(ids) => ids,
|
||||
|
|
@ -2003,27 +2012,57 @@ struct CreateSettings {
|
|||
limits: Option<swarm_queue_client::agent_limits::AgentCaps>,
|
||||
inference: Option<swarm_queue_client::agent_inference::AgentInference>,
|
||||
permissions: Option<swarm_queue_client::agent_permissions::AgentPermissions>,
|
||||
archetype: Option<swarm_queue_client::agent_archetype::ArchetypeRef>,
|
||||
}
|
||||
|
||||
impl CreateSettings {
|
||||
/// Each setting given, or why its `PUT` would refuse the first bad one.
|
||||
/// An archetype is checked here for its shape only; whether the swarm
|
||||
/// lists it is [`agent_archetype::check`].
|
||||
fn parse(
|
||||
limits: Option<agent_limits::Caps>,
|
||||
inference: Option<inference::InferenceSettings>,
|
||||
permissions: Option<Vec<hive_sh4re::permissions::Permission>>,
|
||||
archetype: Option<swarm_queue_client::agent_archetype::ArchetypeRef>,
|
||||
) -> Result<Self, String> {
|
||||
if let Some(a) = &archetype {
|
||||
for part in [&a.source, &a.name] {
|
||||
hive_types::Ident::parse(part)
|
||||
.map_err(|why| format!("archetype {part:?}: {why}"))?;
|
||||
}
|
||||
}
|
||||
Ok(Self {
|
||||
limits: limits.map(agent_limits::Caps::into_stored).transpose()?,
|
||||
inference: inference
|
||||
.map(inference::InferenceSettings::into_stored)
|
||||
.transpose()?,
|
||||
permissions: permissions.map(agent_permissions::at_create).transpose()?,
|
||||
archetype,
|
||||
})
|
||||
}
|
||||
|
||||
/// Whether any setting is given.
|
||||
fn any(&self) -> bool {
|
||||
self.limits.is_some() || self.inference.is_some() || self.permissions.is_some()
|
||||
self.limits.is_some()
|
||||
|| self.inference.is_some()
|
||||
|| self.permissions.is_some()
|
||||
|| self.archetype.is_some()
|
||||
}
|
||||
|
||||
/// The queue to store the settings through, `None` when none is given.
|
||||
/// An archetype is refused here unless the stored list holds it.
|
||||
async fn queue(
|
||||
&self,
|
||||
state: &AppState,
|
||||
) -> Result<Option<async_nats::Client>, problem_details::ProblemDetails> {
|
||||
if !self.any() {
|
||||
return Ok(None);
|
||||
}
|
||||
let client = schedules::queue(state).map_err(schedules::refused)?;
|
||||
if let Some(archetype) = &self.archetype {
|
||||
agent_archetype::check(&client, archetype).await?;
|
||||
}
|
||||
Ok(Some(client))
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -2034,17 +2073,19 @@ struct StoredSettings {
|
|||
limits: agent_limits::Replaced,
|
||||
inference: inference::Replaced,
|
||||
permissions: agent_permissions::Replaced,
|
||||
archetype: agent_archetype::Replaced,
|
||||
}
|
||||
|
||||
impl StoredSettings {
|
||||
/// Store whichever of `limits`, `permissions` and `inference` are given, in
|
||||
/// that order. A failure on one puts back the ones stored before it.
|
||||
/// Store whichever of `limits`, `permissions`, `inference` and `archetype` are
|
||||
/// given, in that order. A failure on one puts back the ones stored before it.
|
||||
async fn store(
|
||||
client: &async_nats::Client,
|
||||
agent: &str,
|
||||
limits: Option<&swarm_queue_client::agent_limits::AgentCaps>,
|
||||
inference: Option<&swarm_queue_client::agent_inference::AgentInference>,
|
||||
permissions: Option<&swarm_queue_client::agent_permissions::AgentPermissions>,
|
||||
archetype: Option<&swarm_queue_client::agent_archetype::ArchetypeRef>,
|
||||
) -> Result<Self, problem_details::ProblemDetails> {
|
||||
let mut stored = Self::default();
|
||||
if let Some(limits) = limits {
|
||||
|
|
@ -2068,6 +2109,15 @@ impl StoredSettings {
|
|||
}
|
||||
}
|
||||
}
|
||||
if let Some(archetype) = archetype {
|
||||
match agent_archetype::store(client, agent, archetype).await {
|
||||
Ok(replaced) => stored.archetype = replaced,
|
||||
Err(e) => {
|
||||
stored.roll_back(client, agent).await;
|
||||
return Err(e);
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(stored)
|
||||
}
|
||||
|
||||
|
|
@ -2075,6 +2125,7 @@ impl StoredSettings {
|
|||
agent_limits::restore(client, agent, self.limits).await;
|
||||
inference::restore(client, agent, self.inference).await;
|
||||
agent_permissions::restore(client, agent, self.permissions).await;
|
||||
agent_archetype::restore(client, agent, self.archetype).await;
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -2430,6 +2481,7 @@ fn declare_agent_job(
|
|||
b: &hive_jobq::builder::JobBuilder<SwarmNodeKind, SwarmResourceKind>,
|
||||
agent: &str,
|
||||
hive: &str,
|
||||
archetype: bool,
|
||||
) -> Vec<hive_jobq::builder::NodeGuid> {
|
||||
let create_identity = b.node(SwarmNodeKind::CreateIdentity {
|
||||
agent: agent.to_owned(),
|
||||
|
|
@ -2440,22 +2492,31 @@ fn declare_agent_job(
|
|||
let create_forge_user = b.node(SwarmNodeKind::CreateForgeUser {
|
||||
agent: agent.to_owned(),
|
||||
});
|
||||
let create_repo = b
|
||||
.node(SwarmNodeKind::CreateRepo {
|
||||
agent: agent.to_owned(),
|
||||
})
|
||||
.after_ok(create_identity);
|
||||
// `AddRepoMember` needs both parents: the repo to add a collaborator to,
|
||||
// and the forge user to add as one — adding a nonexistent user is a
|
||||
// Forgejo validation error, not an idempotent no-op.
|
||||
// `InitAgentConfigRepo` needs only the repo — see the doc comment above
|
||||
// for why.
|
||||
let _add_repo_member = b
|
||||
.node(SwarmNodeKind::AddRepoMember {
|
||||
// An archetype agent has no config repo: its hive builds it from the
|
||||
// archetype's module, so there is no repo to create, share or seed.
|
||||
let init_config = (!archetype).then(|| {
|
||||
let create_repo = b
|
||||
.node(SwarmNodeKind::CreateRepo {
|
||||
agent: agent.to_owned(),
|
||||
})
|
||||
.after_ok(create_identity);
|
||||
// `AddRepoMember` needs both parents: the repo to add a collaborator
|
||||
// to, and the forge user to add as one — adding a nonexistent user is
|
||||
// a Forgejo validation error, not an idempotent no-op.
|
||||
// `InitAgentConfigRepo` needs only the repo — see the doc comment
|
||||
// above for why.
|
||||
let _add_repo_member = b
|
||||
.node(SwarmNodeKind::AddRepoMember {
|
||||
agent: agent.to_owned(),
|
||||
})
|
||||
.after_ok(create_repo)
|
||||
.after_ok(create_forge_user);
|
||||
b.node(SwarmNodeKind::InitAgentConfigRepo {
|
||||
agent: agent.to_owned(),
|
||||
})
|
||||
.after_ok(create_repo)
|
||||
.after_ok(create_forge_user);
|
||||
.guid()
|
||||
});
|
||||
// Read on `internal/docs`, a repo that exists whatever this agent is, so
|
||||
// it needs only the user. Not a parent of the deploy: the agent reads docs
|
||||
// once running, and a failed grant is retried by `forge::docs_grant::spawn`.
|
||||
|
|
@ -2464,11 +2525,6 @@ fn declare_agent_job(
|
|||
agent: agent.to_owned(),
|
||||
})
|
||||
.after_ok(create_forge_user);
|
||||
let init_config = b
|
||||
.node(SwarmNodeKind::InitAgentConfigRepo {
|
||||
agent: agent.to_owned(),
|
||||
})
|
||||
.after_ok(create_repo);
|
||||
// The agent's own identity at the swarm secret store, minted and
|
||||
// published at swarm level so no hive ever needs the capability to mint
|
||||
// one. After `create_identity` because the certificate names a principal
|
||||
|
|
@ -2507,7 +2563,8 @@ fn declare_agent_job(
|
|||
// deploys by reading that repo, so a deploy asked for any earlier would
|
||||
// find nothing to build. This is the edge that makes creating an agent at
|
||||
// swarm level actually put it on a hive, rather than leaving a
|
||||
// provisioned name nobody runs.
|
||||
// provisioned name nobody runs. An archetype agent has no repo, so its
|
||||
// deploy waits on the mints and the declaration alone.
|
||||
//
|
||||
// `after_any` on the mint, not `after_ok`: a hive cannot pass down a
|
||||
// certificate the swarm has not published, so the deploy must not
|
||||
|
|
@ -2526,13 +2583,12 @@ fn declare_agent_job(
|
|||
// queue at all fails this node — but `TriggerDeploy` has nothing to
|
||||
// publish to on such a host either, so the strict edge cancels a node
|
||||
// that could not have succeeded anyway.
|
||||
let _trigger_deploy = b
|
||||
let trigger_deploy = b
|
||||
.node(SwarmNodeKind::TriggerDeploy {
|
||||
hive: hive.to_owned(),
|
||||
agent: agent.to_owned(),
|
||||
rev: None,
|
||||
})
|
||||
.after_ok(init_config)
|
||||
.after_any(mint_identity)
|
||||
.after_any(mint_forge_token)
|
||||
// `after_any` for the same reason: the container reads this token
|
||||
|
|
@ -2541,6 +2597,9 @@ fn declare_agent_job(
|
|||
// and only this node fails, by name.
|
||||
.after_any(mint_matrix)
|
||||
.after_ok(set_wanted);
|
||||
if let Some(init_config) = init_config {
|
||||
let _ = trigger_deploy.after_ok(init_config);
|
||||
}
|
||||
vec![create_identity.guid()]
|
||||
}
|
||||
|
||||
|
|
@ -3904,6 +3963,7 @@ mod tests {
|
|||
limits: None,
|
||||
inference: None,
|
||||
permissions: None,
|
||||
archetype: None,
|
||||
}),
|
||||
)
|
||||
.await
|
||||
|
|
@ -3945,6 +4005,7 @@ mod tests {
|
|||
limits: None,
|
||||
inference: None,
|
||||
permissions: None,
|
||||
archetype: None,
|
||||
}),
|
||||
)
|
||||
.await
|
||||
|
|
@ -4108,6 +4169,7 @@ mod tests {
|
|||
Some(&caps(r#"{"cpu_quota":"400%"}"#)),
|
||||
Some(&inference),
|
||||
None,
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
assert!(stored.is_err(), "{agent}: the inference write must fail");
|
||||
|
|
@ -4246,6 +4308,7 @@ mod tests {
|
|||
limits: None,
|
||||
inference: None,
|
||||
permissions: None,
|
||||
archetype: None,
|
||||
}),
|
||||
)
|
||||
.await
|
||||
|
|
@ -5374,6 +5437,7 @@ mod tests {
|
|||
limits: None,
|
||||
inference: None,
|
||||
permissions: None,
|
||||
archetype: None,
|
||||
}),
|
||||
)
|
||||
.await
|
||||
|
|
@ -5408,6 +5472,171 @@ mod tests {
|
|||
);
|
||||
}
|
||||
|
||||
/// Node labels of the graph [`super::declare_agent_job`] builds, and the
|
||||
/// labels the deploy node waits on.
|
||||
fn creation_graph(archetype: bool) -> (Vec<String>, Vec<String>) {
|
||||
use hive_jobq_wire::WireNode as _;
|
||||
|
||||
let mut sched = hive_jobq::scheduler::Scheduler::new(
|
||||
hive_jobq::Graph::new(),
|
||||
hive_jobq::resources::ResourceTable::new(),
|
||||
);
|
||||
sched
|
||||
.insert_job(None, |b| {
|
||||
super::declare_agent_job(b, "atlas", "pr1ma", archetype)
|
||||
})
|
||||
.expect("the graph inserts");
|
||||
let graph = sched.graph();
|
||||
let labels = graph.nodes().map(|n| n.payload.label()).collect();
|
||||
let deploy = graph
|
||||
.nodes()
|
||||
.find(|n| n.payload.label() == "trigger_deploy")
|
||||
.expect("every creation deploys");
|
||||
let waits = deploy
|
||||
.deps
|
||||
.iter()
|
||||
.filter_map(|d| match d {
|
||||
hive_jobq::Dep::Node { id, .. } => graph.node(*id).map(|n| n.payload.label()),
|
||||
hive_jobq::Dep::Resource { .. } => None,
|
||||
})
|
||||
.collect();
|
||||
(labels, waits)
|
||||
}
|
||||
|
||||
/// An archetype agent's graph has no repo nodes and its deploy waits on
|
||||
/// no repo; a custom agent's has all three, and its deploy waits on the
|
||||
/// seeded repo.
|
||||
#[test]
|
||||
fn an_archetype_agent_is_created_without_a_config_repo() {
|
||||
const REPO_NODES: [&str; 3] = ["create_repo", "add_repo_member", "init_agent_config_repo"];
|
||||
|
||||
let (custom, custom_waits) = creation_graph(false);
|
||||
for repo in REPO_NODES {
|
||||
assert!(custom.iter().any(|l| l == repo), "custom lacks {repo}");
|
||||
}
|
||||
assert!(custom_waits.iter().any(|l| l == "init_agent_config_repo"));
|
||||
|
||||
let (archetype, archetype_waits) = creation_graph(true);
|
||||
for repo in REPO_NODES {
|
||||
assert!(!archetype.iter().any(|l| l == repo), "archetype has {repo}");
|
||||
}
|
||||
for kept in [
|
||||
"create_identity",
|
||||
"create_forge_user",
|
||||
"grant_docs_read",
|
||||
"mint_agent_identity",
|
||||
"mint_agent_forge_token",
|
||||
"mint_agent_matrix_account",
|
||||
"set_agent_wanted",
|
||||
] {
|
||||
assert!(
|
||||
archetype.iter().any(|l| l == kept),
|
||||
"archetype lacks {kept}"
|
||||
);
|
||||
assert!(custom.iter().any(|l| l == kept), "custom lacks {kept}");
|
||||
}
|
||||
assert!(archetype_waits.iter().any(|l| l == "set_agent_wanted"));
|
||||
assert_eq!(archetype.len() + REPO_NODES.len(), custom.len());
|
||||
}
|
||||
|
||||
/// Seed the stored archetype list with `reviewer` under `hyperhive`.
|
||||
async fn seed_archetypes(client: &async_nats::Client) {
|
||||
let record = swarm_queue_client::archetypes::SourceRecord {
|
||||
source: "/nix/store/h".to_owned(),
|
||||
archetypes: std::collections::BTreeMap::from([(
|
||||
"reviewer".to_owned(),
|
||||
swarm_queue_client::archetypes::ArchetypeMeta {
|
||||
description: "reviews".to_owned(),
|
||||
..Default::default()
|
||||
},
|
||||
)]),
|
||||
refused: std::collections::BTreeMap::new(),
|
||||
};
|
||||
swarm_queue_client::archetypes::open_or_create(client)
|
||||
.await
|
||||
.expect("bucket")
|
||||
.put(
|
||||
swarm_queue_client::archetypes::BUILTIN_SOURCE,
|
||||
serde_json::to_vec(&record).expect("encodes").into(),
|
||||
)
|
||||
.await
|
||||
.expect("seeded");
|
||||
}
|
||||
|
||||
/// Through the real handler and a real `nats-server`: a listed archetype
|
||||
/// lands under the agent's key as the hive reads it, and the queued graph
|
||||
/// has no config repo.
|
||||
#[tokio::test]
|
||||
async fn a_creation_from_an_archetype_records_it_and_queues_no_repo() {
|
||||
use hive_jobq_wire::WireNode as _;
|
||||
|
||||
let nats = crate::test_nats::TestNats::start();
|
||||
let client = nats.client().await;
|
||||
seed_archetypes(&client).await;
|
||||
let state = crate::agent_limits::tests::state(&client);
|
||||
let req: super::CreateAgentRequest = serde_json::from_str(
|
||||
r#"{"name":"atlas","hive":"pr1ma",
|
||||
"archetype":{"source":"hyperhive","name":"reviewer"}}"#,
|
||||
)
|
||||
.expect("body parses");
|
||||
let _queued = super::create_agent(axum::extract::State(state.clone()), axum::Json(req))
|
||||
.await
|
||||
.expect("a listed archetype goes through");
|
||||
|
||||
let stored = swarm_queue_client::agent_archetype::open_or_create(&client)
|
||||
.await
|
||||
.expect("bucket")
|
||||
.get("atlas")
|
||||
.await
|
||||
.expect("read")
|
||||
.expect("a value");
|
||||
assert_eq!(&stored[..], br#"{"source":"hyperhive","name":"reviewer"}"#);
|
||||
let labels: Vec<String> = state
|
||||
.jobq
|
||||
.lock()
|
||||
.unwrap_or_else(std::sync::PoisonError::into_inner)
|
||||
.graph()
|
||||
.nodes()
|
||||
.map(|n| n.payload.label())
|
||||
.collect();
|
||||
assert!(labels.iter().any(|l| l == "trigger_deploy"), "{labels:?}");
|
||||
assert!(!labels.iter().any(|l| l == "create_repo"), "{labels:?}");
|
||||
}
|
||||
|
||||
/// An archetype the stored list does not hold is a 400 that stores and
|
||||
/// queues nothing, and so is a name that is not an identifier.
|
||||
#[tokio::test]
|
||||
async fn an_unlisted_archetype_is_refused_before_anything_is_stored() {
|
||||
let nats = crate::test_nats::TestNats::start();
|
||||
let client = nats.client().await;
|
||||
seed_archetypes(&client).await;
|
||||
let state = crate::agent_limits::tests::state(&client);
|
||||
for archetype in [
|
||||
r#"{"source":"hyperhive","name":"nope"}"#,
|
||||
r#"{"source":"elsewhere","name":"reviewer"}"#,
|
||||
r#"{"source":"hyperhive","name":"a.b"}"#,
|
||||
] {
|
||||
let req: super::CreateAgentRequest = serde_json::from_str(&format!(
|
||||
r#"{{"name":"atlas","hive":"pr1ma","permissions":["messaging"],
|
||||
"archetype":{archetype}}}"#
|
||||
))
|
||||
.expect("body parses");
|
||||
let err = super::create_agent(axum::extract::State(state.clone()), axum::Json(req))
|
||||
.await
|
||||
.expect_err("an unlisted archetype must be refused");
|
||||
assert_eq!(
|
||||
err.status,
|
||||
Some(axum::http::StatusCode::BAD_REQUEST),
|
||||
"{archetype}: {err:?}"
|
||||
);
|
||||
}
|
||||
assert_eq!(node_count(&state.jobq), 0, "nothing may be queued");
|
||||
let permissions = swarm_queue_client::agent_permissions::open_or_create(&client)
|
||||
.await
|
||||
.expect("bucket");
|
||||
assert_eq!(permissions.get("atlas").await.expect("read"), None);
|
||||
}
|
||||
|
||||
/// The ordering the operator's ruling requires: a hive cannot pass down
|
||||
/// a certificate the swarm has not published, so the deploy message must
|
||||
/// not leave before the mint is terminal.
|
||||
|
|
@ -5428,6 +5657,7 @@ mod tests {
|
|||
limits: None,
|
||||
inference: None,
|
||||
permissions: None,
|
||||
archetype: None,
|
||||
}),
|
||||
)
|
||||
.await
|
||||
|
|
@ -5478,6 +5708,7 @@ mod tests {
|
|||
limits: None,
|
||||
inference: None,
|
||||
permissions: None,
|
||||
archetype: None,
|
||||
}),
|
||||
)
|
||||
.await
|
||||
|
|
@ -5536,6 +5767,7 @@ mod tests {
|
|||
limits: None,
|
||||
inference: None,
|
||||
permissions: None,
|
||||
archetype: None,
|
||||
}),
|
||||
)
|
||||
.await
|
||||
|
|
@ -5741,6 +5973,7 @@ mod tests {
|
|||
limits: None,
|
||||
inference: None,
|
||||
permissions: None,
|
||||
archetype: None,
|
||||
}),
|
||||
)
|
||||
.await
|
||||
|
|
|
|||
|
|
@ -583,6 +583,13 @@ impl Policy {
|
|||
"$JS.API.DIRECT.GET.{}.>",
|
||||
swarm_queue_client::agent_permissions::STREAM
|
||||
));
|
||||
// The archetype an agent was created from
|
||||
// (`swarm_queue_client::agent_archetype`), read the same way and every
|
||||
// key for the same reason.
|
||||
subjects.push(format!(
|
||||
"$JS.API.DIRECT.GET.{}.>",
|
||||
swarm_queue_client::agent_archetype::STREAM
|
||||
));
|
||||
// The one-shot import of the grants this hive's meta repo held, sent
|
||||
// as a request on this hive's own subject: the controller takes the
|
||||
// hive from the subject, so one hive cannot import for another.
|
||||
|
|
@ -877,6 +884,38 @@ mod tests {
|
|||
}
|
||||
}
|
||||
|
||||
/// A hive reads any agent's archetype and can change none of them, and the
|
||||
/// grant reaches no other bucket.
|
||||
#[test]
|
||||
fn a_hive_may_read_every_agents_archetype_and_write_none() {
|
||||
use swarm_queue_client::agent_archetype::{STREAM, direct_get_subject, key_subject};
|
||||
|
||||
let p = policy()
|
||||
.permissions("hive-alpha")
|
||||
.expect("a hive is admitted");
|
||||
let may = |subject: &str| p.publish.iter().any(|g| covers(g, subject));
|
||||
|
||||
for agent in ["atlas", "iris"] {
|
||||
assert!(may(&direct_get_subject(agent)), "cannot read {agent}");
|
||||
}
|
||||
for forbidden in [
|
||||
key_subject("atlas"),
|
||||
format!("$JS.API.STREAM.CREATE.{STREAM}"),
|
||||
format!("$JS.API.STREAM.UPDATE.{STREAM}"),
|
||||
format!("$JS.API.STREAM.DELETE.{STREAM}"),
|
||||
format!("$JS.API.STREAM.PURGE.{STREAM}"),
|
||||
format!("$JS.API.STREAM.MSG.DELETE.{STREAM}"),
|
||||
format!("$JS.API.CONSUMER.CREATE.{STREAM}"),
|
||||
] {
|
||||
assert!(!may(&forbidden), "hive alpha may publish {forbidden}");
|
||||
}
|
||||
// Not the per-source archetype list, which only the controller reads.
|
||||
assert!(!may(&format!(
|
||||
"$JS.API.DIRECT.GET.KV_{0}.$KV.{0}.hyperhive",
|
||||
swarm_queue_client::archetypes::BUCKET
|
||||
)));
|
||||
}
|
||||
|
||||
/// A hive reads any agent's permissions and can change none of them; an
|
||||
/// agent, with the template `swarm-nats.nix` passes, reads its own key and
|
||||
/// no other, and writes none.
|
||||
|
|
|
|||
|
|
@ -59,6 +59,9 @@ agent-limits = ["async-nats/jetstream"]
|
|||
# `Permission` names, so the crates that only need the bucket name do not link
|
||||
# it.
|
||||
agent-permissions = ["dep:hive-sh4re", "async-nats/jetstream"]
|
||||
# `agent_archetype::read`, for hive-c0re, which reads the archetype of each
|
||||
# agent it provisions or rebuilds. Explicit for the same reason as `notices`.
|
||||
agent-archetype = ["async-nats/jetstream"]
|
||||
|
||||
[dependencies]
|
||||
# Bare (no `kv`/`jetstream`) unless a consumer opts into the `kv` feature
|
||||
|
|
|
|||
159
swarm-queue-client/src/agent_archetype.rs
Normal file
159
swarm-queue-client/src/agent_archetype.rs
Normal file
|
|
@ -0,0 +1,159 @@
|
|||
//! The per-agent archetype KV bucket: one key per agent created from an
|
||||
//! archetype, holding which archetype that is.
|
||||
//!
|
||||
//! **swarm-controller is the only writer**: `POST /api/agents` stores the key
|
||||
//! before it queues the agent's creation, and a destroy deletes it. An agent
|
||||
//! with no key is a custom agent, built from its own `agent-configs/<name>`
|
||||
//! repo.
|
||||
//!
|
||||
//! **The hive running the agent reads the key** when it provisions or
|
||||
//! rebuilds the agent, and renders the archetype's module into its meta flake
|
||||
//! in place of a config repo. Its grant covers every key of this bucket,
|
||||
//! read-only, for the reason [`crate::agent_limits`] gives.
|
||||
//!
|
||||
//! Keyed by the agent alone, like [`crate::agent_limits`]: an agent can move
|
||||
//! between hives, and its archetype goes with it.
|
||||
|
||||
/// The KV bucket archetype choices are published into, keyed by agent name.
|
||||
pub const BUCKET: &str = "agent-archetype";
|
||||
|
||||
/// The stream backing [`BUCKET`]. `create_key_value` prefixes `KV_`.
|
||||
pub const STREAM: &str = "KV_agent-archetype";
|
||||
|
||||
/// The subject `agent`'s entry is stored under.
|
||||
#[must_use]
|
||||
pub fn key_subject(agent: &str) -> String {
|
||||
format!("$KV.{BUCKET}.{agent}")
|
||||
}
|
||||
|
||||
/// The request subject that reads `agent`'s current entry.
|
||||
#[must_use]
|
||||
pub fn direct_get_subject(agent: &str) -> String {
|
||||
format!("$JS.API.DIRECT.GET.{STREAM}.{}", key_subject(agent))
|
||||
}
|
||||
|
||||
/// Which archetype an agent was created from: the value under
|
||||
/// `BUCKET`/`<agent>`. `source` is a key of [`crate::archetypes::BUCKET`]
|
||||
/// and `name` an archetype in that source's record.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
|
||||
pub struct ArchetypeRef {
|
||||
pub source: String,
|
||||
pub name: String,
|
||||
}
|
||||
|
||||
/// Open the archetype-choice bucket for writing, creating it if nothing has
|
||||
/// yet.
|
||||
///
|
||||
/// **Controller-side only.** `history: 1`: a hive wants the current value.
|
||||
#[cfg(feature = "kv")]
|
||||
pub async fn open_or_create(
|
||||
client: &async_nats::Client,
|
||||
) -> Result<async_nats::jetstream::kv::Store, crate::Error> {
|
||||
let js = async_nats::jetstream::new(client.clone());
|
||||
crate::kv::open_or_create(
|
||||
&js,
|
||||
BUCKET,
|
||||
"agent-archetype",
|
||||
"The archetype each agent was created from, set by the swarm controller".to_owned(),
|
||||
)
|
||||
.await
|
||||
}
|
||||
|
||||
/// What reading one agent's key found.
|
||||
#[cfg(feature = "agent-archetype")]
|
||||
#[derive(Debug)]
|
||||
pub enum Read {
|
||||
/// The bucket answered: the agent's archetype, or `None` for an absent or
|
||||
/// deleted key.
|
||||
Current(Option<ArchetypeRef>),
|
||||
/// Nothing answers for the bucket's stream: no archetype agent has been
|
||||
/// created yet, or `JetStream` is not serving it right now.
|
||||
NoBucket,
|
||||
}
|
||||
|
||||
/// Read `agent`'s current archetype with one direct get.
|
||||
///
|
||||
/// # Errors
|
||||
///
|
||||
/// A failed request or an undecodable value. A request the queue refused for
|
||||
/// permissions reaches the client as a timeout, not as a refusal.
|
||||
#[cfg(feature = "agent-archetype")]
|
||||
pub async fn read(
|
||||
client: &async_nats::Client,
|
||||
agent: &str,
|
||||
) -> Result<Read, Box<dyn std::error::Error + Send + Sync>> {
|
||||
use async_nats::jetstream::stream::DirectGetErrorKind;
|
||||
|
||||
let js = async_nats::jetstream::new(client.clone());
|
||||
let stream = js.get_stream_no_info(STREAM).await?;
|
||||
let message = match stream.direct_get_last_for_subject(key_subject(agent)).await {
|
||||
Ok(message) => message,
|
||||
Err(e) if matches!(e.kind(), DirectGetErrorKind::NotFound) => {
|
||||
return Ok(Read::Current(None));
|
||||
}
|
||||
Err(e)
|
||||
if matches!(
|
||||
e.kind(),
|
||||
DirectGetErrorKind::ErrorResponse(status, _)
|
||||
if status == async_nats::StatusCode::NO_RESPONDERS
|
||||
) =>
|
||||
{
|
||||
return Ok(Read::NoBucket);
|
||||
}
|
||||
Err(e) => return Err(e.into()),
|
||||
};
|
||||
Ok(Read::Current(decode(
|
||||
message
|
||||
.headers
|
||||
.get("KV-Operation")
|
||||
.map(async_nats::HeaderValue::as_str),
|
||||
&message.payload,
|
||||
)?))
|
||||
}
|
||||
|
||||
/// One stored message as a value: a KV delete or purge is stored as an empty
|
||||
/// marker carrying the `KV-Operation` header, and reads as no archetype.
|
||||
#[cfg(feature = "agent-archetype")]
|
||||
fn decode(operation: Option<&str>, payload: &[u8]) -> serde_json::Result<Option<ArchetypeRef>> {
|
||||
if matches!(operation, Some("DEL" | "PURGE")) {
|
||||
return Ok(None);
|
||||
}
|
||||
serde_json::from_slice(payload).map(Some)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
/// The names the grant in `swarm-nats-auth` is built from.
|
||||
#[test]
|
||||
fn names_match_the_queue_grant() {
|
||||
assert_eq!(STREAM, format!("KV_{BUCKET}"));
|
||||
assert_eq!(key_subject("iris"), "$KV.agent-archetype.iris");
|
||||
assert_eq!(
|
||||
direct_get_subject("iris"),
|
||||
"$JS.API.DIRECT.GET.KV_agent-archetype.$KV.agent-archetype.iris"
|
||||
);
|
||||
}
|
||||
|
||||
/// A stored choice reads back, a field a newer controller adds is
|
||||
/// ignored, and a deleted key is no archetype.
|
||||
#[cfg(feature = "agent-archetype")]
|
||||
#[test]
|
||||
fn decoding_reads_the_choice_and_a_delete_as_none() {
|
||||
let android = ArchetypeRef {
|
||||
source: "hyperhive".to_owned(),
|
||||
name: "android-dev".to_owned(),
|
||||
};
|
||||
assert_eq!(
|
||||
decode(
|
||||
None,
|
||||
br#"{"source":"hyperhive","name":"android-dev","later":1}"#
|
||||
)
|
||||
.unwrap(),
|
||||
Some(android)
|
||||
);
|
||||
assert_eq!(decode(Some("DEL"), b"").unwrap(), None);
|
||||
assert_eq!(decode(Some("PURGE"), b"").unwrap(), None);
|
||||
}
|
||||
}
|
||||
|
|
@ -232,6 +232,12 @@ pub mod agent_permissions;
|
|||
/// unconditional; writing needs `kv` and reading needs `operator-profiles`.
|
||||
pub mod operator_profiles;
|
||||
|
||||
/// The bucket swarm-controller records each archetype agent's archetype in
|
||||
/// — one key per agent, read by the hive that runs it. The names and the
|
||||
/// stored type are unconditional; writing needs `kv` and reading needs
|
||||
/// `agent-archetype`.
|
||||
pub mod agent_archetype;
|
||||
|
||||
/// The login flag in an agent's turn-state header. No bucket: it travels
|
||||
/// on [`agent_state_subject`] only.
|
||||
pub mod agent_login;
|
||||
|
|
|
|||
Loading…
Reference in a new issue