Compare commits
30 changed files with 130 additions and 1211 deletions
|
|
@ -106,16 +106,13 @@ state dirs work as for every other agent.
|
||||||
deploy log). Any agent holding it gets the same mounts.
|
deploy log). Any agent holding it gets the same mounts.
|
||||||
The operator dashboard's `POST /api/meta-update` is the only path
|
The operator dashboard's `POST /api/meta-update` is the only path
|
||||||
that writes `flake.lock`.
|
that writes `flake.lock`.
|
||||||
- **Prompt** — identical for every container but one block (below):
|
- **Prompt** — identical for every container: `prompts/system.md`
|
||||||
`prompts/system.md`
|
|
||||||
carries `<!-- role:agent -->` / `<!-- role:manager -->` marker
|
carries `<!-- role:agent -->` / `<!-- role:manager -->` marker
|
||||||
blocks, and `prompt::render` filters for `"agent"` unconditionally,
|
blocks, and `prompt::render` filters for `"agent"` unconditionally,
|
||||||
manager included ("always `agent` role — there is only one role",
|
manager included ("always `agent` role — there is only one role",
|
||||||
`hive-agent/src/prompt.rs`'s own module doc). The `role:manager`
|
`hive-agent/src/prompt.rs`'s own module doc). The `role:manager`
|
||||||
blocks are dead in production, exercised only by a unit test
|
blocks are dead in production, exercised only by a unit test
|
||||||
(`filter_blocks(SAMPLE, "role", "manager")`). The one block that differs
|
(`filter_role_blocks(SAMPLE, "manager")`).
|
||||||
is `<!-- config:repo -->` vs `<!-- config:archetype -->`: an agent built
|
|
||||||
from an [archetype](../swarm/archetypes.md) is told it has no config repo.
|
|
||||||
- **Tool allow-list** — the MCP tools claude sees come from the agent's
|
- **Tool allow-list** — the MCP tools claude sees come from the agent's
|
||||||
permissions alone, the same mechanism for every agent
|
permissions alone, the same mechanism for every agent
|
||||||
([`docs/turn-loop/mcp.md`](../turn-loop/mcp.md)). Ruth's permissions
|
([`docs/turn-loop/mcp.md`](../turn-loop/mcp.md)). Ruth's permissions
|
||||||
|
|
|
||||||
|
|
@ -65,8 +65,6 @@ Creating a brand-new agent is swarm-level (`swarmctl agent create`,
|
||||||
an agent. The swarm controller's `InitAgentConfigRepo` job seeds the
|
an agent. The swarm controller's `InitAgentConfigRepo` job seeds the
|
||||||
agent's config repo with a default `agent.nix` template, then asks the
|
agent's config repo with a default `agent.nix` template, then asks the
|
||||||
target hive to deploy it.
|
target hive to deploy it.
|
||||||
An agent created from an [archetype](../swarm/archetypes.md) gets no
|
|
||||||
config repo: its hive builds it from the archetype's module.
|
|
||||||
|
|
||||||
Changing what the template seeded isn't a special case: like every
|
Changing what the template seeded isn't a special case: like every
|
||||||
later change, it's a PR on that config repo, made from a clone and
|
later change, it's a PR on that config repo, made from a clone and
|
||||||
|
|
|
||||||
|
|
@ -420,12 +420,6 @@ reads it at every container start for `manage_root_agent`. Each agent gets
|
||||||
its own entry from it as `swarm-permissions.json` in its runtime dir, which
|
its own entry from it as `swarm-permissions.json` in its runtime dir, which
|
||||||
it holds until its own first read of its key succeeds.
|
it holds until its own first read of its key succeeds.
|
||||||
|
|
||||||
`/var/lib/hyperhive/swarm-agent-archetypes.json` holds the archetype each of
|
|
||||||
this hive's archetype agents runs, as last read from the swarm's
|
|
||||||
`agent-archetype` bucket at the agent's provision and each rebuild. An agent
|
|
||||||
with an entry has no `applied/<n>` repo and no `agent-<n>` meta input; see
|
|
||||||
[archetypes](../swarm/archetypes.md).
|
|
||||||
|
|
||||||
hive-c0re renders containers onto `meta#<n>` at creation.
|
hive-c0re renders containers onto `meta#<n>` at creation.
|
||||||
|
|
||||||
## Destroy vs purge
|
## Destroy vs purge
|
||||||
|
|
|
||||||
|
|
@ -1,24 +1,21 @@
|
||||||
# Agent archetypes
|
# Agent archetypes
|
||||||
|
|
||||||
An archetype is a named agent setup: settings to create the agent with, and
|
An archetype is a named set of settings to create an agent from. The swarm
|
||||||
a NixOS module its container runs. The swarm UI's create form lists the
|
UI's create form lists the archetypes the swarm knows. Picking one fills the
|
||||||
archetypes the swarm knows. Picking one fills the form's permissions,
|
form's permissions, resource caps and inference fields from it, and you can
|
||||||
resource caps and inference fields from it, and you can still change each
|
still change each field before you create the agent.
|
||||||
field before you create the agent.
|
|
||||||
|
|
||||||
An agent created from an archetype has no `agent-configs/<name>` repo of its
|
The agent you create this way is an ordinary agent: it gets its own
|
||||||
own. Its hive builds it from the archetype's module, so every agent of that
|
`agent-configs/<name>` repo like any other, and the request carries only the
|
||||||
archetype runs the same configuration. To change what it runs, change the
|
fields as the form shows them. Nothing records which archetype you picked.
|
||||||
archetype. Picking "none" creates a custom agent, with its own config repo.
|
|
||||||
|
|
||||||
## The built-in archetypes
|
## The built-in archetypes
|
||||||
|
|
||||||
The hyperhive flake ships these:
|
The hyperhive flake ships these:
|
||||||
|
|
||||||
| archetype | what it sets |
|
| archetype | what it sets |
|
||||||
| ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
|
| ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||||
| `reviewer` | permissions `messaging`, `inbox`, `execution`: the default set without `meta`. With `execution` it can run `hive-forge`, `git` and other commands. |
|
| `reviewer` | permissions `messaging`, `inbox`, `execution`: the default set without `meta`. With `execution` it can run `hive-forge`, `git` and other commands. |
|
||||||
| `android-dev` | installs `android-tools` (`adb`, `fastboot`), Gradle and a JDK; default permissions. |
|
|
||||||
|
|
||||||
## Where an archetype comes from
|
## Where an archetype comes from
|
||||||
|
|
||||||
|
|
@ -41,8 +38,8 @@ hyperhiveArchetypes.reviewer = {
|
||||||
- Every archetype needs a `description`. `permissions`, `limits` and
|
- Every archetype needs a `description`. `permissions`, `limits` and
|
||||||
`inference` are optional and take the shapes `POST /api/agents` takes. The swarm ignores
|
`inference` are optional and take the shapes `POST /api/agents` takes. The swarm ignores
|
||||||
other keys in `meta`.
|
other keys in `meta`.
|
||||||
- `module` is a NixOS module for the agent's container. A hive imports it
|
- `module` is a NixOS module for the agent's container. The swarm doesn't
|
||||||
into each agent created from the archetype.
|
apply it yet.
|
||||||
|
|
||||||
Today the swarm reads one flake: the hyperhive source the
|
Today the swarm reads one flake: the hyperhive source the
|
||||||
[builder](builder.md)'s host runs, so only the built-ins appear.
|
[builder](builder.md)'s host runs, so only the built-ins appear.
|
||||||
|
|
@ -88,46 +85,3 @@ The stored value:
|
||||||
"refused": { "broken": "limits: …" }
|
"refused": { "broken": "limits: …" }
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
## Creating an agent from an archetype
|
|
||||||
|
|
||||||
The create form sends the picked archetype as `archetype` on
|
|
||||||
`POST /api/agents`, `{ "source": "hyperhive", "name": "reviewer" }`. The
|
|
||||||
controller refuses with 400 an archetype the stored list doesn't hold, then:
|
|
||||||
|
|
||||||
1. stores the choice in the `agent-archetype` KV bucket, keyed by agent name,
|
|
||||||
next to the agent's permissions, caps and inference;
|
|
||||||
2. queues the creation graph without `CreateRepo`, `AddRepoMember` and
|
|
||||||
`InitAgentConfigRepo`. Identity, forge user, docs read, the mints and the
|
|
||||||
wanted state stay, and the deploy waits on those alone.
|
|
||||||
|
|
||||||
Destroying the agent deletes its key.
|
|
||||||
Every hive may read any key of the bucket
|
|
||||||
(`$JS.API.DIRECT.GET.KV_agent-archetype.>`) and write none; only the
|
|
||||||
controller writes it.
|
|
||||||
|
|
||||||
Deploy `swarm-nats-auth` before the hives: every provision reads the
|
|
||||||
agent's key, and a hive whose queue connection predates the grant gets a
|
|
||||||
refused read, so its `Provision` nodes fail until it reconnects.
|
|
||||||
|
|
||||||
The hive reads the agent's key when it provisions the agent and on every
|
|
||||||
rebuild, and keeps the last value read in
|
|
||||||
`/var/lib/hyperhive/swarm-agent-archetypes.json`, so the meta flake renders
|
|
||||||
without the queue. When the read fails, a rebuild keeps the value
|
|
||||||
stored there. For an archetype agent the hive:
|
|
||||||
|
|
||||||
- seeds no `proposed` or `applied` repo, and gives the meta flake no
|
|
||||||
`agent-<name>` input. `mkAgent` imports
|
|
||||||
`hyperhive.hyperhiveArchetypes.<name>.module` in place of the agent's own
|
|
||||||
flake, and sets `HIVE_ARCHETYPE` in the harness environment;
|
|
||||||
- relocks no agent input on a rebuild or a meta-input bump, and ignores the
|
|
||||||
`rev` of a swarm deploy request;
|
|
||||||
- mounts an empty `/agents/<name>/config/`.
|
|
||||||
|
|
||||||
The agent's system prompt says it has no config repo and names its
|
|
||||||
archetype.
|
|
||||||
|
|
||||||
The hive takes the module from its own `hyperhive` input. When that input
|
|
||||||
predates the archetype, the agent's build fails with
|
|
||||||
`this hive's hyperhive input has no archetype <name>`. Update the hive's
|
|
||||||
`hyperhive` input to fix it.
|
|
||||||
|
|
|
||||||
|
|
@ -255,12 +255,7 @@ needs to `chown` a bind mount), once at startup:
|
||||||
`hive-agent/prompts/system.md` by `hive-agent`'s `prompt::render`:
|
`hive-agent/prompts/system.md` by `hive-agent`'s `prompt::render`:
|
||||||
HTML-comment markers (`<!-- role:agent -->...<!-- /role:agent -->`,
|
HTML-comment markers (`<!-- role:agent -->...<!-- /role:agent -->`,
|
||||||
same for `role:manager`) gate the role-specific blocks; both flavors
|
same for `role:manager`) gate the role-specific blocks; both flavors
|
||||||
share everything else. `<!-- config:repo -->` and
|
share everything else. Five placeholders are then
|
||||||
`<!-- config:archetype -->` blocks gate the config-repo paragraphs: an
|
|
||||||
agent with `HIVE_ARCHETYPE` set (the meta flake sets it on an agent built
|
|
||||||
from an [archetype](../swarm/archetypes.md)) gets the archetype block and
|
|
||||||
every other agent the repo block, with `{archetype}` filled in with the
|
|
||||||
archetype's name. Five placeholders are then
|
|
||||||
substituted: `{label}` (short agent name), `{qualified_label}`
|
substituted: `{label}` (short agent name), `{qualified_label}`
|
||||||
(hive-qualified `name@domain` form),
|
(hive-qualified `name@domain` form),
|
||||||
`{hive_identity}` (for example `` on hive `pr1ma` ``; empty when
|
`{hive_identity}` (for example `` on hive `pr1ma` ``; empty when
|
||||||
|
|
@ -280,17 +275,14 @@ needs to `chown` a bind mount), once at startup:
|
||||||
telling the agent where the docs live.
|
telling the agent where the docs live.
|
||||||
Passed via `--system-prompt-file`.
|
Passed via `--system-prompt-file`.
|
||||||
|
|
||||||
**Marker grammar.** `<!-- KIND:X -->` opens a block; any
|
**Marker grammar.** `<!-- role:X -->` opens a block; any
|
||||||
`<!-- /KIND:X -->` closes the current block of that kind. The renderer
|
`<!-- /role:X -->` closes the current block. The renderer always uses
|
||||||
filters the two kinds one after the other. The renderer always uses
|
role `agent`, so it elides blocks with other role tags. Nesting **isn't**
|
||||||
role `agent`, so it elides blocks with other role tags, and config `repo`
|
|
||||||
or `archetype` as above. Nesting **isn't**
|
|
||||||
supported — a stray opener with no closer runs until end of file.
|
supported — a stray opener with no closer runs until end of file.
|
||||||
The renderer tolerates whitespace inside markers (`<!--role:foo-->` parses the
|
The renderer tolerates whitespace inside markers (`<!--role:foo-->` parses the
|
||||||
same as `<!-- role:foo -->`). Content outside any marker is always
|
same as `<!-- role:foo -->`). Content outside any marker is always
|
||||||
included. Today's `system.md` carries no role markers (single agent
|
included. Today's `system.md` carries no markers (single agent role) —
|
||||||
role), and the role grammar stays wired for a future manager / multi-role
|
the grammar stays wired for a future manager / multi-role prompt.
|
||||||
prompt; it carries one pair of config blocks.
|
|
||||||
|
|
||||||
**`hive_identity` / `swarm_identity` shape.** Each carries a
|
**`hive_identity` / `swarm_identity` shape.** Each carries a
|
||||||
leading space + backticked name (` on hive \`pr1ma\``,
|
leading space + backticked name (` on hive \`pr1ma\``,
|
||||||
|
|
|
||||||
|
|
@ -61,13 +61,3 @@ export function prefill(
|
||||||
},
|
},
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
|
||||||
* The `archetype` field of `POST /api/agents` for `archetype`, or nothing for
|
|
||||||
* a custom agent.
|
|
||||||
*/
|
|
||||||
export function requestArchetype(
|
|
||||||
archetype: Archetype | undefined,
|
|
||||||
): { source: string; name: string } | undefined {
|
|
||||||
return archetype && { source: archetype.source, name: archetype.name };
|
|
||||||
}
|
|
||||||
|
|
|
||||||
|
|
@ -57,7 +57,6 @@ import {
|
||||||
import {
|
import {
|
||||||
archetypeKey,
|
archetypeKey,
|
||||||
prefill,
|
prefill,
|
||||||
requestArchetype,
|
|
||||||
type ArchetypeList,
|
type ArchetypeList,
|
||||||
} from "./CreateAgentArchetype.js";
|
} from "./CreateAgentArchetype.js";
|
||||||
import "./CreateAgentForm.css";
|
import "./CreateAgentForm.css";
|
||||||
|
|
@ -231,7 +230,6 @@ export function CreateAgentForm({
|
||||||
permissions: catalogue
|
permissions: catalogue
|
||||||
? requestPermissions(selected, catalogue.default)
|
? requestPermissions(selected, catalogue.default)
|
||||||
: undefined,
|
: undefined,
|
||||||
archetype: requestArchetype(picked),
|
|
||||||
}),
|
}),
|
||||||
});
|
});
|
||||||
if (!r.ok) {
|
if (!r.ok) {
|
||||||
|
|
@ -323,9 +321,7 @@ export function CreateAgentForm({
|
||||||
/>
|
/>
|
||||||
{picked && (
|
{picked && (
|
||||||
<p class="create-agent-archetype-description">
|
<p class="create-agent-archetype-description">
|
||||||
{picked.description} An agent created from an archetype has
|
{picked.description}
|
||||||
no config repo: its hive builds it from the archetype's
|
|
||||||
module.
|
|
||||||
</p>
|
</p>
|
||||||
)}
|
)}
|
||||||
</>
|
</>
|
||||||
|
|
@ -400,9 +396,7 @@ export function CreateAgentForm({
|
||||||
Submitting this queues everything a new agent needs: a swarm-level
|
Submitting this queues everything a new agent needs: a swarm-level
|
||||||
identity, a config repo on the forge with the operator added as a
|
identity, a config repo on the forge with the operator added as a
|
||||||
collaborator, and a container running the agent on the hive you pick
|
collaborator, and a container running the agent on the hive you pick
|
||||||
above. An agent created from an archetype gets no config repo: its
|
above. Watch it all settle on <Link href="/jobs">jobs</Link>.
|
||||||
container runs the archetype's module. Watch it all settle on{" "}
|
|
||||||
<Link href="/jobs">jobs</Link>.
|
|
||||||
</p>
|
</p>
|
||||||
</Panel>
|
</Panel>
|
||||||
</div>
|
</div>
|
||||||
|
|
|
||||||
|
|
@ -2,11 +2,7 @@
|
||||||
// Run by `npm test` with Node's type stripping.
|
// Run by `npm test` with Node's type stripping.
|
||||||
import assert from "node:assert/strict";
|
import assert from "node:assert/strict";
|
||||||
import { test } from "node:test";
|
import { test } from "node:test";
|
||||||
import {
|
import { archetypeKey, prefill } from "../src/pages/CreateAgentArchetype.ts";
|
||||||
archetypeKey,
|
|
||||||
prefill,
|
|
||||||
requestArchetype,
|
|
||||||
} from "../src/pages/CreateAgentArchetype.ts";
|
|
||||||
|
|
||||||
const defaults = ["messaging", "meta", "inbox", "execution"];
|
const defaults = ["messaging", "meta", "inbox", "execution"];
|
||||||
|
|
||||||
|
|
@ -69,17 +65,3 @@ test("the key tells sources apart", () => {
|
||||||
archetypeKey({ source: "b", name: "x", description: "" }),
|
archetypeKey({ source: "b", name: "x", description: "" }),
|
||||||
);
|
);
|
||||||
});
|
});
|
||||||
|
|
||||||
test("a picked archetype is sent by source and name, none is left out", () => {
|
|
||||||
const reviewer = {
|
|
||||||
source: "hyperhive",
|
|
||||||
name: "reviewer",
|
|
||||||
description: "reviews",
|
|
||||||
permissions: ["messaging"],
|
|
||||||
};
|
|
||||||
assert.deepEqual(requestArchetype(reviewer), {
|
|
||||||
source: "hyperhive",
|
|
||||||
name: "reviewer",
|
|
||||||
});
|
|
||||||
assert.equal(requestArchetype(undefined), undefined);
|
|
||||||
});
|
|
||||||
|
|
|
||||||
|
|
@ -3,24 +3,13 @@ You are hyperhive agent `{label}` (qualified: `{qualified_label}`){hive_identity
|
||||||
Tools (hyperhive surface). Full signature + behavior for each comes from the tool's own MCP description (you already received it via the MCP tool schema) — this is just the map of what exists and which ones are gated, so you know where to look:
|
Tools (hyperhive surface). Full signature + behavior for each comes from the tool's own MCP description (you already received it via the MCP tool schema) — this is just the map of what exists and which ones are gated, so you know where to look:
|
||||||
|
|
||||||
- **Inbox / messaging** (always available): `mcp__hyperhive__recv`, `mcp__hyperhive__ack_until`, `mcp__hyperhive__send`, `mcp__hyperhive__get_loose_ends`, `mcp__hyperhive__cancel_loose_end`, `mcp__hyperhive__mark_todos_done`, `mcp__hyperhive__remind`, `mcp__hyperhive__set_status`, `mcp__hyperhive__get_agent_meta`. `send` reaches peers on this hive only; **reach the operator over matrix** if you have the matrix MCP tools (`mcp__matrix__send_dm` or `mcp__matrix__send_message`); without them the operator reads your `set_status` line and turn output on the dashboard, and your logs. `send(to: "operator")` is refused. One habit worth internalizing beyond the tool descriptions themselves: prefer ending the turn over repeatedly polling `recv` when idle (only turn-boundaries observe in-container todo wakes — bash-task completions, matrix unread, forge activity — and ending the turn is also your checkpoint). For a large todo backlog (`get_loose_ends` caps at 40 rows), clear reviewed ids in bulk with `mark_todos_done` rather than cancelling one at a time — there's no blind range-clear, only ids you've actually looked at.
|
- **Inbox / messaging** (always available): `mcp__hyperhive__recv`, `mcp__hyperhive__ack_until`, `mcp__hyperhive__send`, `mcp__hyperhive__get_loose_ends`, `mcp__hyperhive__cancel_loose_end`, `mcp__hyperhive__mark_todos_done`, `mcp__hyperhive__remind`, `mcp__hyperhive__set_status`, `mcp__hyperhive__get_agent_meta`. `send` reaches peers on this hive only; **reach the operator over matrix** if you have the matrix MCP tools (`mcp__matrix__send_dm` or `mcp__matrix__send_message`); without them the operator reads your `set_status` line and turn output on the dashboard, and your logs. `send(to: "operator")` is refused. One habit worth internalizing beyond the tool descriptions themselves: prefer ending the turn over repeatedly polling `recv` when idle (only turn-boundaries observe in-container todo wakes — bash-task completions, matrix unread, forge activity — and ending the turn is also your checkpoint). For a large todo backlog (`get_loose_ends` caps at 40 rows), clear reviewed ids in bulk with `mark_todos_done` rather than cancelling one at a time — there's no blind range-clear, only ids you've actually looked at.
|
||||||
- **Extra MCP tools** (some agents only): `mcp__<server>__<tool>` — agent-specific (matrix client, scraper, db connector, etc.) declared in your NixOS config under `services.hyperhive.agent.extraMcpServers`. First-class tools, already operator-approved at deploy time.
|
- **Extra MCP tools** (some agents only): `mcp__<server>__<tool>` — agent-specific (matrix client, scraper, db connector, etc.) declared in your `agent.nix` under `services.hyperhive.agent.extraMcpServers`. First-class tools, already operator-approved at deploy time.
|
||||||
|
|
||||||
<!-- config:repo -->
|
|
||||||
|
|
||||||
Need new packages, env vars, or other NixOS config for yourself? You can't edit your own config directly — open a PR on your `agent-configs/<name>` repo, or contact the operator directly. Config repos live at `/agents/{label}/config/` (read-only inside your container). All changes flow through operator-approved commits.
|
Need new packages, env vars, or other NixOS config for yourself? You can't edit your own config directly — open a PR on your `agent-configs/<name>` repo, or contact the operator directly. Config repos live at `/agents/{label}/config/` (read-only inside your container). All changes flow through operator-approved commits.
|
||||||
|
|
||||||
Your config repo is mounted **read-only** at `/agents/{label}/config/` — `agent.nix` plus whatever extra files define you (declared packages, env vars, MCP servers). Read it to see exactly what defines you before asking for a change, so you can point at the precise file and line.
|
Your config repo is mounted **read-only** at `/agents/{label}/config/` — `agent.nix` plus whatever extra files define you (declared packages, env vars, MCP servers). Read it to see exactly what defines you before asking for a change, so you can point at the precise file and line.
|
||||||
|
|
||||||
_Changing_ any agent's config is not a tool call at all — it's a forge PR on the agent's `agent-configs/<name>` repo, which an operator reviews and merges on the forge; the merge deploys it.
|
Operator boundary: starting, stopping and rebuilding containers is the operator's — ask them when any agent needs one. _Creating_ a new agent is not something you can do from here — ask the operator, who scaffolds the new agent's config repo and spawns it from the dashboard. _Changing_ any agent's config is not a tool call at all — it's a forge PR on the agent's `agent-configs/<name>` repo, which an operator reviews and merges on the forge; the merge deploys it. The operator only signs off on changes; you run the day-to-day.
|
||||||
|
|
||||||
<!-- /config:repo -->
|
|
||||||
<!-- config:archetype -->
|
|
||||||
|
|
||||||
You are built from the `{archetype}` archetype and have no config repo of your own: there is no `agent-configs/<name>` repo for you, and `/agents/{label}/config/` is empty. Need new packages, env vars, or other NixOS config for yourself? Ask the operator: what defines you is the archetype, shared with every agent built from it.
|
|
||||||
|
|
||||||
<!-- /config:archetype -->
|
|
||||||
|
|
||||||
Operator boundary: starting, stopping and rebuilding containers is the operator's — ask them when any agent needs one. _Creating_ a new agent is not something you can do from here — ask the operator, who creates agents at swarm level. The operator only signs off on changes; you run the day-to-day.
|
|
||||||
|
|
||||||
Messages from sender `system` are hyperhive helper events (JSON body, `event` field discriminates): `container_crash`, `needs_update`. Use these to react to lifecycle changes:
|
Messages from sender `system` are hyperhive helper events (JSON body, `event` field discriminates): `container_crash`, `needs_update`. Use these to react to lifecycle changes:
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,6 @@
|
||||||
//! System-prompt renderer. Single `prompts/system.md` with
|
//! System-prompt renderer. Single `prompts/system.md` with
|
||||||
//! HTML-comment markers gating role-specific and config-repo-specific
|
//! HTML-comment markers gating role-specific blocks; this module
|
||||||
//! blocks; this module assembles the final prompt (always "agent" role — there is only one
|
//! assembles the final prompt (always "agent" role — there is only one
|
||||||
//! role). Marker grammar + placeholder substitution rules in
|
//! role). Marker grammar + placeholder substitution rules in
|
||||||
//! `docs/turn-loop/claude-invocation.md::On-boot files` (`claude-system-prompt.md`).
|
//! `docs/turn-loop/claude-invocation.md::On-boot files` (`claude-system-prompt.md`).
|
||||||
|
|
||||||
|
|
@ -32,7 +32,7 @@ pub fn render(
|
||||||
docs_dir: Option<&str>,
|
docs_dir: Option<&str>,
|
||||||
operators: &[OperatorProfile],
|
operators: &[OperatorProfile],
|
||||||
) -> String {
|
) -> String {
|
||||||
let body = filter_blocks(template, "role", "agent");
|
let body = filter_role_blocks(template, "agent");
|
||||||
let qualified = crate::identity::qualify(label);
|
let qualified = crate::identity::qualify(label);
|
||||||
let hive_identity = hive_name
|
let hive_identity = hive_name
|
||||||
.filter(|n| !n.is_empty())
|
.filter(|n| !n.is_empty())
|
||||||
|
|
@ -118,27 +118,27 @@ async fn within_timeout<E: std::fmt::Display>(
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Walk `template` line-by-line, stripping `<!-- KIND:X -->` /
|
/// Walk `template` line-by-line, stripping `<!-- role:X -->` /
|
||||||
/// `<!-- /KIND:X -->` marker lines and suppressing the lines inside a block
|
/// `<!-- /role:X -->` marker lines and suppressing the lines inside a block
|
||||||
/// unless `X == target`. A close marker ends the current block. Markers of
|
/// unless `X == target`. A close marker ends the current block. Production
|
||||||
/// any other kind are left in place, for the pass that filters that kind.
|
/// `system.md` carries no markers today (single agent role — see the module
|
||||||
/// Two kinds exist: `role` (always `agent`; the grammar stays wired for a
|
/// doc), so this is effectively a passthrough; the marker grammar stays wired
|
||||||
/// future manager / multi-role prompt) and `config` ([`config_blocks`]).
|
/// for a future manager / multi-role prompt.
|
||||||
fn filter_blocks(template: &str, kind: &str, target: &str) -> String {
|
fn filter_role_blocks(template: &str, target: &str) -> String {
|
||||||
let mut out = String::with_capacity(template.len());
|
let mut out = String::with_capacity(template.len());
|
||||||
// None = outside any block; Some(value) = inside a block of this kind.
|
// None = outside any block; Some(role) = inside role-tagged block.
|
||||||
let mut active: Option<&str> = None;
|
let mut active_role: Option<&str> = None;
|
||||||
for line in template.lines() {
|
for line in template.lines() {
|
||||||
let trimmed = line.trim();
|
let trimmed = line.trim();
|
||||||
if let Some(value) = parse_open_marker(trimmed, kind) {
|
if let Some(role) = parse_open_marker(trimmed) {
|
||||||
active = Some(value);
|
active_role = Some(role);
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
if parse_close_marker(trimmed, kind).is_some() {
|
if parse_close_marker(trimmed).is_some() {
|
||||||
active = None;
|
active_role = None;
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
if active.is_none_or(|value| value == target) {
|
if active_role.is_none_or(|role| role == target) {
|
||||||
out.push_str(line);
|
out.push_str(line);
|
||||||
out.push('\n');
|
out.push('\n');
|
||||||
}
|
}
|
||||||
|
|
@ -146,37 +146,23 @@ fn filter_blocks(template: &str, kind: &str, target: &str) -> String {
|
||||||
out
|
out
|
||||||
}
|
}
|
||||||
|
|
||||||
/// `template` with its `config:repo` blocks for a custom agent, or its
|
/// `<!-- role:agent -->` → `Some("agent")`. Anything else returns
|
||||||
/// `config:archetype` blocks and `{archetype}` filled in for an agent built
|
/// None. Whitespace inside the marker is tolerated so a future
|
||||||
/// from `archetype`. An empty name is a custom agent.
|
|
||||||
fn config_blocks(template: &str, archetype: Option<&str>) -> String {
|
|
||||||
match archetype.filter(|a| !a.is_empty()) {
|
|
||||||
Some(name) => filter_blocks(template, "config", "archetype").replace("{archetype}", name),
|
|
||||||
None => filter_blocks(template, "config", "repo"),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// `<!-- role:agent -->` with `kind = "role"` → `Some("agent")`. Anything
|
|
||||||
/// else returns None. Whitespace inside the marker is tolerated so a future
|
|
||||||
/// author's `<!--role:foo-->` (no spaces) still parses; the dashboard
|
/// author's `<!--role:foo-->` (no spaces) still parses; the dashboard
|
||||||
/// markdown renderer is equally lenient. Close tags (`/role:...`) can't
|
/// markdown renderer is equally lenient. Close tags (`/role:...`)
|
||||||
/// accidentally match — the `strip_prefix(kind)` rejects the leading slash
|
/// can't accidentally match — the `strip_prefix("role:")` rejects
|
||||||
/// before we'd ever see it.
|
/// the leading slash before we'd ever see it.
|
||||||
fn parse_open_marker<'a>(line: &'a str, kind: &str) -> Option<&'a str> {
|
fn parse_open_marker(line: &str) -> Option<&str> {
|
||||||
let inside = line.strip_prefix("<!--")?.strip_suffix("-->")?.trim();
|
let inside = line.strip_prefix("<!--")?.strip_suffix("-->")?.trim();
|
||||||
let value = inside.strip_prefix(kind)?.strip_prefix(':')?.trim();
|
let role = inside.strip_prefix("role:")?.trim();
|
||||||
Some(value)
|
Some(role)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// `<!-- /role:agent -->` with `kind = "role"` → `Some("agent")`. Mirror of
|
/// `<!-- /role:agent -->` → `Some("agent")`. Mirror of
|
||||||
/// [`parse_open_marker`] for the closing tag.
|
/// [`parse_open_marker`] for the closing tag.
|
||||||
fn parse_close_marker<'a>(line: &'a str, kind: &str) -> Option<&'a str> {
|
fn parse_close_marker(line: &str) -> Option<&str> {
|
||||||
let inside = line.strip_prefix("<!--")?.strip_suffix("-->")?.trim();
|
let inside = line.strip_prefix("<!--")?.strip_suffix("-->")?.trim();
|
||||||
inside
|
inside.strip_prefix("/role:").map(str::trim)
|
||||||
.strip_prefix('/')?
|
|
||||||
.strip_prefix(kind)?
|
|
||||||
.strip_prefix(':')
|
|
||||||
.map(str::trim)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Write the assembled prompt to a stable path next to the harness
|
/// Write the assembled prompt to a stable path next to the harness
|
||||||
|
|
@ -210,9 +196,6 @@ pub async fn write_system_prompt(_socket: &Path, label: &str) -> Result<PathBuf>
|
||||||
let docs_dir = std::env::var("HIVE_DOCS_DIR")
|
let docs_dir = std::env::var("HIVE_DOCS_DIR")
|
||||||
.ok()
|
.ok()
|
||||||
.filter(|d| !d.is_empty());
|
.filter(|d| !d.is_empty());
|
||||||
// The meta flake sets HIVE_ARCHETYPE on an agent built from an archetype,
|
|
||||||
// which has no config repo of its own.
|
|
||||||
let template = config_blocks(&template, std::env::var("HIVE_ARCHETYPE").ok().as_deref());
|
|
||||||
let body = render(
|
let body = render(
|
||||||
&template,
|
&template,
|
||||||
label,
|
label,
|
||||||
|
|
@ -274,7 +257,7 @@ shared closer
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn filter_keeps_shared_and_target_role() {
|
fn filter_keeps_shared_and_target_role() {
|
||||||
let agent = filter_blocks(SAMPLE, "role", "agent");
|
let agent = filter_role_blocks(SAMPLE, "agent");
|
||||||
assert!(agent.contains("shared opener"));
|
assert!(agent.contains("shared opener"));
|
||||||
assert!(agent.contains("agent-only line"));
|
assert!(agent.contains("agent-only line"));
|
||||||
assert!(!agent.contains("manager-only line"));
|
assert!(!agent.contains("manager-only line"));
|
||||||
|
|
@ -285,7 +268,7 @@ shared closer
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn filter_for_manager_picks_manager_block() {
|
fn filter_for_manager_picks_manager_block() {
|
||||||
let manager = filter_blocks(SAMPLE, "role", "manager");
|
let manager = filter_role_blocks(SAMPLE, "manager");
|
||||||
assert!(manager.contains("shared opener"));
|
assert!(manager.contains("shared opener"));
|
||||||
assert!(!manager.contains("agent-only line"));
|
assert!(!manager.contains("agent-only line"));
|
||||||
assert!(manager.contains("manager-only line"));
|
assert!(manager.contains("manager-only line"));
|
||||||
|
|
@ -295,38 +278,23 @@ shared closer
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn parse_open_marker_handles_whitespace_variants() {
|
fn parse_open_marker_handles_whitespace_variants() {
|
||||||
assert_eq!(
|
assert_eq!(parse_open_marker("<!-- role:agent -->"), Some("agent"));
|
||||||
parse_open_marker("<!-- role:agent -->", "role"),
|
assert_eq!(parse_open_marker("<!--role:agent-->"), Some("agent"));
|
||||||
Some("agent")
|
assert_eq!(parse_open_marker("<!-- role:manager -->"), Some("manager"));
|
||||||
);
|
|
||||||
assert_eq!(
|
|
||||||
parse_open_marker("<!--role:agent-->", "role"),
|
|
||||||
Some("agent")
|
|
||||||
);
|
|
||||||
assert_eq!(
|
|
||||||
parse_open_marker("<!-- role:manager -->", "role"),
|
|
||||||
Some("manager")
|
|
||||||
);
|
|
||||||
// Close tags must NOT match open-tag parser.
|
// Close tags must NOT match open-tag parser.
|
||||||
assert_eq!(parse_open_marker("<!-- /role:agent -->", "role"), None);
|
assert_eq!(parse_open_marker("<!-- /role:agent -->"), None);
|
||||||
// Non-markers pass through (return None).
|
// Non-markers pass through (return None).
|
||||||
assert_eq!(parse_open_marker("just text", "role"), None);
|
assert_eq!(parse_open_marker("just text"), None);
|
||||||
assert_eq!(parse_open_marker("<!-- not a role -->", "role"), None);
|
assert_eq!(parse_open_marker("<!-- not a role -->"), None);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn parse_close_marker_handles_whitespace_variants() {
|
fn parse_close_marker_handles_whitespace_variants() {
|
||||||
assert_eq!(
|
assert_eq!(parse_close_marker("<!-- /role:agent -->"), Some("agent"));
|
||||||
parse_close_marker("<!-- /role:agent -->", "role"),
|
assert_eq!(parse_close_marker("<!--/role:manager-->"), Some("manager"));
|
||||||
Some("agent")
|
|
||||||
);
|
|
||||||
assert_eq!(
|
|
||||||
parse_close_marker("<!--/role:manager-->", "role"),
|
|
||||||
Some("manager")
|
|
||||||
);
|
|
||||||
// Open tags must NOT match close-tag parser.
|
// Open tags must NOT match close-tag parser.
|
||||||
assert_eq!(parse_close_marker("<!-- role:agent -->", "role"), None);
|
assert_eq!(parse_close_marker("<!-- role:agent -->"), None);
|
||||||
assert_eq!(parse_close_marker("just text", "role"), None);
|
assert_eq!(parse_close_marker("just text"), None);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
|
|
@ -363,49 +331,6 @@ shared closer
|
||||||
assert!(rendered.contains("`/agents/alice/work/`"), "{rendered}");
|
assert!(rendered.contains("`/agents/alice/work/`"), "{rendered}");
|
||||||
}
|
}
|
||||||
|
|
||||||
const CONFIG_FIXTURE: &str = "\
|
|
||||||
shared
|
|
||||||
<!-- config:repo -->
|
|
||||||
open a PR on agent-configs
|
|
||||||
<!-- /config:repo -->
|
|
||||||
<!-- config:archetype -->
|
|
||||||
built from `{archetype}`, no config repo
|
|
||||||
<!-- /config:archetype -->
|
|
||||||
";
|
|
||||||
|
|
||||||
/// A custom agent gets the config-repo lines, an archetype agent the
|
|
||||||
/// archetype lines with its name, and an empty name is a custom agent.
|
|
||||||
#[test]
|
|
||||||
fn config_blocks_pick_the_block_by_archetype() {
|
|
||||||
for custom in [None, Some("")] {
|
|
||||||
let picked = config_blocks(CONFIG_FIXTURE, custom);
|
|
||||||
assert!(picked.contains("open a PR on agent-configs"), "{picked}");
|
|
||||||
assert!(!picked.contains("no config repo"), "{picked}");
|
|
||||||
}
|
|
||||||
let picked = config_blocks(CONFIG_FIXTURE, Some("android-dev"));
|
|
||||||
assert!(picked.contains("built from `android-dev`, no config repo"));
|
|
||||||
assert!(!picked.contains("agent-configs"), "{picked}");
|
|
||||||
assert!(picked.contains("shared"));
|
|
||||||
assert!(!picked.contains("<!--"), "{picked}");
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn the_production_prompt_keeps_no_config_markers() {
|
|
||||||
for archetype in [None, Some("reviewer")] {
|
|
||||||
let rendered = render(
|
|
||||||
&config_blocks(&PRODUCTION_TEMPLATE, archetype),
|
|
||||||
"alice",
|
|
||||||
None,
|
|
||||||
None,
|
|
||||||
None,
|
|
||||||
&[],
|
|
||||||
);
|
|
||||||
assert!(!rendered.contains("<!-- config:"), "{rendered}");
|
|
||||||
assert!(!rendered.contains("<!-- /config:"), "{rendered}");
|
|
||||||
assert!(!rendered.contains("{archetype}"), "{rendered}");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Inline fixture for the {hive_identity} / {swarm_identity}
|
// Inline fixture for the {hive_identity} / {swarm_identity}
|
||||||
// placeholders. Cargo's `cargo test` resolves `PRODUCTION_TEMPLATE`
|
// placeholders. Cargo's `cargo test` resolves `PRODUCTION_TEMPLATE`
|
||||||
// against `$HIVE_ASSETS_DIR/prompts/system.md`, which the flake
|
// against `$HIVE_ASSETS_DIR/prompts/system.md`, which the flake
|
||||||
|
|
|
||||||
|
|
@ -59,7 +59,6 @@ swarm-queue-client = { workspace = true, features = [
|
||||||
"notices",
|
"notices",
|
||||||
"agent-limits",
|
"agent-limits",
|
||||||
"agent-permissions",
|
"agent-permissions",
|
||||||
"agent-archetype",
|
|
||||||
] }
|
] }
|
||||||
swarm-secret-client.workspace = true
|
swarm-secret-client.workspace = true
|
||||||
tokio.workspace = true
|
tokio.workspace = true
|
||||||
|
|
|
||||||
|
|
@ -1,143 +0,0 @@
|
||||||
//! The archetype each of this hive's archetype agents is built from, as last
|
|
||||||
//! read from the swarm's `agent-archetype` bucket
|
|
||||||
//! ([`swarm_queue_client::agent_archetype`]). Stored at
|
|
||||||
//! `/var/lib/hyperhive/swarm-agent-archetypes.json`, outside the meta repo:
|
|
||||||
//! the meta render runs at boot, possibly before the queue answers, so it
|
|
||||||
//! reads this file rather than the bucket.
|
|
||||||
//!
|
|
||||||
//! Format: agent name → [`ArchetypeRef`]. An agent with an entry has no config
|
|
||||||
//! repo: the meta flake builds it from the archetype's module, and the paths
|
|
||||||
//! that seed, relock or advance `applied/<name>` skip it. An agent with no
|
|
||||||
//! entry is a custom agent.
|
|
||||||
//!
|
|
||||||
//! [`refresh`] reads one agent's key; provisioning and every rebuild's meta
|
|
||||||
//! sync call it before they render the meta flake.
|
|
||||||
|
|
||||||
use std::collections::BTreeMap;
|
|
||||||
use std::path::{Path, PathBuf};
|
|
||||||
|
|
||||||
use anyhow::{Context, Result};
|
|
||||||
use swarm_queue_client::agent_archetype::{self, ArchetypeRef, Read};
|
|
||||||
|
|
||||||
const FILE: &str = "swarm-agent-archetypes.json";
|
|
||||||
|
|
||||||
pub type Stored = BTreeMap<String, ArchetypeRef>;
|
|
||||||
|
|
||||||
#[must_use]
|
|
||||||
pub fn path() -> PathBuf {
|
|
||||||
crate::paths::state_root().join(FILE)
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Read the stored archetypes. An absent file is the empty map: every agent
|
|
||||||
/// custom. A file that exists but can't be read or parsed is an error.
|
|
||||||
pub fn read_at(path: &Path) -> std::io::Result<Stored> {
|
|
||||||
super::read_map(path)
|
|
||||||
}
|
|
||||||
|
|
||||||
/// `agent`'s archetype per the file at [`path`], `None` for a custom agent.
|
|
||||||
///
|
|
||||||
/// # Errors
|
|
||||||
///
|
|
||||||
/// As [`read_at`].
|
|
||||||
pub fn of(agent: &str) -> std::io::Result<Option<ArchetypeRef>> {
|
|
||||||
Ok(read_at(&path())?.remove(agent))
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Read `agent`'s key and store what it holds. No queue, or a bucket that
|
|
||||||
/// does not answer, leaves the stored entry as it is.
|
|
||||||
///
|
|
||||||
/// # Errors
|
|
||||||
///
|
|
||||||
/// The queue is configured but not connected, the read failed, the value
|
|
||||||
/// names a source or archetype that is not an identifier, or the file could
|
|
||||||
/// not be read or written.
|
|
||||||
pub async fn refresh(agent: &str) -> Result<()> {
|
|
||||||
let Some(client) = crate::swarm_queue::client().await else {
|
|
||||||
return Ok(());
|
|
||||||
};
|
|
||||||
// An unconnected client does not fail a JetStream request, it hangs on it.
|
|
||||||
swarm_queue_client::ensure_connected(&client)?;
|
|
||||||
let read = agent_archetype::read(&client, agent)
|
|
||||||
.await
|
|
||||||
.map_err(|e| anyhow::anyhow!(e))
|
|
||||||
.with_context(|| format!("reading the swarm's archetype for {agent}"))?;
|
|
||||||
store_read(&path(), agent, read)
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Store `read`, the value of `agent`'s key, in the file at `path`. Written
|
|
||||||
/// only when it changes the entry.
|
|
||||||
fn store_read(path: &Path, agent: &str, read: Read) -> Result<()> {
|
|
||||||
let value = match read {
|
|
||||||
Read::NoBucket => {
|
|
||||||
tracing::debug!(%agent, "archetype: the agent-archetype bucket does not answer");
|
|
||||||
return Ok(());
|
|
||||||
}
|
|
||||||
Read::Current(value) => value,
|
|
||||||
};
|
|
||||||
// Both are rendered into the meta flake as nix strings.
|
|
||||||
if let Some(ArchetypeRef { source, name }) = &value {
|
|
||||||
for part in [source, name] {
|
|
||||||
hive_types::Ident::parse(part)
|
|
||||||
.map_err(|why| anyhow::anyhow!("archetype {part:?} of {agent}: {why}"))?;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
let mut stored = read_at(path).context("reading the stored archetypes")?;
|
|
||||||
let changed = match value {
|
|
||||||
Some(value) => stored.insert(agent.to_owned(), value.clone()) != Some(value),
|
|
||||||
None => stored.remove(agent).is_some(),
|
|
||||||
};
|
|
||||||
if changed {
|
|
||||||
super::write_map(path, &stored).context("storing the archetypes")?;
|
|
||||||
tracing::info!(%agent, archetype = ?stored.get(agent), "archetype: stored the swarm's archetype");
|
|
||||||
}
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
use super::*;
|
|
||||||
|
|
||||||
fn reviewer() -> ArchetypeRef {
|
|
||||||
ArchetypeRef {
|
|
||||||
source: "hyperhive".to_owned(),
|
|
||||||
name: "reviewer".to_owned(),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn a_read_sets_or_clears_the_entry_and_no_bucket_keeps_it() {
|
|
||||||
let dir = tempfile::tempdir().expect("tempdir");
|
|
||||||
let path = dir.path().join(FILE);
|
|
||||||
assert!(read_at(&path).expect("missing is empty").is_empty());
|
|
||||||
|
|
||||||
store_read(&path, "atlas", Read::Current(Some(reviewer()))).expect("store");
|
|
||||||
store_read(&path, "iris", Read::Current(None)).expect("store");
|
|
||||||
assert_eq!(
|
|
||||||
read_at(&path).expect("read"),
|
|
||||||
Stored::from([("atlas".to_owned(), reviewer())])
|
|
||||||
);
|
|
||||||
|
|
||||||
store_read(&path, "atlas", Read::NoBucket).expect("store");
|
|
||||||
assert_eq!(
|
|
||||||
read_at(&path).expect("read").get("atlas"),
|
|
||||||
Some(&reviewer())
|
|
||||||
);
|
|
||||||
|
|
||||||
store_read(&path, "atlas", Read::Current(None)).expect("store");
|
|
||||||
assert!(read_at(&path).expect("read").is_empty());
|
|
||||||
}
|
|
||||||
|
|
||||||
/// The value is rendered into nix, so a name that is not an identifier is
|
|
||||||
/// refused and nothing is stored.
|
|
||||||
#[test]
|
|
||||||
fn a_name_that_is_not_an_identifier_is_refused() {
|
|
||||||
let dir = tempfile::tempdir().expect("tempdir");
|
|
||||||
let path = dir.path().join(FILE);
|
|
||||||
let bad = ArchetypeRef {
|
|
||||||
source: "hyperhive".to_owned(),
|
|
||||||
name: "x\"; y".to_owned(),
|
|
||||||
};
|
|
||||||
store_read(&path, "atlas", Read::Current(Some(bad))).expect_err("refused");
|
|
||||||
assert!(read_at(&path).expect("read").is_empty());
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
@ -1,14 +1,13 @@
|
||||||
//! Per-agent configuration registries: resource limits, topology (JSON files
|
//! Per-agent configuration registries: resource limits, topology (JSON files
|
||||||
//! under `/var/lib/hyperhive/meta/`), the permissions and archetypes the
|
//! under `/var/lib/hyperhive/meta/`), the permissions the swarm grants
|
||||||
//! swarm sets (outside the meta repo) and the shared wire-protocol size
|
//! (outside the meta repo) and the shared wire-protocol size limits. Each
|
||||||
//! limits. Each submodule is re-exported at the crate root, so
|
//! submodule is re-exported at the crate root, so `crate::topology::…` etc.
|
||||||
//! `crate::topology::…` etc. keep working unchanged.
|
//! keep working unchanged.
|
||||||
//!
|
//!
|
||||||
//! Note the two similarly-named modules: [`limits`] caps inline
|
//! Note the two similarly-named modules: [`limits`] caps inline
|
||||||
//! *message body* sizes on the sockets, while [`resource_limits`] holds
|
//! *message body* sizes on the sockets, while [`resource_limits`] holds
|
||||||
//! per-agent CPU/memory caps for the container drop-in.
|
//! per-agent CPU/memory caps for the container drop-in.
|
||||||
|
|
||||||
pub mod archetypes;
|
|
||||||
pub mod limits;
|
pub mod limits;
|
||||||
pub mod permissions;
|
pub mod permissions;
|
||||||
pub mod resource_limits;
|
pub mod resource_limits;
|
||||||
|
|
|
||||||
|
|
@ -396,11 +396,6 @@ async fn run_meta_sync(coord: &Arc<Coordinator>, name: &str, relock: bool) -> Re
|
||||||
let agent_dir = crate::paths::agent_runtime_dir(name);
|
let agent_dir = crate::paths::agent_runtime_dir(name);
|
||||||
let hive = coord.hive_env();
|
let hive = coord.hive_env();
|
||||||
let paths = Coordinator::agent_paths(name, agent_dir)?;
|
let paths = Coordinator::agent_paths(name, agent_dir)?;
|
||||||
// A failed read keeps the archetype last stored, so the agent is built as
|
|
||||||
// it was.
|
|
||||||
if let Err(e) = crate::archetypes::refresh(name).await {
|
|
||||||
tracing::warn!(%name, error = ?e, "rebuild: reading the swarm's archetype failed; keeping the stored one");
|
|
||||||
}
|
|
||||||
crate::lifecycle::prepare_rebuild_dirs(name, &paths).await?;
|
crate::lifecycle::prepare_rebuild_dirs(name, &paths).await?;
|
||||||
// Idempotent meta sync so a manual rebuild can also recover from a
|
// Idempotent meta sync so a manual rebuild can also recover from a
|
||||||
// divergent meta repo; then bump just this agent's input. `relock =
|
// divergent meta repo; then bump just this agent's input. `relock =
|
||||||
|
|
@ -408,8 +403,7 @@ async fn run_meta_sync(coord: &Arc<Coordinator>, name: &str, relock: bool) -> Re
|
||||||
// would revert the bump the cascade just committed.
|
// would revert the bump the cascade just committed.
|
||||||
let agents = crate::lifecycle::agents_for_meta_listing().await?;
|
let agents = crate::lifecycle::agents_for_meta_listing().await?;
|
||||||
crate::meta::sync_agents(&hive, &agents).await?;
|
crate::meta::sync_agents(&hive, &agents).await?;
|
||||||
// An archetype agent has no `agent-<name>` input to relock.
|
if relock {
|
||||||
if relock && crate::archetypes::of(name)?.is_none() {
|
|
||||||
crate::meta::lock_update_for_rebuild(name).await?;
|
crate::meta::lock_update_for_rebuild(name).await?;
|
||||||
}
|
}
|
||||||
Ok(())
|
Ok(())
|
||||||
|
|
@ -559,16 +553,8 @@ async fn run_meta_lock(
|
||||||
// against whatever `applied/<name>` already happened to be locked to —
|
// against whatever `applied/<name>` already happened to be locked to —
|
||||||
// normally current, but silently stale forever if a past deploy failed
|
// normally current, but silently stale forever if a past deploy failed
|
||||||
// and nothing since retried it.
|
// and nothing since retried it.
|
||||||
// An archetype agent has no config repo and no `agent-<name>` input, so
|
|
||||||
// it is left out here; it is still rebuilt with the cascade.
|
|
||||||
let archetypes = crate::archetypes::read_at(&crate::archetypes::path())?;
|
|
||||||
let custom: Vec<String> = cascade
|
|
||||||
.iter()
|
|
||||||
.filter(|name| !archetypes.contains_key(*name))
|
|
||||||
.cloned()
|
|
||||||
.collect();
|
|
||||||
catch_up_cascade_agents(
|
catch_up_cascade_agents(
|
||||||
&custom,
|
&cascade,
|
||||||
crate::forge::fast_forward_applied_main,
|
crate::forge::fast_forward_applied_main,
|
||||||
|agent_inputs| async move { crate::meta::lock_update(&agent_inputs).await },
|
|agent_inputs| async move { crate::meta::lock_update(&agent_inputs).await },
|
||||||
)
|
)
|
||||||
|
|
|
||||||
|
|
@ -305,10 +305,8 @@ pub const CONTAINER_MANAGER_APPLIED_MOUNT: &str = "/applied";
|
||||||
/// and a `ManageRootAgent` holder's view of another agent's) go through
|
/// and a `ManageRootAgent` holder's view of another agent's) go through
|
||||||
/// here so they cannot drift.
|
/// here so they cannot drift.
|
||||||
///
|
///
|
||||||
/// For a custom agent it is never empty under a live container:
|
/// Never empty under a live container: `provision_container` runs
|
||||||
/// `provision_container` runs `setup_applied` before `create_only` makes the
|
/// `setup_applied` before `create_only` makes the container at all.
|
||||||
/// container at all. An archetype agent (`crate::archetypes`) has no applied
|
|
||||||
/// repo, so its mount is the empty directory the caller creates.
|
|
||||||
fn config_bind_source(name: &str) -> PathBuf {
|
fn config_bind_source(name: &str) -> PathBuf {
|
||||||
crate::paths::applied_dir(name)
|
crate::paths::applied_dir(name)
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -230,13 +230,8 @@ pub async fn provision_container(name: &str, hive: &HiveEnv, paths: &AgentPaths)
|
||||||
agent_web_port(name)
|
agent_web_port(name)
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
// Read before anything is seeded: an archetype agent gets no config
|
|
||||||
// repo, and the meta render below builds it from the archetype.
|
|
||||||
crate::archetypes::refresh(name).await?;
|
|
||||||
if crate::archetypes::of(name)?.is_none() {
|
|
||||||
setup_proposed(&paths.proposed, name).await?;
|
setup_proposed(&paths.proposed, name).await?;
|
||||||
setup_applied(&paths.applied, Some(&paths.proposed), name).await?;
|
setup_applied(&paths.applied, Some(&paths.proposed), name).await?;
|
||||||
}
|
|
||||||
ensure_agent_state_subvolume(name).await?;
|
ensure_agent_state_subvolume(name).await?;
|
||||||
ensure_claude_dir(&paths.claude)?;
|
ensure_claude_dir(&paths.claude)?;
|
||||||
ensure_state_dir(&paths.notes)?;
|
ensure_state_dir(&paths.notes)?;
|
||||||
|
|
@ -278,8 +273,6 @@ pub async fn prepare_rebuild_dirs(name: &str, paths: &AgentPaths) -> Result<()>
|
||||||
agent_web_port(name)
|
agent_web_port(name)
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
// An archetype agent has no applied repo to check or recover.
|
|
||||||
if crate::archetypes::of(name)?.is_none() {
|
|
||||||
if !paths.applied.join(".git").exists()
|
if !paths.applied.join(".git").exists()
|
||||||
&& !crate::forge::fetch_config_main_into_applied(name).await
|
&& !crate::forge::fetch_config_main_into_applied(name).await
|
||||||
{
|
{
|
||||||
|
|
@ -291,7 +284,6 @@ pub async fn prepare_rebuild_dirs(name: &str, paths: &AgentPaths) -> Result<()>
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
setup_applied(&paths.applied, None, name).await?;
|
setup_applied(&paths.applied, None, name).await?;
|
||||||
}
|
|
||||||
ensure_agent_state_subvolume(name).await?;
|
ensure_agent_state_subvolume(name).await?;
|
||||||
ensure_claude_dir(&paths.claude)?;
|
ensure_claude_dir(&paths.claude)?;
|
||||||
ensure_state_dir(&paths.notes)?;
|
ensure_state_dir(&paths.notes)?;
|
||||||
|
|
@ -327,14 +319,12 @@ pub async fn swap_update(
|
||||||
/// handle the `Err` themselves with `.unwrap_or_default()`.
|
/// handle the `Err` themselves with `.unwrap_or_default()`.
|
||||||
async fn agents_for_meta(name_to_add: Option<&str>) -> Result<Vec<crate::meta::AgentSpec>> {
|
async fn agents_for_meta(name_to_add: Option<&str>) -> Result<Vec<crate::meta::AgentSpec>> {
|
||||||
let containers = list().await?;
|
let containers = list().await?;
|
||||||
let archetypes = crate::archetypes::read_at(&crate::archetypes::path())?;
|
|
||||||
let mut out: Vec<crate::meta::AgentSpec> = containers
|
let mut out: Vec<crate::meta::AgentSpec> = containers
|
||||||
.into_iter()
|
.into_iter()
|
||||||
.filter_map(|c| {
|
.filter_map(|c| {
|
||||||
let name = c.strip_prefix(AGENT_PREFIX)?.to_owned();
|
let name = c.strip_prefix(AGENT_PREFIX)?.to_owned();
|
||||||
Some(crate::meta::AgentSpec {
|
Some(crate::meta::AgentSpec {
|
||||||
port: agent_web_port(&name),
|
port: agent_web_port(&name),
|
||||||
archetype: archetypes.get(&name).cloned(),
|
|
||||||
name,
|
name,
|
||||||
})
|
})
|
||||||
})
|
})
|
||||||
|
|
@ -344,7 +334,6 @@ async fn agents_for_meta(name_to_add: Option<&str>) -> Result<Vec<crate::meta::A
|
||||||
{
|
{
|
||||||
out.push(crate::meta::AgentSpec {
|
out.push(crate::meta::AgentSpec {
|
||||||
port: agent_web_port(extra),
|
port: agent_web_port(extra),
|
||||||
archetype: archetypes.get(extra).cloned(),
|
|
||||||
name: extra.to_owned(),
|
name: extra.to_owned(),
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -43,7 +43,7 @@ mod swarm_status;
|
||||||
mod test_env;
|
mod test_env;
|
||||||
mod workers;
|
mod workers;
|
||||||
|
|
||||||
pub(crate) use agent_config::{archetypes, limits, permissions, resource_limits, topology};
|
pub(crate) use agent_config::{limits, permissions, resource_limits, topology};
|
||||||
pub(crate) use stats::{container_stats, hive_stats, host_stats, otel_metrics, warnings};
|
pub(crate) use stats::{container_stats, hive_stats, host_stats, otel_metrics, warnings};
|
||||||
pub(crate) use stores::{broker, build_logs, db, power};
|
pub(crate) use stores::{broker, build_logs, db, power};
|
||||||
pub(crate) use workers::{agent_sockets, auto_update, crash_watch, knowledge, mcp_sockets};
|
pub(crate) use workers::{agent_sockets, auto_update, crash_watch, knowledge, mcp_sockets};
|
||||||
|
|
|
||||||
|
|
@ -42,9 +42,6 @@ pub const CONTAINER_MANAGER_META_MOUNT: &str = "/meta";
|
||||||
pub struct AgentSpec {
|
pub struct AgentSpec {
|
||||||
pub name: String,
|
pub name: String,
|
||||||
pub port: u16,
|
pub port: u16,
|
||||||
/// The archetype the agent is built from (`crate::archetypes`), `None`
|
|
||||||
/// for a custom agent built from its own config repo.
|
|
||||||
pub archetype: Option<swarm_queue_client::agent_archetype::ArchetypeRef>,
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Stage topology.json if it exists: it is regenerated by `reconcile`
|
/// Stage topology.json if it exists: it is regenerated by `reconcile`
|
||||||
|
|
@ -789,11 +786,6 @@ where
|
||||||
// (config PRs land there) but are not the flake's build input. deploy +
|
// (config PRs land there) but are not the flake's build input. deploy +
|
||||||
// rebuild re-lock this input to `applied/<name>`'s current `main` head.
|
// rebuild re-lock this input to `applied/<name>`'s current `main` head.
|
||||||
for spec in agents {
|
for spec in agents {
|
||||||
// An archetype agent has no config repo, so no input: `mkAgent`
|
|
||||||
// takes its module from the archetype.
|
|
||||||
if spec.archetype.is_some() {
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
let _ = writeln!(
|
let _ = writeln!(
|
||||||
out,
|
out,
|
||||||
" agent-{name}.url = \"git+file://{applied}\";",
|
" agent-{name}.url = \"git+file://{applied}\";",
|
||||||
|
|
@ -819,26 +811,17 @@ where
|
||||||
out.push_str(" };\n outputs =\n { self, hyperhive, ... }@inputs:\n let\n");
|
out.push_str(" };\n outputs =\n { self, hyperhive, ... }@inputs:\n let\n");
|
||||||
let _ = writeln!(
|
let _ = writeln!(
|
||||||
out,
|
out,
|
||||||
" dashboardPort = {dashboard_port};\n mkAgent = {{ name, port, archetype ? null }}:"
|
" dashboardPort = {dashboard_port};\n mkAgent = {{ name, port }}:"
|
||||||
);
|
);
|
||||||
out.push_str(
|
out.push_str(
|
||||||
r#" let
|
r#" let
|
||||||
base = hyperhive.nixosConfigurations.agent-base;
|
base = hyperhive.nixosConfigurations.agent-base;
|
||||||
# Every flake an archetype can come from, by its source name.
|
input = inputs."agent-${name}";
|
||||||
archetypeSources = { inherit hyperhive; };
|
|
||||||
agentModule =
|
|
||||||
if archetype == null then
|
|
||||||
inputs."agent-${name}".nixosModules.default
|
|
||||||
else
|
|
||||||
(archetypeSources.${archetype.source}
|
|
||||||
or (throw "agent ${name}: archetype source ${archetype.source} is not an input of this hive's meta flake")
|
|
||||||
).hyperhiveArchetypes.${archetype.name}.module
|
|
||||||
or (throw "agent ${name}: this hive's ${archetype.source} input has no archetype ${archetype.name}; update that input");
|
|
||||||
service = "hive-agent";
|
service = "hive-agent";
|
||||||
in
|
in
|
||||||
base.extendModules {
|
base.extendModules {
|
||||||
modules = [
|
modules = [
|
||||||
agentModule
|
input.nixosModules.default
|
||||||
{
|
{
|
||||||
"#,
|
"#,
|
||||||
);
|
);
|
||||||
|
|
@ -1025,10 +1008,6 @@ where
|
||||||
HYPERHIVE_HARNESS_DIR = "/agents/${name}/harness";
|
HYPERHIVE_HARNESS_DIR = "/agents/${name}/harness";
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
# Tells the harness the agent has no config repo of its own.
|
|
||||||
(if archetype == null then { } else {
|
|
||||||
systemd.services.${service}.environment.HIVE_ARCHETYPE = archetype.name;
|
|
||||||
})
|
|
||||||
];
|
];
|
||||||
};
|
};
|
||||||
in
|
in
|
||||||
|
|
@ -1037,17 +1016,9 @@ where
|
||||||
"#,
|
"#,
|
||||||
);
|
);
|
||||||
for spec in agents {
|
for spec in agents {
|
||||||
// `source` and `name` are identifiers (`archetypes::store_read`), so
|
|
||||||
// they need no escaping.
|
|
||||||
let archetype = spec.archetype.as_ref().map_or(String::new(), |a| {
|
|
||||||
format!(
|
|
||||||
" archetype = {{ source = \"{}\"; name = \"{}\"; }};",
|
|
||||||
a.source, a.name
|
|
||||||
)
|
|
||||||
});
|
|
||||||
let _ = writeln!(
|
let _ = writeln!(
|
||||||
out,
|
out,
|
||||||
" {} = mkAgent {{ name = \"{}\"; port = {};{archetype} }};",
|
" {} = mkAgent {{ name = \"{}\"; port = {}; }};",
|
||||||
spec.name, spec.name, spec.port,
|
spec.name, spec.name, spec.port,
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
@ -1264,7 +1235,6 @@ mod tests {
|
||||||
AgentSpec {
|
AgentSpec {
|
||||||
name: name.to_owned(),
|
name: name.to_owned(),
|
||||||
port,
|
port,
|
||||||
archetype: None,
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -1531,53 +1501,6 @@ mod tests {
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
/// An archetype agent has no `agent-<name>` input, and `mkAgent` is
|
|
||||||
/// handed its archetype; a custom agent beside it keeps its input.
|
|
||||||
#[test]
|
|
||||||
fn render_flake_builds_an_archetype_agent_from_its_archetype() {
|
|
||||||
let android = AgentSpec {
|
|
||||||
archetype: Some(swarm_queue_client::agent_archetype::ArchetypeRef {
|
|
||||||
source: "hyperhive".to_owned(),
|
|
||||||
name: "android-dev".to_owned(),
|
|
||||||
}),
|
|
||||||
..sample_spec("droid", 9002)
|
|
||||||
};
|
|
||||||
let out = render_flake_with_lookup(
|
|
||||||
"github:example/hyperhive",
|
|
||||||
"path:/nix/store/bbbb-hyperhive-docs-source",
|
|
||||||
"path:/nix/store/aaaa-nixpkgs-source",
|
|
||||||
None,
|
|
||||||
8000,
|
|
||||||
&[sample_spec("alice", 9001), android],
|
|
||||||
|_| vec!["nixpkgs"],
|
|
||||||
);
|
|
||||||
assert!(
|
|
||||||
!out.contains("agent-droid"),
|
|
||||||
"an archetype agent has no config input:\n{out}"
|
|
||||||
);
|
|
||||||
assert!(
|
|
||||||
out.contains(
|
|
||||||
"droid = mkAgent { name = \"droid\"; port = 9002; \
|
|
||||||
archetype = { source = \"hyperhive\"; name = \"android-dev\"; }; };"
|
|
||||||
),
|
|
||||||
"{out}"
|
|
||||||
);
|
|
||||||
assert!(out.contains("agent-alice.url = \"git+file://"), "{out}");
|
|
||||||
assert!(out.contains("agent-alice.inputs.nixpkgs.follows"), "{out}");
|
|
||||||
assert!(
|
|
||||||
out.contains("alice = mkAgent { name = \"alice\"; port = 9001; };"),
|
|
||||||
"{out}"
|
|
||||||
);
|
|
||||||
assert!(
|
|
||||||
out.contains("hyperhiveArchetypes.${archetype.name}.module"),
|
|
||||||
"{out}"
|
|
||||||
);
|
|
||||||
assert!(
|
|
||||||
out.contains("environment.HIVE_ARCHETYPE = archetype.name"),
|
|
||||||
"{out}"
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn render_flake_forwards_env_into_global_environment() {
|
fn render_flake_forwards_env_into_global_environment() {
|
||||||
// Regression for the isolation breakage where forwarded vars
|
// Regression for the isolation breakage where forwarded vars
|
||||||
|
|
|
||||||
|
|
@ -82,16 +82,7 @@ pub async fn run(coord: &Arc<Coordinator>) -> Result<()> {
|
||||||
|
|
||||||
// Phase 1 + 2: per-agent applied + proposed.
|
// Phase 1 + 2: per-agent applied + proposed.
|
||||||
tracing::debug!("migration: phase 1+2 (applied + proposed repos)");
|
tracing::debug!("migration: phase 1+2 (applied + proposed repos)");
|
||||||
// An archetype agent has neither repo, and seeding one here would hand it
|
|
||||||
// a config it is not built from.
|
|
||||||
let archetypes = crate::archetypes::read_at(&crate::archetypes::path()).unwrap_or_else(|e| {
|
|
||||||
tracing::warn!(error = ?e, "migration: stored archetypes unreadable; treating every agent as custom");
|
|
||||||
crate::archetypes::Stored::new()
|
|
||||||
});
|
|
||||||
for name in &names {
|
for name in &names {
|
||||||
if archetypes.contains_key(name.as_str()) {
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
tracing::debug!(%name, "migration: applied+proposed");
|
tracing::debug!(%name, "migration: applied+proposed");
|
||||||
if let Err(e) = migrate_applied_repo(name.as_str()).await {
|
if let Err(e) = migrate_applied_repo(name.as_str()).await {
|
||||||
tracing::warn!(%name, error = ?e, "migration: applied repo rewrite failed");
|
tracing::warn!(%name, error = ?e, "migration: applied repo rewrite failed");
|
||||||
|
|
|
||||||
|
|
@ -298,16 +298,7 @@ async fn handle_deploy_request(
|
||||||
|
|
||||||
// Only a rebuild applies `rev`. A first deploy ignores it and builds the
|
// Only a rebuild applies `rev`. A first deploy ignores it and builds the
|
||||||
// `applied` repo it seeds from the forge's `main`, or the one it finds.
|
// `applied` repo it seeds from the forge's `main`, or the one it finds.
|
||||||
// An archetype agent has no `applied` repo, so its rebuild ignores it too.
|
|
||||||
let archetype = match crate::archetypes::of(&agent) {
|
|
||||||
Ok(archetype) => archetype,
|
|
||||||
Err(e) => {
|
|
||||||
tracing::warn!(%agent, error = ?e, "swarm events: cannot read the stored archetypes; deploy dropped");
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
};
|
|
||||||
if known
|
if known
|
||||||
&& archetype.is_none()
|
|
||||||
&& let Some(rev) = rev.as_deref()
|
&& let Some(rev) = rev.as_deref()
|
||||||
&& !apply_rev(&agent, rev).await
|
&& !apply_rev(&agent, rev).await
|
||||||
{
|
{
|
||||||
|
|
|
||||||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Reference in a new issue