Compare commits
22 changed files with 30 additions and 1246 deletions
|
|
@ -148,8 +148,7 @@ Swarm-level:
|
||||||
- **`swarm-matrix-ctl/`** — the one binary run inside
|
- **`swarm-matrix-ctl/`** — the one binary run inside
|
||||||
`containers.hive-matrix`; new in-container jobs are verbs here, not
|
`containers.hive-matrix`; new in-container jobs are verbs here, not
|
||||||
crates.
|
crates.
|
||||||
- **`swarm-builder/`** — builds agent config PR heads for commit status,
|
- **`swarm-builder/`** — builds agent config PR heads for commit status.
|
||||||
and evaluates the agent archetypes the create form lists.
|
|
||||||
Design: [`docs/swarm/builder.md`](docs/swarm/builder.md).
|
Design: [`docs/swarm/builder.md`](docs/swarm/builder.md).
|
||||||
- **`swarm-ci-metrics/`** — polls the forge's Actions runs and pushes each
|
- **`swarm-ci-metrics/`** — polls the forge's Actions runs and pushes each
|
||||||
finished run's duration and queue wait to the metrics store. Design:
|
finished run's duration and queue wait to the metrics store. Design:
|
||||||
|
|
@ -177,8 +176,7 @@ Swarm-level:
|
||||||
- **`nix/`** — `host-modules/` (the host stack), `agent-modules/`
|
- **`nix/`** — `host-modules/` (the host stack), `agent-modules/`
|
||||||
(per-agent harness features), `container-modules/` (guest side of
|
(per-agent harness features), `container-modules/` (guest side of
|
||||||
service containers), `templates/agent.nix` (the container entry
|
service containers), `templates/agent.nix` (the container entry
|
||||||
point), `archetypes/` (the built-in agent archetypes), `packages/`,
|
point), `packages/`, `docs/` (options doc), `module-eval/` (NixOS
|
||||||
`docs/` (options doc), `module-eval/` (NixOS
|
|
||||||
module-eval tests), `builder/` (`check-flake`), `script-tests/`, plus
|
module-eval tests), `builder/` (`check-flake`), `script-tests/`, plus
|
||||||
`sources.nix` / `rust.nix` / `checks.nix` / `devshell.nix` /
|
`sources.nix` / `rust.nix` / `checks.nix` / `devshell.nix` /
|
||||||
`treefmt.nix` behind the thin `flake.nix`.
|
`treefmt.nix` behind the thin `flake.nix`.
|
||||||
|
|
|
||||||
|
|
@ -91,8 +91,6 @@ declarations.
|
||||||
(planned, not implemented yet).
|
(planned, not implemented yet).
|
||||||
- **Does an agent config PR build before it's merged?** →
|
- **Does an agent config PR build before it's merged?** →
|
||||||
[`swarm/builder.md`](swarm/builder.md).
|
[`swarm/builder.md`](swarm/builder.md).
|
||||||
- **How do I create an agent from a ready-made set of permissions and
|
|
||||||
settings?** → [`swarm/archetypes.md`](swarm/archetypes.md).
|
|
||||||
- **How long do CI runs take, per repo and PR, and where does the board get
|
- **How long do CI runs take, per repo and PR, and where does the board get
|
||||||
it?** → [`swarm/ci-metrics.md`](swarm/ci-metrics.md).
|
it?** → [`swarm/ci-metrics.md`](swarm/ci-metrics.md).
|
||||||
- **How do I log an agent in to claude from the swarm UI, and why doesn't a
|
- **How do I log an agent in to claude from the swarm UI, and why doesn't a
|
||||||
|
|
|
||||||
|
|
@ -1,87 +0,0 @@
|
||||||
# Agent archetypes
|
|
||||||
|
|
||||||
An archetype is a named set of settings to create an agent from. The swarm
|
|
||||||
UI's create form lists the archetypes the swarm knows. Picking one fills the
|
|
||||||
form's permissions, resource caps and inference fields from it, and you can
|
|
||||||
still change each field before you create the agent.
|
|
||||||
|
|
||||||
The agent you create this way is an ordinary agent: it gets its own
|
|
||||||
`agent-configs/<name>` repo like any other, and the request carries only the
|
|
||||||
fields as the form shows them. Nothing records which archetype you picked.
|
|
||||||
|
|
||||||
## The built-in archetypes
|
|
||||||
|
|
||||||
The hyperhive flake ships these:
|
|
||||||
|
|
||||||
| archetype | what it sets |
|
|
||||||
| ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
||||||
| `reviewer` | permissions `messaging`, `inbox`, `execution`: the default set without `meta`. With `execution` it can run `hive-forge`, `git` and other commands. |
|
|
||||||
|
|
||||||
## Where an archetype comes from
|
|
||||||
|
|
||||||
A flake declares archetypes in its `hyperhiveArchetypes` output:
|
|
||||||
|
|
||||||
```nix
|
|
||||||
hyperhiveArchetypes.reviewer = {
|
|
||||||
module = { };
|
|
||||||
meta = {
|
|
||||||
description = "Reviews what it is sent.";
|
|
||||||
permissions = [ "messaging" "inbox" "execution" ];
|
|
||||||
limits = { memory_max = "4G"; };
|
|
||||||
inference = { effort = "high"; };
|
|
||||||
};
|
|
||||||
};
|
|
||||||
```
|
|
||||||
|
|
||||||
- `meta` is plain data: no derivation, no path, no function. The swarm
|
|
||||||
evaluates `meta` alone.
|
|
||||||
- Every archetype needs a `description`. `permissions`, `limits` and
|
|
||||||
`inference` are optional and take the shapes `POST /api/agents` takes. The swarm ignores
|
|
||||||
other keys in `meta`.
|
|
||||||
- `module` is a NixOS module for the agent's container. The swarm doesn't
|
|
||||||
apply it yet.
|
|
||||||
|
|
||||||
Today the swarm reads one flake: the hyperhive source the
|
|
||||||
[builder](builder.md)'s host runs, so only the built-ins appear.
|
|
||||||
|
|
||||||
## How the list stays current
|
|
||||||
|
|
||||||
1. swarm-controller asks the builder for the list on the queue subject
|
|
||||||
`$SWARM.builder.archetypes` once the queue connects, and every 10 minutes
|
|
||||||
after that.
|
|
||||||
2. The builder evaluates the `meta` of each `hyperhiveArchetypes` entry on
|
|
||||||
its own hyperhive and nixpkgs sources, and answers with the JSON or the
|
|
||||||
last lines `nix` printed:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
nix eval --json <hyperhive>#hyperhiveArchetypes \
|
|
||||||
--apply 'builtins.mapAttrs (_: a: a.meta)'
|
|
||||||
```
|
|
||||||
|
|
||||||
3. The controller checks each archetype the way `POST /api/agents` checks
|
|
||||||
its settings and stores the result in the `agent-archetypes` KV bucket,
|
|
||||||
under the key `hyperhive`, when it differs from the stored one. The list
|
|
||||||
leaves out an archetype that fails the check, and its name and reason go
|
|
||||||
into the record's `refused` map and the controller's journal.
|
|
||||||
4. `GET /api/archetypes` answers from that record, and the create form lists
|
|
||||||
from it.
|
|
||||||
|
|
||||||
A builder host redeploy can change the list. The next pass, within 10
|
|
||||||
minutes, picks it up. When no builder answers, the controller keeps the
|
|
||||||
stored list and logs a warning on each pass. A swarm with no builder enabled
|
|
||||||
shows no archetype select.
|
|
||||||
|
|
||||||
The stored value:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"source": "/nix/store/…-hyperhive-flake-source",
|
|
||||||
"archetypes": {
|
|
||||||
"reviewer": {
|
|
||||||
"description": "…",
|
|
||||||
"permissions": ["messaging", "inbox", "execution"]
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"refused": { "broken": "limits: …" }
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
@ -61,14 +61,6 @@ Where it runs and how it's enabled: [`services.md`](services.md#builder).
|
||||||
push. The jobs graph is in memory, so a controller restart drops a build in
|
push. The jobs graph is in memory, so a controller restart drops a build in
|
||||||
flight and leaves its status `pending`.
|
flight and leaves its status `pending`.
|
||||||
|
|
||||||
## Archetype evaluation
|
|
||||||
|
|
||||||
The builder also answers `$SWARM.builder.archetypes`: it evaluates the
|
|
||||||
`meta` of every agent archetype its own hyperhive source declares, with no
|
|
||||||
forge token, and answers with the JSON. An evaluation doesn't wait for a
|
|
||||||
build in progress and has two minutes. What the controller does with the
|
|
||||||
answer: [`archetypes.md`](archetypes.md).
|
|
||||||
|
|
||||||
## Credentials
|
## Credentials
|
||||||
|
|
||||||
| What | Where | Who reads it |
|
| What | Where | Who reads it |
|
||||||
|
|
|
||||||
|
|
@ -11,7 +11,7 @@ domain, covers host-level detail for one hive.
|
||||||
| ---------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
| ---------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||||
| `/` | the start page: the swarm's name over a digital-rain backdrop, plus a paperclip offering to set up an agent while no `ruth` exists (it opens the create form with `ruth` filled in); the swarm name in the header links here |
|
| `/` | the start page: the swarm's name over a digital-rain backdrop, plus a paperclip offering to set up an agent while no `ruth` exists (it opens the create form with `ruth` filled in); the swarm name in the header links here |
|
||||||
| `/hives` | the hive directory, each hive with its last reported status |
|
| `/hives` | the hive directory, each hive with its last reported status |
|
||||||
| `/agents` | every agent: status, config PR, wanted state, inference settings, resource caps; create agents (with their permissions, optionally from an [archetype](archetypes.md)), link forge, matrix and GitHub accounts |
|
| `/agents` | every agent: status, config PR, wanted state, inference settings, resource caps; create agents (with their permissions), link forge, matrix and GitHub accounts |
|
||||||
| `/agents/<name>/terminal` | one agent's live terminal |
|
| `/agents/<name>/terminal` | one agent's live terminal |
|
||||||
| `/agents/<name>/subagents/<subagent>/terminal` | one of that agent's subagents' live terminal |
|
| `/agents/<name>/subagents/<subagent>/terminal` | one of that agent's subagents' live terminal |
|
||||||
| `/alerts` | the alerts the swarm's Grafana holds now (firing, pending, no data, error), read through the controller with a Viewer service-account token |
|
| `/alerts` | the alerts the swarm's Grafana holds now (firing, pending, no data, error), read through the controller with a Viewer service-account token |
|
||||||
|
|
|
||||||
|
|
@ -206,10 +206,6 @@
|
||||||
agent-base = mkContainer self.nixosModules.agent-base;
|
agent-base = mkContainer self.nixosModules.agent-base;
|
||||||
};
|
};
|
||||||
|
|
||||||
# Agent archetypes, `<name> = { module; meta; }`. Not per system: `meta`
|
|
||||||
# is plain data, and `module` takes its `pkgs` from the agent's system.
|
|
||||||
hyperhiveArchetypes = import ./nix/archetypes;
|
|
||||||
|
|
||||||
devShells = forAllSystems ({ pkgs, rust, ... }: import ./nix/devshell.nix { inherit pkgs rust; });
|
devShells = forAllSystems ({ pkgs, rust, ... }: import ./nix/devshell.nix { inherit pkgs rust; });
|
||||||
|
|
||||||
formatter = forAllSystems ({ treefmt-eval, ... }: treefmt-eval.config.build.wrapper);
|
formatter = forAllSystems ({ treefmt-eval, ... }: treefmt-eval.config.build.wrapper);
|
||||||
|
|
|
||||||
|
|
@ -1,63 +0,0 @@
|
||||||
// The create-agent form's archetype picker: what picking one puts in the
|
|
||||||
// form. Kept out of the component so `npm test` can run it.
|
|
||||||
import type { InferenceDraft } from "../agent/detail/AgentInference.js";
|
|
||||||
|
|
||||||
// Mirrors swarm-controller's `ArchetypeView` (`GET /api/archetypes`).
|
|
||||||
export interface Archetype {
|
|
||||||
source: string;
|
|
||||||
name: string;
|
|
||||||
description: string;
|
|
||||||
permissions?: string[];
|
|
||||||
limits?: { cpu_quota?: string; memory_max?: string };
|
|
||||||
inference?: {
|
|
||||||
model?: string;
|
|
||||||
effort?: string;
|
|
||||||
subagent_models?: string[];
|
|
||||||
max_subagents?: number;
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
// Mirrors swarm-controller's `Archetypes`.
|
|
||||||
export interface ArchetypeList {
|
|
||||||
archetypes: Archetype[];
|
|
||||||
refused: { source: string; name: string; reason: string }[];
|
|
||||||
}
|
|
||||||
|
|
||||||
/** The settings fields a pick fills in. */
|
|
||||||
export interface Prefill {
|
|
||||||
permissions: string[];
|
|
||||||
cpu: string;
|
|
||||||
memory: string;
|
|
||||||
inference: InferenceDraft;
|
|
||||||
}
|
|
||||||
|
|
||||||
/** The select's value for `a`; unique across sources. */
|
|
||||||
export function archetypeKey(a: Archetype): string {
|
|
||||||
return `${a.source}/${a.name}`;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* The form's settings for `archetype`, or for no archetype when it is
|
|
||||||
* `undefined`. A setting the archetype leaves out is the form's empty value,
|
|
||||||
* and permissions it leaves out are `defaults`.
|
|
||||||
*/
|
|
||||||
export function prefill(
|
|
||||||
archetype: Archetype | undefined,
|
|
||||||
defaults: string[],
|
|
||||||
): Prefill {
|
|
||||||
const inference = archetype?.inference ?? {};
|
|
||||||
return {
|
|
||||||
permissions: archetype?.permissions ?? defaults,
|
|
||||||
cpu: archetype?.limits?.cpu_quota ?? "",
|
|
||||||
memory: archetype?.limits?.memory_max ?? "",
|
|
||||||
inference: {
|
|
||||||
model: inference.model ?? "",
|
|
||||||
effort: inference.effort ?? "",
|
|
||||||
subagents: (inference.subagent_models ?? []).join(", "),
|
|
||||||
maxSubagents:
|
|
||||||
inference.max_subagents === undefined
|
|
||||||
? ""
|
|
||||||
: String(inference.max_subagents),
|
|
||||||
},
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
@ -69,8 +69,3 @@
|
||||||
.create-agent-result-ok {
|
.create-agent-result-ok {
|
||||||
color: var(--green);
|
color: var(--green);
|
||||||
}
|
}
|
||||||
/* The picked archetype's description, under its select. */
|
|
||||||
.create-agent-archetype-description {
|
|
||||||
margin: 0;
|
|
||||||
color: var(--muted);
|
|
||||||
}
|
|
||||||
|
|
|
||||||
|
|
@ -54,11 +54,6 @@ import {
|
||||||
requestPermissions,
|
requestPermissions,
|
||||||
togglePermission,
|
togglePermission,
|
||||||
} from "./CreateAgentPermissions.js";
|
} from "./CreateAgentPermissions.js";
|
||||||
import {
|
|
||||||
archetypeKey,
|
|
||||||
prefill,
|
|
||||||
type ArchetypeList,
|
|
||||||
} from "./CreateAgentArchetype.js";
|
|
||||||
import "./CreateAgentForm.css";
|
import "./CreateAgentForm.css";
|
||||||
|
|
||||||
// Mirrors swarm-controller's `HiveEntry` — same shape `HivesPage`
|
// Mirrors swarm-controller's `HiveEntry` — same shape `HivesPage`
|
||||||
|
|
@ -136,16 +131,6 @@ export function CreateAgentForm({
|
||||||
null,
|
null,
|
||||||
);
|
);
|
||||||
const [selected, setSelected] = useState<string[]>([]);
|
const [selected, setSelected] = useState<string[]>([]);
|
||||||
// `null` until `/api/archetypes` answers; `archetype` is the picked one's
|
|
||||||
// `archetypeKey`, `""` for none.
|
|
||||||
const [archetypes, setArchetypes] = useState<ArchetypeList | null>(null);
|
|
||||||
const [archetypesError, setArchetypesError] = useState<ProblemDetails | null>(
|
|
||||||
null,
|
|
||||||
);
|
|
||||||
const [archetype, setArchetype] = useState("");
|
|
||||||
const picked = archetypes?.archetypes.find(
|
|
||||||
(a) => archetypeKey(a) === archetype,
|
|
||||||
);
|
|
||||||
|
|
||||||
// Loaded once, not re-fetched on submit: the roster changes rarely
|
// Loaded once, not re-fetched on submit: the roster changes rarely
|
||||||
// enough (a nix-level swarm config change) that staleness within one
|
// enough (a nix-level swarm config change) that staleness within one
|
||||||
|
|
@ -180,31 +165,6 @@ export function CreateAgentForm({
|
||||||
})().catch((e: unknown) => setCatalogueError({ detail: String(e) }));
|
})().catch((e: unknown) => setCatalogueError({ detail: String(e) }));
|
||||||
}, []);
|
}, []);
|
||||||
|
|
||||||
useEffect(() => {
|
|
||||||
(async () => {
|
|
||||||
const r = await fetch("/api/archetypes");
|
|
||||||
if (!r.ok) {
|
|
||||||
setArchetypesError(await readApiError(r));
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
setArchetypes((await r.json()) as ArchetypeList);
|
|
||||||
})().catch((e: unknown) => setArchetypesError({ detail: String(e) }));
|
|
||||||
}, []);
|
|
||||||
|
|
||||||
// Fills permissions, caps and inference from the picked archetype; each
|
|
||||||
// stays editable, and the request carries only those fields.
|
|
||||||
function pickArchetype(key: string) {
|
|
||||||
setArchetype(key);
|
|
||||||
const p = prefill(
|
|
||||||
archetypes?.archetypes.find((a) => archetypeKey(a) === key),
|
|
||||||
catalogue?.default ?? [],
|
|
||||||
);
|
|
||||||
setSelected(p.permissions);
|
|
||||||
setCpu(p.cpu);
|
|
||||||
setMemory(p.memory);
|
|
||||||
setInference(p.inference);
|
|
||||||
}
|
|
||||||
|
|
||||||
const hiveOptions: SelectOption[] = (hives ?? []).map((h) => ({
|
const hiveOptions: SelectOption[] = (hives ?? []).map((h) => ({
|
||||||
value: h.name,
|
value: h.name,
|
||||||
label: h.name,
|
label: h.name,
|
||||||
|
|
@ -239,7 +199,6 @@ export function CreateAgentForm({
|
||||||
const data = (await r.json()) as CreateAgentResponse;
|
const data = (await r.json()) as CreateAgentResponse;
|
||||||
setResult({ status: "done", nodeId: data.node_id });
|
setResult({ status: "done", nodeId: data.node_id });
|
||||||
setName("");
|
setName("");
|
||||||
pickArchetype("");
|
|
||||||
if (catalogue)
|
if (catalogue)
|
||||||
setSelected(
|
setSelected(
|
||||||
initialSelection(catalogue.all, catalogue.default, extraPermissions),
|
initialSelection(catalogue.all, catalogue.default, extraPermissions),
|
||||||
|
|
@ -271,12 +230,6 @@ export function CreateAgentForm({
|
||||||
problem={catalogueError}
|
problem={catalogueError}
|
||||||
/>
|
/>
|
||||||
)}
|
)}
|
||||||
{archetypesError && (
|
|
||||||
<ApiErrorPanel
|
|
||||||
context="failed to load the archetype list"
|
|
||||||
problem={archetypesError}
|
|
||||||
/>
|
|
||||||
)}
|
|
||||||
<form class="create-agent-form" onSubmit={submit}>
|
<form class="create-agent-form" onSubmit={submit}>
|
||||||
<TextField
|
<TextField
|
||||||
id="agent-name"
|
id="agent-name"
|
||||||
|
|
@ -303,29 +256,6 @@ export function CreateAgentForm({
|
||||||
: "select a hive"
|
: "select a hive"
|
||||||
}
|
}
|
||||||
/>
|
/>
|
||||||
{archetypes && archetypes.archetypes.length > 0 && (
|
|
||||||
<>
|
|
||||||
<SelectField
|
|
||||||
id="agent-archetype"
|
|
||||||
label="archetype"
|
|
||||||
value={archetype}
|
|
||||||
onChange={pickArchetype}
|
|
||||||
disabled={!catalogue}
|
|
||||||
options={[
|
|
||||||
{ value: "", label: "none" },
|
|
||||||
...archetypes.archetypes.map((a) => ({
|
|
||||||
value: archetypeKey(a),
|
|
||||||
label: a.name,
|
|
||||||
})),
|
|
||||||
]}
|
|
||||||
/>
|
|
||||||
{picked && (
|
|
||||||
<p class="create-agent-archetype-description">
|
|
||||||
{picked.description}
|
|
||||||
</p>
|
|
||||||
)}
|
|
||||||
</>
|
|
||||||
)}
|
|
||||||
<FormField label="permissions" htmlFor="agent-permissions">
|
<FormField label="permissions" htmlFor="agent-permissions">
|
||||||
<div id="agent-permissions">
|
<div id="agent-permissions">
|
||||||
{catalogue ? (
|
{catalogue ? (
|
||||||
|
|
|
||||||
|
|
@ -1,67 +0,0 @@
|
||||||
// The create-agent form's archetype picker: what a pick puts in the form.
|
|
||||||
// Run by `npm test` with Node's type stripping.
|
|
||||||
import assert from "node:assert/strict";
|
|
||||||
import { test } from "node:test";
|
|
||||||
import { archetypeKey, prefill } from "../src/pages/CreateAgentArchetype.ts";
|
|
||||||
|
|
||||||
const defaults = ["messaging", "meta", "inbox", "execution"];
|
|
||||||
|
|
||||||
const empty = {
|
|
||||||
permissions: defaults,
|
|
||||||
cpu: "",
|
|
||||||
memory: "",
|
|
||||||
inference: { model: "", effort: "", subagents: "", maxSubagents: "" },
|
|
||||||
};
|
|
||||||
|
|
||||||
test("no archetype is the default permissions and empty settings", () => {
|
|
||||||
assert.deepEqual(prefill(undefined, defaults), empty);
|
|
||||||
});
|
|
||||||
|
|
||||||
test("an archetype's settings fill the form", () => {
|
|
||||||
const reviewer = {
|
|
||||||
source: "hyperhive",
|
|
||||||
name: "reviewer",
|
|
||||||
description: "reviews",
|
|
||||||
permissions: ["messaging", "meta", "inbox"],
|
|
||||||
limits: { cpu_quota: "200%", memory_max: "4G" },
|
|
||||||
inference: {
|
|
||||||
model: "opus",
|
|
||||||
effort: "high",
|
|
||||||
subagent_models: ["haiku", "sonnet"],
|
|
||||||
max_subagents: 0,
|
|
||||||
},
|
|
||||||
};
|
|
||||||
assert.deepEqual(prefill(reviewer, defaults), {
|
|
||||||
permissions: ["messaging", "meta", "inbox"],
|
|
||||||
cpu: "200%",
|
|
||||||
memory: "4G",
|
|
||||||
inference: {
|
|
||||||
model: "opus",
|
|
||||||
effort: "high",
|
|
||||||
subagents: "haiku, sonnet",
|
|
||||||
maxSubagents: "0",
|
|
||||||
},
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
test("an archetype that sets nothing leaves the form as no archetype does", () => {
|
|
||||||
const bare = { source: "hyperhive", name: "bare", description: "d" };
|
|
||||||
assert.deepEqual(prefill(bare, defaults), empty);
|
|
||||||
});
|
|
||||||
|
|
||||||
test("an empty permission list is kept, not replaced by the default", () => {
|
|
||||||
const none = {
|
|
||||||
source: "hyperhive",
|
|
||||||
name: "none",
|
|
||||||
description: "d",
|
|
||||||
permissions: [],
|
|
||||||
};
|
|
||||||
assert.deepEqual(prefill(none, defaults).permissions, []);
|
|
||||||
});
|
|
||||||
|
|
||||||
test("the key tells sources apart", () => {
|
|
||||||
assert.notEqual(
|
|
||||||
archetypeKey({ source: "a", name: "x", description: "" }),
|
|
||||||
archetypeKey({ source: "b", name: "x", description: "" }),
|
|
||||||
);
|
|
||||||
});
|
|
||||||
|
|
@ -1,20 +0,0 @@
|
||||||
# The archetypes this flake ships, exposed as `hyperhiveArchetypes`. Each is
|
|
||||||
# `{ module; meta; }`: `module` is a NixOS module for the agent's container,
|
|
||||||
# `meta` is plain data (no derivations, no `pkgs`), because the swarm
|
|
||||||
# builder evaluates `meta` alone and swarm-controller stores it as JSON.
|
|
||||||
# `meta` fields: `description` (required), `permissions`, `limits`,
|
|
||||||
# `inference`, in the shapes `POST /api/agents` takes them; see
|
|
||||||
# docs/swarm/archetypes.md.
|
|
||||||
{
|
|
||||||
reviewer = {
|
|
||||||
module = { };
|
|
||||||
meta = {
|
|
||||||
description = "Reviews pull requests and replies. It can run hive-forge and git (execution), and holds messaging and inbox; it has no get_agent_meta (meta).";
|
|
||||||
permissions = [
|
|
||||||
"messaging"
|
|
||||||
"inbox"
|
|
||||||
"execution"
|
|
||||||
];
|
|
||||||
};
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
@ -121,35 +121,6 @@ let
|
||||||
&& env.SWARM_BUILDER_CONFIG_ORG == "agent-configs"
|
&& env.SWARM_BUILDER_CONFIG_ORG == "agent-configs"
|
||||||
&& env.SWARM_BUILDER_FORGE_URL == "https://${withBuilder.services.hyperhive.swarm.forge.domain}";
|
&& env.SWARM_BUILDER_FORGE_URL == "https://${withBuilder.services.hyperhive.swarm.forge.domain}";
|
||||||
}
|
}
|
||||||
{
|
|
||||||
# The builder evaluates `builtins.mapAttrs (_: a: a.meta)` over this
|
|
||||||
# output and prints it as JSON. A derivation or a path in `meta` would
|
|
||||||
# make that evaluation build or copy something; a function would not
|
|
||||||
# print at all.
|
|
||||||
name = "the built-in archetypes carry a module and a meta of plain data, and reviewer can run commands";
|
|
||||||
ok =
|
|
||||||
let
|
|
||||||
plain =
|
|
||||||
v:
|
|
||||||
if lib.isDerivation v || lib.isFunction v || builtins.isPath v then
|
|
||||||
false
|
|
||||||
else if lib.isAttrs v then
|
|
||||||
lib.all plain (lib.attrValues v)
|
|
||||||
else if lib.isList v then
|
|
||||||
lib.all plain v
|
|
||||||
else
|
|
||||||
true;
|
|
||||||
archetypes = self.hyperhiveArchetypes;
|
|
||||||
metas = builtins.mapAttrs (_: a: a.meta) archetypes;
|
|
||||||
reviewer = metas.reviewer.permissions;
|
|
||||||
in
|
|
||||||
lib.all (a: a ? module && a.meta.description != "") (lib.attrValues archetypes)
|
|
||||||
&& plain metas
|
|
||||||
&& lib.elem "messaging" reviewer
|
|
||||||
&& lib.elem "inbox" reviewer
|
|
||||||
&& lib.elem "execution" reviewer
|
|
||||||
&& !(lib.elem "meta" reviewer);
|
|
||||||
}
|
|
||||||
{
|
{
|
||||||
# The responder denies a client id it does not know, and a NATS denial
|
# The responder denies a client id it does not know, and a NATS denial
|
||||||
# arrives as a timeout.
|
# arrives as a timeout.
|
||||||
|
|
|
||||||
|
|
@ -31,11 +31,3 @@ read-only forge token at every build.
|
||||||
environment, never in a URL or an argument, and the helper answers only an
|
environment, never in a URL or an argument, and the helper answers only an
|
||||||
`https` request for the forge's host. Its store role reads exactly
|
`https` request for the forge's host. Its store role reads exactly
|
||||||
those two paths.
|
those two paths.
|
||||||
|
|
||||||
## Archetypes
|
|
||||||
|
|
||||||
The service also answers archetype evaluations: `nix eval` of the `meta` of
|
|
||||||
every `hyperhiveArchetypes` entry in this host's hyperhive source, with
|
|
||||||
nixpkgs pointed at this host's. It fetches nothing and uses no token, and it
|
|
||||||
doesn't wait for a running build. See
|
|
||||||
[`docs/swarm/archetypes.md`](../docs/swarm/archetypes.md).
|
|
||||||
|
|
|
||||||
|
|
@ -1,189 +0,0 @@
|
||||||
//! One archetype evaluation: `nix eval` of the `meta` of every entry in this
|
|
||||||
//! host's hyperhive source's `hyperhiveArchetypes` output, as JSON.
|
|
||||||
//!
|
|
||||||
//! Only `meta` is evaluated, never `module`: `meta` is plain data, so the
|
|
||||||
//! evaluation builds nothing. Both inputs are store paths, so it fetches
|
|
||||||
//! nothing and needs no forge token.
|
|
||||||
|
|
||||||
use std::path::PathBuf;
|
|
||||||
use std::process::Stdio;
|
|
||||||
use std::time::Duration;
|
|
||||||
|
|
||||||
use swarm_queue_client::builder::ArchetypesReply;
|
|
||||||
|
|
||||||
/// What every evaluation is run with, fixed at start.
|
|
||||||
pub struct Evaluator {
|
|
||||||
/// The `nix` binary.
|
|
||||||
pub nix: PathBuf,
|
|
||||||
/// The hyperhive source this host was deployed from.
|
|
||||||
pub hyperhive: String,
|
|
||||||
/// The nixpkgs source this host was deployed from, which the hyperhive
|
|
||||||
/// source's own `nixpkgs` input is pointed at, as a hive's meta flake does.
|
|
||||||
pub nixpkgs: String,
|
|
||||||
/// How long one evaluation may run.
|
|
||||||
pub deadline: Duration,
|
|
||||||
}
|
|
||||||
|
|
||||||
impl Evaluator {
|
|
||||||
/// Evaluate the archetypes' `meta`.
|
|
||||||
pub async fn eval(&self) -> ArchetypesReply {
|
|
||||||
let mut cmd = tokio::process::Command::new(&self.nix);
|
|
||||||
cmd.args(self.args())
|
|
||||||
.stdin(Stdio::null())
|
|
||||||
.stdout(Stdio::piped())
|
|
||||||
.stderr(Stdio::piped())
|
|
||||||
.kill_on_drop(true);
|
|
||||||
run(cmd, &self.hyperhive, self.deadline).await
|
|
||||||
}
|
|
||||||
|
|
||||||
/// The `nix eval` arguments.
|
|
||||||
fn args(&self) -> Vec<String> {
|
|
||||||
[
|
|
||||||
"eval",
|
|
||||||
"--json",
|
|
||||||
"--no-write-lock-file",
|
|
||||||
"--option",
|
|
||||||
"accept-flake-config",
|
|
||||||
"false",
|
|
||||||
]
|
|
||||||
.into_iter()
|
|
||||||
.map(str::to_owned)
|
|
||||||
.chain([
|
|
||||||
format!("{}#hyperhiveArchetypes", self.hyperhive),
|
|
||||||
"--override-input".to_owned(),
|
|
||||||
"nixpkgs".to_owned(),
|
|
||||||
self.nixpkgs.clone(),
|
|
||||||
"--apply".to_owned(),
|
|
||||||
"builtins.mapAttrs (_: a: a.meta)".to_owned(),
|
|
||||||
])
|
|
||||||
.collect()
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Run `cmd` to its end or `deadline`, and read the JSON object it printed or
|
|
||||||
/// the tail of what it said.
|
|
||||||
async fn run(
|
|
||||||
mut cmd: tokio::process::Command,
|
|
||||||
source: &str,
|
|
||||||
deadline: Duration,
|
|
||||||
) -> ArchetypesReply {
|
|
||||||
let failed = |error_tail: String| ArchetypesReply::Failed { error_tail };
|
|
||||||
let output = match tokio::time::timeout(deadline, cmd.output()).await {
|
|
||||||
Err(_) => {
|
|
||||||
return failed(format!(
|
|
||||||
"the evaluation did not finish within {} seconds",
|
|
||||||
deadline.as_secs()
|
|
||||||
));
|
|
||||||
}
|
|
||||||
Ok(Err(e)) => return failed(format!("starting nix: {e}")),
|
|
||||||
Ok(Ok(output)) => output,
|
|
||||||
};
|
|
||||||
if !output.status.success() {
|
|
||||||
return failed(crate::nix_build::tail(
|
|
||||||
&String::from_utf8_lossy(&output.stderr),
|
|
||||||
output.status,
|
|
||||||
));
|
|
||||||
}
|
|
||||||
match serde_json::from_slice::<serde_json::Value>(&output.stdout) {
|
|
||||||
Ok(serde_json::Value::Object(archetypes)) => ArchetypesReply::Evaluated {
|
|
||||||
source: source.to_owned(),
|
|
||||||
archetypes,
|
|
||||||
},
|
|
||||||
Ok(other) => failed(format!(
|
|
||||||
"hyperhiveArchetypes is not an attribute set of archetypes: {other}"
|
|
||||||
)),
|
|
||||||
Err(e) => failed(format!("nix printed no JSON: {e}")),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
use super::*;
|
|
||||||
|
|
||||||
fn evaluator() -> Evaluator {
|
|
||||||
Evaluator {
|
|
||||||
nix: PathBuf::from("nix"),
|
|
||||||
hyperhive: "/nix/store/h-source".to_owned(),
|
|
||||||
nixpkgs: "/nix/store/n-source".to_owned(),
|
|
||||||
deadline: swarm_queue_client::builder::ARCHETYPES_DEADLINE,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn only_meta_is_evaluated_on_this_hosts_sources() {
|
|
||||||
let args = evaluator().args();
|
|
||||||
assert!(
|
|
||||||
args.iter()
|
|
||||||
.any(|a| a == "/nix/store/h-source#hyperhiveArchetypes")
|
|
||||||
);
|
|
||||||
let overridden: Vec<&str> = args
|
|
||||||
.windows(2)
|
|
||||||
.filter(|w| w[0] == "--override-input")
|
|
||||||
.map(|w| w[1].as_str())
|
|
||||||
.collect();
|
|
||||||
assert_eq!(overridden, ["nixpkgs"]);
|
|
||||||
let apply = args
|
|
||||||
.windows(2)
|
|
||||||
.find(|w| w[0] == "--apply")
|
|
||||||
.map(|w| w[1].as_str());
|
|
||||||
assert_eq!(apply, Some("builtins.mapAttrs (_: a: a.meta)"));
|
|
||||||
assert!(args.iter().any(|a| a == "--no-write-lock-file"));
|
|
||||||
}
|
|
||||||
|
|
||||||
fn sh(script: &str, deadline: Duration) -> impl Future<Output = ArchetypesReply> {
|
|
||||||
let mut cmd = tokio::process::Command::new("sh");
|
|
||||||
cmd.args(["-c", script])
|
|
||||||
.stdin(Stdio::null())
|
|
||||||
.stdout(Stdio::piped())
|
|
||||||
.stderr(Stdio::piped())
|
|
||||||
.kill_on_drop(true);
|
|
||||||
run(cmd, "/nix/store/h-source", deadline)
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
async fn an_evaluation_replies_with_each_archetypes_meta() {
|
|
||||||
let reply = sh(
|
|
||||||
r#"echo '{"reviewer":{"description":"reads"}}'"#,
|
|
||||||
Duration::from_secs(30),
|
|
||||||
)
|
|
||||||
.await;
|
|
||||||
let ArchetypesReply::Evaluated { source, archetypes } = reply else {
|
|
||||||
panic!("{reply:?}")
|
|
||||||
};
|
|
||||||
assert_eq!(source, "/nix/store/h-source");
|
|
||||||
assert_eq!(archetypes["reviewer"]["description"], "reads");
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
async fn a_failure_replies_with_what_nix_said_last() {
|
|
||||||
let reply = sh(
|
|
||||||
"echo 'error: attribute hyperhiveArchetypes missing' >&2; exit 1",
|
|
||||||
Duration::from_secs(30),
|
|
||||||
)
|
|
||||||
.await;
|
|
||||||
assert!(
|
|
||||||
matches!(&reply, ArchetypesReply::Failed { error_tail } if error_tail.ends_with("missing")),
|
|
||||||
"{reply:?}"
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
async fn output_that_is_not_an_attribute_set_is_refused() {
|
|
||||||
for script in ["echo '[1]'", "echo 'not json'"] {
|
|
||||||
let reply = sh(script, Duration::from_secs(30)).await;
|
|
||||||
assert!(
|
|
||||||
matches!(reply, ArchetypesReply::Failed { .. }),
|
|
||||||
"{script}: {reply:?}"
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
async fn an_evaluation_past_its_deadline_is_stopped() {
|
|
||||||
let reply = sh("sleep 30", Duration::from_millis(200)).await;
|
|
||||||
assert!(
|
|
||||||
matches!(&reply, ArchetypesReply::Failed { error_tail } if error_tail.contains("did not finish")),
|
|
||||||
"{reply:?}"
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
@ -4,9 +4,7 @@
|
||||||
//! from swarm-controller on [`swarm_queue_client::builder::SUBJECT`], builds
|
//! from swarm-controller on [`swarm_queue_client::builder::SUBJECT`], builds
|
||||||
//! each agent config at its commit, answers on the request's reply subject,
|
//! each agent config at its commit, answers on the request's reply subject,
|
||||||
//! and logs one line per build. It writes nothing to the forge: the
|
//! and logs one line per build. It writes nothing to the forge: the
|
||||||
//! controller turns the reply into a commit status. It also answers
|
//! controller turns the reply into a commit status.
|
||||||
//! [`swarm_queue_client::builder::ARCHETYPES_SUBJECT`] with the agent
|
|
||||||
//! archetypes its hyperhive source declares, which the controller stores.
|
|
||||||
//!
|
//!
|
||||||
//! The store identity comes from the `BAO_*` environment. The queue client
|
//! The store identity comes from the `BAO_*` environment. The queue client
|
||||||
//! secret is read at start and again when authelia refuses it, the forge token
|
//! secret is read at start and again when authelia refuses it, the forge token
|
||||||
|
|
@ -17,15 +15,11 @@ use std::sync::Arc;
|
||||||
|
|
||||||
use anyhow::{Context, Result, bail};
|
use anyhow::{Context, Result, bail};
|
||||||
use futures_util::StreamExt as _;
|
use futures_util::StreamExt as _;
|
||||||
use swarm_queue_client::builder::{
|
use swarm_queue_client::builder::{BUILD_DEADLINE, BuildAgent, QUEUE_GROUP, Reply, SUBJECT};
|
||||||
ARCHETYPES_DEADLINE, ARCHETYPES_SUBJECT, BUILD_DEADLINE, BuildAgent, QUEUE_GROUP, Reply,
|
|
||||||
SUBJECT,
|
|
||||||
};
|
|
||||||
use swarm_queue_client::stored_secret::StoredSecret;
|
use swarm_queue_client::stored_secret::StoredSecret;
|
||||||
use swarm_queue_client::{ClientSecret, QueueConfig};
|
use swarm_queue_client::{ClientSecret, QueueConfig};
|
||||||
use swarm_secret_client::forge::Credential;
|
use swarm_secret_client::forge::Credential;
|
||||||
|
|
||||||
mod archetypes;
|
|
||||||
mod nix_build;
|
mod nix_build;
|
||||||
|
|
||||||
/// The cert-auth role this service logs in to the store as. `swarm-bao.nix`
|
/// The cert-auth role this service logs in to the store as. `swarm-bao.nix`
|
||||||
|
|
@ -58,12 +52,6 @@ async fn main() -> Result<()> {
|
||||||
client_secret(&client_id)
|
client_secret(&client_id)
|
||||||
})
|
})
|
||||||
.await?;
|
.await?;
|
||||||
let evaluator = archetypes::Evaluator {
|
|
||||||
nix: builder.nix.clone(),
|
|
||||||
hyperhive: builder.hyperhive.clone(),
|
|
||||||
nixpkgs: builder.nixpkgs.clone(),
|
|
||||||
deadline: ARCHETYPES_DEADLINE,
|
|
||||||
};
|
|
||||||
let reread_id = client_id.clone();
|
let reread_id = client_id.clone();
|
||||||
let secret = StoredSecret::new(client_id.clone(), secret, move || {
|
let secret = StoredSecret::new(client_id.clone(), secret, move || {
|
||||||
let client_id = reread_id.clone();
|
let client_id = reread_id.clone();
|
||||||
|
|
@ -84,12 +72,6 @@ async fn main() -> Result<()> {
|
||||||
.await
|
.await
|
||||||
.with_context(|| format!("subscribing to {SUBJECT}"))?;
|
.with_context(|| format!("subscribing to {SUBJECT}"))?;
|
||||||
tracing::info!(subject = SUBJECT, "builder: awaiting builds");
|
tracing::info!(subject = SUBJECT, "builder: awaiting builds");
|
||||||
let evaluations = queue
|
|
||||||
.queue_subscribe(ARCHETYPES_SUBJECT, QUEUE_GROUP.to_owned())
|
|
||||||
.await
|
|
||||||
.with_context(|| format!("subscribing to {ARCHETYPES_SUBJECT}"))?;
|
|
||||||
// Outside the build slot below: a list refresh does not wait on a build.
|
|
||||||
tokio::spawn(answer_evaluations(evaluations, queue.clone(), evaluator));
|
|
||||||
|
|
||||||
// One build at a time. swarm-controller already sends one at a time; this
|
// One build at a time. swarm-controller already sends one at a time; this
|
||||||
// holds the bound for any request it did not serialise.
|
// holds the bound for any request it did not serialise.
|
||||||
|
|
@ -121,39 +103,6 @@ async fn main() -> Result<()> {
|
||||||
bail!("the subscription to {SUBJECT} ended")
|
bail!("the subscription to {SUBJECT} ended")
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Answer every archetype evaluation request, one at a time, with the
|
|
||||||
/// evaluation's result.
|
|
||||||
async fn answer_evaluations(
|
|
||||||
mut requests: async_nats::Subscriber,
|
|
||||||
queue: async_nats::Client,
|
|
||||||
evaluator: archetypes::Evaluator,
|
|
||||||
) {
|
|
||||||
while let Some(msg) = requests.next().await {
|
|
||||||
let Some(reply_to) = msg.reply.clone() else {
|
|
||||||
tracing::warn!("an archetype evaluation arrived with no reply subject; dropping it");
|
|
||||||
continue;
|
|
||||||
};
|
|
||||||
let reply = evaluator.eval().await;
|
|
||||||
match &reply {
|
|
||||||
swarm_queue_client::builder::ArchetypesReply::Evaluated { archetypes, .. } => {
|
|
||||||
tracing::info!(count = archetypes.len(), "archetypes evaluated");
|
|
||||||
}
|
|
||||||
swarm_queue_client::builder::ArchetypesReply::Failed { error_tail } => {
|
|
||||||
tracing::warn!(%error_tail, "archetype evaluation failed");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
match serde_json::to_vec(&reply) {
|
|
||||||
Ok(body) => {
|
|
||||||
if let Err(e) = queue.publish(reply_to, body.into()).await {
|
|
||||||
tracing::warn!(error = %e, "publishing an evaluation's reply failed");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
Err(e) => tracing::warn!(error = %e, "encoding an evaluation's reply failed"),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
tracing::error!("the subscription to {ARCHETYPES_SUBJECT} ended");
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Decode one request, build it and log the build.
|
/// Decode one request, build it and log the build.
|
||||||
async fn handle(payload: &[u8], builder: &nix_build::Builder) -> Reply {
|
async fn handle(payload: &[u8], builder: &nix_build::Builder) -> Reply {
|
||||||
let job: BuildAgent = match serde_json::from_slice(payload) {
|
let job: BuildAgent = match serde_json::from_slice(payload) {
|
||||||
|
|
|
||||||
|
|
@ -271,7 +271,7 @@ async fn read_tail(
|
||||||
|
|
||||||
/// The last [`TAIL_LINES`] lines of `stderr`, at most [`TAIL_BYTES`] bytes,
|
/// The last [`TAIL_LINES`] lines of `stderr`, at most [`TAIL_BYTES`] bytes,
|
||||||
/// or the exit status when it said nothing.
|
/// or the exit status when it said nothing.
|
||||||
pub(crate) fn tail(stderr: &str, status: std::process::ExitStatus) -> String {
|
fn tail(stderr: &str, status: std::process::ExitStatus) -> String {
|
||||||
let lines: Vec<&str> = stderr.lines().filter(|l| !l.trim().is_empty()).collect();
|
let lines: Vec<&str> = stderr.lines().filter(|l| !l.trim().is_empty()).collect();
|
||||||
if lines.is_empty() {
|
if lines.is_empty() {
|
||||||
return format!("nix exited with {status} and printed nothing");
|
return format!("nix exited with {status} and printed nothing");
|
||||||
|
|
|
||||||
|
|
@ -1,432 +0,0 @@
|
||||||
//! Agent archetypes: the `agent-archetypes` record this daemon keeps from the
|
|
||||||
//! swarm builder's evaluation, and `GET /api/archetypes`, which the create
|
|
||||||
//! screen prefills from.
|
|
||||||
//!
|
|
||||||
//! Every [`REFRESH_INTERVAL`], starting once the queue is connected, the pass
|
|
||||||
//! asks the builder for the `meta` of its hyperhive source's archetypes, checks
|
|
||||||
//! each one the way `POST /api/agents` checks its settings, and writes the
|
|
||||||
//! result under [`BUILTIN_SOURCE`] when it differs from what is stored. A
|
|
||||||
//! failed or unanswered evaluation keeps the stored record.
|
|
||||||
|
|
||||||
use std::time::Duration;
|
|
||||||
|
|
||||||
use axum::Json;
|
|
||||||
use axum::extract::State;
|
|
||||||
use hive_sh4re::permissions::Permission;
|
|
||||||
use problem_details::ProblemDetails;
|
|
||||||
use serde::{Deserialize, Serialize};
|
|
||||||
use swarm_queue_client::archetypes::{ArchetypeMeta, BUILTIN_SOURCE, SourceRecord};
|
|
||||||
use swarm_queue_client::builder::{
|
|
||||||
ARCHETYPES_DEADLINE, ARCHETYPES_SUBJECT, ArchetypesReply, EvalArchetypes,
|
|
||||||
};
|
|
||||||
use utoipa::ToSchema;
|
|
||||||
|
|
||||||
use crate::schedules::{queue, read_or_default, refused, write};
|
|
||||||
use crate::{AppState, agent_limits, agent_permissions, error_problem, inference};
|
|
||||||
|
|
||||||
/// How often the builder is asked. Its source only changes when its host is
|
|
||||||
/// redeployed, so this bounds how long a redeploy takes to reach the create
|
|
||||||
/// screen.
|
|
||||||
const REFRESH_INTERVAL: Duration = Duration::from_mins(10);
|
|
||||||
|
|
||||||
/// One archetype's `meta` as a source flake declares it. Keys this daemon
|
|
||||||
/// does not know, `ui` among them, are ignored, so an archetype written for a
|
|
||||||
/// newer swarm still lists.
|
|
||||||
#[derive(Deserialize)]
|
|
||||||
struct DeclaredMeta {
|
|
||||||
description: String,
|
|
||||||
#[serde(default)]
|
|
||||||
permissions: Option<Vec<Permission>>,
|
|
||||||
#[serde(default)]
|
|
||||||
limits: Option<agent_limits::Caps>,
|
|
||||||
#[serde(default)]
|
|
||||||
inference: Option<inference::InferenceSettings>,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// `meta` as stored, or why a create would refuse it.
|
|
||||||
fn check(name: &str, meta: serde_json::Value) -> Result<ArchetypeMeta, String> {
|
|
||||||
hive_types::Ident::parse(name).map_err(|reason| format!("name: {reason}"))?;
|
|
||||||
let declared: DeclaredMeta = serde_json::from_value(meta).map_err(|e| e.to_string())?;
|
|
||||||
if declared.description.trim().is_empty() {
|
|
||||||
return Err("description is empty".to_owned());
|
|
||||||
}
|
|
||||||
let permissions = match declared.permissions {
|
|
||||||
Some(perms) => {
|
|
||||||
agent_permissions::unique(&perms)?;
|
|
||||||
Some(
|
|
||||||
perms
|
|
||||||
.into_iter()
|
|
||||||
.map(|p| <&str>::from(p).to_owned())
|
|
||||||
.collect(),
|
|
||||||
)
|
|
||||||
}
|
|
||||||
None => None,
|
|
||||||
};
|
|
||||||
Ok(ArchetypeMeta {
|
|
||||||
description: declared.description,
|
|
||||||
permissions,
|
|
||||||
limits: declared
|
|
||||||
.limits
|
|
||||||
.map(agent_limits::Caps::into_stored)
|
|
||||||
.transpose()
|
|
||||||
.map_err(|e| format!("limits: {e}"))?,
|
|
||||||
inference: declared
|
|
||||||
.inference
|
|
||||||
.map(inference::InferenceSettings::into_stored)
|
|
||||||
.transpose()
|
|
||||||
.map_err(|e| format!("inference: {e}"))?,
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
/// The record to store for one evaluation: every archetype that passes
|
|
||||||
/// [`check`], and every one that does not with its reason.
|
|
||||||
fn record(source: String, archetypes: serde_json::Map<String, serde_json::Value>) -> SourceRecord {
|
|
||||||
let mut out = SourceRecord {
|
|
||||||
source,
|
|
||||||
..SourceRecord::default()
|
|
||||||
};
|
|
||||||
for (name, meta) in archetypes {
|
|
||||||
match check(&name, meta) {
|
|
||||||
Ok(meta) => {
|
|
||||||
out.archetypes.insert(name, meta);
|
|
||||||
}
|
|
||||||
Err(why) => {
|
|
||||||
out.refused.insert(name, why);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
out
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Ask the builder once and answer what to store, or why there is nothing to.
|
|
||||||
async fn evaluate(client: &async_nats::Client) -> Result<SourceRecord, String> {
|
|
||||||
use async_nats::RequestErrorKind;
|
|
||||||
|
|
||||||
let body = serde_json::to_vec(&EvalArchetypes {}).map_err(|e| e.to_string())?;
|
|
||||||
let sent = async_nats::Request::new()
|
|
||||||
.payload(body.into())
|
|
||||||
.timeout(Some(ARCHETYPES_DEADLINE));
|
|
||||||
let msg = client
|
|
||||||
.send_request(ARCHETYPES_SUBJECT, sent)
|
|
||||||
.await
|
|
||||||
.map_err(|e| match e.kind() {
|
|
||||||
RequestErrorKind::NoResponders => "no swarm builder is listening: \
|
|
||||||
deploy.builder.enable is set on no host, or its service is not connected to \
|
|
||||||
the swarm queue"
|
|
||||||
.to_owned(),
|
|
||||||
_ => format!("asking the swarm builder failed: {e}"),
|
|
||||||
})?;
|
|
||||||
match serde_json::from_slice(&msg.payload) {
|
|
||||||
Ok(ArchetypesReply::Evaluated { source, archetypes }) => Ok(record(source, archetypes)),
|
|
||||||
Ok(ArchetypesReply::Failed { error_tail }) => {
|
|
||||||
Err(format!("the evaluation failed: {error_tail}"))
|
|
||||||
}
|
|
||||||
Err(e) => Err(format!("the builder's reply did not decode: {e}")),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Store `value` under [`BUILTIN_SOURCE`] unless it is already there.
|
|
||||||
/// Answers whether it wrote.
|
|
||||||
async fn store(client: &async_nats::Client, value: &SourceRecord) -> anyhow::Result<bool> {
|
|
||||||
let store = swarm_queue_client::archetypes::open_or_create(client).await?;
|
|
||||||
let (current, revision, undecodable) =
|
|
||||||
read_or_default::<SourceRecord>(&store, BUILTIN_SOURCE).await?;
|
|
||||||
if current == *value && !undecodable {
|
|
||||||
return Ok(false);
|
|
||||||
}
|
|
||||||
// A lost race is another write of the same evaluation; the next pass
|
|
||||||
// compares again.
|
|
||||||
write(&store, BUILTIN_SOURCE, value, revision).await
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Refresh the record once the queue is connected, then every
|
|
||||||
/// [`REFRESH_INTERVAL`].
|
|
||||||
pub(crate) fn spawn(client: async_nats::Client) {
|
|
||||||
tokio::spawn(async move {
|
|
||||||
const CONNECT_POLL: Duration = Duration::from_secs(5);
|
|
||||||
loop {
|
|
||||||
while swarm_queue_client::ensure_connected(&client).is_err() {
|
|
||||||
tokio::time::sleep(CONNECT_POLL).await;
|
|
||||||
}
|
|
||||||
refresh(&client).await;
|
|
||||||
tokio::time::sleep(REFRESH_INTERVAL).await;
|
|
||||||
}
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
/// One evaluation, stored when it changed anything.
|
|
||||||
async fn refresh(client: &async_nats::Client) {
|
|
||||||
let value = match evaluate(client).await {
|
|
||||||
Ok(value) => value,
|
|
||||||
Err(error) => {
|
|
||||||
tracing::warn!(%error, "archetypes: kept the stored list");
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
};
|
|
||||||
for (name, why) in &value.refused {
|
|
||||||
tracing::warn!(archetype = %name, %why, "archetypes: refused");
|
|
||||||
}
|
|
||||||
match store(client, &value).await {
|
|
||||||
Ok(true) => tracing::info!(
|
|
||||||
count = value.archetypes.len(),
|
|
||||||
source = %value.source,
|
|
||||||
"archetypes: stored"
|
|
||||||
),
|
|
||||||
Ok(false) => {}
|
|
||||||
Err(e) => tracing::warn!(error = %format!("{e:#}"), "archetypes: storing failed"),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// One archetype the create screen can offer, with the settings it prefills.
|
|
||||||
#[derive(Debug, PartialEq, Eq, Serialize, ToSchema)]
|
|
||||||
pub(crate) struct ArchetypeView {
|
|
||||||
/// The source flake's key, `hyperhive` for the built-ins.
|
|
||||||
source: String,
|
|
||||||
name: String,
|
|
||||||
description: String,
|
|
||||||
/// Permission names; absent leaves the default set.
|
|
||||||
#[serde(skip_serializing_if = "Option::is_none")]
|
|
||||||
permissions: Option<Vec<String>>,
|
|
||||||
/// As `PUT /api/agents/{name}/limits` takes them.
|
|
||||||
#[serde(skip_serializing_if = "Option::is_none")]
|
|
||||||
#[schema(value_type = Option<Object>)]
|
|
||||||
limits: Option<swarm_queue_client::agent_limits::AgentCaps>,
|
|
||||||
/// As `PUT /api/agents/{name}/inference` takes them.
|
|
||||||
#[serde(skip_serializing_if = "Option::is_none")]
|
|
||||||
#[schema(value_type = Option<Object>)]
|
|
||||||
inference: Option<swarm_queue_client::agent_inference::AgentInference>,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// One archetype a source declared and this daemon refused.
|
|
||||||
#[derive(Debug, PartialEq, Eq, Serialize, ToSchema)]
|
|
||||||
pub(crate) struct RefusedView {
|
|
||||||
source: String,
|
|
||||||
name: String,
|
|
||||||
reason: String,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Debug, Default, PartialEq, Eq, Serialize, ToSchema)]
|
|
||||||
pub(crate) struct Archetypes {
|
|
||||||
archetypes: Vec<ArchetypeView>,
|
|
||||||
refused: Vec<RefusedView>,
|
|
||||||
}
|
|
||||||
|
|
||||||
fn view(key: &str, record: SourceRecord) -> Archetypes {
|
|
||||||
let mut out = Archetypes::default();
|
|
||||||
for (name, meta) in record.archetypes {
|
|
||||||
out.archetypes.push(ArchetypeView {
|
|
||||||
source: key.to_owned(),
|
|
||||||
name,
|
|
||||||
description: meta.description,
|
|
||||||
permissions: meta.permissions,
|
|
||||||
limits: meta.limits,
|
|
||||||
inference: meta.inference,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
for (name, reason) in record.refused {
|
|
||||||
out.refused.push(RefusedView {
|
|
||||||
source: key.to_owned(),
|
|
||||||
name,
|
|
||||||
reason,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
out
|
|
||||||
}
|
|
||||||
|
|
||||||
#[utoipa::path(
|
|
||||||
get,
|
|
||||||
path = "/api/archetypes",
|
|
||||||
responses(
|
|
||||||
(status = 200, description = "every archetype the swarm builder's evaluation passed, \
|
|
||||||
and every one it refused; both empty before the first evaluation", body = Archetypes),
|
|
||||||
(status = 503, description = "no swarm queue is configured on this host, or it is \
|
|
||||||
not connected (problem+json)", body = String),
|
|
||||||
(status = 500, description = "the archetypes could not be read (problem+json)",
|
|
||||||
body = String),
|
|
||||||
),
|
|
||||||
tag = "agents"
|
|
||||||
)]
|
|
||||||
pub(crate) async fn get_archetypes(
|
|
||||||
State(state): State<AppState>,
|
|
||||||
) -> Result<Json<Archetypes>, ProblemDetails> {
|
|
||||||
let client = queue(&state).map_err(refused)?;
|
|
||||||
let failed = |e: anyhow::Error| {
|
|
||||||
tracing::warn!(error = %format!("{e:#}"), "archetypes: read failed");
|
|
||||||
error_problem(
|
|
||||||
axum::http::StatusCode::INTERNAL_SERVER_ERROR,
|
|
||||||
&format!("{e:#}"),
|
|
||||||
)
|
|
||||||
};
|
|
||||||
let Some(store) = swarm_queue_client::archetypes::open(&client)
|
|
||||||
.await
|
|
||||||
.map_err(|e| failed(e.into()))?
|
|
||||||
else {
|
|
||||||
return Ok(Json(Archetypes::default()));
|
|
||||||
};
|
|
||||||
let (record, _, _) = read_or_default::<SourceRecord>(&store, BUILTIN_SOURCE)
|
|
||||||
.await
|
|
||||||
.map_err(failed)?;
|
|
||||||
Ok(Json(view(BUILTIN_SOURCE, record)))
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
use super::*;
|
|
||||||
use serde_json::json;
|
|
||||||
|
|
||||||
fn evaluated(archetypes: serde_json::Value) -> SourceRecord {
|
|
||||||
let serde_json::Value::Object(map) = archetypes else {
|
|
||||||
panic!("an object")
|
|
||||||
};
|
|
||||||
record("/nix/store/h".to_owned(), map)
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn a_valid_archetype_is_stored_with_its_settings() {
|
|
||||||
let r = evaluated(json!({
|
|
||||||
"reviewer": {
|
|
||||||
"description": "reviews",
|
|
||||||
"permissions": ["messaging", "meta", "inbox"],
|
|
||||||
"limits": { "memory_max": "4G" },
|
|
||||||
"inference": { "effort": "high" },
|
|
||||||
"ui": { "anything": true }
|
|
||||||
}
|
|
||||||
}));
|
|
||||||
assert!(r.refused.is_empty(), "{:?}", r.refused);
|
|
||||||
let meta = &r.archetypes["reviewer"];
|
|
||||||
assert_eq!(meta.description, "reviews");
|
|
||||||
assert_eq!(
|
|
||||||
meta.permissions.as_deref(),
|
|
||||||
Some(
|
|
||||||
&[
|
|
||||||
"messaging".to_owned(),
|
|
||||||
"meta".to_owned(),
|
|
||||||
"inbox".to_owned()
|
|
||||||
][..]
|
|
||||||
)
|
|
||||||
);
|
|
||||||
assert_eq!(
|
|
||||||
meta.limits.as_ref().and_then(|l| l.memory_max.as_deref()),
|
|
||||||
Some("4G")
|
|
||||||
);
|
|
||||||
assert!(meta.inference.as_ref().is_some_and(|i| i.effort.is_some()));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn a_bad_archetype_is_refused_by_name_and_the_rest_stay() {
|
|
||||||
let r = evaluated(json!({
|
|
||||||
"ok": { "description": "fine" },
|
|
||||||
"no-description": { "description": " " },
|
|
||||||
"bad-permission": { "description": "x", "permissions": ["root"] },
|
|
||||||
"twice": { "description": "x", "permissions": ["meta", "meta"] },
|
|
||||||
"bad-limits": { "description": "x", "limits": { "cpu_quota": "lots" } },
|
|
||||||
"typo-limits": { "description": "x", "limits": { "memory": "4G" } },
|
|
||||||
"bad-inference": { "description": "x", "inference": { "effort": "huge" } },
|
|
||||||
"Bad_Name": { "description": "x" }
|
|
||||||
}));
|
|
||||||
assert_eq!(r.archetypes.keys().collect::<Vec<_>>(), ["ok"]);
|
|
||||||
let refused: Vec<&str> = r.refused.keys().map(String::as_str).collect();
|
|
||||||
assert_eq!(
|
|
||||||
refused,
|
|
||||||
[
|
|
||||||
"Bad_Name",
|
|
||||||
"bad-inference",
|
|
||||||
"bad-limits",
|
|
||||||
"bad-permission",
|
|
||||||
"no-description",
|
|
||||||
"twice",
|
|
||||||
"typo-limits"
|
|
||||||
]
|
|
||||||
);
|
|
||||||
assert!(r.refused["bad-limits"].starts_with("limits: "));
|
|
||||||
assert!(r.refused["twice"].contains("listed twice"));
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Against a real server: no builder listening names the option; a
|
|
||||||
/// listener's evaluation is checked and stored, and storing the same
|
|
||||||
/// record again writes nothing.
|
|
||||||
#[tokio::test]
|
|
||||||
async fn an_evaluation_is_stored_and_rewritten_only_when_it_changes() {
|
|
||||||
use futures_util::StreamExt as _;
|
|
||||||
|
|
||||||
let nats = crate::test_nats::TestNats::start();
|
|
||||||
let client = nats.client().await;
|
|
||||||
let unanswered = evaluate(&client).await;
|
|
||||||
assert!(
|
|
||||||
matches!(&unanswered, Err(e) if e.contains("deploy.builder.enable")),
|
|
||||||
"{unanswered:?}"
|
|
||||||
);
|
|
||||||
|
|
||||||
let builder = nats.client().await;
|
|
||||||
let mut requests = builder
|
|
||||||
.queue_subscribe(
|
|
||||||
ARCHETYPES_SUBJECT,
|
|
||||||
swarm_queue_client::builder::QUEUE_GROUP.to_owned(),
|
|
||||||
)
|
|
||||||
.await
|
|
||||||
.expect("subscribe");
|
|
||||||
// A message the builder sends itself and receives back proves the
|
|
||||||
// server processed the subscription before it.
|
|
||||||
let mut probe = builder
|
|
||||||
.subscribe("archetypes-test.probe")
|
|
||||||
.await
|
|
||||||
.expect("subscribe");
|
|
||||||
builder
|
|
||||||
.publish("archetypes-test.probe", "".into())
|
|
||||||
.await
|
|
||||||
.expect("publish");
|
|
||||||
probe.next().await.expect("the probe comes back");
|
|
||||||
tokio::spawn(async move {
|
|
||||||
let msg = requests.next().await.expect("a request");
|
|
||||||
let serde_json::Value::Object(archetypes) = json!({
|
|
||||||
"reviewer": { "description": "reviews", "permissions": ["meta"] },
|
|
||||||
"broken": { "description": "" }
|
|
||||||
}) else {
|
|
||||||
unreachable!()
|
|
||||||
};
|
|
||||||
let reply = serde_json::to_vec(&ArchetypesReply::Evaluated {
|
|
||||||
source: "/nix/store/h".to_owned(),
|
|
||||||
archetypes,
|
|
||||||
})
|
|
||||||
.expect("encodes");
|
|
||||||
builder
|
|
||||||
.publish(msg.reply.expect("a reply subject"), reply.into())
|
|
||||||
.await
|
|
||||||
.expect("publish");
|
|
||||||
});
|
|
||||||
let value = evaluate(&client).await.expect("evaluated");
|
|
||||||
assert_eq!(value.archetypes.keys().collect::<Vec<_>>(), ["reviewer"]);
|
|
||||||
assert_eq!(value.refused.keys().collect::<Vec<_>>(), ["broken"]);
|
|
||||||
|
|
||||||
assert!(store(&client, &value).await.expect("stored"));
|
|
||||||
assert!(!store(&client, &value).await.expect("compared"));
|
|
||||||
let changed = SourceRecord {
|
|
||||||
source: "/nix/store/h2".to_owned(),
|
|
||||||
..value
|
|
||||||
};
|
|
||||||
assert!(store(&client, &changed).await.expect("stored"));
|
|
||||||
let bucket = swarm_queue_client::archetypes::open(&client)
|
|
||||||
.await
|
|
||||||
.expect("opens")
|
|
||||||
.expect("exists");
|
|
||||||
let (stored, _, _) = read_or_default::<SourceRecord>(&bucket, BUILTIN_SOURCE)
|
|
||||||
.await
|
|
||||||
.expect("reads");
|
|
||||||
assert_eq!(stored, changed);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn the_view_lists_each_archetype_under_its_source() {
|
|
||||||
let r = evaluated(json!({
|
|
||||||
"reviewer": { "description": "reviews" },
|
|
||||||
"broken": { "description": "" }
|
|
||||||
}));
|
|
||||||
let v = view(BUILTIN_SOURCE, r);
|
|
||||||
assert_eq!(v.archetypes.len(), 1);
|
|
||||||
assert_eq!(v.archetypes[0].source, "hyperhive");
|
|
||||||
assert_eq!(v.archetypes[0].name, "reviewer");
|
|
||||||
assert_eq!(v.archetypes[0].permissions, None);
|
|
||||||
assert_eq!(v.refused.len(), 1);
|
|
||||||
assert_eq!(v.refused[0].name, "broken");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
@ -53,7 +53,6 @@ mod agent_renewal;
|
||||||
mod agent_state_stream;
|
mod agent_state_stream;
|
||||||
mod agent_status;
|
mod agent_status;
|
||||||
mod alerts;
|
mod alerts;
|
||||||
mod archetypes;
|
|
||||||
mod auth;
|
mod auth;
|
||||||
mod builder;
|
mod builder;
|
||||||
mod claude_login;
|
mod claude_login;
|
||||||
|
|
@ -2989,16 +2988,12 @@ fn spawn_subagent_streams(status: Option<&Arc<status::StatusReader>>, hives: &[H
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The per-agent background passes: subagent and terminal streams, and
|
/// The per-agent background passes: subagent and terminal streams, and
|
||||||
/// schedule keys, and the permissions bucket and its import, and the
|
/// schedule keys, and the permissions bucket and its import.
|
||||||
/// archetypes agents are created from.
|
|
||||||
/// Lifted out of `main` for `clippy::too_many_lines`.
|
/// Lifted out of `main` for `clippy::too_many_lines`.
|
||||||
fn spawn_per_agent_upkeep(status: Option<&Arc<status::StatusReader>>, hives: &[HiveEntry]) {
|
fn spawn_per_agent_upkeep(status: Option<&Arc<status::StatusReader>>, hives: &[HiveEntry]) {
|
||||||
spawn_subagent_streams(status, hives);
|
spawn_subagent_streams(status, hives);
|
||||||
spawn_schedule_reconcile(status, hives);
|
spawn_schedule_reconcile(status, hives);
|
||||||
spawn_agent_permissions(status, hives);
|
spawn_agent_permissions(status, hives);
|
||||||
if let Some(status) = status {
|
|
||||||
archetypes::spawn(status.queue_client());
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Create the agent-permissions bucket and answer hives' imports when a swarm
|
/// Create the agent-permissions bucket and answer hives' imports when a swarm
|
||||||
|
|
@ -3629,7 +3624,6 @@ fn build_app(state: AppState) -> axum::Router {
|
||||||
agent_permissions::delete_agent_permissions
|
agent_permissions::delete_agent_permissions
|
||||||
))
|
))
|
||||||
.routes(routes!(agent_permissions::get_permission_catalogue))
|
.routes(routes!(agent_permissions::get_permission_catalogue))
|
||||||
.routes(routes!(archetypes::get_archetypes))
|
|
||||||
.routes(routes!(get_config_prs))
|
.routes(routes!(get_config_prs))
|
||||||
.routes(routes!(create_agent))
|
.routes(routes!(create_agent))
|
||||||
.routes(routes!(mint_agent_forge_token))
|
.routes(routes!(mint_agent_forge_token))
|
||||||
|
|
|
||||||
|
|
@ -263,16 +263,14 @@ impl Policy {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// What the builder may do: hear build and archetype evaluation requests
|
/// What the builder may do: hear build requests and answer each on its
|
||||||
/// and answer each on its reply subject, for as long as a build may take.
|
/// reply subject, for as long as a build may take. Its publish list is
|
||||||
/// Its publish list is its own inbox, for
|
/// its own inbox, for [`Self::review_classifier_permissions`]' reason.
|
||||||
/// [`Self::review_classifier_permissions`]' reason.
|
|
||||||
fn builder_permissions(client_id: &str) -> Permissions {
|
fn builder_permissions(client_id: &str) -> Permissions {
|
||||||
Permissions {
|
Permissions {
|
||||||
publish: vec![Self::inbox(client_id)],
|
publish: vec![Self::inbox(client_id)],
|
||||||
subscribe: Some(vec![
|
subscribe: Some(vec![
|
||||||
swarm_queue_client::builder::SUBJECT.to_owned(),
|
swarm_queue_client::builder::SUBJECT.to_owned(),
|
||||||
swarm_queue_client::builder::ARCHETYPES_SUBJECT.to_owned(),
|
|
||||||
Self::inbox(client_id),
|
Self::inbox(client_id),
|
||||||
]),
|
]),
|
||||||
responses: Some(swarm_queue_client::builder::REPLY_DEADLINE),
|
responses: Some(swarm_queue_client::builder::REPLY_DEADLINE),
|
||||||
|
|
@ -645,8 +643,6 @@ impl Policy {
|
||||||
swarm_queue_client::review_classifier::SUBJECT.to_owned(),
|
swarm_queue_client::review_classifier::SUBJECT.to_owned(),
|
||||||
// A build of an agent config, sent as a request the same way.
|
// A build of an agent config, sent as a request the same way.
|
||||||
swarm_queue_client::builder::SUBJECT.to_owned(),
|
swarm_queue_client::builder::SUBJECT.to_owned(),
|
||||||
// An archetype evaluation, sent as a request the same way.
|
|
||||||
swarm_queue_client::builder::ARCHETYPES_SUBJECT.to_owned(),
|
|
||||||
// An operator's claude login step, sent as a request the same way.
|
// An operator's claude login step, sent as a request the same way.
|
||||||
swarm_queue_client::claude_login::SUBJECT.to_owned(),
|
swarm_queue_client::claude_login::SUBJECT.to_owned(),
|
||||||
// The wanted-state buckets — one per hive, all created and written
|
// The wanted-state buckets — one per hive, all created and written
|
||||||
|
|
@ -1467,7 +1463,7 @@ mod tests {
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn the_builder_hears_its_subject_and_inbox_only_and_answers_by_reply() {
|
fn the_builder_hears_its_subject_and_inbox_only_and_answers_by_reply() {
|
||||||
use swarm_queue_client::builder::{ARCHETYPES_SUBJECT, REPLY_DEADLINE, SUBJECT};
|
use swarm_queue_client::builder::{REPLY_DEADLINE, SUBJECT};
|
||||||
use swarm_queue_client::client_inbox_prefix;
|
use swarm_queue_client::client_inbox_prefix;
|
||||||
|
|
||||||
let p = policy()
|
let p = policy()
|
||||||
|
|
@ -1476,7 +1472,6 @@ mod tests {
|
||||||
.expect("a configured builder is admitted");
|
.expect("a configured builder is admitted");
|
||||||
let inbox = client_inbox_prefix(BUILDER);
|
let inbox = client_inbox_prefix(BUILDER);
|
||||||
assert!(may_subscribe(&p, SUBJECT));
|
assert!(may_subscribe(&p, SUBJECT));
|
||||||
assert!(may_subscribe(&p, ARCHETYPES_SUBJECT));
|
|
||||||
assert!(may_subscribe(&p, &reply(&inbox)));
|
assert!(may_subscribe(&p, &reply(&inbox)));
|
||||||
for refused in [
|
for refused in [
|
||||||
">".to_owned(),
|
">".to_owned(),
|
||||||
|
|
@ -1497,12 +1492,11 @@ mod tests {
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn only_the_reader_may_send_a_build() {
|
fn only_the_reader_may_send_a_build() {
|
||||||
use swarm_queue_client::builder::{ARCHETYPES_SUBJECT, SUBJECT};
|
use swarm_queue_client::builder::SUBJECT;
|
||||||
|
|
||||||
let p = with_classifier().with_builders(vec![BUILDER.to_owned()]);
|
let p = with_classifier().with_builders(vec![BUILDER.to_owned()]);
|
||||||
let reader = p.permissions("swarm-controller").expect("the reader");
|
let reader = p.permissions("swarm-controller").expect("the reader");
|
||||||
for subject in [SUBJECT, ARCHETYPES_SUBJECT] {
|
assert!(reader.publish.iter().any(|g| covers(g, SUBJECT)));
|
||||||
assert!(reader.publish.iter().any(|g| covers(g, subject)));
|
|
||||||
for (who, grant) in [
|
for (who, grant) in [
|
||||||
("a hive", p.permissions("hive-alpha").expect("a hive")),
|
("a hive", p.permissions("hive-alpha").expect("a hive")),
|
||||||
("the builder", p.permissions(BUILDER).expect("admitted")),
|
("the builder", p.permissions(BUILDER).expect("admitted")),
|
||||||
|
|
@ -1512,13 +1506,12 @@ mod tests {
|
||||||
),
|
),
|
||||||
] {
|
] {
|
||||||
assert!(
|
assert!(
|
||||||
!grant.publish.iter().any(|g| covers(g, subject)),
|
!grant.publish.iter().any(|g| covers(g, SUBJECT)),
|
||||||
"{who} may send to {subject}: {:?}",
|
"{who} may send a build: {:?}",
|
||||||
grant.publish
|
grant.publish
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
|
||||||
|
|
||||||
const CLAUDE_LOGIN: &str = "swarm-claude-login";
|
const CLAUDE_LOGIN: &str = "swarm-claude-login";
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,117 +0,0 @@
|
||||||
//! The agent archetype KV bucket: one key per archetype source flake, holding
|
|
||||||
//! the `meta` of every archetype that flake's `hyperhiveArchetypes` output
|
|
||||||
//! declares, as swarm-controller last checked it.
|
|
||||||
//!
|
|
||||||
//! **swarm-controller is the only writer and the only reader.** The swarm
|
|
||||||
//! builder evaluates the source and answers on
|
|
||||||
//! [`crate::builder::ARCHETYPES_SUBJECT`]; the controller checks each
|
|
||||||
//! archetype's settings the way `POST /api/agents` checks them and writes the
|
|
||||||
//! result here, so the create screen lists archetypes from a value that a
|
|
||||||
//! create would accept.
|
|
||||||
|
|
||||||
use std::collections::BTreeMap;
|
|
||||||
|
|
||||||
use crate::agent_inference::AgentInference;
|
|
||||||
use crate::agent_limits::AgentCaps;
|
|
||||||
|
|
||||||
/// The KV bucket archetype sources are published into, keyed by source name.
|
|
||||||
pub const BUCKET: &str = "agent-archetypes";
|
|
||||||
|
|
||||||
/// The key of the archetypes the hyperhive flake itself ships, the one source
|
|
||||||
/// there is so far.
|
|
||||||
pub const BUILTIN_SOURCE: &str = "hyperhive";
|
|
||||||
|
|
||||||
/// One source's archetypes: the value under `BUCKET`/`<source>`.
|
|
||||||
#[derive(Debug, Clone, Default, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
|
|
||||||
pub struct SourceRecord {
|
|
||||||
/// The flake reference the builder evaluated.
|
|
||||||
pub source: String,
|
|
||||||
/// Every archetype whose `meta` passed the controller's checks, by name.
|
|
||||||
#[serde(default)]
|
|
||||||
pub archetypes: BTreeMap<String, ArchetypeMeta>,
|
|
||||||
/// Every archetype whose `meta` did not, by name, with the reason.
|
|
||||||
#[serde(default, skip_serializing_if = "BTreeMap::is_empty")]
|
|
||||||
pub refused: BTreeMap<String, String>,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// One archetype's `meta`, as checked: the settings a create takes, in the
|
|
||||||
/// shapes the per-agent buckets store them in.
|
|
||||||
#[derive(Debug, Clone, Default, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
|
|
||||||
pub struct ArchetypeMeta {
|
|
||||||
pub description: String,
|
|
||||||
/// Permission names, `snake_case`. Absent leaves the default set.
|
|
||||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
|
||||||
pub permissions: Option<Vec<String>>,
|
|
||||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
|
||||||
pub limits: Option<AgentCaps>,
|
|
||||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
|
||||||
pub inference: Option<AgentInference>,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Open the archetype bucket for writing, creating it if nothing has yet.
|
|
||||||
///
|
|
||||||
/// **Controller-side only.** `history: 1`: a reader wants the current value.
|
|
||||||
#[cfg(feature = "kv")]
|
|
||||||
pub async fn open_or_create(
|
|
||||||
client: &async_nats::Client,
|
|
||||||
) -> Result<async_nats::jetstream::kv::Store, crate::Error> {
|
|
||||||
let js = async_nats::jetstream::new(client.clone());
|
|
||||||
crate::kv::open_or_create(
|
|
||||||
&js,
|
|
||||||
BUCKET,
|
|
||||||
"agent-archetypes",
|
|
||||||
"Agent archetypes the swarm builder evaluated, one key per source flake".to_owned(),
|
|
||||||
)
|
|
||||||
.await
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Open the archetype bucket for reading, or `None` when nothing has created
|
|
||||||
/// it yet, which reads the same as no archetypes.
|
|
||||||
///
|
|
||||||
/// # Errors
|
|
||||||
///
|
|
||||||
/// [`crate::Error::OpenBucket`] for every failure other than the bucket not
|
|
||||||
/// existing.
|
|
||||||
#[cfg(feature = "kv")]
|
|
||||||
pub async fn open(
|
|
||||||
client: &async_nats::Client,
|
|
||||||
) -> Result<Option<async_nats::jetstream::kv::Store>, crate::Error> {
|
|
||||||
let js = async_nats::jetstream::new(client.clone());
|
|
||||||
match js.get_key_value(BUCKET).await {
|
|
||||||
Ok(store) => Ok(Some(store)),
|
|
||||||
Err(e) if crate::wanted::is_bucket_not_found(&e) => Ok(None),
|
|
||||||
Err(source) => Err(crate::Error::OpenBucket {
|
|
||||||
bucket: BUCKET.to_owned(),
|
|
||||||
source,
|
|
||||||
}),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
use super::*;
|
|
||||||
|
|
||||||
/// An unset setting is left out on the wire, and so is an empty refusal
|
|
||||||
/// list.
|
|
||||||
#[test]
|
|
||||||
fn unset_fields_are_absent() {
|
|
||||||
let mut archetypes = BTreeMap::new();
|
|
||||||
archetypes.insert(
|
|
||||||
"reviewer".to_owned(),
|
|
||||||
ArchetypeMeta {
|
|
||||||
description: "reads".to_owned(),
|
|
||||||
permissions: Some(vec!["messaging".to_owned()]),
|
|
||||||
..ArchetypeMeta::default()
|
|
||||||
},
|
|
||||||
);
|
|
||||||
let record = SourceRecord {
|
|
||||||
source: "/nix/store/h".to_owned(),
|
|
||||||
archetypes,
|
|
||||||
refused: BTreeMap::new(),
|
|
||||||
};
|
|
||||||
assert_eq!(
|
|
||||||
serde_json::to_string(&record).unwrap(),
|
|
||||||
r#"{"source":"/nix/store/h","archetypes":{"reviewer":{"description":"reads","permissions":["messaging"]}}}"#
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Reference in a new issue