Watch
0
0
Fork
You've already forked hyperhive
0

Compare commits

...
18 changed files with 1188 additions and 50 deletions

View file

@ -94,6 +94,11 @@ below. A hive without them reads
| `never_reported` | this hive has never reported at all — normally a deployment that hasn't happened, not an outage |
| `unknown` | something is publishing under a name that's not in `swarm.hives` — a typo in the roster, or a hive removed from the roster but still running |
The snapshot is the hive's `/health/ready` document plus `capabilities`, a list
of names for what the hive's release can do. Two exist: `tolerant-permissions`, and `no-config-checkout`, which
[turning an archetype agent into a custom one](archetypes.md#turn-into-custom)
needs. A hive on an older release lists none.
Every row also carries `last_seen_unix` and `age_seconds` if you want to
apply your own threshold. The timestamp is the one the queue recorded on
arrival, not one the hive put in its own payload. A roster hive's row

View file

@ -9,7 +9,9 @@ field before you create the agent.
An agent created from an archetype has no `agent-configs/<name>` repo of its
own. The swarm's `internal/meta` flake builds it from the archetype's module,
so every agent of that archetype runs the same configuration. To change what it runs, change the
archetype. Picking "none" creates a custom agent, with its own config repo.
archetype. Picking "none" creates a custom agent, with its own config repo. The
swarm can also turn an archetype agent into a custom one later
([below](#turn-into-custom)).
## The built-in archetypes
@ -139,3 +141,57 @@ The module comes from the `hyperhive` input that `internal/meta` pins, not
from the hive's own. When that pin predates the archetype, the agent's build
fails with `this hive's hyperhive input has no archetype <name>; update that
input`. Update the `hyperhive` pin in `internal/meta` to fix it.
## Turn into custom
An archetype agent can become a custom agent with its own
`agent-configs/<name>` repo, either from the agent's overview in the swarm UI
(the "turn into custom" button asks first) or with `POST /api/agents/{name}/custom`.
The agent keeps its state, permissions, caps, inference settings and accounts.
Its configuration stays the same, except that it no longer has `HIVE_ARCHETYPE`
in its environment. Keeping the config current is then up to the repo's owner.
Nothing here undoes it.
The controller refuses with 409 an agent that no hive or more than one hive
places, and one whose hive doesn't list the
`no-config-checkout` capability (see below). Otherwise it queues a job graph:
1. `CreateRepo`, `CreateForgeUser` and `AddRepoMember`: the repo and the
agent's access to the repo, as for any custom agent. Each is a no-op for what
exists.
2. `SeedArchetypeConfig` commits two files to the repo. It copies nothing from
the archetype:
- `flake.nix` declares the archetype's source as an input, at the URL
`internal/meta` holds for that source today (a pinned rev, which is the input's only pin: the repo has no lock, and a source meta doesn't pin to a commit fails the node), and imports
`inputs.<source>.hyperhiveArchetypes.<name>.module` next to `./agent.nix`;
- `agent.nix` is an empty module for the owner's edits.
The node leaves a repo whose `flake.nix` already is this file alone, whatever
URL it pins. Any other `flake.nix` fails the node.
3. `DropArchetype` deletes the agent's `agent-archetype` key, only after the
seed and the member grant succeeded. A failure before it leaves the agent
archetype-built and working, and you can send the same request again.
4. `UpdateMeta`, then the build and the deploy, as for a merged config PR.
If the build fails, the agent is custom and has no key. Sending the request
again then queues only this step, as the earlier ones have run.
When the source is `hyperhive`, the meta flake sets
`agent-<name>.inputs.hyperhive.follows = "hyperhive"` for an agent whose
`flake.nix` or `flake.lock` declares a `hyperhive` input, so the lock holds one
hyperhive, and the module comes from the pin `internal/meta` holds. Any other
source keeps the pinned URL from its seeded `flake.nix`, and the meta lock
records that rev. The `follows` applies to every custom agent whose `flake.nix`
declares a `hyperhive` input, not only converted ones. A `hyperhive` node left
in a lock doesn't count.
### Which hives may convert
A hive that keeps a checkout of a custom agent's config would create one for
the converted agent, under the container's running mount. A hive that lists
`no-config-checkout` under `capabilities` in the status it publishes (the
document `GET /api/hives/status` serves as `snapshot`) keeps none: it builds a
custom agent from `internal/meta` alone, and the agent clones its own repo from
the forge. The controller reads the agent's hive's latest status and refuses
unless it's `fresh` and lists the capability; an older hive lists none. Deploy
the release to every hive that places archetype agents before converting theirs.

View file

@ -43,6 +43,17 @@ export interface AgentRow {
// no header since swarm-controller started, or an agent that does not
// send the flag — not known, so not "needs login".
needs_login: boolean | null;
// The archetype the agent is built from; `null` for a custom agent.
archetype: { source: string; name: string } | null;
}
// "Turn into custom" is offered to an archetype agent whose hive is known and
// that has not been destroyed: the controller places the conversion on that
// hive.
export function canTurnIntoCustom(
row: Pick<AgentRow, "archetype" | "hive" | "wanted">,
): boolean {
return !!row.archetype && !!row.hive && row.wanted !== "destroyed";
}
// Only an agent that said so gets the badge; `null` shows nothing.

View file

@ -11,6 +11,7 @@
// change or a link made here shows wherever else the same row is drawn.
import { ApiErrorPanel } from "@hive/shared/api-error-panel.js";
import { Badge } from "@hive/shared/badge.js";
import { Button } from "../../ui/button/Button.js";
import { RelativeTime } from "../../ui/relative-time/RelativeTime.js";
import type { AgentActions } from "../roster/AgentActions.js";
import { AgentInference } from "./AgentInference.js";
@ -20,7 +21,7 @@ import { AgentPermissions } from "./AgentPermissions.js";
import { AllowedDomains } from "./AllowedDomains.js";
import { AgentStateRow } from "./AgentStateRow.js";
import { AgentTermPreview } from "../terminal/AgentTermPreview.js";
import { FRESHNESS, type AgentRow } from "../AgentTypes.js";
import { canTurnIntoCustom, FRESHNESS, type AgentRow } from "../AgentTypes.js";
import { LinkAccountMenu } from "./LinkAccountMenu.js";
import { LinkedAccounts } from "./LinkedAccounts.js";
import { LazyDisclosure } from "../../ui/disclosure/LazyDisclosure.js";
@ -90,7 +91,13 @@ function HeaderRows({
);
}
function OverviewRows({ row }: { row: AgentRow }) {
function OverviewRows({
row,
actions,
}: {
row: AgentRow;
actions: AgentActions;
}) {
return (
<>
<dt>state</dt>
@ -122,6 +129,25 @@ function OverviewRows({ row }: { row: AgentRow }) {
"—"
)}
</dd>
{row.archetype ? (
<>
<dt>archetype</dt>
<dd>
{row.archetype.source}/{row.archetype.name}
{canTurnIntoCustom(row) ? (
<>
{" "}
<Button
disabled={actions.pendingAgents.has(row.name)}
onClick={() => actions.requestCustom(row)}
>
turn into custom
</Button>
</>
) : null}
</dd>
</>
) : null}
</>
);
}
@ -211,7 +237,7 @@ function SectionBody({
case "overview":
return (
<dl class="ui-agent-detail-fields">
<OverviewRows row={row} />
<OverviewRows row={row} actions={actions} />
</dl>
);
case "settings":
@ -272,7 +298,7 @@ export function AgentDetail({
<div class="ui-agent-detail">
<dl class="ui-agent-detail-fields">
<HeaderRows row={row} actions={actions} />
<OverviewRows row={row} />
<OverviewRows row={row} actions={actions} />
<SettingsRows agent={row.name} />
<AccountsRows row={row} actions={actions} />
</dl>

View file

@ -71,6 +71,7 @@ export interface AgentActions {
selectOffline: (row: AgentRow) => void;
selectPaused: (row: AgentRow) => void;
requestDestroy: (row: AgentRow) => void;
requestCustom: (row: AgentRow) => void;
openLink: (row: AgentRow, kind: LinkKind) => void;
/** The confirm and link dialogs; render once, anywhere. */
dialogs: ComponentChildren;
@ -94,6 +95,7 @@ export function useAgentActions(
row: AgentRow;
state: "offline" | "paused";
} | null>(null);
const [customTarget, setCustomTarget] = useState<AgentRow | null>(null);
const [matrixTarget, setMatrixTarget] = useState<AgentRow | null>(null);
const [forgeTarget, setForgeTarget] = useState<AgentRow | null>(null);
const [githubTarget, setGithubTarget] = useState<AgentRow | null>(null);
@ -172,6 +174,38 @@ export function useAgentActions(
await declareState(row, "destroyed");
}
// The conversion is queued as a job graph; the roster shows the agent as
// custom once it has run, so a success needs no handling here.
async function turnIntoCustom(row: AgentRow) {
setCustomTarget(null);
setPendingAgents((prev) => new Set(prev).add(row.name));
setActionErrors((prev) => {
const next = new Map(prev);
next.delete(row.name);
return next;
});
try {
const res = await fetch(
`/api/agents/${encodeURIComponent(row.name)}/custom`,
{ method: "POST" },
);
if (!res.ok) {
const problem = await readApiError(res);
setActionErrors((prev) => new Map(prev).set(row.name, problem));
}
} catch (e: unknown) {
setActionErrors((prev) =>
new Map(prev).set(row.name, { detail: String(e) }),
);
} finally {
setPendingAgents((prev) => {
const next = new Set(prev);
next.delete(row.name);
return next;
});
}
}
const dialogs = (
<>
<Dialog
@ -248,6 +282,22 @@ export function useAgentActions(
? CONFIRM_COPY[confirmTarget.state].message(confirmTarget.row.name)
: null}
</ConfirmDialog>
<ConfirmDialog
open={customTarget !== null}
label="turn into custom"
onCancel={() => setCustomTarget(null)}
onConfirm={() => customTarget && void turnIntoCustom(customTarget)}
confirmLabel="turn into custom"
>
{customTarget ? (
<p>
Turn <strong>{customTarget.name}</strong> into a custom agent? It
gets its own config repo, which imports the archetype it runs today,
and is rebuilt from that repo. Keeping the config current is then up
to the repo's owner. This is not reversible from here.
</p>
) : null}
</ConfirmDialog>
<ConfirmDialog
open={destroyTarget !== null}
label="destroy agent"
@ -276,6 +326,7 @@ export function useAgentActions(
selectOffline: (row) => setConfirmTarget({ row, state: "offline" }),
selectPaused: (row) => setConfirmTarget({ row, state: "paused" }),
requestDestroy: setDestroyTarget,
requestCustom: setCustomTarget,
openLink: (row, kind) => {
if (kind === "matrix") setMatrixTarget(row);
else if (kind === "forge") setForgeTarget(row);

View file

@ -3,7 +3,10 @@
// imported as it is.
import assert from "node:assert/strict";
import { test } from "node:test";
import { showsLoginNeeded } from "../src/agent/AgentTypes.ts";
import {
canTurnIntoCustom,
showsLoginNeeded,
} from "../src/agent/AgentTypes.ts";
test("an agent that says it needs a login gets the badge", () => {
assert.equal(showsLoginNeeded({ needs_login: true }), true);
@ -17,3 +20,25 @@ test("an agent that has not said gets none", () => {
assert.equal(showsLoginNeeded({ needs_login: null }), false);
assert.equal(showsLoginNeeded({}), false);
});
test("turn into custom is offered to a placed archetype agent only", () => {
const archetype = { source: "hyperhive", name: "reviewer" };
assert.equal(
canTurnIntoCustom({ archetype, hive: "alpha", wanted: "up" }),
true,
);
// A custom agent has no archetype to turn from.
assert.equal(
canTurnIntoCustom({ archetype: null, hive: "alpha", wanted: "up" }),
false,
);
// The controller places the conversion on a hive, so one is needed.
assert.equal(
canTurnIntoCustom({ archetype, hive: null, wanted: "up" }),
false,
);
assert.equal(
canTurnIntoCustom({ archetype, hive: "alpha", wanted: "destroyed" }),
false,
);
});

View file

@ -407,7 +407,10 @@ struct Snapshot {
fn snapshot() -> Snapshot {
Snapshot {
readiness: crate::warnings::readiness(),
capabilities: &[swarm_wire::hive_status::TOLERANT_PERMISSIONS],
capabilities: &[
swarm_wire::hive_status::TOLERANT_PERMISSIONS,
swarm_wire::hive_status::NO_CONFIG_CHECKOUT,
],
}
}
@ -449,12 +452,16 @@ mod snapshot_tests {
/// The controller reads the capability out of the published bytes with
/// the wire type, and the readiness keeps its own top-level fields.
#[test]
fn the_snapshot_advertises_tolerant_permissions_beside_the_readiness() {
fn the_snapshot_advertises_its_capabilities_beside_the_readiness() {
let value = serde_json::to_value(snapshot()).unwrap();
assert!(
swarm_wire::hive_status::Capabilities::of_snapshot(&value)
.has(swarm_wire::hive_status::TOLERANT_PERMISSIONS)
);
assert!(
swarm_wire::hive_status::Capabilities::of_snapshot(&value)
.has(swarm_wire::hive_status::NO_CONFIG_CHECKOUT)
);
assert!(value.get("status").is_some() && value.get("warnings").is_some());
}
}

View file

@ -43,12 +43,12 @@ pub fn pinned_git_url(forge_git_url: &str, rev: Option<&str>) -> Result<String>
Ok(format!("git+{forge_git_url}?rev={rev}"))
}
/// Canonical inputs meta knows how to dedup. An agent that declares one
/// of these as a top-level input in its own `flake.nix` will get a
/// Canonical inputs meta knows how to dedup. An agent whose `flake.lock` or
/// `flake.nix` declares one of these as a top-level input gets a
/// `follows = "<name>"` line emitted in meta — collapsing the
/// otherwise-separate-but-identical `nixpkgs_N` nodes into a single
/// meta-level reference.
pub const CANONICAL_INPUTS: &[&str] = &["nixpkgs"];
/// otherwise-separate-but-identical `nixpkgs_N` and `hyperhive_N` nodes into
/// a single meta-level reference.
pub const CANONICAL_INPUTS: &[&str] = &["nixpkgs", "hyperhive"];
/// Env vars the meta flake sets in every agent container, from the
/// `(name, value)` pairs [`render_flake`] is given. A caller passes only the
@ -143,6 +143,32 @@ pub fn push_forwarded_var_options(out: &mut String, vars: &[(&'static str, Strin
}
}
/// The subset of `CANONICAL_INPUTS` that an agent config declares, in its
/// `flake.lock` text `lock` or its `flake.nix` text `flake_nix`. A repo that
/// was only just seeded has no lock yet, so its `flake.nix` counts too: a line
/// starting `<input>.url` or `inputs.<input>.url`.
///
/// A `hyperhive` node in a lock does not count: it can be a leftover from an
/// input the config dropped from its `flake.nix`, and a `follows` for an input
/// the flake does not declare makes nix warn.
#[must_use]
pub fn canonical_inputs_declared(lock: Option<&str>, flake_nix: Option<&str>) -> Vec<&'static str> {
let locked = lock.map(canonical_inputs_in_lock).unwrap_or_default();
CANONICAL_INPUTS
.iter()
.copied()
.filter(|canon| {
(*canon != "hyperhive" && locked.contains(canon))
|| flake_nix.is_some_and(|text| {
text.lines().map(str::trim_start).any(|l| {
l.starts_with(&format!("{canon}.url"))
|| l.starts_with(&format!("inputs.{canon}.url"))
})
})
})
.collect()
}
/// The subset of `CANONICAL_INPUTS` that the `flake.lock` text `raw` declares
/// as direct (root-level) inputs. Empty when the lock is unparsable — safe
/// degradation, the worst case is no dedup for that agent.
@ -460,17 +486,26 @@ pub fn is_rev(rev: &str) -> bool {
rev.len() == 40 && rev.bytes().all(|b| matches!(b, b'0'..=b'9' | b'a'..=b'f'))
}
/// The URL of `input` in a flake [`render_flake`] rendered: the text of its
/// `<input>.url = "<url>";` line. `None` when the flake has no such input.
#[must_use]
pub fn input_url<'a>(flake_nix: &'a str, input: &str) -> Option<&'a str> {
let prefix = format!(" {input}.url = \"");
Some(
flake_nix
.lines()
.find_map(|l| l.strip_prefix(prefix.as_str()))?
.split_once('"')?
.0,
)
}
/// The commit `input` is pinned to in a flake [`render_flake`] rendered:
/// the `<rev>` of its `<input>.url = "git+<url>?rev=<rev>";` line. `None` when
/// the flake has no such input, or its rev is not a commit id.
#[must_use]
pub fn pinned_rev<'a>(flake_nix: &'a str, input: &str) -> Option<&'a str> {
let prefix = format!(" {input}.url = \"");
let url = flake_nix
.lines()
.find_map(|l| l.strip_prefix(prefix.as_str()))?
.split_once('"')?
.0;
let url = input_url(flake_nix, input)?;
let (_, rev) = url.split_once("?rev=")?;
let rev = rev.split('&').next()?;
is_rev(rev).then_some(rev)
@ -695,4 +730,51 @@ mod tests {
assert!(!builds_agent_at(&flake, "gus", None, Some(ARGUS)));
assert!(!builds_agent_at(&flake, "v", Some(&reviewer()), None));
}
/// An agent that declares `hyperhive` follows meta's own pin, so the
/// lock holds one hyperhive; one that declares nothing gets no line (the
/// control that the line is the declared input's, not every agent's).
#[test]
fn a_declared_hyperhive_input_follows_meta_s_own() {
let url = |repo: &str, rev: &str| {
pinned_git_url(&format!("https://f.t/{repo}.git"), Some(rev)).unwrap()
};
let agents = |canonical_inputs: Vec<&'static str>| {
render_flake(
&url("hyperhive/hyperhive", HYPERHIVE),
&[spec(
"argus",
None,
Some(AgentPin {
url: url("agent-configs/argus", ARGUS),
canonical_inputs,
}),
)],
&[],
false,
)
.unwrap()
};
let line = "agent-argus.inputs.hyperhive.follows = \"hyperhive\";";
assert!(agents(vec!["hyperhive"]).contains(line));
assert!(!agents(vec![]).contains(line));
}
/// A seeded repo has only a `flake.nix`; its input counts, from either
/// spelling, and a lock-declared input still does.
#[test]
fn canonical_inputs_are_read_from_the_flake_text_and_the_lock() {
let flake = "{\n inputs = {\n hyperhive.url = \"git+x\";\n };\n}\n";
assert_eq!(canonical_inputs_declared(None, Some(flake)), ["hyperhive"]);
let dotted = "{\n inputs.nixpkgs.url = \"x\";\n}\n";
assert_eq!(canonical_inputs_declared(None, Some(dotted)), ["nixpkgs"]);
let lock = r#"{"root":"root","nodes":{"root":{"inputs":{"nixpkgs":"n"}}}}"#;
assert_eq!(canonical_inputs_declared(Some(lock), None), ["nixpkgs"]);
assert!(canonical_inputs_declared(None, Some("{ inputs = { }; }")).is_empty());
// A leftover `hyperhive` node in a lock declares nothing.
let stale = r#"{"root":"root","nodes":{"root":{"inputs":{"hyperhive":"hyperhive"}}}}"#;
assert!(canonical_inputs_declared(Some(stale), Some("{ inputs = { }; }")).is_empty());
// Another archetype source is not canonical: meta does not follow it.
let other = "{\n inputs = {\n extra.url = \"git+x\";\n };\n}\n";
assert!(canonical_inputs_declared(None, Some(other)).is_empty());
}
}

View file

@ -114,6 +114,22 @@ pub(crate) async fn restore(client: &async_nats::Client, agent: &str, replaced:
}
}
/// The archetype of each agent in `rows` that has one.
pub(crate) async fn of_agents(
client: &async_nats::Client,
rows: &[crate::agent_status::AgentStatusRow],
) -> anyhow::Result<std::collections::HashMap<String, ArchetypeRef>> {
let store = agent_archetype::open_or_create(client).await?;
let mut found = std::collections::HashMap::new();
for row in rows {
let (archetype, _, _) = read_or_default::<Option<ArchetypeRef>>(&store, &row.name).await?;
if let Some(archetype) = archetype {
found.insert(row.name.clone(), archetype);
}
}
Ok(found)
}
/// Delete a destroyed agent's key, so the next agent of that name is not
/// built from this one's archetype. A failure is logged and does not stop the
/// destroy, as in `agent_permissions::clear_on_destroy`.

View file

@ -0,0 +1,611 @@
//! `POST /api/agents/{name}/custom`: turn an archetype agent into a custom
//! one, with its own `agent-configs/<name>` repo.
//!
//! The repo's `flake.nix` declares the archetype's source flake as an input, at
//! the URL `internal/meta` holds for that source, and imports the archetype's
//! module from it. Nothing is copied: any source `internal/meta` has a pinned
//! input for works. The URL's `rev` is the only pin, since the seeded repo has
//! no lock. The meta flake points the agent's `hyperhive` input at its own, so
//! the lock holds one hyperhive. From then on the repo's owner keeps the config
//! current. The job graph is [`crate::jobq::declare_custom_job`]: repo and seed
//! first, the archetype key deleted only after the seed succeeded, then the
//! meta update and the deploy. A failure before the delete leaves the agent
//! archetype-built and working, and the same request can be sent again. A
//! failure after it leaves the agent custom, and the same request then only
//! re-queues the meta update and the deploy.
use axum::Json;
use axum::extract::{Path, State};
use axum::http::StatusCode;
use problem_details::ProblemDetails;
use serde::Serialize;
use swarm_queue_client::agent_archetype::ArchetypeRef;
use utoipa::ToSchema;
use crate::routes::agents::{claiming_hives, read_declarations};
use crate::routes::{AppState, error_problem};
use swarm_wire::hive_status::{Capabilities, NO_CONFIG_CHECKOUT};
use crate::status::{Freshness, HiveStatus};
use crate::{forge, meta_flake, schedules};
/// Answer of a queued conversion.
#[derive(Serialize, ToSchema)]
pub struct ConvertResponse {
/// A node of the queued graph; `/api/jobq/graph` serves the rest.
node_id: u64,
}
/// Turn an archetype agent into a custom agent.
#[utoipa::path(
post,
path = "/api/agents/{name}/custom",
params(("name" = String, Path, description = "archetype agent to turn into a custom one")),
responses(
(status = 200, description = "the conversion is queued", body = ConvertResponse),
(status = 400, description = "the agent name is not shaped like an identifier (problem+json)", body = String),
(status = 409, description = "the agent is placed on no hive or on several, or its hive has not reported the capability a converted agent's rebuild needs (problem+json)", body = String),
(status = 503, description = "no swarm queue is configured here or it is not connected (problem+json)", body = String),
(status = 500, description = "the queue or the job graph failed (problem+json)", body = String),
),
tag = "agents"
)]
pub(crate) async fn post_agent_custom(
State(state): State<AppState>,
Path(agent): Path<String>,
) -> Result<Json<ConvertResponse>, ProblemDetails> {
let agent = hive_types::Ident::parse(&agent)
.map_err(|reason| error_problem(StatusCode::BAD_REQUEST, reason))?
.into_string();
let client = schedules::queue(&state).map_err(schedules::refused)?;
let internal =
|e: anyhow::Error| error_problem(StatusCode::INTERNAL_SERVER_ERROR, &format!("{e:#}"));
let store = swarm_queue_client::agent_archetype::open_or_create(&client)
.await
.map_err(|e| internal(e.into()))?;
let (archetype, _, _) = schedules::read_or_default::<Option<ArchetypeRef>>(&store, &agent)
.await
.map_err(internal)?;
let declared = read_declarations(&state, state.hives.iter())
.await
.map_err(|(hive, e)| {
internal(e.context(format!("read where hive {hive} places agents")))
})?;
let Some(status) = state.status.as_ref() else {
return Err(error_problem(
StatusCode::SERVICE_UNAVAILABLE,
"no swarm queue is configured on this host",
));
};
let rows = status
.view(&state.hives, std::time::SystemTime::now())
.await
.map_err(internal)?;
let (archetype, hive) = plan(&agent, archetype, &declared, &rows)
.map_err(|why| error_problem(StatusCode::CONFLICT, &why))?;
let queued = {
let mut sched = state
.jobq
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
sched.insert_job(None, |b| {
crate::jobq::declare_custom_job(b, &agent, &hive, archetype.as_ref())
})
};
let ids = queued.map_err(|e| internal(anyhow::anyhow!("{e}")))?;
let [id] = ids[..] else {
unreachable!("exactly one handle was asked for");
};
tracing::info!(%agent, %hive, ?archetype, "turn into custom queued");
Ok(Json(ConvertResponse { node_id: id.get() }))
}
/// What the request does, or why it is refused: `archetype` is the agent's
/// stored one, and `None` for a custom agent, which a request only re-deploys
/// (the retry of a conversion whose build failed after the key was deleted).
/// The agent's hive is the one hive that places it, and must pass
/// [`offers_no_config_checkout`].
fn plan(
agent: &str,
archetype: Option<ArchetypeRef>,
declared: &[(String, swarm_wire::wanted::HiveWanted)],
rows: &[HiveStatus],
) -> Result<(Option<ArchetypeRef>, String), String> {
let hive = match claiming_hives(agent, declared).as_slice() {
[hive] => hive.clone(),
other => {
return Err(format!(
"agent {agent:?} is placed on {} hives, so its hive is unknown",
other.len()
));
}
};
offers_no_config_checkout(rows, &hive)?;
Ok((archetype, hive))
}
/// Whether `hive`'s latest status advertises [`NO_CONFIG_CHECKOUT`]. An older
/// hive keeps a checkout of a custom agent's config and would create it for a
/// converted agent under the running container's mount, so a conversion waits
/// for this.
///
/// # Errors
/// The reason, naming the hive: it has no recent status, or its status does
/// not list it. An older hive lists no capabilities at all.
fn offers_no_config_checkout(rows: &[HiveStatus], hive: &str) -> Result<(), String> {
let capability = NO_CONFIG_CHECKOUT;
let fresh = rows
.iter()
.find(|row| row.name == hive && row.freshness == Freshness::Fresh);
let Some(snapshot) = fresh.and_then(|row| row.snapshot.as_ref()) else {
return Err(format!(
"hive {hive:?} has no recent status, so whether it can rebuild a converted agent is \
unknown; retry once it reports"
));
};
if Capabilities::of_snapshot(snapshot).has(capability) {
return Ok(());
}
Err(format!(
"hive {hive:?} does not advertise the {capability:?} capability, so a converted agent \
would get a config checkout on it; deploy this release to hive {hive:?} first"
))
}
/// What [`seed`] reads and writes on the forge: a fake in the tests.
pub(crate) trait SeedForge {
/// `internal/meta`'s `flake.nix` at its head.
fn meta_flake(&self) -> impl Future<Output = anyhow::Result<Option<String>>> + Send;
/// `path` on the default branch of `agent`'s config repo; `None` when the
/// repo is empty or has no such file.
fn config_file(
&self,
agent: &str,
path: &str,
) -> impl Future<Output = anyhow::Result<Option<String>>> + Send;
/// Commit `files` to the default branch of `agent`'s config repo.
fn create_files(
&self,
agent: &str,
files: &[(&str, String)],
) -> impl Future<Output = anyhow::Result<()>> + Send;
}
impl SeedForge for forge::Client {
async fn meta_flake(&self) -> anyhow::Result<Option<String>> {
let head = self
.default_branch_files(
forge::objects::SHARED_ORG,
meta_flake::META_REPO,
&["flake.nix"],
)
.await?;
Ok(head
.and_then(|h| h.files.into_iter().next())
.map(|f| String::from_utf8_lossy(&f.bytes).into_owned()))
}
async fn config_file(&self, agent: &str, path: &str) -> anyhow::Result<Option<String>> {
let head = self
.default_branch_files(forge::CONFIG_ORG, agent, &[path])
.await?;
Ok(head
.and_then(|h| h.files.into_iter().next())
.map(|f| String::from_utf8_lossy(&f.bytes).into_owned()))
}
async fn create_files(&self, agent: &str, files: &[(&str, String)]) -> anyhow::Result<()> {
self.create_config_files(agent, files, "swarm-controller: turn into custom")
.await
}
}
/// A nix identifier, so the names can be written as attribute names unquoted.
fn is_nix_ident(name: &str) -> bool {
let mut chars = name.chars();
chars
.next()
.is_some_and(|c| c.is_ascii_lowercase() || c == '_')
&& chars.all(|c| c.is_ascii_lowercase() || c.is_ascii_digit() || c == '_' || c == '-')
}
/// The `flake.nix` of a converted agent: the shape every config repo has, plus
/// the archetype's source as an input at `url`, and the archetype's module
/// among the imports. The module is imported here, from `inputs`, because a
/// module cannot take its own imports from `_module.args`.
fn converted_flake_nix(archetype: &ArchetypeRef, url: &str) -> String {
let ArchetypeRef { source, name } = archetype;
format!(
"{{\n description = \"hyperhive agent\";\n inputs = {{\n {source}.url = \"{url}\";\n }};\n outputs =\n {{ self, ... }}@inputs:\n {{\n nixosModules.default = {{\n imports = [\n ./agent.nix\n inputs.{source}.hyperhiveArchetypes.{name}.module\n ];\n _module.args.flakeInputs = builtins.removeAttrs inputs [ \"self\" ];\n }};\n }};\n}}\n"
)
}
/// The `agent.nix` of a converted agent: an empty module for the owner's edits.
fn converted_agent_nix(agent: &str, archetype: &ArchetypeRef) -> String {
let ArchetypeRef { source, name } = archetype;
format!(
"{{ config, pkgs, lib, ... }}:\n{{\n # Per-agent overrides for {agent}. flake.nix imports the {source} archetype\n # {name}; keeping that input current is up to this repo's owner.\n}}\n"
)
}
/// Seed `agent`'s config repo so it builds what the archetype built: its
/// `flake.nix` declares the archetype's source as an input at the URL
/// `internal/meta` holds for it today (a pinned rev), and imports the module.
/// The job behind `SeedArchetypeConfig`.
///
/// Safe to run again: a repo whose `flake.nix` is what this writes, whatever
/// URL it holds, is left as it is, since both files go in one commit. A
/// `flake.nix` that differs belongs to someone, and is an error.
///
/// # Errors
/// A forge failure, a name that is not a nix identifier, `internal/meta`
/// holding no input for the source or one not pinned to a commit, or a different
/// `flake.nix` already in the repo.
pub(crate) async fn seed(
forge: &impl SeedForge,
agent: &str,
archetype: &ArchetypeRef,
) -> anyhow::Result<()> {
use anyhow::Context as _;
let ArchetypeRef { source, name } = archetype;
for part in [source, name] {
anyhow::ensure!(
is_nix_ident(part),
"archetype name {part:?} is not a nix identifier"
);
}
let meta = forge
.meta_flake()
.await?
.context("internal/meta has no flake.nix to read the archetype source from")?;
let url = hive_meta_flake::input_url(&meta, source).with_context(|| {
format!("internal/meta has no input {source:?}, so the archetype source has no URL to pin")
})?;
// The seeded repo has no lock, so the URL's `rev` is the only pin the input has.
anyhow::ensure!(
hive_meta_flake::pinned_rev(&meta, source).is_some(),
"internal/meta's input {source:?} is not pinned to a commit, so the converted agent would float"
);
if let Some(have) = forge.config_file(agent, "flake.nix").await? {
let ours = hive_meta_flake::input_url(&have, source)
.is_some_and(|held| have == converted_flake_nix(archetype, held));
anyhow::ensure!(
ours,
"agent-configs/{agent} already holds a different flake.nix; the conversion does not \
overwrite it"
);
tracing::debug!(%agent, "turn into custom: config already seeded");
return Ok(());
}
forge
.create_files(
agent,
&[
("agent.nix", converted_agent_nix(agent, archetype)),
("flake.nix", converted_flake_nix(archetype, url)),
],
)
.await?;
tracing::info!(%agent, %url, "turn into custom: seeded the config repo");
Ok(())
}
/// Delete `agent`'s archetype key, so the next render and rebuild treat it as
/// custom. The job behind `DropArchetype`; deleting an absent key succeeds.
///
/// # Errors
/// The bucket cannot be opened or written.
pub(crate) async fn drop_archetype(client: &async_nats::Client, agent: &str) -> anyhow::Result<()> {
swarm_queue_client::agent_archetype::open_or_create(client)
.await?
.delete(agent)
.await?;
tracing::info!(%agent, "turn into custom: archetype key deleted");
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use std::collections::BTreeMap;
use std::sync::Mutex;
const URL: &str =
"git+http://forge/hyperhive/hyperhive.git?rev=1111111111111111111111111111111111111111";
fn reviewer() -> ArchetypeRef {
ArchetypeRef {
source: "hyperhive".to_owned(),
name: "reviewer".to_owned(),
}
}
/// An in-memory forge: `internal/meta`'s flake and the files of one config
/// repo. `fail_create` makes the next `create_files` fail once, without
/// writing.
#[derive(Default)]
struct FakeForge {
meta_flake: Option<String>,
repo: Mutex<BTreeMap<String, String>>,
fail_create: Mutex<bool>,
creates: Mutex<usize>,
}
impl FakeForge {
fn with_inputs(inputs: &[(&str, &str)]) -> Self {
use std::fmt::Write as _;
let mut lines = String::new();
for (name, url) in inputs {
let _ = writeln!(lines, " {name}.url = \"{url}\";");
}
Self {
meta_flake: Some(lines),
..Self::default()
}
}
fn pinned() -> Self {
Self::with_inputs(&[("hyperhive", URL)])
}
}
impl SeedForge for FakeForge {
async fn meta_flake(&self) -> anyhow::Result<Option<String>> {
Ok(self.meta_flake.clone())
}
async fn config_file(&self, _agent: &str, path: &str) -> anyhow::Result<Option<String>> {
Ok(self.repo.lock().unwrap().get(path).cloned())
}
async fn create_files(&self, _agent: &str, files: &[(&str, String)]) -> anyhow::Result<()> {
if std::mem::take(&mut *self.fail_create.lock().unwrap()) {
anyhow::bail!("the forge refused the commit");
}
*self.creates.lock().unwrap() += 1;
let mut repo = self.repo.lock().unwrap();
for (path, content) in files {
repo.insert((*path).to_owned(), content.clone());
}
Ok(())
}
}
/// The seeded flake declares the source as an input at meta's URL and
/// imports the module from it; nothing is copied into the repo.
#[tokio::test]
async fn the_seed_declares_the_source_as_an_input_and_imports_its_module() {
let forge = FakeForge::pinned();
seed(&forge, "atlas", &reviewer()).await.expect("seeds");
let repo = forge.repo.lock().unwrap();
let flake = &repo["flake.nix"];
assert!(
flake.contains(&format!(" hyperhive.url = \"{URL}\";")),
"{flake}"
);
assert!(
flake.contains("inputs.hyperhive.hyperhiveArchetypes.reviewer.module"),
"{flake}"
);
assert!(flake.contains("_module.args.flakeInputs"), "{flake}");
assert_eq!(
repo.keys().map(String::as_str).collect::<Vec<_>>(),
["agent.nix", "flake.nix"]
);
}
/// A source other than hyperhive takes the URL `internal/meta` holds for
/// that name, and one meta has no input for is an error.
#[tokio::test]
async fn another_source_takes_its_url_from_meta_and_an_unknown_one_fails() {
let other = ArchetypeRef {
source: "extra".to_owned(),
name: "builder".to_owned(),
};
let forge = FakeForge::with_inputs(&[
("hyperhive", URL),
(
"extra",
"git+http://x/y.git?rev=2222222222222222222222222222222222222222",
),
]);
seed(&forge, "atlas", &other).await.expect("seeds");
let flake = forge.repo.lock().unwrap()["flake.nix"].clone();
assert!(
flake.contains(" extra.url = \"git+http://x/y.git?rev=2222"),
"{flake}"
);
assert!(
flake.contains("inputs.extra.hyperhiveArchetypes.builder.module"),
"{flake}"
);
let why = format!(
"{:#}",
seed(&FakeForge::pinned(), "atlas", &other)
.await
.unwrap_err()
);
assert!(why.contains("no input \"extra\""), "{why}");
}
/// A seed that failed wrote nothing, and sending it again completes; a
/// third run, after the repo is seeded, commits nothing more — even when
/// meta's pin has moved since. The presence control: the second run did
/// write.
#[tokio::test]
async fn a_seed_that_failed_is_retried_and_a_seeded_repo_is_left_alone() {
let forge = FakeForge::pinned();
*forge.fail_create.lock().unwrap() = true;
seed(&forge, "atlas", &reviewer())
.await
.expect_err("refused");
assert!(forge.repo.lock().unwrap().is_empty(), "a failed seed wrote");
seed(&forge, "atlas", &reviewer()).await.expect("retry");
assert_eq!(*forge.creates.lock().unwrap(), 1);
let moved = FakeForge {
repo: Mutex::new(forge.repo.lock().unwrap().clone()),
..FakeForge::with_inputs(&[(
"hyperhive",
"git+http://forge/h.git?rev=3333333333333333333333333333333333333333",
)])
};
seed(&moved, "atlas", &reviewer()).await.expect("again");
assert_eq!(
*moved.creates.lock().unwrap(),
0,
"a seeded repo was committed to again"
);
}
/// Someone else's `flake.nix` is never overwritten.
#[tokio::test]
async fn a_different_flake_nix_is_an_error() {
let forge = FakeForge::pinned();
forge
.repo
.lock()
.unwrap()
.insert("flake.nix".to_owned(), "{ }".to_owned());
let why = format!(
"{:#}",
seed(&forge, "atlas", &reviewer()).await.unwrap_err()
);
assert!(why.contains("different flake.nix"), "{why}");
assert_eq!(*forge.creates.lock().unwrap(), 0);
}
#[tokio::test]
async fn a_missing_meta_flake_or_a_bad_name_is_an_error() {
let forge = FakeForge::default();
seed(&forge, "atlas", &reviewer())
.await
.expect_err("no meta flake");
let bad = ArchetypeRef {
source: "hyperhive".to_owned(),
name: "9; evil".to_owned(),
};
seed(&FakeForge::pinned(), "atlas", &bad)
.await
.expect_err("bad name");
}
fn row(name: &str, freshness: Freshness, snapshot: Option<serde_json::Value>) -> HiveStatus {
HiveStatus {
name: name.to_owned(),
domain: None,
dashboard: None,
freshness,
last_seen_unix: None,
age_seconds: None,
snapshot,
}
}
/// The rollout gate: a hive listing the capability is allowed; one that
/// lists others, lists none (an older hive's payload), has gone quiet or
/// never reported is refused, naming the hive.
#[test]
fn only_a_fresh_hive_listing_the_capability_may_convert() {
let with = serde_json::json!({"status": "ok", "warnings": [], "capabilities": ["no-config-checkout"]});
let old = serde_json::json!({"status": "ok", "warnings": []});
let other = serde_json::json!({"status": "ok", "warnings": [], "capabilities": ["x"]});
let rows = |hive: Freshness, snapshot| {
vec![
row("alpha", hive, Some(snapshot)),
row("beta", Freshness::Fresh, Some(with.clone())),
]
};
assert_eq!(
offers_no_config_checkout(&rows(Freshness::Fresh, with.clone()), "alpha"),
Ok(())
);
for refused in [old.clone(), other] {
let why =
offers_no_config_checkout(&rows(Freshness::Fresh, refused), "alpha").unwrap_err();
assert!(
why.contains("\"alpha\"") && why.contains("no-config-checkout"),
"{why}"
);
}
let stale =
offers_no_config_checkout(&rows(Freshness::Stale, with.clone()), "alpha").unwrap_err();
assert!(stale.contains("no recent status"), "{stale}");
let never = vec![row("alpha", Freshness::NeverReported, None)];
offers_no_config_checkout(&never, "alpha").unwrap_err();
offers_no_config_checkout(&[], "alpha").unwrap_err();
}
/// An old-shaped payload, without `capabilities`, still parses as the
/// opaque snapshot the status reader serves.
#[test]
fn an_old_shaped_payload_still_parses_and_lists_nothing() {
let old: serde_json::Value =
serde_json::from_str(r#"{"status":"ok","warnings":[]}"#).unwrap();
assert!(old["capabilities"].as_array().is_none());
let why = offers_no_config_checkout(&[row("alpha", Freshness::Fresh, Some(old))], "alpha")
.unwrap_err();
assert!(why.contains("alpha"), "{why}");
}
fn placing(hive: &str, agent: &str) -> Vec<(String, swarm_wire::wanted::HiveWanted)> {
let wanted = swarm_wire::wanted::HiveWanted {
agents: BTreeMap::from([(
agent.to_owned(),
swarm_wire::wanted::AgentWanted {
state: swarm_wire::wanted::AgentState::Up,
},
)]),
};
vec![(hive.to_owned(), wanted)]
}
fn reporting(hive: &str, capabilities: &[&str]) -> Vec<HiveStatus> {
let snapshot =
serde_json::json!({"status": "ok", "warnings": [], "capabilities": capabilities});
vec![row(hive, Freshness::Fresh, Some(snapshot))]
}
/// The refusals and the outcomes of a request, decided before anything is
/// queued: an agent no hive places and a hive that doesn't list the
/// capability are refused; an archetype agent converts, and a custom agent
/// (its key already gone) is only re-deployed. The conversion is the
/// control that the refusals' inputs get through otherwise.
#[test]
fn a_request_is_refused_for_an_unplaced_agent_or_an_old_hive() {
let placed = placing("alpha", "atlas");
let ready = reporting("alpha", &["no-config-checkout"]);
assert_eq!(
plan("atlas", Some(reviewer()), &placed, &ready),
Ok((Some(reviewer()), "alpha".to_owned()))
);
assert_eq!(
plan("atlas", None, &placed, &ready),
Ok((None, "alpha".to_owned()))
);
let unplaced = plan("atlas", Some(reviewer()), &[], &ready).unwrap_err();
assert!(unplaced.contains("0 hives"), "{unplaced}");
let old = plan("atlas", Some(reviewer()), &placed, &reporting("alpha", &[])).unwrap_err();
assert!(old.contains("\"alpha\""), "{old}");
// The retry of a custom agent waits for the capability too.
plan("atlas", None, &placed, &reporting("alpha", &[])).unwrap_err();
}
/// A source `internal/meta` does not pin to a commit would float in a repo
/// with no lock, so the seed refuses it and writes nothing.
#[tokio::test]
async fn a_source_that_meta_does_not_pin_is_not_seeded() {
let other = ArchetypeRef {
source: "extra".to_owned(),
name: "builder".to_owned(),
};
let forge = FakeForge::with_inputs(&[("hyperhive", URL), ("extra", "git+http://x/y.git")]);
let why = format!("{:#}", seed(&forge, "atlas", &other).await.unwrap_err());
assert!(why.contains("not pinned"), "{why}");
assert!(forge.repo.lock().unwrap().is_empty());
}
}

View file

@ -73,6 +73,14 @@ pub struct AgentStatusRow {
/// `AppState::agent_login` — this reader sees the hive's bucket, and the
/// flag comes from the agent itself.
pub needs_login: Option<bool>,
/// The archetype the agent is built from; `None` for a custom agent. Always
/// `None` out of this reader, filled in by the `GET /api/agents/status`
/// handler from the `agent-archetype` bucket.
#[schema(
value_type = Option<Object>,
example = json!({"source": "hyperhive", "name": "reviewer"})
)]
pub archetype: Option<swarm_queue_client::agent_archetype::ArchetypeRef>,
}
/// A snapshot as retained by the bucket, with the hive it was published
@ -194,6 +202,7 @@ fn render(
config_pr: None,
wanted: None,
needs_login: None,
archetype: None,
},
})
.collect();
@ -239,6 +248,7 @@ fn row(
config_pr: None,
wanted: None,
needs_login: None,
archetype: None,
}
}

View file

@ -807,22 +807,38 @@ impl Client {
tracing::debug!(%repo, "swarm forge: config already seeded");
return Ok(());
}
let files = vec![
ChangeFileOperation {
content: Some(base64_encode(&initial_agent_nix(agent))),
self.create_config_files(
repo,
&[
("agent.nix", initial_agent_nix(agent)),
("flake.nix", initial_flake_nix().to_owned()),
],
"swarm-controller: seed agent config",
)
.await?;
tracing::info!(%repo, %agent, "swarm forge: seeded agent.nix + flake.nix");
Ok(())
}
/// Commit `files` (path, content) to `repo`'s default branch in one
/// `repo_change_files` call, creating each path. No presence check: a
/// path that already exists makes the forge refuse the whole commit.
pub(crate) async fn create_config_files(
&self,
repo: &str,
files: &[(&str, String)],
message: &str,
) -> Result<()> {
let files: Vec<ChangeFileOperation> = files
.iter()
.map(|(path, content)| ChangeFileOperation {
content: Some(base64_encode(content)),
from_path: None,
operation: ChangeFileOperationOperation::Create,
path: "agent.nix".to_owned(),
path: (*path).to_owned(),
sha: None,
},
ChangeFileOperation {
content: Some(base64_encode(initial_flake_nix())),
from_path: None,
operation: ChangeFileOperationOperation::Create,
path: "flake.nix".to_owned(),
sha: None,
},
];
})
.collect();
self.api
.call(|api| {
api.repo_change_files(
@ -835,7 +851,7 @@ impl Client {
dates: None,
files: files.clone(),
force_overwrite_new_branch: None,
message: Some("swarm-controller: seed agent config".to_owned()),
message: Some(message.to_owned()),
new_branch: None,
signoff: None,
},
@ -843,8 +859,7 @@ impl Client {
.into_future()
})
.await
.with_context(|| format!("seed config files in {CONFIG_ORG}/{repo}"))?;
tracing::info!(%repo, %agent, "swarm forge: seeded agent.nix + flake.nix");
.with_context(|| format!("commit config files to {CONFIG_ORG}/{repo}"))?;
Ok(())
}

View file

@ -57,6 +57,15 @@ pub(crate) enum SwarmNodeKind {
/// swarm routes a deploy message to — it belongs on the node that
/// sends that message, not on this one.
InitAgentConfigRepo { agent: String },
/// Seed the agent's config repo with the archetype's module, for turn
/// into custom. See `agent_custom::seed`.
SeedArchetypeConfig {
agent: String,
archetype: swarm_wire::agent_archetype::ArchetypeRef,
},
/// Delete the agent's archetype key, which makes it custom. See
/// `agent_custom::drop_archetype`.
DropArchetype { agent: String },
/// Mint `agent`'s own client certificate for the swarm secret store,
/// publish it there, grant it, and log in with it. See
/// `agent_identity::mint_and_verify` — including why this node does not
@ -168,6 +177,8 @@ impl hive_jobq_wire::WireNode for SwarmNodeKind {
SwarmNodeKind::AddRepoMember { .. } => "add_repo_member".to_owned(),
SwarmNodeKind::GrantDocsRead { .. } => "grant_docs_read".to_owned(),
SwarmNodeKind::InitAgentConfigRepo { .. } => "init_agent_config_repo".to_owned(),
SwarmNodeKind::SeedArchetypeConfig { .. } => "seed_archetype_config".to_owned(),
SwarmNodeKind::DropArchetype { .. } => "drop_archetype".to_owned(),
SwarmNodeKind::MintAgentIdentity { .. } => "mint_agent_identity".to_owned(),
SwarmNodeKind::MintAgentForgeToken { .. } => "mint_agent_forge_token".to_owned(),
SwarmNodeKind::MintAgentMatrixAccount { .. } => "mint_agent_matrix_account".to_owned(),
@ -202,6 +213,7 @@ impl hive_jobq_wire::WireNode for SwarmNodeKind {
| SwarmNodeKind::AddRepoMember { agent }
| SwarmNodeKind::GrantDocsRead { agent }
| SwarmNodeKind::InitAgentConfigRepo { agent }
| SwarmNodeKind::DropArchetype { agent }
| SwarmNodeKind::MintAgentIdentity { agent }
| SwarmNodeKind::MintAgentForgeToken { agent }
| SwarmNodeKind::MintAgentMatrixAccount { agent }
@ -211,6 +223,9 @@ impl hive_jobq_wire::WireNode for SwarmNodeKind {
SwarmNodeKind::BuildMeta { agent, pr } => {
serde_json::json!({ "agent": agent, "pr": pr })
}
SwarmNodeKind::SeedArchetypeConfig { agent, archetype } => {
serde_json::json!({ "agent": agent, "archetype": archetype.name })
}
SwarmNodeKind::MintHiveSenderToken { hive }
| SwarmNodeKind::MintHiveForgeToken { hive } => serde_json::json!({ "hive": hive }),
SwarmNodeKind::MintClassifierForgeToken
@ -404,6 +419,9 @@ async fn run_node_work(
},
SwarmNodeKind::GrantDocsRead { agent } => grant_docs_read(forge, &agent).await,
SwarmNodeKind::InitAgentConfigRepo { agent } => init_agent_config_repo(forge, &agent).await,
SwarmNodeKind::SeedArchetypeConfig { .. } | SwarmNodeKind::DropArchetype { .. } => {
convert_node(&kind, forge, deps.queue.as_ref()).await
}
SwarmNodeKind::MintAgentIdentity { agent } => {
mint_identity(deps.wanted.as_deref(), &agent).await
}
@ -592,6 +610,44 @@ async fn init_agent_config_repo(
}
}
/// The `SeedArchetypeConfig` and `DropArchetype` arms, lifted out for the
/// same reason. Any other kind is not this function's to run.
async fn convert_node(
kind: &SwarmNodeKind,
forge: Option<Arc<forge::Client>>,
queue: Option<&async_nats::Client>,
) -> hive_jobq::scheduler::Outcome {
use hive_jobq::scheduler::Outcome;
let done = |result: Result<()>| match result {
Ok(()) => Outcome::Done,
Err(e) => Outcome::Failed(format!("{e:#}")),
};
match kind {
SwarmNodeKind::SeedArchetypeConfig { agent, archetype } => {
let Some(client) = forge else {
return Outcome::Failed(
"no forge access here (SWARM_CONTROLLER_FORGE_URL unset, or its token \
not in the swarm secret store)"
.to_owned(),
);
};
done(
forge_node(
"seed_archetype_config",
crate::agent_custom::seed(client.as_ref(), agent, archetype),
)
.await,
)
}
SwarmNodeKind::DropArchetype { agent } => match queue {
Some(client) => done(crate::agent_custom::drop_archetype(client, agent).await),
None => Outcome::Failed("no swarm queue is configured on this host".to_owned()),
},
other => unreachable!("{other:?} is not a conversion node"),
}
}
/// The `RenewAgentQueueCredential` arm, lifted out for the same reason.
async fn renew_queue_credential(agent: &str) -> hive_jobq::scheduler::Outcome {
use hive_jobq::scheduler::Outcome;
@ -1421,6 +1477,65 @@ pub(crate) fn declare_agent_job(
vec![create_identity.guid()]
}
/// The sub-DAG turning the archetype agent `agent` into a custom one: its
/// config repo is created and seeded, and only then is its archetype key
/// deleted, so a failed step leaves the agent archetype-built. The meta
/// update and the deploy follow the delete (`after_ok`), as in
/// [`merged_deploy_job`]. Returns the repo node, the handle the response
/// reports.
///
/// `CreateRepo`, `CreateForgeUser` and `AddRepoMember` are no-ops for what
/// already exists, so the same graph can be queued again after a failure.
///
/// With `archetype` `None` the agent is already custom: its key was deleted and
/// a later step failed, so the graph is the meta update and the deploy alone,
/// and the handle is the update. A build that fails after the key delete leaves
/// the agent custom, which is how that failure is retried.
pub(crate) fn declare_custom_job(
b: &hive_jobq::builder::JobBuilder<SwarmNodeKind, SwarmResourceKind>,
agent: &str,
hive: &str,
archetype: Option<&swarm_wire::agent_archetype::ArchetypeRef>,
) -> Vec<hive_jobq::builder::NodeGuid> {
let Some(archetype) = archetype else {
let update_meta = b
.node(SwarmNodeKind::UpdateMeta)
.needs(SwarmResourceKind::Meta);
let _deploy = gated_deploy(b, hive, agent, None, None, Some(update_meta.guid()));
return vec![update_meta.guid()];
};
let create_repo = b.node(SwarmNodeKind::CreateRepo {
agent: agent.to_owned(),
});
let create_forge_user = b.node(SwarmNodeKind::CreateForgeUser {
agent: agent.to_owned(),
});
let add_repo_member = b
.node(SwarmNodeKind::AddRepoMember {
agent: agent.to_owned(),
})
.after_ok(create_repo)
.after_ok(create_forge_user);
let seed = b
.node(SwarmNodeKind::SeedArchetypeConfig {
agent: agent.to_owned(),
archetype: archetype.clone(),
})
.after_ok(create_repo);
let drop_archetype = b
.node(SwarmNodeKind::DropArchetype {
agent: agent.to_owned(),
})
.after_ok(seed)
.after_ok(add_repo_member);
let update_meta = b
.node(SwarmNodeKind::UpdateMeta)
.needs(SwarmResourceKind::Meta)
.after_ok(drop_archetype);
let _deploy = gated_deploy(b, hive, agent, None, None, Some(update_meta.guid()));
vec![create_repo.guid()]
}
/// Insert one job per agent: `CreateForgeUser`, then `MintAgentForgeToken`
/// `after_ok` it — the same edge [`declare_agent_job`] uses. Returns the mint
/// nodes' ids.
@ -2389,6 +2504,87 @@ mod tests {
let _ = super::merged_deploy_job(b, "pr1ma", "atlas", "abc", 7);
}
fn custom_dag(b: &TestBuilder) {
let archetype = swarm_wire::agent_archetype::ArchetypeRef {
source: "hyperhive".to_owned(),
name: "reviewer".to_owned(),
};
let _ = super::declare_custom_job(b, "atlas", "pr1ma", Some(&archetype));
}
/// Turn into custom: with every node succeeding the agent is seeded, loses
/// its key, and is deployed.
#[tokio::test]
async fn a_conversion_seeds_then_drops_the_key_then_deploys() {
use hive_jobq::State;
let done = run_dag_failing(custom_dag, None).await;
for label in [
"create_repo",
"seed_archetype_config",
"drop_archetype",
"update_meta",
"trigger_deploy",
] {
assert_eq!(state_of(&done, label), State::Done, "{label}");
}
// The key is deleted only after the seed and the member grant, and
// each edge refuses a failure. The deploy edge is the control that
// the lookup finds an `after_ok` edge in this graph.
for (from, on) in [
("drop_archetype", "seed_archetype_config"),
("drop_archetype", "add_repo_member"),
("update_meta", "drop_archetype"),
("trigger_deploy", "update_meta"),
] {
assert!(
!edge_accepts_failure(custom_dag, from, on),
"{from} -> {on}"
);
}
}
/// The retry of a conversion whose build failed after the key was deleted:
/// with no archetype the graph is the meta update and the deploy, with no
/// repo, seed or key delete. The conversion is the control that those
/// nodes exist when there is one.
#[tokio::test]
async fn a_retry_of_a_custom_agent_is_only_the_update_and_the_deploy() {
use hive_jobq::State;
fn retry_dag(b: &TestBuilder) {
let _ = super::declare_custom_job(b, "atlas", "pr1ma", None);
}
let done = run_dag_failing(retry_dag, None).await;
let labels: Vec<&str> = done.iter().map(|(l, _)| l.as_str()).collect();
for absent in ["create_repo", "seed_archetype_config", "drop_archetype"] {
assert!(!labels.contains(&absent), "{absent} in {labels:?}");
}
assert_eq!(state_of(&done, "update_meta"), State::Done);
assert_eq!(state_of(&done, "trigger_deploy"), State::Done);
let converting = run_dag_failing(custom_dag, None).await;
assert!(converting.iter().any(|(l, _)| l == "drop_archetype"));
// A failed build leaves the deploy skipped, so a retry has work to do.
let red = run_dag_failing(retry_dag, Some("build_meta")).await;
assert_eq!(state_of(&red, "trigger_deploy"), State::Skipped);
}
/// A seed that fails leaves the key in place and nothing deployed; the
/// same graph, queued again with the seed succeeding, completes.
#[tokio::test]
async fn a_failed_seed_keeps_the_key_and_a_retry_completes() {
use hive_jobq::State;
let failed = run_dag_failing(custom_dag, Some("seed_archetype_config")).await;
assert_eq!(state_of(&failed, "seed_archetype_config"), State::Failed);
for label in ["drop_archetype", "update_meta", "trigger_deploy"] {
assert_eq!(state_of(&failed, label), State::Skipped, "{label}");
}
let retried = run_dag_failing(custom_dag, None).await;
assert_eq!(state_of(&retried, "drop_archetype"), State::Done);
assert_eq!(state_of(&retried, "trigger_deploy"), State::Done);
}
/// A deploy needs the meta update to succeed: its edge refuses a failed
/// update, in both DAGs that queue one.
#[test]

View file

@ -23,6 +23,7 @@ use anyhow::{Context, Result};
mod agent_archetype;
mod agent_control;
mod agent_custom;
mod agent_entity;
mod agent_icon;
mod agent_identity;

View file

@ -418,26 +418,29 @@ async fn render(
CONFIG_ORG,
&agent.name,
Some(CONFIG_BRANCH),
&["flake.lock"],
&["flake.lock", "flake.nix"],
)
.await?
else {
tracing::warn!(agent = %agent.name, "meta flake: its config repo has no commit on {CONFIG_BRANCH}; leaving the agent out");
continue;
};
let lock = head
.files
.first()
.map(|f| String::from_utf8_lossy(&f.bytes).into_owned());
let read = |path: &str| {
head.files
.iter()
.find(|f| f.path == path)
.map(|f| String::from_utf8_lossy(&f.bytes).into_owned())
};
let (lock, flake) = (read("flake.lock"), read("flake.nix"));
Some(hive_meta_flake::AgentPin {
url: hive_meta_flake::pinned_git_url(
&settings.git_url(CONFIG_ORG, &agent.name),
Some(&head.commit),
)?,
canonical_inputs: lock
.as_deref()
.map(hive_meta_flake::canonical_inputs_in_lock)
.unwrap_or_default(),
canonical_inputs: hive_meta_flake::canonical_inputs_declared(
lock.as_deref(),
flake.as_deref(),
),
})
};
specs.push(hive_meta_flake::AgentSpec {

View file

@ -472,6 +472,21 @@ pub(crate) async fn get_agents_status(
row.needs_login = login.get(&row.name);
}
}
// Absent on a failed read, like `wanted`: one unreadable bucket
// must not blank the roster.
if let Some(reader) = state.status.as_ref() {
match agent_archetype::of_agents(&reader.queue_client(), &rows).await {
Ok(mut archetypes) => {
for row in &mut rows {
row.archetype = archetypes.remove(&row.name);
}
}
Err(e) => tracing::warn!(
error = %format!("{e:#}"),
"reading the agent archetypes failed; rows show none"
),
}
}
Ok(Json(rows))
}
Err(e) => {
@ -1800,6 +1815,7 @@ mod tests {
config_pr: None,
wanted: None,
needs_login: None,
archetype: None,
}
}

View file

@ -28,12 +28,12 @@ use crate::jobq::{
use crate::routes::hives::{HiveEntry, ServiceLink};
use crate::routes::{self, AppState, agents, config_prs, forge_admin, hives, jobq_graph};
use crate::{
agent_control, agent_icon, agent_input, agent_limits, agent_login, agent_permissions,
agent_renewal, agent_state_stream, agent_status, agents_md, alerts, archetypes, auth,
backend_env, builder, claude_login, config_pr, egress_allow, forge, forge_account,
github_account, inference, issue_report, linked_accounts, matrix_account, message_stream,
notify_flows, notify_router_api, pulls, queue_identity, review_classifier, schedules, status,
subagent_term, term_stream, wanted, webhook,
agent_control, agent_custom, agent_icon, agent_input, agent_limits, agent_login,
agent_permissions, agent_renewal, agent_state_stream, agent_status, agents_md, alerts,
archetypes, auth, backend_env, builder, claude_login, config_pr, egress_allow, forge,
forge_account, github_account, inference, issue_report, linked_accounts, matrix_account,
message_stream, notify_flows, notify_router_api, pulls, queue_identity, review_classifier,
schedules, status, subagent_term, term_stream, wanted, webhook,
};
/// Where the daemon binds, overridable via `SWARM_CONTROLLER_SOCKET`.
@ -656,6 +656,7 @@ pub(crate) fn build_app(state: AppState) -> axum::Router {
.routes(routes!(agent_input::post_agent_input))
.routes(routes!(agent_input::post_agents_broadcast))
.routes(routes!(agent_control::post_agent_cancel))
.routes(routes!(agent_custom::post_agent_custom))
.routes(routes!(
claude_login::post_claude_login,
claude_login::delete_claude_login

View file

@ -12,6 +12,12 @@
/// this.
pub const TOLERANT_PERMISSIONS: &str = "tolerant-permissions";
/// A hive lists this when it builds a custom agent from `internal/meta` alone
/// and keeps no checkout of its config: no `applied` or `proposed` repo and no
/// config mount in the container. swarm-controller turns an archetype agent
/// into a custom one only on a hive that advertises it.
pub const NO_CONFIG_CHECKOUT: &str = "no-config-checkout";
/// The part of a hive's status snapshot this module describes.
#[derive(Debug, Clone, Default, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
pub struct Capabilities {