diff --git a/docs/swarm/README.md b/docs/swarm/README.md index 71f8bb2b6..8c7e92ed8 100644 --- a/docs/swarm/README.md +++ b/docs/swarm/README.md @@ -94,11 +94,6 @@ below. A hive without them reads | `never_reported` | this hive has never reported at all — normally a deployment that hasn't happened, not an outage | | `unknown` | something is publishing under a name that's not in `swarm.hives` — a typo in the roster, or a hive removed from the roster but still running | -The snapshot is the hive's `/health/ready` document plus `capabilities`, a list -of names for what the hive's release can do. Two exist: `tolerant-permissions`, and `no-config-checkout`, which -[turning an archetype agent into a custom one](archetypes.md#turn-into-custom) -needs. A hive on an older release lists none. - Every row also carries `last_seen_unix` and `age_seconds` if you want to apply your own threshold. The timestamp is the one the queue recorded on arrival, not one the hive put in its own payload. A roster hive's row diff --git a/docs/swarm/archetypes.md b/docs/swarm/archetypes.md index 6494599a8..6ae72a29f 100644 --- a/docs/swarm/archetypes.md +++ b/docs/swarm/archetypes.md @@ -9,9 +9,7 @@ field before you create the agent. An agent created from an archetype has no `agent-configs/` repo of its own. The swarm's `internal/meta` flake builds it from the archetype's module, so every agent of that archetype runs the same configuration. To change what it runs, change the -archetype. Picking "none" creates a custom agent, with its own config repo. The -swarm can also turn an archetype agent into a custom one later -([below](#turn-into-custom)). +archetype. Picking "none" creates a custom agent, with its own config repo. ## The built-in archetypes @@ -141,57 +139,3 @@ The module comes from the `hyperhive` input that `internal/meta` pins, not from the hive's own. When that pin predates the archetype, the agent's build fails with `this hive's hyperhive input has no archetype ; update that input`. Update the `hyperhive` pin in `internal/meta` to fix it. - -## Turn into custom - -An archetype agent can become a custom agent with its own -`agent-configs/` repo, either from the agent's overview in the swarm UI -(the "turn into custom" button asks first) or with `POST /api/agents/{name}/custom`. -The agent keeps its state, permissions, caps, inference settings and accounts. -Its configuration stays the same, except that it no longer has `HIVE_ARCHETYPE` -in its environment. Keeping the config current is then up to the repo's owner. -Nothing here undoes it. - -The controller refuses with 409 an agent that no hive or more than one hive -places, and one whose hive doesn't list the -`no-config-checkout` capability (see below). Otherwise it queues a job graph: - -1. `CreateRepo`, `CreateForgeUser` and `AddRepoMember`: the repo and the - agent's access to the repo, as for any custom agent. Each is a no-op for what - exists. -2. `SeedArchetypeConfig` commits two files to the repo. It copies nothing from - the archetype: - - `flake.nix` declares the archetype's source as an input, at the URL - `internal/meta` holds for that source today (a pinned rev, which is the input's only pin: the repo has no lock, and a source meta doesn't pin to a commit fails the node), and imports - `inputs..hyperhiveArchetypes..module` next to `./agent.nix`; - - `agent.nix` is an empty module for the owner's edits. - - The node leaves a repo whose `flake.nix` already is this file alone, whatever - URL it pins. Any other `flake.nix` fails the node. - -3. `DropArchetype` deletes the agent's `agent-archetype` key, only after the - seed and the member grant succeeded. A failure before it leaves the agent - archetype-built and working, and you can send the same request again. -4. `UpdateMeta`, then the build and the deploy, as for a merged config PR. - If the build fails, the agent is custom and has no key. Sending the request - again then queues only this step, as the earlier ones have run. - -When the source is `hyperhive`, the meta flake sets -`agent-.inputs.hyperhive.follows = "hyperhive"` for an agent whose -`flake.nix` or `flake.lock` declares a `hyperhive` input, so the lock holds one -hyperhive, and the module comes from the pin `internal/meta` holds. Any other -source keeps the pinned URL from its seeded `flake.nix`, and the meta lock -records that rev. The `follows` applies to every custom agent whose `flake.nix` -declares a `hyperhive` input, not only converted ones. A `hyperhive` node left -in a lock doesn't count. - -### Which hives may convert - -A hive that keeps a checkout of a custom agent's config would create one for -the converted agent, under the container's running mount. A hive that lists -`no-config-checkout` under `capabilities` in the status it publishes (the -document `GET /api/hives/status` serves as `snapshot`) keeps none: it builds a -custom agent from `internal/meta` alone, and the agent clones its own repo from -the forge. The controller reads the agent's hive's latest status and refuses -unless it's `fresh` and lists the capability; an older hive lists none. Deploy -the release to every hive that places archetype agents before converting theirs. diff --git a/frontend/packages/swarm-ui/src/agent/AgentTypes.ts b/frontend/packages/swarm-ui/src/agent/AgentTypes.ts index 1969c63b8..dbcc29d45 100644 --- a/frontend/packages/swarm-ui/src/agent/AgentTypes.ts +++ b/frontend/packages/swarm-ui/src/agent/AgentTypes.ts @@ -43,17 +43,6 @@ export interface AgentRow { // no header since swarm-controller started, or an agent that does not // send the flag — not known, so not "needs login". needs_login: boolean | null; - // The archetype the agent is built from; `null` for a custom agent. - archetype: { source: string; name: string } | null; -} - -// "Turn into custom" is offered to an archetype agent whose hive is known and -// that has not been destroyed: the controller places the conversion on that -// hive. -export function canTurnIntoCustom( - row: Pick, -): boolean { - return !!row.archetype && !!row.hive && row.wanted !== "destroyed"; } // Only an agent that said so gets the badge; `null` shows nothing. diff --git a/frontend/packages/swarm-ui/src/agent/detail/AgentDetail.tsx b/frontend/packages/swarm-ui/src/agent/detail/AgentDetail.tsx index bae373eff..70cdaf42a 100644 --- a/frontend/packages/swarm-ui/src/agent/detail/AgentDetail.tsx +++ b/frontend/packages/swarm-ui/src/agent/detail/AgentDetail.tsx @@ -11,7 +11,6 @@ // change or a link made here shows wherever else the same row is drawn. import { ApiErrorPanel } from "@hive/shared/api-error-panel.js"; import { Badge } from "@hive/shared/badge.js"; -import { Button } from "../../ui/button/Button.js"; import { RelativeTime } from "../../ui/relative-time/RelativeTime.js"; import type { AgentActions } from "../roster/AgentActions.js"; import { AgentInference } from "./AgentInference.js"; @@ -21,7 +20,7 @@ import { AgentPermissions } from "./AgentPermissions.js"; import { AllowedDomains } from "./AllowedDomains.js"; import { AgentStateRow } from "./AgentStateRow.js"; import { AgentTermPreview } from "../terminal/AgentTermPreview.js"; -import { canTurnIntoCustom, FRESHNESS, type AgentRow } from "../AgentTypes.js"; +import { FRESHNESS, type AgentRow } from "../AgentTypes.js"; import { LinkAccountMenu } from "./LinkAccountMenu.js"; import { LinkedAccounts } from "./LinkedAccounts.js"; import { LazyDisclosure } from "../../ui/disclosure/LazyDisclosure.js"; @@ -91,13 +90,7 @@ function HeaderRows({ ); } -function OverviewRows({ - row, - actions, -}: { - row: AgentRow; - actions: AgentActions; -}) { +function OverviewRows({ row }: { row: AgentRow }) { return ( <>
state
@@ -129,25 +122,6 @@ function OverviewRows({ "—" )} - {row.archetype ? ( - <> -
archetype
-
- {row.archetype.source}/{row.archetype.name} - {canTurnIntoCustom(row) ? ( - <> - {" "} - - - ) : null} -
- - ) : null} ); } @@ -237,7 +211,7 @@ function SectionBody({ case "overview": return (
- +
); case "settings": @@ -298,7 +272,7 @@ export function AgentDetail({
- +
diff --git a/frontend/packages/swarm-ui/src/agent/roster/AgentActions.tsx b/frontend/packages/swarm-ui/src/agent/roster/AgentActions.tsx index f56abea7d..2e0e8f505 100644 --- a/frontend/packages/swarm-ui/src/agent/roster/AgentActions.tsx +++ b/frontend/packages/swarm-ui/src/agent/roster/AgentActions.tsx @@ -71,7 +71,6 @@ export interface AgentActions { selectOffline: (row: AgentRow) => void; selectPaused: (row: AgentRow) => void; requestDestroy: (row: AgentRow) => void; - requestCustom: (row: AgentRow) => void; openLink: (row: AgentRow, kind: LinkKind) => void; /** The confirm and link dialogs; render once, anywhere. */ dialogs: ComponentChildren; @@ -95,7 +94,6 @@ export function useAgentActions( row: AgentRow; state: "offline" | "paused"; } | null>(null); - const [customTarget, setCustomTarget] = useState(null); const [matrixTarget, setMatrixTarget] = useState(null); const [forgeTarget, setForgeTarget] = useState(null); const [githubTarget, setGithubTarget] = useState(null); @@ -174,38 +172,6 @@ export function useAgentActions( await declareState(row, "destroyed"); } - // The conversion is queued as a job graph; the roster shows the agent as - // custom once it has run, so a success needs no handling here. - async function turnIntoCustom(row: AgentRow) { - setCustomTarget(null); - setPendingAgents((prev) => new Set(prev).add(row.name)); - setActionErrors((prev) => { - const next = new Map(prev); - next.delete(row.name); - return next; - }); - try { - const res = await fetch( - `/api/agents/${encodeURIComponent(row.name)}/custom`, - { method: "POST" }, - ); - if (!res.ok) { - const problem = await readApiError(res); - setActionErrors((prev) => new Map(prev).set(row.name, problem)); - } - } catch (e: unknown) { - setActionErrors((prev) => - new Map(prev).set(row.name, { detail: String(e) }), - ); - } finally { - setPendingAgents((prev) => { - const next = new Set(prev); - next.delete(row.name); - return next; - }); - } - } - const dialogs = ( <> - setCustomTarget(null)} - onConfirm={() => customTarget && void turnIntoCustom(customTarget)} - confirmLabel="turn into custom" - > - {customTarget ? ( -

- Turn {customTarget.name} into a custom agent? It - gets its own config repo, which imports the archetype it runs today, - and is rebuilt from that repo. Keeping the config current is then up - to the repo's owner. This is not reversible from here. -

- ) : null} - setConfirmTarget({ row, state: "offline" }), selectPaused: (row) => setConfirmTarget({ row, state: "paused" }), requestDestroy: setDestroyTarget, - requestCustom: setCustomTarget, openLink: (row, kind) => { if (kind === "matrix") setMatrixTarget(row); else if (kind === "forge") setForgeTarget(row); diff --git a/frontend/packages/swarm-ui/test/agent-types.test.mjs b/frontend/packages/swarm-ui/test/agent-types.test.mjs index ea059531e..758d8dc6f 100644 --- a/frontend/packages/swarm-ui/test/agent-types.test.mjs +++ b/frontend/packages/swarm-ui/test/agent-types.test.mjs @@ -3,10 +3,7 @@ // imported as it is. import assert from "node:assert/strict"; import { test } from "node:test"; -import { - canTurnIntoCustom, - showsLoginNeeded, -} from "../src/agent/AgentTypes.ts"; +import { showsLoginNeeded } from "../src/agent/AgentTypes.ts"; test("an agent that says it needs a login gets the badge", () => { assert.equal(showsLoginNeeded({ needs_login: true }), true); @@ -20,25 +17,3 @@ test("an agent that has not said gets none", () => { assert.equal(showsLoginNeeded({ needs_login: null }), false); assert.equal(showsLoginNeeded({}), false); }); - -test("turn into custom is offered to a placed archetype agent only", () => { - const archetype = { source: "hyperhive", name: "reviewer" }; - assert.equal( - canTurnIntoCustom({ archetype, hive: "alpha", wanted: "up" }), - true, - ); - // A custom agent has no archetype to turn from. - assert.equal( - canTurnIntoCustom({ archetype: null, hive: "alpha", wanted: "up" }), - false, - ); - // The controller places the conversion on a hive, so one is needed. - assert.equal( - canTurnIntoCustom({ archetype, hive: null, wanted: "up" }), - false, - ); - assert.equal( - canTurnIntoCustom({ archetype, hive: "alpha", wanted: "destroyed" }), - false, - ); -}); diff --git a/hive-c0re/src/swarm_status.rs b/hive-c0re/src/swarm_status.rs index 49053fe8b..48fb9aa0e 100644 --- a/hive-c0re/src/swarm_status.rs +++ b/hive-c0re/src/swarm_status.rs @@ -407,10 +407,7 @@ struct Snapshot { fn snapshot() -> Snapshot { Snapshot { readiness: crate::warnings::readiness(), - capabilities: &[ - swarm_wire::hive_status::TOLERANT_PERMISSIONS, - swarm_wire::hive_status::NO_CONFIG_CHECKOUT, - ], + capabilities: &[swarm_wire::hive_status::TOLERANT_PERMISSIONS], } } @@ -452,16 +449,12 @@ mod snapshot_tests { /// The controller reads the capability out of the published bytes with /// the wire type, and the readiness keeps its own top-level fields. #[test] - fn the_snapshot_advertises_its_capabilities_beside_the_readiness() { + fn the_snapshot_advertises_tolerant_permissions_beside_the_readiness() { let value = serde_json::to_value(snapshot()).unwrap(); assert!( swarm_wire::hive_status::Capabilities::of_snapshot(&value) .has(swarm_wire::hive_status::TOLERANT_PERMISSIONS) ); - assert!( - swarm_wire::hive_status::Capabilities::of_snapshot(&value) - .has(swarm_wire::hive_status::NO_CONFIG_CHECKOUT) - ); assert!(value.get("status").is_some() && value.get("warnings").is_some()); } } diff --git a/hive-meta-flake/src/lib.rs b/hive-meta-flake/src/lib.rs index ed40b9dd1..8b35b1853 100644 --- a/hive-meta-flake/src/lib.rs +++ b/hive-meta-flake/src/lib.rs @@ -43,12 +43,12 @@ pub fn pinned_git_url(forge_git_url: &str, rev: Option<&str>) -> Result Ok(format!("git+{forge_git_url}?rev={rev}")) } -/// Canonical inputs meta knows how to dedup. An agent whose `flake.lock` or -/// `flake.nix` declares one of these as a top-level input gets a +/// Canonical inputs meta knows how to dedup. An agent that declares one +/// of these as a top-level input in its own `flake.nix` will get a /// `follows = ""` line emitted in meta — collapsing the -/// otherwise-separate-but-identical `nixpkgs_N` and `hyperhive_N` nodes into -/// a single meta-level reference. -pub const CANONICAL_INPUTS: &[&str] = &["nixpkgs", "hyperhive"]; +/// otherwise-separate-but-identical `nixpkgs_N` nodes into a single +/// meta-level reference. +pub const CANONICAL_INPUTS: &[&str] = &["nixpkgs"]; /// Env vars the meta flake sets in every agent container, from the /// `(name, value)` pairs [`render_flake`] is given. A caller passes only the @@ -143,32 +143,6 @@ pub fn push_forwarded_var_options(out: &mut String, vars: &[(&'static str, Strin } } -/// The subset of `CANONICAL_INPUTS` that an agent config declares, in its -/// `flake.lock` text `lock` or its `flake.nix` text `flake_nix`. A repo that -/// was only just seeded has no lock yet, so its `flake.nix` counts too: a line -/// starting `.url` or `inputs..url`. -/// -/// A `hyperhive` node in a lock does not count: it can be a leftover from an -/// input the config dropped from its `flake.nix`, and a `follows` for an input -/// the flake does not declare makes nix warn. -#[must_use] -pub fn canonical_inputs_declared(lock: Option<&str>, flake_nix: Option<&str>) -> Vec<&'static str> { - let locked = lock.map(canonical_inputs_in_lock).unwrap_or_default(); - CANONICAL_INPUTS - .iter() - .copied() - .filter(|canon| { - (*canon != "hyperhive" && locked.contains(canon)) - || flake_nix.is_some_and(|text| { - text.lines().map(str::trim_start).any(|l| { - l.starts_with(&format!("{canon}.url")) - || l.starts_with(&format!("inputs.{canon}.url")) - }) - }) - }) - .collect() -} - /// The subset of `CANONICAL_INPUTS` that the `flake.lock` text `raw` declares /// as direct (root-level) inputs. Empty when the lock is unparsable — safe /// degradation, the worst case is no dedup for that agent. @@ -486,26 +460,17 @@ pub fn is_rev(rev: &str) -> bool { rev.len() == 40 && rev.bytes().all(|b| matches!(b, b'0'..=b'9' | b'a'..=b'f')) } -/// The URL of `input` in a flake [`render_flake`] rendered: the text of its -/// `.url = "";` line. `None` when the flake has no such input. -#[must_use] -pub fn input_url<'a>(flake_nix: &'a str, input: &str) -> Option<&'a str> { - let prefix = format!(" {input}.url = \""); - Some( - flake_nix - .lines() - .find_map(|l| l.strip_prefix(prefix.as_str()))? - .split_once('"')? - .0, - ) -} - /// The commit `input` is pinned to in a flake [`render_flake`] rendered: /// the `` of its `.url = "git+?rev=";` line. `None` when /// the flake has no such input, or its rev is not a commit id. #[must_use] pub fn pinned_rev<'a>(flake_nix: &'a str, input: &str) -> Option<&'a str> { - let url = input_url(flake_nix, input)?; + let prefix = format!(" {input}.url = \""); + let url = flake_nix + .lines() + .find_map(|l| l.strip_prefix(prefix.as_str()))? + .split_once('"')? + .0; let (_, rev) = url.split_once("?rev=")?; let rev = rev.split('&').next()?; is_rev(rev).then_some(rev) @@ -730,51 +695,4 @@ mod tests { assert!(!builds_agent_at(&flake, "gus", None, Some(ARGUS))); assert!(!builds_agent_at(&flake, "v", Some(&reviewer()), None)); } - /// An agent that declares `hyperhive` follows meta's own pin, so the - /// lock holds one hyperhive; one that declares nothing gets no line (the - /// control that the line is the declared input's, not every agent's). - #[test] - fn a_declared_hyperhive_input_follows_meta_s_own() { - let url = |repo: &str, rev: &str| { - pinned_git_url(&format!("https://f.t/{repo}.git"), Some(rev)).unwrap() - }; - let agents = |canonical_inputs: Vec<&'static str>| { - render_flake( - &url("hyperhive/hyperhive", HYPERHIVE), - &[spec( - "argus", - None, - Some(AgentPin { - url: url("agent-configs/argus", ARGUS), - canonical_inputs, - }), - )], - &[], - false, - ) - .unwrap() - }; - let line = "agent-argus.inputs.hyperhive.follows = \"hyperhive\";"; - assert!(agents(vec!["hyperhive"]).contains(line)); - assert!(!agents(vec![]).contains(line)); - } - - /// A seeded repo has only a `flake.nix`; its input counts, from either - /// spelling, and a lock-declared input still does. - #[test] - fn canonical_inputs_are_read_from_the_flake_text_and_the_lock() { - let flake = "{\n inputs = {\n hyperhive.url = \"git+x\";\n };\n}\n"; - assert_eq!(canonical_inputs_declared(None, Some(flake)), ["hyperhive"]); - let dotted = "{\n inputs.nixpkgs.url = \"x\";\n}\n"; - assert_eq!(canonical_inputs_declared(None, Some(dotted)), ["nixpkgs"]); - let lock = r#"{"root":"root","nodes":{"root":{"inputs":{"nixpkgs":"n"}}}}"#; - assert_eq!(canonical_inputs_declared(Some(lock), None), ["nixpkgs"]); - assert!(canonical_inputs_declared(None, Some("{ inputs = { }; }")).is_empty()); - // A leftover `hyperhive` node in a lock declares nothing. - let stale = r#"{"root":"root","nodes":{"root":{"inputs":{"hyperhive":"hyperhive"}}}}"#; - assert!(canonical_inputs_declared(Some(stale), Some("{ inputs = { }; }")).is_empty()); - // Another archetype source is not canonical: meta does not follow it. - let other = "{\n inputs = {\n extra.url = \"git+x\";\n };\n}\n"; - assert!(canonical_inputs_declared(None, Some(other)).is_empty()); - } } diff --git a/swarm-controller/src/agent_archetype.rs b/swarm-controller/src/agent_archetype.rs index d355634b0..d793a150c 100644 --- a/swarm-controller/src/agent_archetype.rs +++ b/swarm-controller/src/agent_archetype.rs @@ -114,22 +114,6 @@ pub(crate) async fn restore(client: &async_nats::Client, agent: &str, replaced: } } -/// The archetype of each agent in `rows` that has one. -pub(crate) async fn of_agents( - client: &async_nats::Client, - rows: &[crate::agent_status::AgentStatusRow], -) -> anyhow::Result> { - let store = agent_archetype::open_or_create(client).await?; - let mut found = std::collections::HashMap::new(); - for row in rows { - let (archetype, _, _) = read_or_default::>(&store, &row.name).await?; - if let Some(archetype) = archetype { - found.insert(row.name.clone(), archetype); - } - } - Ok(found) -} - /// Delete a destroyed agent's key, so the next agent of that name is not /// built from this one's archetype. A failure is logged and does not stop the /// destroy, as in `agent_permissions::clear_on_destroy`. diff --git a/swarm-controller/src/agent_custom.rs b/swarm-controller/src/agent_custom.rs deleted file mode 100644 index 1b9a9cdfe..000000000 --- a/swarm-controller/src/agent_custom.rs +++ /dev/null @@ -1,611 +0,0 @@ -//! `POST /api/agents/{name}/custom`: turn an archetype agent into a custom -//! one, with its own `agent-configs/` repo. -//! -//! The repo's `flake.nix` declares the archetype's source flake as an input, at -//! the URL `internal/meta` holds for that source, and imports the archetype's -//! module from it. Nothing is copied: any source `internal/meta` has a pinned -//! input for works. The URL's `rev` is the only pin, since the seeded repo has -//! no lock. The meta flake points the agent's `hyperhive` input at its own, so -//! the lock holds one hyperhive. From then on the repo's owner keeps the config -//! current. The job graph is [`crate::jobq::declare_custom_job`]: repo and seed -//! first, the archetype key deleted only after the seed succeeded, then the -//! meta update and the deploy. A failure before the delete leaves the agent -//! archetype-built and working, and the same request can be sent again. A -//! failure after it leaves the agent custom, and the same request then only -//! re-queues the meta update and the deploy. - -use axum::Json; -use axum::extract::{Path, State}; -use axum::http::StatusCode; -use problem_details::ProblemDetails; -use serde::Serialize; -use swarm_queue_client::agent_archetype::ArchetypeRef; -use utoipa::ToSchema; - -use crate::routes::agents::{claiming_hives, read_declarations}; -use crate::routes::{AppState, error_problem}; -use swarm_wire::hive_status::{Capabilities, NO_CONFIG_CHECKOUT}; - -use crate::status::{Freshness, HiveStatus}; -use crate::{forge, meta_flake, schedules}; - -/// Answer of a queued conversion. -#[derive(Serialize, ToSchema)] -pub struct ConvertResponse { - /// A node of the queued graph; `/api/jobq/graph` serves the rest. - node_id: u64, -} - -/// Turn an archetype agent into a custom agent. -#[utoipa::path( - post, - path = "/api/agents/{name}/custom", - params(("name" = String, Path, description = "archetype agent to turn into a custom one")), - responses( - (status = 200, description = "the conversion is queued", body = ConvertResponse), - (status = 400, description = "the agent name is not shaped like an identifier (problem+json)", body = String), - (status = 409, description = "the agent is placed on no hive or on several, or its hive has not reported the capability a converted agent's rebuild needs (problem+json)", body = String), - (status = 503, description = "no swarm queue is configured here or it is not connected (problem+json)", body = String), - (status = 500, description = "the queue or the job graph failed (problem+json)", body = String), - ), - tag = "agents" -)] -pub(crate) async fn post_agent_custom( - State(state): State, - Path(agent): Path, -) -> Result, ProblemDetails> { - let agent = hive_types::Ident::parse(&agent) - .map_err(|reason| error_problem(StatusCode::BAD_REQUEST, reason))? - .into_string(); - let client = schedules::queue(&state).map_err(schedules::refused)?; - let internal = - |e: anyhow::Error| error_problem(StatusCode::INTERNAL_SERVER_ERROR, &format!("{e:#}")); - - let store = swarm_queue_client::agent_archetype::open_or_create(&client) - .await - .map_err(|e| internal(e.into()))?; - let (archetype, _, _) = schedules::read_or_default::>(&store, &agent) - .await - .map_err(internal)?; - let declared = read_declarations(&state, state.hives.iter()) - .await - .map_err(|(hive, e)| { - internal(e.context(format!("read where hive {hive} places agents"))) - })?; - let Some(status) = state.status.as_ref() else { - return Err(error_problem( - StatusCode::SERVICE_UNAVAILABLE, - "no swarm queue is configured on this host", - )); - }; - let rows = status - .view(&state.hives, std::time::SystemTime::now()) - .await - .map_err(internal)?; - let (archetype, hive) = plan(&agent, archetype, &declared, &rows) - .map_err(|why| error_problem(StatusCode::CONFLICT, &why))?; - - let queued = { - let mut sched = state - .jobq - .lock() - .unwrap_or_else(std::sync::PoisonError::into_inner); - sched.insert_job(None, |b| { - crate::jobq::declare_custom_job(b, &agent, &hive, archetype.as_ref()) - }) - }; - let ids = queued.map_err(|e| internal(anyhow::anyhow!("{e}")))?; - let [id] = ids[..] else { - unreachable!("exactly one handle was asked for"); - }; - tracing::info!(%agent, %hive, ?archetype, "turn into custom queued"); - Ok(Json(ConvertResponse { node_id: id.get() })) -} - -/// What the request does, or why it is refused: `archetype` is the agent's -/// stored one, and `None` for a custom agent, which a request only re-deploys -/// (the retry of a conversion whose build failed after the key was deleted). -/// The agent's hive is the one hive that places it, and must pass -/// [`offers_no_config_checkout`]. -fn plan( - agent: &str, - archetype: Option, - declared: &[(String, swarm_wire::wanted::HiveWanted)], - rows: &[HiveStatus], -) -> Result<(Option, String), String> { - let hive = match claiming_hives(agent, declared).as_slice() { - [hive] => hive.clone(), - other => { - return Err(format!( - "agent {agent:?} is placed on {} hives, so its hive is unknown", - other.len() - )); - } - }; - offers_no_config_checkout(rows, &hive)?; - Ok((archetype, hive)) -} - -/// Whether `hive`'s latest status advertises [`NO_CONFIG_CHECKOUT`]. An older -/// hive keeps a checkout of a custom agent's config and would create it for a -/// converted agent under the running container's mount, so a conversion waits -/// for this. -/// -/// # Errors -/// The reason, naming the hive: it has no recent status, or its status does -/// not list it. An older hive lists no capabilities at all. -fn offers_no_config_checkout(rows: &[HiveStatus], hive: &str) -> Result<(), String> { - let capability = NO_CONFIG_CHECKOUT; - let fresh = rows - .iter() - .find(|row| row.name == hive && row.freshness == Freshness::Fresh); - let Some(snapshot) = fresh.and_then(|row| row.snapshot.as_ref()) else { - return Err(format!( - "hive {hive:?} has no recent status, so whether it can rebuild a converted agent is \ - unknown; retry once it reports" - )); - }; - if Capabilities::of_snapshot(snapshot).has(capability) { - return Ok(()); - } - Err(format!( - "hive {hive:?} does not advertise the {capability:?} capability, so a converted agent \ - would get a config checkout on it; deploy this release to hive {hive:?} first" - )) -} - -/// What [`seed`] reads and writes on the forge: a fake in the tests. -pub(crate) trait SeedForge { - /// `internal/meta`'s `flake.nix` at its head. - fn meta_flake(&self) -> impl Future>> + Send; - /// `path` on the default branch of `agent`'s config repo; `None` when the - /// repo is empty or has no such file. - fn config_file( - &self, - agent: &str, - path: &str, - ) -> impl Future>> + Send; - /// Commit `files` to the default branch of `agent`'s config repo. - fn create_files( - &self, - agent: &str, - files: &[(&str, String)], - ) -> impl Future> + Send; -} - -impl SeedForge for forge::Client { - async fn meta_flake(&self) -> anyhow::Result> { - let head = self - .default_branch_files( - forge::objects::SHARED_ORG, - meta_flake::META_REPO, - &["flake.nix"], - ) - .await?; - Ok(head - .and_then(|h| h.files.into_iter().next()) - .map(|f| String::from_utf8_lossy(&f.bytes).into_owned())) - } - - async fn config_file(&self, agent: &str, path: &str) -> anyhow::Result> { - let head = self - .default_branch_files(forge::CONFIG_ORG, agent, &[path]) - .await?; - Ok(head - .and_then(|h| h.files.into_iter().next()) - .map(|f| String::from_utf8_lossy(&f.bytes).into_owned())) - } - - async fn create_files(&self, agent: &str, files: &[(&str, String)]) -> anyhow::Result<()> { - self.create_config_files(agent, files, "swarm-controller: turn into custom") - .await - } -} - -/// A nix identifier, so the names can be written as attribute names unquoted. -fn is_nix_ident(name: &str) -> bool { - let mut chars = name.chars(); - chars - .next() - .is_some_and(|c| c.is_ascii_lowercase() || c == '_') - && chars.all(|c| c.is_ascii_lowercase() || c.is_ascii_digit() || c == '_' || c == '-') -} - -/// The `flake.nix` of a converted agent: the shape every config repo has, plus -/// the archetype's source as an input at `url`, and the archetype's module -/// among the imports. The module is imported here, from `inputs`, because a -/// module cannot take its own imports from `_module.args`. -fn converted_flake_nix(archetype: &ArchetypeRef, url: &str) -> String { - let ArchetypeRef { source, name } = archetype; - format!( - "{{\n description = \"hyperhive agent\";\n inputs = {{\n {source}.url = \"{url}\";\n }};\n outputs =\n {{ self, ... }}@inputs:\n {{\n nixosModules.default = {{\n imports = [\n ./agent.nix\n inputs.{source}.hyperhiveArchetypes.{name}.module\n ];\n _module.args.flakeInputs = builtins.removeAttrs inputs [ \"self\" ];\n }};\n }};\n}}\n" - ) -} - -/// The `agent.nix` of a converted agent: an empty module for the owner's edits. -fn converted_agent_nix(agent: &str, archetype: &ArchetypeRef) -> String { - let ArchetypeRef { source, name } = archetype; - format!( - "{{ config, pkgs, lib, ... }}:\n{{\n # Per-agent overrides for {agent}. flake.nix imports the {source} archetype\n # {name}; keeping that input current is up to this repo's owner.\n}}\n" - ) -} - -/// Seed `agent`'s config repo so it builds what the archetype built: its -/// `flake.nix` declares the archetype's source as an input at the URL -/// `internal/meta` holds for it today (a pinned rev), and imports the module. -/// The job behind `SeedArchetypeConfig`. -/// -/// Safe to run again: a repo whose `flake.nix` is what this writes, whatever -/// URL it holds, is left as it is, since both files go in one commit. A -/// `flake.nix` that differs belongs to someone, and is an error. -/// -/// # Errors -/// A forge failure, a name that is not a nix identifier, `internal/meta` -/// holding no input for the source or one not pinned to a commit, or a different -/// `flake.nix` already in the repo. -pub(crate) async fn seed( - forge: &impl SeedForge, - agent: &str, - archetype: &ArchetypeRef, -) -> anyhow::Result<()> { - use anyhow::Context as _; - - let ArchetypeRef { source, name } = archetype; - for part in [source, name] { - anyhow::ensure!( - is_nix_ident(part), - "archetype name {part:?} is not a nix identifier" - ); - } - let meta = forge - .meta_flake() - .await? - .context("internal/meta has no flake.nix to read the archetype source from")?; - let url = hive_meta_flake::input_url(&meta, source).with_context(|| { - format!("internal/meta has no input {source:?}, so the archetype source has no URL to pin") - })?; - // The seeded repo has no lock, so the URL's `rev` is the only pin the input has. - anyhow::ensure!( - hive_meta_flake::pinned_rev(&meta, source).is_some(), - "internal/meta's input {source:?} is not pinned to a commit, so the converted agent would float" - ); - if let Some(have) = forge.config_file(agent, "flake.nix").await? { - let ours = hive_meta_flake::input_url(&have, source) - .is_some_and(|held| have == converted_flake_nix(archetype, held)); - anyhow::ensure!( - ours, - "agent-configs/{agent} already holds a different flake.nix; the conversion does not \ - overwrite it" - ); - tracing::debug!(%agent, "turn into custom: config already seeded"); - return Ok(()); - } - forge - .create_files( - agent, - &[ - ("agent.nix", converted_agent_nix(agent, archetype)), - ("flake.nix", converted_flake_nix(archetype, url)), - ], - ) - .await?; - tracing::info!(%agent, %url, "turn into custom: seeded the config repo"); - Ok(()) -} - -/// Delete `agent`'s archetype key, so the next render and rebuild treat it as -/// custom. The job behind `DropArchetype`; deleting an absent key succeeds. -/// -/// # Errors -/// The bucket cannot be opened or written. -pub(crate) async fn drop_archetype(client: &async_nats::Client, agent: &str) -> anyhow::Result<()> { - swarm_queue_client::agent_archetype::open_or_create(client) - .await? - .delete(agent) - .await?; - tracing::info!(%agent, "turn into custom: archetype key deleted"); - Ok(()) -} - -#[cfg(test)] -mod tests { - use super::*; - use std::collections::BTreeMap; - use std::sync::Mutex; - - const URL: &str = - "git+http://forge/hyperhive/hyperhive.git?rev=1111111111111111111111111111111111111111"; - - fn reviewer() -> ArchetypeRef { - ArchetypeRef { - source: "hyperhive".to_owned(), - name: "reviewer".to_owned(), - } - } - - /// An in-memory forge: `internal/meta`'s flake and the files of one config - /// repo. `fail_create` makes the next `create_files` fail once, without - /// writing. - #[derive(Default)] - struct FakeForge { - meta_flake: Option, - repo: Mutex>, - fail_create: Mutex, - creates: Mutex, - } - - impl FakeForge { - fn with_inputs(inputs: &[(&str, &str)]) -> Self { - use std::fmt::Write as _; - let mut lines = String::new(); - for (name, url) in inputs { - let _ = writeln!(lines, " {name}.url = \"{url}\";"); - } - Self { - meta_flake: Some(lines), - ..Self::default() - } - } - - fn pinned() -> Self { - Self::with_inputs(&[("hyperhive", URL)]) - } - } - - impl SeedForge for FakeForge { - async fn meta_flake(&self) -> anyhow::Result> { - Ok(self.meta_flake.clone()) - } - async fn config_file(&self, _agent: &str, path: &str) -> anyhow::Result> { - Ok(self.repo.lock().unwrap().get(path).cloned()) - } - async fn create_files(&self, _agent: &str, files: &[(&str, String)]) -> anyhow::Result<()> { - if std::mem::take(&mut *self.fail_create.lock().unwrap()) { - anyhow::bail!("the forge refused the commit"); - } - *self.creates.lock().unwrap() += 1; - let mut repo = self.repo.lock().unwrap(); - for (path, content) in files { - repo.insert((*path).to_owned(), content.clone()); - } - Ok(()) - } - } - - /// The seeded flake declares the source as an input at meta's URL and - /// imports the module from it; nothing is copied into the repo. - #[tokio::test] - async fn the_seed_declares_the_source_as_an_input_and_imports_its_module() { - let forge = FakeForge::pinned(); - seed(&forge, "atlas", &reviewer()).await.expect("seeds"); - let repo = forge.repo.lock().unwrap(); - let flake = &repo["flake.nix"]; - assert!( - flake.contains(&format!(" hyperhive.url = \"{URL}\";")), - "{flake}" - ); - assert!( - flake.contains("inputs.hyperhive.hyperhiveArchetypes.reviewer.module"), - "{flake}" - ); - assert!(flake.contains("_module.args.flakeInputs"), "{flake}"); - assert_eq!( - repo.keys().map(String::as_str).collect::>(), - ["agent.nix", "flake.nix"] - ); - } - - /// A source other than hyperhive takes the URL `internal/meta` holds for - /// that name, and one meta has no input for is an error. - #[tokio::test] - async fn another_source_takes_its_url_from_meta_and_an_unknown_one_fails() { - let other = ArchetypeRef { - source: "extra".to_owned(), - name: "builder".to_owned(), - }; - let forge = FakeForge::with_inputs(&[ - ("hyperhive", URL), - ( - "extra", - "git+http://x/y.git?rev=2222222222222222222222222222222222222222", - ), - ]); - seed(&forge, "atlas", &other).await.expect("seeds"); - let flake = forge.repo.lock().unwrap()["flake.nix"].clone(); - assert!( - flake.contains(" extra.url = \"git+http://x/y.git?rev=2222"), - "{flake}" - ); - assert!( - flake.contains("inputs.extra.hyperhiveArchetypes.builder.module"), - "{flake}" - ); - - let why = format!( - "{:#}", - seed(&FakeForge::pinned(), "atlas", &other) - .await - .unwrap_err() - ); - assert!(why.contains("no input \"extra\""), "{why}"); - } - - /// A seed that failed wrote nothing, and sending it again completes; a - /// third run, after the repo is seeded, commits nothing more — even when - /// meta's pin has moved since. The presence control: the second run did - /// write. - #[tokio::test] - async fn a_seed_that_failed_is_retried_and_a_seeded_repo_is_left_alone() { - let forge = FakeForge::pinned(); - *forge.fail_create.lock().unwrap() = true; - seed(&forge, "atlas", &reviewer()) - .await - .expect_err("refused"); - assert!(forge.repo.lock().unwrap().is_empty(), "a failed seed wrote"); - - seed(&forge, "atlas", &reviewer()).await.expect("retry"); - assert_eq!(*forge.creates.lock().unwrap(), 1); - - let moved = FakeForge { - repo: Mutex::new(forge.repo.lock().unwrap().clone()), - ..FakeForge::with_inputs(&[( - "hyperhive", - "git+http://forge/h.git?rev=3333333333333333333333333333333333333333", - )]) - }; - seed(&moved, "atlas", &reviewer()).await.expect("again"); - assert_eq!( - *moved.creates.lock().unwrap(), - 0, - "a seeded repo was committed to again" - ); - } - - /// Someone else's `flake.nix` is never overwritten. - #[tokio::test] - async fn a_different_flake_nix_is_an_error() { - let forge = FakeForge::pinned(); - forge - .repo - .lock() - .unwrap() - .insert("flake.nix".to_owned(), "{ }".to_owned()); - let why = format!( - "{:#}", - seed(&forge, "atlas", &reviewer()).await.unwrap_err() - ); - assert!(why.contains("different flake.nix"), "{why}"); - assert_eq!(*forge.creates.lock().unwrap(), 0); - } - - #[tokio::test] - async fn a_missing_meta_flake_or_a_bad_name_is_an_error() { - let forge = FakeForge::default(); - seed(&forge, "atlas", &reviewer()) - .await - .expect_err("no meta flake"); - let bad = ArchetypeRef { - source: "hyperhive".to_owned(), - name: "9; evil".to_owned(), - }; - seed(&FakeForge::pinned(), "atlas", &bad) - .await - .expect_err("bad name"); - } - - fn row(name: &str, freshness: Freshness, snapshot: Option) -> HiveStatus { - HiveStatus { - name: name.to_owned(), - domain: None, - dashboard: None, - freshness, - last_seen_unix: None, - age_seconds: None, - snapshot, - } - } - - /// The rollout gate: a hive listing the capability is allowed; one that - /// lists others, lists none (an older hive's payload), has gone quiet or - /// never reported is refused, naming the hive. - #[test] - fn only_a_fresh_hive_listing_the_capability_may_convert() { - let with = serde_json::json!({"status": "ok", "warnings": [], "capabilities": ["no-config-checkout"]}); - let old = serde_json::json!({"status": "ok", "warnings": []}); - let other = serde_json::json!({"status": "ok", "warnings": [], "capabilities": ["x"]}); - let rows = |hive: Freshness, snapshot| { - vec![ - row("alpha", hive, Some(snapshot)), - row("beta", Freshness::Fresh, Some(with.clone())), - ] - }; - - assert_eq!( - offers_no_config_checkout(&rows(Freshness::Fresh, with.clone()), "alpha"), - Ok(()) - ); - for refused in [old.clone(), other] { - let why = - offers_no_config_checkout(&rows(Freshness::Fresh, refused), "alpha").unwrap_err(); - assert!( - why.contains("\"alpha\"") && why.contains("no-config-checkout"), - "{why}" - ); - } - let stale = - offers_no_config_checkout(&rows(Freshness::Stale, with.clone()), "alpha").unwrap_err(); - assert!(stale.contains("no recent status"), "{stale}"); - let never = vec![row("alpha", Freshness::NeverReported, None)]; - offers_no_config_checkout(&never, "alpha").unwrap_err(); - offers_no_config_checkout(&[], "alpha").unwrap_err(); - } - - /// An old-shaped payload, without `capabilities`, still parses as the - /// opaque snapshot the status reader serves. - #[test] - fn an_old_shaped_payload_still_parses_and_lists_nothing() { - let old: serde_json::Value = - serde_json::from_str(r#"{"status":"ok","warnings":[]}"#).unwrap(); - assert!(old["capabilities"].as_array().is_none()); - let why = offers_no_config_checkout(&[row("alpha", Freshness::Fresh, Some(old))], "alpha") - .unwrap_err(); - assert!(why.contains("alpha"), "{why}"); - } - fn placing(hive: &str, agent: &str) -> Vec<(String, swarm_wire::wanted::HiveWanted)> { - let wanted = swarm_wire::wanted::HiveWanted { - agents: BTreeMap::from([( - agent.to_owned(), - swarm_wire::wanted::AgentWanted { - state: swarm_wire::wanted::AgentState::Up, - }, - )]), - }; - vec![(hive.to_owned(), wanted)] - } - - fn reporting(hive: &str, capabilities: &[&str]) -> Vec { - let snapshot = - serde_json::json!({"status": "ok", "warnings": [], "capabilities": capabilities}); - vec![row(hive, Freshness::Fresh, Some(snapshot))] - } - - /// The refusals and the outcomes of a request, decided before anything is - /// queued: an agent no hive places and a hive that doesn't list the - /// capability are refused; an archetype agent converts, and a custom agent - /// (its key already gone) is only re-deployed. The conversion is the - /// control that the refusals' inputs get through otherwise. - #[test] - fn a_request_is_refused_for_an_unplaced_agent_or_an_old_hive() { - let placed = placing("alpha", "atlas"); - let ready = reporting("alpha", &["no-config-checkout"]); - assert_eq!( - plan("atlas", Some(reviewer()), &placed, &ready), - Ok((Some(reviewer()), "alpha".to_owned())) - ); - assert_eq!( - plan("atlas", None, &placed, &ready), - Ok((None, "alpha".to_owned())) - ); - - let unplaced = plan("atlas", Some(reviewer()), &[], &ready).unwrap_err(); - assert!(unplaced.contains("0 hives"), "{unplaced}"); - let old = plan("atlas", Some(reviewer()), &placed, &reporting("alpha", &[])).unwrap_err(); - assert!(old.contains("\"alpha\""), "{old}"); - // The retry of a custom agent waits for the capability too. - plan("atlas", None, &placed, &reporting("alpha", &[])).unwrap_err(); - } - - /// A source `internal/meta` does not pin to a commit would float in a repo - /// with no lock, so the seed refuses it and writes nothing. - #[tokio::test] - async fn a_source_that_meta_does_not_pin_is_not_seeded() { - let other = ArchetypeRef { - source: "extra".to_owned(), - name: "builder".to_owned(), - }; - let forge = FakeForge::with_inputs(&[("hyperhive", URL), ("extra", "git+http://x/y.git")]); - let why = format!("{:#}", seed(&forge, "atlas", &other).await.unwrap_err()); - assert!(why.contains("not pinned"), "{why}"); - assert!(forge.repo.lock().unwrap().is_empty()); - } -} diff --git a/swarm-controller/src/agent_status.rs b/swarm-controller/src/agent_status.rs index 705f36efd..4f7036272 100644 --- a/swarm-controller/src/agent_status.rs +++ b/swarm-controller/src/agent_status.rs @@ -73,14 +73,6 @@ pub struct AgentStatusRow { /// `AppState::agent_login` — this reader sees the hive's bucket, and the /// flag comes from the agent itself. pub needs_login: Option, - /// The archetype the agent is built from; `None` for a custom agent. Always - /// `None` out of this reader, filled in by the `GET /api/agents/status` - /// handler from the `agent-archetype` bucket. - #[schema( - value_type = Option, - example = json!({"source": "hyperhive", "name": "reviewer"}) - )] - pub archetype: Option, } /// A snapshot as retained by the bucket, with the hive it was published @@ -202,7 +194,6 @@ fn render( config_pr: None, wanted: None, needs_login: None, - archetype: None, }, }) .collect(); @@ -248,7 +239,6 @@ fn row( config_pr: None, wanted: None, needs_login: None, - archetype: None, } } diff --git a/swarm-controller/src/forge.rs b/swarm-controller/src/forge.rs index ea78b4a05..3092c518a 100644 --- a/swarm-controller/src/forge.rs +++ b/swarm-controller/src/forge.rs @@ -807,38 +807,22 @@ impl Client { tracing::debug!(%repo, "swarm forge: config already seeded"); return Ok(()); } - self.create_config_files( - repo, - &[ - ("agent.nix", initial_agent_nix(agent)), - ("flake.nix", initial_flake_nix().to_owned()), - ], - "swarm-controller: seed agent config", - ) - .await?; - tracing::info!(%repo, %agent, "swarm forge: seeded agent.nix + flake.nix"); - Ok(()) - } - - /// Commit `files` (path, content) to `repo`'s default branch in one - /// `repo_change_files` call, creating each path. No presence check: a - /// path that already exists makes the forge refuse the whole commit. - pub(crate) async fn create_config_files( - &self, - repo: &str, - files: &[(&str, String)], - message: &str, - ) -> Result<()> { - let files: Vec = files - .iter() - .map(|(path, content)| ChangeFileOperation { - content: Some(base64_encode(content)), + let files = vec![ + ChangeFileOperation { + content: Some(base64_encode(&initial_agent_nix(agent))), from_path: None, operation: ChangeFileOperationOperation::Create, - path: (*path).to_owned(), + path: "agent.nix".to_owned(), sha: None, - }) - .collect(); + }, + ChangeFileOperation { + content: Some(base64_encode(initial_flake_nix())), + from_path: None, + operation: ChangeFileOperationOperation::Create, + path: "flake.nix".to_owned(), + sha: None, + }, + ]; self.api .call(|api| { api.repo_change_files( @@ -851,7 +835,7 @@ impl Client { dates: None, files: files.clone(), force_overwrite_new_branch: None, - message: Some(message.to_owned()), + message: Some("swarm-controller: seed agent config".to_owned()), new_branch: None, signoff: None, }, @@ -859,7 +843,8 @@ impl Client { .into_future() }) .await - .with_context(|| format!("commit config files to {CONFIG_ORG}/{repo}"))?; + .with_context(|| format!("seed config files in {CONFIG_ORG}/{repo}"))?; + tracing::info!(%repo, %agent, "swarm forge: seeded agent.nix + flake.nix"); Ok(()) } diff --git a/swarm-controller/src/jobq.rs b/swarm-controller/src/jobq.rs index cd21a7380..0d6c346d4 100644 --- a/swarm-controller/src/jobq.rs +++ b/swarm-controller/src/jobq.rs @@ -57,15 +57,6 @@ pub(crate) enum SwarmNodeKind { /// swarm routes a deploy message to — it belongs on the node that /// sends that message, not on this one. InitAgentConfigRepo { agent: String }, - /// Seed the agent's config repo with the archetype's module, for turn - /// into custom. See `agent_custom::seed`. - SeedArchetypeConfig { - agent: String, - archetype: swarm_wire::agent_archetype::ArchetypeRef, - }, - /// Delete the agent's archetype key, which makes it custom. See - /// `agent_custom::drop_archetype`. - DropArchetype { agent: String }, /// Mint `agent`'s own client certificate for the swarm secret store, /// publish it there, grant it, and log in with it. See /// `agent_identity::mint_and_verify` — including why this node does not @@ -177,8 +168,6 @@ impl hive_jobq_wire::WireNode for SwarmNodeKind { SwarmNodeKind::AddRepoMember { .. } => "add_repo_member".to_owned(), SwarmNodeKind::GrantDocsRead { .. } => "grant_docs_read".to_owned(), SwarmNodeKind::InitAgentConfigRepo { .. } => "init_agent_config_repo".to_owned(), - SwarmNodeKind::SeedArchetypeConfig { .. } => "seed_archetype_config".to_owned(), - SwarmNodeKind::DropArchetype { .. } => "drop_archetype".to_owned(), SwarmNodeKind::MintAgentIdentity { .. } => "mint_agent_identity".to_owned(), SwarmNodeKind::MintAgentForgeToken { .. } => "mint_agent_forge_token".to_owned(), SwarmNodeKind::MintAgentMatrixAccount { .. } => "mint_agent_matrix_account".to_owned(), @@ -213,7 +202,6 @@ impl hive_jobq_wire::WireNode for SwarmNodeKind { | SwarmNodeKind::AddRepoMember { agent } | SwarmNodeKind::GrantDocsRead { agent } | SwarmNodeKind::InitAgentConfigRepo { agent } - | SwarmNodeKind::DropArchetype { agent } | SwarmNodeKind::MintAgentIdentity { agent } | SwarmNodeKind::MintAgentForgeToken { agent } | SwarmNodeKind::MintAgentMatrixAccount { agent } @@ -223,9 +211,6 @@ impl hive_jobq_wire::WireNode for SwarmNodeKind { SwarmNodeKind::BuildMeta { agent, pr } => { serde_json::json!({ "agent": agent, "pr": pr }) } - SwarmNodeKind::SeedArchetypeConfig { agent, archetype } => { - serde_json::json!({ "agent": agent, "archetype": archetype.name }) - } SwarmNodeKind::MintHiveSenderToken { hive } | SwarmNodeKind::MintHiveForgeToken { hive } => serde_json::json!({ "hive": hive }), SwarmNodeKind::MintClassifierForgeToken @@ -419,9 +404,6 @@ async fn run_node_work( }, SwarmNodeKind::GrantDocsRead { agent } => grant_docs_read(forge, &agent).await, SwarmNodeKind::InitAgentConfigRepo { agent } => init_agent_config_repo(forge, &agent).await, - SwarmNodeKind::SeedArchetypeConfig { .. } | SwarmNodeKind::DropArchetype { .. } => { - convert_node(&kind, forge, deps.queue.as_ref()).await - } SwarmNodeKind::MintAgentIdentity { agent } => { mint_identity(deps.wanted.as_deref(), &agent).await } @@ -610,44 +592,6 @@ async fn init_agent_config_repo( } } -/// The `SeedArchetypeConfig` and `DropArchetype` arms, lifted out for the -/// same reason. Any other kind is not this function's to run. -async fn convert_node( - kind: &SwarmNodeKind, - forge: Option>, - queue: Option<&async_nats::Client>, -) -> hive_jobq::scheduler::Outcome { - use hive_jobq::scheduler::Outcome; - - let done = |result: Result<()>| match result { - Ok(()) => Outcome::Done, - Err(e) => Outcome::Failed(format!("{e:#}")), - }; - match kind { - SwarmNodeKind::SeedArchetypeConfig { agent, archetype } => { - let Some(client) = forge else { - return Outcome::Failed( - "no forge access here (SWARM_CONTROLLER_FORGE_URL unset, or its token \ - not in the swarm secret store)" - .to_owned(), - ); - }; - done( - forge_node( - "seed_archetype_config", - crate::agent_custom::seed(client.as_ref(), agent, archetype), - ) - .await, - ) - } - SwarmNodeKind::DropArchetype { agent } => match queue { - Some(client) => done(crate::agent_custom::drop_archetype(client, agent).await), - None => Outcome::Failed("no swarm queue is configured on this host".to_owned()), - }, - other => unreachable!("{other:?} is not a conversion node"), - } -} - /// The `RenewAgentQueueCredential` arm, lifted out for the same reason. async fn renew_queue_credential(agent: &str) -> hive_jobq::scheduler::Outcome { use hive_jobq::scheduler::Outcome; @@ -1477,65 +1421,6 @@ pub(crate) fn declare_agent_job( vec![create_identity.guid()] } -/// The sub-DAG turning the archetype agent `agent` into a custom one: its -/// config repo is created and seeded, and only then is its archetype key -/// deleted, so a failed step leaves the agent archetype-built. The meta -/// update and the deploy follow the delete (`after_ok`), as in -/// [`merged_deploy_job`]. Returns the repo node, the handle the response -/// reports. -/// -/// `CreateRepo`, `CreateForgeUser` and `AddRepoMember` are no-ops for what -/// already exists, so the same graph can be queued again after a failure. -/// -/// With `archetype` `None` the agent is already custom: its key was deleted and -/// a later step failed, so the graph is the meta update and the deploy alone, -/// and the handle is the update. A build that fails after the key delete leaves -/// the agent custom, which is how that failure is retried. -pub(crate) fn declare_custom_job( - b: &hive_jobq::builder::JobBuilder, - agent: &str, - hive: &str, - archetype: Option<&swarm_wire::agent_archetype::ArchetypeRef>, -) -> Vec { - let Some(archetype) = archetype else { - let update_meta = b - .node(SwarmNodeKind::UpdateMeta) - .needs(SwarmResourceKind::Meta); - let _deploy = gated_deploy(b, hive, agent, None, None, Some(update_meta.guid())); - return vec![update_meta.guid()]; - }; - let create_repo = b.node(SwarmNodeKind::CreateRepo { - agent: agent.to_owned(), - }); - let create_forge_user = b.node(SwarmNodeKind::CreateForgeUser { - agent: agent.to_owned(), - }); - let add_repo_member = b - .node(SwarmNodeKind::AddRepoMember { - agent: agent.to_owned(), - }) - .after_ok(create_repo) - .after_ok(create_forge_user); - let seed = b - .node(SwarmNodeKind::SeedArchetypeConfig { - agent: agent.to_owned(), - archetype: archetype.clone(), - }) - .after_ok(create_repo); - let drop_archetype = b - .node(SwarmNodeKind::DropArchetype { - agent: agent.to_owned(), - }) - .after_ok(seed) - .after_ok(add_repo_member); - let update_meta = b - .node(SwarmNodeKind::UpdateMeta) - .needs(SwarmResourceKind::Meta) - .after_ok(drop_archetype); - let _deploy = gated_deploy(b, hive, agent, None, None, Some(update_meta.guid())); - vec![create_repo.guid()] -} - /// Insert one job per agent: `CreateForgeUser`, then `MintAgentForgeToken` /// `after_ok` it — the same edge [`declare_agent_job`] uses. Returns the mint /// nodes' ids. @@ -2504,87 +2389,6 @@ mod tests { let _ = super::merged_deploy_job(b, "pr1ma", "atlas", "abc", 7); } - fn custom_dag(b: &TestBuilder) { - let archetype = swarm_wire::agent_archetype::ArchetypeRef { - source: "hyperhive".to_owned(), - name: "reviewer".to_owned(), - }; - let _ = super::declare_custom_job(b, "atlas", "pr1ma", Some(&archetype)); - } - - /// Turn into custom: with every node succeeding the agent is seeded, loses - /// its key, and is deployed. - #[tokio::test] - async fn a_conversion_seeds_then_drops_the_key_then_deploys() { - use hive_jobq::State; - - let done = run_dag_failing(custom_dag, None).await; - for label in [ - "create_repo", - "seed_archetype_config", - "drop_archetype", - "update_meta", - "trigger_deploy", - ] { - assert_eq!(state_of(&done, label), State::Done, "{label}"); - } - // The key is deleted only after the seed and the member grant, and - // each edge refuses a failure. The deploy edge is the control that - // the lookup finds an `after_ok` edge in this graph. - for (from, on) in [ - ("drop_archetype", "seed_archetype_config"), - ("drop_archetype", "add_repo_member"), - ("update_meta", "drop_archetype"), - ("trigger_deploy", "update_meta"), - ] { - assert!( - !edge_accepts_failure(custom_dag, from, on), - "{from} -> {on}" - ); - } - } - - /// The retry of a conversion whose build failed after the key was deleted: - /// with no archetype the graph is the meta update and the deploy, with no - /// repo, seed or key delete. The conversion is the control that those - /// nodes exist when there is one. - #[tokio::test] - async fn a_retry_of_a_custom_agent_is_only_the_update_and_the_deploy() { - use hive_jobq::State; - - fn retry_dag(b: &TestBuilder) { - let _ = super::declare_custom_job(b, "atlas", "pr1ma", None); - } - let done = run_dag_failing(retry_dag, None).await; - let labels: Vec<&str> = done.iter().map(|(l, _)| l.as_str()).collect(); - for absent in ["create_repo", "seed_archetype_config", "drop_archetype"] { - assert!(!labels.contains(&absent), "{absent} in {labels:?}"); - } - assert_eq!(state_of(&done, "update_meta"), State::Done); - assert_eq!(state_of(&done, "trigger_deploy"), State::Done); - let converting = run_dag_failing(custom_dag, None).await; - assert!(converting.iter().any(|(l, _)| l == "drop_archetype")); - // A failed build leaves the deploy skipped, so a retry has work to do. - let red = run_dag_failing(retry_dag, Some("build_meta")).await; - assert_eq!(state_of(&red, "trigger_deploy"), State::Skipped); - } - - /// A seed that fails leaves the key in place and nothing deployed; the - /// same graph, queued again with the seed succeeding, completes. - #[tokio::test] - async fn a_failed_seed_keeps_the_key_and_a_retry_completes() { - use hive_jobq::State; - - let failed = run_dag_failing(custom_dag, Some("seed_archetype_config")).await; - assert_eq!(state_of(&failed, "seed_archetype_config"), State::Failed); - for label in ["drop_archetype", "update_meta", "trigger_deploy"] { - assert_eq!(state_of(&failed, label), State::Skipped, "{label}"); - } - let retried = run_dag_failing(custom_dag, None).await; - assert_eq!(state_of(&retried, "drop_archetype"), State::Done); - assert_eq!(state_of(&retried, "trigger_deploy"), State::Done); - } - /// A deploy needs the meta update to succeed: its edge refuses a failed /// update, in both DAGs that queue one. #[test] diff --git a/swarm-controller/src/main.rs b/swarm-controller/src/main.rs index 46d1cdc92..bac047425 100644 --- a/swarm-controller/src/main.rs +++ b/swarm-controller/src/main.rs @@ -23,7 +23,6 @@ use anyhow::{Context, Result}; mod agent_archetype; mod agent_control; -mod agent_custom; mod agent_entity; mod agent_icon; mod agent_identity; diff --git a/swarm-controller/src/meta_flake.rs b/swarm-controller/src/meta_flake.rs index 0172a4e5f..d0232c134 100644 --- a/swarm-controller/src/meta_flake.rs +++ b/swarm-controller/src/meta_flake.rs @@ -418,29 +418,26 @@ async fn render( CONFIG_ORG, &agent.name, Some(CONFIG_BRANCH), - &["flake.lock", "flake.nix"], + &["flake.lock"], ) .await? else { tracing::warn!(agent = %agent.name, "meta flake: its config repo has no commit on {CONFIG_BRANCH}; leaving the agent out"); continue; }; - let read = |path: &str| { - head.files - .iter() - .find(|f| f.path == path) - .map(|f| String::from_utf8_lossy(&f.bytes).into_owned()) - }; - let (lock, flake) = (read("flake.lock"), read("flake.nix")); + let lock = head + .files + .first() + .map(|f| String::from_utf8_lossy(&f.bytes).into_owned()); Some(hive_meta_flake::AgentPin { url: hive_meta_flake::pinned_git_url( &settings.git_url(CONFIG_ORG, &agent.name), Some(&head.commit), )?, - canonical_inputs: hive_meta_flake::canonical_inputs_declared( - lock.as_deref(), - flake.as_deref(), - ), + canonical_inputs: lock + .as_deref() + .map(hive_meta_flake::canonical_inputs_in_lock) + .unwrap_or_default(), }) }; specs.push(hive_meta_flake::AgentSpec { diff --git a/swarm-controller/src/routes/agents.rs b/swarm-controller/src/routes/agents.rs index 4c8baf013..e44c7131c 100644 --- a/swarm-controller/src/routes/agents.rs +++ b/swarm-controller/src/routes/agents.rs @@ -472,21 +472,6 @@ pub(crate) async fn get_agents_status( row.needs_login = login.get(&row.name); } } - // Absent on a failed read, like `wanted`: one unreadable bucket - // must not blank the roster. - if let Some(reader) = state.status.as_ref() { - match agent_archetype::of_agents(&reader.queue_client(), &rows).await { - Ok(mut archetypes) => { - for row in &mut rows { - row.archetype = archetypes.remove(&row.name); - } - } - Err(e) => tracing::warn!( - error = %format!("{e:#}"), - "reading the agent archetypes failed; rows show none" - ), - } - } Ok(Json(rows)) } Err(e) => { @@ -1815,7 +1800,6 @@ mod tests { config_pr: None, wanted: None, needs_login: None, - archetype: None, } } diff --git a/swarm-controller/src/startup.rs b/swarm-controller/src/startup.rs index dc8e2627e..4adf557ff 100644 --- a/swarm-controller/src/startup.rs +++ b/swarm-controller/src/startup.rs @@ -28,12 +28,12 @@ use crate::jobq::{ use crate::routes::hives::{HiveEntry, ServiceLink}; use crate::routes::{self, AppState, agents, config_prs, forge_admin, hives, jobq_graph}; use crate::{ - agent_control, agent_custom, agent_icon, agent_input, agent_limits, agent_login, - agent_permissions, agent_renewal, agent_state_stream, agent_status, agents_md, alerts, - archetypes, auth, backend_env, builder, claude_login, config_pr, egress_allow, forge, - forge_account, github_account, inference, issue_report, linked_accounts, matrix_account, - message_stream, notify_flows, notify_router_api, pulls, queue_identity, review_classifier, - schedules, status, subagent_term, term_stream, wanted, webhook, + agent_control, agent_icon, agent_input, agent_limits, agent_login, agent_permissions, + agent_renewal, agent_state_stream, agent_status, agents_md, alerts, archetypes, auth, + backend_env, builder, claude_login, config_pr, egress_allow, forge, forge_account, + github_account, inference, issue_report, linked_accounts, matrix_account, message_stream, + notify_flows, notify_router_api, pulls, queue_identity, review_classifier, schedules, status, + subagent_term, term_stream, wanted, webhook, }; /// Where the daemon binds, overridable via `SWARM_CONTROLLER_SOCKET`. @@ -656,7 +656,6 @@ pub(crate) fn build_app(state: AppState) -> axum::Router { .routes(routes!(agent_input::post_agent_input)) .routes(routes!(agent_input::post_agents_broadcast)) .routes(routes!(agent_control::post_agent_cancel)) - .routes(routes!(agent_custom::post_agent_custom)) .routes(routes!( claude_login::post_claude_login, claude_login::delete_claude_login diff --git a/swarm-wire/src/hive_status.rs b/swarm-wire/src/hive_status.rs index 8da854591..babb5e6db 100644 --- a/swarm-wire/src/hive_status.rs +++ b/swarm-wire/src/hive_status.rs @@ -12,12 +12,6 @@ /// this. pub const TOLERANT_PERMISSIONS: &str = "tolerant-permissions"; -/// A hive lists this when it builds a custom agent from `internal/meta` alone -/// and keeps no checkout of its config: no `applied` or `proposed` repo and no -/// config mount in the container. swarm-controller turns an archetype agent -/// into a custom one only on a hive that advertises it. -pub const NO_CONFIG_CHECKOUT: &str = "no-config-checkout"; - /// The part of a hive's status snapshot this module describes. #[derive(Debug, Clone, Default, PartialEq, Eq, serde::Serialize, serde::Deserialize)] pub struct Capabilities {