Watch
0
0
Fork
You've already forked hyperhive
0
hyperhive/swarm-controller/src/forge/objects.rs
atlas f1c695c212 config PRs: an operator's Forgejo merge deploys the merged rev
A config PR merged in the Forgejo UI changed nothing on the hive: the
hive's webhook ignores `closed`, its poll then cancels the dashboard
card, and `applied/main` stays where it was.

swarm-controller reads `merged`/`merge_commit_sha` off the
`pull_request` delivery it already receives for `agent-configs`, finds
the hive placing the agent by scanning every hive's wanted state (the
scan `declarations_elsewhere` already ran, factored out), and queues a
`TriggerDeploy` carrying the rev. Zero or several claimants deploy
nothing and log the claimants.

`DeployRequest` gains `rev: Option<String>` with `serde(default)`, so
rev-less payloads from either side keep decoding.

hive-c0re, given a rev for an agent it runs: a no-op when
`applied/main` already is the rev (a dashboard merge deploys its own
PR); otherwise it fetches the forge `main` with the core token,
requires the rev to descend from `applied/main` (the ancestry gate,
factored out of `run_deploy_merge_verify`), fast-forwards by CAS and
queues the usual relocking rebuild. No eval-verify on this path, per
mara (#4850 c90075). A refusal is commented on the PR that merged the
rev, found by commit.

swarm-controller's forge-objects pass converges every config repo's
`main` rule to merge whitelist `operators` + `core` and approval
whitelist `operators`. The hive's boot PATCH stops forcing
`enable_approvals_whitelist` off, so the two do not fight.

Refs #4850
2026-10-02 23:13:03 +02:00

1392 lines
52 KiB
Rust

//! The swarm-wide forge objects: the three seeded orgs (plus every mirror's
//! owner org), the `operators` merge-gate team in `agents` and
//! `agent-configs`, the merge gate on every `agent-configs` repo's `main`, the
//! operator-declared pull-mirrors, `internal/docs`, `internal/knowledge`
//! (public, README-seeded) and the `agent-configs` org avatar.
//!
//! Every hive's `hive-c0re` used to ensure these in its boot sweep, but only
//! the one hive co-located with the forge container ever did (the sweep bails
//! without a local `hive-forge` container), and only as `core`, a site-admin
//! token that hive should not need. They are one set per forge, not per hive,
//! so they are ensured here, at start and every [`RECONCILE_INTERVAL`] after.
//!
//! Split into observe → [`plan`] → apply so the decision is pure: [`plan`]
//! takes what the forge reported and returns the writes, which is what the
//! tests pin. The IO on either side only reads or executes.
//!
//! A pass that fails anywhere logs each failure at `warn` plus a summary line,
//! and the next tick retries. Nothing here stops the daemon.
use std::collections::BTreeMap;
use std::path::PathBuf;
use std::sync::Arc;
use anyhow::{Context, Result};
use base64::Engine as _;
use forgejo_api::structs::{
BranchProtection, ChangeFileOperation, ChangeFileOperationOperation, ChangeFilesOptions,
CreateOrgOption, CreateTeamOption, CreateTeamOptionPermission, EditBranchProtectionOption,
EditRepoOption, EditTeamOption, EditTeamOptionPermission, MigrateRepoOptions,
MigrateRepoOptionsService, Team, TeamPermission, UpdateUserAvatarOption,
};
use forgejo_api::{ApiErrorKind, ForgejoError};
use reqwest::StatusCode;
use serde::Deserialize;
use super::legacy_tokens::CORE_USER;
use super::{
CONFIG_ORG, Client, KNOWLEDGE_ORG, KNOWLEDGE_REPO, OPERATORS_TEAM, base64_encode,
folds_into_success, is_ambiguous_validation_failure, is_confirmed_conflict,
};
/// Forgejo org that owns agent-created repos. Agents can't create repos with
/// their own token (`max_repo_creation = 0`); a repo an agent asks for is
/// created here and the agent added as a **write** member (not owner/admin).
/// Because the org, not the agent, owns the repo, perms stay centrally
/// managed and branch protection (referencing [`OPERATORS_TEAM`]) can block
/// the author from merging their own PR.
const AGENTS_ORG: &str = "agents";
/// Org hosting the operator-curated shared repos. Same org as the knowledge
/// repo's, named separately for the docs repo it also holds.
const SHARED_ORG: &str = KNOWLEDGE_ORG;
/// The shared docs repo inside [`SHARED_ORG`]. **Private**: every agent gets
/// read-only collaborator access, granted per agent by its hive. Agents use it
/// as a common reference without the operator having to bake content into the
/// system prompt.
const SHARED_DOCS_REPO: &str = "docs";
/// The orgs ensured on every pass. The meta repo lives at `core/meta`, the
/// `core` user's own namespace, so no org is needed for it (and it is not a
/// swarm object: it stays with `hive-c0re`).
const SEEDED_ORGS: [&str; 3] = [CONFIG_ORG, SHARED_ORG, AGENTS_ORG];
/// Where the [`OPERATORS_TEAM`] must exist. Gitea teams are org-scoped, so a
/// config-repo branch-protection rule referencing `operators` needs the team
/// in `agent-configs` too. Missing it there 422'd every config-repo
/// protection apply, leaving those repos unprotected, so operator-merged
/// config PRs bypassed the deploy pipeline and silently didn't apply.
const OPERATORS_TEAM_ORGS: [&str; 2] = [AGENTS_ORG, CONFIG_ORG];
const OPERATORS_TEAM_DESCRIPTION: &str = "hyperhive operators — merge gate for agent repos";
/// Repo-unit access flags for the `operators` team. Explicit list so Forgejo
/// doesn't reject a null/absent `units` field; a `write`-permission team needs
/// at least `repo.code` + `repo.pulls` to review and merge PRs.
const OPERATORS_TEAM_UNITS: [&str; 7] = [
"repo.code",
"repo.issues",
"repo.pulls",
"repo.releases",
"repo.wiki",
"repo.projects",
"repo.packages",
];
/// Periodic sync interval for pull-mirrors. Forgejo syncs mirrors on-access by
/// default, which re-introduces external DNS latency on every `git clone` (the
/// hive-ci runner shares the host netns and is therefore affected by host
/// resolver blips). A fixed periodic interval isolates CI from transient DNS
/// failures — a stale mirror is acceptable; a broken clone because of a
/// momentary DNS blip is not. Written in the form Forgejo echoes back, so an
/// unchanged mirror compares equal and is not re-patched.
const MIRROR_INTERVAL: &str = "8h0m0s";
/// README pushed to a freshly created, still-empty `internal/knowledge`.
/// Short explanation + empty table-of-contents with an HTML comment
/// instructing contributors to add entries when they create new files.
const KNOWLEDGE_README: &str = "\
# knowledge
Hive-wide reference documents: conventions, runbooks, and anything that \
every agent should know.
## How to contribute
1. Fork this repo into your own namespace on the forge.
2. Create a branch, add or update a document.
3. Open a pull request — the operator reviews and merges.
4. Every agent container updates automatically on merge.
Do **not** push directly to `main` — agents have read-only access.
## Contents
<!-- Add an entry here each time you create a new document:
- [Title](path/to/file.md) - one-line description
-->
";
/// The operator-declared pull-mirrors (nix `deploy.forgejo.mirrors`, plus the
/// CI-auto `actions/checkout` entry), JSON-encoded by `hive-forge/default.nix`.
/// Absent or empty means no mirrors.
const MIRRORS_ENV: &str = "SWARM_CONTROLLER_FORGE_MIRRORS";
/// The `agent-configs` org avatar PNG, set by `swarm-controller.nix` from
/// `deploy.swarm-controller.configOrgAvatarPng` or the bundled default.
const AVATAR_PNG_ENV: &str = "SWARM_CONTROLLER_CONFIG_ORG_AVATAR_PNG";
/// Marker file under the state directory: the org avatar has been uploaded.
/// One-shot so every pass does not re-upload the same image; delete it to
/// force a re-upload after changing the PNG.
const AVATAR_MARKER: &str = "forge-config-org-avatar-set";
/// How often the pass re-runs. The objects rarely change, so this is about how
/// long a failed pass (the forge still starting, most often) waits for its
/// retry. Matches `config_pr`'s poll: the same forge, a similar handful of
/// reads, the same staleness tolerance.
const RECONCILE_INTERVAL: std::time::Duration = std::time::Duration::from_mins(5);
/// One operator-declared pull-mirror, as the env carries it.
#[derive(Deserialize)]
struct MirrorEntry {
/// Upstream clone URL to mirror from (e.g. `https://github.com/actions/checkout`).
upstream: String,
/// Local `<owner>/<repo>` the mirror is created at.
dest: String,
}
/// A pull-mirror to ensure, with its `dest` already split.
#[derive(Clone, Debug, PartialEq)]
struct MirrorSpec {
owner: String,
repo: String,
upstream: String,
}
/// A plain (non-mirror) repo to ensure.
#[derive(Clone, Debug, PartialEq)]
struct RepoSpec {
owner: &'static str,
name: &'static str,
private: bool,
/// Push [`KNOWLEDGE_README`] while the repo is still empty.
seed_readme: bool,
}
/// Everything a pass should make true. Built once per process: its inputs are
/// env vars, which do not change under a running daemon.
#[derive(Clone, Debug)]
pub struct Desired {
/// Seeded orgs first, then mirror owners, without duplicates.
orgs: Vec<String>,
repos: Vec<RepoSpec>,
mirrors: Vec<MirrorSpec>,
/// `None` when no avatar is configured, which the nix module never does
/// on a host with forge access; logged at build time.
avatar_png: Option<PathBuf>,
/// Where [`AVATAR_MARKER`] lives.
state_dir: PathBuf,
/// Converge every config repo's `main` rule on [`config_rule_edit`].
config_rules: bool,
}
impl Desired {
fn new(mirrors: Vec<MirrorSpec>, avatar_png: Option<PathBuf>, state_dir: PathBuf) -> Self {
let mut orgs: Vec<String> = SEEDED_ORGS.iter().map(|&o| o.to_owned()).collect();
for m in &mirrors {
// The mirror can't land without its owner existing.
if !orgs.contains(&m.owner) {
orgs.push(m.owner.clone());
}
}
Self {
orgs,
repos: vec![
RepoSpec {
owner: SHARED_ORG,
name: SHARED_DOCS_REPO,
private: true,
seed_readme: false,
},
// Public, so any agent with a forge account can fork it and
// open PRs without an explicit collaborator grant. A repo
// that exists as private (an older deployment) is patched
// to public.
RepoSpec {
owner: KNOWLEDGE_ORG,
name: KNOWLEDGE_REPO,
private: false,
seed_readme: true,
},
],
mirrors,
avatar_png,
state_dir,
config_rules: true,
}
}
/// Read [`MIRRORS_ENV`] and [`AVATAR_PNG_ENV`]. A malformed mirror list
/// or entry is logged and skipped, never fatal, so one bad entry cannot
/// stop the orgs and the merge-gate team from being ensured.
pub fn from_env() -> Self {
let mirrors = std::env::var(MIRRORS_ENV)
.ok()
.map(|raw| parse_mirrors(&raw))
.unwrap_or_default();
let avatar_png = std::env::var_os(AVATAR_PNG_ENV).map(PathBuf::from);
if avatar_png.is_none() {
tracing::warn!(
"{AVATAR_PNG_ENV} unset; the {CONFIG_ORG} org avatar will not be set \
(the nix module sets it whenever forge access is configured)"
);
}
Self::new(mirrors, avatar_png, crate::webhook::state_dir())
}
/// Just the merge gate's prerequisites: the `agent-configs` org and the
/// `operators` team in it. What [`Client::ensure_merge_gate_prerequisites`]
/// reconciles ahead of a config repo's branch protection.
fn merge_gate_only() -> Self {
Self {
orgs: vec![CONFIG_ORG.to_owned()],
repos: Vec::new(),
mirrors: Vec::new(),
avatar_png: None,
state_dir: PathBuf::new(),
config_rules: false,
}
}
fn team_orgs(&self) -> impl Iterator<Item = &'static str> + '_ {
OPERATORS_TEAM_ORGS
.into_iter()
.filter(|org| self.orgs.iter().any(|o| o == org))
}
fn avatar_marker(&self) -> PathBuf {
self.state_dir.join(AVATAR_MARKER)
}
}
/// Parse the mirror env. Invalid JSON skips every mirror; an entry whose
/// `dest` is not `<owner>/<repo>` skips just that entry. Both warn.
fn parse_mirrors(raw: &str) -> Vec<MirrorSpec> {
if raw.trim().is_empty() {
return Vec::new();
}
let entries: Vec<MirrorEntry> = match serde_json::from_str(raw) {
Ok(m) => m,
Err(e) => {
tracing::warn!(error = %e, "swarm forge: {MIRRORS_ENV} is not valid JSON; skipping mirror seed");
return Vec::new();
}
};
entries
.into_iter()
.filter_map(|m| {
let Some((owner, repo)) = m.dest.split_once('/') else {
tracing::warn!(dest = %m.dest, "swarm forge: mirror dest is not <owner>/<repo>; skipping");
return None;
};
Some(MirrorSpec {
owner: owner.to_owned(),
repo: repo.to_owned(),
upstream: m.upstream,
})
})
.collect()
}
/// What a read found. `Unknown` means the read itself failed (already
/// logged); the planner writes nothing for such an object this pass rather
/// than guess, and the next pass reads again.
#[derive(Clone, Debug, PartialEq)]
enum Seen<T> {
Absent,
Present(T),
Unknown,
}
/// The [`OPERATORS_TEAM`] as found in one org.
#[derive(Clone, Debug, PartialEq)]
struct TeamState {
id: i64,
/// Whether every setting this module manages already has its desired
/// value. Membership is not one of them: the operator manages that.
matches: bool,
}
/// The parts of a repo this module manages.
#[derive(Clone, Debug, PartialEq)]
struct RepoState {
private: bool,
empty: bool,
mirror_interval: Option<String>,
}
/// Everything a pass read, keyed the way [`plan`] looks it up.
#[derive(Clone, Debug, Default)]
struct Observed {
orgs: BTreeMap<String, Seen<()>>,
teams: BTreeMap<String, Seen<TeamState>>,
repos: BTreeMap<(String, String), Seen<RepoState>>,
avatar_set: bool,
/// Per config repo, whether its `main` rule already matches
/// [`config_rule_edit`]. `Absent` is a repo with no `main` rule.
config_rules: BTreeMap<String, Seen<bool>>,
/// The config org's repo list could not be read, so `config_rules` is
/// empty without meaning every rule matches.
config_repos_unread: bool,
}
impl Observed {
fn org(&self, org: &str) -> &Seen<()> {
self.orgs.get(org).unwrap_or(&Seen::Unknown)
}
/// A repo in an org that does not exist does not exist either: no read
/// is made for it, and the planner creates it after the org.
fn repo(&self, owner: &str, name: &str) -> Seen<&RepoState> {
match self.org(owner) {
Seen::Absent => Seen::Absent,
Seen::Unknown => Seen::Unknown,
Seen::Present(()) => match self.repos.get(&(owner.to_owned(), name.to_owned())) {
Some(Seen::Present(r)) => Seen::Present(r),
Some(Seen::Absent) => Seen::Absent,
Some(Seen::Unknown) | None => Seen::Unknown,
},
}
}
fn team(&self, org: &str) -> Seen<&TeamState> {
match self.org(org) {
Seen::Absent => Seen::Absent,
Seen::Unknown => Seen::Unknown,
Seen::Present(()) => match self.teams.get(org) {
Some(Seen::Present(t)) => Seen::Present(t),
Some(Seen::Absent) => Seen::Absent,
Some(Seen::Unknown) | None => Seen::Unknown,
},
}
}
}
/// One write. Ordered by [`plan`] so an org always precedes what lives in it.
#[derive(Clone, Debug, PartialEq)]
enum Action {
CreateOrg {
org: String,
},
CreateTeam {
org: String,
},
/// The team exists with the wrong shape (an older or hand-edited one):
/// PATCH it to the desired settings.
ReconcileTeam {
org: String,
id: i64,
},
/// PATCH a config repo's `main` rule to [`config_rule_edit`].
ConvergeConfigRule {
repo: String,
},
CreateRepo {
owner: String,
name: String,
private: bool,
},
SetRepoPublic {
owner: String,
name: String,
},
SeedReadme {
owner: String,
name: String,
},
CreateMirror {
owner: String,
repo: String,
upstream: String,
},
/// Mirrors seeded before the interval was introduced (or with another
/// value) converge on [`MIRROR_INTERVAL`].
SetMirrorInterval {
owner: String,
repo: String,
},
SetConfigOrgAvatar {
png: PathBuf,
},
}
impl Action {
/// The org this write needs to exist first, if it is not the one
/// creating it.
fn needs_org(&self) -> Option<&str> {
match self {
Self::CreateOrg { .. } => None,
Self::CreateTeam { org } | Self::ReconcileTeam { org, .. } => Some(org),
Self::CreateRepo { owner, .. }
| Self::SetRepoPublic { owner, .. }
| Self::SeedReadme { owner, .. }
| Self::CreateMirror { owner, .. }
| Self::SetMirrorInterval { owner, .. } => Some(owner),
Self::ConvergeConfigRule { .. } | Self::SetConfigOrgAvatar { .. } => Some(CONFIG_ORG),
}
}
}
/// Decide the writes that take `observed` to `desired`. Pure. An object whose
/// read failed gets no write; one that already has its desired state gets
/// none either, so a converged forge sees no writes at all.
fn plan(desired: &Desired, observed: &Observed) -> Vec<Action> {
let mut actions = Vec::new();
for org in &desired.orgs {
if *observed.org(org) == Seen::Absent {
actions.push(Action::CreateOrg { org: org.clone() });
}
}
for org in desired.team_orgs() {
match observed.team(org) {
Seen::Absent => actions.push(Action::CreateTeam {
org: org.to_owned(),
}),
Seen::Present(t) if !t.matches => actions.push(Action::ReconcileTeam {
org: org.to_owned(),
id: t.id,
}),
Seen::Present(_) | Seen::Unknown => {}
}
}
// After the teams: the rule names the config org's `operators` team.
for (repo, rule) in &observed.config_rules {
if *rule == Seen::Present(false) {
actions.push(Action::ConvergeConfigRule { repo: repo.clone() });
}
}
for r in &desired.repos {
let (owner, name) = (r.owner.to_owned(), r.name.to_owned());
let (create, set_public, seed) = match observed.repo(r.owner, r.name) {
Seen::Absent => (true, false, r.seed_readme),
Seen::Present(s) => (false, !r.private && s.private, r.seed_readme && s.empty),
Seen::Unknown => (false, false, false),
};
if create {
actions.push(Action::CreateRepo {
owner: owner.clone(),
name: name.clone(),
private: r.private,
});
}
if set_public {
actions.push(Action::SetRepoPublic {
owner: owner.clone(),
name: name.clone(),
});
}
if seed {
actions.push(Action::SeedReadme { owner, name });
}
}
for m in &desired.mirrors {
match observed.repo(&m.owner, &m.repo) {
Seen::Absent => actions.push(Action::CreateMirror {
owner: m.owner.clone(),
repo: m.repo.clone(),
upstream: m.upstream.clone(),
}),
Seen::Present(s) if s.mirror_interval.as_deref() != Some(MIRROR_INTERVAL) => {
actions.push(Action::SetMirrorInterval {
owner: m.owner.clone(),
repo: m.repo.clone(),
});
}
Seen::Present(_) | Seen::Unknown => {}
}
}
if let Some(png) = &desired.avatar_png
&& !observed.avatar_set
{
actions.push(Action::SetConfigOrgAvatar { png: png.clone() });
}
actions
}
/// Whether `t` has every setting [`Client::create_operators_team`] would
/// give it. Units compare as a set: Forgejo does not promise their order.
fn team_matches(t: &Team) -> bool {
let units_match = t.units.as_ref().is_some_and(|units| {
let mut have: Vec<&str> = units.iter().map(String::as_str).collect();
let mut want = OPERATORS_TEAM_UNITS.to_vec();
have.sort_unstable();
want.sort_unstable();
have == want
});
units_match
&& t.permission == Some(TeamPermission::Write)
&& t.includes_all_repositories == Some(true)
&& t.can_create_org_repo == Some(false)
&& t.description.as_deref() == Some(OPERATORS_TEAM_DESCRIPTION)
}
/// The merge gate on every config repo's `main`: the `operators` team approves
/// and merges, and `core` merges too, for the hive's dashboard approval.
///
/// Forgejo replaces each list this sends wholesale and keeps every field left
/// `None`, `required_approvals` included — the hive's own boot PATCH sets
/// that one.
fn config_rule_edit() -> EditBranchProtectionOption {
EditBranchProtectionOption {
apply_to_admins: None,
approvals_whitelist_teams: Some(vec![OPERATORS_TEAM.to_owned()]),
approvals_whitelist_username: None,
block_on_official_review_requests: None,
block_on_outdated_branch: None,
block_on_rejected_reviews: None,
dismiss_stale_approvals: None,
enable_approvals_whitelist: Some(true),
enable_merge_whitelist: Some(true),
enable_push: None,
enable_push_whitelist: None,
enable_status_check: None,
ignore_stale_approvals: None,
merge_whitelist_teams: Some(vec![OPERATORS_TEAM.to_owned()]),
merge_whitelist_usernames: Some(vec![CORE_USER.to_owned()]),
protected_file_patterns: None,
push_whitelist_deploy_keys: None,
push_whitelist_teams: None,
push_whitelist_usernames: None,
require_signed_commits: None,
required_approvals: None,
status_check_contexts: None,
unprotected_file_patterns: None,
}
}
/// Whether `rule` already has every field [`config_rule_edit`] sets.
fn config_rule_matches(rule: &BranchProtection) -> bool {
let only = |list: &Option<Vec<String>>, want: &str| matches!(list.as_deref(), Some([entry]) if entry == want);
rule.enable_merge_whitelist == Some(true)
&& only(&rule.merge_whitelist_teams, OPERATORS_TEAM)
&& only(&rule.merge_whitelist_usernames, CORE_USER)
&& rule.enable_approvals_whitelist == Some(true)
&& only(&rule.approvals_whitelist_teams, OPERATORS_TEAM)
}
/// Whether an error is Forgejo saying 404: the object is absent, as opposed
/// to a transport, auth or server failure.
fn is_not_found(e: &ForgejoError) -> bool {
match e {
ForgejoError::ApiError(api) => matches!(api.error_kind(), ApiErrorKind::NotFound { .. }),
ForgejoError::UnexpectedStatusCode(s) => *s == StatusCode::NOT_FOUND,
_ => false,
}
}
/// Fold a read into [`Seen`], logging a failure that is not a 404.
fn seen<T, U>(res: Result<T, ForgejoError>, what: &str, f: impl FnOnce(T) -> U) -> Seen<U> {
match res {
Ok(v) => Seen::Present(f(v)),
Err(e) if is_not_found(&e) => Seen::Absent,
Err(e) => {
tracing::warn!(error = %e, "swarm forge objects: reading {what} failed; skipping it this pass");
Seen::Unknown
}
}
}
/// `EditRepoOption` with every field unset — repo edits only ever change the
/// one field the caller sets on top (Forgejo leaves `None` fields untouched).
fn sparse_edit_repo_option() -> EditRepoOption {
EditRepoOption {
allow_fast_forward_only_merge: None,
allow_manual_merge: None,
allow_merge_commits: None,
allow_rebase: None,
allow_rebase_explicit: None,
allow_rebase_update: None,
allow_squash_merge: None,
archived: None,
autodetect_manual_merge: None,
default_allow_maintainer_edit: None,
default_branch: None,
default_delete_branch_after_merge: None,
default_merge_style: None,
default_update_style: None,
description: None,
enable_prune: None,
external_tracker: None,
external_wiki: None,
globally_editable_wiki: None,
has_actions: None,
has_issues: None,
has_packages: None,
has_projects: None,
has_pull_requests: None,
has_releases: None,
has_wiki: None,
ignore_whitespace_conflicts: None,
internal_tracker: None,
mirror_interval: None,
name: None,
private: None,
template: None,
website: None,
wiki_branch: None,
}
}
/// How a pass went, for the one summary line.
#[derive(Debug, Default, PartialEq)]
struct PassOutcome {
applied: usize,
failed: usize,
}
impl Client {
/// Read the current state of every object `desired` names.
async fn observe(&self, desired: &Desired) -> Observed {
let mut observed = Observed::default();
for org in &desired.orgs {
let org_seen = seen(self.api.org_get(org).await, &format!("org {org}"), drop);
observed.orgs.insert(org.clone(), org_seen);
}
for org in desired.team_orgs() {
if *observed.org(org) != Seen::Present(()) {
continue;
}
let teams = seen(
self.api.org_list_teams(org).await,
&format!("teams of {org}"),
|(_headers, teams)| teams,
);
let team = match teams {
Seen::Present(teams) => teams
.into_iter()
.find(|t| t.name.as_deref() == Some(OPERATORS_TEAM))
.map_or(Seen::Absent, |t| match t.id {
Some(id) => Seen::Present(TeamState {
id,
matches: team_matches(&t),
}),
None => Seen::Unknown,
}),
Seen::Absent => Seen::Absent,
Seen::Unknown => Seen::Unknown,
};
observed.teams.insert(org.to_owned(), team);
}
let repo_keys = desired
.repos
.iter()
.map(|r| (r.owner.to_owned(), r.name.to_owned()))
.chain(
desired
.mirrors
.iter()
.map(|m| (m.owner.clone(), m.repo.clone())),
);
for (owner, name) in repo_keys {
if *observed.org(&owner) != Seen::Present(()) {
continue;
}
let repo = seen(
self.api.repo_get(&owner, &name).await,
&format!("repo {owner}/{name}"),
|r| RepoState {
private: r.private.unwrap_or(false),
empty: r.empty.unwrap_or(false),
mirror_interval: r.mirror_interval,
},
);
observed.repos.insert((owner, name), repo);
}
observed.avatar_set = desired.avatar_png.is_some() && desired.avatar_marker().exists();
if desired.config_rules && *observed.org(CONFIG_ORG) == Seen::Present(()) {
self.observe_config_rules(&mut observed).await;
}
observed
}
/// Read every config repo's `main` rule into `observed.config_rules`.
async fn observe_config_rules(&self, observed: &mut Observed) {
let repos = match self.api.org_list_repos(CONFIG_ORG).all().await {
Ok(repos) => repos,
Err(e) => {
tracing::warn!(error = %e, "swarm forge objects: listing {CONFIG_ORG} repos failed; skipping their branch rules this pass");
observed.config_repos_unread = true;
return;
}
};
for name in repos.into_iter().filter_map(|r| r.name) {
let rule = seen(
self.api
.repo_get_branch_protection(CONFIG_ORG, &name, "main")
.await,
&format!("branch protection of {CONFIG_ORG}/{name}"),
|rule| config_rule_matches(&rule),
);
observed.config_rules.insert(name, rule);
}
}
/// Execute `actions` in order. An action whose org failed to be created
/// this pass is skipped: it would fail anyway, and its org's failure is
/// the line worth reading.
async fn apply(&self, desired: &Desired, actions: &[Action]) -> PassOutcome {
let mut outcome = PassOutcome::default();
let mut failed_orgs: Vec<&str> = Vec::new();
for action in actions {
if let Some(org) = action.needs_org()
&& failed_orgs.contains(&org)
{
tracing::warn!(?action, %org, "swarm forge objects: skipped, its org could not be created");
outcome.failed += 1;
continue;
}
match self.apply_one(desired, action).await {
Ok(()) => outcome.applied += 1,
Err(e) => {
tracing::warn!(?action, error = %format!("{e:#}"), "swarm forge objects: write failed; retrying next pass");
outcome.failed += 1;
if let Action::CreateOrg { org } = action {
failed_orgs.push(org);
}
}
}
}
outcome
}
async fn apply_one(&self, desired: &Desired, action: &Action) -> Result<()> {
match action {
Action::CreateOrg { org } => self.create_org(org).await,
Action::CreateTeam { org } => self.create_operators_team(org).await,
Action::ReconcileTeam { org, id } => self.reconcile_operators_team(org, *id).await,
Action::ConvergeConfigRule { repo } => {
self.api
.repo_edit_branch_protection(CONFIG_ORG, repo, "main", config_rule_edit())
.await
.with_context(|| format!("converge {CONFIG_ORG}/{repo} main branch rule"))?;
tracing::info!(%repo, "swarm forge objects: config repo merge gate converged");
Ok(())
}
Action::CreateRepo {
owner,
name,
private,
} => self.create_shared_repo(owner, name, *private).await,
Action::SetRepoPublic { owner, name } => {
let mut edit = sparse_edit_repo_option();
edit.private = Some(false);
self.api
.repo_edit(owner, name, edit)
.await
.with_context(|| format!("edit {owner}/{name} (set public)"))?;
tracing::info!(%owner, %name, "swarm forge objects: repo set to public");
Ok(())
}
Action::SeedReadme { owner, name } => self.seed_readme(owner, name).await,
Action::CreateMirror {
owner,
repo,
upstream,
} => self.create_mirror(owner, repo, upstream).await,
Action::SetMirrorInterval { owner, repo } => {
let mut edit = sparse_edit_repo_option();
edit.mirror_interval = Some(MIRROR_INTERVAL.to_owned());
self.api
.repo_edit(owner, repo, edit)
.await
.with_context(|| format!("set mirror_interval on {owner}/{repo}"))?;
tracing::info!(%owner, %repo, interval = MIRROR_INTERVAL, "swarm forge objects: pull-mirror interval updated");
Ok(())
}
Action::SetConfigOrgAvatar { png } => {
self.set_config_org_avatar(png, &desired.avatar_marker())
.await
}
}
}
/// Create `org`. A 409, or a 422 with the org confirmed present, is a
/// race with another writer and counts as done.
async fn create_org(&self, org: &str) -> Result<()> {
let option = CreateOrgOption {
description: None,
email: None,
full_name: None,
location: None,
repo_admin_change_team_access: None,
username: org.to_owned(),
visibility: None,
website: None,
};
match self.api.org_create(option).await {
Ok(_) => {
tracing::info!(%org, "swarm forge objects: created org");
Ok(())
}
Err(e) => {
let existing =
is_ambiguous_validation_failure(&e) && self.api.org_get(org).await.is_ok();
if folds_into_success(&e, existing) {
Ok(())
} else {
Err(e).with_context(|| format!("create org {org}"))
}
}
}
}
/// Provision the [`OPERATORS_TEAM`] inside `org` as an **empty** team.
/// Branch protection on that org's repos references it as the
/// merge/approval whitelist; the operator adds herself as a member via the
/// forge UI. `includes_all_repositories` so the gate applies to every repo
/// in the org; `write` is enough to approve + merge. This module never
/// manages membership.
///
/// A 422 from `org_create_team` that is not a duplicate is a real
/// validation error (bad request shape, missing units) and surfaces, so it
/// can be fixed rather than leave the team silently uncreated every pass.
async fn create_operators_team(&self, org: &str) -> Result<()> {
let team = CreateTeamOption {
can_create_org_repo: Some(false),
description: Some(OPERATORS_TEAM_DESCRIPTION.to_owned()),
includes_all_repositories: Some(true),
name: OPERATORS_TEAM.to_owned(),
permission: Some(CreateTeamOptionPermission::Write),
units: Some(OPERATORS_TEAM_UNITS.map(str::to_owned).to_vec()),
units_map: None,
};
match self.api.org_create_team(org, team).await {
Ok(_) => {
tracing::info!(%org, "swarm forge objects: created {OPERATORS_TEAM} team");
Ok(())
}
// Forgejo signals a duplicate team as a 409 OR (this build) a 422
// `validation failed: team already exists`. Confirm the 422 by
// listing; the next pass reconciles its settings if they differ.
Err(e) => {
let existing = is_ambiguous_validation_failure(&e)
&& self.api.org_list_teams(org).await.is_ok_and(|(_, teams)| {
teams
.iter()
.any(|t| t.name.as_deref() == Some(OPERATORS_TEAM))
});
if folds_into_success(&e, existing) {
Ok(())
} else {
Err(e).with_context(|| format!("create team {org}/{OPERATORS_TEAM}"))
}
}
}
}
/// PATCH an existing [`OPERATORS_TEAM`] to the desired settings, so a
/// team created with an older or wrong shape self-heals. Members are a
/// separate endpoint and untouched here.
async fn reconcile_operators_team(&self, org: &str, id: i64) -> Result<()> {
let edit = EditTeamOption {
can_create_org_repo: Some(false),
description: Some(OPERATORS_TEAM_DESCRIPTION.to_owned()),
includes_all_repositories: Some(true),
name: OPERATORS_TEAM.to_owned(),
permission: Some(EditTeamOptionPermission::Write),
units: Some(OPERATORS_TEAM_UNITS.map(str::to_owned).to_vec()),
units_map: None,
};
self.api
.org_edit_team(id, edit)
.await
.with_context(|| format!("reconcile {org}/{OPERATORS_TEAM} team settings"))?;
tracing::info!(%org, "swarm forge objects: reconciled {OPERATORS_TEAM} team settings");
Ok(())
}
/// Create `owner/name`, empty, defaulting to `main`.
async fn create_shared_repo(&self, owner: &str, name: &str, private: bool) -> Result<()> {
match self
.api
.create_org_repo(owner, Self::repo_option(name, private))
.await
{
Ok(_) => {
tracing::info!(%owner, %name, private, "swarm forge objects: created repo");
Ok(())
}
Err(e) => {
let existing = is_ambiguous_validation_failure(&e)
&& self.api.repo_get(owner, name).await.is_ok();
if folds_into_success(&e, existing) {
Ok(())
} else {
Err(e).with_context(|| format!("create repo {owner}/{name}"))
}
}
}
}
/// Commit [`KNOWLEDGE_README`] to the empty repo's default branch. Only
/// planned while the repo reports itself empty, so a README the operator
/// has since edited is never overwritten.
async fn seed_readme(&self, owner: &str, name: &str) -> Result<()> {
let files = vec![ChangeFileOperation {
content: Some(base64_encode(KNOWLEDGE_README)),
from_path: None,
operation: ChangeFileOperationOperation::Create,
path: "README.md".to_owned(),
sha: None,
}];
self.api
.repo_change_files(
owner,
name,
ChangeFilesOptions {
author: None,
branch: None,
committer: None,
dates: None,
files,
force_overwrite_new_branch: None,
message: Some("init: seed README".to_owned()),
new_branch: None,
signoff: None,
},
)
.await
.with_context(|| format!("seed README.md in {owner}/{name}"))?;
tracing::info!(%owner, %name, "swarm forge objects: seeded README.md");
Ok(())
}
/// Create `owner/repo` as a pull-mirror of `upstream` via the migrate API.
/// Only a 409 folds into success (a race created it since the read). NOT
/// 422: for the migrate endpoint 422 is a validation error (bad
/// `clone_addr` / service), so it must surface rather than be swallowed as
/// "already exists".
async fn create_mirror(&self, owner: &str, repo: &str, upstream: &str) -> Result<()> {
let opts = MigrateRepoOptions {
auth_password: None,
auth_token: None,
auth_username: None,
clone_addr: upstream.to_owned(),
description: None,
issues: None,
labels: None,
lfs: None,
lfs_endpoint: None,
milestones: None,
mirror: Some(true),
// Periodic refresh instead of on-access sync — keeps CI isolated
// from external DNS failures at clone time.
mirror_interval: Some(MIRROR_INTERVAL.to_owned()),
private: Some(false),
pull_requests: None,
releases: None,
repo_name: repo.to_owned(),
repo_owner: Some(owner.to_owned()),
service: Some(MigrateRepoOptionsService::Git),
uid: None,
wiki: None,
};
match self.api.repo_migrate(opts).await {
Ok(_) => {
tracing::info!(%owner, %repo, %upstream, interval = MIRROR_INTERVAL, "swarm forge objects: created pull-mirror");
Ok(())
}
Err(e) if is_confirmed_conflict(&e) => Ok(()),
Err(e) => Err(e).with_context(|| format!("migrate pull-mirror {owner}/{repo}")),
}
}
/// Upload `png` as the `agent-configs` org avatar, then write `marker` so
/// later passes skip it. Best-effort: the forge runs fine with the default
/// identicon, and a marker that fails to write only costs a re-upload.
async fn set_config_org_avatar(
&self,
png: &std::path::Path,
marker: &std::path::Path,
) -> Result<()> {
let bytes = tokio::fs::read(png)
.await
.with_context(|| format!("read {CONFIG_ORG} avatar PNG from {}", png.display()))?;
self.api
.org_update_avatar(
CONFIG_ORG,
UpdateUserAvatarOption {
image: Some(base64::engine::general_purpose::STANDARD.encode(&bytes)),
},
)
.await
.with_context(|| format!("set {CONFIG_ORG} avatar"))?;
if let Err(e) = std::fs::write(marker, "") {
tracing::warn!(error = %e, marker = %marker.display(), "swarm forge objects: avatar set, but its marker could not be written");
}
tracing::info!(org = CONFIG_ORG, "swarm forge objects: set org avatar");
Ok(())
}
/// One observe → plan → apply pass over `desired`.
async fn reconcile(&self, desired: &Desired) -> PassOutcome {
let observed = self.observe(desired).await;
let unknown = observed
.orgs
.values()
.filter(|s| **s == Seen::Unknown)
.count()
+ observed
.teams
.values()
.filter(|s| **s == Seen::Unknown)
.count()
+ observed
.repos
.values()
.filter(|s| **s == Seen::Unknown)
.count()
+ observed
.config_rules
.values()
.filter(|s| **s == Seen::Unknown)
.count()
+ usize::from(observed.config_repos_unread);
let actions = plan(desired, &observed);
let mut outcome = self.apply(desired, &actions).await;
outcome.failed += unknown;
outcome
}
/// Make sure the `agent-configs` org and its `operators` team exist, so a
/// branch-protection rule naming the team can be applied. The periodic
/// pass normally has done this long before, but an agent can be created
/// while that pass is still failing (a controller that started with the
/// forge), so [`Client::create_repo`] checks first rather than rely on
/// the order.
pub(super) async fn ensure_merge_gate_prerequisites(&self) -> Result<()> {
let outcome = self.reconcile(&Desired::merge_gate_only()).await;
if outcome.failed > 0 {
anyhow::bail!(
"the {CONFIG_ORG} org or its {OPERATORS_TEAM} team could not be ensured \
({} failure(s), logged above)",
outcome.failed
);
}
Ok(())
}
}
/// Ensure the swarm-wide forge objects now and every [`RECONCILE_INTERVAL`].
///
/// The first tick fires immediately (`tokio::time::interval`'s default), the
/// same shape as `config_pr::spawn`. A pass that fails leaves a `warn` per
/// failed object plus the summary line below in the controller's journal, and
/// is retried on the next tick; it never stops the daemon.
pub fn spawn(client: Arc<Client>) {
let desired = Desired::from_env();
tokio::spawn(async move {
let mut ticker = tokio::time::interval(RECONCILE_INTERVAL);
loop {
ticker.tick().await;
let outcome = client.reconcile(&desired).await;
if outcome.failed > 0 {
tracing::warn!(
applied = outcome.applied,
failed = outcome.failed,
retry_in_s = RECONCILE_INTERVAL.as_secs(),
"swarm forge objects: pass incomplete; retrying next tick"
);
} else if outcome.applied > 0 {
tracing::info!(
applied = outcome.applied,
"swarm forge objects: pass converged"
);
} else {
tracing::debug!("swarm forge objects: already converged");
}
}
});
}
#[cfg(test)]
mod tests {
use super::*;
fn mirror(owner: &str, repo: &str) -> MirrorSpec {
MirrorSpec {
owner: owner.to_owned(),
repo: repo.to_owned(),
upstream: format!("https://example.invalid/{repo}"),
}
}
fn desired() -> Desired {
Desired::new(
vec![mirror("actions", "checkout")],
Some(PathBuf::from("/avatar.png")),
PathBuf::from("/state"),
)
}
fn repo_state(private: bool, empty: bool, interval: Option<&str>) -> RepoState {
RepoState {
private,
empty,
mirror_interval: interval.map(str::to_owned),
}
}
/// Every object of [`desired`] present with its desired settings.
fn converged() -> Observed {
let mut o = Observed::default();
for org in [CONFIG_ORG, SHARED_ORG, AGENTS_ORG, "actions"] {
o.orgs.insert(org.to_owned(), Seen::Present(()));
}
for (id, org) in [(1, AGENTS_ORG), (2, CONFIG_ORG)] {
o.teams.insert(
org.to_owned(),
Seen::Present(TeamState { id, matches: true }),
);
}
let key = |a: &str, b: &str| (a.to_owned(), b.to_owned());
o.repos.insert(
key(SHARED_ORG, SHARED_DOCS_REPO),
Seen::Present(repo_state(true, true, None)),
);
o.repos.insert(
key(KNOWLEDGE_ORG, KNOWLEDGE_REPO),
Seen::Present(repo_state(false, false, None)),
);
o.repos.insert(
key("actions", "checkout"),
Seen::Present(repo_state(false, false, Some(MIRROR_INTERVAL))),
);
o.avatar_set = true;
o
}
/// Every object of [`desired`] reported absent.
fn nothing() -> Observed {
let mut o = Observed::default();
for org in [CONFIG_ORG, SHARED_ORG, AGENTS_ORG, "actions"] {
o.orgs.insert(org.to_owned(), Seen::Absent);
}
o
}
fn s(v: &str) -> String {
v.to_owned()
}
#[test]
fn nothing_exists_so_everything_is_created_orgs_first() {
assert_eq!(
plan(&desired(), &nothing()),
vec![
Action::CreateOrg { org: s(CONFIG_ORG) },
Action::CreateOrg { org: s(SHARED_ORG) },
Action::CreateOrg { org: s(AGENTS_ORG) },
Action::CreateOrg { org: s("actions") },
Action::CreateTeam { org: s(AGENTS_ORG) },
Action::CreateTeam { org: s(CONFIG_ORG) },
Action::CreateRepo {
owner: s(SHARED_ORG),
name: s(SHARED_DOCS_REPO),
private: true,
},
Action::CreateRepo {
owner: s(KNOWLEDGE_ORG),
name: s(KNOWLEDGE_REPO),
private: false,
},
Action::SeedReadme {
owner: s(KNOWLEDGE_ORG),
name: s(KNOWLEDGE_REPO),
},
Action::CreateMirror {
owner: s("actions"),
repo: s("checkout"),
upstream: s("https://example.invalid/checkout"),
},
Action::SetConfigOrgAvatar {
png: PathBuf::from("/avatar.png"),
},
]
);
}
#[test]
fn everything_exists_so_nothing_is_written() {
assert_eq!(plan(&desired(), &converged()), Vec::<Action>::new());
}
#[test]
fn partial_state_writes_only_the_gaps() {
let mut o = converged();
// The team is missing in agent-configs only, the agents one has an
// old shape, knowledge is private and still empty, the mirror
// predates the interval, and the avatar was never uploaded.
o.teams.insert(s(CONFIG_ORG), Seen::Absent);
o.teams.insert(
s(AGENTS_ORG),
Seen::Present(TeamState {
id: 7,
matches: false,
}),
);
o.repos.insert(
(s(KNOWLEDGE_ORG), s(KNOWLEDGE_REPO)),
Seen::Present(repo_state(true, true, None)),
);
o.repos.insert(
(s("actions"), s("checkout")),
Seen::Present(repo_state(false, false, None)),
);
o.avatar_set = false;
assert_eq!(
plan(&desired(), &o),
vec![
Action::ReconcileTeam {
org: s(AGENTS_ORG),
id: 7,
},
Action::CreateTeam { org: s(CONFIG_ORG) },
Action::SetRepoPublic {
owner: s(KNOWLEDGE_ORG),
name: s(KNOWLEDGE_REPO),
},
Action::SeedReadme {
owner: s(KNOWLEDGE_ORG),
name: s(KNOWLEDGE_REPO),
},
Action::SetMirrorInterval {
owner: s("actions"),
repo: s("checkout"),
},
Action::SetConfigOrgAvatar {
png: PathBuf::from("/avatar.png"),
},
]
);
}
#[test]
fn an_unreadable_object_gets_no_write() {
let mut o = nothing();
o.orgs.insert(s(SHARED_ORG), Seen::Unknown);
let actions = plan(&desired(), &o);
assert!(!actions.contains(&Action::CreateOrg { org: s(SHARED_ORG) }));
assert!(
!actions.iter().any(|a| a.needs_org() == Some(SHARED_ORG)),
"nothing inside an org whose state is unknown: {actions:?}"
);
}
#[test]
fn the_merge_gate_subset_is_the_config_org_and_its_team() {
let mut o = Observed::default();
o.orgs.insert(s(CONFIG_ORG), Seen::Absent);
assert_eq!(
plan(&Desired::merge_gate_only(), &o),
vec![
Action::CreateOrg { org: s(CONFIG_ORG) },
Action::CreateTeam { org: s(CONFIG_ORG) },
]
);
}
#[test]
fn only_a_config_rule_that_does_not_match_is_converged() {
let mut o = converged();
o.config_rules.insert(s("atlas"), Seen::Present(true));
o.config_rules.insert(s("damocles"), Seen::Present(false));
o.config_rules.insert(s("iris"), Seen::Absent);
o.config_rules.insert(s("argus"), Seen::Unknown);
assert_eq!(
plan(&desired(), &o),
vec![Action::ConvergeConfigRule {
repo: s("damocles")
}]
);
}
fn rule(merge_users: &[&str]) -> BranchProtection {
serde_json::from_value(serde_json::json!({
"enable_merge_whitelist": true,
"merge_whitelist_teams": [OPERATORS_TEAM],
"merge_whitelist_usernames": merge_users,
"enable_approvals_whitelist": true,
"approvals_whitelist_teams": [OPERATORS_TEAM],
}))
.expect("branch protection json")
}
#[test]
fn a_config_rule_matches_only_with_operators_and_core() {
assert!(config_rule_matches(&rule(&[CORE_USER])));
assert!(!config_rule_matches(&rule(&[])));
assert!(!config_rule_matches(&rule(&[CORE_USER, "mallory"])));
let mut core_only = rule(&[CORE_USER]);
core_only.merge_whitelist_teams = None;
assert!(!config_rule_matches(&core_only));
let mut approvals_off = rule(&[CORE_USER]);
approvals_off.enable_approvals_whitelist = Some(false);
assert!(!config_rule_matches(&approvals_off));
}
/// The edit is what the match checks for, so one PATCH converges a rule.
#[test]
fn the_config_rule_edit_produces_a_matching_rule() {
let edit = config_rule_edit();
let applied: BranchProtection = serde_json::from_value(serde_json::json!({
"enable_merge_whitelist": edit.enable_merge_whitelist,
"merge_whitelist_teams": edit.merge_whitelist_teams,
"merge_whitelist_usernames": edit.merge_whitelist_usernames,
"enable_approvals_whitelist": edit.enable_approvals_whitelist,
"approvals_whitelist_teams": edit.approvals_whitelist_teams,
}))
.expect("branch protection json");
assert!(config_rule_matches(&applied));
}
#[test]
fn mirror_owners_join_the_seeded_orgs_once() {
let d = Desired::new(
vec![mirror("actions", "checkout"), mirror("actions", "cache")],
None,
PathBuf::new(),
);
assert_eq!(d.orgs, [CONFIG_ORG, SHARED_ORG, AGENTS_ORG, "actions"]);
}
#[test]
fn a_malformed_mirror_entry_is_dropped_alone() {
let parsed = parse_mirrors(
r#"[{"upstream":"https://x/a","dest":"no-slash"},{"upstream":"https://x/b","dest":"o/b"}]"#,
);
assert_eq!(
parsed,
vec![MirrorSpec {
owner: s("o"),
repo: s("b"),
upstream: s("https://x/b"),
}]
);
assert!(parse_mirrors("not json").is_empty());
assert!(parse_mirrors("").is_empty());
}
fn team(units: &[&str]) -> Team {
serde_json::from_value(serde_json::json!({
"id": 1,
"name": OPERATORS_TEAM,
"description": OPERATORS_TEAM_DESCRIPTION,
"permission": "write",
"includes_all_repositories": true,
"can_create_org_repo": false,
"units": units,
}))
.expect("team json")
}
#[test]
fn team_units_compare_as_a_set() {
let mut reversed = OPERATORS_TEAM_UNITS;
reversed.reverse();
assert!(team_matches(&team(&reversed)));
assert!(!team_matches(&team(&OPERATORS_TEAM_UNITS[1..])));
}
}