forgejo-api's client has no request timeout, so a forge that accepts a connection and never answers held a forge-calling job node forever: the node never reached Done or Failed, and kept its executor task. Every executor arm that awaits the forge now runs its work under the same deadline the periodic passes use. On elapsing, the node fails through its existing Outcome::Failed path with "<node>: no answer from the forge within 120s". Nodes: create_repo, create_forge_user, add_repo_member, grant_docs_read, init_agent_config_repo, mint_agent_forge_token, mint_hive_forge_token. The bound is per node, not per request: the nodes make 1 to about 15 sequential forge and store requests each (mint_hive_forge_token the most), and the requests live inside forge::Client methods the periodic passes share. One 2-minute bound covers both, so #5065's within_pass_deadline / PASS_DEADLINE become within_forge_deadline / FORGE_DEADLINE, with a log line that no longer says "retrying next tick" (true of a pass, not of a node). The InitAgentConfigRepo arm is lifted out of run_swarm_node, like its siblings, to keep that function under clippy::too_many_lines. Refs #5066
1400 lines
52 KiB
Rust
1400 lines
52 KiB
Rust
//! The swarm-wide forge objects: the three seeded orgs (plus every mirror's
|
|
//! owner org), the `operators` merge-gate team in `agents` and
|
|
//! `agent-configs`, the merge gate on every `agent-configs` repo's `main`, the
|
|
//! operator-declared pull-mirrors, `internal/docs`, `internal/knowledge`
|
|
//! (public, README-seeded) and the `agent-configs` org avatar.
|
|
//!
|
|
//! Every hive's `hive-c0re` used to ensure these in its boot sweep, but only
|
|
//! the one hive co-located with the forge container ever did (the sweep bails
|
|
//! without a local `hive-forge` container), and only as `core`, a site-admin
|
|
//! token that hive should not need. They are one set per forge, not per hive,
|
|
//! so they are ensured here, at start and every [`RECONCILE_INTERVAL`] after.
|
|
//!
|
|
//! Split into observe → [`plan`] → apply so the decision is pure: [`plan`]
|
|
//! takes what the forge reported and returns the writes, which is what the
|
|
//! tests pin. The IO on either side only reads or executes.
|
|
//!
|
|
//! A pass that fails anywhere logs each failure at `warn` plus a summary line,
|
|
//! and the next tick retries. Nothing here stops the daemon.
|
|
|
|
use std::collections::BTreeMap;
|
|
use std::path::PathBuf;
|
|
use std::sync::Arc;
|
|
|
|
use anyhow::{Context, Result};
|
|
use base64::Engine as _;
|
|
use forgejo_api::structs::{
|
|
BranchProtection, ChangeFileOperation, ChangeFileOperationOperation, ChangeFilesOptions,
|
|
CreateOrgOption, CreateTeamOption, CreateTeamOptionPermission, EditBranchProtectionOption,
|
|
EditRepoOption, EditTeamOption, EditTeamOptionPermission, MigrateRepoOptions,
|
|
MigrateRepoOptionsService, Team, TeamPermission, UpdateUserAvatarOption,
|
|
};
|
|
use forgejo_api::{ApiErrorKind, ForgejoError};
|
|
use reqwest::StatusCode;
|
|
use serde::Deserialize;
|
|
|
|
use super::{
|
|
CONFIG_ORG, Client, KNOWLEDGE_ORG, KNOWLEDGE_REPO, OPERATORS_TEAM, base64_encode,
|
|
folds_into_success, is_ambiguous_validation_failure, is_confirmed_conflict,
|
|
};
|
|
|
|
/// Forgejo org that owns agent-created repos. Agents can't create repos with
|
|
/// their own token (`max_repo_creation = 0`); a repo an agent asks for is
|
|
/// created here and the agent added as a **write** member (not owner/admin).
|
|
/// Because the org, not the agent, owns the repo, perms stay centrally
|
|
/// managed and branch protection (referencing [`OPERATORS_TEAM`]) can block
|
|
/// the author from merging their own PR.
|
|
const AGENTS_ORG: &str = "agents";
|
|
|
|
/// Org hosting the operator-curated shared repos. Same org as the knowledge
|
|
/// repo's, named separately for the docs repo it also holds.
|
|
pub(super) const SHARED_ORG: &str = KNOWLEDGE_ORG;
|
|
|
|
/// The shared docs repo inside [`SHARED_ORG`]. **Private**: every agent gets
|
|
/// read-only collaborator access, granted per agent by [`super::docs_grant`].
|
|
/// Agents use it as a common reference without the operator having to bake
|
|
/// content into the system prompt.
|
|
pub(super) const SHARED_DOCS_REPO: &str = "docs";
|
|
|
|
/// The orgs ensured on every pass. The meta repo lives at `core/meta`, the
|
|
/// `core` user's own namespace, so no org is needed for it (and it is not a
|
|
/// swarm object: it stays with `hive-c0re`).
|
|
const SEEDED_ORGS: [&str; 3] = [CONFIG_ORG, SHARED_ORG, AGENTS_ORG];
|
|
|
|
/// Where the [`OPERATORS_TEAM`] must exist. Gitea teams are org-scoped, so a
|
|
/// config-repo branch-protection rule referencing `operators` needs the team
|
|
/// in `agent-configs` too. Missing it there 422'd every config-repo
|
|
/// protection apply, leaving those repos unprotected, so operator-merged
|
|
/// config PRs bypassed the deploy pipeline and silently didn't apply.
|
|
const OPERATORS_TEAM_ORGS: [&str; 2] = [AGENTS_ORG, CONFIG_ORG];
|
|
|
|
const OPERATORS_TEAM_DESCRIPTION: &str = "hyperhive operators — merge gate for agent repos";
|
|
|
|
/// Repo-unit access flags for the `operators` team. Explicit list so Forgejo
|
|
/// doesn't reject a null/absent `units` field; a `write`-permission team needs
|
|
/// at least `repo.code` + `repo.pulls` to review and merge PRs.
|
|
const OPERATORS_TEAM_UNITS: [&str; 7] = [
|
|
"repo.code",
|
|
"repo.issues",
|
|
"repo.pulls",
|
|
"repo.releases",
|
|
"repo.wiki",
|
|
"repo.projects",
|
|
"repo.packages",
|
|
];
|
|
|
|
/// Periodic sync interval for pull-mirrors. Forgejo syncs mirrors on-access by
|
|
/// default, which re-introduces external DNS latency on every `git clone`. A
|
|
/// fixed periodic interval isolates CI from transient DNS
|
|
/// failures — a stale mirror is acceptable; a broken clone because of a
|
|
/// momentary DNS blip is not. Written in the form Forgejo echoes back, so an
|
|
/// unchanged mirror compares equal and is not re-patched.
|
|
const MIRROR_INTERVAL: &str = "8h0m0s";
|
|
|
|
/// README pushed to a freshly created, still-empty `internal/knowledge`.
|
|
/// Short explanation + empty table-of-contents with an HTML comment
|
|
/// instructing contributors to add entries when they create new files.
|
|
const KNOWLEDGE_README: &str = "\
|
|
# knowledge
|
|
|
|
Hive-wide reference documents: conventions, runbooks, and anything that \
|
|
every agent should know.
|
|
|
|
## How to contribute
|
|
|
|
1. Fork this repo into your own namespace on the forge.
|
|
2. Create a branch, add or update a document.
|
|
3. Open a pull request — the operator reviews and merges.
|
|
4. Every agent container updates automatically on merge.
|
|
|
|
Do **not** push directly to `main` — agents have read-only access.
|
|
|
|
## Contents
|
|
|
|
<!-- Add an entry here each time you create a new document:
|
|
- [Title](path/to/file.md) - one-line description
|
|
-->
|
|
";
|
|
|
|
/// The operator-declared pull-mirrors (nix `deploy.forgejo.mirrors`, plus the
|
|
/// `actions/*` entries `deploy.forgejo.actionsFromSelf` adds), JSON-encoded by `hive-forge/default.nix`.
|
|
/// Absent or empty means no mirrors.
|
|
const MIRRORS_ENV: &str = "SWARM_CONTROLLER_FORGE_MIRRORS";
|
|
|
|
/// The `agent-configs` org avatar PNG, set by `swarm-controller.nix` from
|
|
/// `deploy.swarm-controller.configOrgAvatarPng` or the bundled default.
|
|
const AVATAR_PNG_ENV: &str = "SWARM_CONTROLLER_CONFIG_ORG_AVATAR_PNG";
|
|
|
|
/// Marker file under the state directory: the org avatar has been uploaded.
|
|
/// One-shot so every pass does not re-upload the same image; delete it to
|
|
/// force a re-upload after changing the PNG.
|
|
const AVATAR_MARKER: &str = "forge-config-org-avatar-set";
|
|
|
|
/// How often the pass re-runs. The objects rarely change, so this is about how
|
|
/// long a failed pass (the forge still starting, most often) waits for its
|
|
/// retry. Matches `config_pr`'s poll: the same forge, a similar handful of
|
|
/// reads, the same staleness tolerance.
|
|
const RECONCILE_INTERVAL: std::time::Duration = std::time::Duration::from_mins(5);
|
|
|
|
/// One operator-declared pull-mirror, as the env carries it.
|
|
#[derive(Deserialize)]
|
|
struct MirrorEntry {
|
|
/// Upstream clone URL to mirror from (e.g. `https://github.com/actions/checkout`).
|
|
upstream: String,
|
|
/// Local `<owner>/<repo>` the mirror is created at.
|
|
dest: String,
|
|
}
|
|
|
|
/// A pull-mirror to ensure, with its `dest` already split.
|
|
#[derive(Clone, Debug, PartialEq)]
|
|
struct MirrorSpec {
|
|
owner: String,
|
|
repo: String,
|
|
upstream: String,
|
|
}
|
|
|
|
/// A plain (non-mirror) repo to ensure.
|
|
#[derive(Clone, Debug, PartialEq)]
|
|
struct RepoSpec {
|
|
owner: &'static str,
|
|
name: &'static str,
|
|
private: bool,
|
|
/// Push [`KNOWLEDGE_README`] while the repo is still empty.
|
|
seed_readme: bool,
|
|
}
|
|
|
|
/// Everything a pass should make true. Built once per process: its inputs are
|
|
/// env vars, which do not change under a running daemon.
|
|
#[derive(Clone, Debug)]
|
|
pub struct Desired {
|
|
/// Seeded orgs first, then mirror owners, without duplicates.
|
|
orgs: Vec<String>,
|
|
repos: Vec<RepoSpec>,
|
|
mirrors: Vec<MirrorSpec>,
|
|
/// `None` when no avatar is configured, which the nix module never does
|
|
/// on a host with forge access; logged at build time.
|
|
avatar_png: Option<PathBuf>,
|
|
/// Where [`AVATAR_MARKER`] lives.
|
|
state_dir: PathBuf,
|
|
/// Converge every config repo's `main` rule on [`config_rule_edit`].
|
|
config_rules: bool,
|
|
}
|
|
|
|
impl Desired {
|
|
fn new(mirrors: Vec<MirrorSpec>, avatar_png: Option<PathBuf>, state_dir: PathBuf) -> Self {
|
|
let mut orgs: Vec<String> = SEEDED_ORGS.iter().map(|&o| o.to_owned()).collect();
|
|
for m in &mirrors {
|
|
// The mirror can't land without its owner existing.
|
|
if !orgs.contains(&m.owner) {
|
|
orgs.push(m.owner.clone());
|
|
}
|
|
}
|
|
Self {
|
|
orgs,
|
|
repos: vec![
|
|
RepoSpec {
|
|
owner: SHARED_ORG,
|
|
name: SHARED_DOCS_REPO,
|
|
private: true,
|
|
seed_readme: false,
|
|
},
|
|
// Public, so any agent with a forge account can fork it and
|
|
// open PRs without an explicit collaborator grant. A repo
|
|
// that exists as private (an older deployment) is patched
|
|
// to public.
|
|
RepoSpec {
|
|
owner: KNOWLEDGE_ORG,
|
|
name: KNOWLEDGE_REPO,
|
|
private: false,
|
|
seed_readme: true,
|
|
},
|
|
],
|
|
mirrors,
|
|
avatar_png,
|
|
state_dir,
|
|
config_rules: true,
|
|
}
|
|
}
|
|
|
|
/// Read [`MIRRORS_ENV`] and [`AVATAR_PNG_ENV`]. A malformed mirror list
|
|
/// or entry is logged and skipped, never fatal, so one bad entry cannot
|
|
/// stop the orgs and the merge-gate team from being ensured.
|
|
pub fn from_env() -> Self {
|
|
let mirrors = std::env::var(MIRRORS_ENV)
|
|
.ok()
|
|
.map(|raw| parse_mirrors(&raw))
|
|
.unwrap_or_default();
|
|
let avatar_png = std::env::var_os(AVATAR_PNG_ENV).map(PathBuf::from);
|
|
if avatar_png.is_none() {
|
|
tracing::warn!(
|
|
"{AVATAR_PNG_ENV} unset; the {CONFIG_ORG} org avatar will not be set \
|
|
(the nix module sets it whenever forge access is configured)"
|
|
);
|
|
}
|
|
Self::new(mirrors, avatar_png, crate::webhook::state_dir())
|
|
}
|
|
|
|
/// Just the merge gate's prerequisites: the `agent-configs` org and the
|
|
/// `operators` team in it. What [`Client::ensure_merge_gate_prerequisites`]
|
|
/// reconciles ahead of a config repo's branch protection.
|
|
fn merge_gate_only() -> Self {
|
|
Self {
|
|
orgs: vec![CONFIG_ORG.to_owned()],
|
|
repos: Vec::new(),
|
|
mirrors: Vec::new(),
|
|
avatar_png: None,
|
|
state_dir: PathBuf::new(),
|
|
config_rules: false,
|
|
}
|
|
}
|
|
|
|
fn team_orgs(&self) -> impl Iterator<Item = &'static str> + '_ {
|
|
OPERATORS_TEAM_ORGS
|
|
.into_iter()
|
|
.filter(|org| self.orgs.iter().any(|o| o == org))
|
|
}
|
|
|
|
fn avatar_marker(&self) -> PathBuf {
|
|
self.state_dir.join(AVATAR_MARKER)
|
|
}
|
|
}
|
|
|
|
/// Parse the mirror env. Invalid JSON skips every mirror; an entry whose
|
|
/// `dest` is not `<owner>/<repo>` skips just that entry. Both warn.
|
|
fn parse_mirrors(raw: &str) -> Vec<MirrorSpec> {
|
|
if raw.trim().is_empty() {
|
|
return Vec::new();
|
|
}
|
|
let entries: Vec<MirrorEntry> = match serde_json::from_str(raw) {
|
|
Ok(m) => m,
|
|
Err(e) => {
|
|
tracing::warn!(error = %e, "swarm forge: {MIRRORS_ENV} is not valid JSON; skipping mirror seed");
|
|
return Vec::new();
|
|
}
|
|
};
|
|
entries
|
|
.into_iter()
|
|
.filter_map(|m| {
|
|
let Some((owner, repo)) = m.dest.split_once('/') else {
|
|
tracing::warn!(dest = %m.dest, "swarm forge: mirror dest is not <owner>/<repo>; skipping");
|
|
return None;
|
|
};
|
|
Some(MirrorSpec {
|
|
owner: owner.to_owned(),
|
|
repo: repo.to_owned(),
|
|
upstream: m.upstream,
|
|
})
|
|
})
|
|
.collect()
|
|
}
|
|
|
|
/// What a read found. `Unknown` means the read itself failed (already
|
|
/// logged); the planner writes nothing for such an object this pass rather
|
|
/// than guess, and the next pass reads again.
|
|
#[derive(Clone, Debug, PartialEq)]
|
|
enum Seen<T> {
|
|
Absent,
|
|
Present(T),
|
|
Unknown,
|
|
}
|
|
|
|
/// The [`OPERATORS_TEAM`] as found in one org.
|
|
#[derive(Clone, Debug, PartialEq)]
|
|
struct TeamState {
|
|
id: i64,
|
|
/// Whether every setting this module manages already has its desired
|
|
/// value. Membership is not one of them: the operator manages that.
|
|
matches: bool,
|
|
}
|
|
|
|
/// The parts of a repo this module manages.
|
|
#[derive(Clone, Debug, PartialEq)]
|
|
struct RepoState {
|
|
private: bool,
|
|
empty: bool,
|
|
mirror_interval: Option<String>,
|
|
}
|
|
|
|
/// Everything a pass read, keyed the way [`plan`] looks it up.
|
|
#[derive(Clone, Debug, Default)]
|
|
struct Observed {
|
|
orgs: BTreeMap<String, Seen<()>>,
|
|
teams: BTreeMap<String, Seen<TeamState>>,
|
|
repos: BTreeMap<(String, String), Seen<RepoState>>,
|
|
avatar_set: bool,
|
|
/// Per config repo, whether its `main` rule already matches
|
|
/// [`config_rule_edit`]. `Absent` is a repo with no `main` rule.
|
|
config_rules: BTreeMap<String, Seen<bool>>,
|
|
/// The config org's repo list could not be read, so `config_rules` is
|
|
/// empty without meaning every rule matches.
|
|
config_repos_unread: bool,
|
|
}
|
|
|
|
impl Observed {
|
|
fn org(&self, org: &str) -> &Seen<()> {
|
|
self.orgs.get(org).unwrap_or(&Seen::Unknown)
|
|
}
|
|
|
|
/// A repo in an org that does not exist does not exist either: no read
|
|
/// is made for it, and the planner creates it after the org.
|
|
fn repo(&self, owner: &str, name: &str) -> Seen<&RepoState> {
|
|
match self.org(owner) {
|
|
Seen::Absent => Seen::Absent,
|
|
Seen::Unknown => Seen::Unknown,
|
|
Seen::Present(()) => match self.repos.get(&(owner.to_owned(), name.to_owned())) {
|
|
Some(Seen::Present(r)) => Seen::Present(r),
|
|
Some(Seen::Absent) => Seen::Absent,
|
|
Some(Seen::Unknown) | None => Seen::Unknown,
|
|
},
|
|
}
|
|
}
|
|
|
|
fn team(&self, org: &str) -> Seen<&TeamState> {
|
|
match self.org(org) {
|
|
Seen::Absent => Seen::Absent,
|
|
Seen::Unknown => Seen::Unknown,
|
|
Seen::Present(()) => match self.teams.get(org) {
|
|
Some(Seen::Present(t)) => Seen::Present(t),
|
|
Some(Seen::Absent) => Seen::Absent,
|
|
Some(Seen::Unknown) | None => Seen::Unknown,
|
|
},
|
|
}
|
|
}
|
|
}
|
|
|
|
/// One write. Ordered by [`plan`] so an org always precedes what lives in it.
|
|
#[derive(Clone, Debug, PartialEq)]
|
|
enum Action {
|
|
CreateOrg {
|
|
org: String,
|
|
},
|
|
CreateTeam {
|
|
org: String,
|
|
},
|
|
/// The team exists with the wrong shape (an older or hand-edited one):
|
|
/// PATCH it to the desired settings.
|
|
ReconcileTeam {
|
|
org: String,
|
|
id: i64,
|
|
},
|
|
/// PATCH a config repo's `main` rule to [`config_rule_edit`].
|
|
ConvergeConfigRule {
|
|
repo: String,
|
|
},
|
|
CreateRepo {
|
|
owner: String,
|
|
name: String,
|
|
private: bool,
|
|
},
|
|
SetRepoPublic {
|
|
owner: String,
|
|
name: String,
|
|
},
|
|
SeedReadme {
|
|
owner: String,
|
|
name: String,
|
|
},
|
|
CreateMirror {
|
|
owner: String,
|
|
repo: String,
|
|
upstream: String,
|
|
},
|
|
/// Mirrors seeded before the interval was introduced (or with another
|
|
/// value) converge on [`MIRROR_INTERVAL`].
|
|
SetMirrorInterval {
|
|
owner: String,
|
|
repo: String,
|
|
},
|
|
SetConfigOrgAvatar {
|
|
png: PathBuf,
|
|
},
|
|
}
|
|
|
|
impl Action {
|
|
/// The org this write needs to exist first, if it is not the one
|
|
/// creating it.
|
|
fn needs_org(&self) -> Option<&str> {
|
|
match self {
|
|
Self::CreateOrg { .. } => None,
|
|
Self::CreateTeam { org } | Self::ReconcileTeam { org, .. } => Some(org),
|
|
Self::CreateRepo { owner, .. }
|
|
| Self::SetRepoPublic { owner, .. }
|
|
| Self::SeedReadme { owner, .. }
|
|
| Self::CreateMirror { owner, .. }
|
|
| Self::SetMirrorInterval { owner, .. } => Some(owner),
|
|
Self::ConvergeConfigRule { .. } | Self::SetConfigOrgAvatar { .. } => Some(CONFIG_ORG),
|
|
}
|
|
}
|
|
}
|
|
|
|
/// Decide the writes that take `observed` to `desired`. Pure. An object whose
|
|
/// read failed gets no write; one that already has its desired state gets
|
|
/// none either, so a converged forge sees no writes at all.
|
|
fn plan(desired: &Desired, observed: &Observed) -> Vec<Action> {
|
|
let mut actions = Vec::new();
|
|
for org in &desired.orgs {
|
|
if *observed.org(org) == Seen::Absent {
|
|
actions.push(Action::CreateOrg { org: org.clone() });
|
|
}
|
|
}
|
|
for org in desired.team_orgs() {
|
|
match observed.team(org) {
|
|
Seen::Absent => actions.push(Action::CreateTeam {
|
|
org: org.to_owned(),
|
|
}),
|
|
Seen::Present(t) if !t.matches => actions.push(Action::ReconcileTeam {
|
|
org: org.to_owned(),
|
|
id: t.id,
|
|
}),
|
|
Seen::Present(_) | Seen::Unknown => {}
|
|
}
|
|
}
|
|
// After the teams: the rule names the config org's `operators` team.
|
|
for (repo, rule) in &observed.config_rules {
|
|
if *rule == Seen::Present(false) {
|
|
actions.push(Action::ConvergeConfigRule { repo: repo.clone() });
|
|
}
|
|
}
|
|
for r in &desired.repos {
|
|
let (owner, name) = (r.owner.to_owned(), r.name.to_owned());
|
|
let (create, set_public, seed) = match observed.repo(r.owner, r.name) {
|
|
Seen::Absent => (true, false, r.seed_readme),
|
|
Seen::Present(s) => (false, !r.private && s.private, r.seed_readme && s.empty),
|
|
Seen::Unknown => (false, false, false),
|
|
};
|
|
if create {
|
|
actions.push(Action::CreateRepo {
|
|
owner: owner.clone(),
|
|
name: name.clone(),
|
|
private: r.private,
|
|
});
|
|
}
|
|
if set_public {
|
|
actions.push(Action::SetRepoPublic {
|
|
owner: owner.clone(),
|
|
name: name.clone(),
|
|
});
|
|
}
|
|
if seed {
|
|
actions.push(Action::SeedReadme { owner, name });
|
|
}
|
|
}
|
|
for m in &desired.mirrors {
|
|
match observed.repo(&m.owner, &m.repo) {
|
|
Seen::Absent => actions.push(Action::CreateMirror {
|
|
owner: m.owner.clone(),
|
|
repo: m.repo.clone(),
|
|
upstream: m.upstream.clone(),
|
|
}),
|
|
Seen::Present(s) if s.mirror_interval.as_deref() != Some(MIRROR_INTERVAL) => {
|
|
actions.push(Action::SetMirrorInterval {
|
|
owner: m.owner.clone(),
|
|
repo: m.repo.clone(),
|
|
});
|
|
}
|
|
Seen::Present(_) | Seen::Unknown => {}
|
|
}
|
|
}
|
|
if let Some(png) = &desired.avatar_png
|
|
&& !observed.avatar_set
|
|
{
|
|
actions.push(Action::SetConfigOrgAvatar { png: png.clone() });
|
|
}
|
|
actions
|
|
}
|
|
|
|
/// Whether `t` has every setting [`Client::create_operators_team`] would
|
|
/// give it. Units compare as a set: Forgejo does not promise their order.
|
|
fn team_matches(t: &Team) -> bool {
|
|
let units_match = t.units.as_ref().is_some_and(|units| {
|
|
let mut have: Vec<&str> = units.iter().map(String::as_str).collect();
|
|
let mut want = OPERATORS_TEAM_UNITS.to_vec();
|
|
have.sort_unstable();
|
|
want.sort_unstable();
|
|
have == want
|
|
});
|
|
units_match
|
|
&& t.permission == Some(TeamPermission::Write)
|
|
&& t.includes_all_repositories == Some(true)
|
|
&& t.can_create_org_repo == Some(false)
|
|
&& t.description.as_deref() == Some(OPERATORS_TEAM_DESCRIPTION)
|
|
}
|
|
|
|
/// The merge gate on every config repo's `main`: the `operators` team approves
|
|
/// and merges, and no user does. The empty user list removes any user an
|
|
/// older rule carries.
|
|
///
|
|
/// Forgejo replaces each list this sends wholesale and keeps every field left
|
|
/// `None` as the rule has it, `required_approvals` included.
|
|
fn config_rule_edit() -> EditBranchProtectionOption {
|
|
EditBranchProtectionOption {
|
|
apply_to_admins: None,
|
|
approvals_whitelist_teams: Some(vec![OPERATORS_TEAM.to_owned()]),
|
|
approvals_whitelist_username: None,
|
|
block_on_official_review_requests: None,
|
|
block_on_outdated_branch: None,
|
|
block_on_rejected_reviews: None,
|
|
dismiss_stale_approvals: None,
|
|
enable_approvals_whitelist: Some(true),
|
|
enable_merge_whitelist: Some(true),
|
|
enable_push: None,
|
|
enable_push_whitelist: None,
|
|
enable_status_check: None,
|
|
ignore_stale_approvals: None,
|
|
merge_whitelist_teams: Some(vec![OPERATORS_TEAM.to_owned()]),
|
|
merge_whitelist_usernames: Some(Vec::new()),
|
|
protected_file_patterns: None,
|
|
push_whitelist_deploy_keys: None,
|
|
push_whitelist_teams: None,
|
|
push_whitelist_usernames: None,
|
|
require_signed_commits: None,
|
|
required_approvals: None,
|
|
status_check_contexts: None,
|
|
unprotected_file_patterns: None,
|
|
}
|
|
}
|
|
|
|
/// Whether `rule` already has every field [`config_rule_edit`] sets.
|
|
fn config_rule_matches(rule: &BranchProtection) -> bool {
|
|
let only = |list: &Option<Vec<String>>, want: &str| matches!(list.as_deref(), Some([entry]) if entry == want);
|
|
rule.enable_merge_whitelist == Some(true)
|
|
&& only(&rule.merge_whitelist_teams, OPERATORS_TEAM)
|
|
&& rule
|
|
.merge_whitelist_usernames
|
|
.as_deref()
|
|
.is_none_or(<[String]>::is_empty)
|
|
&& rule.enable_approvals_whitelist == Some(true)
|
|
&& only(&rule.approvals_whitelist_teams, OPERATORS_TEAM)
|
|
}
|
|
|
|
/// Whether an error is Forgejo saying 404: the object is absent, as opposed
|
|
/// to a transport, auth or server failure.
|
|
fn is_not_found(e: &ForgejoError) -> bool {
|
|
match e {
|
|
ForgejoError::ApiError(api) => matches!(api.error_kind(), ApiErrorKind::NotFound { .. }),
|
|
ForgejoError::UnexpectedStatusCode(s) => *s == StatusCode::NOT_FOUND,
|
|
_ => false,
|
|
}
|
|
}
|
|
|
|
/// Fold a read into [`Seen`], logging a failure that is not a 404.
|
|
fn seen<T, U>(res: Result<T, ForgejoError>, what: &str, f: impl FnOnce(T) -> U) -> Seen<U> {
|
|
match res {
|
|
Ok(v) => Seen::Present(f(v)),
|
|
Err(e) if is_not_found(&e) => Seen::Absent,
|
|
Err(e) => {
|
|
tracing::warn!(error = %e, "swarm forge objects: reading {what} failed; skipping it this pass");
|
|
Seen::Unknown
|
|
}
|
|
}
|
|
}
|
|
|
|
/// `EditRepoOption` with every field unset — repo edits only ever change the
|
|
/// one field the caller sets on top (Forgejo leaves `None` fields untouched).
|
|
fn sparse_edit_repo_option() -> EditRepoOption {
|
|
EditRepoOption {
|
|
allow_fast_forward_only_merge: None,
|
|
allow_manual_merge: None,
|
|
allow_merge_commits: None,
|
|
allow_rebase: None,
|
|
allow_rebase_explicit: None,
|
|
allow_rebase_update: None,
|
|
allow_squash_merge: None,
|
|
archived: None,
|
|
autodetect_manual_merge: None,
|
|
default_allow_maintainer_edit: None,
|
|
default_branch: None,
|
|
default_delete_branch_after_merge: None,
|
|
default_merge_style: None,
|
|
default_update_style: None,
|
|
description: None,
|
|
enable_prune: None,
|
|
external_tracker: None,
|
|
external_wiki: None,
|
|
globally_editable_wiki: None,
|
|
has_actions: None,
|
|
has_issues: None,
|
|
has_packages: None,
|
|
has_projects: None,
|
|
has_pull_requests: None,
|
|
has_releases: None,
|
|
has_wiki: None,
|
|
ignore_whitespace_conflicts: None,
|
|
internal_tracker: None,
|
|
mirror_interval: None,
|
|
name: None,
|
|
private: None,
|
|
template: None,
|
|
website: None,
|
|
wiki_branch: None,
|
|
}
|
|
}
|
|
|
|
/// How a pass went, for the one summary line.
|
|
#[derive(Debug, Default, PartialEq)]
|
|
struct PassOutcome {
|
|
applied: usize,
|
|
failed: usize,
|
|
}
|
|
|
|
impl Client {
|
|
/// Read the current state of every object `desired` names.
|
|
async fn observe(&self, desired: &Desired) -> Observed {
|
|
let mut observed = Observed::default();
|
|
for org in &desired.orgs {
|
|
let org_seen = seen(self.api.org_get(org).await, &format!("org {org}"), drop);
|
|
observed.orgs.insert(org.clone(), org_seen);
|
|
}
|
|
for org in desired.team_orgs() {
|
|
if *observed.org(org) != Seen::Present(()) {
|
|
continue;
|
|
}
|
|
let teams = seen(
|
|
self.api.org_list_teams(org).await,
|
|
&format!("teams of {org}"),
|
|
|(_headers, teams)| teams,
|
|
);
|
|
let team = match teams {
|
|
Seen::Present(teams) => teams
|
|
.into_iter()
|
|
.find(|t| t.name.as_deref() == Some(OPERATORS_TEAM))
|
|
.map_or(Seen::Absent, |t| match t.id {
|
|
Some(id) => Seen::Present(TeamState {
|
|
id,
|
|
matches: team_matches(&t),
|
|
}),
|
|
None => Seen::Unknown,
|
|
}),
|
|
Seen::Absent => Seen::Absent,
|
|
Seen::Unknown => Seen::Unknown,
|
|
};
|
|
observed.teams.insert(org.to_owned(), team);
|
|
}
|
|
let repo_keys = desired
|
|
.repos
|
|
.iter()
|
|
.map(|r| (r.owner.to_owned(), r.name.to_owned()))
|
|
.chain(
|
|
desired
|
|
.mirrors
|
|
.iter()
|
|
.map(|m| (m.owner.clone(), m.repo.clone())),
|
|
);
|
|
for (owner, name) in repo_keys {
|
|
if *observed.org(&owner) != Seen::Present(()) {
|
|
continue;
|
|
}
|
|
let repo = seen(
|
|
self.api.repo_get(&owner, &name).await,
|
|
&format!("repo {owner}/{name}"),
|
|
|r| RepoState {
|
|
private: r.private.unwrap_or(false),
|
|
empty: r.empty.unwrap_or(false),
|
|
mirror_interval: r.mirror_interval,
|
|
},
|
|
);
|
|
observed.repos.insert((owner, name), repo);
|
|
}
|
|
observed.avatar_set = desired.avatar_png.is_some() && desired.avatar_marker().exists();
|
|
if desired.config_rules && *observed.org(CONFIG_ORG) == Seen::Present(()) {
|
|
self.observe_config_rules(&mut observed).await;
|
|
}
|
|
observed
|
|
}
|
|
|
|
/// Read every config repo's `main` rule into `observed.config_rules`.
|
|
async fn observe_config_rules(&self, observed: &mut Observed) {
|
|
let repos = match self.api.org_list_repos(CONFIG_ORG).all().await {
|
|
Ok(repos) => repos,
|
|
Err(e) => {
|
|
tracing::warn!(error = %e, "swarm forge objects: listing {CONFIG_ORG} repos failed; skipping their branch rules this pass");
|
|
observed.config_repos_unread = true;
|
|
return;
|
|
}
|
|
};
|
|
for name in repos.into_iter().filter_map(|r| r.name) {
|
|
let rule = seen(
|
|
self.api
|
|
.repo_get_branch_protection(CONFIG_ORG, &name, "main")
|
|
.await,
|
|
&format!("branch protection of {CONFIG_ORG}/{name}"),
|
|
|rule| config_rule_matches(&rule),
|
|
);
|
|
observed.config_rules.insert(name, rule);
|
|
}
|
|
}
|
|
|
|
/// Execute `actions` in order. An action whose org failed to be created
|
|
/// this pass is skipped: it would fail anyway, and its org's failure is
|
|
/// the line worth reading.
|
|
async fn apply(&self, desired: &Desired, actions: &[Action]) -> PassOutcome {
|
|
let mut outcome = PassOutcome::default();
|
|
let mut failed_orgs: Vec<&str> = Vec::new();
|
|
for action in actions {
|
|
if let Some(org) = action.needs_org()
|
|
&& failed_orgs.contains(&org)
|
|
{
|
|
tracing::warn!(?action, %org, "swarm forge objects: skipped, its org could not be created");
|
|
outcome.failed += 1;
|
|
continue;
|
|
}
|
|
match self.apply_one(desired, action).await {
|
|
Ok(()) => outcome.applied += 1,
|
|
Err(e) => {
|
|
tracing::warn!(?action, error = %format!("{e:#}"), "swarm forge objects: write failed; retrying next pass");
|
|
outcome.failed += 1;
|
|
if let Action::CreateOrg { org } = action {
|
|
failed_orgs.push(org);
|
|
}
|
|
}
|
|
}
|
|
}
|
|
outcome
|
|
}
|
|
|
|
async fn apply_one(&self, desired: &Desired, action: &Action) -> Result<()> {
|
|
match action {
|
|
Action::CreateOrg { org } => self.create_org(org).await,
|
|
Action::CreateTeam { org } => self.create_operators_team(org).await,
|
|
Action::ReconcileTeam { org, id } => self.reconcile_operators_team(org, *id).await,
|
|
Action::ConvergeConfigRule { repo } => {
|
|
self.api
|
|
.repo_edit_branch_protection(CONFIG_ORG, repo, "main", config_rule_edit())
|
|
.await
|
|
.with_context(|| format!("converge {CONFIG_ORG}/{repo} main branch rule"))?;
|
|
tracing::info!(%repo, "swarm forge objects: config repo merge gate converged");
|
|
Ok(())
|
|
}
|
|
Action::CreateRepo {
|
|
owner,
|
|
name,
|
|
private,
|
|
} => self.create_shared_repo(owner, name, *private).await,
|
|
Action::SetRepoPublic { owner, name } => {
|
|
let mut edit = sparse_edit_repo_option();
|
|
edit.private = Some(false);
|
|
self.api
|
|
.repo_edit(owner, name, edit)
|
|
.await
|
|
.with_context(|| format!("edit {owner}/{name} (set public)"))?;
|
|
tracing::info!(%owner, %name, "swarm forge objects: repo set to public");
|
|
Ok(())
|
|
}
|
|
Action::SeedReadme { owner, name } => self.seed_readme(owner, name).await,
|
|
Action::CreateMirror {
|
|
owner,
|
|
repo,
|
|
upstream,
|
|
} => self.create_mirror(owner, repo, upstream).await,
|
|
Action::SetMirrorInterval { owner, repo } => {
|
|
let mut edit = sparse_edit_repo_option();
|
|
edit.mirror_interval = Some(MIRROR_INTERVAL.to_owned());
|
|
self.api
|
|
.repo_edit(owner, repo, edit)
|
|
.await
|
|
.with_context(|| format!("set mirror_interval on {owner}/{repo}"))?;
|
|
tracing::info!(%owner, %repo, interval = MIRROR_INTERVAL, "swarm forge objects: pull-mirror interval updated");
|
|
Ok(())
|
|
}
|
|
Action::SetConfigOrgAvatar { png } => {
|
|
self.set_config_org_avatar(png, &desired.avatar_marker())
|
|
.await
|
|
}
|
|
}
|
|
}
|
|
|
|
/// Create `org`. A 409, or a 422 with the org confirmed present, is a
|
|
/// race with another writer and counts as done.
|
|
async fn create_org(&self, org: &str) -> Result<()> {
|
|
let option = CreateOrgOption {
|
|
description: None,
|
|
email: None,
|
|
full_name: None,
|
|
location: None,
|
|
repo_admin_change_team_access: None,
|
|
username: org.to_owned(),
|
|
visibility: None,
|
|
website: None,
|
|
};
|
|
match self.api.org_create(option).await {
|
|
Ok(_) => {
|
|
tracing::info!(%org, "swarm forge objects: created org");
|
|
Ok(())
|
|
}
|
|
Err(e) => {
|
|
let existing =
|
|
is_ambiguous_validation_failure(&e) && self.api.org_get(org).await.is_ok();
|
|
if folds_into_success(&e, existing) {
|
|
Ok(())
|
|
} else {
|
|
Err(e).with_context(|| format!("create org {org}"))
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
/// Provision the [`OPERATORS_TEAM`] inside `org` as an **empty** team.
|
|
/// Branch protection on that org's repos references it as the
|
|
/// merge/approval whitelist; the operator adds herself as a member via the
|
|
/// forge UI. `includes_all_repositories` so the gate applies to every repo
|
|
/// in the org; `write` is enough to approve + merge. This module never
|
|
/// manages membership.
|
|
///
|
|
/// A 422 from `org_create_team` that is not a duplicate is a real
|
|
/// validation error (bad request shape, missing units) and surfaces, so it
|
|
/// can be fixed rather than leave the team silently uncreated every pass.
|
|
async fn create_operators_team(&self, org: &str) -> Result<()> {
|
|
let team = CreateTeamOption {
|
|
can_create_org_repo: Some(false),
|
|
description: Some(OPERATORS_TEAM_DESCRIPTION.to_owned()),
|
|
includes_all_repositories: Some(true),
|
|
name: OPERATORS_TEAM.to_owned(),
|
|
permission: Some(CreateTeamOptionPermission::Write),
|
|
units: Some(OPERATORS_TEAM_UNITS.map(str::to_owned).to_vec()),
|
|
units_map: None,
|
|
};
|
|
match self.api.org_create_team(org, team).await {
|
|
Ok(_) => {
|
|
tracing::info!(%org, "swarm forge objects: created {OPERATORS_TEAM} team");
|
|
Ok(())
|
|
}
|
|
// Forgejo signals a duplicate team as a 409 OR (this build) a 422
|
|
// `validation failed: team already exists`. Confirm the 422 by
|
|
// listing; the next pass reconciles its settings if they differ.
|
|
Err(e) => {
|
|
let existing = is_ambiguous_validation_failure(&e)
|
|
&& self.api.org_list_teams(org).await.is_ok_and(|(_, teams)| {
|
|
teams
|
|
.iter()
|
|
.any(|t| t.name.as_deref() == Some(OPERATORS_TEAM))
|
|
});
|
|
if folds_into_success(&e, existing) {
|
|
Ok(())
|
|
} else {
|
|
Err(e).with_context(|| format!("create team {org}/{OPERATORS_TEAM}"))
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
/// PATCH an existing [`OPERATORS_TEAM`] to the desired settings, so a
|
|
/// team created with an older or wrong shape self-heals. Members are a
|
|
/// separate endpoint and untouched here.
|
|
async fn reconcile_operators_team(&self, org: &str, id: i64) -> Result<()> {
|
|
let edit = EditTeamOption {
|
|
can_create_org_repo: Some(false),
|
|
description: Some(OPERATORS_TEAM_DESCRIPTION.to_owned()),
|
|
includes_all_repositories: Some(true),
|
|
name: OPERATORS_TEAM.to_owned(),
|
|
permission: Some(EditTeamOptionPermission::Write),
|
|
units: Some(OPERATORS_TEAM_UNITS.map(str::to_owned).to_vec()),
|
|
units_map: None,
|
|
};
|
|
self.api
|
|
.org_edit_team(id, edit)
|
|
.await
|
|
.with_context(|| format!("reconcile {org}/{OPERATORS_TEAM} team settings"))?;
|
|
tracing::info!(%org, "swarm forge objects: reconciled {OPERATORS_TEAM} team settings");
|
|
Ok(())
|
|
}
|
|
|
|
/// Create `owner/name`, empty, defaulting to `main`.
|
|
async fn create_shared_repo(&self, owner: &str, name: &str, private: bool) -> Result<()> {
|
|
match self
|
|
.api
|
|
.create_org_repo(owner, Self::repo_option(name, private))
|
|
.await
|
|
{
|
|
Ok(_) => {
|
|
tracing::info!(%owner, %name, private, "swarm forge objects: created repo");
|
|
Ok(())
|
|
}
|
|
Err(e) => {
|
|
let existing = is_ambiguous_validation_failure(&e)
|
|
&& self.api.repo_get(owner, name).await.is_ok();
|
|
if folds_into_success(&e, existing) {
|
|
Ok(())
|
|
} else {
|
|
Err(e).with_context(|| format!("create repo {owner}/{name}"))
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
/// Commit [`KNOWLEDGE_README`] to the empty repo's default branch. Only
|
|
/// planned while the repo reports itself empty, so a README the operator
|
|
/// has since edited is never overwritten.
|
|
async fn seed_readme(&self, owner: &str, name: &str) -> Result<()> {
|
|
let files = vec![ChangeFileOperation {
|
|
content: Some(base64_encode(KNOWLEDGE_README)),
|
|
from_path: None,
|
|
operation: ChangeFileOperationOperation::Create,
|
|
path: "README.md".to_owned(),
|
|
sha: None,
|
|
}];
|
|
self.api
|
|
.repo_change_files(
|
|
owner,
|
|
name,
|
|
ChangeFilesOptions {
|
|
author: None,
|
|
branch: None,
|
|
committer: None,
|
|
dates: None,
|
|
files,
|
|
force_overwrite_new_branch: None,
|
|
message: Some("init: seed README".to_owned()),
|
|
new_branch: None,
|
|
signoff: None,
|
|
},
|
|
)
|
|
.await
|
|
.with_context(|| format!("seed README.md in {owner}/{name}"))?;
|
|
tracing::info!(%owner, %name, "swarm forge objects: seeded README.md");
|
|
Ok(())
|
|
}
|
|
|
|
/// Create `owner/repo` as a pull-mirror of `upstream` via the migrate API.
|
|
/// Only a 409 folds into success (a race created it since the read). NOT
|
|
/// 422: for the migrate endpoint 422 is a validation error (bad
|
|
/// `clone_addr` / service), so it must surface rather than be swallowed as
|
|
/// "already exists".
|
|
async fn create_mirror(&self, owner: &str, repo: &str, upstream: &str) -> Result<()> {
|
|
let opts = MigrateRepoOptions {
|
|
auth_password: None,
|
|
auth_token: None,
|
|
auth_username: None,
|
|
clone_addr: upstream.to_owned(),
|
|
description: None,
|
|
issues: None,
|
|
labels: None,
|
|
lfs: None,
|
|
lfs_endpoint: None,
|
|
milestones: None,
|
|
mirror: Some(true),
|
|
// Periodic refresh instead of on-access sync — keeps CI isolated
|
|
// from external DNS failures at clone time.
|
|
mirror_interval: Some(MIRROR_INTERVAL.to_owned()),
|
|
private: Some(false),
|
|
pull_requests: None,
|
|
releases: None,
|
|
repo_name: repo.to_owned(),
|
|
repo_owner: Some(owner.to_owned()),
|
|
service: Some(MigrateRepoOptionsService::Git),
|
|
uid: None,
|
|
wiki: None,
|
|
};
|
|
match self.api.repo_migrate(opts).await {
|
|
Ok(_) => {
|
|
tracing::info!(%owner, %repo, %upstream, interval = MIRROR_INTERVAL, "swarm forge objects: created pull-mirror");
|
|
Ok(())
|
|
}
|
|
Err(e) if is_confirmed_conflict(&e) => Ok(()),
|
|
Err(e) => Err(e).with_context(|| format!("migrate pull-mirror {owner}/{repo}")),
|
|
}
|
|
}
|
|
|
|
/// Upload `png` as the `agent-configs` org avatar, then write `marker` so
|
|
/// later passes skip it. Best-effort: the forge runs fine with the default
|
|
/// identicon, and a marker that fails to write only costs a re-upload.
|
|
async fn set_config_org_avatar(
|
|
&self,
|
|
png: &std::path::Path,
|
|
marker: &std::path::Path,
|
|
) -> Result<()> {
|
|
let bytes = tokio::fs::read(png)
|
|
.await
|
|
.with_context(|| format!("read {CONFIG_ORG} avatar PNG from {}", png.display()))?;
|
|
self.api
|
|
.org_update_avatar(
|
|
CONFIG_ORG,
|
|
UpdateUserAvatarOption {
|
|
image: Some(base64::engine::general_purpose::STANDARD.encode(&bytes)),
|
|
},
|
|
)
|
|
.await
|
|
.with_context(|| format!("set {CONFIG_ORG} avatar"))?;
|
|
if let Err(e) = std::fs::write(marker, "") {
|
|
tracing::warn!(error = %e, marker = %marker.display(), "swarm forge objects: avatar set, but its marker could not be written");
|
|
}
|
|
tracing::info!(org = CONFIG_ORG, "swarm forge objects: set org avatar");
|
|
Ok(())
|
|
}
|
|
|
|
/// One observe → plan → apply pass over `desired`.
|
|
async fn reconcile(&self, desired: &Desired) -> PassOutcome {
|
|
let observed = self.observe(desired).await;
|
|
let unknown = observed
|
|
.orgs
|
|
.values()
|
|
.filter(|s| **s == Seen::Unknown)
|
|
.count()
|
|
+ observed
|
|
.teams
|
|
.values()
|
|
.filter(|s| **s == Seen::Unknown)
|
|
.count()
|
|
+ observed
|
|
.repos
|
|
.values()
|
|
.filter(|s| **s == Seen::Unknown)
|
|
.count()
|
|
+ observed
|
|
.config_rules
|
|
.values()
|
|
.filter(|s| **s == Seen::Unknown)
|
|
.count()
|
|
+ usize::from(observed.config_repos_unread);
|
|
let actions = plan(desired, &observed);
|
|
let mut outcome = self.apply(desired, &actions).await;
|
|
outcome.failed += unknown;
|
|
outcome
|
|
}
|
|
|
|
/// Make sure the `agent-configs` org and its `operators` team exist, so a
|
|
/// branch-protection rule naming the team can be applied. The periodic
|
|
/// pass normally has done this long before, but an agent can be created
|
|
/// while that pass is still failing (a controller that started with the
|
|
/// forge), so [`Client::create_repo`] checks first rather than rely on
|
|
/// the order.
|
|
pub(super) async fn ensure_merge_gate_prerequisites(&self) -> Result<()> {
|
|
let outcome = self.reconcile(&Desired::merge_gate_only()).await;
|
|
if outcome.failed > 0 {
|
|
anyhow::bail!(
|
|
"the {CONFIG_ORG} org or its {OPERATORS_TEAM} team could not be ensured \
|
|
({} failure(s), logged above)",
|
|
outcome.failed
|
|
);
|
|
}
|
|
Ok(())
|
|
}
|
|
}
|
|
|
|
/// Ensure the swarm-wide forge objects now and every [`RECONCILE_INTERVAL`].
|
|
///
|
|
/// The first tick fires immediately (`tokio::time::interval`'s default), the
|
|
/// same shape as `config_pr::spawn`. A pass that fails leaves a `warn` per
|
|
/// failed object plus the summary line below in the controller's journal, and
|
|
/// is retried on the next tick; it never stops the daemon.
|
|
pub fn spawn(client: Arc<Client>) {
|
|
let desired = Desired::from_env();
|
|
tokio::spawn(async move {
|
|
let mut ticker = tokio::time::interval(RECONCILE_INTERVAL);
|
|
loop {
|
|
ticker.tick().await;
|
|
let Some(outcome) =
|
|
super::within_forge_deadline("swarm forge objects", client.reconcile(&desired))
|
|
.await
|
|
else {
|
|
continue;
|
|
};
|
|
if outcome.failed > 0 {
|
|
tracing::warn!(
|
|
applied = outcome.applied,
|
|
failed = outcome.failed,
|
|
retry_in_s = RECONCILE_INTERVAL.as_secs(),
|
|
"swarm forge objects: pass incomplete; retrying next tick"
|
|
);
|
|
} else if outcome.applied > 0 {
|
|
tracing::info!(
|
|
applied = outcome.applied,
|
|
"swarm forge objects: pass converged"
|
|
);
|
|
} else {
|
|
tracing::debug!("swarm forge objects: already converged");
|
|
}
|
|
}
|
|
});
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
fn mirror(owner: &str, repo: &str) -> MirrorSpec {
|
|
MirrorSpec {
|
|
owner: owner.to_owned(),
|
|
repo: repo.to_owned(),
|
|
upstream: format!("https://example.invalid/{repo}"),
|
|
}
|
|
}
|
|
|
|
fn desired() -> Desired {
|
|
Desired::new(
|
|
vec![mirror("actions", "checkout")],
|
|
Some(PathBuf::from("/avatar.png")),
|
|
PathBuf::from("/state"),
|
|
)
|
|
}
|
|
|
|
fn repo_state(private: bool, empty: bool, interval: Option<&str>) -> RepoState {
|
|
RepoState {
|
|
private,
|
|
empty,
|
|
mirror_interval: interval.map(str::to_owned),
|
|
}
|
|
}
|
|
|
|
/// Every object of [`desired`] present with its desired settings.
|
|
fn converged() -> Observed {
|
|
let mut o = Observed::default();
|
|
for org in [CONFIG_ORG, SHARED_ORG, AGENTS_ORG, "actions"] {
|
|
o.orgs.insert(org.to_owned(), Seen::Present(()));
|
|
}
|
|
for (id, org) in [(1, AGENTS_ORG), (2, CONFIG_ORG)] {
|
|
o.teams.insert(
|
|
org.to_owned(),
|
|
Seen::Present(TeamState { id, matches: true }),
|
|
);
|
|
}
|
|
let key = |a: &str, b: &str| (a.to_owned(), b.to_owned());
|
|
o.repos.insert(
|
|
key(SHARED_ORG, SHARED_DOCS_REPO),
|
|
Seen::Present(repo_state(true, true, None)),
|
|
);
|
|
o.repos.insert(
|
|
key(KNOWLEDGE_ORG, KNOWLEDGE_REPO),
|
|
Seen::Present(repo_state(false, false, None)),
|
|
);
|
|
o.repos.insert(
|
|
key("actions", "checkout"),
|
|
Seen::Present(repo_state(false, false, Some(MIRROR_INTERVAL))),
|
|
);
|
|
o.avatar_set = true;
|
|
o
|
|
}
|
|
|
|
/// Every object of [`desired`] reported absent.
|
|
fn nothing() -> Observed {
|
|
let mut o = Observed::default();
|
|
for org in [CONFIG_ORG, SHARED_ORG, AGENTS_ORG, "actions"] {
|
|
o.orgs.insert(org.to_owned(), Seen::Absent);
|
|
}
|
|
o
|
|
}
|
|
|
|
fn s(v: &str) -> String {
|
|
v.to_owned()
|
|
}
|
|
|
|
#[test]
|
|
fn nothing_exists_so_everything_is_created_orgs_first() {
|
|
assert_eq!(
|
|
plan(&desired(), ¬hing()),
|
|
vec![
|
|
Action::CreateOrg { org: s(CONFIG_ORG) },
|
|
Action::CreateOrg { org: s(SHARED_ORG) },
|
|
Action::CreateOrg { org: s(AGENTS_ORG) },
|
|
Action::CreateOrg { org: s("actions") },
|
|
Action::CreateTeam { org: s(AGENTS_ORG) },
|
|
Action::CreateTeam { org: s(CONFIG_ORG) },
|
|
Action::CreateRepo {
|
|
owner: s(SHARED_ORG),
|
|
name: s(SHARED_DOCS_REPO),
|
|
private: true,
|
|
},
|
|
Action::CreateRepo {
|
|
owner: s(KNOWLEDGE_ORG),
|
|
name: s(KNOWLEDGE_REPO),
|
|
private: false,
|
|
},
|
|
Action::SeedReadme {
|
|
owner: s(KNOWLEDGE_ORG),
|
|
name: s(KNOWLEDGE_REPO),
|
|
},
|
|
Action::CreateMirror {
|
|
owner: s("actions"),
|
|
repo: s("checkout"),
|
|
upstream: s("https://example.invalid/checkout"),
|
|
},
|
|
Action::SetConfigOrgAvatar {
|
|
png: PathBuf::from("/avatar.png"),
|
|
},
|
|
]
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn everything_exists_so_nothing_is_written() {
|
|
assert_eq!(plan(&desired(), &converged()), Vec::<Action>::new());
|
|
}
|
|
|
|
#[test]
|
|
fn partial_state_writes_only_the_gaps() {
|
|
let mut o = converged();
|
|
// The team is missing in agent-configs only, the agents one has an
|
|
// old shape, knowledge is private and still empty, the mirror
|
|
// predates the interval, and the avatar was never uploaded.
|
|
o.teams.insert(s(CONFIG_ORG), Seen::Absent);
|
|
o.teams.insert(
|
|
s(AGENTS_ORG),
|
|
Seen::Present(TeamState {
|
|
id: 7,
|
|
matches: false,
|
|
}),
|
|
);
|
|
o.repos.insert(
|
|
(s(KNOWLEDGE_ORG), s(KNOWLEDGE_REPO)),
|
|
Seen::Present(repo_state(true, true, None)),
|
|
);
|
|
o.repos.insert(
|
|
(s("actions"), s("checkout")),
|
|
Seen::Present(repo_state(false, false, None)),
|
|
);
|
|
o.avatar_set = false;
|
|
assert_eq!(
|
|
plan(&desired(), &o),
|
|
vec![
|
|
Action::ReconcileTeam {
|
|
org: s(AGENTS_ORG),
|
|
id: 7,
|
|
},
|
|
Action::CreateTeam { org: s(CONFIG_ORG) },
|
|
Action::SetRepoPublic {
|
|
owner: s(KNOWLEDGE_ORG),
|
|
name: s(KNOWLEDGE_REPO),
|
|
},
|
|
Action::SeedReadme {
|
|
owner: s(KNOWLEDGE_ORG),
|
|
name: s(KNOWLEDGE_REPO),
|
|
},
|
|
Action::SetMirrorInterval {
|
|
owner: s("actions"),
|
|
repo: s("checkout"),
|
|
},
|
|
Action::SetConfigOrgAvatar {
|
|
png: PathBuf::from("/avatar.png"),
|
|
},
|
|
]
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn an_unreadable_object_gets_no_write() {
|
|
let mut o = nothing();
|
|
o.orgs.insert(s(SHARED_ORG), Seen::Unknown);
|
|
let actions = plan(&desired(), &o);
|
|
assert!(!actions.contains(&Action::CreateOrg { org: s(SHARED_ORG) }));
|
|
assert!(
|
|
!actions.iter().any(|a| a.needs_org() == Some(SHARED_ORG)),
|
|
"nothing inside an org whose state is unknown: {actions:?}"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn the_merge_gate_subset_is_the_config_org_and_its_team() {
|
|
let mut o = Observed::default();
|
|
o.orgs.insert(s(CONFIG_ORG), Seen::Absent);
|
|
assert_eq!(
|
|
plan(&Desired::merge_gate_only(), &o),
|
|
vec![
|
|
Action::CreateOrg { org: s(CONFIG_ORG) },
|
|
Action::CreateTeam { org: s(CONFIG_ORG) },
|
|
]
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn only_a_config_rule_that_does_not_match_is_converged() {
|
|
let mut o = converged();
|
|
o.config_rules.insert(s("atlas"), Seen::Present(true));
|
|
o.config_rules.insert(s("damocles"), Seen::Present(false));
|
|
o.config_rules.insert(s("iris"), Seen::Absent);
|
|
o.config_rules.insert(s("argus"), Seen::Unknown);
|
|
assert_eq!(
|
|
plan(&desired(), &o),
|
|
vec![Action::ConvergeConfigRule {
|
|
repo: s("damocles")
|
|
}]
|
|
);
|
|
}
|
|
|
|
fn rule(merge_users: &[&str]) -> BranchProtection {
|
|
serde_json::from_value(serde_json::json!({
|
|
"enable_merge_whitelist": true,
|
|
"merge_whitelist_teams": [OPERATORS_TEAM],
|
|
"merge_whitelist_usernames": merge_users,
|
|
"enable_approvals_whitelist": true,
|
|
"approvals_whitelist_teams": [OPERATORS_TEAM],
|
|
}))
|
|
.expect("branch protection json")
|
|
}
|
|
|
|
#[test]
|
|
fn a_config_rule_matches_only_with_operators_and_no_user() {
|
|
assert!(config_rule_matches(&rule(&[])));
|
|
let mut users_absent = rule(&[]);
|
|
users_absent.merge_whitelist_usernames = None;
|
|
assert!(config_rule_matches(&users_absent));
|
|
assert!(!config_rule_matches(&rule(&["core"])));
|
|
let mut no_team = rule(&[]);
|
|
no_team.merge_whitelist_teams = None;
|
|
assert!(!config_rule_matches(&no_team));
|
|
let mut approvals_off = rule(&[]);
|
|
approvals_off.enable_approvals_whitelist = Some(false);
|
|
assert!(!config_rule_matches(&approvals_off));
|
|
}
|
|
|
|
/// The edit is what the match checks for, so one PATCH converges a rule.
|
|
#[test]
|
|
fn the_config_rule_edit_produces_a_matching_rule() {
|
|
let edit = config_rule_edit();
|
|
let applied: BranchProtection = serde_json::from_value(serde_json::json!({
|
|
"enable_merge_whitelist": edit.enable_merge_whitelist,
|
|
"merge_whitelist_teams": edit.merge_whitelist_teams,
|
|
"merge_whitelist_usernames": edit.merge_whitelist_usernames,
|
|
"enable_approvals_whitelist": edit.enable_approvals_whitelist,
|
|
"approvals_whitelist_teams": edit.approvals_whitelist_teams,
|
|
}))
|
|
.expect("branch protection json");
|
|
assert!(config_rule_matches(&applied));
|
|
}
|
|
|
|
#[test]
|
|
fn mirror_owners_join_the_seeded_orgs_once() {
|
|
let d = Desired::new(
|
|
vec![mirror("actions", "checkout"), mirror("actions", "cache")],
|
|
None,
|
|
PathBuf::new(),
|
|
);
|
|
assert_eq!(d.orgs, [CONFIG_ORG, SHARED_ORG, AGENTS_ORG, "actions"]);
|
|
}
|
|
|
|
#[test]
|
|
fn a_malformed_mirror_entry_is_dropped_alone() {
|
|
let parsed = parse_mirrors(
|
|
r#"[{"upstream":"https://x/a","dest":"no-slash"},{"upstream":"https://x/b","dest":"o/b"}]"#,
|
|
);
|
|
assert_eq!(
|
|
parsed,
|
|
vec![MirrorSpec {
|
|
owner: s("o"),
|
|
repo: s("b"),
|
|
upstream: s("https://x/b"),
|
|
}]
|
|
);
|
|
assert!(parse_mirrors("not json").is_empty());
|
|
assert!(parse_mirrors("").is_empty());
|
|
}
|
|
|
|
fn team(units: &[&str]) -> Team {
|
|
serde_json::from_value(serde_json::json!({
|
|
"id": 1,
|
|
"name": OPERATORS_TEAM,
|
|
"description": OPERATORS_TEAM_DESCRIPTION,
|
|
"permission": "write",
|
|
"includes_all_repositories": true,
|
|
"can_create_org_repo": false,
|
|
"units": units,
|
|
}))
|
|
.expect("team json")
|
|
}
|
|
|
|
#[test]
|
|
fn team_units_compare_as_a_set() {
|
|
let mut reversed = OPERATORS_TEAM_UNITS;
|
|
reversed.reverse();
|
|
assert!(team_matches(&team(&reversed)));
|
|
assert!(!team_matches(&team(&OPERATORS_TEAM_UNITS[1..])));
|
|
}
|
|
}
|