vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

swarm-controller: own the swarm-wide forge objects; hive-c0re stops creating them

Browse source 
The orgs agent-configs/internal/agents (plus mirror owners), the
operators team in agents and agent-configs, the pull-mirrors,
internal/docs, internal/knowledge (public, README-seeded) and the
agent-configs org avatar are one set per forge. hive-c0re ensured them in
its boot sweep, as the core admin, and only on the hive co-located with
the forge container.

swarm-controller now reconciles them at start and every 5 minutes
(forge/objects.rs: observe -> pure plan -> apply). A failed object logs
a warn line plus a pass summary and is retried next tick. create_repo
ensures the agent-configs org and its operators team first, so a config
repo's merge gate never depends on the periodic pass having run.

hive-c0re drops ensure_org, SEEDED_ORGS, ensure_mirrors/ensure_mirror_repo,
ensure_operators_team, ensure_shared_docs_repo, ensure_knowledge_repo/
set_repo_public, seed_readme, ensure_config_org_avatar and the one-shot
knowledge::remove_webhook cleanup, with their now-unused helpers.

nix: the mirror list moves from the hive-c0re unit
(HYPERHIVE_FORGE_MIRRORS) to the swarm-controller unit
(SWARM_CONTROLLER_FORGE_MIRRORS), with an eval warning when mirrors are
declared on a host that runs no controller. c0re.orgAvatarPng is renamed
to deploy.swarm-controller.configOrgAvatarPng.

Refs #3782
This commit is contained in:
atlas 2026-09-24 15:00:10 +02:00
commit 20419ccd41
18 changed files with 1456 additions and 822 deletions
Download patch file Download diff file Expand all files Collapse all files

424
hive-c0re/src/forge/repos.rs
View file

@ -10,9 +10,7 @@ use std::sync::Mutex;
use anyhow::{Context, Result};
use forgejo_api::structs::{
AddCollaboratorOption, AddCollaboratorOptionPermission, CreateBranchProtectionOption,
CreateOrgOption, CreateRepoOption, CreateTeamOption, CreateTeamOptionPermission,
EditBranchProtectionOption, EditRepoOption, EditTeamOption, EditTeamOptionPermission,
MigrateRepoOptions, MigrateRepoOptionsService, Repository,
CreateRepoOption, EditBranchProtectionOption, Repository,
};
use forgejo_api::{ApiErrorKind, ForgejoError};
use reqwest::StatusCode;
@ -20,8 +18,8 @@ use reqwest::StatusCode;
use crate::coordinator::Coordinator;
use super::{
AGENTS_ORG, CONFIG_ORG, KNOWLEDGE_REPO, OPERATORS_TEAM, SHARED_DOCS_REPO, SHARED_ORG, api,
core_auth_header, core_token, forge_git_url, forge_http_base, is_present,
AGENTS_ORG, CONFIG_ORG, OPERATORS_TEAM, SHARED_DOCS_REPO, SHARED_ORG, api, core_auth_header,
core_token, forge_git_url, forge_http_base, is_present,
};
/// Creation options for an empty repo defaulting to `main`.
@ -42,48 +40,6 @@ fn repo_option(name: &str, private: bool) -> CreateRepoOption {
}
}
/// `EditRepoOption` with every field unset — repo edits only ever
/// change the one field the caller sets on top (Forgejo leaves `None`
/// fields untouched).
fn sparse_edit_repo_option() -> EditRepoOption {
EditRepoOption {
allow_fast_forward_only_merge: None,
allow_manual_merge: None,
allow_merge_commits: None,
allow_rebase: None,
allow_rebase_explicit: None,
allow_rebase_update: None,
allow_squash_merge: None,
archived: None,
autodetect_manual_merge: None,
default_allow_maintainer_edit: None,
default_branch: None,
default_delete_branch_after_merge: None,
default_merge_style: None,
default_update_style: None,
description: None,
enable_prune: None,
external_tracker: None,
external_wiki: None,
globally_editable_wiki: None,
has_actions: None,
has_issues: None,
has_packages: None,
has_projects: None,
has_pull_requests: None,
has_releases: None,
has_wiki: None,
ignore_whitespace_conflicts: None,
internal_tracker: None,
mirror_interval: None,
name: None,
private: None,
template: None,
website: None,
wiki_branch: None,
}
}
/// Whether a create-style call failed because the object already
/// exists. Forgejo signals this as HTTP 409 (conflict) or 422
/// (validation). The typed client surfaces those as
@ -104,59 +60,6 @@ fn is_already_exists(e: &ForgejoError) -> bool {
}
}
/// Whether an error is specifically an HTTP 409 conflict (and NOT a
/// 422): the migrate endpoint's 422 is a validation error (bad
/// `clone_addr` / service) and must surface, so it can't share
/// [`is_already_exists`]'s 422 tolerance.
fn is_conflict(e: &ForgejoError) -> bool {
match e {
ForgejoError::ApiError(api) => {
matches!(api.error_kind(), ApiErrorKind::Other(s) if *s == StatusCode::CONFLICT)
}
ForgejoError::UnexpectedStatusCode(s) => *s == StatusCode::CONFLICT,
_ => false,
}
}
/// Whether a rendered error message names the "already exists" case. The
/// discriminator that tells a *benign* already-exists 422 apart from a
/// *real* validation 422 (invalid units, etc.). Case-insensitive.
fn message_says_already_exists(rendered: &str) -> bool {
rendered.to_lowercase().contains("already exists")
}
/// Whether `e` is Forgejo saying the resource already exists — matching
/// the 409 conflict shape *and* the 422 shape this Forgejo build actually
/// returns for a duplicate team: `validation failed: team already exists`.
/// A 409-only [`is_conflict`] check misses that 422, so the caller fires
/// a spurious "provisioning failed" warning every boot and skips its
/// settings reconcile. This still surfaces *other* 422s (bad request
/// body) as real failures — only a 422 whose message names the
/// already-exists case is folded in.
fn is_already_exists_lenient(e: &ForgejoError) -> bool {
if is_conflict(e) {
return true;
}
let is_unprocessable = match e {
ForgejoError::ApiError(api) => matches!(api.error_kind(), ApiErrorKind::ValidationFailed),
ForgejoError::UnexpectedStatusCode(s) => *s == StatusCode::UNPROCESSABLE_ENTITY,
_ => false,
};
is_unprocessable && message_says_already_exists(&e.to_string())
}
/// Whether an error is Forgejo saying 404 — the resource is absent,
/// as opposed to a transport / auth / server failure.
fn is_not_found(e: &ForgejoError) -> bool {
match e {
ForgejoError::ApiError(api) => {
matches!(api.error_kind(), ApiErrorKind::NotFound { .. })
}
ForgejoError::UnexpectedStatusCode(s) => *s == StatusCode::NOT_FOUND,
_ => false,
}
}
/// Fold a repo-creation result's "already exists" (409 / 422) into
/// success. `label` is `<owner>/<name>` — purely for log + error
/// context.
@ -174,28 +77,6 @@ fn created_or_exists(res: Result<Repository, ForgejoError>, label: &str) -> Resu
}
}
/// Set an existing repo to public visibility. No-op if the repo is
/// already public. Used for `internal/knowledge` which may have been
/// created as private on an older deployment.
async fn set_repo_public(owner: &str, repo: &str, token: &str) -> Result<()> {
let mut edit = sparse_edit_repo_option();
edit.private = Some(false);
api(token)?
.repo_edit(owner, repo, edit)
.await
.with_context(|| format!("edit {owner}/{repo} (set public)"))?;
tracing::debug!(%owner, %repo, "forge: repo set to public");
Ok(())
}
/// Create `name` inside org `org` as a public repo. Idempotent.
async fn ensure_org_repo_public(org: &str, name: &str, token: &str) -> Result<()> {
let res = api(token)?
.create_org_repo(org, repo_option(name, false))
.await;
created_or_exists(res, &format!("{org}/{name}"))
}
/// Create a repo in the token-owner's own namespace. `token` belongs
/// to the user we want the repo owned by (we use `core`'s token for
/// `core/meta`). Idempotent.
@ -356,13 +237,6 @@ fn record_branch_protection_result(name: &str, ok: bool) {
}
}
/// Ensure the `internal/docs` repo exists. Called once at startup
/// after `ensure_org(SHARED_ORG)`. Idempotent — `ensure_org_repo`
/// treats 409 as success.
pub async fn ensure_shared_docs_repo(core_token: &str) -> Result<()> {
ensure_org_repo(SHARED_ORG, SHARED_DOCS_REPO, core_token).await
}
/// Grant agent `name` read-only collaborator access to `internal/docs`.
/// Idempotent: re-adding an existing collaborator succeeds (Forgejo
/// answers 204 either way). Mirrors `meta_read_access` so agents can
@ -380,19 +254,6 @@ pub async fn shared_docs_access(name: &str, core_token: &str) -> Result<()> {
Ok(())
}
/// Ensure the `internal/knowledge` repo exists and is public.
/// Called once at startup after `ensure_org(SHARED_ORG)`. Idempotent.
///
/// The repo is created as public so any agent with a forge account can
/// fork it and open PRs to contribute. Existing deployments that ended
/// up with a private repo are patched to public on the next hive-c0re
/// startup via `set_repo_public`.
pub async fn ensure_knowledge_repo(core_token: &str) -> Result<()> {
ensure_org_repo_public(SHARED_ORG, KNOWLEDGE_REPO, core_token).await?;
// Ensure public even if the repo already existed as private (older deployment).
set_repo_public(SHARED_ORG, KNOWLEDGE_REPO, core_token).await
}
/// Grant agent `name` read-only collaborator access to `core/meta` on
/// the forge so the agent can clone/fetch the meta flake. Idempotent:
/// re-adding an existing collaborator succeeds (Forgejo answers 204
@ -732,261 +593,6 @@ async fn run_config_push(
.context("invoke git push agent-configs")
}
/// Create an org named `name` (`org_create`). Idempotent: HTTP 422
/// ("user already exists") / 409 is treated as success.
pub(super) async fn ensure_org(name: &str, admin_token: &str) -> Result<()> {
let org = CreateOrgOption {
description: None,
email: None,
full_name: None,
location: None,
repo_admin_change_team_access: None,
username: name.to_owned(),
visibility: None,
website: None,
};
match api(admin_token)?.org_create(org).await {
Ok(_) => {
tracing::info!(%name, "forge: created org");
Ok(())
}
Err(e) if is_already_exists(&e) => {
tracing::debug!(%name, "forge: org already exists");
Ok(())
}
Err(e) => Err(e).with_context(|| format!("create org {name}")),
}
}
/// One operator-declared pull-mirror, forwarded from the nix
/// `services.hyperhive.forge.mirrors` option as JSON in
/// `HYPERHIVE_FORGE_MIRRORS`.
#[derive(serde::Deserialize)]
struct Mirror {
/// Upstream clone URL to mirror from (e.g. `https://github.com/actions/checkout`).
upstream: String,
/// Local `<owner>/<repo>` the mirror is created at.
dest: String,
}
/// Ensure each `HYPERHIVE_FORGE_MIRRORS` entry exists as a real Forgejo
/// pull-mirror. The env carries the JSON-encoded nix `forge.mirrors` list
/// (plus the CI-auto `actions/checkout` entry). Absent/empty env = no-op.
/// Per-mirror failures warn and continue — never abort the startup sweep.
pub(super) async fn ensure_mirrors(admin_token: &str) {
let raw = match std::env::var("HYPERHIVE_FORGE_MIRRORS") {
Ok(s) if !s.trim().is_empty() => s,
_ => return,
};
let mirrors: Vec<Mirror> = match serde_json::from_str(&raw) {
Ok(m) => m,
Err(e) => {
tracing::warn!(error = ?e, "forge: HYPERHIVE_FORGE_MIRRORS is not valid JSON; skipping mirror seed");
return;
}
};
for m in mirrors {
let Some((owner, repo)) = m.dest.split_once('/') else {
tracing::warn!(dest = %m.dest, "forge: mirror dest is not <owner>/<repo>; skipping");
continue;
};
// Create the dest org first (idempotent); the mirror can't land
// without its owner existing.
if let Err(e) = ensure_org(owner, admin_token).await {
tracing::warn!(%owner, error = ?e, "forge: ensure_org for mirror failed");
continue;
}
if let Err(e) = ensure_mirror_repo(&m.upstream, owner, repo, admin_token).await {
tracing::warn!(dest = %m.dest, error = ?e, "forge: ensure_mirror_repo failed");
}
}
}
/// Periodic sync interval for pull-mirrors. Forgejo syncs mirrors
/// on-access by default, which re-introduces external DNS latency on
/// every `git clone` (the hive-ci runner shares the host netns and is
/// therefore affected by host resolver blips). A fixed periodic interval
/// isolates CI from transient DNS failures — a stale mirror is
/// acceptable; a broken clone because of a momentary DNS blip is not.
const MIRROR_INTERVAL: &str = "8h0m0s";
/// Create `owner/repo` as a pull-mirror of `upstream` via the migrate API.
/// Idempotent: if the repo already exists this function patches its
/// `mirror_interval` to ensure it matches (covers mirrors that were
/// created before the interval was introduced). A 409 on the migrate
/// call (a race between the existence check and the migrate) is also
/// success.
async fn ensure_mirror_repo(
upstream: &str,
owner: &str,
repo: &str,
admin_token: &str,
) -> Result<()> {
let client = api(admin_token)?;
match client.repo_get(owner, repo).await {
Ok(_) => {
// Mirror already present. Patch interval so mirrors seeded before
// this field was introduced (or with a different value) converge.
let mut edit = sparse_edit_repo_option();
edit.mirror_interval = Some(MIRROR_INTERVAL.to_owned());
match client.repo_edit(owner, repo, edit).await {
Ok(_) => {
tracing::debug!(%owner, %repo, interval = MIRROR_INTERVAL, "forge: pull-mirror interval updated");
}
Err(e) => {
tracing::warn!(
%owner, %repo, error = %e,
"forge: failed to set mirror_interval on existing pull-mirror"
);
}
}
return Ok(());
}
// Absent — fall through to migrate.
Err(e) if is_not_found(&e) => {}
// Anything else (transport, auth, 5xx) leaves the repo's existence
// unknown: migrating anyway would fold a 409 into success and skip
// the interval patch this pass. Surface it instead.
Err(e) => {
return Err(e).with_context(|| format!("get pull-mirror {owner}/{repo}"));
}
}
let opts = MigrateRepoOptions {
auth_password: None,
auth_token: None,
auth_username: None,
clone_addr: upstream.to_owned(),
description: None,
issues: None,
labels: None,
lfs: None,
lfs_endpoint: None,
milestones: None,
mirror: Some(true),
// Periodic refresh instead of on-access sync — keeps CI isolated
// from external DNS failures at clone time.
mirror_interval: Some(MIRROR_INTERVAL.to_owned()),
private: Some(false),
pull_requests: None,
releases: None,
repo_name: repo.to_owned(),
repo_owner: Some(owner.to_owned()),
service: Some(MigrateRepoOptionsService::Git),
uid: None,
wiki: None,
};
match client.repo_migrate(opts).await {
Ok(_) => {
tracing::info!(%owner, %repo, %upstream, interval = MIRROR_INTERVAL, "forge: created pull-mirror");
Ok(())
}
// 409 = a race created it between our existence check and here (the
// check is the real idempotency guard). NOT 422: for the migrate
// endpoint 422 is a validation error (bad clone_addr / service), so
// it must surface via the error arm, not be swallowed as "already
// exists" — hence `is_conflict`, not `is_already_exists`.
Err(e) if is_conflict(&e) => {
tracing::debug!(%owner, %repo, "forge: pull-mirror already exists (race)");
Ok(())
}
Err(e) => Err(e).with_context(|| format!("migrate pull-mirror {owner}/{repo}")),
}
}
/// Provision the [`OPERATORS_TEAM`] inside `org` as an **empty** team.
/// Branch protection on that org's repos references it as the
/// merge/approval whitelist; the operator adds herself as a member via the
/// forge UI / hivectl. `includes_all_repositories` so the gate applies to
/// every repo in the org; `write` is enough to approve + merge. hive-c0re
/// never manages membership. Idempotent (409 = already exists).
///
/// Must run for BOTH [`AGENTS_ORG`] and [`CONFIG_ORG`]: Gitea teams are
/// org-scoped, so a config-repo branch-protection rule referencing
/// `operators` needs the team to exist in `agent-configs` too. Missing it
/// there 422'd every `apply_config_repo_branch_protection`, leaving config
/// repos unprotected — operator-merged config PRs then bypassed the deploy
/// pipeline and silently didn't apply.
///
/// Uses `is_conflict` (409 only) — NOT `is_already_exists` (which also
/// folds 422 into "already exists"). A 422 from `org_create_team` is a
/// real validation error (bad request shape, missing units, etc.) that
/// must surface so it can be fixed; the previous 422-swallowing hid the
/// true cause and left the team silently uncreated every boot.
/// Repo-unit access flags for the `operators` team.
/// Explicit list so Forgejo doesn't reject a null/absent `units` field;
/// a `write`-permission team needs at least `repo.code` + `repo.pulls`
/// to review and merge PRs.
const OPERATORS_TEAM_UNITS: &[&str] = &[
"repo.code",
"repo.issues",
"repo.pulls",
"repo.releases",
"repo.wiki",
"repo.projects",
"repo.packages",
];
pub(super) async fn ensure_operators_team(org: &str, token: &str) -> Result<()> {
let units_vec: Vec<String> = OPERATORS_TEAM_UNITS.iter().map(|&s| s.to_owned()).collect();
let team = CreateTeamOption {
can_create_org_repo: Some(false),
description: Some("hyperhive operators — merge gate for agent repos".to_owned()),
includes_all_repositories: Some(true),
name: OPERATORS_TEAM.to_owned(),
permission: Some(CreateTeamOptionPermission::Write),
units: Some(units_vec.clone()),
units_map: None,
};
let client = api(token)?;
match client.org_create_team(org, team).await {
Ok(_) => {
tracing::info!(%org, "forge: created {OPERATORS_TEAM} team");
Ok(())
}
// Team already exists — reconcile settings to desired state so a
// team created with an older/wrong shape self-heals on next boot.
// Forgejo signals the duplicate as a 409 conflict OR (this build) a
// 422 `validation failed: team already exists`; both mean the same
// thing, so fold both in via `is_already_exists_lenient` — a 409-only
// check missed the 422 and warned every boot. List teams to find the
// id (required by org_edit_team), then unconditionally PATCH to the
// desired settings. Members are a separate endpoint; untouched here.
Err(e) if is_already_exists_lenient(&e) => {
let (_headers, teams) = client
.org_list_teams(org)
.await
.with_context(|| format!("list teams for {org}"))?;
let team_id = teams
.into_iter()
.find(|t| t.name.as_deref() == Some(OPERATORS_TEAM))
.and_then(|t| t.id)
.with_context(|| {
format!("{OPERATORS_TEAM} team not found in {org} after 409 conflict")
})?;
let edit = EditTeamOption {
can_create_org_repo: Some(false),
description: Some("hyperhive operators — merge gate for agent repos".to_owned()),
includes_all_repositories: Some(true),
name: OPERATORS_TEAM.to_owned(),
permission: Some(EditTeamOptionPermission::Write),
units: Some(units_vec),
units_map: None,
};
client
.org_edit_team(team_id, edit)
.await
.with_context(|| format!("reconcile {org}/{OPERATORS_TEAM} team settings"))?;
tracing::debug!(%org, "forge: reconciled {OPERATORS_TEAM} team settings");
Ok(())
}
// Any OTHER error (a 422 that is NOT already-exists, or a transport
// / auth failure): surface it — don't mask a real failure. A
// non-already-exists 422 means the request body is invalid (e.g.
// Forgejo rejected the units list); it repeats every boot until fixed.
Err(e) => Err(e).with_context(|| format!("create team {org}/{OPERATORS_TEAM}")),
}
}
/// Add `user` as a collaborator on `owner/repo` at `permission`.
/// Idempotent: adding an existing collaborator just updates its
/// permission (Forgejo answers 204 either way; a 201 from older
@ -1224,27 +830,3 @@ pub async fn create_agent_repo(agent: &str, repo: &str, core_token: &str) -> Res
tracing::info!(%agent, %repo, "forge: created agent repo in {AGENTS_ORG} with operator merge gate");
Ok(format!("{AGENTS_ORG}/{repo}"))
}
#[cfg(test)]
mod tests {
use super::message_says_already_exists;
#[test]
fn already_exists_message_is_recognised() {
// The exact 422 body this Forgejo build returns for a duplicate team.
assert!(message_says_already_exists(
"validation failed: team already exists [org_id: 10, name: operators]"
));
// Case-insensitive.
assert!(message_says_already_exists("Repository Already Exists"));
}
#[test]
fn real_validation_error_is_not_treated_as_already_exists() {
// A genuine bad-request 422 must still surface, not be folded in.
assert!(!message_says_already_exists(
"validation failed: units must not be empty"
));
assert!(!message_says_already_exists("not found"));
}
}