vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(#3255): hives stop owning the knowledge webhook, and clean up their own

Browse source 
A webhook has exactly one target URL, so every hive registering one
against the shared internal/knowledge repository was last-writer-wins
rather than idempotent: all but the most recent silently stopped
receiving deliveries. The swarm controller holds the single registration
and now addresses an event to each hive over the queue instead.

This is a migration, not a deletion. Not registering any more fixes
nothing on a hive that has already run — the hook it created persists on
the forge, so the contention would survive on exactly the deployments
that have it while fresh installs looked fixed. The hive that created a
hook removes it.

It removes only its OWN, matched on the full URL rather than the
/webhook/knowledge suffix. A hook with that suffix and a different base
belongs to another hive, possibly one not yet upgraded, and deleting it
would break that hive's knowledge sync until it caught up. Reaping a
neighbour's registration is the behaviour being removed here; doing it
while fixing it would only invert the direction.

The predecessor did reap by suffix, to clear loopback hooks left by an
older single-hive layout. That was safe when a hive was alone on its
forge and is not safe now. The hive-side registrars also acted as reapers
of hooks under their own path, which is why the swarm hook lives under
/webhook/forge/; removing this registrar removes that reaper too.
Intended, and stated because no reviewer would infer it from the diff.

The receive endpoint goes with it. A live HMAC-verified
/webhook/knowledge that nothing can legitimately reach would tell the
next reader that this is how a hive learns about knowledge changes.

Docs move in the same commit: docs/swarm/README.md said two hooks exist
per swarm-wide repo and neither should be deleted, which is now true for
agent-configs and wrong for internal/knowledge — a half-correct
description being worse than an uncorrected one.
This commit is contained in:
atlas 2026-08-19 19:25:30 +02:00 • committed by mara
commit d2a550e685
6 changed files with 111 additions and 218 deletions
Download patch file Download diff file Expand all files Collapse all files

153
hive-c0re/src/workers/knowledge.rs
View file

@ -6,15 +6,18 @@
//! contribute by forking the repo and opening PRs — they never write
//! to the bind-mounted path inside the container.
//!
//! hive-c0re maintains the local clone. A Forgejo webhook notifies it
//! on push to main so agents always see an up-to-date snapshot. The
//! webhook is auto-created by [`ensure_webhook`] at startup. A
//! hive-c0re maintains the local clone. It learns that the repository
//! moved from the **swarm controller**, which owns the one Forgejo
//! webhook and addresses an event to each hive over the queue; a
//! periodic pull in `main.rs` provides a fallback cadence.
use std::collections::BTreeMap;
//!
//! A hive used to register that webhook itself, pointing at its own
//! `/webhook/knowledge`. A webhook has exactly one target URL, so with
//! more than one hive that was last-writer-wins rather than idempotent —
//! every hive but the most recent silently stopped receiving deliveries.
//! [`remove_webhook`] is the migration off it.
use anyhow::{Context, Result};
use forgejo_api::structs::{CreateHookOption, CreateHookOptionConfig, CreateHookOptionType};
use crate::coordinator::Coordinator;
use crate::forge::{core_auth_header, forge_git_url};
@ -142,103 +145,71 @@ async fn seed_readme(core_token: &str) -> Result<()> {
}
}
/// Ensure a Forgejo push webhook for `internal/knowledge` exists and
/// points at hive-c0re's `/webhook/knowledge` endpoint. Idempotent —
/// lists existing hooks first and skips creation when one is already
/// targeting the correct URL.
/// Delete this hive's own `internal/knowledge` push webhook if it is
/// still registered, so the swarm controller is the only party holding
/// one.
///
/// `hive_domain` is the public domain name of the hive; the webhook URL is
/// `https://<hive_domain>/webhook/knowledge` (routed through the gateway,
/// avoiding the Forgejo SSRF guard that blocks loopback delivery).
/// # Why this is a migration and not just a deletion
///
/// `webhook_secret` is the HMAC secret Forgejo will attach as
/// `X-Hub-Signature-256` on each delivery; hive-c0re verifies this header
/// in the dashboard webhook handler (`post_webhook_knowledge`).
/// Not registering any more fixes nothing on a hive that has already
/// run: the hook it created persists on the forge, so the contention
/// this removes would survive on exactly the deployments that have it
/// while fresh installs looked fixed. The hive that created a hook is
/// the one that removes it.
///
/// Called at startup alongside [`ensure_local_clone`]. No-op when the
/// core token is absent (forge not yet provisioned).
pub async fn ensure_webhook(
core_token: &str,
hive_domain: &str,
webhook_secret: &str,
) -> Result<()> {
/// # It removes only its OWN hook, never a neighbour's
///
/// The match is the full URL, not the `/webhook/knowledge` suffix. A
/// hook with that suffix and a different base belongs to *another hive* —
/// one that may not have been upgraded yet — and deleting it would break
/// its knowledge sync until it was. Reaping a neighbour's registration is
/// the very behaviour this issue is about; doing it in the name of fixing
/// it would just invert the direction.
///
/// (The predecessor did reap by suffix, to clear loopback hooks left by
/// an older single-hive layout. That was safe when a hive was alone on
/// its forge and is not safe now.)
///
/// A listing failure is an error rather than a silent skip: there is no
/// create attempt left to fall through to, so swallowing it would leave
/// the hook in place with nothing said. The caller logs and continues —
/// boot does not depend on this.
pub async fn remove_webhook(core_token: &str, hive_domain: &str) -> Result<()> {
// The typed client carries no per-request timeout, so each call is
// wrapped in one: this runs as a detached startup task, and a forge
// that accepts connections but never answers would otherwise hang
// it forever (and the hourly pull fallback masks the missing hook).
// that accepts connections but never answers would otherwise hang it
// forever.
const HTTP_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(10);
let target_url = format!("https://{hive_domain}/webhook/knowledge");
let own_url = format!("https://{hive_domain}/webhook/knowledge");
let client = crate::forge::api(core_token)?;
// List existing hooks — skip creation if ours is already there.
// Best-effort like the raw-HTTP predecessor: a listing failure
// falls through to the create attempt.
let listed = tokio::time::timeout(HTTP_TIMEOUT, client.repo_list_hooks(ORG, REPO).all())
.await
.map_err(anyhow::Error::from)
.and_then(|r| r.map_err(anyhow::Error::from));
match listed {
Ok(hooks) => {
let already_exists = hooks.iter().any(|h| {
h.config
.as_ref()
.and_then(|c| c.get("url"))
.map(String::as_str)
== Some(target_url.as_str())
});
if already_exists {
tracing::debug!(%target_url, "knowledge: push webhook already configured");
return Ok(());
}
// Delete stale hooks that point at our path but a different base
// (e.g. old loopback hooks from before the SSRF-bypass migration).
for h in &hooks {
let hook_url = h
.config
.as_ref()
.and_then(|c| c.get("url"))
.map_or("", String::as_str);
if hook_url.ends_with("/webhook/knowledge")
&& hook_url != target_url
&& let Some(id) = h.id
{
tracing::info!(hook_url, "knowledge: deleting stale webhook (wrong base)");
let _ = tokio::time::timeout(
HTTP_TIMEOUT,
client.repo_delete_hook(ORG, REPO, id).send(),
)
.await;
}
}
}
Err(e) => {
tracing::debug!(error = %e, "knowledge: listing hooks failed; attempting create");
}
}
// Create the webhook.
let mut additional = BTreeMap::new();
additional.insert("secret".to_owned(), webhook_secret.to_owned());
let hook = CreateHookOption {
active: Some(true),
authorization_header: None,
branch_filter: None,
config: CreateHookOptionConfig {
content_type: "json".to_owned(),
url: url::Url::parse(&target_url).context("parse webhook target url")?,
additional,
},
events: Some(vec!["push".to_owned()]),
r#type: CreateHookOptionType::Forgejo,
};
tokio::time::timeout(HTTP_TIMEOUT, client.repo_create_hook(ORG, REPO, hook))
let hooks = tokio::time::timeout(HTTP_TIMEOUT, client.repo_list_hooks(ORG, REPO).all())
.await
.map_err(anyhow::Error::from)
.and_then(|r| r.map_err(anyhow::Error::from))
.with_context(|| format!("create webhook for {ORG}/{REPO}"))?;
tracing::info!(%target_url, "knowledge: push webhook created");
.with_context(|| format!("list webhooks for {ORG}/{REPO}"))?;
for h in &hooks {
let hook_url = h
.config
.as_ref()
.and_then(|c| c.get("url"))
.map_or("", String::as_str);
if hook_url == own_url
&& let Some(id) = h.id
{
tokio::time::timeout(HTTP_TIMEOUT, client.repo_delete_hook(ORG, REPO, id).send())
.await
.map_err(anyhow::Error::from)
.and_then(|r| r.map_err(anyhow::Error::from))
.with_context(|| format!("delete webhook {id} for {ORG}/{REPO}"))?;
tracing::info!(
%own_url,
"knowledge: removed this hive's push webhook — the swarm controller owns it now"
);
}
}
Ok(())
}