Watch
0
0
Fork
You've already forked hyperhive
0

Compare commits

...
Author SHA1 Message Date
atlas
19c9f0c815 docs(#3255): record why the public URL is gated on the swarm UI
mara asked whether SWARM_CONTROLLER_PUBLIC_URL should also be set when the
controller does not run on the same host, and then said the two read as one
service to her. Both are worth answering in the file rather than only in the
thread: that split is not representable today, and the reason is two hops
away from this line.

The UI's /api/ location proxies http://unix:<socketPath> -- a path that
resolves nowhere else -- and this daemon binds no TCP address at all. So the
vhost and the daemon are co-located by construction, and ui.enable is the
flag that declares the vhost rather than a guess about some host serving it.

Also names the tripwire: the day the daemon grows a TCP listener, this line
silently stops being right (env unset, registration quietly skipped, no
error). That is when an explicit publicUrl option becomes correct -- not
before, while there is exactly one derivable answer.
2026-08-18 12:28:09 +02:00
atlas
4573865745 feat(#3255): register the swarm-wide forge hooks against the controller
The endpoint landed inert: nothing pointed at it, so the only way to see
it work was to mint an HMAC by hand. Register the two swarm-wide hooks
at startup so a real forge event produces a journal line.

Registered ALONGSIDE the per-hive hooks, not instead of them. Every hive
keeps receiving and acting on its own deliveries; the controller gets a
copy and logs it. Moving the registration is a later step and has to be:
fan-out swarm->hive does not exist yet, so a hook moved now would point
at a receiver that forwards nowhere, silently on both sides.

Deliberately no stale-hook deletion arm, unlike the two per-hive
registrars this otherwise mirrors: theirs delete hooks matching their own
path with a foreign base, and the hives' hooks are not stale.

The route prefix is what keeps this safe. Both hive-side registrars
delete any hook ending in /webhook/knowledge or /webhook/config-pr with a
different base, so a swarm hook under those paths would be deleted by
every hive on every boot. Serving them under /webhook/forge/ avoids it,
and a test pins it -- there is nothing else that can.

SWARM_CONTROLLER_PUBLIC_URL is set only where the swarm vhost is served,
because a hook whose target_url nothing answers is worse than no hook.
2026-08-18 12:28:09 +02:00
atlas
a8bc084262 feat(#3255): expose the controller's webhook endpoint through the swarm vhost
A forge webhook is a machine POST carrying an HMAC and no session cookie,
so it cannot pass the authelia auth-request subrequest every other location
on this vhost uses. This location deliberately omits it; the HMAC check in
the controller is what guards the path.

Scoped to /webhook/forge/ rather than /webhook/ so a future endpoint under
the same prefix does not inherit the bypass.
2026-08-18 12:28:09 +02:00
atlas
645fd0d56c docs(#3255): record the controller's first persisted file
The crate's module doc claimed no persistence and no writes, which this
change makes false, and docs/persistence.md had no swarm-controller entry
at all — the webhook secret is the daemon's first persisted state.
2026-08-18 12:28:09 +02:00
atlas
b2596097d8 feat(#3255): receive swarm-wide forge webhooks in the controller
A Forgejo webhook has one target_url, so every hive registering the same
swarm-wide hooks is last-writer-wins rather than idempotent. The controller
is the only swarm-wide thing in the deployment, so it becomes the receiver.

It verifies the HMAC and treats the payload as opaque bytes keyed by the hook
kind in the URL path; it deliberately does not parse the payload, because the
hives' existing handlers already decide what a delivery means.

Nothing is registered against the endpoint yet. The replacement path is built
and observable before anything takes the old one away, so the swarm's single
target_url never points at a receiver that forwards nowhere.
2026-08-18 12:28:09 +02:00
10 changed files with 924 additions and 7 deletions

2
Cargo.lock generated
View file

@ -4605,10 +4605,12 @@ dependencies = [
"hive-jobq", "hive-jobq",
"hive-jobq-wire", "hive-jobq-wire",
"hive-types", "hive-types",
"hmac 0.13.0",
"problem_details", "problem_details",
"reqwest 0.13.1", "reqwest 0.13.1",
"serde", "serde",
"serde_json", "serde_json",
"sha2 0.11.0",
"swarm-authelia-bridge-sock", "swarm-authelia-bridge-sock",
"swarm-queue-client", "swarm-queue-client",
"tokio", "tokio",

View file

@ -47,6 +47,10 @@ hive-c0re maintains the local clone at
The endpoint is protected by an auto-generated HMAC secret that The endpoint is protected by an auto-generated HMAC secret that
hive-c0re verifies on every delivery. hive-c0re verifies on every delivery.
On a swarm, a **second** hook on the same repo points at the
swarm controller — see `docs/swarm/README.md` § Swarm-wide forge
webhooks. Both are expected; neither should be deleted.
2. **Periodic pull** — a background task in `hive-c0re::main` 2. **Periodic pull** — a background task in `hive-c0re::main`
pulls on a fixed cadence as a fallback (webhook missed, c0re pulls on a fixed cadence as a fallback (webhook missed, c0re
restarted between pushes). The pull is best-effort — a failure restarted between pushes). The pull is best-effort — a failure

View file

@ -468,6 +468,23 @@ sends to the swarm's snapshot store — see
[`docs/snapshot-store.md`](snapshot-store.md) for what a pushed [`docs/snapshot-store.md`](snapshot-store.md) for what a pushed
snapshot contains and how the store authenticates a sender. snapshot contains and how the store authenticates a sender.
## `/var/lib/swarm-controller/` (swarm-controller host only)
Only present on the one host running
`services.hyperhive.swarm.controller.enable`. systemd `StateDirectory=`,
so it survives restarts and redeploys.
- `webhook-secret` — the HMAC key the swarm's forge webhooks are signed
with. **Keep it.** It is handed to Forgejo when a hook is registered,
so replacing the file means every subsequent delivery fails
verification until the hook is re-registered with the new value. It is
generated automatically on first start; there is nothing to configure.
If the file is unreadable at startup the daemon still starts and logs
`webhook secret unavailable`; the webhook endpoint then answers 503
rather than accepting deliveries it cannot verify. Everything else the
controller serves is unaffected.
## Run-time dirs ## Run-time dirs
`/run/hyperhive/` is tmpfs-backed (systemd `RuntimeDirectory=`) but `/run/hyperhive/` is tmpfs-backed (systemd `RuntimeDirectory=`) but

View file

@ -349,6 +349,30 @@ list, which would look like a silent swarm rather than a controller that
cannot see. The body says which. Status survives a controller restart: cannot see. The body says which. Status survives a controller restart:
it is stored in the queue, not in the daemon. it is stored in the queue, not in the daemon.
### Swarm-wide forge webhooks
At startup the controller registers two Forgejo hooks pointing at
itself — a `push` hook on `internal/knowledge` and a `pull_request` hook
on the `agent-configs` org, both under
`https://<swarm.ui.domain>/webhook/forge/`.
**You will see two hooks where there used to be one**, and that is the
intended state for now: each hive still registers and receives its own,
and the controller's is an additional copy that it currently logs and
nothing more. **Do not delete either.** Removing a hive's hook stops that
hive acting on knowledge pushes and config PRs; removing the
controller's just gets recreated on its next start.
Nothing to configure. The hooks are registered only when this host also
serves the swarm UI vhost — that is what publishes the endpoint, and a
hook the forge cannot reach would collect failed deliveries while
looking healthy. The HMAC secret is generated on first start and kept
(see [`docs/persistence.md`](../persistence.md)).
To check it is working, push to `internal/knowledge` and look for
`webhook: verified delivery` in `journalctl -u swarm-controller`. A
refused delivery logs `webhook: refused delivery` with the reason.
## Cross-references ## Cross-references
- `docs/snapshot-store.md` — the swarm's `btrfs receive` endpoint, and - `docs/snapshot-store.md` — the swarm's `btrfs receive` endpoint, and

View file

@ -33,6 +33,7 @@ let
natsCfg = config.services.hyperhive.swarm.nats; natsCfg = config.services.hyperhive.swarm.nats;
forgeCfg = config.services.hyperhive.swarm.forge; forgeCfg = config.services.hyperhive.swarm.forge;
uiCfg = config.services.hyperhive.swarm.ui;
# The controller's own OAuth2 client. It is NOT a hive: the per-hive # The controller's own OAuth2 client. It is NOT a hive: the per-hive
# clients the roster issues belong to hives, and the responder's client # clients the roster issues belong to hives, and the responder's client
@ -85,6 +86,33 @@ let
SWARM_CONTROLLER_FORGE_TOKEN_FILE = "%d/forge-token"; SWARM_CONTROLLER_FORGE_TOKEN_FILE = "%d/forge-token";
}; };
# How the forge must address this controller to deliver a swarm-wide
# webhook. Gated on the swarm UI being served *here*, because that module
# is what declares the vhost and the `/webhook/forge/` location inside it:
# absent, nothing outside this host can reach the endpoint.
#
# 🔑 The gate is the point, not a detail. The daemon registers hooks only
# when this is set, and a hook whose `target_url` nothing answers is worse
# than no hook at all — forgejo keeps the registration, marks every
# delivery failed, and the hook still reads as configured.
#
# ⚠️ Correct ONLY while the vhost and this daemon share a host, and they do
# by construction: the UI's `/api/` location proxies
# `http://unix:<socketPath>`, a path that resolves nowhere else, and this
# daemon binds no TCP address at all (see the header). `swarm.ui.enable`
# therefore is not a guess about *some* host publishing the endpoint — it is
# the flag that declares that vhost, on the box holding the socket.
#
# ⇒ The day this daemon grows a TCP listener so another host can front it,
# THIS LINE SILENTLY STOPS BEING RIGHT: the env goes unset on the controller
# host, registration quietly does not happen, and nothing errors. That is
# the moment to add an explicit `publicUrl` option — not before, because
# until then there is exactly one derivable answer and an option would only
# be a second place to get it wrong.
webhookEnv = lib.optionalAttrs uiCfg.enable {
SWARM_CONTROLLER_PUBLIC_URL = "https://${uiCfg.domain}";
};
# Not a secret to deliver — `swarm-authelia-bridge`'s own bearer check # Not a secret to deliver — `swarm-authelia-bridge`'s own bearer check
# is satisfied by THIS daemon's existing queue OIDC identity # is satisfied by THIS daemon's existing queue OIDC identity
# (`queueEnv` above): "one identity per principal" already covers this, # (`queueEnv` above): "one identity per principal" already covers this,
@ -594,6 +622,7 @@ in
} }
// queueEnv // queueEnv
// forgeEnv // forgeEnv
// webhookEnv
// authBridgeEnv; // authBridgeEnv;
}; };

View file

@ -188,6 +188,27 @@ in
proxyPass = "http://unix:${controllerCfg.socketPath}:"; proxyPass = "http://unix:${controllerCfg.socketPath}:";
extraConfig = swarmAuthRequest; extraConfig = swarmAuthRequest;
}; };
# ⚠️ THE ONE LOCATION ON THIS VHOST WITH NO `swarmAuthRequest`,
# and that is deliberate rather than an omission.
#
# A forge webhook is a machine POST carrying an HMAC signature
# and no session cookie. An authelia auth-request subrequest
# authenticates a *browser session*; there is nothing here for it
# to check, so guarding this location would not make it safer, it
# would make it permanently unreachable.
#
# What replaces it: swarm-controller verifies the
# `X-Hub-Signature-256` HMAC over the raw body before looking at
# anything else, and answers 401 on any mismatch. That check is
# the access control for this path — see the `webhook` module.
#
# Scoped to `/webhook/forge/` rather than `/webhook/` so the
# carve-out is exactly as wide as the endpoint that justifies it:
# a future `/webhook/<something-else>` does not inherit the
# bypass by living under a shared prefix.
"/webhook/forge/" = {
proxyPass = "http://unix:${controllerCfg.socketPath}:";
};
# Swagger UI: same "nginx hosts the themed dist straight from # Swagger UI: same "nginx hosts the themed dist straight from
# the store, only /api/openapi.json is dynamic" shape as the # the store, only /api/openapi.json is dynamic" shape as the
# per-hive gateway's `swaggerUiLocations`. # per-hive gateway's `swaggerUiLocations`.

View file

@ -38,6 +38,12 @@ problem_details = { version = "0.9.0", features = ["axum"] }
# same shape `hive-c0re/src/job_queue/scheduler.rs` uses over its own graph. # same shape `hive-c0re/src/job_queue/scheduler.rs` uses over its own graph.
hive-jobq.workspace = true hive-jobq.workspace = true
hive-jobq-wire.workspace = true hive-jobq-wire.workspace = true
# The forge webhook HMAC (`webhook.rs`). Kept in this crate rather than
# shared with hive-c0re's equivalent: c0re's copy is scheduled to be deleted
# with its webhook routes once registration moves here, so the second holder
# is departing, not arriving — see that module's docs.
hmac.workspace = true
sha2.workspace = true
# Validates `POST /api/agents`' `name` before it becomes `agent`/`repo` # Validates `POST /api/agents`' `name` before it becomes `agent`/`repo`
# everywhere downstream — see `create_agent`'s doc comment for why this is # everywhere downstream — see `create_agent`'s doc comment for why this is
# defense-in-depth, not the only gate (per an argus review finding). # defense-in-depth, not the only gate (per an argus review finding).

View file

@ -19,10 +19,14 @@ use anyhow::{Context, Result};
use forgejo_api::structs::{ use forgejo_api::structs::{
AddCollaboratorOption, AddCollaboratorOptionPermission, ChangeFileOperation, AddCollaboratorOption, AddCollaboratorOptionPermission, ChangeFileOperation,
ChangeFileOperationOperation, ChangeFilesOptions, CreateBranchProtectionOption, ChangeFileOperationOperation, ChangeFilesOptions, CreateBranchProtectionOption,
CreateRepoOption, RepoGetContentsQuery, CreateHookOption, CreateHookOptionConfig, CreateHookOptionType, CreateRepoOption,
RepoGetContentsQuery,
}; };
use forgejo_api::{ApiErrorKind, Auth, Forgejo, ForgejoError}; use forgejo_api::{ApiErrorKind, Auth, Forgejo, ForgejoError};
use reqwest::StatusCode; use reqwest::StatusCode;
use std::collections::BTreeMap;
use crate::webhook::DeliveryKind;
/// The forge org that owns agent repos. Same org `hive-c0re::forge` /// The forge org that owns agent repos. Same org `hive-c0re::forge`
/// already uses for its own single-hive `CreateRepo` path — this is /// already uses for its own single-hive `CreateRepo` path — this is
@ -38,6 +42,20 @@ pub const AGENTS_ORG: &str = "agents";
/// [`Client::create_repo`]'s doc comment. /// [`Client::create_repo`]'s doc comment.
const OPERATORS_TEAM: &str = "operators"; const OPERATORS_TEAM: &str = "operators";
/// The org owning per-agent config repos, and where the `pull_request` hook
/// lives. Same value as `hive-c0re::forge::CONFIG_ORG` — one forge, one org.
///
/// ⚠️ Duplicated across the crate boundary (this crate deliberately does not
/// depend on `hive-c0re`), so nothing makes the two fail together. The
/// failure mode if they drift is quiet: the hook is created on an org that
/// exists, forgejo reports it healthy, and it simply never fires.
const CONFIG_ORG: &str = "agent-configs";
/// The hive-wide knowledge repo, where the `push` hook lives. Same values as
/// `hive-c0re::workers::knowledge::{ORG, REPO}`.
const KNOWLEDGE_ORG: &str = "internal";
const KNOWLEDGE_REPO: &str = "knowledge";
/// The env vars `swarm-controller.nix`'s `forgeEnv` sets. Named here /// The env vars `swarm-controller.nix`'s `forgeEnv` sets. Named here
/// once so [`Client::from_env`] and the nix module can't drift silently /// once so [`Client::from_env`] and the nix module can't drift silently
/// — a rename on one side without the other fails loudly (env var /// — a rename on one side without the other fails loudly (env var
@ -325,6 +343,136 @@ impl Client {
Err(e) => Err(e).with_context(|| format!("check for {path} in {AGENTS_ORG}/{repo}")), Err(e) => Err(e).with_context(|| format!("check for {path} in {AGENTS_ORG}/{repo}")),
} }
} }
/// Register the swarm-wide hooks against this controller, so a real
/// forge event reaches [`crate::webhook`] instead of the endpoint only
/// being reachable by hand.
///
/// **These are registered ALONGSIDE the per-hive hooks, not instead of
/// them.** Every hive keeps receiving and acting on its own deliveries
/// exactly as today; the controller receives a copy and (for now) logs
/// it. Taking the hive-side registration away is a later step, and it
/// has to be later: fan-out swarm→hive does not exist yet, so a hook
/// moved now would point at a receiver that forwards nowhere — silent on
/// both sides, indistinguishable from no activity.
///
/// ⛔ **No stale-hook deletion arm, unlike the two per-hive registrars
/// this otherwise mirrors.** Their arm deletes hooks matching their own
/// path with a foreign base; copying it here would delete the hives'
/// live hooks, which are not stale — they are the path still in
/// production. The controller only ever adds its own.
///
/// Idempotent: an existing hook with the same `target_url` is left
/// alone, so this is safe on every boot.
///
/// # Errors
///
/// Returns the first failure. A listing failure is not fatal — it falls
/// through to the create attempt, which is idempotent server-side by way
/// of the already-exists fold, the same best-effort shape `hive-c0re`
/// uses.
pub async fn ensure_swarm_webhooks(&self, public_base: &str, secret: &str) -> Result<()> {
for kind in DeliveryKind::ALL {
let target_url = kind.target_url(public_base);
let (event, scope) = match kind {
DeliveryKind::Knowledge => (
"push",
HookScope::Repo {
org: KNOWLEDGE_ORG,
repo: KNOWLEDGE_REPO,
},
),
DeliveryKind::ConfigPr => ("pull_request", HookScope::Org { org: CONFIG_ORG }),
};
self.ensure_hook(&scope, &target_url, event, secret)
.await
.with_context(|| format!("register swarm webhook {target_url}"))?;
}
Ok(())
}
/// One hook, idempotently. Split out of [`Self::ensure_swarm_webhooks`]
/// so the org-scoped and repo-scoped calls do not each grow their own
/// copy of the list-then-create logic.
async fn ensure_hook(
&self,
scope: &HookScope<'_>,
target_url: &str,
event: &str,
secret: &str,
) -> Result<()> {
match scope.list_hook_urls(&self.api).await {
Ok(urls) => {
if urls.iter().any(|u| u == target_url) {
tracing::debug!(%target_url, "swarm forge: webhook already registered");
return Ok(());
}
}
Err(e) => {
// Best-effort, same as the per-hive registrars: a forge that
// cannot be listed may still accept a create, and a
// duplicate create is folded into success below.
tracing::debug!(error = %e, %target_url, "swarm forge: listing hooks failed; attempting create");
}
}
let mut additional = BTreeMap::new();
additional.insert("secret".to_owned(), secret.to_owned());
let hook = CreateHookOption {
active: Some(true),
authorization_header: None,
branch_filter: None,
config: CreateHookOptionConfig {
content_type: "json".to_owned(),
url: url::Url::parse(target_url)
.with_context(|| format!("parse webhook target url {target_url}"))?,
additional,
},
events: Some(vec![event.to_owned()]),
r#type: CreateHookOptionType::Forgejo,
};
match scope.create_hook(&self.api, hook).await {
Ok(()) => {
tracing::info!(%target_url, %event, "swarm forge: webhook registered");
Ok(())
}
Err(e) if is_already_exists(&e) => {
tracing::debug!(%target_url, "swarm forge: webhook already present");
Ok(())
}
Err(e) => Err(e.into()),
}
}
}
/// Where a hook lives. The knowledge hook is repo-scoped and the config-PR
/// hook is org-scoped, mirroring exactly where the per-hive registrars put
/// theirs — a hook on the wrong scope would never fire, and forgejo would
/// report that as a perfectly healthy hook with no deliveries.
enum HookScope<'a> {
Repo { org: &'a str, repo: &'a str },
Org { org: &'a str },
}
impl HookScope<'_> {
/// The `url` config value of every hook currently on this scope.
async fn list_hook_urls(&self, api: &Forgejo) -> Result<Vec<String>, ForgejoError> {
let hooks = match self {
Self::Repo { org, repo } => api.repo_list_hooks(org, repo).all().await?,
Self::Org { org } => api.org_list_hooks(org).send().await?,
};
Ok(hooks
.iter()
.filter_map(|h| h.config.as_ref()?.get("url").cloned())
.collect())
}
async fn create_hook(&self, api: &Forgejo, hook: CreateHookOption) -> Result<(), ForgejoError> {
match self {
Self::Repo { org, repo } => api.repo_create_hook(org, repo, hook).await.map(drop),
Self::Org { org } => api.org_create_hook(org, hook).await.map(drop),
}
}
} }
/// Base64-encode `content` for a forgejo content-API call — every file /// Base64-encode `content` for a forgejo content-API call — every file

View file

@ -3,11 +3,14 @@
//! `services.hyperhive.swarm.controller.enable` on, and serves HTTP over a //! `services.hyperhive.swarm.controller.enable` on, and serves HTTP over a
//! unix socket that the hive-gateway's nginx proxies to. //! unix socket that the hive-gateway's nginx proxies to.
//! //!
//! Holds one piece of read-only state: the swarm's hive directory, loaded //! Configuration is read-only and loaded once at startup from env vars the
//! once at startup from an env var the NixOS module sets //! NixOS module sets (`services.hyperhive.swarm.controller`) — see
//! (`services.hyperhive.swarm.controller`) — see `load_hives`. Still no //! `load_hives`. A config change means a redeploy, same as every other
//! persistence and no writes; a config change means a redeploy, same as //! option this process reads.
//! every other option this process reads. //!
//! The one thing it does persist is `webhook-secret` under its
//! `StateDirectory` (see `webhook`), because that key is handed to Forgejo
//! at registration and so cannot be regenerated per boot.
//! //!
//! Distinct from `hive-c0re`, which is per-hive: c0re owns the agents on //! Distinct from `hive-c0re`, which is per-hive: c0re owns the agents on
//! one host, this owns what is true across hives. //! one host, this owns what is true across hives.
@ -35,6 +38,7 @@ use utoipa_axum::{router::OpenApiRouter, routes};
mod auth; mod auth;
mod forge; mod forge;
mod status; mod status;
mod webhook;
/// Node payload for the swarm-level job graph. Named `Swarm*` rather than /// Node payload for the swarm-level job graph. Named `Swarm*` rather than
/// the bare `NodeKind`/`Resource` `hive-c0re::job_queue::model` already /// the bare `NodeKind`/`Resource` `hive-c0re::job_queue::model` already
@ -273,6 +277,7 @@ fn socket_path() -> PathBuf {
(name = "links", description = "swarm service quick links"), (name = "links", description = "swarm service quick links"),
(name = "jobq", description = "the swarm-level job graph"), (name = "jobq", description = "the swarm-level job graph"),
(name = "agents", description = "creating agent identities at swarm level"), (name = "agents", description = "creating agent identities at swarm level"),
(name = "webhook", description = "swarm-wide forge webhook receipt"),
) )
)] )]
struct ApiDoc; struct ApiDoc;
@ -329,6 +334,17 @@ struct AppState {
/// claimed — same "queue now, fail per-job" shape as an unreachable /// claimed — same "queue now, fail per-job" shape as an unreachable
/// swarm queue. /// swarm queue.
jobq: Arc<Mutex<hive_jobq::scheduler::Scheduler<SwarmNodeKind, SwarmResourceKind>>>, jobq: Arc<Mutex<hive_jobq::scheduler::Scheduler<SwarmNodeKind, SwarmResourceKind>>>,
/// HMAC secret for swarm-wide forge webhooks, loaded once at startup.
/// `None` when it could not be read or created — the webhook endpoint
/// then refuses every delivery with 503 rather than admitting one it
/// cannot verify. Deliberately not fatal to startup: nothing is
/// registered against that endpoint yet, and the rest of this daemon's
/// surface is unaffected. See [`webhook::load_or_generate_secret`].
///
/// `Arc<str>`, not `Arc<String>`: the value is never mutated after
/// startup, and this way `as_deref()` yields the `&str` the verifier
/// takes without a second hop through `String`.
webhook_secret: Option<Arc<str>>,
} }
/// Env var the controller's NixOS module sets from /// Env var the controller's NixOS module sets from
@ -645,6 +661,47 @@ async fn get_jobq_rollup(State(state): State<AppState>) -> Json<Vec<hive_jobq_wi
Json(hive_jobq_wire::state_rollup(graph, roots)) Json(hive_jobq_wire::state_rollup(graph, roots))
} }
/// The swarm's own public base URL, as the forge must address it.
///
/// Set by `swarm-controller.nix` **only when this host actually serves the
/// swarm vhost** that carries the `/webhook/forge/` location. Absent means
/// "the endpoint is not reachable from outside", and the right response to
/// that is to register nothing: a hook pointing at a URL nothing answers is
/// worse than no hook, because forgejo records failed deliveries against a
/// registration that looks configured.
const PUBLIC_URL_ENV: &str = "SWARM_CONTROLLER_PUBLIC_URL";
/// Register the swarm-wide forge hooks against this controller, in the
/// background.
///
/// Detached rather than awaited, and never fatal: the forge may be slow or
/// briefly down at boot, and none of the daemon's other routes depend on a
/// hook existing. Registration is idempotent, so the next restart retries.
///
/// Silently does nothing when any of the three preconditions is missing —
/// each is a legitimate deployment shape (no forge here, no state directory
/// to hold a secret, no public vhost), and each is already logged where it
/// is discovered.
fn register_swarm_webhooks(forge: Option<Arc<forge::Client>>, secret: Option<Arc<str>>) {
let Some(forge) = forge else { return };
let Some(secret) = secret else { return };
let Ok(public_url) = std::env::var(PUBLIC_URL_ENV) else {
tracing::info!(
"{PUBLIC_URL_ENV} unset; not registering swarm-wide forge webhooks (the \
endpoint is not published on this host)"
);
return;
};
tokio::spawn(async move {
if let Err(e) = forge.ensure_swarm_webhooks(&public_url, &secret).await {
tracing::warn!(
error = %format!("{e:#}"),
"registering swarm-wide forge webhooks failed; retrying on next start"
);
}
});
}
#[tokio::main] #[tokio::main]
async fn main() -> Result<()> { async fn main() -> Result<()> {
tracing_subscriber::fmt() tracing_subscriber::fmt()
@ -747,7 +804,7 @@ async fn main() -> Result<()> {
}; };
let deps = WorkerDeps { let deps = WorkerDeps {
auth, auth,
forge: forge_client, forge: forge_client.clone(),
}; };
let jobq = Arc::new(Mutex::new(hive_jobq::scheduler::Scheduler::new( let jobq = Arc::new(Mutex::new(hive_jobq::scheduler::Scheduler::new(
@ -756,11 +813,29 @@ async fn main() -> Result<()> {
))); )));
spawn_jobq_worker(Arc::clone(&jobq), deps); spawn_jobq_worker(Arc::clone(&jobq), deps);
// Same "log and carry on" shape as the queue/bridge/forge wiring above.
// A controller that cannot hold a webhook secret still serves every
// other route; the webhook endpoint answers 503, which is the honest
// answer rather than a silent accept.
let webhook_secret = match webhook::load_or_generate_secret() {
Ok(secret) => Some(Arc::from(secret)),
Err(e) => {
tracing::warn!(
error = %format!("{e:#}"),
"webhook secret unavailable; swarm-wide forge webhooks are off"
);
None
}
};
register_swarm_webhooks(forge_client, webhook_secret.clone());
let state = AppState { let state = AppState {
hives: Arc::new(load_hives()), hives: Arc::new(load_hives()),
links: Arc::new(load_links()), links: Arc::new(load_links()),
status, status,
jobq, jobq,
webhook_secret,
}; };
let (router, api) = OpenApiRouter::<AppState>::with_openapi(ApiDoc::openapi()) let (router, api) = OpenApiRouter::<AppState>::with_openapi(ApiDoc::openapi())
@ -771,6 +846,7 @@ async fn main() -> Result<()> {
.routes(routes!(get_jobq_graph)) .routes(routes!(get_jobq_graph))
.routes(routes!(get_jobq_rollup)) .routes(routes!(get_jobq_rollup))
.routes(routes!(create_agent)) .routes(routes!(create_agent))
.routes(routes!(webhook::post_webhook_forge))
.split_for_parts(); .split_for_parts();
// Just the JSON, not the UI — Swagger UI itself is nginx-hosted from // Just the JSON, not the UI — Swagger UI itself is nginx-hosted from
// the nix store (see the module doc comment above). `api` is // the nix store (see the module doc comment above). `api` is

View file

@ -0,0 +1,590 @@
//! Forgejo webhook receipt at the swarm level.
//!
//! A Forgejo webhook has exactly **one** `target_url`, so every hive
//! registering the same swarm-wide hooks is last-writer-wins rather than
//! idempotent, and every hive but the most recent silently stops receiving
//! deliveries. One owner is the only non-racing shape, and the only
//! swarm-wide thing in the deployment is this daemon.
//!
//! **This relays; it does not interpret.** The payload stays opaque bytes,
//! keyed by a [`DeliveryKind`] from the URL path. The hives already know
//! what a delivery *means* — which repo is the knowledge repo, which config
//! PR action queues an approval — so parsing it here would be a second place
//! deciding that, and the two would drift.
//!
//! **The HMAC code is not shared with `hive-c0re` because c0re's copy is
//! leaving, not staying.** Once registration moves here, hives stop
//! registering *and* receiving, and c0re's routes and secret go with it. A
//! shared crate is right when a second consumer *arrives*; here it is
//! departing. The two verifiers meanwhile check different hooks with
//! different secrets and never need to agree.
//!
//! **These hooks are registered ALONGSIDE the per-hive ones** — see
//! [`crate::forge::Client::ensure_swarm_webhooks`], which carries why, and
//! why moving them cannot come before fan-out.
use anyhow::{Context as _, Result};
use axum::{
body::Bytes,
extract::{Path, State},
http::{HeaderMap, StatusCode},
response::{IntoResponse as _, Response},
};
use hmac::{Hmac, KeyInit as _, Mac as _};
use sha2::Sha256;
use super::AppState;
/// Where the HMAC secret is kept, relative to the daemon's state directory.
///
/// The secret is generated on first start and handed to Forgejo when the
/// hook is registered, so it must survive a restart — a rotated secret
/// would make every subsequent delivery fail verification with a correctly
/// configured hook on both sides.
const SECRET_FILE: &str = "webhook-secret";
/// Fallback state directory, used only when `STATE_DIRECTORY` is unset (a
/// dev run outside systemd). Under the unit it is always set, because
/// `StateDirectory = "swarm-controller"` is declared there.
const DEFAULT_STATE_DIR: &str = "/var/lib/swarm-controller";
/// Path to the HMAC secret file.
///
/// Reads systemd's `STATE_DIRECTORY` rather than introducing a
/// `SWARM_CONTROLLER_*` env var for it: the unit already declares
/// `StateDirectory=`, and a second env var would be a second declaration of
/// the same fact, free to drift from the first.
///
/// ⚠️ `StateDirectory=` accepts a *list*, in which case systemd exports the
/// paths colon-separated. Only the first is taken — a bare `PathBuf::from`
/// of the whole variable would silently produce a path containing a colon
/// the day someone adds a second directory to the unit.
fn secret_path() -> std::path::PathBuf {
secret_path_from(std::env::var("STATE_DIRECTORY").ok().as_deref())
}
/// The pure half of [`secret_path`], split out so the colon-list handling is
/// testable **without touching the environment**.
///
/// Not a stylistic split: `std::env::set_var` mutates process-global state,
/// and cargo runs a crate's tests on parallel threads in one process, so two
/// tests setting it race — which is not hypothetical here, it is how the
/// first version of this module's tests failed.
fn secret_path_from(raw: Option<&str>) -> std::path::PathBuf {
let dir = raw
.and_then(|raw| raw.split(':').next())
.filter(|first| !first.is_empty())
.unwrap_or(DEFAULT_STATE_DIR);
std::path::PathBuf::from(dir).join(SECRET_FILE)
}
/// Load the swarm's webhook HMAC secret, generating and persisting it if the
/// file is absent or malformed.
///
/// Returns a hex-encoded 32-byte secret (64 hex chars).
///
/// # Errors
///
/// Returns an error when the secret cannot be generated (no `/dev/urandom`)
/// or cannot be persisted. A malformed existing file is **not** an error: it
/// is regenerated, because a secret that cannot be parsed is
/// indistinguishable from one that was never written, and failing startup
/// over it would strand the daemon with no path forward.
pub(super) fn load_or_generate_secret() -> Result<String> {
load_or_generate_at(&secret_path())
}
/// The path-taking half of [`load_or_generate_secret`], split out for the
/// same reason as [`secret_path_from`]: a test can point it at a scratch
/// file without mutating a process-global env var.
fn load_or_generate_at(path: &std::path::Path) -> Result<String> {
if let Ok(raw) = std::fs::read_to_string(path) {
let trimmed = raw.trim().to_owned();
if trimmed.len() == 64 && trimmed.chars().all(|c| c.is_ascii_hexdigit()) {
return Ok(trimmed);
}
tracing::warn!(
path = %path.display(),
"webhook-secret file malformed (wrong length/chars); regenerating"
);
}
let secret = generate_hex_secret()?;
std::fs::create_dir_all(path.parent().unwrap_or(path))
.with_context(|| format!("create dir for {}", path.display()))?;
std::fs::write(path, format!("{secret}\n"))
.with_context(|| format!("write webhook secret to {}", path.display()))?;
tracing::info!(path = %path.display(), "webhook secret generated and persisted");
Ok(secret)
}
/// Read 32 random bytes from `/dev/urandom` and hex-encode them.
fn generate_hex_secret() -> Result<String> {
use std::io::Read as _;
let mut buf = [0u8; 32];
let mut f =
std::fs::File::open("/dev/urandom").context("open /dev/urandom for secret generation")?;
f.read_exact(&mut buf)
.context("read 32 bytes from /dev/urandom")?;
Ok(hex_encode(&buf))
}
/// Hex-encode `bytes` as a lowercase string.
fn hex_encode(bytes: &[u8]) -> String {
let mut out = String::with_capacity(bytes.len() * 2);
for b in bytes {
out.push(char::from_digit(u32::from(b >> 4), 16).unwrap_or('0'));
out.push(char::from_digit(u32::from(b & 0xf), 16).unwrap_or('0'));
}
out
}
/// Decode a lowercase hex string into bytes; returns `None` on invalid input.
fn hex_decode(s: &str) -> Option<Vec<u8>> {
if !s.len().is_multiple_of(2) {
return None;
}
let mut out = Vec::with_capacity(s.len() / 2);
let mut chars = s.chars();
while let (Some(hi), Some(lo)) = (chars.next(), chars.next()) {
let hi = u8::try_from(hi.to_digit(16)?).ok()?;
let lo = u8::try_from(lo.to_digit(16)?).ok()?;
out.push((hi << 4) | lo);
}
Some(out)
}
/// Which swarm-wide hook a delivery arrived on.
///
/// The kind comes from the URL path rather than from Forgejo's
/// `X-Forgejo-Event` header, so the routing key is decided by *the
/// registration we made* instead of by a field the sender chooses. Each
/// registered hook gets its own `target_url`, exactly as the per-hive hooks
/// do today.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub(super) enum DeliveryKind {
/// Push events on the hive-wide knowledge repo.
Knowledge,
/// `pull_request` events on the agent-config repos.
ConfigPr,
}
/// The route prefix a registered `target_url` must point at.
///
/// ⚠️ Deliberately **not** `/webhook/knowledge` or `/webhook/config-pr`, the
/// paths the per-hive receivers use. Both hive-side registrars delete any
/// hook whose URL ends with *their* path but has a different base — see
/// `hive-c0re`'s `forge::ensure_config_pr_webhook` and
/// `workers::knowledge::ensure_webhook`. A swarm-level hook under those
/// paths would therefore be deleted by every hive on every boot, and the
/// symptom is a hook that silently stops existing. `webhook_urls_survive_the_hive_side_reapers`
/// pins that.
const ROUTE_PREFIX: &str = "/webhook/forge/";
impl DeliveryKind {
/// Every kind, for the registration sweep. An array rather than a
/// hand-written list at the call site, so adding a kind cannot leave one
/// hook unregistered.
pub(super) const ALL: [Self; 2] = [Self::Knowledge, Self::ConfigPr];
/// The `target_url` to register with Forgejo for this kind, given the
/// swarm's public base URL.
///
/// Built here rather than at the registration call site so the URL that
/// is *registered* and the route that *serves* it are the same fact in
/// one place. A trailing slash on the base is tolerated — it arrives from
/// config, and `https://swarm//webhook/...` would be a hook that 404s.
pub(super) fn target_url(self, public_base: &str) -> String {
format!(
"{}{ROUTE_PREFIX}{}",
public_base.trim_end_matches('/'),
self.as_str()
)
}
/// Parse the `{kind}` path segment. Unknown values are rejected rather
/// than accepted-and-ignored: a typo in a registered `target_url` must
/// be *observable*, and a 200 for an unrecognised path is exactly the
/// silence this issue exists to remove.
fn parse(segment: &str) -> Option<Self> {
match segment {
"knowledge" => Some(Self::Knowledge),
"config-pr" => Some(Self::ConfigPr),
_ => None,
}
}
/// Stable string form, used for logging and (once fan-out lands) as the
/// event's routing key. Deliberately the same spelling as the path
/// segment so a journal line can be matched against a registered URL.
fn as_str(self) -> &'static str {
match self {
Self::Knowledge => "knowledge",
Self::ConfigPr => "config-pr",
}
}
}
/// Why a delivery was refused.
///
/// A value rather than a string, so the status code is chosen by matching on
/// the reason instead of on message text — `hive-c0re`'s equivalent does
/// `e.contains("unavailable")`, which couples an HTTP response to the exact
/// wording of a log message.
#[derive(Debug, PartialEq, Eq)]
pub(super) enum Refusal {
/// No secret was loaded at startup: the endpoint cannot verify anything,
/// which is this daemon's fault and not the caller's. 503, so a retry is
/// meaningful.
SecretUnavailable,
/// The signature was missing, malformed, or did not match. 401.
BadSignature(String),
}
impl Refusal {
fn status(&self) -> StatusCode {
match self {
Self::SecretUnavailable => StatusCode::SERVICE_UNAVAILABLE,
Self::BadSignature(_) => StatusCode::UNAUTHORIZED,
}
}
fn message(&self) -> String {
match self {
Self::SecretUnavailable => {
"webhook HMAC secret unavailable; endpoint disabled".to_owned()
}
Self::BadSignature(detail) => detail.clone(),
}
}
}
/// Verify the `X-Hub-Signature-256` header Forgejo attaches to a delivery
/// (`sha256=<hex>`).
///
/// The body is the raw wire bytes, not a re-serialised structure: an HMAC is
/// over exactly what was sent, and any parse-then-reserialise step would
/// change whitespace or key order and fail every signature.
///
/// The refusal messages are safe to log — they never contain the secret or
/// the expected digest, so a caller can surface *why* a delivery was refused
/// without handing an attacker the answer.
pub(super) fn verify(
secret: Option<&str>,
headers: &HeaderMap,
body: &Bytes,
) -> Result<(), Refusal> {
let secret = secret.ok_or(Refusal::SecretUnavailable)?;
let sig = headers
.get("x-hub-signature-256")
.and_then(|v| v.to_str().ok())
.unwrap_or("");
if sig.is_empty() {
return Err(Refusal::BadSignature(
"missing X-Hub-Signature-256 header".to_owned(),
));
}
let Some(sig_hex) = sig.strip_prefix("sha256=") else {
return Err(Refusal::BadSignature(
"X-Hub-Signature-256 missing 'sha256=' prefix".to_owned(),
));
};
let Some(expected) = hex_decode(sig_hex) else {
return Err(Refusal::BadSignature(
"X-Hub-Signature-256 contains non-hex chars".to_owned(),
));
};
let mut mac = Hmac::<Sha256>::new_from_slice(secret.as_bytes())
.map_err(|e| Refusal::BadSignature(format!("HMAC key error: {e}")))?;
mac.update(body);
// `verify_slice` is constant-time and also rejects a wrong-length digest,
// which is why the comparison is not written by hand here.
mac.verify_slice(&expected)
.map_err(|_| Refusal::BadSignature("X-Hub-Signature-256 mismatch".to_owned()))
}
/// POST `/webhook/forge/{kind}` — a swarm-wide Forgejo delivery.
///
/// Verifies the HMAC over the raw body, then (once fan-out lands) relays the
/// delivery to every hive over the swarm queue. The payload is never parsed
/// here — see the module docs.
///
/// Returns 200 on an accepted delivery so Forgejo does not retry. A refused
/// one answers 401 (bad signature) or 503 (this daemon has no secret), and
/// an unknown `{kind}` answers 404 rather than a cheerful 200.
#[utoipa::path(
post,
path = "/webhook/forge/{kind}",
params(
("kind" = String, Path, description = "which swarm-wide hook this delivery arrived on: \
`knowledge` or `config-pr`")
),
request_body(
content = String,
content_type = "application/json",
description = "Forgejo webhook payload, taken as raw bytes so HMAC \
verification runs over the exact wire bytes and the \
delivery can be relayed unmodified"
),
responses(
(status = 200, description = "delivery accepted", body = String),
(status = 401, description = "bad or missing HMAC signature"),
(status = 404, description = "unknown hook kind in the path"),
(status = 503, description = "HMAC secret unavailable at startup"),
),
tag = "webhook"
)]
pub(super) async fn post_webhook_forge(
State(state): State<AppState>,
Path(kind): Path<String>,
headers: HeaderMap,
body: Bytes,
) -> Response {
// Order matters: verify BEFORE looking at the path. An unauthenticated
// caller learning which `{kind}` values exist, from the difference
// between 404 and 401, is a small leak — but it is free to close.
if let Err(refusal) = verify(state.webhook_secret.as_deref(), &headers, &body) {
let message = refusal.message();
tracing::warn!(%kind, %message, "webhook: refused delivery");
return (refusal.status(), message).into_response();
}
let Some(kind) = DeliveryKind::parse(&kind) else {
tracing::warn!(%kind, "webhook: delivery on an unknown hook kind");
return (StatusCode::NOT_FOUND, "unknown hook kind").into_response();
};
tracing::info!(
kind = kind.as_str(),
bytes = body.len(),
"webhook: verified delivery"
);
(StatusCode::OK, "ok").into_response()
}
#[cfg(test)]
mod tests {
use super::{DeliveryKind, Refusal, load_or_generate_at, secret_path_from, verify};
use axum::{body::Bytes, http::HeaderMap, http::StatusCode};
use hmac::{Hmac, KeyInit as _, Mac as _};
use sha2::Sha256;
use std::fmt::Write as _;
/// Mint the header Forgejo would send. Written out rather than reusing
/// the verifier's internals so a test cannot pass by sharing a bug with
/// the code under test.
fn sign(secret: &str, body: &[u8]) -> String {
let mut mac = Hmac::<Sha256>::new_from_slice(secret.as_bytes()).expect("key");
mac.update(body);
let mut hex = String::new();
for byte in mac.finalize().into_bytes() {
write!(hex, "{byte:02x}").expect("writing to a String cannot fail");
}
format!("sha256={hex}")
}
fn headers_with(sig: &str) -> HeaderMap {
let mut h = HeaderMap::new();
h.insert("x-hub-signature-256", sig.parse().expect("header value"));
h
}
/// Each refusal reached separately, so a change that collapses them into
/// one always-refuse path is visible — and the accept case alongside
/// them, so a verifier that refuses *everything* cannot pass either.
#[test]
fn a_delivery_is_admitted_only_with_a_matching_signature() {
let body = Bytes::from_static(b"{\"action\":\"opened\"}");
let good = sign("s3cret", &body);
assert_eq!(
verify(Some("s3cret"), &headers_with(&good), &body),
Ok(()),
"a correctly signed delivery must be admitted"
);
assert!(
matches!(
verify(Some("other"), &headers_with(&good), &body),
Err(Refusal::BadSignature(_))
),
"a signature minted with a different secret must be refused"
);
assert!(
matches!(
verify(
Some("s3cret"),
&headers_with(&good),
&Bytes::from_static(b"tampered")
),
Err(Refusal::BadSignature(_))
),
"a body that does not match the digest must be refused"
);
assert!(
matches!(
verify(
Some("s3cret"),
&headers_with(good.trim_start_matches("sha256=")),
&body
),
Err(Refusal::BadSignature(_))
),
"a bare hex digest with no 'sha256=' prefix is not what Forgejo sends"
);
assert!(
matches!(
verify(Some("s3cret"), &HeaderMap::new(), &body),
Err(Refusal::BadSignature(_))
),
"a delivery with no signature header at all must be refused"
);
assert_eq!(
verify(None, &headers_with(&good), &body),
Err(Refusal::SecretUnavailable),
"with no secret loaded the endpoint must refuse, not admit"
);
}
/// The status codes, asserted on the value rather than on message text.
#[test]
fn a_missing_secret_is_this_daemons_fault_and_a_bad_signature_is_the_callers() {
assert_eq!(
Refusal::SecretUnavailable.status(),
StatusCode::SERVICE_UNAVAILABLE,
"no secret is a server-side condition: a retry is meaningful"
);
assert_eq!(
Refusal::BadSignature(String::new()).status(),
StatusCode::UNAUTHORIZED,
"a bad signature is the caller's, and a retry will not help"
);
}
/// An unknown segment must not parse. The whole point of the 404 is that
/// a mistyped `target_url` is observable rather than silently accepted.
#[test]
fn only_the_registered_hook_kinds_parse() {
assert_eq!(
DeliveryKind::parse("knowledge"),
Some(DeliveryKind::Knowledge)
);
assert_eq!(
DeliveryKind::parse("config-pr"),
Some(DeliveryKind::ConfigPr)
);
assert_eq!(
DeliveryKind::parse("config_pr"),
None,
"underscore is not the registered spelling; accepting both would make a journal line ambiguous"
);
assert_eq!(DeliveryKind::parse(""), None);
assert_eq!(DeliveryKind::parse("../knowledge"), None);
}
/// A malformed stored secret is replaced rather than fatal — and the
/// replacement is a *usable* secret that survives a reload, because a
/// version rotating on every call would pass a naive length check while
/// silently breaking every registration made with the previous value.
///
#[test]
fn a_malformed_secret_file_is_regenerated_and_then_stable() {
let dir = std::env::temp_dir().join(format!("swarm-hook-{}", std::process::id()));
std::fs::create_dir_all(&dir).expect("mkdir");
let path = dir.join("webhook-secret");
std::fs::write(&path, "not-a-hex-secret\n").expect("seed");
let secret = load_or_generate_at(&path).expect("regenerates");
let again = load_or_generate_at(&path).expect("second load");
std::fs::remove_dir_all(&dir).ok();
assert_eq!(secret.len(), 64, "hex-encoded 32 bytes");
assert!(secret.chars().all(|c| c.is_ascii_hexdigit()));
assert_eq!(
again, secret,
"a valid secret must be read back, not rotated"
);
}
/// `StateDirectory=` may name several directories, in which case systemd
/// exports them colon-separated — take the first, never the raw value.
///
/// Asserted against the pure half, so this never touches the process
/// environment. The first version of these tests *did* use
/// `std::env::set_var`, and it failed: cargo runs a crate's tests on
/// parallel threads within **one** process, so the sibling env-setting
/// test removed the variable mid-run and the other silently fell back to
/// `/var/lib/swarm-controller`.
#[test]
fn the_secret_path_survives_a_multi_valued_state_directory() {
assert_eq!(
secret_path_from(Some("/var/lib/a:/var/lib/b")),
std::path::Path::new("/var/lib/a/webhook-secret")
);
assert_eq!(
secret_path_from(Some("/var/lib/only")),
std::path::Path::new("/var/lib/only/webhook-secret")
);
assert_eq!(
secret_path_from(None),
std::path::Path::new("/var/lib/swarm-controller/webhook-secret"),
"unset falls back to the directory the unit declares"
);
assert_eq!(
secret_path_from(Some("")),
std::path::Path::new("/var/lib/swarm-controller/webhook-secret"),
"an empty value must not resolve to a relative path"
);
}
/// The URL that gets registered must match the route that serves it, and
/// a trailing slash on the configured base must not produce `//webhook`.
#[test]
fn a_target_url_is_the_route_this_module_serves() {
assert_eq!(
DeliveryKind::Knowledge.target_url("https://swarm.example"),
"https://swarm.example/webhook/forge/knowledge"
);
assert_eq!(
DeliveryKind::ConfigPr.target_url("https://swarm.example/"),
"https://swarm.example/webhook/forge/config-pr",
"a trailing slash on the base must not double up"
);
for kind in DeliveryKind::ALL {
let url = kind.target_url("https://swarm.example");
let segment = url.rsplit('/').next().expect("a last segment");
assert_eq!(
DeliveryKind::parse(segment),
Some(kind),
"the last path segment of a registered URL must parse back \
to the kind that built it"
);
}
}
/// A cross-daemon invariant with nothing else to enforce it: both
/// per-hive registrars in `hive-c0re` **delete** hooks whose URL ends
/// with their own path but carries a different base — see
/// `forge::ensure_config_pr_webhook` and `knowledge::ensure_webhook`.
///
/// While the swarm-level hooks live alongside the per-hive ones, a
/// controller URL matching either suffix would be deleted by every hive
/// on every boot: the swarm hook would simply cease to exist, with the
/// cause in a different daemon's startup sweep. Serving these under
/// `/webhook/forge/` is what avoids it, and this is the only place that
/// says so in a form that fails.
#[test]
fn webhook_urls_survive_the_hive_side_reapers() {
for suffix in ["/webhook/knowledge", "/webhook/config-pr"] {
for kind in DeliveryKind::ALL {
let url = kind.target_url("https://swarm.example");
assert!(
!url.ends_with(suffix),
"{url} ends with {suffix}, which every hive's startup \
sweep deletes as a stale hook"
);
}
}
}
}