swarm-controller: bound forge calls in job executors

forgejo-api's client has no request timeout, so a forge that accepts a
connection and never answers held a forge-calling job node forever:
the node never reached Done or Failed, and kept its executor task.

Every executor arm that awaits the forge now runs its work under the
same deadline the periodic passes use. On elapsing, the node fails
through its existing Outcome::Failed path with
"<node>: no answer from the forge within 120s".

Nodes: create_repo, create_forge_user, add_repo_member,
grant_docs_read, init_agent_config_repo, mint_agent_forge_token,
mint_hive_forge_token.

The bound is per node, not per request: the nodes make 1 to about 15
sequential forge and store requests each (mint_hive_forge_token the
most), and the requests live inside forge::Client methods the periodic
passes share. One 2-minute bound covers both, so #5065's
within_pass_deadline / PASS_DEADLINE become within_forge_deadline /
FORGE_DEADLINE, with a log line that no longer says "retrying next
tick" (true of a pass, not of a node).

The InitAgentConfigRepo arm is lifted out of run_swarm_node, like its
siblings, to keep that function under clippy::too_many_lines.

Refs #5066

swarm-controller/src/forge.rs

@@ -231,26 +231,27 @@ where
     }
 }
 
-/// The longest one periodic forge pass may run before it is abandoned until
-/// its next tick. forgejo-api's client has no request timeout, so without
-/// this a forge that accepts a connection and never answers holds the pass,
-/// and the loop running it, forever. Under every pass's 5-minute interval and
-/// far over what a pass that is making progress takes.
-pub(crate) const PASS_DEADLINE: std::time::Duration = std::time::Duration::from_mins(2);
+/// The longest one unit of forge work, a periodic pass or a job node, may run
+/// before it is abandoned. forgejo-api's client has no request timeout, so
+/// without this a forge that accepts a connection and never answers holds the
+/// pass's loop, or the node, forever. Under every pass's 5-minute interval,
+/// and far over what the longest node (`MintHiveForgeToken`, about fifteen
+/// sequential forge and store requests) takes while it is making progress.
+pub(crate) const FORGE_DEADLINE: std::time::Duration = std::time::Duration::from_mins(2);
 
-/// Run `fut`, one periodic pass named `pass` in the log, under
-/// [`PASS_DEADLINE`]. `None` means it was abandoned at the deadline; that is
-/// logged here, and the caller moves on to its next tick.
-pub(crate) async fn within_pass_deadline<T>(
-    pass: &str,
+/// Run `fut`, the forge work named `work` in the log, under
+/// [`FORGE_DEADLINE`]. `None` means it was abandoned at the deadline; that is
+/// logged here.
+pub(crate) async fn within_forge_deadline<T>(
+    work: &str,
     fut: impl std::future::Future<Output = T>,
 ) -> Option<T> {
-    let result = tokio::time::timeout(PASS_DEADLINE, fut).await.ok();
+    let result = tokio::time::timeout(FORGE_DEADLINE, fut).await.ok();
     if result.is_none() {
         tracing::warn!(
-            pass,
-            deadline_s = PASS_DEADLINE.as_secs(),
-            "forge pass: no answer within the deadline; abandoned, retrying next tick"
+            work,
+            deadline_s = FORGE_DEADLINE.as_secs(),
+            "forge work: no answer within the deadline; abandoned"
         );
     }
     result
@@ -1411,6 +1412,15 @@ fn conflict_error() -> ForgejoError {
     ForgejoError::ApiError(ApiErrorKind::Other(StatusCode::CONFLICT).into())
 }
 
+#[cfg(test)]
+impl Client {
+    /// A client for a test's own forge at `url`, holding a placeholder token.
+    pub(crate) fn for_test_forge(url: url::Url) -> Self {
+        let api = Forgejo::new(Auth::Token("stub-token"), url.clone()).unwrap();
+        Self { api, url }
+    }
+}
+
 #[cfg(test)]
 mod tests {
     use super::*;
@@ -1483,14 +1493,14 @@ mod tests {
     #[tokio::test(start_paused = true)]
     async fn a_pass_that_never_answers_is_abandoned_at_the_deadline() {
         let start = tokio::time::Instant::now();
-        let got = within_pass_deadline("test", std::future::pending::<()>()).await;
+        let got = within_forge_deadline("test", std::future::pending::<()>()).await;
         assert_eq!(got, None);
-        assert_eq!(start.elapsed(), PASS_DEADLINE);
+        assert_eq!(start.elapsed(), FORGE_DEADLINE);
     }
 
     #[tokio::test(start_paused = true)]
     async fn a_pass_that_answers_returns_its_result() {
-        assert_eq!(within_pass_deadline("test", async { 7 }).await, Some(7));
+        assert_eq!(within_forge_deadline("test", async { 7 }).await, Some(7));
     }
 
     #[tokio::test]