swarm-controller reads its forge token from the store, not a host file

The controller's forge token never entered bao: the forge minted it in its
container, hive-forge-swarm-controller-token copied it to a host file, and the
controller LoadCredential'ed that file. forgeTokenFile defaulted to that path
only where the forge ran, so a controller on any other host had no forge
access unless an operator copied the file by hand.

Now the forge's host publishes it and the controller reads it:

- swarm-bao-forge-publisher (replaces hive-forge-swarm-controller-token)
  waits for the in-container token, logs in with its own forge-publisher leaf
  and writes it to swarm/controller/swarm-controller/forge/token. It runs on
  every forge container start, the old unit's trigger.
- New principal swarm-forge-publisher: leaf minted by glue-bao-tls.nix,
  policy create/update on that one path, written by
  swarm-bao-forge-publisher-policy, CN reserved as a hive name.
- The controller policy gains read on that path. swarm-controller reads it
  at start under its own certificate with the OIDC secret's retry, and runs
  without forge access if it still cannot.
- forgeTokenFile, its LoadCredential and the forgeEnv gate are gone;
  SWARM_CONTROLLER_FORGE_URL is always set. hostSwarmControllerTokenFile is
  removed, and a tmpfiles rule deletes the old host copy on the forge host.

The in-container file stays: the mint unit's only-once guard reads it, and
it is the publisher's source.

Refs #4998

swarm-controller/src/queue_identity.rs

@@ -25,7 +25,7 @@ const PREFIX: &str = "SWARM_CONTROLLER";
 /// Seconds to wait after each failed fetch before the next one. About a minute
 /// in all, which rides out a store restarting alongside this daemon. A longer
 /// outage fails the start, and the unit's `Restart=` tries again from the top.
-const RETRY_DELAYS_S: [u64; 6] = [1, 2, 4, 8, 16, 30];
+pub(crate) const RETRY_DELAYS_S: [u64; 6] = [1, 2, 4, 8, 16, 30];
 
 static IDENTITY: OnceLock<Option<QueueConfig>> = OnceLock::new();
 
@@ -61,7 +61,9 @@ pub async fn init() -> Result<()> {
         None => None,
         Some(coordinates) => {
             let delays = RETRY_DELAYS_S.map(Duration::from_secs);
-            Some(coordinates.with_secret(fetch_with_retry(&delays, fetch_secret).await?))
+            Some(coordinates.with_secret(
+                fetch_with_retry(&delays, "the queue client secret", fetch_secret).await?,
+            ))
         }
     };
     IDENTITY
@@ -118,8 +120,13 @@ async fn fetch_secret() -> Result<String> {
 }
 
 /// Run `attempt`, waiting each of `delays` after a failure, then once more.
-/// Answers with the first success, or the last attempt's error.
-async fn fetch_with_retry<F, Fut>(delays: &[Duration], mut attempt: F) -> Result<String>
+/// Answers with the first success, or the last attempt's error. `what` names the
+/// secret in the log line and the error.
+pub(crate) async fn fetch_with_retry<F, Fut>(
+    delays: &[Duration],
+    what: &str,
+    mut attempt: F,
+) -> Result<String>
 where
     F: FnMut() -> Fut,
     Fut: Future<Output = Result<String>>,
@@ -131,7 +138,7 @@ where
                 tracing::warn!(
                     error = %format!("{e:#}"),
                     retry_in = ?delay,
-                    "fetching the queue client secret from the swarm secret store failed"
+                    "fetching {what} from the swarm secret store failed"
                 );
                 tokio::time::sleep(*delay).await;
             }
@@ -139,7 +146,7 @@ where
     }
     attempt().await.with_context(|| {
         format!(
-            "fetching the queue client secret from the swarm secret store, after {} attempts",
+            "fetching {what} from the swarm secret store, after {} attempts",
             delays.len() + 1
         )
     })
@@ -217,7 +224,7 @@ mod tests {
     #[tokio::test]
     async fn a_store_that_comes_back_within_the_window_is_waited_for() {
         let mut calls = 0;
-        let got = fetch_with_retry(&[Duration::ZERO; 3], || {
+        let got = fetch_with_retry(&[Duration::ZERO; 3], "a secret", || {
             calls += 1;
             let n = calls;
             async move {
@@ -236,7 +243,7 @@ mod tests {
     #[tokio::test]
     async fn a_store_that_stays_down_fails_after_one_attempt_per_delay_plus_one() {
         let mut calls = 0;
-        let err = fetch_with_retry(&[Duration::ZERO; 3], || {
+        let err = fetch_with_retry(&[Duration::ZERO; 3], "a secret", || {
             calls += 1;
             async { bail!("store sealed") }
         })