hive-matrix: load the swarm's appservice and promote its sender

The matrix container renders the swarm registration before tuwunel starts
(`requiredBy` it, no network), tuwunel loads it as a second `.yaml`
credential, and a publish unit hands its token to the store. tuwunel 1.9.1
refuses only a duplicate id or as_token, not overlapping non-exclusive
namespaces, so it sits beside the hive's `hyperhive` registration.

`admin_execute` promotes exactly `@swarm` at boot. The module-eval pin
narrows from "no account" to that one list, compared whole so a second
entry fails; `admin_execute_errors_ignore` is pinned too. matrix-ctl may
write the swarm token's leaf, and the controller may only read it.

Everything is gated on matrix-ctl's store identity: with nobody to publish
the token, the registration would be an admin credential nobody reads.

nix/host-modules/hive-matrix.nix

@@ -172,6 +172,26 @@ let
     isReadOnly = true;
   });
 
+  # ── the swarm's own appservice ──────────────────────────────────────
+  #
+  # A second registration beside the hive's `hyperhive` one, and the swarm's
+  # identity on this homeserver: `swarm-controller` creates every agent's
+  # account with its token. Minted INSIDE this container by
+  # `swarm-matrix-ctl appservice render` and published to the store by
+  # `appservice publish`, which is why it is gated on the same identity as
+  # the sender mint: with nobody to publish it, a registration here would be
+  # an admin credential nobody reads.
+  #
+  # Its sender is promoted to homeserver admin at boot (`admin_execute`
+  # below), so its token goes only to a store path no hive's policy reaches.
+  # `swarm` is in ../reserved-names.nix, so no agent can be created as this
+  # account.
+  swarmSenderLocalpart = "swarm";
+  # Inside the container: the render unit's `StateDirectory`. `ephemeral =
+  # false` keeps it across restarts, so "mint when absent" is mint once.
+  swarmAppserviceDir = "/var/lib/swarm-matrix-appservice";
+  swarmAppserviceCredentialId = "swarm-appservice.yaml";
+
   # Where a reader of the published credential is told the token is good for.
   # Empty when this hive serves no vhost: `matrix::Credential.homeserver` is an
   # `Option`, and matrix-ctl reads an empty variable as absent rather than as
@@ -1304,21 +1324,27 @@ in
               # `Services::start()`, before the listener accepts anything.
               appservice_dir = appserviceCredentialDir;
 
-              # No `admin_execute` promotion for `@${hiveLocalpart}`. The
+            }
-              # hive's account is an ordinary user: it creates the hive
+            # Exactly one account is promoted to homeserver admin at boot:
-              # Space and chat room and invites agents into them, all of
+            # the swarm appservice's sender. Its token is read by
-              # which ride on being the rooms' own creator at power level
+            # matrix-ctl and swarm-controller only. `@${hiveLocalpart}`
-              # 100, and none of which is a homeserver-admin capability.
+            # stays an ordinary user, because every hive reads its
-              # Granting it server admin at boot would hand a credential
+            # credential; what it does (the Space, the chat room, the
-              # that every hive reads far more than the work needs.
+            # invites) rides on being the rooms' creator, not on admin.
-              #
+            #
-              # The two operations that do need an admin sender —
+            # The registration load creates the sender inside
-              # `!admin users make-user-admin` and
+            # `Services::start()`, and `admin_execute` runs after that and
-              # `!admin users reset-password`, both messages into
+            # before the listener accepts anything. `make-user-admin` is a
-              # `#admins:${effectiveServerName}` — therefore have no
+            # no-op on an account that is already admin.
-              # working sender here. They are swarm-level operations and
+            # `admin_execute_errors_ignore` is load-bearing: a failing
-              # are being rehomed as such; until then they fail, loudly,
+            # command aborts startup when it is false.
-              # rather than being served by an over-privileged token.
+            // lib.optionalAttrs ctlActive {
+              admin_execute = [
+                "users make-user-admin @${swarmSenderLocalpart}:${effectiveServerName}"
+              ];
+              admin_execute_errors_ignore = true;
+            }
+            // {
               # Server-side E2EE is opt-in (default off); the agent matrix
               # client always supports decryption regardless.
               allow_encryption = cfg.allowEncryption;
@@ -1387,7 +1413,13 @@ in
             # every OAuth exchange, not just at startup, so it has to
             # outlive the unit's start — a credentials path does.
             "oidc_client_secret:${toString deployCfg.matrix.sso.clientSecretFile}"
-          ];
+          ]
+          # The swarm registration, a second `.yaml` in the same directory:
+          # tuwunel loads every one it finds (`appservice/mod.rs`), and
+          # refuses only a duplicate `id` or `as_token`, never an
+          # overlapping namespace. A missing source fails this unit, which
+          # is why the render below is `requiredBy` it and local only.
+          ++ lib.optional ctlActive "${swarmAppserviceCredentialId}:${swarmAppserviceDir}/swarm.yaml";
 
           # Publish the appservice sender account's access token to the swarm
           # store, once, under an identity that belongs to this container and
@@ -1449,6 +1481,61 @@ in
             };
           };
 
+          # The swarm registration, minted and rendered before the homeserver
+          # loads it. No network and no store: this is on tuwunel's start
+          # path, and a store outage must not keep the homeserver down.
+          systemd.services.swarm-matrix-appservice-render = lib.mkIf ctlActive {
+            description = "render the swarm's appservice registration";
+            before = [ "tuwunel.service" ];
+            requiredBy = [ "tuwunel.service" ];
+            serviceConfig = {
+              Type = "oneshot";
+              RemainAfterExit = true;
+              ExecStart = "${deployCfg.matrix.ctlPackage}/bin/swarm-matrix-ctl appservice render";
+              StateDirectory = baseNameOf swarmAppserviceDir;
+              StateDirectoryMode = "0700";
+              UMask = "0077";
+              SyslogIdentifier = "swarm-matrix-appservice";
+            };
+            environment = {
+              MATRIX_APPSERVICE_DIR = swarmAppserviceDir;
+              MATRIX_APPSERVICE_SENDER = swarmSenderLocalpart;
+              # The hive registration's namespace. Agents' localparts are
+              # bare `[a-z0-9-]`, so nothing narrower covers them without
+              # also covering people.
+              MATRIX_APPSERVICE_USER_REGEX = appserviceUserRegex;
+            };
+          };
+
+          # Hand the swarm registration's token to swarm-controller, through
+          # the store. Same identity and retry shape as `swarm-matrix-ctl`
+          # above; the write lands at a path only matrix-ctl and the
+          # controller are granted (./swarm-bao.nix).
+          systemd.services.swarm-matrix-appservice-publish = lib.mkIf ctlActive {
+            description = "publish the swarm's appservice token to the swarm secret store";
+            after = [ "swarm-matrix-appservice-render.service" ];
+            requires = [ "swarm-matrix-appservice-render.service" ];
+            wantedBy = [ "multi-user.target" ];
+            serviceConfig = {
+              Type = "oneshot";
+              ExecStart = "${deployCfg.matrix.ctlPackage}/bin/swarm-matrix-ctl appservice publish";
+              Restart = "on-failure";
+              RestartSec = 30;
+              TimeoutStartSec = 60;
+              SyslogIdentifier = "swarm-matrix-appservice";
+            };
+            environment = {
+              BAO_ADDR = "https://${baoCfg.domain}:${toString baoCfg.port}";
+              BAO_CLIENT_CERT = deployCfg.matrix.ctlBaoClientCertFile;
+              BAO_CLIENT_KEY = deployCfg.matrix.ctlBaoClientKeyFile;
+              MATRIX_APPSERVICE_CERT_ROLE = ctlCertRole;
+              MATRIX_APPSERVICE_DIR = swarmAppserviceDir;
+            }
+            // lib.optionalAttrs (deployCfg.bao.serverCaFile != null) {
+              BAO_CACERT = deployCfg.bao.serverCaFile;
+            };
+          };
+
           environment.systemPackages = [ deployCfg.matrix.package ];
         };
     };

nix/host-modules/swarm-bao.nix

@@ -194,6 +194,10 @@ let
   # `read` too: `mint_and_verify` reads a credential back before writing so a
   # re-run keeps the value a live agent already holds instead of rotating it —
   # the read is required, not incidental.
+  #
+  # The swarm appservice token, read-only: the controller creates agents'
+  # matrix accounts with it and never writes it. matrix-ctl mints and publishes
+  # it (`matrixCtlPolicyText` below).
   controllerPolicyText = ''
     path "auth/cert/certs/hive-*" {
       capabilities = ["create", "update", "read", "delete"]
@@ -214,6 +218,10 @@ let
     path "${credentialMountPath}/data/swarm/agents/*" {
       capabilities = ["create", "read", "update"]
     }
+
+    path "${credentialMountPath}/data/${swarmAppserviceTokenLeaf}" {
+      capabilities = ["read"]
+    }
   '';
 
   # The identity that copies authelia's minted OIDC client secrets into the
@@ -278,10 +286,18 @@ let
   # restart would mint a second access token and invalidate the hive's. A read
   # here recovers one secret this principal itself wrote, which is a much
   # narrower grant than the publisher's would have been.
+  #
+  # The second stanza is the swarm appservice's token, which matrix-ctl mints
+  # inside the container and publishes here for the controller. `read` for the
+  # same reason: publish compares before it writes.
   matrixCtlPolicyText = ''
     path "${credentialMountPath}/data/swarm/hives/${matrixCtlHive}/matrix/sender-token" {
       capabilities = ["create", "update", "read"]
     }
+
+    path "${credentialMountPath}/data/${swarmAppserviceTokenLeaf}" {
+      capabilities = ["create", "update", "read"]
+    }
   '';
 
   # Which hive matrix-ctl mints for. This host's own by default, which is right
@@ -295,6 +311,12 @@ let
   # locally, which is the same degrade a store that was never deployed gives.
   matrixCtlHive = baoDeploy.matrixCtlHiveName;
 
+  # The swarm appservice token's leaf, the nix half of
+  # `swarm_secret_client::matrix::swarm_appservice_token_path`. Under
+  # `controller/`, the one kind no hive's policy reads: its sender is the
+  # homeserver's admin.
+  swarmAppserviceTokenLeaf = "swarm/controller/swarm-controller/matrix/appservice-token";
+
   # The KV v2 engine the controller writes agent credentials through. Named
   # once because the grant above and the `secrets enable` in the bootstrap unit
   # have to agree: a policy pointing at a mount nobody created is precisely the

nix/module-eval/bao-grants.nix

@@ -334,12 +334,18 @@ let
       # are what keep it from drifting back — neither the `services/*` tree nor
       # a `hives/*` wildcard may appear, since either one hands matrix-ctl (or
       # a hive) reach beyond the single leaf it owns.
-      name = "matrix-ctl's grant is one hive's sender token path and nothing else";
+      #
+      # The one other leaf is the swarm appservice token, which matrix-ctl
+      # mints and publishes for the controller. Counted, so a third stanza
+      # fails rather than riding along beside two correct ones.
+      name = "matrix-ctl's grant is one hive's sender token and the swarm appservice token, nothing else";
       ok =
         let
           s = baoGrantHere.systemd.services.swarm-bao-matrix-ctl-policy.script;
         in
         lib.hasInfix "path \"secret/data/swarm/hives/h1/matrix/sender-token\" {" s
+        && lib.hasInfix "path \"secret/data/swarm/controller/swarm-controller/matrix/appservice-token\" {\n  capabilities = [\"create\", \"update\", \"read\"]\n}" s
+        && lib.length (lib.splitString "path \"" s) == 3
         && !(lib.hasInfix "secret/data/swarm/services" s)
         && !(lib.hasInfix "secret/data/swarm/agents" s)
         && !(lib.hasInfix "secret/data/swarm/hives/*" s)
@@ -747,6 +753,14 @@ let
         in
         lib.hasInfix "path \"secret/data/swarm/agents/*\" {\n  capabilities = [\"create\", \"read\", \"update\"]" s;
     }
+    {
+      # The swarm appservice token is a homeserver-admin credential. The
+      # controller mints agents' accounts with it and has no business replacing
+      # it: matrix-ctl is its one writer. Pinned as the whole stanza, so an
+      # added capability fails.
+      name = "the controller reads the swarm appservice token and cannot write it";
+      ok = lib.hasInfix "path \"secret/data/swarm/controller/swarm-controller/matrix/appservice-token\" {\n  capabilities = [\"read\"]\n}" baoGrantHere.systemd.services.swarm-bao-controller-policy.script;
+    }
     {
       # The policy above grants paths under a mount nothing else creates, so
       # the unit that writes the policy has to create it too — otherwise every

nix/module-eval/bao-matrix-reader.nix

@@ -392,18 +392,64 @@ let
         && !(matrixNoBaoIdentity.containers.hive-matrix.bindMounts ? "/var/lib/swarm-bao-pki");
     }
     {
-      # 🩸 The privilege arm of the credential this slice publishes: the
+      # 🩸 The privilege arm: exactly one account is a homeserver admin, the
-      # account it belongs to must not be a homeserver admin. Read on the
+      # swarm appservice's sender, whose token only matrix-ctl and the
-      # rendered homeserver settings rather than on an option, because the
+      # controller read. Read on the rendered settings rather than on an
-      # grant was never an option — it was a boot command in `admin_execute`,
+      # option, because the grant is a boot command in `admin_execute`, and a
-      # and a command list is exactly the shape a later edit re-adds without
+      # command list is exactly the shape a later edit extends without anything
-      # anything noticing.
+      # noticing — so the list is compared whole, and a second entry fails.
-      name = "the homeserver promotes no account to admin at boot";
+      name = "the homeserver promotes exactly the swarm sender to admin at boot, and no hive's sender";
       ok =
         let
           g = baoWithMatrix.containers.hive-matrix.config.services.matrix-tuwunel.settings.global;
         in
-        !(g ? admin_execute) || g.admin_execute == [ ];
+        g.admin_execute == [ "users make-user-admin @swarm:${g.server_name}" ]
+        && !(lib.any (c: lib.hasInfix "@hive-" c) g.admin_execute);
+    }
+    {
+      # Load-bearing rather than lenient: tuwunel aborts startup on a failing
+      # `admin_execute` command when this is false, and the homeserver must not
+      # stay down over a promotion.
+      name = "a failed admin promotion does not stop the homeserver";
+      ok =
+        baoWithMatrix.containers.hive-matrix.config.services.matrix-tuwunel.settings.global.admin_execute_errors_ignore
+          or false;
+    }
+    {
+      # The swarm registration reaches tuwunel the way the hive's does: a
+      # `.yaml` credential in the directory `appservice_dir` names.
+      name = "tuwunel loads the swarm registration as a yaml credential";
+      ok = lib.elem "swarm-appservice.yaml:/var/lib/swarm-matrix-appservice/swarm.yaml" (
+        baoWithMatrix.containers.hive-matrix.config.systemd.services.tuwunel.serviceConfig.LoadCredential
+      );
+    }
+    {
+      # A missing `LoadCredential` source fails tuwunel, so the render has to
+      # have run first — and it is local only, so it cannot fail on a store.
+      name = "the swarm registration is rendered before tuwunel and required by it, without the store";
+      ok =
+        let
+          r = baoWithMatrix.containers.hive-matrix.config.systemd.services.swarm-matrix-appservice-render;
+        in
+        lib.elem "tuwunel.service" r.before
+        && lib.elem "tuwunel.service" r.requiredBy
+        && lib.hasSuffix "swarm-matrix-ctl appservice render" r.serviceConfig.ExecStart
+        && !(r.environment ? BAO_ADDR);
+    }
+    {
+      # The absence arm for the four above: with no matrix-ctl identity there
+      # is nobody to publish the swarm token, so no admin, no registration and
+      # no render — rather than an admin credential nobody reads.
+      name = "a matrix container with no store identity has no swarm appservice and promotes nobody";
+      ok =
+        let
+          c = matrixNoBaoIdentity.containers.hive-matrix.config;
+          g = c.services.matrix-tuwunel.settings.global;
+        in
+        !(g ? admin_execute)
+        && !(c.systemd.services ? swarm-matrix-appservice-render)
+        && !(c.systemd.services ? swarm-matrix-appservice-publish)
+        && !(lib.any (lib.hasPrefix "swarm-appservice.yaml") c.systemd.services.tuwunel.serviceConfig.LoadCredential);
     }
     {
       # 🩸 The arm the whole refusal exists for. Both readers are gated on their