swarm UI: an alerts page, read by the controller with a Viewer Grafana token from bao

swarm-bao-grafana-controller-token, on the host that runs Grafana, logs in
to Grafana's API over its socket as the built-in admin (the password
swarm-bao-grafana-admin minted, #5138) and keeps a `swarm-controller`
service account that is an enabled Viewer. It mints a token for that
account and writes it to bao at
swarm/controller/swarm-controller/grafana/token under Grafana's leaf. A
ten-minute timer runs it again. The unit keeps a stored token Grafana
accepts. If Grafana refuses the stored token with 401, or bao holds
none, the unit deletes every token of the account and mints a new one.
Grafana's grant gains create/read/update on that leaf, and the
controller's gains read.

swarm-controller serves GET /api/alerts from Grafana's
/api/prometheus/grafana/api/v1/alerts. It reads the token from bao,
keeps it in memory, and re-reads bao once on a 401. The HTTP client has
5s connect and 15s request timeouts. Any failure is a 503 that names its
cause. swarm-ui gets an /alerts page listing them, built the same way as
the hives page.

This is option (c) on #5129, the operator's choice.

docs/swarm/secrets.md

@@ -60,6 +60,7 @@ neither is a renaming of the other.
 | a hive's copy of its own OIDC secret                           | hive-c0re reads it out of the swarm secret store when it first connects to the queue, under the hive's own leaf                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  | the daemon's memory only; never on disk                                                                                                                                                                                                                                                                                                                                            | same read, co-located or not. Nothing for an operator to place beyond the hive's store leaf (`deploy.bao.clientCertFile`)                                                                                                                                                                                                                                                                                                                                                                                 |
 | the hive collector's copy of its hive's OIDC secret            | `swarm-bao-hive-otel-oidc.service` reads it out of the swarm secret store under the hive's own leaf, **on every hive that runs the collector and holds that leaf**                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               | `/var/lib/hive-otel-oidc/hive-<name>.secret` on the hive's host, 0400 root, loaded by `LoadCredential`                                                                                                                                                                                                                                                                             | same unit, same path, co-located or not. Nothing for an operator to place beyond the hive's store leaf (`deploy.bao.clientCertFile`)                                                                                                                                                                                                                                                                                                                                                                      |
 | the swarm controller's forge token                             | the forge's `forgejo-swarm-controller-account`, in-container, once                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               | `/var/lib/forgejo/swarm-controller-token` inside the forge container, the minter's record that it has minted; `swarm-bao-forge-publisher` copies it into the swarm secret store at `swarm/controller/swarm-controller/forge/token` each time the forge container starts and each time the mint writes the file; the controller holds it in memory only                             | nothing to place beyond the controller's store leaf. A forge host without the store needs the publisher's leaf placed (`deploy.bao.forgePublisherClientCertFile`)                                                                                                                                                                                                                                                                                                                                         |
+| the swarm controller's Grafana token                           | `swarm-bao-grafana-controller-token.service`, **on the host that runs Grafana**, as Grafana's built-in admin over its socket: it creates the `swarm-controller` service account as a Viewer (and sets it back to Viewer on every run), mints a token for it and writes it to the swarm secret store at `swarm/controller/swarm-controller/grafana/token` under Grafana's leaf. Re-run every ten minutes: it keeps a stored token Grafana accepts, and replaces one Grafana refuses with 401 (or a missing one), deleting every other token of that account first                                                                                                                                                 | the swarm secret store only; the controller reads it for `GET /api/alerts` and holds it in memory, re-reading the store once when Grafana answers 401                                                                                                                                                                                                                              | nothing to place: Grafana's leaf is already required to run Grafana. Until the unit has run, `/api/alerts` answers 503 naming the empty path                                                                                                                                                                                                                                                                                                                                                              |
 | an agent's claude login (`.credentials.json`)                  | `swarm-claude-login.service`, in its container, when an operator completes `claude auth login` from the agent's card in the swarm UI; it stores the file as a new version at `swarm/agents/<agent>/claude-credentials` under its own leaf, which may create and update that path and read nothing under an agent                                                                                                                                                                                                                                                                                                                                                                                                 | the swarm secret store at that path; the agent's `hive-agent` reads it under the agent's own leaf every 30 seconds while it waits for a login and writes it to `~/.claude/.credentials.json`                                                                                                                                                                                       | the same read, co-located or not; nothing for an operator to place beyond the agent's store leaf. When the store's copy and the agent's differ the KV version decides: the agent installs only a version newer than the last it installed (`claude-credentials-version` under the harness dir), because claude rotates the refresh token in its own copy and the stored one goes stale. The agent ignores a stale or equal version, and a relogin writes a newer one — [claude-login.md](claude-login.md) |
 | the queue responder's copy of its OIDC secret                  | the responder reads it out of the swarm secret store itself as it starts, under its own leaf, and again when authelia refuses it                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 | the daemon's memory only; never on disk                                                                                                                                                                                                                                                                                                                                            | same read, wherever authelia runs. Nothing for an operator to place beyond the responder's store leaf (`deploy.nats.authBaoClientCertFile`)                                                                                                                                                                                                                                                                                                                                                               |
 | authelia subject store                                         | `swarmctl` and `swarm-authelia-bridge`                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           | `users.yml` — one file, read and written by both                                                                                                                                                                                                                                                                                                                                   | `swarmctl`, on the host that runs authelia                                                                                                                                                                                                                                                                                                                                                                                                                                                                |

docs/swarm/ui.md

@@ -8,12 +8,13 @@ domain, covers host-level detail for one hive.
 ## What it shows
 
 | route                                          | what                                                                                                                                        |
-| ---------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- |
+| ---------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
 | `/`                                            | the start page: the swarm's name over a digital-rain backdrop; the swarm name in the header links here                                      |
 | `/hives`                                       | the hive directory, each hive with its last reported status                                                                                 |
 | `/agents`                                      | every agent: status, config PR, wanted state, inference settings, resource caps; create agents, link forge, matrix and GitHub accounts      |
 | `/agents/<name>/terminal`                      | one agent's live terminal                                                                                                                   |
 | `/agents/<name>/subagents/<subagent>/terminal` | one of that agent's subagents' live terminal                                                                                                |
+| `/alerts`                                      | the alerts the swarm's Grafana holds now (firing, pending, no data, error), read through the controller with a Viewer service-account token |
 | `/jobs`                                        | the controller's job graph — where agent creation and credential mints show progress                                                        |
 | `/schedules`                                   | the scheduled prompts: add, edit, pause, resume and remove                                                                                  |
 | `/issues`                                      | a cross-repo issue report                                                                                                                   |

frontend/packages/swarm-ui/src/App.tsx

@@ -4,6 +4,7 @@ import { Route, Switch } from "wouter-preact";
 import { Shell } from "./shell/Shell.js";
 import { AgentsPage } from "./pages/agents/AgentsPage.js";
 import { AgentTerminalPage } from "./pages/agents/AgentTerminalPage.js";
+import { AlertsPage } from "./pages/AlertsPage.js";
 import { ComponentsPage } from "./pages/ComponentsPage.js";
 import { JobsPage } from "./pages/JobsPage.js";
 import { HivesPage } from "./pages/HivesPage.js";
@@ -27,6 +28,7 @@ export function App() {
       <Switch>
         <Route path="/" component={StartPage} />
         <Route path="/hives" component={HivesPage} />
+        <Route path="/alerts" component={AlertsPage} />
         <Route path="/agents" component={AgentsPage} />
         <Route path="/agents/:name/terminal" component={AgentTerminalPage} />
         <Route

frontend/packages/swarm-ui/src/pages/AlertsPage.tsx

@@ -0,0 +1,138 @@
+// <AlertsPage> — `/alerts`' content: the alerts the swarm's Grafana holds
+// now, from swarm-controller's `GET /api/alerts`. Same shape as `HivesPage`:
+// one fetch on a `RefreshIntervalPicker` cadence, rendered as a `Table`.
+import { useState } from "preact/hooks";
+import { ApiErrorPanel } from "@hive/shared/api-error-panel.js";
+import { readApiError, type ProblemDetails } from "@hive/shared/api-error.js";
+import { Badge, type BadgeTone } from "@hive/shared/badge.js";
+import { Panel } from "../ui/panel/Panel.js";
+import { RelativeTime } from "../ui/relative-time/RelativeTime.js";
+import {
+  RefreshIntervalPicker,
+  useRefreshInterval,
+  type RefreshIntervalMs,
+} from "../ui/refresh-interval/RefreshInterval.js";
+import { Table, type TableColumn } from "../ui/table/Table.js";
+
+interface Alert {
+  name: string;
+  // Grafana's state for the instance: `Alerting`, `Pending`, `NoData` or
+  // `Error`. Anything else renders neutral.
+  state: string;
+  active_at: string | null;
+  summary: string | null;
+  labels: Record<string, string>;
+}
+
+const STATE_TONE: Record<string, BadgeTone> = {
+  Alerting: "negative",
+  Error: "negative",
+  NoData: "warning",
+  Pending: "warning",
+};
+
+// Grafana reports an instance that never became active with the zero time,
+// which `Date.parse` reads as year 1.
+function activeSinceMs(a: Alert): number | null {
+  if (a.active_at === null) return null;
+  const ms = Date.parse(a.active_at);
+  return Number.isNaN(ms) || ms <= 0 ? null : ms;
+}
+
+// A row needs a key that is unique among instances of the same rule.
+function alertKey(a: Alert): string {
+  return JSON.stringify(Object.entries(a.labels).sort());
+}
+
+const COLUMNS: TableColumn<Alert>[] = [
+  {
+    key: "name",
+    header: "alert",
+    render: (a) => a.name,
+    sortBy: (a) => a.name,
+    filterValue: (a) => a.name,
+  },
+  {
+    key: "state",
+    header: "state",
+    render: (a) => {
+      const since = activeSinceMs(a);
+      return (
+        <Badge
+          tone={STATE_TONE[a.state] ?? "neutral"}
+          value={
+            since !== null ? (
+              <>
+                {a.state} (<RelativeTime epochMs={since} />)
+              </>
+            ) : (
+              a.state
+            )
+          }
+        />
+      );
+    },
+    sortBy: (a) => a.state,
+    filterValues: (a) => [a.state],
+    filterMode: "multiselect",
+  },
+  {
+    key: "summary",
+    header: "summary",
+    render: (a) => a.summary ?? "—",
+    sortBy: (a) => a.summary ?? "",
+    filterValue: (a) => a.summary ?? "",
+  },
+];
+
+const DEFAULT_INTERVAL_MS: RefreshIntervalMs = 30_000;
+
+export function AlertsPage() {
+  const [alerts, setAlerts] = useState<Alert[] | null>(null);
+  const [error, setError] = useState<ProblemDetails | null>(null);
+  const [intervalMs, setIntervalMs] =
+    useState<RefreshIntervalMs>(DEFAULT_INTERVAL_MS);
+
+  useRefreshInterval(intervalMs, () => {
+    (async () => {
+      const r = await fetch("/api/alerts");
+      if (!r.ok) {
+        setError(await readApiError(r));
+        return;
+      }
+      setAlerts((await r.json()) as Alert[]);
+      setError(null);
+    })().catch((e: unknown) => setError({ detail: String(e) }));
+  });
+
+  return (
+    <Panel
+      title="alerts"
+      icon="🚨"
+      actions={
+        <RefreshIntervalPicker
+          id="alerts-refresh"
+          value={intervalMs}
+          onChange={setIntervalMs}
+        />
+      }
+    >
+      {error ? (
+        <ApiErrorPanel
+          context="failed to load Grafana's alerts"
+          problem={error}
+        />
+      ) : null}
+      {!error && alerts === null ? <p>loading…</p> : null}
+      {alerts && alerts.length === 0 ? <p>no alerts.</p> : null}
+      {alerts && alerts.length > 0 ? (
+        <Table
+          columns={COLUMNS}
+          rows={alerts}
+          rowKey={alertKey}
+          storageKey="swarm-ui:alerts:table-filters"
+        />
+      ) : null}
+    </Panel>
+  );
+}

frontend/packages/swarm-ui/src/shell/Shell.tsx

@@ -51,6 +51,7 @@ const MOTION_KEY = "swarm-ui:motion-override";
 const NAV_ITEMS: { href: string; label: string; accent: string }[] = [
   { href: "/hives", label: "hives", accent: "var(--purple)" },
   { href: "/agents", label: "agents", accent: "var(--green)" },
+  { href: "/alerts", label: "alerts", accent: "var(--red)" },
   { href: "/jobs", label: "jobs", accent: "var(--pink)" },
   { href: "/schedules", label: "schedules", accent: "var(--cyan)" },
   { href: "/issues", label: "issues", accent: "var(--yellow)" },

nix/host-modules/glue-bao-readers-policy-order.nix

@@ -67,7 +67,8 @@ let
       ];
     };
     # ./swarm-grafana.nix, under Grafana's leaf, which may create the admin
-    # password and the metrics bearer only once that policy is written.
+    # password, the metrics bearer and the controller's Grafana token only
+    # once that policy is written.
     swarm-bao-grafana-admin = {
       here = deployCfg.grafana.enable;
       policies = [ "swarm-bao-grafana-oidc-policy" ];
@@ -76,6 +77,10 @@ let
       here = deployCfg.grafana.enable;
       policies = [ "swarm-bao-grafana-oidc-policy" ];
     };
+    swarm-bao-grafana-controller-token = {
+      here = deployCfg.grafana.enable;
+      policies = [ "swarm-bao-grafana-oidc-policy" ];
+    };
     # ./hive-matrix.nix, under matrix-ctl's leaf.
     swarm-bao-matrix-oidc = {
       here = deployCfg.matrix.enable;

nix/host-modules/glue-bao-readers-store-order.nix

@@ -47,6 +47,7 @@ let
     swarm-bao-grafana-oidc = deployCfg.grafana.enable;
     swarm-bao-grafana-admin = deployCfg.grafana.enable;
     swarm-bao-grafana-metrics-bearer = deployCfg.grafana.enable;
+    swarm-bao-grafana-controller-token = deployCfg.grafana.enable;
     # ./swarm-nats.nix
     swarm-bao-nats-callout-xkey = deployCfg.nats.enable;
     # ./swarm-ci-runner.nix

nix/host-modules/swarm-bao.nix

@@ -369,9 +369,9 @@ let
   # Each hive's matrix sender token (`matrix_account::hive_sender`) grants `read`
   # for the same keep-if-live reason, and `+` for the same glob-only-as-last-segment reason.
   #
-  # Three leaves it reads and never writes: the swarm appservice token
+  # Four leaves it reads and never writes: the swarm appservice token (matrix-ctl),
-  # (matrix-ctl, `matrixCtlPolicyText` below), its own OIDC client secret (the
+  # its OIDC client secret (the publisher), its forge token (the forge's host)
-  # publisher) and its own forge token (the forge's host, `forgePublisherWriters`).
+  # and its Grafana token (Grafana's host). Each writer's grant is named below.
   #
   # The agent PKI grant is its only path on that mount: it can ask the one role
   # for a certificate, not write that role or reach the issuer.
@@ -457,6 +457,10 @@ let
       capabilities = ["read"]
     }
 
+    path "${credentialMountPath}/data/${controllerGrafanaTokenLeaf}" {
+      capabilities = ["read"]
+    }
+
     path "${agentPkiMountPath}/issue/${agentPkiRoleName}" {
       capabilities = ["update"]
     }
@@ -582,6 +586,11 @@ let
   # it; the controller reads it and holds no other copy.
   controllerForgeTokenLeaf = "swarm/controller/swarm-controller/forge/token";
 
+  # The controller's Viewer token for Grafana's API, the nix half of
+  # `swarm_secret_client::grafana::controller_token_path`. Grafana's host
+  # writes it; the controller reads it and holds no other copy.
+  controllerGrafanaTokenLeaf = "swarm/controller/swarm-controller/grafana/token";
+
   # The forge's runner registration token, the nix half of
   # `swarm_secret_client::ci::registration_token_path`. The controller writes
   # it and cannot read it back (`forge::runner_token` keeps the value it last
@@ -911,6 +920,10 @@ let
   # in ./swarm-grafana.nix) and the bearer its metrics datasource presents
   # (`swarm-bao-grafana-metrics-bearer`). `create` without `update`, so a stored
   # value is never replaced.
+  #
+  # The controller's Grafana token is the one it may replace:
+  # `swarm-bao-grafana-controller-token` mints a new one when Grafana refuses
+  # the stored one, as it does after its database is recreated.
   grafanaOidcReaders = [
     {
       name = "swarm-grafana-oidc";
@@ -918,7 +931,12 @@ let
       policyText =
         readStanza "${credentialMountPath}/data/swarm/services/${hyperhiveCfg.swarm.grafana.oidc.clientId}/oidc/client"
         + createStanza "${credentialMountPath}/data/${grafanaAdminLeaf}"
-        + createStanza "${credentialMountPath}/data/${grafanaMetricsBearerLeaf}";
+        + createStanza "${credentialMountPath}/data/${grafanaMetricsBearerLeaf}"
+        + ''
+          path "${credentialMountPath}/data/${controllerGrafanaTokenLeaf}" {
+            capabilities = ["create", "read", "update"]
+          }
+        '';
     }
   ];
 

nix/host-modules/swarm-controller.nix

@@ -811,6 +811,11 @@ in
         # separated: a matrix user id cannot contain a space. Consumed by
         # swarm-controller/src/matrix_account/operator_admin.rs.
         SWARM_CONTROLLER_MATRIX_ADMIN_SYNC_IGNORE_USERS = lib.concatStringsSep " " matrixCfg.adminSync.ignoreUsers;
+        # Where `GET /api/alerts` reads alerts (swarm-controller/src/alerts.rs).
+        # Every swarm has one Grafana, reached by name wherever it runs. The
+        # token is in the store, written by `swarm-bao-grafana-controller-token`
+        # on Grafana's host.
+        SWARM_CONTROLLER_GRAFANA_URL = "https://${swarmCfg.grafana.domain}";
       }
       // queueEnv
       // forgeEnv

nix/host-modules/swarm-grafana.nix

@@ -273,6 +273,14 @@ let
   metricsBearerPath = "/var/lib/grafana-metrics/bearer";
   hostMetricsBearerPath = "/var/lib/nixos-containers/${cfg.machine}${metricsBearerPath}";
 
+  # swarm-controller's token for Grafana's API, minted by
+  # `swarm-bao-grafana-controller-token` below for a Viewer service account of
+  # this name. `controllerGrafanaTokenLeaf` in ./swarm-bao.nix is the nix half
+  # of the path, `swarm_secret_client::grafana::controller_token_path` the Rust
+  # half.
+  controllerAccountName = "swarm-controller";
+  controllerTokenStorePath = "secret/swarm/controller/swarm-controller/grafana/token";
+
   socketPath = "${deployCfg.grafana.socketDir}/grafana.sock";
 
   # Both static NixOS ids, and both checked rather than assumed: nginx has
@@ -849,6 +857,148 @@ in
       '';
     };
 
+    # swarm-controller reads Grafana's alerts with a token of a Viewer service
+    # account, created here through Grafana's API as the built-in admin and
+    # handed over through the store.
+    #
+    # Re-run by the timer below. A stored token Grafana still accepts is kept;
+    # one it refuses with 401, or none at all, is replaced: every token of the
+    # account is deleted and one new one is minted and stored. The account's
+    # role is set back to Viewer on every run.
+    systemd.services.swarm-bao-grafana-controller-token = {
+      description = "give swarm-controller a Viewer token for Grafana's API through the swarm secret store";
+      # Ordered after the store where it runs here:
+      # ./glue-bao-readers-store-order.nix.
+      after = [
+        "container@${cfg.machine}.service"
+        "swarm-bao-grafana-admin.service"
+      ];
+      wantedBy = [ "multi-user.target" ];
+      path = [
+        baoDeploy.package
+        pkgs.coreutils
+        pkgs.curl
+        pkgs.jq
+      ];
+      # ./lib/store-retry.nix. A Grafana still starting refuses the
+      # connection, and that attempt fails into the same retry.
+      inherit (storeRetry) startLimitBurst startLimitIntervalSec;
+      serviceConfig = storeRetry.serviceConfig // {
+        # No `RemainAfterExit`: the timer below starts this again.
+        Type = "oneshot";
+        SyslogIdentifier = "swarm-bao-grafana-controller-token";
+        # Grafana's answers, the new token among them, pass through a temp file.
+        PrivateTmp = true;
+        # Each call below is bounded by curl's own timeouts; a run makes up to
+        # about ten of them.
+        TimeoutStartSec = "3min";
+      };
+      environment = {
+        BAO_ADDR = "https://${baoCfg.domain}:${toString baoCfg.port}";
+        BAO_CLIENT_CERT = baoDeploy.grafanaOidcClientCertFile;
+        BAO_CLIENT_KEY = baoDeploy.grafanaOidcClientKeyFile;
+      }
+      // lib.optionalAttrs (baoDeploy.serverCaFile != null) {
+        BAO_CACERT = baoDeploy.serverCaFile;
+      };
+      script = ''
+        set -euo pipefail
+
+        err="$(mktemp)"
+        body="$(mktemp)"
+        trap 'rm -f "$err" "$body"' EXIT
+
+        # One call to Grafana's API through its socket. Prints the HTTP
+        # status; the body lands in "$body". The credential reaches curl as
+        # a header file, never as an argument.
+        grafana() {
+          local auth="$1" method="$2" path="$3"
+          shift 3
+          curl --silent --show-error --connect-timeout 5 --max-time 15 \
+            --unix-socket ${lib.escapeShellArg socketPath} \
+            -H @<(printf 'Authorization: %s\n' "$auth") \
+            -H 'Content-Type: application/json' \
+            -X "$method" -o "$body" -w '%{http_code}' "$@" \
+            "http://localhost$path"
+        }
+
+        # Fails the attempt unless the last call answered one of the statuses
+        # given.
+        expect() {
+          local what="$1" status="$2"
+          shift 2
+          for ok in "$@"; do
+            [ "$status" = "$ok" ] && return 0
+          done
+          echo "Grafana answered $status to $what:" >&2
+          cat "$body" >&2
+          exit 1
+        }
+
+        if [ ! -s ${lib.escapeShellArg hostAdminPasswordPath} ]; then
+          echo "no Grafana admin password at ${hostAdminPasswordPath} yet; swarm-bao-grafana-admin writes it." >&2
+          exit 1
+        fi
+        admin="Basic $(printf 'admin:%s' "$(< ${lib.escapeShellArg hostAdminPasswordPath})" | base64 -w0)"
+
+        status="$(grafana "$admin" GET '/api/serviceaccounts/search?perpage=1000&query=${controllerAccountName}')"
+        expect "the service account search" "$status" 200
+        account="$(jq -c --arg n ${controllerAccountName} '[.serviceAccounts[] | select(.name == $n)][0] // empty' "$body")"
+        if [ -z "$account" ]; then
+          status="$(grafana "$admin" POST /api/serviceaccounts \
+            --data '{"name":"${controllerAccountName}","role":"Viewer","isDisabled":false}')"
+          expect "creating the ${controllerAccountName} service account" "$status" 200 201
+          account="$(jq -c . "$body")"
+          echo "created the ${controllerAccountName} service account as a Viewer"
+        fi
+        id="$(jq -r .id <<<"$account")"
+        if [ "$(jq -r '"\(.role) \(.isDisabled // false)"' <<<"$account")" != "Viewer false" ]; then
+          status="$(grafana "$admin" PATCH "/api/serviceaccounts/$id" \
+            --data '{"role":"Viewer","isDisabled":false}')"
+          expect "setting the ${controllerAccountName} service account to an enabled Viewer" "$status" 200
+          echo "set the ${controllerAccountName} service account back to an enabled Viewer"
+        fi
+
+        if ! BAO_TOKEN="$(bao login -method=cert -token-only 2>"$err")"; then
+          echo "could not log in to swarm-bao with Grafana's certificate; leaving swarm-controller's Grafana token as it is." >&2
+          cat "$err" >&2
+          exit 1
+        fi
+        export BAO_TOKEN
+
+        # Checked against the call the controller makes with it.
+        if stored="$(bao kv get -field=value ${lib.escapeShellArg controllerTokenStorePath} 2>"$err")" && [ -n "$stored" ]; then
+          status="$(grafana "Bearer $stored" GET /api/prometheus/grafana/api/v1/alerts)"
+          case "$status" in
+            200) exit 0 ;;
+            401) echo "Grafana refuses the stored token for ${controllerAccountName}; replacing it." ;;
+            *) expect "reading alerts with the stored token" "$status" 200 ;;
+          esac
+        fi
+
+        status="$(grafana "$admin" GET "/api/serviceaccounts/$id/tokens")"
+        expect "listing the ${controllerAccountName} tokens" "$status" 200
+        for token_id in $(jq -r '.[].id' "$body"); do
+          status="$(grafana "$admin" DELETE "/api/serviceaccounts/$id/tokens/$token_id")"
+          expect "deleting token $token_id of ${controllerAccountName}" "$status" 200
+        done
+
+        status="$(grafana "$admin" POST "/api/serviceaccounts/$id/tokens" \
+          --data "{\"name\":\"${controllerAccountName}-$(date +%s)\"}")"
+        expect "minting a ${controllerAccountName} token" "$status" 200
+        key="$(jq -r .key "$body")"
+        : >"$body"
+        if ! printf '%s' "$key" | bao kv put ${lib.escapeShellArg controllerTokenStorePath} value=- >/dev/null 2>"$err"; then
+          echo "could not store the new ${controllerAccountName} token at ${controllerTokenStorePath}; the next run replaces it." >&2
+          cat "$err" >&2
+          exit 1
+        fi
+        echo "stored a new ${controllerAccountName} token at ${controllerTokenStorePath}"
+      '';
+    };
+
+    systemd.timers.swarm-bao-grafana-controller-token = storeRefreshTimer "check swarm-controller's Grafana token and replace it if Grafana refuses it";
+
     containers.${cfg.machine} = {
       autoStart = true;
       ephemeral = false;

nix/module-eval/bao-controller.nix

@@ -221,10 +221,10 @@ let
         && builtins.elem "bao-client-key.pem:/var/lib/swarm-bao-pki/controller-key.pem" s.swarm-controller.serviceConfig.LoadCredential;
     }
     {
-      # The forge token comes from the store, so the unit is handed no file for
+      # The forge and Grafana tokens come from the store, so the unit is handed
-      # it, and neither fixture runs the forge: a controller reaches it by name
+      # no file for either, and neither fixture runs the forge or Grafana: a
-      # wherever it runs.
+      # controller reaches both by name wherever they run.
-      name = "the controller is handed no forge token file, and is pointed at the forge away from it";
+      name = "the controller is handed no forge token file, and is pointed at the forge and Grafana away from it";
       ok =
         lib.all
           (
@@ -235,6 +235,7 @@ let
             !(lib.any (c: lib.hasPrefix "forge-token:" c) u.serviceConfig.LoadCredential)
             && !(u.environment ? SWARM_CONTROLLER_FORGE_TOKEN_FILE)
             && u.environment.SWARM_CONTROLLER_FORGE_URL == "https://forge.t.local"
+            && u.environment.SWARM_CONTROLLER_GRAFANA_URL == "https://grafana.t.local"
           )
           [
             baoControllerHere

nix/module-eval/bao-grants.nix

@@ -597,8 +597,10 @@ let
       #
       # The admin password and the metrics bearer are the leaves it may create,
       # and without `update`: a grant that could replace one would let a second
-      # run rotate a value Grafana or the metrics store already holds.
+      # run rotate a value Grafana or the metrics store already holds. The
-      name = "the Grafana reader reads its own client secret and creates its admin password and metrics bearer, and nothing else";
+      # controller's Grafana token is the one leaf it may replace, which it
+      # does when Grafana refuses the stored token.
+      name = "the Grafana reader reads its own client secret, creates its admin password and metrics bearer, and writes the controller's Grafana token, and nothing else";
       ok =
         let
           s = baoGrantHere.systemd.services.swarm-bao-grafana-oidc-policy.script;
@@ -622,6 +624,14 @@ let
               "read"
             ];
           }
+          {
+            path = "secret/data/swarm/controller/swarm-controller/grafana/token";
+            caps = [
+              "create"
+              "read"
+              "update"
+            ];
+          }
         ]
         && !(lib.hasInfix "secret/data/swarm/agents" s)
         && !(lib.hasInfix "secret/data/swarm/hives" s)
@@ -938,18 +948,27 @@ let
         && !(noReaders ? swarm-bao-nats-callout-xkey);
     }
     {
-      # The admin password is created under Grafana's leaf, so its unit waits
+      # The admin password and the controller's Grafana token are written under
-      # for the unit writing that leaf's role and policy; before it, the create
+      # Grafana's leaf, so their units wait for the unit writing that leaf's
-      # is refused. The second arm is the absence control.
+      # role and policy; before it, the write is refused. The last arm is the
-      name = "grafana's admin password unit is ordered after grafana's policy unit";
+      # absence control.
+      name = "grafana's admin password and controller token units are ordered after grafana's policy unit";
       ok =
+        lib.all
+          (
+            unit:
             let
-          u = baoGrantWithConsumers.systemd.services.swarm-bao-grafana-admin;
+              u = baoGrantWithConsumers.systemd.services.${unit};
             in
             lib.elem "swarm-bao-grafana-oidc-policy.service" u.after
             && lib.elem "swarm-bao-grafana-oidc-policy.service" u.wants
             && !(lib.elem "swarm-bao-grafana-oidc-policy.service" u.requires)
-        && !(baoGrantNoReaders.systemd.services ? swarm-bao-grafana-admin);
+            && !(baoGrantNoReaders.systemd.services ? ${unit})
+          )
+          [
+            "swarm-bao-grafana-admin"
+            "swarm-bao-grafana-controller-token"
+          ];
     }
     {
       # The ordering is set apart from each reader's own definition, so it can
@@ -1100,6 +1119,7 @@ let
               "swarm-bao-forwarder-oidc"
               "swarm-bao-grafana-admin"
               "swarm-bao-grafana-metrics-bearer"
+              "swarm-bao-grafana-controller-token"
             ]
             ++ policyReaders
           );
@@ -1715,6 +1735,25 @@ let
           "swarm-bao-forge-publisher-policy"
         ];
     }
+    {
+      # The controller's Grafana token: read on the one leaf, pinned as the
+      # whole stanza so an added write fails, and named by no policy but the
+      # controller's and Grafana's, which writes it.
+      name = "the controller reads its Grafana token, and only Grafana's grant writes it";
+      ok =
+        let
+          s = baoGrantHere.systemd.services.swarm-bao-controller-policy.script;
+          naming = lib.filter (
+            u: lib.hasInfix "swarm-controller/grafana/token" baoGrantHere.systemd.services.${u}.script
+          ) grantingUnitNames;
+        in
+        lib.hasInfix "path \"secret/data/swarm/controller/swarm-controller/grafana/token\" {\n  capabilities = [\"read\"]\n}" s
+        &&
+          lib.sort lib.lessThan naming == [
+            "swarm-bao-controller-policy"
+            "swarm-bao-grafana-oidc-policy"
+          ];
+    }
     {
       # The forge's OIDC client secret by name: the forge publisher reads it,
       # and the secret publisher writes it under its `swarm/services/*` grant,

nix/module-eval/grafana.nix

@@ -249,6 +249,33 @@ let
         && lib.hasInfix "admin reset-admin-password --password-from-stdin < /var/lib/grafana-admin/password" service.preStart
         && lib.hasInfix "-config ${resetConfig}" service.script;
     }
+    {
+      # swarm-controller reads alerts with this token, so the account it
+      # belongs to is created and kept a Viewer, the token lands at the path
+      # the controller's grant reads, and an attempt against a Grafana or a
+      # store that never answers ends and retries. The timer re-runs it, so
+      # the unit must not stay active.
+      name = "the controller's Grafana token is a Viewer account's, stored under Grafana's leaf, and each attempt is bounded";
+      ok =
+        let
+          unit = grafanaRemoteAuthelia.systemd.services.swarm-bao-grafana-controller-token;
+          s = unit.script;
+        in
+        !(lib.elem (toString (unit.serviceConfig.TimeoutStartSec or "infinity")) [
+          "infinity"
+          "0"
+        ])
+        && lib.hasInfix "curl --silent --show-error --connect-timeout 5 --max-time 15" s
+        && lib.hasInfix "--data '{\"name\":\"swarm-controller\",\"role\":\"Viewer\",\"isDisabled\":false}'" s
+        && lib.hasInfix "--data '{\"role\":\"Viewer\",\"isDisabled\":false}'" s
+        && lib.hasInfix "bao kv put secret/swarm/controller/swarm-controller/grafana/token value=-" s
+        && lib.hasInfix "--unix-socket /run/swarm-grafana/grafana.sock" s
+        && unit.environment.BAO_CLIENT_CERT == "/etc/pki/bao-grafana-oidc.pem"
+        && unit.serviceConfig.Restart == "on-failure"
+        && !(unit.serviceConfig.RemainAfterExit or false)
+        && lib.elem "swarm-bao-grafana-admin.service" unit.after
+        && grafanaRemoteAuthelia.systemd.timers ? swarm-bao-grafana-controller-token;
+    }
     {
       # Same 403-not-a-miss reason as the matrix and queue arms below: the
       # reader's grant covers the `services` prefix, so a path outside it is

swarm-controller/README.md

@@ -73,6 +73,9 @@ The swarm's control plane. The swarm UI and `swarmctl` are its clients.
   [`docs/swarm/review-classifier.md`](../docs/swarm/review-classifier.md#swarm-level-settings).
 - **Relays** — each agent's terminal and turn-state header as SSE, agent
   icons, the cross-repo issue report, and the UI's quick links.
+- **Alerts** — the swarm Grafana's current alerts (`GET /api/alerts`), read
+  at request time with a Viewer service-account token from the secret store
+  → [`docs/swarm/secrets.md`](../docs/swarm/secrets.md).
 
 It reads its configuration once at startup, from the environment the nix module
 sets. It persists `webhook-secret` and `operator-matrix-admins.json` in its

swarm-controller/src/agent_limits.rs

@@ -508,6 +508,7 @@ pub(crate) mod tests {
             reviews: Arc::default(),
             try_outs: crate::review_classifier::TryOuts::default(),
             runs: crate::review_classifier::runs::Runs::Live(None),
+            alerts: None,
         }
     }
 

swarm-controller/src/alerts.rs

@@ -0,0 +1,391 @@
+//! `GET /api/alerts`: the alerts the swarm's Grafana holds right now, read at
+//! request time for swarm-ui's alerts page.
+//!
+//! Grafana's URL comes from `swarm-controller.nix`
+//! (`SWARM_CONTROLLER_GRAFANA_URL`). The token is a Viewer service account's,
+//! read from the swarm's secret store at
+//! `swarm_secret_client::grafana::controller_token_path` under this daemon's
+//! own certificate. `swarm-bao-grafana-controller-token`, on Grafana's host,
+//! mints it and replaces it whenever Grafana refuses the stored one, so a 401
+//! here re-reads the store once before giving up.
+
+use std::collections::BTreeMap;
+use std::future::Future;
+use std::pin::Pin;
+use std::sync::Arc;
+use std::time::Duration;
+
+use anyhow::{Context, Result};
+use axum::Json;
+use axum::extract::State;
+use axum::http::StatusCode;
+use problem_details::ProblemDetails;
+use serde::{Deserialize, Serialize};
+use utoipa::ToSchema;
+
+use crate::{AppState, error_problem};
+
+/// The env var `swarm-controller.nix` sets.
+const URL_ENV: &str = "SWARM_CONTROLLER_GRAFANA_URL";
+
+/// Grafana's Prometheus-compatible list of alert instances, relative to its
+/// base URL. Instances in the `Normal` state are not listed.
+const ALERTS_PATH: &str = "api/prometheus/grafana/api/v1/alerts";
+
+/// Bound on reaching Grafana.
+const CONNECT_TIMEOUT: Duration = Duration::from_secs(5);
+/// Bound on one whole request to Grafana, body included.
+const TIMEOUT: Duration = Duration::from_secs(15);
+
+type TokenFuture = Pin<Box<dyn Future<Output = Result<String>> + Send>>;
+
+/// Where the token comes from: the store in the daemon, a stand-in in tests.
+pub(crate) type TokenSource = Arc<dyn Fn() -> TokenFuture + Send + Sync>;
+
+/// One alert instance, as `GET /api/alerts` serves it.
+#[derive(Clone, Debug, PartialEq, Serialize, ToSchema)]
+pub(crate) struct Alert {
+    /// The rule's title (Grafana's `alertname` label).
+    pub name: String,
+    /// Grafana's state for the instance: `Alerting`, `Pending`, `NoData` or
+    /// `Error`.
+    pub state: String,
+    /// When the instance entered its state, RFC 3339, as Grafana reports it.
+    pub active_at: Option<String>,
+    /// The rule's `summary` annotation, templated by Grafana.
+    pub summary: Option<String>,
+    /// Every label of the instance, as Grafana reports them.
+    pub labels: BTreeMap<String, String>,
+}
+
+#[derive(Deserialize)]
+struct PromResponse {
+    data: PromData,
+}
+
+#[derive(Deserialize)]
+struct PromData {
+    alerts: Vec<PromAlert>,
+}
+
+#[derive(Deserialize)]
+struct PromAlert {
+    #[serde(default)]
+    labels: BTreeMap<String, String>,
+    #[serde(default)]
+    annotations: BTreeMap<String, String>,
+    state: String,
+    #[serde(rename = "activeAt")]
+    active_at: Option<String>,
+}
+
+impl From<PromAlert> for Alert {
+    fn from(a: PromAlert) -> Self {
+        Self {
+            name: a.labels.get("alertname").cloned().unwrap_or_default(),
+            state: a.state,
+            active_at: a.active_at,
+            summary: a.annotations.get("summary").cloned(),
+            labels: a.labels,
+        }
+    }
+}
+
+/// Grafana's alerts API, with the token last read from the store.
+pub(crate) struct Grafana {
+    http: reqwest::Client,
+    alerts_url: url::Url,
+    source: TokenSource,
+    token: tokio::sync::Mutex<Option<String>>,
+}
+
+impl Grafana {
+    /// # Errors
+    /// A base URL the alerts path cannot be joined onto, or a client that
+    /// cannot be built.
+    pub(crate) fn new(base: &url::Url, source: TokenSource, timeout: Duration) -> Result<Self> {
+        let http = reqwest::Client::builder()
+            .connect_timeout(CONNECT_TIMEOUT)
+            .timeout(timeout)
+            .build()
+            .context("building the Grafana HTTP client")?;
+        Ok(Self {
+            http,
+            alerts_url: base
+                .join(ALERTS_PATH)
+                .with_context(|| format!("joining {ALERTS_PATH} onto {base}"))?,
+            source,
+            token: tokio::sync::Mutex::new(None),
+        })
+    }
+
+    /// The held token, or a fresh one from the source when `reread` or none is
+    /// held yet.
+    async fn token(&self, reread: bool) -> Result<String> {
+        let mut held = self.token.lock().await;
+        if let Some(t) = held.as_ref().filter(|_| !reread) {
+            return Ok(t.clone());
+        }
+        let fresh = (self.source)().await?;
+        *held = Some(fresh.clone());
+        Ok(fresh)
+    }
+
+    async fn get(&self, token: &str) -> Result<reqwest::Response> {
+        self.http
+            .get(self.alerts_url.clone())
+            .bearer_auth(token)
+            .send()
+            .await
+            .with_context(|| format!("GET {}", self.alerts_url))
+    }
+
+    /// Every alert instance Grafana lists now.
+    ///
+    /// # Errors
+    /// No token in the store, Grafana unreachable or slower than the client's
+    /// timeout, a 401 for both the held token and a re-read one, any other
+    /// non-success status, or a body that is not the alerts shape.
+    pub(crate) async fn alerts(&self) -> Result<Vec<Alert>> {
+        let mut res = self.get(&self.token(false).await?).await?;
+        if res.status() == reqwest::StatusCode::UNAUTHORIZED {
+            res = self.get(&self.token(true).await?).await?;
+        }
+        let status = res.status();
+        if !status.is_success() {
+            anyhow::bail!("Grafana answered {status} to GET {}", self.alerts_url);
+        }
+        let body: PromResponse = res
+            .json()
+            .await
+            .with_context(|| format!("reading Grafana's answer to GET {}", self.alerts_url))?;
+        Ok(body.data.alerts.into_iter().map(Alert::from).collect())
+    }
+}
+
+/// One login and one read. An empty value is refused like an absent one.
+async fn token_from_store() -> Result<String> {
+    let path = swarm_secret_client::grafana::controller_token_path()?;
+    let store = crate::store::connect()
+        .await
+        .context("logging in to the swarm secret store")?;
+    let stored: Option<swarm_secret_client::grafana::ControllerToken> = store
+        .read_optional(&path)
+        .await
+        .with_context(|| format!("reading {path}"))?;
+    match stored {
+        Some(t) if !t.value.trim().is_empty() => Ok(t.value.trim().to_owned()),
+        _ => anyhow::bail!(
+            "nothing at {path}: swarm-bao-grafana-controller-token, on the host that runs \
+             Grafana, has not stored this daemon's Grafana token"
+        ),
+    }
+}
+
+/// The daemon's Grafana client, or `None` when `SWARM_CONTROLLER_GRAFANA_URL`
+/// is unset or unusable, which is logged.
+pub(crate) fn from_env() -> Option<Arc<Grafana>> {
+    let raw = std::env::var(URL_ENV).ok()?;
+    let built = url::Url::parse(&raw)
+        .with_context(|| format!("parse {URL_ENV} ({raw}) as a URL"))
+        .and_then(|base| {
+            let source: TokenSource = Arc::new(|| Box::pin(token_from_store()));
+            Grafana::new(&base, source, TIMEOUT)
+        });
+    match built {
+        Ok(g) => Some(Arc::new(g)),
+        Err(e) => {
+            tracing::warn!(error = %format!("{e:#}"), "Grafana misconfigured; /api/alerts is off");
+            None
+        }
+    }
+}
+
+/// The alerts the swarm's Grafana holds now: firing, pending, and those whose
+/// rule has no data or failed to evaluate.
+#[utoipa::path(
+    get,
+    path = "/api/alerts",
+    responses(
+        (status = 200, description = "every alert instance Grafana lists now", body = Vec<Alert>),
+        (status = 503, description = "no Grafana is configured here, its token is not in the store \
+            yet, or Grafana refused, failed or did not answer in time (problem+json)", body = String),
+    ),
+    tag = "alerts"
+)]
+pub(crate) async fn get_alerts(
+    State(state): State<AppState>,
+) -> Result<Json<Vec<Alert>>, ProblemDetails> {
+    let Some(grafana) = state.alerts.as_ref() else {
+        return Err(error_problem(
+            StatusCode::SERVICE_UNAVAILABLE,
+            "no Grafana is configured on this host",
+        ));
+    };
+    grafana.alerts().await.map(Json).map_err(|e| {
+        let detail = format!("{e:#}");
+        tracing::warn!(error = %detail, "reading Grafana's alerts failed");
+        error_problem(StatusCode::SERVICE_UNAVAILABLE, &detail)
+    })
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use axum::body::Body;
+    use axum::http::{HeaderMap, Request};
+    use std::sync::Mutex;
+    use tower::ServiceExt as _;
+
+    /// What the stand-in Grafana answers to a request with the accepted token.
+    const ALERTS_BODY: &str = r#"{"status":"success","data":{"alerts":[{
+        "labels":{"alertname":"swarm queue connect failed","hive":"pr1ma","agent":"atlas"},
+        "annotations":{"summary":"pr1ma/atlas could not connect"},
+        "state":"Alerting","activeAt":"2026-10-15T09:00:00Z","value":"1e+00"}]}}"#;
+
+    /// A stand-in Grafana on a loopback port: the alerts path answers
+    /// [`ALERTS_BODY`] to `Bearer <accepts>` and 401 to anything else, after
+    /// `delay`.
+    async fn fake_grafana(accepts: &'static str, delay: Duration) -> url::Url {
+        let app = axum::Router::new().route(
+            &format!("/{ALERTS_PATH}"),
+            axum::routing::get(move |headers: HeaderMap| async move {
+                tokio::time::sleep(delay).await;
+                let ok = headers
+                    .get(axum::http::header::AUTHORIZATION)
+                    .is_some_and(|v| v == format!("Bearer {accepts}").as_str());
+                if ok {
+                    (StatusCode::OK, ALERTS_BODY)
+                } else {
+                    (StatusCode::UNAUTHORIZED, "")
+                }
+            }),
+        );
+        let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
+        let addr = listener.local_addr().unwrap();
+        tokio::spawn(async move { axum::serve(listener, app).await });
+        url::Url::parse(&format!("http://{addr}/")).unwrap()
+    }
+
+    /// A token source handing out `tokens` in order, the last one forever,
+    /// counting its calls.
+    fn tokens(tokens: &'static [&'static str]) -> (TokenSource, Arc<Mutex<usize>>) {
+        let calls = Arc::new(Mutex::new(0));
+        let counted = Arc::clone(&calls);
+        let source: TokenSource = Arc::new(move || {
+            let mut n = counted.lock().unwrap();
+            let t = tokens[(*n).min(tokens.len() - 1)];
+            *n += 1;
+            Box::pin(async move { Ok(t.to_owned()) })
+        });
+        (source, calls)
+    }
+
+    async fn get_alerts_with(grafana: Option<Grafana>) -> (StatusCode, serde_json::Value) {
+        let (mut state, _) = crate::tests::state_with_roster();
+        state.alerts = grafana.map(Arc::new);
+        let req = Request::get("/api/alerts")
+            .body(Body::empty())
+            .expect("a request");
+        let res = crate::build_app(state)
+            .oneshot(req)
+            .await
+            .expect("infallible");
+        let status = res.status();
+        let bytes = axum::body::to_bytes(res.into_body(), usize::MAX)
+            .await
+            .expect("a body");
+        (status, serde_json::from_slice(&bytes).unwrap_or_default())
+    }
+
+    #[tokio::test]
+    async fn serves_grafanas_alerts_read_with_the_stored_token() {
+        let base = fake_grafana("viewer-token", Duration::ZERO).await;
+        let (source, calls) = tokens(&["viewer-token"]);
+        let grafana = Grafana::new(&base, source, TIMEOUT).unwrap();
+        let (status, body) = get_alerts_with(Some(grafana)).await;
+        assert_eq!(status, StatusCode::OK, "{body}");
+        assert_eq!(
+            body,
+            serde_json::json!([{
+                "name": "swarm queue connect failed",
+                "state": "Alerting",
+                "active_at": "2026-10-15T09:00:00Z",
+                "summary": "pr1ma/atlas could not connect",
+                "labels": {
+                    "alertname": "swarm queue connect failed",
+                    "hive": "pr1ma",
+                    "agent": "atlas",
+                },
+            }])
+        );
+        assert_eq!(*calls.lock().unwrap(), 1);
+    }
+
+    #[tokio::test]
+    async fn the_held_token_is_reused_across_requests() {
+        let base = fake_grafana("viewer-token", Duration::ZERO).await;
+        let (source, calls) = tokens(&["viewer-token"]);
+        let grafana = Grafana::new(&base, source, TIMEOUT).unwrap();
+        grafana.alerts().await.unwrap();
+        grafana.alerts().await.unwrap();
+        assert_eq!(*calls.lock().unwrap(), 1);
+    }
+
+    /// The minting unit replaces a token Grafana refuses, so a 401 re-reads
+    /// the store and sends the request once more.
+    #[tokio::test]
+    async fn a_refused_token_is_read_again_from_the_store() {
+        let base = fake_grafana("replaced", Duration::ZERO).await;
+        let (source, calls) = tokens(&["stale", "replaced"]);
+        let grafana = Grafana::new(&base, source, TIMEOUT).unwrap();
+        let alerts = grafana.alerts().await.unwrap();
+        assert_eq!(alerts.len(), 1);
+        assert_eq!(*calls.lock().unwrap(), 2);
+    }
+
+    #[tokio::test]
+    async fn a_token_refused_twice_is_a_503_naming_the_status() {
+        let base = fake_grafana("replaced", Duration::ZERO).await;
+        let (source, calls) = tokens(&["stale"]);
+        let grafana = Grafana::new(&base, source, TIMEOUT).unwrap();
+        let (status, body) = get_alerts_with(Some(grafana)).await;
+        assert_eq!(status, StatusCode::SERVICE_UNAVAILABLE);
+        assert!(body["detail"].as_str().unwrap().contains("401"), "{body}");
+        assert_eq!(*calls.lock().unwrap(), 2);
+    }
+
+    #[tokio::test]
+    async fn a_grafana_slower_than_the_timeout_is_a_503() {
+        let base = fake_grafana("viewer-token", Duration::from_secs(5)).await;
+        let (source, _) = tokens(&["viewer-token"]);
+        let grafana = Grafana::new(&base, source, Duration::from_millis(200)).unwrap();
+        let started = std::time::Instant::now();
+        let (status, _) = get_alerts_with(Some(grafana)).await;
+        assert_eq!(status, StatusCode::SERVICE_UNAVAILABLE);
+        assert!(started.elapsed() < Duration::from_secs(4));
+    }
+
+    #[tokio::test]
+    async fn no_token_in_the_store_is_a_503_saying_so() {
+        let base = fake_grafana("viewer-token", Duration::ZERO).await;
+        let source: TokenSource =
+            Arc::new(|| Box::pin(async { anyhow::bail!("nothing at the token path") }));
+        let grafana = Grafana::new(&base, source, TIMEOUT).unwrap();
+        let (status, body) = get_alerts_with(Some(grafana)).await;
+        assert_eq!(status, StatusCode::SERVICE_UNAVAILABLE);
+        assert!(
+            body["detail"]
+                .as_str()
+                .unwrap()
+                .contains("nothing at the token path"),
+            "{body}"
+        );
+    }
+
+    #[tokio::test]
+    async fn no_grafana_configured_is_a_503() {
+        let (status, _) = get_alerts_with(None).await;
+        assert_eq!(status, StatusCode::SERVICE_UNAVAILABLE);
+    }
+}

swarm-controller/src/forge_account.rs

@@ -168,6 +168,7 @@ mod tests {
             try_outs: super::super::review_classifier::TryOuts::default(),
             reviews: std::sync::Arc::default(),
             runs: crate::review_classifier::runs::Runs::Live(None),
+            alerts: None,
         }
     }
 

swarm-controller/src/github_account.rs

@@ -138,6 +138,7 @@ mod tests {
             try_outs: super::super::review_classifier::TryOuts::default(),
             reviews: std::sync::Arc::default(),
             runs: crate::review_classifier::runs::Runs::Live(None),
+            alerts: None,
         }
     }
 

swarm-controller/src/linked_accounts.rs

@@ -973,6 +973,7 @@ pub(crate) mod tests {
             try_outs: super::super::review_classifier::TryOuts::default(),
             reviews: std::sync::Arc::default(),
             runs: crate::review_classifier::runs::Runs::Live(None),
+            alerts: None,
         }
     }
 

swarm-controller/src/main.rs

@@ -50,6 +50,7 @@ mod agent_login;
 mod agent_renewal;
 mod agent_state_stream;
 mod agent_status;
+mod alerts;
 mod auth;
 mod builder;
 mod claude_login;
@@ -839,6 +840,7 @@ fn socket_path() -> PathBuf {
         (name = "schedules", description = "scheduled prompts (`swarmctl schedule`)"),
         (name = "review-classifier", description = "trying review classifications against a PR (swarm-ui's try-out page), and the classifier's swarm-wide settings"),
         (name = "pulls", description = "open pull requests, for the try-out page's dropdowns"),
+        (name = "alerts", description = "the swarm Grafana's current alerts (swarm-ui's alerts page)"),
     )
 )]
 struct ApiDoc;
@@ -972,6 +974,9 @@ struct AppState {
     reviews: Arc<review_classifier::review::Reviews>,
     /// The review classifier's finished runs.
     runs: review_classifier::runs::Runs,
+    /// Grafana's alerts API, for `GET /api/alerts`. `None` when no Grafana URL
+    /// is configured.
+    alerts: Option<Arc<alerts::Grafana>>,
 }
 
 /// Env var the controller's NixOS module sets from
@@ -3451,6 +3456,7 @@ async fn main() -> Result<()> {
         try_outs: review_classifier::TryOuts::default(),
         reviews: Arc::default(),
         runs,
+        alerts: alerts::from_env(),
     };
 
     let app = build_app(state);
@@ -3468,6 +3474,7 @@ async fn main() -> Result<()> {
 fn build_app(state: AppState) -> axum::Router {
     let (router, api) = OpenApiRouter::<AppState>::with_openapi(ApiDoc::openapi())
         .routes(routes!(health))
+        .routes(routes!(alerts::get_alerts))
         .routes(routes!(get_hives))
         .routes(routes!(get_hives_status))
         .routes(routes!(get_links))
@@ -3660,6 +3667,7 @@ mod tests {
             try_outs: super::review_classifier::TryOuts::default(),
             reviews: std::sync::Arc::default(),
             runs: super::review_classifier::runs::Runs::Live(None),
+            alerts: None,
         };
         (state, sched)
     }

swarm-controller/src/matrix_account.rs

@@ -585,6 +585,7 @@ mod tests {
             try_outs: super::super::review_classifier::TryOuts::default(),
             reviews: std::sync::Arc::default(),
             runs: crate::review_classifier::runs::Runs::Live(None),
+            alerts: None,
         }
     }
 

swarm-secret-client/src/grafana.rs

@@ -0,0 +1,54 @@
+//! The Grafana agreement: where `swarm-controller`'s token for Grafana's API
+//! lives, and what the object at that path holds.
+//!
+//! `swarm-bao-grafana-controller-token`, on the host that runs Grafana, mints
+//! the token for a Viewer service account and writes it here under Grafana's
+//! own certificate; `swarm-controller` reads it under its own. Those two
+//! principals' stanzas are the only ones in the store that cover this path.
+
+use serde::{Deserialize, Serialize};
+
+use crate::{
+    Error,
+    path::{Kind, principal_prefix},
+};
+
+/// The path holding `swarm-controller`'s Grafana token.
+///
+/// Under [`Kind::Controller`], which no hive's policy reads. The nix half is
+/// `swarm-bao.nix`'s `controllerGrafanaTokenLeaf`.
+///
+/// # Errors
+/// Never in practice: the name segment is a constant. The `Result` is
+/// [`principal_prefix`]'s.
+pub fn controller_token_path() -> Result<String, Error> {
+    let prefix = principal_prefix(Kind::Controller, crate::matrix::CONTROLLER)?;
+    Ok(format!("{prefix}/grafana/token"))
+}
+
+/// What [`controller_token_path`] holds.
+///
+/// No `Debug` derive: `value` reads every alert and dashboard in the swarm's
+/// Grafana, and a derived `Debug` is one `{:?}` away from a log line.
+#[derive(Clone, PartialEq, Eq, Serialize, Deserialize)]
+pub struct ControllerToken {
+    /// The token itself. The minting unit writes it with
+    /// `bao kv put … value=-`, so this name is load-bearing for a writer this
+    /// crate does not control.
+    pub value: String,
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn the_controllers_grafana_token_is_under_the_controllers_prefix() {
+        // The literal is what `swarm-bao.nix` grants Grafana's role and the
+        // controller's, and what the minting unit writes.
+        assert_eq!(
+            controller_token_path().expect("a constant segment is legal"),
+            "swarm/controller/swarm-controller/grafana/token"
+        );
+    }
+}

swarm-secret-client/src/lib.rs

@@ -5,10 +5,10 @@
 //! the rules every path obeys ([`path`]), the translation from this
 //! deployment's environment into a logged-in client ([`client`]), and, per kind
 //! of secret, the path it lives at together with the fields it holds
-//! ([`matrix`], [`queue`], [`mtls`], [`forge`], [`github`], [`acp`], [`ci`],
+//! ([`matrix`], [`queue`], [`mtls`], [`forge`], [`github`], [`grafana`],
-//! [`review_classifier`], [`builder`], [`claude_login`]). Each of those is a
+//! [`acp`], [`ci`], [`review_classifier`], [`builder`], [`claude_login`]).
-//! thing the controller and a hive must say identically, so it is said once
+//! Each of those is a thing the controller and a hive must say identically, so
-//! here.
+//! it is said once here.
 //!
 //! [`policy`] is the same kind of agreement seen from the other side: which of
 //! those paths a given principal's own token may read — a hive's, and an
@@ -31,6 +31,7 @@ pub mod claude_login;
 pub mod client;
 pub mod forge;
 pub mod github;
+pub mod grafana;
 pub mod matrix;
 pub mod mtls;
 pub mod path;