diff --git a/docs/2-handbook/swarm/archetypes.md b/docs/2-handbook/swarm/archetypes.md index c133ec1d7..fe4c325d5 100644 --- a/docs/2-handbook/swarm/archetypes.md +++ b/docs/2-handbook/swarm/archetypes.md @@ -54,48 +54,14 @@ an archetype from any other source name. ## How the list stays current -1. swarm-controller asks the builder for the list on the queue subject - `$SWARM.builder.archetypes` once the queue connects, and every 10 minutes - after that. The controller asks again after a pass that gets no answer or - a failed evaluation: after 10 seconds, then 20, doubling up to 10 minutes, - until one succeeds. -2. The builder evaluates the `meta` of each `hyperhiveArchetypes` entry on - its own hyperhive and nixpkgs sources, and answers with the JSON or the - last lines `nix` printed: +The swarm controller asks the builder for the list when the queue connects and +every 10 minutes after. A builder host redeploy can change it, and the next +pass picks the change up. The controller leaves an archetype that fails its check +off the list and logs it. When no builder answers, the controller keeps the +stored list, and a swarm with no builder shows no archetype select. - ```sh - nix eval --json #hyperhiveArchetypes \ - --apply 'builtins.mapAttrs (_: a: a.meta)' - ``` - -3. The controller checks each archetype the way `POST /api/agents` checks - its settings and stores the result in the `agent-archetypes` KV bucket, - under the key `hyperhive`, when it differs from the stored one. The list - leaves out an archetype that fails the check, and its name and reason go - into the record's `refused` map and the controller's journal. -4. `GET /api/archetypes` answers from that record, and the create form lists - from it. - -A builder host redeploy can change the list. The next pass, within 10 -minutes, picks it up. When no builder answers, the controller keeps the -stored list and logs a warning on each pass. A swarm whose -builder connects after the controller has no list until a retry succeeds. A swarm with no builder enabled shows no -archetype select. - -The stored value: - -```json -{ - "source": "/nix/store/…-hyperhive-flake-source", - "archetypes": { - "reviewer": { - "description": "…", - "permissions": ["messaging", "inbox", "execution"] - } - }, - "refused": { "broken": "limits: …" } -} -``` +The query, the stored record and the retry timing: +[`archetypes.md`](../../4-implementation/swarm/archetypes.md). ## Creating an agent from an archetype diff --git a/docs/2-handbook/swarm/forge.md b/docs/2-handbook/swarm/forge.md index 6f9152754..00d5788cc 100644 --- a/docs/2-handbook/swarm/forge.md +++ b/docs/2-handbook/swarm/forge.md @@ -33,95 +33,34 @@ by a test). swarm-controller mints every agent token with it: | `write:notification` | Mark notifications read via `PATCH /notifications/threads/{id}`. | **`HIVE_TOKEN_SCOPES`** (swarm-controller's `forge::hive_token`, pinned by a -test): `read:repository` alone, for each hive's read-only account (next -section). +test): `read:repository` alone, for each hive's read-only account (see +[Hive and agent accounts](#hive-and-agent-accounts)). --- -## How a hive reads the forge +## Hive and agent accounts -Each hive in swarm-controller's hive directory has its own forge account, -`hive-`. swarm-controller creates it, makes it a member of the `hives` -team in `agent-configs`, and mints one token named `swarm-hive` into the -swarm secret store at `swarm/hives//forge/token`. The team has read -permission on `repo.code` and `repo.pulls` of every repo in the org, -including repos created later. A pass at start and every five minutes -creates what's missing and re-mints a token that's missing or no longer -matches the forge. A pass whose store or forge reads fail queues nothing, so -an outage never rotates every hive's token. - -swarm-controller refuses to create an agent whose name starts with `hive-`: -these accounts own that prefix. An agent that already had such a -name keeps it, so the accounts are also told apart by email. swarm-controller -creates the hive's account with the email `hive-@hives.hyperhive.local` -and refuses an account by that name with any other email: it never adds an -agent's account to the team or mints a token for it. An agent's own forge -account setup refuses the hive's account the same way. - -hive-c0re reads the token with `forge::reader_token()`, under the hive's own -store certificate, every time it reads the forge. A hive whose store holds -none reads nothing. These reads go through it, and none of them needs a local -forge container: - -- cloning and pulling `/knowledge` - -The hive builds nothing, so no nix process reads the forge as the hive's -account. hive-c0re hands the token to each `git` command it runs -(`http.extraHeader`) and keeps no copy on disk. A hive whose store holds no token -reads nothing private from the forge. This also holds on a hive with no forge -container. - -Roll a change to any of this out store host first (its granter writes the -controller's grant on `swarm/hives/+/forge/token`), then the controller host, -then the hives. A hive that starts before its token exists reads nothing, and -builds nothing, until the controller's next pass. - ---- - -## Per-agent forge accounts - -Each agent gets its own Forgejo user and access token. swarm-controller -creates the user when it creates the agent, and mints one token named -`swarm-agent` with the admin API into the swarm secret store at -`swarm/agents//forge-token`. A pass at start and every five minutes -covers every agent with a store identity: it creates a missing forge user and -re-mints any token that's missing or no longer matches the forge; -a rotation deletes the old `swarm-agent` token first, so each agent holds at -most one. The agent fetches the token under its own store certificate into -`/run/hive-agent-forge-token/token` (`nix/agent-modules/forge-token.nix`), -and `hive-forge`, the git credential helper, the `forge_notify` poller and -the avatar sync read it from there, falling back to `/forge-token`, -the file hive-c0re wrote before. hive-c0re no longer creates agent users or -mints agent tokens. swarm-controller deletes the `hyperhive-` -tokens it left behind, for each agent whose current `swarm-agent` token is -live; tokens of other names are never touched. +Each hive has a read-only forge account, `hive-`, which it uses to clone +and pull `/knowledge`. Each agent has its own Forgejo user and access token. The +swarm controller creates both and keeps the tokens in the swarm secret store, +so you set up nothing by hand. A token that goes missing or stops matching the +forge is re-minted within five minutes. Two things live in the `agent-configs` Forgejo organization: -- A config repo per agent (`agent-configs/`). The - agent is a **write collaborator on its own** repo — it can push - config-change branches and open config PRs — but `main` is - branch-protected by swarm-controller: the merge and approval - allowlists are the `operators` team, and the agent can neither push - `main` directly nor self-merge. An operator's merge in the Forgejo UI +- A config repo per agent, `agent-configs/`. The agent can push + config-change branches and open config PRs there, but `main` is + branch-protected: only the `operators` team can merge, and the agent can + neither push `main` nor self-merge. An operator's merge in the Forgejo UI deploys the merged commit (see [config-changes.md § Config changes](../agents/config-changes.md#config-changes)). - hive-c0re pushes nothing to this org and reads nothing from it. - Repos stay private, so an agent can't read another - agent's config. The agent's **persistent meta flake input** is this repo's `main` head, - pinned by sha in `internal/meta` (`agent-.url = -git+/agent-configs/.git?rev=`; see - [config-changes.md § Meta flake](../agents/config-changes.md#meta-flake)). + Repos stay private, so an agent can't read another agent's config. -The `hive-forge` CLI (separate workspace crate, see -[`README.md`](../../README.md) file map) wraps the Forgejo REST API -with the per-agent token; agents call it for issue / PR / comment -ops as if it were a peer. All REST calls across the workspace -(`hive-forge` verbs, hive-c0re provisioning, this poller) go through -the typed `forgejo-api` crate; only non-`/api/v1` web-router routes -(attachment / artifact downloads, log streaming) and the poller's -enrichment fetches of server-provided subject URLs stay on raw -reqwest. +The `hive-forge` CLI wraps the Forgejo REST API with the agent's own token; +agents call it for issue, PR and comment operations. + +How the controller mints, rotates and reads each token, and the team that gives a hive read +access: [`forge-accounts.md`](../../4-implementation/swarm/forge-accounts.md). ## Notifications diff --git a/docs/2-handbook/swarm/knowledge.md b/docs/2-handbook/swarm/knowledge.md index 53e35e073..0d5b01547 100644 --- a/docs/2-handbook/swarm/knowledge.md +++ b/docs/2-handbook/swarm/knowledge.md @@ -21,7 +21,7 @@ read-only — agents never write through it. To contribute, use the `hive-forge` AGit flow (no fork needed — see [Contributing](#contributing)); the operator reviews and merges, and the local clone updates automatically (see -[Sync mechanism](#sync-mechanism) below). +[Staying current](#staying-current) below). @@ -38,91 +38,25 @@ and every few minutes after, and seeds an empty one with a `README.md` containing a contribution guide and a blank table of contents. Add an entry to that ToC each time you create a new document. -## Sync mechanism +## Staying current -hive-c0re maintains the local clone at -`/var/lib/hyperhive/knowledge` via two paths: +Each hive keeps a local clone of the repo and mounts it read-only into every +agent container. The swarm controller holds the single push hook on +`internal/knowledge`, and on any push to main it sends an event that makes +every hive pull. A periodic pull closes the gap for a hive that was offline, so +one hive may briefly show older content than another. When a pull changes +anything, every agent gets a `/knowledge updated` notice. -1. **Swarm event** — the swarm controller holds the single push hook on - `internal/knowledge` (see - [swarm-wide forge webhooks](../../4-implementation/swarm/controller.md#swarm-wide-forge-webhooks)). - On any push to main, including merge commits, it sends an - event to every hive over the swarm queue and each hive runs `git -pull`, so agents see the new content on their next turn. +**Don't add a per-hive hook.** A webhook has one target URL, so a second +registration takes delivery away from whoever registered first. - A hive that's offline when the swarm controller sends the event - doesn't get it on reconnect — the periodic pull below is what closes that gap. One - hive briefly showing older `/knowledge` content than another is - expected, and resolves by itself within the fallback interval. +The clone lives on the host at `/var/lib/hyperhive/knowledge` and survives +hive-c0re restarts and agent destroy or recreate; only a manual delete removes +it. The mount leaves out `.git`, so agents see the documents but not the +history. - **don't add a per-hive hook.** A webhook has exactly one target - URL, so a second registration against the same repo doesn't add a - recipient — it takes delivery away from whoever registered first. - Earlier versions had each hive register its own, and later ones - removed that leftover at startup. A hive upgraded straight past those - versions still holds its old hook: delete any hook on the repo that - points at a hive's own `/webhook/knowledge`. - -2. **Periodic pull** — a background task in `hive-c0re::main` - queues a pull on a fixed cadence as a fallback (webhook missed, c0re - restarted between pushes). The pull is best-effort — a failure - shows as a failed `KnowledgePull` node on the job queue, counts toward - the `knowledge_pull` hive warning, and doesn't affect the rest of the daemon. - -A failed pull of either kind, or the one at boot, also queues a retry after 5 s, -doubling per failure in a row up to 5 min, until a pull succeeds. The -`knowledge_pull` hive warning appears after seven failures in a row, about five -minutes in, so a forge still starting at boot raises nothing. - -Both paths, and the one pull at boot, queue the same `KnowledgePull` job -node. It holds the working tree for its duration, so two pulls never run -over each other — each trigger queues its own pass, and a pass that lands -while another is already queued or running waits its turn, since the -one ahead of it may have fetched before the push. The node runs -`knowledge::pull()`, which also handles the change notice below. - -### Change notice - -When a pull actually moves the local clone's `HEAD` (a real change, -not a no-op — for example the periodic pull finding nothing new), hive-c0re -broadcasts a short notice to every currently registered agent's inbox: -sender `system`, body `[system] /knowledge updated:` followed by a -`git diff --stat ..` summary of what changed (or a generic -"see the repo" fallback if computing the diff itself fails). This is -the same broadcast mechanism used for other hive-wide notices — inbox -message only, no forced wake, and it carries the standard "this was a -broadcast" hint. hive-c0re logs a diff or per-agent send failure -without blocking the pull itself. - -On a hive with no local clone yet, `knowledge::pull()` takes the clone -instead of pulling, so every hive gets `/knowledge` from its first pull, -whether or not it runs the forge. It authenticates with the hive's forge -read token when it has one (see -[forge.md](forge.md#how-a-hive-reads-the-forge)) and clones the public -repo anonymously otherwise. - -## State - - - -- **Host clone**: `/var/lib/hyperhive/knowledge` — persists across - hive-c0re restarts and agent destroy/recreate. Deleted only by - manual operator action. -- **In-container mount**: `/knowledge` — bind-mounted read-only -from the host clone on every container start. Gone when container -is stopped; reappears on next start with the current clone state. - - -The mount deliberately **excludes `.git`**: hive-priv overlays an empty -tmpfs at `/knowledge/.git`, so agents see the documents, not the branches or -history. An older clone also kept a token in `.git/config`; `knowledge::pull()` -rewrites that remote to the plain URL. - -hive-priv adds the overlay only when the host clone has a `.git`. Before the -first clone succeeds, the agent starts with an empty `/knowledge` and no -overlay, because nspawn can't create the mount point inside the read-only -bind. A container that started in that window sees the `.git` once the clone -lands, until its next start. +The sync jobs, retry timing, the change notice and the mount rules: +[`knowledge.md`](../../4-implementation/swarm/knowledge.md). ## Contributing @@ -161,7 +95,7 @@ required. 4. The operator reviews and merges. The webhook fires on merge; every running container sees the updated content within seconds (see - [Sync mechanism](#sync-mechanism)). + [Staying current](#staying-current)). Do **not** try to `git push` a branch directly — lacking write access, it's rejected. The `--agit` flow above is the no-fork path that works diff --git a/docs/2-handbook/swarm/matrix.md b/docs/2-handbook/swarm/matrix.md index e79842269..20faf2584 100644 --- a/docs/2-handbook/swarm/matrix.md +++ b/docs/2-handbook/swarm/matrix.md @@ -115,94 +115,16 @@ leases an address from the gateway's DHCP, resolves through its dnsmasq, and leaves through the bridge's NAT. It reaches the secret store, authelia and federation peers by name. -## Provisioning flow (appservice) +## Accounts - -Registration is closed. **The swarm's appservice** creates accounts: agents -never see an appservice token, and an agent only ever receives its own -`access_token`. - +The homeserver refuses registration. **The swarm's appservice** creates every account: +agents never see an appservice token, and an agent only ever receives its own +`access_token`. `swarm-controller` creates each agent's accounts and one sender +account per hive, `@hive-:`, an ordinary account that +nothing promotes. -The appservice has no URL (`url: null` in its registration), so the -homeserver never calls out to it and there is no service to run. What the -registration buys is an identity the homeserver recognises — which is why -no secret has to be equal on both sides of the wire, and why account -creation doesn't depend on registration being open to anyone who learns -a token. - -1. **`swarm-matrix-ctl appservice render`**, inside the `hive-matrix` - container and before tuwunel starts, fetches the `as_token` and - spec-required `hs_token` from the store and renders the registration to - `/run/swarm-matrix-appservice/swarm.yaml` (mode `0600 root:root`) on the - container's tmpfs, so no token sits on disk. It mints a pair only when - the store answers and holds none. While the store doesn't answer, `render` - fails and retries, and tuwunel stays down. -2. **systemd `LoadCredential=`** copies the registration into - `/run/credentials/tuwunel.service/swarm-appservice.yaml`, owned by - tuwunel's dynamic user with mode `0400`, at service start. The source - file stays `root:root 0600` — no `chown :tuwunel` / `chmod 0640` / - GID-pin gymnastics required. Keeps `DynamicUser = true` + - `PrivateUsers = true` intact. -3. tuwunel's `appservice_dir` points at the credentials directory. It - reads only `.yaml`/`.yml` entries from there, so the sibling - credentials are invisible to it. The `.yaml` suffix on the credential - id is what makes this work. -4. **hive-c0re** holds no appservice token, creates no account and does - not talk to matrix. -5. **`swarm-controller` creates every account**: each agent's, with its - token stored at `swarm/agents//matrix/main` for the agent's - `hive-matrix-daemon` to read as the agent itself, and each hive's - sender account (below). - -A homeserver whose `swarm-matrix-ctl` has no store identity -(`services.hyperhive.deploy.matrix.ctlBaoClientCertFile`) loads no -registration, and nothing creates accounts on it. - -On the store's host `ctlBaoClientCertFile` / `ctlBaoClientKeyFile` default -to the leaf `glue-bao-tls.nix` mints there. A homeserver host without the -store gets no default, even when it holds the hive's own reader -`deploy.bao.clientCertFile`: issue a leaf whose subject is -`deploy.bao.matrixCtlCommonName`, place it, and name it in both options. -Until you do, `swarm-bao-matrix-oidc` fails naming both. - -### The swarm's appservice, and its admin sender - -The registration's id is `swarm`, its sender `@swarm`, and its user -namespace non-exclusive. It's the swarm's identity on the homeserver: -`swarm-controller` creates every agent's account and every hive's sender -account with it. - -The swarm secret store holds the registration's tokens, at -`swarm/controller/swarm-controller/matrix/appservice-registration`. -`swarm-matrix-ctl appservice render` fetches them and renders the -registration tuwunel loads as a credential onto the `hive-matrix` -container's tmpfs, before tuwunel starts. -A render that can't reach the store fails and retries, and tuwunel, which fails to start without the registration, retries until one has landed. -`appservice sync` runs beside tuwunel on a timer. The store's tokens win: -when they differ from the rendered copy, it rewrites it. It -restarts tuwunel whenever the registration is newer than tuwunel's last -start, so the next run retries one that failed after writing, and starts -one that failed because `render` couldn't reach the store. A store that -holds none takes the tokens an earlier version kept in -`/var/lib/swarm-matrix-appservice` with a create-only write, so a running -swarm keeps its value; the files stay, unused. A host with neither files nor stored tokens mints a pair. `sync` then writes the `as_token` -to `swarm/controller/swarm-controller/matrix/appservice-token`. No hive's -policy reaches either path: only matrix-ctl (which holds `create` and -`read` on the registration, and writes the token) and `swarm-controller` -(which reads only the token) hold a grant. - -`@swarm` is the **one** account the homeserver promotes to admin at boot -(`admin_execute`). Only those two principals read its token, which is -the difference from the hive sender below: every hive reads its own -sender's token. `swarm` is a reserved name, so nothing can create an agent -as this account. - -`swarm-controller` mints each agent's account with the device id -`hyperhive-`, and a login on that device replaces its token. The controller therefore reads -the stored token back with `whoami` every five minutes and re-mints only -when it's missing, unknown to the homeserver, or someone else's. An -agent's daemon picks a re-minted token up on its next start, and a timer -restarts it while it's down. +How the registration gets rendered, where its tokens live and how the +controller keeps an agent's token: [`matrix-provisioning.md`](../../4-implementation/swarm/matrix-provisioning.md). ### Operators are homeserver admins @@ -241,101 +163,25 @@ lands on. `services.hyperhive.deploy.swarm-controller.authBridgeUrl` and `services.hyperhive.deploy.swarm-controller.matrixHomeserverUrl` set. -### A hive's sender account, and why it isn't an admin -`@hive-:` is the account a hive acts as. -`swarm-controller` creates it with the swarm's appservice token, for every -hive in its directory. It's an **ordinary account**: nothing promotes it; -the homeserver's `admin_execute` promotes only `@swarm`. +### Settings -**One account per hive.** The localpart carries the hive's name, so a swarm -whose hives share a homeserver gives each of them its own identity: the -homeserver attributes an action to the hive that took it, and revoking one -hive's standing leaves the others alone. +- **Federation** is on at the protocol level, but `trusted_servers = []` keeps + it closed until you list peers in `trustedServers` (see + [Configuration tuning](#configuration-tuning)). +- **Registration** is off. The swarm's appservice still creates accounts, and + the homeserver refuses everyone else. +- **Encryption**: `services.hyperhive.deploy.matrix.allowEncryption` (default + `false`) lets rooms on this homeserver use encryption. Turn it on for + encrypted rooms with external homeservers, or to keep contents opaque to the + homeserver admin. The agents' matrix client reads encrypted rooms either way. +- **Key backup.** Every matrix account an agent has gets a recovery passphrase + in the swarm secret store, so a crypto store lost to `--purge` or a token + reset comes back with its history. -Its access token is the **sender token**. `swarm-controller`, its only -minter, mints it for every hive with the swarm's appservice token and -publishes it to `swarm/hives//matrix/sender-token`, inside the hive's own -read grant (`swarm/hives//*`). Nothing on the hive uses it any more: -hive-c0re used to present it for its Space and invite calls and no longer -makes any. The name says what it authenticates as — an account the -swarm's appservice creates — rather than any privilege level, because it -carries none. - -Promoting a user to homeserver admin and resetting a password both need an -admin **sender**: `!admin …` messages into `#admins:`, and -tuwunel only treats a message as a command when its sender is already an -admin. Of the swarm's own accounts only `@swarm` is a homeserver admin -(operators are too, see -[Operators are homeserver admins](#operators-are-homeserver-admins)), so -both are swarm-level operations. - -
Appservice identity - -`swarm-matrix-ctl appservice render` renders the registration from the -store's copy of the appservice tokens before tuwunel starts, so every -boot has both halves. With the store down or sealed, tuwunel waits for it. - - -- **Access tokens live on the device that minted them.** - `login_with_password` stays on, so the password fallback works - too. -- **The homeserver's `admin_execute` promotes only `@swarm` at boot** - (above); a hive's sender account is never promoted. -- **The value that matters lives at - `swarm/controller/swarm-controller/matrix/appservice-token`.** - `appservice sync` writes it there whenever the store's differs from the - registration's `as_token`, and `swarm-controller` reads it from there. - - -
- -Initial rollout settings: - -- `allow_federation = true` at the protocol level so swarms can be - wired up later by extending `trustedServers` without a homeserver - restart. `trusted_servers = []` keeps it effectively closed - until you list peers. -- `allow_registration = false`. tuwunel checks this flag only for - requests that arrive **without** an appservice token, so the swarm's - appservice still creates accounts and tuwunel refuses everyone else. With no - registration token configured, `allow_registration = true` makes - tuwunel refuse to start unless - `yes_i_am_very_very_sure_…_open_registration_…` is also set. -- `allow_encryption` — server-side E2EE switch, sourced from - `services.hyperhive.deploy.matrix.allowEncryption` (**default `false`**, opt-in). - Off by default because on the swarm's own homeserver the operator - already controls the transport; turn it on for encrypted rooms on - external / federated homeservers or to keep contents opaque to the - homeserver admin. **The agent matrix client always supports decryption - regardless of this flag** — it uses the `e2e-encryption` feature of - `matrix-sdk` so it can read encrypted rooms it's invited to even when - this homeserver doesn't permit room encryption. matrix-sdk stores - crypto keys in the per-agent sqlite store under the state dir; they persist across - restarts. `read_room` decrypts via - `room.messages()` — UTD events surface as `event_type = - "m.room.encrypted"` with `body = "[unable to decrypt]"`. - The daemon bootstraps cross-signing on the bearer token alone, which - works where the homeserver skips re-auth for a first key upload - (MSC3967, as tuwunel does). -- **Key backup.** `swarm-controller` mints a random recovery passphrase - for every matrix account an agent has, at - `swarm/agents//matrix-recovery/` in the swarm secret - store, and never replaces it. When `hive-matrix-daemon` brings an - account up it reads that passphrase: with no backup on the account's - homeserver it turns on server-side key backup and secret storage - (log line `matrix recovery enabled`), unless the account already has - secret storage, which it never replaces (a `WARN` instead). With a - backup there it recovers from it (`recovered from the server-side key backup`), which downloads the - room keys and signs the new device with the account's existing - cross-signing identity. A crypto store lost to `--purge` or to a - primary-token reset therefore comes back with its history. With no passphrase - stored yet the account comes up without backup and picks it up on the - next daemon start. A failure is a `WARN` and never stops the daemon. - -The module's retired hive-side code, its evaluation assertions and the -fluffychat-web build fixes are in the -[implementation notes](../../4-implementation/networking/matrix-homeserver.md). +The details, and the retired hive-side code, are in +[`matrix-provisioning.md`](../../4-implementation/swarm/matrix-provisioning.md) +and the [implementation notes](../../4-implementation/networking/matrix-homeserver.md). ## Configuration tuning diff --git a/docs/2-handbook/swarm/sso.md b/docs/2-handbook/swarm/sso.md index d00e7d329..50e8dc871 100644 --- a/docs/2-handbook/swarm/sso.md +++ b/docs/2-handbook/swarm/sso.md @@ -104,186 +104,39 @@ this password is stored nowhere — record it now Authelia's file store has no SMTP notifier, so this is the only way a human account gets a new password once an operator forgets the old one. -## What secrets exist, and where each one lives +## Logging in to the forge and matrix -[`credentials-reference.md`](credentials-reference.md) tabulates every secret in the swarm, with its -generator and its path, in one place — including authelia's own keys -(session, JWT, storage-encryption, OIDC HMAC, OIDC issuer) and the two -halves of each client secret. That page's two rules — a secret is always -a path, never a value, and the generator and the reader typically live in -different containers — are why the client secret's plaintext half needs -the delivery step below and the rest of authelia's keys don't. +- **A person's first forge login creates their forge account**, named after + their authelia username, provided the subject has an email. It starts as an + ordinary user; `swarmctl forge make-admin ` makes it a site admin. A + name that already has a local forge account isn't handed over: forgejo asks + for that account's own password first. Agents, `core` and `swarm-controller` + all have local accounts. +- **A member of `admins` becomes a matrix homeserver admin** once their first + matrix login has created the account. `swarm-controller` promotes them within + five minutes, and never demotes. See + [`matrix.md`](matrix.md#operators-are-homeserver-admins). -## Getting the plaintext to the relying party +You place no client secret by hand. Authelia mints each one at its first boot +and the host-side units deliver it to the forge and the homeserver, retrying +until it lands, so there is no start order to get right. A forge on another host +than the store needs the forge publisher's store leaf, in +`deploy.bao.forgePublisherClientCertFile` and `forgePublisherClientKeyFile`. +Without it the forge has no login source, and the unit fails naming both options. -Three cases, and they're genuinely different mechanisms rather than one -mechanism with flags. - -### 1. All-local — one host runs both - -Nothing to configure at all. authelia mints each client's secret at its -first boot, and the mint in its container creates it in the swarm -secret store at `swarm/services//oidc/client`, keeping only the digest on disk. A host-side unit -then reads it from the store and writes it into the service's container: -`swarm-bao-forge-oidc` for the forge, under the forge publisher's store -leaf, into `/run/hive-forge-oidc/.secret` on the host, which -`forgejo-sso-source` loads as a credential; `swarm-bao-matrix-oidc` for matrix, under -matrix-ctl's. A unit that can't read the secret yet fails and retries -every 30 seconds for a day, so there is no start order to get right. The -service's own module contributes its client entry — callback URL -included — to authelia's client list. - -Each relying service's own glue module builds its callback once, and -both authelia's client list and the service's own OIDC client config -read that same value, so the redirect URI authelia accepts and the one -the service actually sends can't drift apart. A mismatch there is a -rejected login with no error text worth reading. - - - -⚠️ nixos-container refuses to start a container whose bind source is -missing, and this secret doesn't exist until authelia's first boot has run -and the mint has stored it. The forge therefore binds the **directory** -`/run/hive-forge-oidc`, which a tmpfiles rule creates at boot, never the -file. The directory is tmpfs, so the secret is gone after a reboot until -the fetch runs again; forgejo keeps it in its login-source row, so logins -still work meanwhile. Matrix's secret is written into the container's state -directory instead. - - - -### 2. Swarm-managed services - -The controller side owns provisioning: `swarmctl` writes both halves, the -same way it already owns authelia's user store (`users.yml`, read and -written in place). - -### 3. A hive elsewhere - -The forge's host needn't be the store's or authelia's host, and the -operator places no secret file: `swarm-bao-forge-oidc` runs on the -forge's host as in the all-local case and fetches the secret over the -store's address. What that host needs is the forge publisher's store -leaf, issued from the store's CA and copied over from the store's host: - -```nix -services.hyperhive = { - swarm.authelia.url = "https://auth.example.com"; - deploy.bao = { - forgePublisherClientCertFile = "/var/lib/swarm-bao-pki/forge-publisher.pem"; - forgePublisherClientKeyFile = "/var/lib/swarm-bao-pki/forge-publisher-key.pem"; - }; -}; -``` - -**A forge host without that leaf still evaluates.** `swarm-bao-forge-oidc` -renders and fails at start, naming both options, and the forge has no -login source until an operator places the leaf. Where the store runs on -the same host, `glue-bao-tls.nix` defaults both options to the leaf it -mints. - -## Where each relying party differs - -The registration half is identical; what each service does with the -result isn't. - -| | forge | matrix | -| ------------------------- | ---------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | -| how it learns the config | a oneshot calls `forgejo admin auth`, writing a login-source row into its database | tuwunel reads a `[[global.identity_provider]]` entry from its config file | -| how it reads the secret | a path inside its container | the same path, handed on by `LoadCredential` | -| callback URL | `/user/oauth2//callback` | `/_matrix/client/unstable/login/sso/callback/`, a shape tuwunel fixes rather than accepts | -| cost of a malformed entry | the login source is missing | the homeserver can refuse to start | - -Four consequences worth stating plainly: - -- **A person's first forge login creates their forge account**, named - after authelia's `preferred_username` (`[oauth2_client]` in the forge - module), provided the subject has an email. It starts as an ordinary user; `swarmctl forge make-admin -` makes it a site admin. A name that already has a local forge - account doesn't get it handed over: forgejo asks for that account's - own password first (`ACCOUNT_LINKING = login`). Agents, `core` and - `swarm-controller` all have local accounts, and `swarmctl user add` - refuses none of those names. - -- **A member of `admins` becomes a matrix homeserver admin** once their - first matrix login has created the account: `swarm-controller` promotes - them within five minutes, and never demotes. See - [`2-handbook/swarm/matrix.md`](matrix.md#operators-are-homeserver-admins). - -- **tuwunel re-reads its secret file on every OAuth exchange**, not only - at startup, and its own sandboxing hides most paths from it. It gets the - file through `LoadCredential` for the same reason the appservice - registration does — that keeps `DynamicUser` and `PrivateUsers` intact, - with no host-side ownership arrangement to maintain. -- **Matrix SSO lives inside the homeserver.** The client-server API is - spoken by non-browser clients holding matrix access tokens — every - agent's own daemon — as well as by federation, so the homeserver - serves `/_matrix/` directly and authenticates it itself. The - forward-auth vhosts protect browser surfaces; this isn't one of them. +Where each secret lives, the delivery mechanics and how forge and matrix +differ: [`sso.md`](../../4-implementation/swarm/sso.md). ## Machine clients -Everything above is a person in a browser. A swarm also has callers that -hold no session and follow no redirect: the telemetry collector, the -queue's auth-callout responder, and each hive. - -**One client per hive, not one per service.** A hive's identity belongs to -the directory rather than to whichever service happens to consume it, so a -hive holds a single OIDC client — `` — and mints a -different token per service from it. The alternative, letting each -consuming subsystem declare its own list, collides on the same client id -the moment a second consumer appears. - -**The audience is that client id.** A swarm service that has to tell hives -apart needs one name both sides already agree on, and the client id is -already that name. A parallel per-hive naming scheme would be a second -thing to keep in step, and it drifts silently — a mismatch presents as a -valid token refused at the target, which reads like a broken credential -rather than a broken name. - -**Authelia signs tokens (`RS256`), not opaque ones**, because a resource server -that can't call the provider back is a real case here: the telemetry -collector verifies offline against `/jwks.json`, and an opaque token gives -it nothing to verify. The queue's responder introspects instead — a -different question asked of the same token, and the reason both -`/api/oidc/introspection` and `/jwks.json` have to stay reachable. - -### Machine callers must fail closed - -⚠️ **An error page that answers `200` is a security bug, not a cosmetic -one.** The browser surface intercepts upstream errors and serves a -friendly "SSO is unavailable" page; that page is a file, so it returns -`200`. Any machine caller routed through it receives a success carrying -HTML instead of the failure that actually happened: - -- `/api/authz/auth-request` — nginx `auth_request` treats **any 2xx as - success**, so a down provider means _access granted_ -- `/api/oidc/introspection` — a token check that answers `200` -- `/api/oidc/token`, `/.well-known/openid-configuration` — a client - parsing an error page as its JSON document - -The gateway routes authelia's `/api/` and `/.well-known/` prefixes, and -the bridge's `/bridge/`, **without** error interception. The split is by -_audience_, not by an enumerated path list: a human gets the page, every -machine caller gets the status. -Enumerating endpoints individually would leave the next one added -silently intercepted. - -The same shape bites any machine route behind a browser-shaped gate: the -caller follows a `302` to a login page, the login page answers `200`, -and the caller reports success while nothing happened. Log ingest hit exactly this -and lost eleven hours of delivery in silence. - -**Checking it, if you change this routing.** Point the vhost at a dead -upstream and compare three requests, not one: - -1. through `/` — must still serve the friendly page -2. through `/api/` — must deny -3. a direct dial to authelia — must match what (2) did - -All three matter. A change that silently deleted the browser page would -pass a deny-only check, and one that quietly stopped denying would pass a -page-only check. The person who introduced the split verified it this way. +Everything above is a person in a browser. A swarm also has callers that hold +no session and follow no redirect: the telemetry collector, the queue's +auth-callout responder, and each hive. A hive holds a single OIDC client, +``, and mints a different token per service from it. +The gateway answers machine callers with the real status rather than the +friendly "SSO is unavailable" page, so a down provider denies access instead of +granting it. The reasoning and how to check it after a routing change: +[`sso.md`](../../4-implementation/swarm/sso.md#machine-callers-must-fail-closed). ## Operator CLI login diff --git a/docs/4-implementation/swarm/archetypes.md b/docs/4-implementation/swarm/archetypes.md new file mode 100644 index 000000000..5be6bc021 --- /dev/null +++ b/docs/4-implementation/swarm/archetypes.md @@ -0,0 +1,50 @@ +# Agent archetypes: how the list stays current + +How swarm-controller learns which archetypes the create form offers, and what +it stores. The operator's view is +[the archetypes page](../../2-handbook/swarm/archetypes.md). + +## The refresh + +1. swarm-controller asks the builder for the list on the queue subject + `$SWARM.builder.archetypes` once the queue connects, and every 10 minutes + after that. The controller asks again after a pass that gets no answer or + a failed evaluation: after 10 seconds, then 20, doubling up to 10 minutes, + until one succeeds. +2. The builder evaluates the `meta` of each `hyperhiveArchetypes` entry on + its own hyperhive and nixpkgs sources, and answers with the JSON or the + last lines `nix` printed: + + ```sh + nix eval --json #hyperhiveArchetypes \ + --apply 'builtins.mapAttrs (_: a: a.meta)' + ``` + +3. The controller checks each archetype the way `POST /api/agents` checks + its settings and stores the result in the `agent-archetypes` KV bucket, + under the key `hyperhive`, when it differs from the stored one. The list + leaves out an archetype that fails the check, and its name and reason go + into the record's `refused` map and the controller's journal. +4. `GET /api/archetypes` answers from that record, and the create form lists + from it. + +A builder host redeploy can change the list. The next pass, within 10 +minutes, picks it up. When no builder answers, the controller keeps the +stored list and logs a warning on each pass. A swarm whose +builder connects after the controller has no list until a retry succeeds. A swarm with no builder enabled shows no +archetype select. + +The stored value: + +```json +{ + "source": "/nix/store/…-hyperhive-flake-source", + "archetypes": { + "reviewer": { + "description": "…", + "permissions": ["messaging", "inbox", "execution"] + } + }, + "refused": { "broken": "limits: …" } +} +``` diff --git a/docs/4-implementation/swarm/forge-accounts.md b/docs/4-implementation/swarm/forge-accounts.md new file mode 100644 index 000000000..278c7fde7 --- /dev/null +++ b/docs/4-implementation/swarm/forge-accounts.md @@ -0,0 +1,88 @@ +# Forge accounts: hives and agents + +How swarm-controller gives each hive a read-only forge account and each agent +its own user and token, and where each token lives. The operator's view of the +forge is [the forge page](../../2-handbook/swarm/forge.md). + +## How a hive reads the forge + +Each hive in swarm-controller's hive directory has its own forge account, +`hive-`. swarm-controller creates it, makes it a member of the `hives` +team in `agent-configs`, and mints one token named `swarm-hive` into the +swarm secret store at `swarm/hives//forge/token`. The team has read +permission on `repo.code` and `repo.pulls` of every repo in the org, +including repos created later. A pass at start and every five minutes +creates what's missing and re-mints a token that's missing or no longer +matches the forge. A pass whose store or forge reads fail queues nothing, so +an outage never rotates every hive's token. + +swarm-controller refuses to create an agent whose name starts with `hive-`: +these accounts own that prefix. An agent that already had such a +name keeps it, so the accounts are also told apart by email. swarm-controller +creates the hive's account with the email `hive-@hives.hyperhive.local` +and refuses an account by that name with any other email: it never adds an +agent's account to the team or mints a token for it. An agent's own forge +account setup refuses the hive's account the same way. + +hive-c0re reads the token with `forge::reader_token()`, under the hive's own +store certificate, every time it reads the forge. A hive whose store holds +none reads nothing. These reads go through it, and none of them needs a local +forge container: + +- cloning and pulling `/knowledge` + +The hive builds nothing, so no nix process reads the forge as the hive's +account. hive-c0re hands the token to each `git` command it runs +(`http.extraHeader`) and keeps no copy on disk. A hive whose store holds no token +reads nothing private from the forge. This also holds on a hive with no forge +container. + +Roll a change to any of this out store host first (its granter writes the +controller's grant on `swarm/hives/+/forge/token`), then the controller host, +then the hives. A hive that starts before its token exists reads nothing, and +builds nothing, until the controller's next pass. + +## Per-agent forge accounts + +Each agent gets its own Forgejo user and access token. swarm-controller +creates the user when it creates the agent, and mints one token named +`swarm-agent` with the admin API into the swarm secret store at +`swarm/agents//forge-token`. A pass at start and every five minutes +covers every agent with a store identity: it creates a missing forge user and +re-mints any token that's missing or no longer matches the forge; +a rotation deletes the old `swarm-agent` token first, so each agent holds at +most one. The agent fetches the token under its own store certificate into +`/run/hive-agent-forge-token/token` (`nix/agent-modules/forge-token.nix`), +and `hive-forge`, the git credential helper, the `forge_notify` poller and +the avatar sync read it from there, falling back to `/forge-token`, +the file hive-c0re wrote before. hive-c0re no longer creates agent users or +mints agent tokens. swarm-controller deletes the `hyperhive-` +tokens it left behind, for each agent whose current `swarm-agent` token is +live; tokens of other names are never touched. + +Two things live in the `agent-configs` Forgejo organization: + +- A config repo per agent (`agent-configs/`). The + agent is a **write collaborator on its own** repo — it can push + config-change branches and open config PRs — but `main` is + branch-protected by swarm-controller: the merge and approval + allowlists are the `operators` team, and the agent can neither push + `main` directly nor self-merge. An operator's merge in the Forgejo UI + deploys the merged commit (see + [config-changes.md § Config changes](../../2-handbook/agents/config-changes.md#config-changes)). + hive-c0re pushes nothing to this org and reads nothing from it. + Repos stay private, so an agent can't read another + agent's config. The agent's **persistent meta flake input** is this repo's `main` head, + pinned by sha in `internal/meta` (`agent-.url = +git+/agent-configs/.git?rev=`; see + [config-changes.md § Meta flake](../../2-handbook/agents/config-changes.md#meta-flake)). + +The `hive-forge` CLI (separate workspace crate, see +[`README.md`](../../README.md) file map) wraps the Forgejo REST API +with the per-agent token; agents call it for issue / PR / comment +ops as if it were a peer. All REST calls across the workspace +(`hive-forge` verbs, hive-c0re provisioning, this poller) go through +the typed `forgejo-api` crate; only non-`/api/v1` web-router routes +(attachment / artifact downloads, log streaming) and the poller's +enrichment fetches of server-provided subject URLs stay on raw +reqwest. diff --git a/docs/4-implementation/swarm/knowledge.md b/docs/4-implementation/swarm/knowledge.md new file mode 100644 index 000000000..9d8e7cc3e --- /dev/null +++ b/docs/4-implementation/swarm/knowledge.md @@ -0,0 +1,90 @@ +# Knowledge repo: sync, change notice and state + +How a hive keeps its clone of `internal/knowledge` current, tells agents when it +changes, and mounts it into each container. The operator's view is +[the knowledge page](../../2-handbook/swarm/knowledge.md). + +## Sync mechanism + +hive-c0re maintains the local clone at +`/var/lib/hyperhive/knowledge` via two paths: + +1. **Swarm event** — the swarm controller holds the single push hook on + `internal/knowledge` (see + [swarm-wide forge webhooks](controller.md#swarm-wide-forge-webhooks)). On any push to main, including merge commits, it sends an + event to every hive over the swarm queue and each hive runs `git +pull`, so agents see the new content on their next turn. + + A hive that's offline when the swarm controller sends the event + doesn't get it on reconnect — the periodic pull below is what closes that gap. One + hive briefly showing older `/knowledge` content than another is + expected, and resolves by itself within the fallback interval. + + **don't add a per-hive hook.** A webhook has exactly one target + URL, so a second registration against the same repo doesn't add a + recipient — it takes delivery away from whoever registered first. + Earlier versions had each hive register its own, and later ones + removed that leftover at startup. A hive upgraded straight past those + versions still holds its old hook: delete any hook on the repo that + points at a hive's own `/webhook/knowledge`. + +2. **Periodic pull** — a background task in `hive-c0re::main` + queues a pull on a fixed cadence as a fallback (webhook missed, c0re + restarted between pushes). The pull is best-effort — a failure + shows as a failed `KnowledgePull` node on the job queue, counts toward + the `knowledge_pull` hive warning, and doesn't affect the rest of the daemon. + +A failed pull of either kind, or the one at boot, also queues a retry after 5 s, +doubling per failure in a row up to 5 min, until a pull succeeds. The +`knowledge_pull` hive warning appears after seven failures in a row, about five +minutes in, so a forge still starting at boot raises nothing. + +Both paths, and the one pull at boot, queue the same `KnowledgePull` job +node. It holds the working tree for its duration, so two pulls never run +over each other — each trigger queues its own pass, and a pass that lands +while another is already queued or running waits its turn, since the +one ahead of it may have fetched before the push. The node runs +`knowledge::pull()`, which also handles the change notice below. + +### Change notice + +When a pull actually moves the local clone's `HEAD` (a real change, +not a no-op — for example the periodic pull finding nothing new), hive-c0re +broadcasts a short notice to every currently registered agent's inbox: +sender `system`, body `[system] /knowledge updated:` followed by a +`git diff --stat ..` summary of what changed (or a generic +"see the repo" fallback if computing the diff itself fails). This is +the same broadcast mechanism used for other hive-wide notices — inbox +message only, no forced wake, and it carries the standard "this was a +broadcast" hint. hive-c0re logs a diff or per-agent send failure +without blocking the pull itself. + +On a hive with no local clone yet, `knowledge::pull()` takes the clone +instead of pulling, so every hive gets `/knowledge` from its first pull, +whether or not it runs the forge. It authenticates with the hive's forge +read token when it has one (see +[forge-accounts.md](forge-accounts.md#how-a-hive-reads-the-forge)) and clones the public +repo anonymously otherwise. + +## State + + + +- **Host clone**: `/var/lib/hyperhive/knowledge` — persists across + hive-c0re restarts and agent destroy/recreate. Deleted only by + manual operator action. +- **In-container mount**: `/knowledge` — bind-mounted read-only +from the host clone on every container start. Gone when container +is stopped; reappears on next start with the current clone state. + + +The mount deliberately **excludes `.git`**: hive-priv overlays an empty +tmpfs at `/knowledge/.git`, so agents see the documents, not the branches or +history. An older clone also kept a token in `.git/config`; `knowledge::pull()` +rewrites that remote to the plain URL. + +hive-priv adds the overlay only when the host clone has a `.git`. Before the +first clone succeeds, the agent starts with an empty `/knowledge` and no +overlay, because nspawn can't create the mount point inside the read-only +bind. A container that started in that window sees the `.git` once the clone +lands, until its next start. diff --git a/docs/4-implementation/swarm/matrix-provisioning.md b/docs/4-implementation/swarm/matrix-provisioning.md new file mode 100644 index 000000000..af2de73ba --- /dev/null +++ b/docs/4-implementation/swarm/matrix-provisioning.md @@ -0,0 +1,193 @@ +# Matrix homeserver: account provisioning + +How the swarm's appservice creates accounts on the homeserver, where its tokens +live, and the settings the homeserver starts with. The operator's view is +[the matrix page](../../2-handbook/swarm/matrix.md). + +## Provisioning flow (appservice) + + + +Registration is closed. **The swarm's appservice** creates accounts: agents +never see an appservice token, and an agent only ever receives its own +`access_token`. + + + +The appservice has no URL (`url: null` in its registration), so the +homeserver never calls out to it and there is no service to run. What the +registration buys is an identity the homeserver recognises — which is why +no secret has to be equal on both sides of the wire, and why account +creation doesn't depend on registration being open to anyone who learns +a token. + +1. **`swarm-matrix-ctl appservice render`**, inside the `hive-matrix` + container and before tuwunel starts, fetches the `as_token` and + spec-required `hs_token` from the store and renders the registration to + `/run/swarm-matrix-appservice/swarm.yaml` (mode `0600 root:root`) on the + container's tmpfs, so no token sits on disk. It mints a pair only when + the store answers and holds none. While the store doesn't answer, `render` + fails and retries, and tuwunel stays down. +2. **systemd `LoadCredential=`** copies the registration into + `/run/credentials/tuwunel.service/swarm-appservice.yaml`, owned by + tuwunel's dynamic user with mode `0400`, at service start. The source + file stays `root:root 0600` — no `chown :tuwunel` / `chmod 0640` / + GID-pin gymnastics required. Keeps `DynamicUser = true` + + `PrivateUsers = true` intact. +3. tuwunel's `appservice_dir` points at the credentials directory. It + reads only `.yaml`/`.yml` entries from there, so the sibling + credentials are invisible to it. The `.yaml` suffix on the credential + id is what makes this work. +4. **hive-c0re** holds no appservice token, creates no account and does + not talk to matrix. +5. **`swarm-controller` creates every account**: each agent's, with its + token stored at `swarm/agents//matrix/main` for the agent's + `hive-matrix-daemon` to read as the agent itself, and each hive's + sender account (below). + +A homeserver whose `swarm-matrix-ctl` has no store identity +(`services.hyperhive.deploy.matrix.ctlBaoClientCertFile`) loads no +registration, and nothing creates accounts on it. + +On the store's host `ctlBaoClientCertFile` / `ctlBaoClientKeyFile` default +to the leaf `glue-bao-tls.nix` mints there. A homeserver host without the +store gets no default, even when it holds the hive's own reader +`deploy.bao.clientCertFile`: issue a leaf whose subject is +`deploy.bao.matrixCtlCommonName`, place it, and name it in both options. +Until you do, `swarm-bao-matrix-oidc` fails naming both. + +### The swarm's appservice, and its admin sender + +The registration's id is `swarm`, its sender `@swarm`, and its user +namespace non-exclusive. It's the swarm's identity on the homeserver: +`swarm-controller` creates every agent's account and every hive's sender +account with it. + +The swarm secret store holds the registration's tokens, at +`swarm/controller/swarm-controller/matrix/appservice-registration`. +`swarm-matrix-ctl appservice render` fetches them and renders the +registration tuwunel loads as a credential onto the `hive-matrix` +container's tmpfs, before tuwunel starts. +A render that can't reach the store fails and retries, and tuwunel, which fails to start without the registration, retries until one has landed. +`appservice sync` runs beside tuwunel on a timer. The store's tokens win: +when they differ from the rendered copy, it rewrites it. It +restarts tuwunel whenever the registration is newer than tuwunel's last +start, so the next run retries one that failed after writing, and starts +one that failed because `render` couldn't reach the store. A store that +holds none takes the tokens an earlier version kept in +`/var/lib/swarm-matrix-appservice` with a create-only write, so a running +swarm keeps its value; the files stay, unused. A host with neither files nor stored tokens mints a pair. `sync` then writes the `as_token` +to `swarm/controller/swarm-controller/matrix/appservice-token`. No hive's +policy reaches either path: only matrix-ctl (which holds `create` and +`read` on the registration, and writes the token) and `swarm-controller` +(which reads only the token) hold a grant. + +`@swarm` is the **one** account the homeserver promotes to admin at boot +(`admin_execute`). Only those two principals read its token, which is +the difference from the hive sender below: every hive reads its own +sender's token. `swarm` is a reserved name, so nothing can create an agent +as this account. + +`swarm-controller` mints each agent's account with the device id +`hyperhive-`, and a login on that device replaces its token. The controller therefore reads +the stored token back with `whoami` every five minutes and re-mints only +when it's missing, unknown to the homeserver, or someone else's. An +agent's daemon picks a re-minted token up on its next start, and a timer +restarts it while it's down. + +### A hive's sender account, and why it isn't an admin + +`@hive-:` is the account a hive acts as. +`swarm-controller` creates it with the swarm's appservice token, for every +hive in its directory. It's an **ordinary account**: nothing promotes it; +the homeserver's `admin_execute` promotes only `@swarm`. + +**One account per hive.** The localpart carries the hive's name, so a swarm +whose hives share a homeserver gives each of them its own identity: the +homeserver attributes an action to the hive that took it, and revoking one +hive's standing leaves the others alone. + +Its access token is the **sender token**. `swarm-controller`, its only +minter, mints it for every hive with the swarm's appservice token and +publishes it to `swarm/hives//matrix/sender-token`, inside the hive's own +read grant (`swarm/hives//*`). Nothing on the hive uses it any more: +hive-c0re used to present it for its Space and invite calls and no longer +makes any. The name says what it authenticates as — an account the +swarm's appservice creates — rather than any privilege level, because it +carries none. + +Promoting a user to homeserver admin and resetting a password both need an +admin **sender**: `!admin …` messages into `#admins:`, and +tuwunel only treats a message as a command when its sender is already an +admin. Of the swarm's own accounts only `@swarm` is a homeserver admin +(operators are too, see +[Operators are homeserver admins](../../2-handbook/swarm/matrix.md#operators-are-homeserver-admins)), so +both are swarm-level operations. + +
Appservice identity + +`swarm-matrix-ctl appservice render` renders the registration from the +store's copy of the appservice tokens before tuwunel starts, so every +boot has both halves. With the store down or sealed, tuwunel waits for it. + + + +- **Access tokens live on the device that minted them.** + `login_with_password` stays on, so the password fallback works + too. +- **The homeserver's `admin_execute` promotes only `@swarm` at boot** + (above); a hive's sender account is never promoted. +- **The value that matters lives at +`swarm/controller/swarm-controller/matrix/appservice-token`.** +`appservice sync` writes it there whenever the store's differs from the +registration's `as_token`, and `swarm-controller` reads it from there. + + +
+ +## Rollout settings + +- `allow_federation = true` at the protocol level so swarms can be + wired up later by extending `trustedServers` without a homeserver + restart. `trusted_servers = []` keeps it effectively closed + until you list peers. +- `allow_registration = false`. tuwunel checks this flag only for + requests that arrive **without** an appservice token, so the swarm's + appservice still creates accounts and tuwunel refuses everyone else. With no + registration token configured, `allow_registration = true` makes + tuwunel refuse to start unless + `yes_i_am_very_very_sure_…_open_registration_…` is also set. +- `allow_encryption` — server-side E2EE switch, sourced from + `services.hyperhive.deploy.matrix.allowEncryption` (**default `false`**, opt-in). + Off by default because on the swarm's own homeserver the operator + already controls the transport; turn it on for encrypted rooms on + external / federated homeservers or to keep contents opaque to the + homeserver admin. **The agent matrix client always supports decryption + regardless of this flag** — it uses the `e2e-encryption` feature of + `matrix-sdk` so it can read encrypted rooms it's invited to even when + this homeserver doesn't permit room encryption. matrix-sdk stores + crypto keys in the per-agent sqlite store under the state dir; they persist across + restarts. `read_room` decrypts via + `room.messages()` — UTD events surface as `event_type = +"m.room.encrypted"` with `body = "[unable to decrypt]"`. + The daemon bootstraps cross-signing on the bearer token alone, which + works where the homeserver skips re-auth for a first key upload + (MSC3967, as tuwunel does). +- **Key backup.** `swarm-controller` mints a random recovery passphrase + for every matrix account an agent has, at + `swarm/agents//matrix-recovery/` in the swarm secret + store, and never replaces it. When `hive-matrix-daemon` brings an + account up it reads that passphrase: with no backup on the account's + homeserver it turns on server-side key backup and secret storage + (log line `matrix recovery enabled`), unless the account already has + secret storage, which it never replaces (a `WARN` instead). With a + backup there it recovers from it (`recovered from the server-side key backup`), which downloads the + room keys and signs the new device with the account's existing + cross-signing identity. A crypto store lost to `--purge` or to a + primary-token reset therefore comes back with its history. With no passphrase + stored yet the account comes up without backup and picks it up on the + next daemon start. A failure is a `WARN` and never stops the daemon. + +The module's retired hive-side code, its evaluation assertions and the +fluffychat-web build fixes are in the +[implementation notes](../../4-implementation/networking/matrix-homeserver.md). diff --git a/docs/4-implementation/swarm/sso.md b/docs/4-implementation/swarm/sso.md new file mode 100644 index 000000000..f954b3a5b --- /dev/null +++ b/docs/4-implementation/swarm/sso.md @@ -0,0 +1,186 @@ +# SSO: relying-party secrets and machine callers + +How authelia's client secrets reach the forge and the homeserver, how the two +differ, and why machine callers must fail closed. The operator's view is +[the SSO page](../../2-handbook/swarm/sso.md). + +## What secrets exist, and where each one lives + +[`credentials-reference.md`](../../2-handbook/swarm/credentials-reference.md) tabulates every secret in the swarm, with its +generator and its path, in one place — including authelia's own keys +(session, JWT, storage-encryption, OIDC HMAC, OIDC issuer) and the two +halves of each client secret. That page's two rules — a secret is always +a path, never a value, and the generator and the reader typically live in +different containers — are why the client secret's plaintext half needs +the delivery step below and the rest of authelia's keys don't. + +## Getting the plaintext to the relying party + +Three cases, and they're genuinely different mechanisms rather than one +mechanism with flags. + +### 1. All-local — one host runs both + +Nothing to configure at all. authelia mints each client's secret at its +first boot, and the mint in its container creates it in the swarm +secret store at `swarm/services//oidc/client`, keeping only the digest on disk. A host-side unit +then reads it from the store and writes it into the service's container: +`swarm-bao-forge-oidc` for the forge, under the forge publisher's store +leaf, into `/run/hive-forge-oidc/.secret` on the host, which +`forgejo-sso-source` loads as a credential; `swarm-bao-matrix-oidc` for matrix, under +matrix-ctl's. A unit that can't read the secret yet fails and retries +every 30 seconds for a day, so there is no start order to get right. The +service's own module contributes its client entry — callback URL +included — to authelia's client list. + +Each relying service's own glue module builds its callback once, and +both authelia's client list and the service's own OIDC client config +read that same value, so the redirect URI authelia accepts and the one +the service actually sends can't drift apart. A mismatch there is a +rejected login with no error text worth reading. + + + +⚠️ nixos-container refuses to start a container whose bind source is +missing, and this secret doesn't exist until authelia's first boot has run +and the mint has stored it. The forge therefore binds the **directory** +`/run/hive-forge-oidc`, which a tmpfiles rule creates at boot, never the +file. The directory is tmpfs, so the secret is gone after a reboot until +the fetch runs again; forgejo keeps it in its login-source row, so logins +still work meanwhile. Matrix's secret is written into the container's state +directory instead. + + + +### 2. Swarm-managed services + +The controller side owns provisioning: `swarmctl` writes both halves, the +same way it already owns authelia's user store (`users.yml`, read and +written in place). + +### 3. A hive elsewhere + +The forge's host needn't be the store's or authelia's host, and the +operator places no secret file: `swarm-bao-forge-oidc` runs on the +forge's host as in the all-local case and fetches the secret over the +store's address. What that host needs is the forge publisher's store +leaf, issued from the store's CA and copied over from the store's host: + +```nix +services.hyperhive = { + swarm.authelia.url = "https://auth.example.com"; + deploy.bao = { + forgePublisherClientCertFile = "/var/lib/swarm-bao-pki/forge-publisher.pem"; + forgePublisherClientKeyFile = "/var/lib/swarm-bao-pki/forge-publisher-key.pem"; + }; +}; +``` + +**A forge host without that leaf still evaluates.** `swarm-bao-forge-oidc` +renders and fails at start, naming both options, and the forge has no +login source until an operator places the leaf. Where the store runs on +the same host, `glue-bao-tls.nix` defaults both options to the leaf it +mints. + +## Where each relying party differs + +The registration half is identical; what each service does with the +result isn't. + +| | forge | matrix | +| ------------------------- | ---------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | +| how it learns the config | a oneshot calls `forgejo admin auth`, writing a login-source row into its database | tuwunel reads a `[[global.identity_provider]]` entry from its config file | +| how it reads the secret | a path inside its container | the same path, handed on by `LoadCredential` | +| callback URL | `/user/oauth2//callback` | `/_matrix/client/unstable/login/sso/callback/`, a shape tuwunel fixes rather than accepts | +| cost of a malformed entry | the login source is missing | the homeserver can refuse to start | + +Four consequences worth stating plainly: + +- **A person's first forge login creates their forge account**, named + after authelia's `preferred_username` (`[oauth2_client]` in the forge + module), provided the subject has an email. It starts as an ordinary user; `swarmctl forge make-admin +` makes it a site admin. A name that already has a local forge + account doesn't get it handed over: forgejo asks for that account's + own password first (`ACCOUNT_LINKING = login`). Agents, `core` and + `swarm-controller` all have local accounts, and `swarmctl user add` + refuses none of those names. + +- **A member of `admins` becomes a matrix homeserver admin** once their + first matrix login has created the account: `swarm-controller` promotes + them within five minutes, and never demotes. See + [`2-handbook/swarm/matrix.md`](../../2-handbook/swarm/matrix.md#operators-are-homeserver-admins). + +- **tuwunel re-reads its secret file on every OAuth exchange**, not only + at startup, and its own sandboxing hides most paths from it. It gets the + file through `LoadCredential` for the same reason the appservice + registration does — that keeps `DynamicUser` and `PrivateUsers` intact, + with no host-side ownership arrangement to maintain. +- **Matrix SSO lives inside the homeserver.** The client-server API is + spoken by non-browser clients holding matrix access tokens — every + agent's own daemon — as well as by federation, so the homeserver + serves `/_matrix/` directly and authenticates it itself. The + forward-auth vhosts protect browser surfaces; this isn't one of them. + +## Machine clients + +Everything above is a person in a browser. A swarm also has callers that +hold no session and follow no redirect: the telemetry collector, the +queue's auth-callout responder, and each hive. + +**One client per hive, not one per service.** A hive's identity belongs to +the directory rather than to whichever service happens to consume it, so a +hive holds a single OIDC client — `` — and mints a +different token per service from it. The alternative, letting each +consuming subsystem declare its own list, collides on the same client id +the moment a second consumer appears. + +**The audience is that client id.** A swarm service that has to tell hives +apart needs one name both sides already agree on, and the client id is +already that name. A parallel per-hive naming scheme would be a second +thing to keep in step, and it drifts silently — a mismatch presents as a +valid token refused at the target, which reads like a broken credential +rather than a broken name. + +**Authelia signs tokens (`RS256`), not opaque ones**, because a resource server +that can't call the provider back is a real case here: the telemetry +collector verifies offline against `/jwks.json`, and an opaque token gives +it nothing to verify. The queue's responder introspects instead — a +different question asked of the same token, and the reason both +`/api/oidc/introspection` and `/jwks.json` have to stay reachable. + +## Machine callers must fail closed + +⚠️ **An error page that answers `200` is a security bug, not a cosmetic +one.** The browser surface intercepts upstream errors and serves a +friendly "SSO is unavailable" page; that page is a file, so it returns +`200`. Any machine caller routed through it receives a success carrying +HTML instead of the failure that actually happened: + +- `/api/authz/auth-request` — nginx `auth_request` treats **any 2xx as + success**, so a down provider means _access granted_ +- `/api/oidc/introspection` — a token check that answers `200` +- `/api/oidc/token`, `/.well-known/openid-configuration` — a client + parsing an error page as its JSON document + +The gateway routes authelia's `/api/` and `/.well-known/` prefixes, and +the bridge's `/bridge/`, **without** error interception. The split is by +_audience_, not by an enumerated path list: a human gets the page, every +machine caller gets the status. +Enumerating endpoints individually would leave the next one added +silently intercepted. + +The same shape bites any machine route behind a browser-shaped gate: the +caller follows a `302` to a login page, the login page answers `200`, +and the caller reports success while nothing happened. Log ingest hit exactly this +and lost eleven hours of delivery in silence. + +**Checking it, if you change this routing.** Point the vhost at a dead +upstream and compare three requests, not one: + +1. through `/` — must still serve the friendly page +2. through `/api/` — must deny +3. a direct dial to authelia — must match what (2) did + +All three matter. A change that silently deleted the browser page would +pass a deny-only check, and one that quietly stopped denying would pass a +page-only check. The person who introduced the split verified it this way.