matrix: swarm-controller is the only minter
Every hive is in a swarm and every swarm runs matrix, so every swarm has a swarm-controller, and since #4810 its hive_sender pass mints each hive's @hive-<hive>: sender token into the store every five minutes. The two other minters of that token go: - swarm-matrix-ctl mint: the systemd.services.swarm-matrix-ctl unit in the hive-matrix container, Command::Mint and src/mint.rs. The binary, its appservice render/publish verbs, ctlPackage, ctlActive and the ctl cert role stay. bao-matrix-reader's checks on the deleted unit are removed; the leaf-identity and no-token-in-env checks now look at swarm-matrix-appservice-publish, which runs under the same identity. - the hive-side mint ladder in hive-c0re's ensure_hive_user (register/appservice-login/password-login with the local as_token), with read_appservice_token, paths::matrix_appservice_token and the helpers only it used. ensure_hive_user now takes the store's token, keeps the file when the store has none or can't be reached, and fails otherwise. - hivectl matrix sync-admin: the verb, HostRequest::MatrixSyncAdmin and handle_matrix_sync_admin. The periodic MatrixSweep (ensure_all) is unchanged apart from no longer reading the local as_token. This removes the double-mint race #4810's review flagged: two minters logging in on one pinned device could leave a dead token in the store until the next pass. Closes #4813 Closes #4814
This commit is contained in:
atlas
mara
parent
91e47732a6
commit
ddb7d7196d
22 changed files with 187 additions and 1162 deletions
|
|
@ -280,9 +280,6 @@ control: [`swarm/ui.md`](../swarm/ui.md).
|
|||
### 6 · Matrix
|
||||
|
||||
```bash
|
||||
# Ensure the appservice's sender account exists first
|
||||
hivectl matrix sync-admin
|
||||
|
||||
# Invite the operator to the hive Space (and optionally to rooms)
|
||||
hivectl matrix invite mara
|
||||
hivectl matrix invite @mara:yourserver --room '#hive-chat:yourserver'
|
||||
|
|
|
|||
|
|
@ -93,9 +93,8 @@ delegation (the latter lives in `gateway.md::Discovery flow`).
|
|||
## Provisioning flow (appservice)
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
Registration is closed. The hive's own **appservice** creates accounts:
|
||||
hive-c0re holds the appservice token, agents never see
|
||||
it, and an agent only ever receives its own `access_token`.
|
||||
Registration is closed. **Appservices** create accounts: agents never see
|
||||
an appservice token, and an agent only ever receives its own `access_token`.
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
The appservice has no URL (`url: null` in its registration), so the
|
||||
|
|
@ -110,7 +109,7 @@ a token.
|
|||
spec-required `hs_token` sibling, mode `0600 root:root`, then renders
|
||||
the registration to
|
||||
`/var/lib/hyperhive/matrix-appservice/hyperhive.yaml` (also `0600`).
|
||||
hive-c0re mints the tokens only when missing; the registration is
|
||||
The script mints the tokens only when missing; the registration is
|
||||
re-rendered every time, because the token file can be overwritten in
|
||||
place by the swarm secret store and a registration naming a stale
|
||||
token authenticates nobody. Runs at activation time, before any
|
||||
|
|
@ -129,10 +128,8 @@ a token.
|
|||
the bind-mount path. It reads only `.yaml`/`.yml` entries from there,
|
||||
so the sibling credentials are invisible to it. The `.yaml` suffix on
|
||||
the credential id is what makes this work.
|
||||
5. **hive-c0re** reads the appservice token and creates its own
|
||||
`@hive-<hive>:` account. It never mints the token itself: the value
|
||||
has to be the one the rendered registration names, and only the nix
|
||||
side writes that.
|
||||
5. **hive-c0re** doesn't read the appservice token and creates no
|
||||
account. Its `@hive-<hive>:` token comes from the swarm store (below).
|
||||
6. **Agents' accounts aren't this hive's.** `swarm-controller` creates each
|
||||
one with the swarm's own appservice token (next section), stores its
|
||||
token at `swarm/agents/<agent>/matrix/main`, and the agent's
|
||||
|
|
@ -183,9 +180,8 @@ hive's standing leaves the others alone.
|
|||
Its access token is the **sender token**, and it's the credential
|
||||
hive-c0re presents for every homeserver call it makes on the hive's
|
||||
behalf. It's per hive for the same reason the account is:
|
||||
`swarm-controller` mints it for every hive with the swarm's appservice token (and
|
||||
`swarm-matrix-ctl`, inside the `hive-matrix` container, for its own hive) and
|
||||
publishes it to `swarm/hives/<hive>/matrix/sender-token`, and the hive
|
||||
`swarm-controller`, its only minter, mints it for every hive with the swarm's
|
||||
appservice token and publishes it to `swarm/hives/<hive>/matrix/sender-token`, and the hive
|
||||
reads it from there under its own certificate. That path sits inside the
|
||||
hive's own read grant (`swarm/hives/<hive>/*`), so a hive fetches its own
|
||||
token and gets a refusal on any other hive's. The name says what it
|
||||
|
|
@ -212,34 +208,19 @@ Nothing to do, and no window where the hive is without an account.
|
|||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
- **The old shared value at `swarm/services/matrix/sender-token` is read by
|
||||
nothing.** `hive-c0re` and `swarm-matrix-ctl` both build the path from the
|
||||
nothing.** `hive-c0re` and `swarm-controller` both build the path from the
|
||||
same function, and it now carries the hive's name — so the old object
|
||||
stays in the store, unread, until an operator deletes it. Delete it or
|
||||
leave it; the accounts it authenticates as keep their own standing either
|
||||
way, since an access token lives on the device that minted it.
|
||||
- **The hive re-mints, per hive, on the next sweep.** `ensure_hive_user`
|
||||
reads the new per-hive store path **first, on every sweep**, not just when
|
||||
the file is missing (`sender_source`'s decision). If a per-hive token is
|
||||
already there — `swarm-controller` mints one for every hive — the sweep
|
||||
takes it and overwrites the file, so the shared token stops being served
|
||||
as soon as one exists in the store, no boot required. If the store has
|
||||
nothing yet, the file is left untouched (still the shared token, right
|
||||
after the upgrade), so nothing breaks mid-sweep. Only when neither the
|
||||
store nor the file holds anything does the sweep fall through to the
|
||||
register-or-appservice-login ladder against `@hive-<hive>:`, using only
|
||||
the `as_token`, which is per hive and on local disk, so that step works
|
||||
with or without a reachable store.
|
||||
- **To move a hive onto its own account now**, clear both copies: the
|
||||
sender-token file, and the store's `swarm/hives/<hive>/matrix/sender-token`
|
||||
if `swarm-controller` or `swarm-matrix-ctl` has already published one for
|
||||
it (otherwise the next sweep just re-adopts that value instead of minting
|
||||
a new one). With both empty, the next sweep — or `hivectl matrix
|
||||
sync-admin` — runs the ladder, registers `@hive-<hive>:`, and persists
|
||||
that account's token to the file. The ladder never writes the store — on
|
||||
a swarm that runs `swarm-controller`, its own mint pass will reach the
|
||||
same account on its next tick and write a token there too, on the same
|
||||
pinned device, which replaces whichever token was minted last. Only once
|
||||
both copies agree does the hive stop presenting the shared one.
|
||||
- **The hive switches on the next sweep.** `ensure_hive_user` reads the
|
||||
per-hive store path **first, on every sweep**, not just when the file is
|
||||
missing (`sender_source`'s decision). `swarm-controller` mints a token
|
||||
there for every hive within five minutes; the sweep takes it and
|
||||
overwrites the file, so the shared token stops being served as soon as
|
||||
one exists in the store, no boot required. While the store has nothing
|
||||
yet, the file is left untouched (still the shared token, right after the
|
||||
upgrade), so nothing breaks mid-sweep.
|
||||
- **The rooms the shared account created don't follow the new account, and
|
||||
this is the one step that needs a decision.** Membership is per account.
|
||||
`ensure_hive_space` takes the stored room id first, so the sweep hands the
|
||||
|
|
|
|||
|
|
@ -59,20 +59,20 @@ of the cell says how.
|
|||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
| store path | minter | reader — pulls at runtime, holds in memory | automatic re-mint | automatic re-pull |
|
||||
| ----------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `swarm/agents/<agent>/matrix/main` | `swarm-controller`, with the swarm's appservice token, at agent creation and in a five-minute pass | the agent container itself, under the certificate its hive passed in | ✅ the pass re-mints when the stored token is missing, unknown to the homeserver, or someone else's | ✅ `hive-matrix-daemon` exits when the homeserver rejects its token, and a five-minute timer restarts it, which reads the store again |
|
||||
| `swarm/agents/<agent>/matrix/<account>` | `swarm-controller` | the agent container itself, under the certificate its hive passed in | must be stated | must be stated |
|
||||
| `swarm/controller/swarm-controller/matrix/appservice-token` | `swarm-matrix-ctl`, inside the `hive-matrix` container, once | `swarm-controller`, under its own certificate | ❌ `swarm-matrix-ctl` mints it once; the container keeps its copy and republishes it when the store's differs | ✅ the controller reads it on every five-minute matrix pass |
|
||||
| `swarm/controller/swarm-controller/oidc/client` | authelia, at its first boot, where the controller registers its client; `swarm-secret-publish` copies it in | `swarm-controller`, under its own certificate, once at start | ❌ authelia mints it once. A re-mint is republished by `swarm-secret-publish`'s path unit | ❌ read once at start; the controller holds the old value until it restarts |
|
||||
| `swarm/agents/<agent>/bao-mtls` | the store's agent PKI mount (`deploy.bao.agentPkiMountPath`), which generates the key, at `swarm-controller`'s request at agent creation | `hive-c0re`, under the hive's own certificate, when it writes the agent's container config | ✅ `swarm-controller`'s five-minute pass re-issues a live agent's leaf once it's past half its validity (45 of 90 days, read from the certificate itself) | ❌ `hive-c0re` reads it when it writes the container config, so the agent presents a new leaf from its next start; the old leaf stays valid until it expires |
|
||||
| `swarm/agents/<agent>/queue` | `swarm-controller`, at agent creation | `hive-agent` in the agent container, under the agent's own certificate, held in memory — the identity it presents to the swarm queue, naming that one agent rather than its hive | ✅ `swarm-controller`'s five-minute pass re-mints a live agent's secret once it's 45 days old by `minted_at` on the stored object; a secret with no `minted_at` gets one stamped, value unchanged. The pass skips agents declared `Destroyed` — declaring an agent destroyed deletes every version of the path instead, the undo of the mint rather than another one | ✅ `hive-agent` reads the path before its first connect and again on every reconnect attempt, so a reconnect after a re-mint presents the new secret. An open connection keeps the secret it connected with; after a revocation the agent keeps retrying under the queue client's backoff |
|
||||
| `swarm/agents/<agent>/forge-token` | `swarm-controller`, at agent creation and in a pass every 5 minutes over every agent with a store identity | the agent container itself, under its own certificate, fetched to `/run/hive-agent-forge-token/token` | ✅ the controller re-mints when the stored token is missing or no longer matches the forge (last eight characters and scopes) | ✅ the agent re-fetches on a 10-minute timer |
|
||||
| `swarm/hives/<hive>/matrix/appservice-token` | one minter, on the authelia host | the hive process that presents the token to its homeserver, under the hive's own certificate | must be stated | must be stated |
|
||||
| `swarm/hives/<hive>/matrix/sender-token` | `swarm-controller`, with the swarm's appservice token, for every hive in its directory in a five-minute pass; also `swarm-matrix-ctl` in the `hive-matrix` container, for its own hive, when the path is empty | `swarm-controller` and `swarm-matrix-ctl` under their own certificates, before they decide whether to mint, and hive-c0re's `stored_sender_token()`, under the hive's own certificate | ✅ the controller's pass re-mints when the stored token is missing, unknown to the homeserver, or someone else's | ✅ hive-c0re's matrix sweep reads the store every run and overwrites its token file when the store's token differs |
|
||||
| `swarm/hives/<hive>/queue/agent` | authelia | `swarm-bao-queue-agent` on the hive's host, under its own per-hive certificate; no agent's policy reaches it | must be stated | must be stated |
|
||||
| `swarm/services/<clientId>/oidc/client` | authelia | the service process that presents the client secret, under the certificate of the host it runs on | must be stated | must be stated |
|
||||
| _(not in the store)_ a hive's mTLS leaf | the store's own PKI, or an operator placing it by hand | its own client, off disk — the exception above, because it's what makes every other row's pull possible | must be stated | must be stated |
|
||||
| store path | minter | reader — pulls at runtime, holds in memory | automatic re-mint | automatic re-pull |
|
||||
| ----------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `swarm/agents/<agent>/matrix/main` | `swarm-controller`, with the swarm's appservice token, at agent creation and in a five-minute pass | the agent container itself, under the certificate its hive passed in | ✅ the pass re-mints when the stored token is missing, unknown to the homeserver, or someone else's | ✅ `hive-matrix-daemon` exits when the homeserver rejects its token, and a five-minute timer restarts it, which reads the store again |
|
||||
| `swarm/agents/<agent>/matrix/<account>` | `swarm-controller` | the agent container itself, under the certificate its hive passed in | must be stated | must be stated |
|
||||
| `swarm/controller/swarm-controller/matrix/appservice-token` | `swarm-matrix-ctl`, inside the `hive-matrix` container, once | `swarm-controller`, under its own certificate | ❌ `swarm-matrix-ctl` mints it once; the container keeps its copy and republishes it when the store's differs | ✅ the controller reads it on every five-minute matrix pass |
|
||||
| `swarm/controller/swarm-controller/oidc/client` | authelia, at its first boot, where the controller registers its client; `swarm-secret-publish` copies it in | `swarm-controller`, under its own certificate, once at start | ❌ authelia mints it once. A re-mint is republished by `swarm-secret-publish`'s path unit | ❌ read once at start; the controller holds the old value until it restarts |
|
||||
| `swarm/agents/<agent>/bao-mtls` | the store's agent PKI mount (`deploy.bao.agentPkiMountPath`), which generates the key, at `swarm-controller`'s request at agent creation | `hive-c0re`, under the hive's own certificate, when it writes the agent's container config | ✅ `swarm-controller`'s five-minute pass re-issues a live agent's leaf once it's past half its validity (45 of 90 days, read from the certificate itself) | ❌ `hive-c0re` reads it when it writes the container config, so the agent presents a new leaf from its next start; the old leaf stays valid until it expires |
|
||||
| `swarm/agents/<agent>/queue` | `swarm-controller`, at agent creation | `hive-agent` in the agent container, under the agent's own certificate, held in memory — the identity it presents to the swarm queue, naming that one agent rather than its hive | ✅ `swarm-controller`'s five-minute pass re-mints a live agent's secret once it's 45 days old by `minted_at` on the stored object; a secret with no `minted_at` gets one stamped, value unchanged. The pass skips agents declared `Destroyed` — declaring an agent destroyed deletes every version of the path instead, the undo of the mint rather than another one | ✅ `hive-agent` reads the path before its first connect and again on every reconnect attempt, so a reconnect after a re-mint presents the new secret. An open connection keeps the secret it connected with; after a revocation the agent keeps retrying under the queue client's backoff |
|
||||
| `swarm/agents/<agent>/forge-token` | `swarm-controller`, at agent creation and in a pass every 5 minutes over every agent with a store identity | the agent container itself, under its own certificate, fetched to `/run/hive-agent-forge-token/token` | ✅ the controller re-mints when the stored token is missing or no longer matches the forge (last eight characters and scopes) | ✅ the agent re-fetches on a 10-minute timer |
|
||||
| `swarm/hives/<hive>/matrix/appservice-token` | one minter, on the authelia host | the hive process that presents the token to its homeserver, under the hive's own certificate | must be stated | must be stated |
|
||||
| `swarm/hives/<hive>/matrix/sender-token` | `swarm-controller`, with the swarm's appservice token, for every hive in its directory in a five-minute pass | `swarm-controller` under its own certificate, before it decides whether to mint, and hive-c0re's `stored_sender_token()`, under the hive's own certificate | ✅ the controller's pass re-mints when the stored token is missing, unknown to the homeserver, or someone else's | ✅ hive-c0re's matrix sweep reads the store every run and overwrites its token file when the store's token differs |
|
||||
| `swarm/hives/<hive>/queue/agent` | authelia | `swarm-bao-queue-agent` on the hive's host, under its own per-hive certificate; no agent's policy reaches it | must be stated | must be stated |
|
||||
| `swarm/services/<clientId>/oidc/client` | authelia | the service process that presents the client secret, under the certificate of the host it runs on | must be stated | must be stated |
|
||||
| _(not in the store)_ a hive's mTLS leaf | the store's own PKI, or an operator placing it by hand | its own client, off disk — the exception above, because it's what makes every other row's pull possible | must be stated | must be stated |
|
||||
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
|
|
|
|||
|
|
@ -8,7 +8,6 @@ This document contains the help content for the `hivectl` command-line program.
|
|||
* [`hivectl forge`↴](#hivectl-forge)
|
||||
* [`hivectl forge reconcile-config`↴](#hivectl-forge-reconcile-config)
|
||||
* [`hivectl matrix`↴](#hivectl-matrix)
|
||||
* [`hivectl matrix sync-admin`↴](#hivectl-matrix-sync-admin)
|
||||
* [`hivectl matrix invite`↴](#hivectl-matrix-invite)
|
||||
* [`hivectl github`↴](#hivectl-github)
|
||||
* [`hivectl github set-token`↴](#hivectl-github-set-token)
|
||||
|
|
@ -64,7 +63,7 @@ Sibling to the `hive-c0re` daemon binary. Covers host-side admin operations that
|
|||
###### **Subcommands:**
|
||||
|
||||
* `forge` — Reconcile an agent's config between this hive and the forge
|
||||
* `matrix` — matrix-tuwunel user provisioning
|
||||
* `matrix` — matrix-tuwunel invites
|
||||
* `github` — GitHub account provisioning
|
||||
* `gateway` — Gateway htpasswd user management
|
||||
* `agent` — Lifecycle actions on ONE managed agent container. Needs the hive-c0re daemon running
|
||||
|
|
@ -129,29 +128,18 @@ Always prints the diff first. `--from forge` resets the local checkout to forge
|
|||
|
||||
## `hivectl matrix`
|
||||
|
||||
matrix-tuwunel user provisioning.
|
||||
matrix-tuwunel invites.
|
||||
|
||||
Manual entry point to the same idempotent provisioning c0re runs at boot — for re-registering an agent the boot sweep skipped, or after wiping a token file.
|
||||
Manual invites into the hive Space and its rooms; c0re's periodic matrix sweep provisions the Space, the chat room and the agents' invites on its own.
|
||||
|
||||
**Usage:** `hivectl matrix <COMMAND>`
|
||||
|
||||
###### **Subcommands:**
|
||||
|
||||
* `sync-admin` — Provision (or re-provision) the matrix appservice's sender account
|
||||
* `invite` — Invite a matrix user to the hive Space, or a specific room with `--room`. Idempotent
|
||||
|
||||
|
||||
|
||||
## `hivectl matrix sync-admin`
|
||||
|
||||
Provision (or re-provision) the matrix appservice's sender account.
|
||||
|
||||
Runs automatically on startup; run manually to recover a missing access token.
|
||||
|
||||
**Usage:** `hivectl matrix sync-admin`
|
||||
|
||||
|
||||
|
||||
## `hivectl matrix invite`
|
||||
|
||||
Invite a matrix user to the hive Space, or a specific room with `--room`. Idempotent
|
||||
|
|
|
|||
|
|
@ -50,7 +50,6 @@ Manual entry to the same idempotent matrix provisioning flow
|
|||
running (`services.hyperhive.deploy.matrix.enable = true`).
|
||||
|
||||
```bash
|
||||
hivectl matrix sync-admin # provision / refresh the appservice's sender account
|
||||
hivectl matrix invite mara # invite a user to the hive Space
|
||||
hivectl matrix invite @mara:server --room '#hive-chat:server' # ...or to a specific room/alias
|
||||
```
|
||||
|
|
@ -59,10 +58,6 @@ Human matrix accounts come from SSO, not `hivectl`; matrix homeserver
|
|||
admin should eventually come from membership in authelia's `admins`
|
||||
group; nobody has built that sync yet.
|
||||
|
||||
- `sync-admin`: ensures this hive's appservice sender account
|
||||
(`@hive-<hive>:<server_name>`, one per hive) exists
|
||||
(the account `hive-c0re` provisions rooms with). Token persisted to the
|
||||
access token path. Safe to run again — idempotent.
|
||||
- `invite`: invites a matrix user (full `@user:server` or a bare
|
||||
localpart, qualified with the homeserver's `server_name`) to the hive
|
||||
Space by default, or to a `--room` id / `#alias`. Uses the sender
|
||||
|
|
|
|||
Loading…
Reference in a new issue