render the matrix appservice registration from the store onto tmpfs
swarm-matrix-ctl appservice render now reads the registration's tokens from the store and renders swarm.yaml under /run/swarm-matrix-appservice, which tuwunel loads through LoadCredential. It mints a pair only when the store answers and holds none; a host that has the old files stores their pair once instead. A store that does not answer fails the unit and keeps tuwunel down; sync starts it once the store answers. Refs #4998 (item 3)
This commit is contained in:
atlas
mara
parent
7813337548
commit
3a4b0ffabf
8 changed files with 594 additions and 378 deletions
|
|
@ -125,11 +125,12 @@ creation doesn't depend on registration being open to anyone who learns
|
|||
a token.
|
||||
|
||||
1. **`swarm-matrix-ctl appservice render`**, inside the `hive-matrix`
|
||||
container and before tuwunel starts, renders the registration to
|
||||
`/var/lib/swarm-matrix-appservice/swarm.yaml` (mode `0600 root:root`)
|
||||
from the `as_token` and spec-required `hs_token` beside it. It mints
|
||||
them only on a container that has none, and never reaches the store, so
|
||||
a store outage can't keep the homeserver down.
|
||||
container and before tuwunel starts, fetches the `as_token` and
|
||||
spec-required `hs_token` from the store and renders the registration to
|
||||
`/run/swarm-matrix-appservice/swarm.yaml` (mode `0600 root:root`) on the
|
||||
container's tmpfs, so no token sits on disk. It mints a pair only when
|
||||
the store answers and holds none. While the store doesn't answer, `render`
|
||||
fails and retries, and tuwunel stays down.
|
||||
2. **systemd `LoadCredential=`** copies the registration into
|
||||
`/run/credentials/tuwunel.service/swarm-appservice.yaml`, owned by
|
||||
tuwunel's dynamic user with mode `0400`, at service start. The source
|
||||
|
|
@ -167,15 +168,17 @@ account with it.
|
|||
|
||||
The swarm secret store holds the registration's tokens, at
|
||||
`swarm/controller/swarm-controller/matrix/appservice-registration`.
|
||||
`swarm-matrix-ctl appservice render` renders the registration tuwunel
|
||||
loads as a credential from the copy of them in the `hive-matrix`
|
||||
container, before tuwunel starts and without the store.
|
||||
`swarm-matrix-ctl appservice render` fetches them and renders the
|
||||
registration tuwunel loads as a credential onto the `hive-matrix`
|
||||
container's tmpfs, before tuwunel starts.
|
||||
`appservice sync` runs beside tuwunel on a timer. The store's tokens win:
|
||||
when they differ from the container's, it writes them there. It
|
||||
when they differ from the rendered copy, it rewrites it. It
|
||||
restarts tuwunel whenever the registration is newer than tuwunel's last
|
||||
start, so the next run retries one that failed after writing. A store that holds none takes the container's
|
||||
tokens with a create-only write, so a running swarm keeps its value, and
|
||||
a container that has none mints a pair. `sync` then writes the `as_token`
|
||||
start, so the next run retries one that failed after writing, and starts
|
||||
one that failed because `render` couldn't reach the store. A store that
|
||||
holds none takes the tokens an earlier version kept in
|
||||
`/var/lib/swarm-matrix-appservice` with a create-only write, so a running
|
||||
swarm keeps its value; the files stay, unused. A host with neither files nor stored tokens mints a pair. `sync` then writes the `as_token`
|
||||
to `swarm/controller/swarm-controller/matrix/appservice-token`. No hive's
|
||||
policy reaches either path: only matrix-ctl (which holds `create` and
|
||||
`read` on the registration, and writes the token) and `swarm-controller`
|
||||
|
|
@ -263,8 +266,8 @@ both are swarm-level operations.
|
|||
<details><summary>Appservice identity</summary>
|
||||
|
||||
`swarm-matrix-ctl appservice render` renders the registration from the
|
||||
container's copy of the appservice tokens before tuwunel starts, so every
|
||||
boot has both halves, with the store up or down.
|
||||
store's copy of the appservice tokens before tuwunel starts, so every
|
||||
boot has both halves. With the store down or sealed, tuwunel waits for it.
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
- **Access tokens live on the device that minted them.**
|
||||
|
|
|
|||
|
|
@ -22,7 +22,7 @@ the rest of the cell says how.
|
|||
| `swarm/agents/<agent>/matrix/<account>` | `swarm-controller`, from a credential you hand it | the agent | must be stated | ✅ retried within 2 to 16 minutes after a rejection |
|
||||
| `swarm/agents/<agent>/matrix-recovery/<account>` | `swarm-controller`, a random passphrase per account | the agent | ❌ never replaced: the account's key backup opens only with this passphrase | ❌ read when the account comes up |
|
||||
| `swarm/controller/swarm-controller/matrix/appservice-token` | `swarm-matrix-ctl appservice sync` in `hive-matrix`, from the registration's `as_token` | `swarm-controller` | ❌ follows the registration below | ✅ read every 5 minutes |
|
||||
| `swarm/controller/swarm-controller/matrix/appservice-registration` | `swarm-matrix-ctl appservice sync` in `hive-matrix`: seeded from the container's own tokens when the store holds none, a new pair only when the container has none either | `swarm-matrix-ctl` in `hive-matrix`; the controller has no grant to it | ❌ minted once; an operator who replaces it makes `sync` write the new pair into the container and restart the homeserver | ✅ `sync` re-reads it every 10 minutes |
|
||||
| `swarm/controller/swarm-controller/matrix/appservice-registration` | `swarm-matrix-ctl appservice render` in `hive-matrix`, when the store answers and holds none; before that, the tokens an earlier version kept in the container, stored unchanged | `swarm-matrix-ctl` in `hive-matrix`; the controller has no grant to it | ❌ minted once; an operator who replaces it makes `sync` render the new pair and restart the homeserver | ✅ `sync` re-reads it every 10 minutes |
|
||||
| `swarm/controller/swarm-controller/oidc/client` | authelia, first boot | `swarm-controller` | ❌ minted once, straight into the store; minted again only when the store holds none | ✅ re-read when authelia refuses it |
|
||||
| `swarm/controller/swarm-controller/forge/token` | the forge, once | `swarm-controller` | ❌ minted once; delete `/var/lib/forgejo/swarm-controller-token-name` in the forge container to mint a new one | ✅ re-read on a 401 |
|
||||
| `swarm/controller/swarm-controller/grafana/token` | `swarm-bao-grafana-controller-token` on Grafana's host | `swarm-controller` | ✅ every 10 minutes, when Grafana refuses it or none is stored | ✅ re-read on a 401 |
|
||||
|
|
@ -113,7 +113,7 @@ pieces fit, and the reasoning behind them, is in [`credentials.md`](credentials.
|
|||
| swarm-services root CA (cert + key) | `swarm-bao.nix`'s granting unit, inside the store's `pki` mount | the **key** never leaves the mount; the **cert** lands at `/var/lib/swarm-bao-services-pki/services-root.pem` (0644) and at `/var/lib/hive-tls/swarm-services-root.pem` on every hive | nothing to copy — every hive receives the cert with the leaf it asks the store for, or from `pki/ca/pem` where it fronts no swarm name and needs a store identity |
|
||||
| authelia session, JWT, storage-encryption and OIDC HMAC keys | `authelia-swarm-secrets` in authelia's container. It stores each in the swarm secret store at `swarm/services/swarm-authelia/keys/<name>` when the store holds none: the value of the key an earlier release generated, or a new one when there is no such file. The files stay on disk. While the store's policy doesn't grant the leaf yet, or the store is down, the unit runs on the files. A container with no secret publisher identity generates the files, as before | `/var/lib/authelia-swarm/{session,jwt,storage-encryption,oidc-hmac}.key` | nothing to place: the publisher's store leaf is the one in the row below. A store that stays unreachable on a start with no key file keeps authelia from starting |
|
||||
| authelia OIDC issuer key (RSA) | same unit, same store path (`oidc-issuer`) | `/var/lib/authelia-swarm/oidc-issuer.key` | same — relying parties verify against the **public** half at `/jwks.json` |
|
||||
| swarm appservice registration (`as_token`, `hs_token`) | `swarm-matrix-ctl appservice sync` in `hive-matrix`. The store's pair wins; a store holding none takes the container's own tokens, so the value doesn't change by moving. Only `appservice render` on a container with neither mints a new pair | `/var/lib/swarm-matrix-appservice/{as-token,hs-token,swarm.yaml}` in `hive-matrix`, the last copy `sync` took, which tuwunel starts from; the store path is `swarm/controller/swarm-controller/matrix/appservice-registration` | nothing to place: matrix-ctl's store leaf is the one the container binds. A store that's down leaves the homeserver running on the files |
|
||||
| swarm appservice registration (`as_token`, `hs_token`) | `swarm-matrix-ctl appservice render` in `hive-matrix`. The store's pair wins; a store holding none takes the tokens an earlier version kept in the container, so the value doesn't change by moving. Only a store that answers, holds none, and a container with no such files gets a new pair | `/run/swarm-matrix-appservice/swarm.yaml` in `hive-matrix` (tmpfs), which tuwunel loads through `LoadCredential`. The earlier `/var/lib/swarm-matrix-appservice/{as-token,hs-token,swarm.yaml}` stay, unused, and bao wins if they differ; the store path is `swarm/controller/swarm-controller/matrix/appservice-registration` | nothing to place: matrix-ctl's store leaf is the one the container binds. A store that's down keeps the homeserver down and mints nothing, and `render` retries every 30 seconds for 24 hours |
|
||||
| OIDC client secret, plaintext half | `authelia crypto hash generate --random` | the swarm secret store, at the path its reader fetches: `swarm/hives/<hive>/oidc/client`, `swarm/controller/swarm-controller/oidc/client` or `swarm/services/<client id>/oidc/client`. The bridge reads its own into `/run/swarm-authelia-bridge/client-secret` before each start. A container with no secret publisher identity keeps it in `/var/lib/authelia-swarm/oidc-clients/<id>.secret`; a file an earlier release left seeds the store once when it holds no value, and stays unused | nothing to place: readers fetch it from the store ([how](credentials.md#hive-level-secrets)) |
|
||||
| OIDC client secret, digest half | the same mint | `oidc-clients/<id>.digest` | authelia's own half; merged at runtime via `settingsFiles` |
|
||||
| the swarm collector's copy of its OIDC secret | `swarm-bao-otel-oidc.service` reads it out of the swarm secret store, **on every host that runs the collector and holds a store identity** | `/run/swarm-otel-oidc/<id>.secret` on the host's tmpfs, bind-mounted read-only into the `swarm-otel` container — never on persistent disk, so a reboot fetches it again | same unit, same path — one route, co-located or not. A collector with no store identity has `clientSecretFile == null`, its already-supported unauthenticated-push degrade — see [swarm-level secrets](credentials.md#swarm-level-secrets) |
|
||||
|
|
|
|||
|
|
@ -100,6 +100,25 @@ let
|
|||
# environment, so the binary itself spells no role at all.
|
||||
ctlCertRole = "swarm-matrix-ctl";
|
||||
|
||||
# What `swarm-matrix-ctl appservice render` and `sync` are both given: the
|
||||
# store address and matrix-ctl's identity, and the registration's shape. A
|
||||
# secret is a path, never a value, so nothing here is a token.
|
||||
appserviceEnv = {
|
||||
BAO_ADDR = "https://${baoCfg.domain}:${toString baoCfg.port}";
|
||||
BAO_CLIENT_CERT = deployCfg.matrix.ctlBaoClientCertFile;
|
||||
BAO_CLIENT_KEY = deployCfg.matrix.ctlBaoClientKeyFile;
|
||||
MATRIX_APPSERVICE_CERT_ROLE = ctlCertRole;
|
||||
MATRIX_APPSERVICE_RUNTIME_DIR = swarmAppserviceRuntimeDir;
|
||||
MATRIX_APPSERVICE_LEGACY_DIR = swarmAppserviceLegacyDir;
|
||||
MATRIX_APPSERVICE_SENDER = swarmSenderLocalpart;
|
||||
# Agents' localparts are bare `[a-z0-9-]`, so nothing narrower covers
|
||||
# them without also covering people.
|
||||
MATRIX_APPSERVICE_USER_REGEX = appserviceUserRegex;
|
||||
}
|
||||
// lib.optionalAttrs (deployCfg.bao.serverCaFile != null) {
|
||||
BAO_CACERT = deployCfg.bao.serverCaFile;
|
||||
};
|
||||
|
||||
# Every host directory matrix-ctl's bao identity is spread across. Normally
|
||||
# one — ./glue-bao-tls.nix puts all three files in ./glue's PKI dir — but
|
||||
# derived rather than assumed, because an operator naming paths by hand is
|
||||
|
|
@ -140,20 +159,25 @@ let
|
|||
# The homeserver's one appservice registration, and the swarm's identity on
|
||||
# it: `swarm-controller` creates every agent's account and every hive's
|
||||
# `@hive-<hive>` sender with its token. The store holds its tokens;
|
||||
# `swarm-matrix-ctl appservice sync` keeps this container's copy in step with
|
||||
# them, which is why it is gated on matrix-ctl's store identity: with nobody
|
||||
# to sync it, a registration here would be an admin credential nobody reads.
|
||||
# `appservice render` minted a copy here only for a host that has none, and
|
||||
# renders the one the homeserver starts from.
|
||||
# `swarm-matrix-ctl appservice render` fetches them into this container's
|
||||
# tmpfs before each homeserver start, and `appservice sync` keeps that copy
|
||||
# in step with them. Both are gated on matrix-ctl's store identity: with
|
||||
# nobody to fetch it, a registration here would be an admin credential
|
||||
# nobody reads. `render` mints a pair only when the store answers and holds
|
||||
# none, and the homeserver stays down while the store does not answer.
|
||||
#
|
||||
# Its sender is promoted to homeserver admin at boot (`admin_execute`
|
||||
# below), so its token goes only to a store path no hive's policy reaches.
|
||||
# `swarm` is in ../reserved-names.nix, so no agent can be created as this
|
||||
# account.
|
||||
swarmSenderLocalpart = "swarm";
|
||||
# Inside the container: the render unit's `StateDirectory`. `ephemeral =
|
||||
# false` keeps it across restarts, so "mint when absent" is mint once.
|
||||
swarmAppserviceDir = "/var/lib/swarm-matrix-appservice";
|
||||
# Inside the container, on its tmpfs: the registration tuwunel loads, owned
|
||||
# by the two units below through `RuntimeDirectory`.
|
||||
swarmAppserviceRuntimeName = "swarm-matrix-appservice";
|
||||
swarmAppserviceRuntimeDir = "/run/${swarmAppserviceRuntimeName}";
|
||||
# Where earlier versions kept the tokens. Read to seed an empty store, never
|
||||
# written.
|
||||
swarmAppserviceLegacyDir = "/var/lib/swarm-matrix-appservice";
|
||||
swarmAppserviceCredentialId = "swarm-appservice.yaml";
|
||||
|
||||
# Every local user the swarm appservice may create — agents and every
|
||||
|
|
@ -1021,77 +1045,71 @@ in
|
|||
# suffix on purpose: `appservice_dir` above names this very
|
||||
# directory, and tuwunel takes only `.yaml`/`.yml` entries from
|
||||
# it. A missing source fails this unit, which is why the render
|
||||
# below is `requiredBy` it and local only.
|
||||
++ lib.optional ctlActive "${swarmAppserviceCredentialId}:${swarmAppserviceDir}/swarm.yaml";
|
||||
# below is `requiredBy` it.
|
||||
++ lib.optional ctlActive "${swarmAppserviceCredentialId}:${swarmAppserviceRuntimeDir}/swarm.yaml";
|
||||
|
||||
# The swarm registration, rendered before the homeserver loads it from
|
||||
# the tokens in the state dir: the last copy of the store's that
|
||||
# `swarm-matrix-appservice-sync` took, or a minted pair on a host with
|
||||
# none. No network and no store: this is on tuwunel's start path, and
|
||||
# a store outage must not keep the homeserver down.
|
||||
# The swarm registration, fetched from the store and rendered onto
|
||||
# the tmpfs before the homeserver loads it. A store that does not
|
||||
# answer fails this unit, and with it the homeserver's start: the
|
||||
# tokens are on no disk to start from. It retries like the other
|
||||
# store fetches, and `swarm-matrix-appservice-sync` below starts the
|
||||
# homeserver once the registration is there.
|
||||
#
|
||||
# `RuntimeDirectoryPreserve` because the sync unit shares the
|
||||
# directory: neither unit's stop may remove it from under the other.
|
||||
systemd.services.swarm-matrix-appservice-render = lib.mkIf ctlActive {
|
||||
description = "render the swarm's appservice registration";
|
||||
before = [ "tuwunel.service" ];
|
||||
requiredBy = [ "tuwunel.service" ];
|
||||
serviceConfig = {
|
||||
inherit (storeRetry) startLimitBurst startLimitIntervalSec;
|
||||
serviceConfig = storeRetry.serviceConfig // {
|
||||
Type = "oneshot";
|
||||
RemainAfterExit = true;
|
||||
ExecStart = "${deployCfg.matrix.ctlPackage}/bin/swarm-matrix-ctl appservice render";
|
||||
StateDirectory = baseNameOf swarmAppserviceDir;
|
||||
StateDirectoryMode = "0700";
|
||||
# A sealed store answers on the port and never answers the read.
|
||||
TimeoutStartSec = 60;
|
||||
RuntimeDirectory = swarmAppserviceRuntimeName;
|
||||
RuntimeDirectoryMode = "0700";
|
||||
RuntimeDirectoryPreserve = "yes";
|
||||
UMask = "0077";
|
||||
SyslogIdentifier = "swarm-matrix-appservice";
|
||||
};
|
||||
environment = {
|
||||
MATRIX_APPSERVICE_DIR = swarmAppserviceDir;
|
||||
MATRIX_APPSERVICE_SENDER = swarmSenderLocalpart;
|
||||
# Agents' localparts are bare `[a-z0-9-]`, so nothing narrower
|
||||
# covers them without also covering people.
|
||||
MATRIX_APPSERVICE_USER_REGEX = appserviceUserRegex;
|
||||
};
|
||||
environment = appserviceEnv;
|
||||
};
|
||||
|
||||
# The store holds the swarm registration; this keeps the state dir
|
||||
# the render above reads in step with it, and publishes the
|
||||
# `as_token` to the path `swarm-controller` reads, under matrix-ctl's
|
||||
# identity (the grants are in ./swarm-bao.nix). A store without the
|
||||
# registration is seeded from the state dir, so nothing rotates.
|
||||
# Keeps the rendered registration in step with the store, and
|
||||
# publishes the `as_token` to the path `swarm-controller` reads,
|
||||
# under matrix-ctl's identity (the grants are in ./swarm-bao.nix).
|
||||
# When the registration is newer than tuwunel's last start, this
|
||||
# restarts tuwunel onto it; a tuwunel that failed because the render
|
||||
# did is started the same way.
|
||||
#
|
||||
# ⚠️ Not `before`/`requiredBy` tuwunel, and nothing orders tuwunel
|
||||
# after it: the homeserver starts from the state dir's last copy
|
||||
# whether or not the store answers. When the registration is newer than
|
||||
# tuwunel's last start, this restarts tuwunel onto it.
|
||||
#
|
||||
# `Restart=on-failure` covers a sealed store or one still starting;
|
||||
# the timer below repeats the run, which a unit without `RemainAfterExit` allows.
|
||||
# `wants`, not `requires`, the render: this unit must still run when
|
||||
# the render has failed, since it is what brings the homeserver up
|
||||
# once the store answers. `Restart=on-failure` with the shared budget
|
||||
# covers a sealed store or one still starting; the timer below
|
||||
# repeats the run, which a unit without `RemainAfterExit` allows.
|
||||
# The start timeout is bounded because a sealed store answers on the
|
||||
# port and never answers the read.
|
||||
systemd.services.swarm-matrix-appservice-sync = lib.mkIf ctlActive {
|
||||
description = "keep the swarm's appservice registration in step with the swarm secret store";
|
||||
after = [ "swarm-matrix-appservice-render.service" ];
|
||||
requires = [ "swarm-matrix-appservice-render.service" ];
|
||||
wants = [ "swarm-matrix-appservice-render.service" ];
|
||||
wantedBy = [ "multi-user.target" ];
|
||||
path = [ pkgs.systemd ];
|
||||
serviceConfig = {
|
||||
inherit (storeRetry) startLimitBurst startLimitIntervalSec;
|
||||
serviceConfig = storeRetry.serviceConfig // {
|
||||
Type = "oneshot";
|
||||
ExecStart = "${deployCfg.matrix.ctlPackage}/bin/swarm-matrix-ctl appservice sync";
|
||||
Restart = "on-failure";
|
||||
RestartSec = 30;
|
||||
TimeoutStartSec = 60;
|
||||
RuntimeDirectory = swarmAppserviceRuntimeName;
|
||||
RuntimeDirectoryMode = "0700";
|
||||
RuntimeDirectoryPreserve = "yes";
|
||||
UMask = "0077";
|
||||
SyslogIdentifier = "swarm-matrix-appservice";
|
||||
};
|
||||
environment = {
|
||||
BAO_ADDR = "https://${baoCfg.domain}:${toString baoCfg.port}";
|
||||
BAO_CLIENT_CERT = deployCfg.matrix.ctlBaoClientCertFile;
|
||||
BAO_CLIENT_KEY = deployCfg.matrix.ctlBaoClientKeyFile;
|
||||
MATRIX_APPSERVICE_CERT_ROLE = ctlCertRole;
|
||||
MATRIX_APPSERVICE_DIR = swarmAppserviceDir;
|
||||
MATRIX_APPSERVICE_SENDER = swarmSenderLocalpart;
|
||||
MATRIX_APPSERVICE_USER_REGEX = appserviceUserRegex;
|
||||
environment = appserviceEnv // {
|
||||
MATRIX_APPSERVICE_RESTART_UNIT = "tuwunel.service";
|
||||
}
|
||||
// lib.optionalAttrs (deployCfg.bao.serverCaFile != null) {
|
||||
BAO_CACERT = deployCfg.bao.serverCaFile;
|
||||
};
|
||||
};
|
||||
|
||||
|
|
|
|||
|
|
@ -653,7 +653,8 @@ let
|
|||
# write, unlike either sibling, because sync compares before it writes.
|
||||
#
|
||||
# The second is the registration's tokens: the store's copy, which matrix-ctl
|
||||
# seeds from the container's state dir and then follows. `create` and `read`
|
||||
# creates (from the tokens an earlier version kept in the container, or a
|
||||
# minted pair) and then follows. `create` and `read`
|
||||
# only, never `update`: a registration that exists is replaced by an
|
||||
# operator, not by a unit.
|
||||
#
|
||||
|
|
@ -2008,9 +2009,9 @@ in
|
|||
example = "swarm-matrix-ctl.svc";
|
||||
description = ''
|
||||
Subject the store's matrix-ctl cert-auth role accepts — the
|
||||
identity the matrix container's `swarm-matrix-appservice-sync`
|
||||
unit presents when it syncs the swarm appservice's registration and
|
||||
publishes its token.
|
||||
identity the matrix container's `swarm-matrix-appservice-render` and
|
||||
`swarm-matrix-appservice-sync` units present when they fetch the swarm
|
||||
appservice's registration and publish its token.
|
||||
|
||||
A **third** identity rather than reuse of either sibling above. Its
|
||||
grant is that credential and the registration it derives from, plus
|
||||
|
|
|
|||
|
|
@ -258,13 +258,15 @@ let
|
|||
creds =
|
||||
baoWithMatrix.containers.hive-matrix.config.systemd.services.tuwunel.serviceConfig.LoadCredential;
|
||||
in
|
||||
lib.elem "swarm-appservice.yaml:/var/lib/swarm-matrix-appservice/swarm.yaml" creds
|
||||
lib.elem "swarm-appservice.yaml:/run/swarm-matrix-appservice/swarm.yaml" creds
|
||||
&& lib.length (lib.filter (c: lib.hasInfix ".yaml:" c) creds) == 1;
|
||||
}
|
||||
{
|
||||
# A missing `LoadCredential` source fails tuwunel, so the render has to
|
||||
# have run first — and it is local only, so it cannot fail on a store.
|
||||
name = "the swarm registration is rendered before tuwunel and required by it, without the store";
|
||||
# have run first. It fetches from the store, so it carries the store's
|
||||
# address and matrix-ctl's identity, and retries on failure: a store
|
||||
# that does not answer keeps the homeserver down until it does.
|
||||
name = "the swarm registration is rendered before tuwunel and required by it, from the store, with retries";
|
||||
ok =
|
||||
let
|
||||
r = baoWithMatrix.containers.hive-matrix.config.systemd.services.swarm-matrix-appservice-render;
|
||||
|
|
@ -272,17 +274,19 @@ let
|
|||
lib.elem "tuwunel.service" r.before
|
||||
&& lib.elem "tuwunel.service" r.requiredBy
|
||||
&& lib.hasSuffix "swarm-matrix-ctl appservice render" r.serviceConfig.ExecStart
|
||||
&& !(r.environment ? BAO_ADDR);
|
||||
&& r.environment.BAO_ADDR != ""
|
||||
&& r.environment.BAO_CLIENT_CERT == "/var/lib/swarm-bao-pki/matrix-ctl.pem"
|
||||
&& r.environment.MATRIX_APPSERVICE_CERT_ROLE == "swarm-matrix-ctl"
|
||||
&& r.serviceConfig.Restart == "on-failure";
|
||||
}
|
||||
{
|
||||
# The start path stays free of the store. The render is what tuwunel
|
||||
# waits for and it carries no store address or identity; the sync unit
|
||||
# that does is neither before tuwunel nor required by it, and tuwunel
|
||||
# names it nowhere, so a store outage leaves the homeserver starting on
|
||||
# the state dir's last copy. The presence arms are the render's own
|
||||
# ordering (the same lookup finds `before`/`requiredBy` where they are)
|
||||
# and the sync's order after the render.
|
||||
name = "the sync unit is off tuwunel's start path and the render holds no store identity";
|
||||
# The sync unit is what starts a homeserver the render's failure left
|
||||
# down, so it must run when the render has failed: it is after the
|
||||
# render, and neither requires nor binds to it. It stays off tuwunel's
|
||||
# own start path, and tuwunel names it nowhere. The presence arms are the
|
||||
# render's own ordering (the same lookup finds `before`/`requiredBy`
|
||||
# where they are) and the sync's order after the render.
|
||||
name = "the sync unit runs after the render without requiring it, and is off tuwunel's start path";
|
||||
ok =
|
||||
let
|
||||
c = baoWithMatrix.containers.hive-matrix.config.systemd;
|
||||
|
|
@ -298,15 +302,51 @@ let
|
|||
(u.bindsTo or [ ])
|
||||
];
|
||||
sync = "swarm-matrix-appservice-sync.service";
|
||||
render = "swarm-matrix-appservice-render.service";
|
||||
in
|
||||
lib.elem "tuwunel.service" r.before
|
||||
&& lib.elem "swarm-matrix-appservice-render.service" s.after
|
||||
&& lib.elem render s.after
|
||||
&& !(lib.elem render (s.requires or [ ]))
|
||||
&& !(lib.elem render (s.bindsTo or [ ]))
|
||||
&& !(lib.elem "tuwunel.service" (s.before or [ ]))
|
||||
&& !(lib.elem "tuwunel.service" (s.requiredBy or [ ]))
|
||||
&& !(lib.elem "tuwunel.service" (s.wantedBy or [ ]))
|
||||
&& !(lib.elem sync (names t))
|
||||
&& !(lib.elem sync (r.before or [ ]))
|
||||
&& !(lib.any (lib.hasPrefix "BAO_") (lib.attrNames r.environment));
|
||||
&& !(lib.elem sync (r.before or [ ]));
|
||||
}
|
||||
{
|
||||
# 🩸 The registration holds the tokens that create users, so it is
|
||||
# rendered onto the container's tmpfs and no unit keeps the tokens in
|
||||
# /var/lib. tuwunel's credential source, both units' runtime directory
|
||||
# and the directory they are told to render into are one path under
|
||||
# /run (the presence arms), and neither unit has a state directory or
|
||||
# a writable path under /var/lib. The legacy directory is named in the
|
||||
# environment, to be read, and nowhere else.
|
||||
name = "the swarm registration is rendered under /run and no unit writes it under /var/lib";
|
||||
ok =
|
||||
let
|
||||
c = baoWithMatrix.containers.hive-matrix.config.systemd;
|
||||
units = [
|
||||
c.services.swarm-matrix-appservice-render
|
||||
c.services.swarm-matrix-appservice-sync
|
||||
];
|
||||
creds = c.services.tuwunel.serviceConfig.LoadCredential;
|
||||
runDir = "/run/swarm-matrix-appservice";
|
||||
writesVarLib =
|
||||
u:
|
||||
lib.any (lib.hasPrefix "/var/lib") (
|
||||
lib.toList (u.serviceConfig.ReadWritePaths or [ ]) ++ lib.toList (u.serviceConfig.BindPaths or [ ])
|
||||
)
|
||||
|| (u.serviceConfig ? StateDirectory);
|
||||
in
|
||||
lib.elem "swarm-appservice.yaml:${runDir}/swarm.yaml" creds
|
||||
&& lib.all (u: u.environment.MATRIX_APPSERVICE_RUNTIME_DIR == runDir) units
|
||||
&& lib.all (u: u.serviceConfig.RuntimeDirectory == "swarm-matrix-appservice") units
|
||||
&& !(lib.any writesVarLib units)
|
||||
&& !(lib.any (cr: lib.hasInfix "/var/lib/swarm-matrix-appservice" cr) creds)
|
||||
&& lib.all (
|
||||
u: u.environment.MATRIX_APPSERVICE_LEGACY_DIR == "/var/lib/swarm-matrix-appservice"
|
||||
) units;
|
||||
}
|
||||
{
|
||||
# A timer repeats the sync, which needs a unit that does not stay
|
||||
|
|
|
|||
|
|
@ -13,18 +13,19 @@ crate.
|
|||
|
||||
### `appservice render`
|
||||
|
||||
Renders the swarm appservice registration tuwunel loads from the tokens in the
|
||||
state dir, minting them only on a host that has none. Runs before the
|
||||
homeserver and needs no network: the tokens on disk are the last copy `sync`
|
||||
took from the store.
|
||||
Renders the swarm appservice registration tuwunel loads from the store's tokens
|
||||
onto the container's tmpfs, minting a pair only when the store answers and holds
|
||||
none. Runs before the homeserver, which stays down while the store does not
|
||||
answer. Tokens an earlier version kept in the container's `/var/lib` are stored
|
||||
first when the store holds none; the files stay, unused.
|
||||
|
||||
### `appservice sync`
|
||||
|
||||
The store holds the registration's tokens. This writes them into the state dir
|
||||
The store holds the registration's tokens. This rewrites the rendered copy
|
||||
when they differ, restarts the homeserver whenever the registration is
|
||||
newer than its last start, and publishes the `as_token` for `swarm-controller`. A store holding none is seeded from the
|
||||
state dir, create-only, so nothing rotates. It runs beside the homeserver, so a
|
||||
store outage leaves the homeserver running on the last copy.
|
||||
newer than its last start, and publishes the `as_token` for `swarm-controller`.
|
||||
It runs the same reconciliation as `render`, and starts a homeserver that
|
||||
failed because `render` could not reach the store.
|
||||
|
||||
Both are configured entirely by the `MATRIX_APPSERVICE_*` environment the units
|
||||
set — no flags. A systemd `Environment=` block is what a nix module can render; a
|
||||
|
|
|
|||
|
|
@ -1,21 +1,22 @@
|
|||
//! `swarm-matrix-ctl appservice` — the **swarm's** own appservice
|
||||
//! registration. The store holds it; this container keeps a copy the
|
||||
//! homeserver starts from.
|
||||
//! registration. The store holds it; this container renders a copy the
|
||||
//! homeserver starts from onto its own tmpfs, and keeps none on disk.
|
||||
//!
|
||||
//! Two verbs, because they have opposite failure rules:
|
||||
//! Two verbs, both reading the store:
|
||||
//!
|
||||
//! - [`render`] runs before tuwunel and touches nothing but this container's
|
||||
//! state dir. tuwunel loads the file it writes through `LoadCredential`, and
|
||||
//! a missing credential source fails the homeserver's start, so this half
|
||||
//! must not be able to fail on a network. It renders from the tokens on
|
||||
//! disk, which are the last copy [`sync`] took from the store, and mints
|
||||
//! them only on a host that has none.
|
||||
//! - [`sync`] runs beside the homeserver, never before it, and needs the store.
|
||||
//! The store's tokens win: it writes the state dir only when they differ, and
|
||||
//! restarts the homeserver whenever the registration is newer than the
|
||||
//! homeserver's last start. A store holding none is seeded from
|
||||
//! the state dir, create-only, so nothing rotates. A sealed store delays it
|
||||
//! and nothing else.
|
||||
//! - [`render`] runs before tuwunel. tuwunel loads the file it writes through
|
||||
//! `LoadCredential`, and a missing credential source fails the homeserver's
|
||||
//! start, so a store that does not answer fails this unit and keeps the
|
||||
//! homeserver down until it does. It mints a pair only when the store
|
||||
//! answers and holds none.
|
||||
//! - [`sync`] runs beside the homeserver and follows the store: it rewrites
|
||||
//! the file when the store's tokens differ, restarts the homeserver
|
||||
//! whenever the registration is newer than the homeserver's last start, and
|
||||
//! publishes the `as_token` for the controller.
|
||||
//!
|
||||
//! Where the store holds none, both first store the tokens an earlier version
|
||||
//! kept in the legacy state dir (see [`resolve`]), so a host that has run for
|
||||
//! months keeps its value. The files stay, unused.
|
||||
//!
|
||||
//! This registration's sender is promoted to homeserver admin at boot
|
||||
//! (`nix/host-modules/hive-matrix.nix`), which is why its `as_token` goes to
|
||||
|
|
@ -33,14 +34,17 @@ use swarm_secret_client::{
|
|||
matrix::{self, Registration},
|
||||
};
|
||||
|
||||
/// This container's state dir for the registration and its two tokens.
|
||||
const ENV_DIR: &str = "MATRIX_APPSERVICE_DIR";
|
||||
/// Where the homeserver's copy is rendered: a runtime directory on this
|
||||
/// container's tmpfs.
|
||||
const ENV_RUNTIME_DIR: &str = "MATRIX_APPSERVICE_RUNTIME_DIR";
|
||||
/// The directory earlier versions kept the tokens in. Read once, never written.
|
||||
const ENV_LEGACY_DIR: &str = "MATRIX_APPSERVICE_LEGACY_DIR";
|
||||
/// The registration's `sender_localpart`: the account the homeserver creates
|
||||
/// for it and the one `admin_execute` promotes.
|
||||
const ENV_SENDER: &str = "MATRIX_APPSERVICE_SENDER";
|
||||
/// The user namespace regex, rendered by nix.
|
||||
const ENV_USER_REGEX: &str = "MATRIX_APPSERVICE_USER_REGEX";
|
||||
/// Role on the store's `cert` auth mount that [`sync`] logs in with.
|
||||
/// Role on the store's `cert` auth mount that both verbs log in with.
|
||||
const ENV_CERT_ROLE: &str = "MATRIX_APPSERVICE_CERT_ROLE";
|
||||
/// The unit that loads the registration, which [`sync`] restarts when the
|
||||
/// registration is newer than its last start.
|
||||
|
|
@ -48,7 +52,8 @@ const ENV_RESTART_UNIT: &str = "MATRIX_APPSERVICE_RESTART_UNIT";
|
|||
|
||||
/// The registration's `id`.
|
||||
const ID: &str = "swarm";
|
||||
/// File names inside [`ENV_DIR`]. `REGISTRATION` is what tuwunel loads.
|
||||
/// `REGISTRATION` is what tuwunel loads, in [`ENV_RUNTIME_DIR`]. All three are
|
||||
/// the names earlier versions used in [`ENV_LEGACY_DIR`].
|
||||
const AS_TOKEN: &str = "as-token";
|
||||
const HS_TOKEN: &str = "hs-token";
|
||||
const REGISTRATION: &str = "swarm.yaml";
|
||||
|
|
@ -62,41 +67,51 @@ fn required(get: &impl Fn(&str) -> Option<String>, var: &'static str) -> Result<
|
|||
.with_context(|| format!("{var} is unset or empty"))
|
||||
}
|
||||
|
||||
/// Mint the tokens if absent and render the registration from them.
|
||||
/// The store, logged in as matrix-ctl.
|
||||
async fn connect(cert_role: &str) -> Result<Bao> {
|
||||
let settings = Settings::from_env().context("reading the store's BAO_* environment")?;
|
||||
let store = SecretStore::connect(&settings, cert_role, DEFAULT_CERT_MOUNT)
|
||||
.await
|
||||
.with_context(|| {
|
||||
format!("logging in to the swarm secret store as cert role {cert_role}")
|
||||
})?;
|
||||
Ok(Bao(store))
|
||||
}
|
||||
|
||||
/// Render the registration from the store's tokens, which are minted first
|
||||
/// when the store holds none.
|
||||
///
|
||||
/// # Errors
|
||||
/// If a variable is missing, or the state dir cannot be read or written.
|
||||
pub fn render() -> Result<()> {
|
||||
/// If a variable is missing, or the store refuses the login, a read or a
|
||||
/// write, or the runtime dir cannot be written.
|
||||
pub async fn render() -> Result<()> {
|
||||
let get = |k: &str| std::env::var(k).ok();
|
||||
let dir = PathBuf::from(required(&get, ENV_DIR)?);
|
||||
let runtime = PathBuf::from(required(&get, ENV_RUNTIME_DIR)?);
|
||||
let legacy = PathBuf::from(required(&get, ENV_LEGACY_DIR)?);
|
||||
let sender = required(&get, ENV_SENDER)?;
|
||||
let regex = required(&get, ENV_USER_REGEX)?;
|
||||
render_into(&dir, &sender, ®ex)?;
|
||||
tracing::info!(path = %dir.join(REGISTRATION).display(), "rendered the swarm appservice registration");
|
||||
let store = connect(&required(&get, ENV_CERT_ROLE)?).await?;
|
||||
render_with(&store, &runtime, &legacy, &sender, ®ex, mint).await?;
|
||||
tracing::info!(path = %runtime.join(REGISTRATION).display(), "rendered the swarm appservice registration");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// [`render`] against an explicit directory, so a test can run it twice.
|
||||
fn render_into(dir: &Path, sender: &str, regex: &str) -> Result<()> {
|
||||
let as_token = existing_or_minted(&dir.join(AS_TOKEN))?;
|
||||
let hs_token = existing_or_minted(&dir.join(HS_TOKEN))?;
|
||||
/// [`render`] against an explicit store and directories.
|
||||
async fn render_with(
|
||||
store: &impl Store,
|
||||
runtime: &Path,
|
||||
legacy: &Path,
|
||||
sender: &str,
|
||||
regex: &str,
|
||||
mint: impl FnOnce() -> Result<Registration>,
|
||||
) -> Result<()> {
|
||||
let current = resolve(store, legacy, mint).await?;
|
||||
write_secret(
|
||||
&dir.join(REGISTRATION),
|
||||
®istration_yaml(sender, regex, &as_token, &hs_token),
|
||||
&runtime.join(REGISTRATION),
|
||||
®istration_yaml(sender, regex, ¤t.as_token, ¤t.hs_token),
|
||||
)
|
||||
}
|
||||
|
||||
/// The token at `path`, minting and writing one first when there is none.
|
||||
fn existing_or_minted(path: &Path) -> Result<String> {
|
||||
if let Some(t) = read_token(path)? {
|
||||
return Ok(t);
|
||||
}
|
||||
let token = swarm_matrix_client::random_hex(TOKEN_BYTES)?;
|
||||
write_secret(path, &token)?;
|
||||
tracing::info!(path = %path.display(), "minted a swarm appservice token");
|
||||
Ok(token)
|
||||
}
|
||||
|
||||
/// The trimmed token at `path`; `None` when the file is missing or blank.
|
||||
fn read_token(path: &Path) -> Result<Option<String>> {
|
||||
match std::fs::read_to_string(path) {
|
||||
|
|
@ -107,21 +122,43 @@ fn read_token(path: &Path) -> Result<Option<String>> {
|
|||
}
|
||||
}
|
||||
|
||||
/// A temp path beside `path`, unique per call and per process, so two writers
|
||||
/// of the same file never share one. Same directory, so the rename stays
|
||||
/// atomic.
|
||||
fn temp_path(path: &Path) -> PathBuf {
|
||||
static NEXT: std::sync::atomic::AtomicU32 = std::sync::atomic::AtomicU32::new(0);
|
||||
let mut name = path.file_name().unwrap_or_default().to_os_string();
|
||||
name.push(format!(
|
||||
".{}-{}.tmp",
|
||||
std::process::id(),
|
||||
NEXT.fetch_add(1, std::sync::atomic::Ordering::Relaxed)
|
||||
));
|
||||
path.with_file_name(name)
|
||||
}
|
||||
|
||||
/// Write `contents` to `path` as a `0600` file, through a rename so a reader
|
||||
/// never sees half of it.
|
||||
/// never sees half of it. A failure removes the temp file.
|
||||
fn write_secret(path: &Path, contents: &str) -> Result<()> {
|
||||
let tmp = path.with_extension("tmp");
|
||||
let _ = std::fs::remove_file(&tmp);
|
||||
let tmp = temp_path(path);
|
||||
let written = write_temp(&tmp, contents).and_then(|()| {
|
||||
std::fs::rename(&tmp, path).with_context(|| format!("renaming onto {}", path.display()))
|
||||
});
|
||||
if written.is_err() {
|
||||
let _ = std::fs::remove_file(&tmp);
|
||||
}
|
||||
written
|
||||
}
|
||||
|
||||
fn write_temp(tmp: &Path, contents: &str) -> Result<()> {
|
||||
let mut f = std::fs::OpenOptions::new()
|
||||
.write(true)
|
||||
.create_new(true)
|
||||
.mode(0o600)
|
||||
.open(&tmp)
|
||||
.open(tmp)
|
||||
.with_context(|| format!("creating {}", tmp.display()))?;
|
||||
f.write_all(contents.as_bytes())
|
||||
.and_then(|()| f.sync_all())
|
||||
.with_context(|| format!("writing {}", tmp.display()))?;
|
||||
std::fs::rename(&tmp, path).with_context(|| format!("renaming onto {}", path.display()))
|
||||
.with_context(|| format!("writing {}", tmp.display()))
|
||||
}
|
||||
|
||||
/// The registration tuwunel loads.
|
||||
|
|
@ -146,7 +183,7 @@ fn registration_yaml(sender: &str, regex: &str, as_token: &str, hs_token: &str)
|
|||
)
|
||||
}
|
||||
|
||||
/// What [`sync`] needs of the store, so the decisions can be tested without
|
||||
/// What [`render`] and [`sync`] need of the store, so the decisions can be tested without
|
||||
/// one.
|
||||
trait Store {
|
||||
/// The registration's tokens, or `None` when nothing is stored.
|
||||
|
|
@ -201,55 +238,55 @@ impl Store for Bao {
|
|||
}
|
||||
}
|
||||
|
||||
/// Make the state dir hold the store's registration, and the controller's token
|
||||
/// match it.
|
||||
/// Make the runtime dir hold the store's registration, and the controller's
|
||||
/// token match it.
|
||||
///
|
||||
/// # Errors
|
||||
/// If a variable is missing, the store refuses the login, a read or a write,
|
||||
/// the state dir cannot be written, or the homeserver cannot be restarted.
|
||||
/// the runtime dir cannot be written, or the homeserver cannot be restarted.
|
||||
pub async fn sync() -> Result<()> {
|
||||
let get = |k: &str| std::env::var(k).ok();
|
||||
let dir = PathBuf::from(required(&get, ENV_DIR)?);
|
||||
let runtime = PathBuf::from(required(&get, ENV_RUNTIME_DIR)?);
|
||||
let legacy = PathBuf::from(required(&get, ENV_LEGACY_DIR)?);
|
||||
let sender = required(&get, ENV_SENDER)?;
|
||||
let regex = required(&get, ENV_USER_REGEX)?;
|
||||
let cert_role = required(&get, ENV_CERT_ROLE)?;
|
||||
let unit = required(&get, ENV_RESTART_UNIT)?;
|
||||
|
||||
let settings = Settings::from_env().context("reading the store's BAO_* environment")?;
|
||||
let store = SecretStore::connect(&settings, &cert_role, DEFAULT_CERT_MOUNT)
|
||||
.await
|
||||
.with_context(|| {
|
||||
format!("logging in to the swarm secret store as cert role {cert_role}")
|
||||
})?;
|
||||
sync_with(&Bao(store), &dir, &sender, ®ex, mint, &Systemd(unit)).await
|
||||
let store = connect(&required(&get, ENV_CERT_ROLE)?).await?;
|
||||
sync_with(
|
||||
&store,
|
||||
&runtime,
|
||||
&legacy,
|
||||
&sender,
|
||||
®ex,
|
||||
mint,
|
||||
&Systemd(unit),
|
||||
)
|
||||
.await
|
||||
}
|
||||
|
||||
/// [`sync`] against an explicit store, directory and homeserver unit.
|
||||
/// [`sync`] against an explicit store, directories and homeserver unit.
|
||||
///
|
||||
/// The store's registration wins. Where it holds none, the state dir's tokens
|
||||
/// seed it, and only a host with neither mints a pair. The homeserver is
|
||||
/// restarted whenever the registration was written after it last started, so
|
||||
/// a run that failed between the write and the restart is finished by the next.
|
||||
/// The homeserver is restarted whenever the registration was written after it
|
||||
/// last started, so a run that failed between the write and the restart is
|
||||
/// finished by the next.
|
||||
async fn sync_with(
|
||||
store: &impl Store,
|
||||
dir: &Path,
|
||||
runtime: &Path,
|
||||
legacy: &Path,
|
||||
sender: &str,
|
||||
regex: &str,
|
||||
mint: impl FnOnce() -> Result<Registration>,
|
||||
unit: &impl Homeserver,
|
||||
) -> Result<()> {
|
||||
let local = read_local(dir)?;
|
||||
let current = stored_or_seeded(store, local.as_ref(), mint).await?;
|
||||
let current = resolve(store, legacy, mint).await?;
|
||||
|
||||
let path = runtime.join(REGISTRATION);
|
||||
let want = registration_yaml(sender, regex, ¤t.as_token, ¤t.hs_token);
|
||||
let have = std::fs::read_to_string(dir.join(REGISTRATION)).ok();
|
||||
if local.as_ref() != Some(¤t) || have.as_deref() != Some(&want) {
|
||||
write_secret(&dir.join(AS_TOKEN), ¤t.as_token)?;
|
||||
write_secret(&dir.join(HS_TOKEN), ¤t.hs_token)?;
|
||||
write_secret(&dir.join(REGISTRATION), &want)?;
|
||||
if std::fs::read_to_string(&path).ok().as_deref() != Some(&want) {
|
||||
write_secret(&path, &want)?;
|
||||
tracing::info!("took the swarm appservice registration from the store");
|
||||
}
|
||||
restart_if_stale(unit, &dir.join(REGISTRATION))?;
|
||||
restart_if_stale(unit, &path)?;
|
||||
|
||||
if needs_publish(store.controller_token().await?.as_ref(), ¤t.as_token) {
|
||||
store.write_controller_token(¤t.as_token).await?;
|
||||
|
|
@ -258,17 +295,31 @@ async fn sync_with(
|
|||
Ok(())
|
||||
}
|
||||
|
||||
/// The store's registration; where it holds none, `local` or a minted pair is
|
||||
/// created there first. A refused create is a lost race or a denial, and the
|
||||
/// read-back tells which: it finds the winner's value, or nothing.
|
||||
async fn stored_or_seeded(
|
||||
/// The registration the store holds. Where it holds none, the tokens an
|
||||
/// earlier version kept in `legacy` are stored once (create-only, so nothing
|
||||
/// rotates), and only a host with neither mints a pair. A store that cannot
|
||||
/// be read is an error and mints nothing. The files stay where they are,
|
||||
/// unused.
|
||||
async fn resolve(
|
||||
store: &impl Store,
|
||||
legacy: &Path,
|
||||
mint: impl FnOnce() -> Result<Registration>,
|
||||
) -> Result<Registration> {
|
||||
match store.registration().await? {
|
||||
Some(stored) => Ok(stored),
|
||||
None => seed_store(store, read_local(legacy)?.as_ref(), mint).await,
|
||||
}
|
||||
}
|
||||
|
||||
/// Create the registration in the empty store from `local`, or from a minted
|
||||
/// pair when there are no files, and return what the store then holds. A
|
||||
/// refused create is a lost race or a denial, and the read-back tells which:
|
||||
/// it finds the winner's pair, or nothing.
|
||||
async fn seed_store(
|
||||
store: &impl Store,
|
||||
local: Option<&Registration>,
|
||||
mint: impl FnOnce() -> Result<Registration>,
|
||||
) -> Result<Registration> {
|
||||
if let Some(stored) = store.registration().await? {
|
||||
return Ok(stored);
|
||||
}
|
||||
let seed = match local {
|
||||
Some(l) => l.clone(),
|
||||
None => mint()?,
|
||||
|
|
@ -282,7 +333,7 @@ async fn stored_or_seeded(
|
|||
.context("the store holds no swarm appservice registration after creating one")
|
||||
}
|
||||
|
||||
/// The tokens on disk, when both are there.
|
||||
/// The tokens in `dir`, when both are there.
|
||||
fn read_local(dir: &Path) -> Result<Option<Registration>> {
|
||||
let as_token = read_token(&dir.join(AS_TOKEN))?;
|
||||
let hs_token = read_token(&dir.join(HS_TOKEN))?;
|
||||
|
|
@ -447,18 +498,24 @@ mod tests {
|
|||
|
||||
const REGEX: &str = "^@[a-z0-9._=/-]+:t\\.local$";
|
||||
|
||||
/// A directory of its own per call. A counter, not the clock, keeps two
|
||||
/// tests that start in the same clock tick out of one directory.
|
||||
fn scratch() -> PathBuf {
|
||||
/// A runtime dir and a legacy dir of their own per call; the legacy one
|
||||
/// is not created, as on a host that never had the files. A counter, not
|
||||
/// the clock, keeps two tests that start in the same clock tick out of one
|
||||
/// directory.
|
||||
fn scratch() -> (PathBuf, PathBuf) {
|
||||
static NEXT: std::sync::atomic::AtomicU32 = std::sync::atomic::AtomicU32::new(0);
|
||||
let dir = std::env::temp_dir().join(format!(
|
||||
let root = std::env::temp_dir().join(format!(
|
||||
"swarm-appservice-{}-{}",
|
||||
std::process::id(),
|
||||
NEXT.fetch_add(1, std::sync::atomic::Ordering::Relaxed)
|
||||
));
|
||||
std::fs::remove_dir_all(&dir).ok();
|
||||
std::fs::create_dir_all(&dir).expect("temp dir");
|
||||
dir
|
||||
std::fs::remove_dir_all(&root).ok();
|
||||
std::fs::create_dir_all(root.join("run")).expect("temp dir");
|
||||
(root.join("run"), root.join("legacy"))
|
||||
}
|
||||
|
||||
fn cleanup(runtime: &Path) {
|
||||
std::fs::remove_dir_all(runtime.parent().expect("a parent")).ok();
|
||||
}
|
||||
|
||||
fn reg(tag: &str) -> Registration {
|
||||
|
|
@ -469,18 +526,34 @@ mod tests {
|
|||
}
|
||||
|
||||
/// A store in memory. `down` fails every call, as a sealed or unreachable
|
||||
/// store does; `racer` is what another writer stores between this run's
|
||||
/// read and its create.
|
||||
/// store does; `deny_create` refuses a create and stores nothing; `racer`
|
||||
/// is what another writer stores between this run's read and its create.
|
||||
#[derive(Default)]
|
||||
struct Fake {
|
||||
registration: RefCell<Option<Registration>>,
|
||||
token: RefCell<Option<String>>,
|
||||
down: bool,
|
||||
deny_create: bool,
|
||||
racer: RefCell<Option<Registration>>,
|
||||
creates: Cell<u32>,
|
||||
token_writes: Cell<u32>,
|
||||
}
|
||||
|
||||
impl Fake {
|
||||
fn holding(r: Registration) -> Self {
|
||||
let f = Self::default();
|
||||
*f.registration.borrow_mut() = Some(r);
|
||||
f
|
||||
}
|
||||
|
||||
fn down() -> Self {
|
||||
Self {
|
||||
down: true,
|
||||
..Self::default()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl Store for Fake {
|
||||
async fn registration(&self) -> Result<Option<Registration>> {
|
||||
if self.down {
|
||||
|
|
@ -491,6 +564,9 @@ mod tests {
|
|||
|
||||
async fn create_registration(&self, r: &Registration) -> Result<()> {
|
||||
self.creates.set(self.creates.get() + 1);
|
||||
if self.deny_create {
|
||||
bail!("permission denied");
|
||||
}
|
||||
if let Some(winner) = self.racer.borrow_mut().take() {
|
||||
*self.registration.borrow_mut() = Some(winner);
|
||||
bail!("already created");
|
||||
|
|
@ -513,8 +589,9 @@ mod tests {
|
|||
}
|
||||
}
|
||||
|
||||
/// Write the state dir as `render` leaves it for `r`.
|
||||
fn host_with(dir: &Path, r: &Registration) {
|
||||
/// The three files an earlier version left in the legacy dir for `r`.
|
||||
fn legacy_with(dir: &Path, r: &Registration) {
|
||||
std::fs::create_dir_all(dir).expect("legacy dir");
|
||||
write_secret(&dir.join(AS_TOKEN), &r.as_token).expect("as");
|
||||
write_secret(&dir.join(HS_TOKEN), &r.hs_token).expect("hs");
|
||||
write_secret(
|
||||
|
|
@ -524,11 +601,25 @@ mod tests {
|
|||
.expect("yaml");
|
||||
}
|
||||
|
||||
fn files(dir: &Path) -> Vec<String> {
|
||||
[AS_TOKEN, HS_TOKEN, REGISTRATION]
|
||||
.iter()
|
||||
.map(|f| std::fs::read_to_string(dir.join(f)).unwrap_or_default())
|
||||
.collect()
|
||||
/// What is left of the legacy files: the dir's file names, sorted.
|
||||
fn legacy_left(dir: &Path) -> Vec<String> {
|
||||
let mut names: Vec<String> = std::fs::read_dir(dir)
|
||||
.map(|d| {
|
||||
d.map(|e| e.expect("entry").file_name().to_string_lossy().into_owned())
|
||||
.collect()
|
||||
})
|
||||
.unwrap_or_default();
|
||||
names.sort();
|
||||
names
|
||||
}
|
||||
|
||||
/// The registration the runtime dir holds, or empty.
|
||||
fn rendered(runtime: &Path) -> String {
|
||||
std::fs::read_to_string(runtime.join(REGISTRATION)).unwrap_or_default()
|
||||
}
|
||||
|
||||
fn yaml_for(r: &Registration) -> String {
|
||||
registration_yaml("swarm", REGEX, &r.as_token, &r.hs_token)
|
||||
}
|
||||
|
||||
/// The homeserver unit. `started` is when it last became active; a restart
|
||||
|
|
@ -574,12 +665,24 @@ mod tests {
|
|||
}
|
||||
}
|
||||
|
||||
/// Run one render, returning how often it minted.
|
||||
async fn render_run(store: &Fake, runtime: &Path, legacy: &Path) -> (Result<()>, u32) {
|
||||
let mints = Cell::new(0);
|
||||
let out = render_with(store, runtime, legacy, "swarm", REGEX, || {
|
||||
mints.set(mints.get() + 1);
|
||||
Ok(reg("minted"))
|
||||
})
|
||||
.await;
|
||||
(out, mints.get())
|
||||
}
|
||||
|
||||
/// Run one sync against `unit`, returning how often it minted.
|
||||
async fn run(store: &Fake, dir: &Path, unit: &Unit) -> (Result<()>, u32) {
|
||||
async fn run(store: &Fake, runtime: &Path, legacy: &Path, unit: &Unit) -> (Result<()>, u32) {
|
||||
let mints = Cell::new(0);
|
||||
let out = sync_with(
|
||||
store,
|
||||
dir,
|
||||
runtime,
|
||||
legacy,
|
||||
"swarm",
|
||||
REGEX,
|
||||
|| {
|
||||
|
|
@ -592,28 +695,154 @@ mod tests {
|
|||
(out, mints.get())
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_rendered_registration_carries_the_token_it_minted() {
|
||||
let dir = scratch();
|
||||
render_into(&dir, "swarm", REGEX).expect("renders");
|
||||
let yaml = std::fs::read_to_string(dir.join(REGISTRATION)).expect("rendered");
|
||||
let token = std::fs::read_to_string(dir.join(AS_TOKEN)).expect("minted");
|
||||
assert_eq!(token.len(), TOKEN_BYTES * 2);
|
||||
assert!(yaml.contains(&format!("\nas_token: {token}\n")), "{yaml}");
|
||||
std::fs::remove_dir_all(&dir).ok();
|
||||
#[tokio::test]
|
||||
async fn an_empty_store_and_no_files_mint_a_pair_into_the_store() {
|
||||
let (runtime, legacy) = scratch();
|
||||
let store = Fake::default();
|
||||
|
||||
let (out, mints) = render_run(&store, &runtime, &legacy).await;
|
||||
|
||||
out.expect("renders");
|
||||
assert_eq!(mints, 1);
|
||||
assert_eq!(*store.registration.borrow(), Some(reg("minted")));
|
||||
assert_eq!(rendered(&runtime), yaml_for(®("minted")));
|
||||
assert!(!legacy.exists(), "nothing is written to the legacy dir");
|
||||
cleanup(&runtime);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn an_empty_store_takes_the_files_value_once() {
|
||||
let (runtime, legacy) = scratch();
|
||||
legacy_with(&legacy, ®("old"));
|
||||
let store = Fake::default();
|
||||
|
||||
let (out, mints) = render_run(&store, &runtime, &legacy).await;
|
||||
|
||||
out.expect("renders");
|
||||
assert_eq!(mints, 0, "a host that has the value mints nothing");
|
||||
assert_eq!(*store.registration.borrow(), Some(reg("old")));
|
||||
assert_eq!(rendered(&runtime), yaml_for(®("old")));
|
||||
cleanup(&runtime);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_store_that_is_down_mints_nothing_and_renders_nothing() {
|
||||
let (runtime, legacy) = scratch();
|
||||
legacy_with(&legacy, ®("disk"));
|
||||
|
||||
let (out, mints) = render_run(&Fake::down(), &runtime, &legacy).await;
|
||||
|
||||
assert!(out.is_err(), "the unit fails and retries");
|
||||
assert_eq!(mints, 0);
|
||||
assert_eq!(rendered(&runtime), "", "nothing for tuwunel to load");
|
||||
cleanup(&runtime);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_store_that_is_down_on_a_fresh_host_mints_nothing() {
|
||||
let (runtime, legacy) = scratch();
|
||||
|
||||
let (out, mints) = render_run(&Fake::down(), &runtime, &legacy).await;
|
||||
|
||||
assert!(out.is_err());
|
||||
assert_eq!(
|
||||
mints, 0,
|
||||
"a pair minted while the store is away is a rotation"
|
||||
);
|
||||
assert_eq!(rendered(&runtime), "");
|
||||
cleanup(&runtime);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_lost_race_adopts_the_winners_pair() {
|
||||
let (runtime, legacy) = scratch();
|
||||
legacy_with(&legacy, ®("disk"));
|
||||
let store = Fake::default();
|
||||
*store.racer.borrow_mut() = Some(reg("winner"));
|
||||
|
||||
let (out, _) = render_run(&store, &runtime, &legacy).await;
|
||||
|
||||
out.expect("renders");
|
||||
assert_eq!(rendered(&runtime), yaml_for(®("winner")));
|
||||
cleanup(&runtime);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_refused_create_fails_the_run() {
|
||||
let (runtime, legacy) = scratch();
|
||||
legacy_with(&legacy, ®("disk"));
|
||||
let store = Fake {
|
||||
deny_create: true,
|
||||
..Fake::default()
|
||||
};
|
||||
|
||||
let (out, _) = render_run(&store, &runtime, &legacy).await;
|
||||
|
||||
assert!(out.is_err(), "no read-back finds the pair");
|
||||
assert_eq!(rendered(&runtime), "");
|
||||
cleanup(&runtime);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_second_render_keeps_the_pair() {
|
||||
// "Only once": a re-mint on every boot would hand the controller a
|
||||
// token the homeserver no longer loads until sync catches up.
|
||||
let (runtime, legacy) = scratch();
|
||||
let store = Fake::default();
|
||||
render_run(&store, &runtime, &legacy)
|
||||
.await
|
||||
.0
|
||||
.expect("first");
|
||||
let first = rendered(&runtime);
|
||||
|
||||
let (out, mints) = render_run(&store, &runtime, &legacy).await;
|
||||
|
||||
out.expect("second");
|
||||
assert_eq!(mints, 0);
|
||||
assert_eq!(rendered(&runtime), first);
|
||||
cleanup(&runtime);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_second_render_keeps_the_token() {
|
||||
// "Only once": a re-mint on every boot would hand the controller a
|
||||
// token the homeserver no longer loads until sync catches up.
|
||||
let dir = scratch();
|
||||
render_into(&dir, "swarm", REGEX).expect("first");
|
||||
let first = std::fs::read_to_string(dir.join(REGISTRATION)).expect("rendered");
|
||||
render_into(&dir, "swarm", REGEX).expect("second");
|
||||
let second = std::fs::read_to_string(dir.join(REGISTRATION)).expect("rendered");
|
||||
assert_eq!(first, second);
|
||||
std::fs::remove_dir_all(&dir).ok();
|
||||
fn two_writers_of_one_file_use_different_temp_files() {
|
||||
let path = Path::new("/run/x/swarm.yaml");
|
||||
let (a, b) = (temp_path(path), temp_path(path));
|
||||
assert_ne!(a, b);
|
||||
assert_eq!(
|
||||
a.parent(),
|
||||
path.parent(),
|
||||
"the rename stays in one directory"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn overlapping_writers_all_succeed_and_leave_no_temp_file() {
|
||||
let (runtime, _) = scratch();
|
||||
let path = runtime.join(REGISTRATION);
|
||||
let writers: Vec<_> = (0..8)
|
||||
.map(|_| {
|
||||
let path = path.clone();
|
||||
std::thread::spawn(move || {
|
||||
(0..50)
|
||||
.try_for_each(|_| write_secret(&path, "same\n"))
|
||||
.is_ok()
|
||||
})
|
||||
})
|
||||
.collect();
|
||||
assert!(writers.into_iter().all(|w| w.join().expect("joins")));
|
||||
assert_eq!(std::fs::read_to_string(&path).expect("written"), "same\n");
|
||||
assert_eq!(legacy_left(&runtime), [REGISTRATION]);
|
||||
cleanup(&runtime);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_failed_write_removes_its_temp_file() {
|
||||
let (runtime, _) = scratch();
|
||||
// A directory where the file goes: the rename fails.
|
||||
std::fs::create_dir(runtime.join(REGISTRATION)).expect("dir");
|
||||
write_secret(&runtime.join(REGISTRATION), "x").expect_err("cannot replace a directory");
|
||||
assert_eq!(legacy_left(&runtime), [REGISTRATION]);
|
||||
cleanup(&runtime);
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
|
@ -624,172 +853,86 @@ mod tests {
|
|||
assert!(y.contains("\n - exclusive: false\n"), "{y}");
|
||||
assert!(y.contains(&format!("regex: '{REGEX}'")), "{y}");
|
||||
assert!(y.contains("\nurl: null\n"), "{y}");
|
||||
assert!(y.contains("\nas_token: aa\n"), "{y}");
|
||||
assert!(y.contains("\nhs_token: bb\n"), "{y}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_registration_and_tokens_are_owner_only() {
|
||||
#[tokio::test]
|
||||
async fn the_rendered_registration_is_owner_only() {
|
||||
use std::os::unix::fs::PermissionsExt as _;
|
||||
let dir = scratch();
|
||||
render_into(&dir, "swarm", REGEX).expect("renders");
|
||||
for f in [AS_TOKEN, HS_TOKEN, REGISTRATION] {
|
||||
let mode = std::fs::metadata(dir.join(f))
|
||||
.expect("exists")
|
||||
.permissions()
|
||||
.mode();
|
||||
assert_eq!(mode & 0o777, 0o600, "{f}");
|
||||
}
|
||||
std::fs::remove_dir_all(&dir).ok();
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn seeding_from_the_host_leaves_every_file_unchanged() {
|
||||
// A host that has run for months with the store empty: the store takes
|
||||
// its value, and nothing on the host moves, so nothing rotates.
|
||||
let dir = scratch();
|
||||
let on_disk = reg("old");
|
||||
host_with(&dir, &on_disk);
|
||||
let before = files(&dir);
|
||||
let store = Fake::default();
|
||||
|
||||
let unit = Unit::just_started();
|
||||
let (out, mints) = run(&store, &dir, &unit).await;
|
||||
let restarts = unit.restarts.get();
|
||||
|
||||
out.expect("syncs");
|
||||
assert_eq!(*store.registration.borrow(), Some(on_disk));
|
||||
assert_eq!(files(&dir), before);
|
||||
assert_eq!(mints, 0, "a host that has the value mints nothing");
|
||||
assert_eq!(restarts, 0, "an unchanged host does not restart");
|
||||
assert_eq!(store.token.borrow().as_deref(), Some("as-old"));
|
||||
std::fs::remove_dir_all(&dir).ok();
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn the_store_wins_when_both_hold_a_value() {
|
||||
let dir = scratch();
|
||||
host_with(&dir, ®("disk"));
|
||||
let store = Fake::default();
|
||||
*store.registration.borrow_mut() = Some(reg("store"));
|
||||
*store.token.borrow_mut() = Some("as-disk".to_owned());
|
||||
|
||||
let unit = Unit::long_running();
|
||||
let (out, _) = run(&store, &dir, &unit).await;
|
||||
let restarts = unit.restarts.get();
|
||||
|
||||
out.expect("syncs");
|
||||
let want = registration_yaml("swarm", REGEX, "as-store", "hs-store");
|
||||
assert_eq!(
|
||||
files(&dir),
|
||||
vec!["as-store".to_owned(), "hs-store".to_owned(), want]
|
||||
);
|
||||
assert_eq!(*store.registration.borrow(), Some(reg("store")));
|
||||
assert_eq!(store.creates.get(), 0, "a held value is never rewritten");
|
||||
assert_eq!(restarts, 1);
|
||||
assert_eq!(store.token.borrow().as_deref(), Some("as-store"));
|
||||
std::fs::remove_dir_all(&dir).ok();
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_host_without_the_files_takes_them_from_the_store() {
|
||||
let dir = scratch();
|
||||
let store = Fake::default();
|
||||
*store.registration.borrow_mut() = Some(reg("store"));
|
||||
|
||||
let unit = Unit::long_running();
|
||||
let (out, mints) = run(&store, &dir, &unit).await;
|
||||
let restarts = unit.restarts.get();
|
||||
|
||||
out.expect("syncs");
|
||||
assert_eq!(read_local(&dir).expect("reads"), Some(reg("store")));
|
||||
assert!(dir.join(REGISTRATION).exists());
|
||||
assert_eq!((restarts, mints), (1, 0));
|
||||
std::fs::remove_dir_all(&dir).ok();
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_fresh_host_mints_into_the_store() {
|
||||
let dir = scratch();
|
||||
let store = Fake::default();
|
||||
|
||||
let unit = Unit::long_running();
|
||||
let (out, mints) = run(&store, &dir, &unit).await;
|
||||
let restarts = unit.restarts.get();
|
||||
|
||||
out.expect("syncs");
|
||||
assert_eq!(mints, 1);
|
||||
assert_eq!(*store.registration.borrow(), Some(reg("minted")));
|
||||
assert_eq!(read_local(&dir).expect("reads"), Some(reg("minted")));
|
||||
assert_eq!(restarts, 1);
|
||||
std::fs::remove_dir_all(&dir).ok();
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_lost_race_adopts_the_winners_registration() {
|
||||
let dir = scratch();
|
||||
host_with(&dir, ®("disk"));
|
||||
let store = Fake::default();
|
||||
*store.racer.borrow_mut() = Some(reg("winner"));
|
||||
|
||||
let unit = Unit::long_running();
|
||||
let (out, _) = run(&store, &dir, &unit).await;
|
||||
let restarts = unit.restarts.get();
|
||||
|
||||
out.expect("syncs");
|
||||
assert_eq!(read_local(&dir).expect("reads"), Some(reg("winner")));
|
||||
assert_eq!(restarts, 1);
|
||||
std::fs::remove_dir_all(&dir).ok();
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_store_that_is_down_leaves_the_host_as_it_is() {
|
||||
let dir = scratch();
|
||||
host_with(&dir, ®("disk"));
|
||||
let before = files(&dir);
|
||||
let store = Fake {
|
||||
down: true,
|
||||
..Fake::default()
|
||||
};
|
||||
|
||||
let unit = Unit::just_started();
|
||||
let (out, mints) = run(&store, &dir, &unit).await;
|
||||
let restarts = unit.restarts.get();
|
||||
|
||||
assert!(out.is_err(), "the unit reports the outage");
|
||||
assert_eq!(files(&dir), before, "the homeserver keeps the last copy");
|
||||
assert_eq!((restarts, mints), (0, 0));
|
||||
std::fs::remove_dir_all(&dir).ok();
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn the_start_path_renders_the_copy_sync_took_from_the_store() {
|
||||
// `render` has no store: after an outage it renders what the last sync
|
||||
// wrote, without minting over it.
|
||||
let dir = scratch();
|
||||
let store = Fake::default();
|
||||
*store.registration.borrow_mut() = Some(reg("store"));
|
||||
run(&store, &dir, &Unit::long_running())
|
||||
let (runtime, legacy) = scratch();
|
||||
render_run(&Fake::default(), &runtime, &legacy)
|
||||
.await
|
||||
.0
|
||||
.expect("syncs");
|
||||
let synced = files(&dir);
|
||||
.expect("renders");
|
||||
let mode = std::fs::metadata(runtime.join(REGISTRATION))
|
||||
.expect("exists")
|
||||
.permissions()
|
||||
.mode();
|
||||
assert_eq!(mode & 0o777, 0o600);
|
||||
cleanup(&runtime);
|
||||
}
|
||||
|
||||
render_into(&dir, "swarm", REGEX).expect("renders");
|
||||
#[tokio::test]
|
||||
async fn the_store_wins_when_the_rendered_copy_differs() {
|
||||
let (runtime, legacy) = scratch();
|
||||
std::fs::write(runtime.join(REGISTRATION), yaml_for(®("old"))).expect("rendered");
|
||||
let store = Fake::holding(reg("store"));
|
||||
*store.token.borrow_mut() = Some("as-old".to_owned());
|
||||
|
||||
assert_eq!(files(&dir), synced);
|
||||
std::fs::remove_dir_all(&dir).ok();
|
||||
let unit = Unit::long_running();
|
||||
let (out, _) = run(&store, &runtime, &legacy, &unit).await;
|
||||
|
||||
out.expect("syncs");
|
||||
assert_eq!(rendered(&runtime), yaml_for(®("store")));
|
||||
assert_eq!(*store.registration.borrow(), Some(reg("store")));
|
||||
assert_eq!(store.creates.get(), 0, "a held value is never rewritten");
|
||||
assert_eq!(unit.restarts.get(), 1);
|
||||
assert_eq!(store.token.borrow().as_deref(), Some("as-store"));
|
||||
cleanup(&runtime);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_sync_with_nothing_rendered_takes_the_stores_pair() {
|
||||
let (runtime, legacy) = scratch();
|
||||
let store = Fake::holding(reg("store"));
|
||||
|
||||
let unit = Unit::long_running();
|
||||
let (out, mints) = run(&store, &runtime, &legacy, &unit).await;
|
||||
|
||||
out.expect("syncs");
|
||||
assert_eq!(rendered(&runtime), yaml_for(®("store")));
|
||||
assert_eq!((unit.restarts.get(), mints), (1, 0));
|
||||
cleanup(&runtime);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_store_that_is_down_leaves_the_rendered_copy_as_it_is() {
|
||||
let (runtime, legacy) = scratch();
|
||||
std::fs::write(runtime.join(REGISTRATION), yaml_for(®("disk"))).expect("rendered");
|
||||
|
||||
let unit = Unit::just_started();
|
||||
let (out, mints) = run(&Fake::down(), &runtime, &legacy, &unit).await;
|
||||
|
||||
assert!(out.is_err(), "the unit reports the outage");
|
||||
assert_eq!(rendered(&runtime), yaml_for(®("disk")));
|
||||
assert_eq!((unit.restarts.get(), mints), (0, 0));
|
||||
cleanup(&runtime);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_second_sync_restarts_nothing() {
|
||||
let dir = scratch();
|
||||
let store = Fake::default();
|
||||
*store.registration.borrow_mut() = Some(reg("store"));
|
||||
let (runtime, legacy) = scratch();
|
||||
let store = Fake::holding(reg("store"));
|
||||
|
||||
let unit = Unit::long_running();
|
||||
|
||||
run(&store, &dir, &unit).await.0.expect("first");
|
||||
run(&store, &runtime, &legacy, &unit)
|
||||
.await
|
||||
.0
|
||||
.expect("first");
|
||||
let first = unit.restarts.get();
|
||||
let (out, _) = run(&store, &dir, &unit).await;
|
||||
let (out, _) = run(&store, &runtime, &legacy, &unit).await;
|
||||
let second = unit.restarts.get() - first;
|
||||
|
||||
out.expect("syncs");
|
||||
|
|
@ -799,48 +942,48 @@ mod tests {
|
|||
1,
|
||||
"the controller's copy is written once"
|
||||
);
|
||||
std::fs::remove_dir_all(&dir).ok();
|
||||
cleanup(&runtime);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_restart_that_failed_is_retried_by_the_next_run() {
|
||||
// The files are written and the restart is refused. The next run finds
|
||||
// The file is written and the restart is refused. The next run finds
|
||||
// nothing to write, and still restarts: the decision comes from the
|
||||
// files' age against the unit's start, not from this run's writes.
|
||||
let dir = scratch();
|
||||
let store = Fake::default();
|
||||
*store.registration.borrow_mut() = Some(reg("store"));
|
||||
// file's age against the unit's start, not from this run's writes.
|
||||
let (runtime, legacy) = scratch();
|
||||
let store = Fake::holding(reg("store"));
|
||||
let unit = Unit::long_running();
|
||||
unit.fail_restart.set(true);
|
||||
|
||||
let (first, _) = run(&store, &dir, &unit).await;
|
||||
let (first, _) = run(&store, &runtime, &legacy, &unit).await;
|
||||
assert!(first.is_err(), "the refused restart fails the run");
|
||||
let written = files(&dir);
|
||||
let written = rendered(&runtime);
|
||||
|
||||
unit.fail_restart.set(false);
|
||||
let (second, _) = run(&store, &dir, &unit).await;
|
||||
let (second, _) = run(&store, &runtime, &legacy, &unit).await;
|
||||
|
||||
second.expect("the retry succeeds");
|
||||
assert_eq!(files(&dir), written, "nothing was left to write");
|
||||
assert_eq!(rendered(&runtime), written, "nothing was left to write");
|
||||
assert_eq!(unit.restarts.get(), 2);
|
||||
assert_eq!(read_local(&dir).expect("reads"), Some(reg("store")));
|
||||
std::fs::remove_dir_all(&dir).ok();
|
||||
cleanup(&runtime);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_unit_that_started_after_the_files_is_left_alone() {
|
||||
// The control for the retry above: the same files, a unit that started
|
||||
// after them, and no restart.
|
||||
let dir = scratch();
|
||||
host_with(&dir, ®("store"));
|
||||
let store = Fake::default();
|
||||
*store.registration.borrow_mut() = Some(reg("store"));
|
||||
async fn a_unit_that_started_after_the_file_is_left_alone() {
|
||||
// The control for the retry above: the same file, a unit that started
|
||||
// after it, and no restart.
|
||||
let (runtime, legacy) = scratch();
|
||||
std::fs::write(runtime.join(REGISTRATION), yaml_for(®("store"))).expect("rendered");
|
||||
let store = Fake::holding(reg("store"));
|
||||
let unit = Unit::just_started();
|
||||
|
||||
run(&store, &dir, &unit).await.0.expect("syncs");
|
||||
run(&store, &runtime, &legacy, &unit)
|
||||
.await
|
||||
.0
|
||||
.expect("syncs");
|
||||
|
||||
assert_eq!(unit.restarts.get(), 0);
|
||||
std::fs::remove_dir_all(&dir).ok();
|
||||
cleanup(&runtime);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
|
|
@ -855,17 +998,25 @@ mod tests {
|
|||
Ok(())
|
||||
}
|
||||
}
|
||||
let dir = scratch();
|
||||
let store = Fake::default();
|
||||
*store.registration.borrow_mut() = Some(reg("store"));
|
||||
let (runtime, legacy) = scratch();
|
||||
let store = Fake::holding(reg("store"));
|
||||
let unit = Unreadable(Cell::new(0));
|
||||
|
||||
let out = sync_with(&store, &dir, "swarm", REGEX, || Ok(reg("minted")), &unit).await;
|
||||
let out = sync_with(
|
||||
&store,
|
||||
&runtime,
|
||||
&legacy,
|
||||
"swarm",
|
||||
REGEX,
|
||||
|| Ok(reg("minted")),
|
||||
&unit,
|
||||
)
|
||||
.await;
|
||||
|
||||
out.expect("the run goes on");
|
||||
assert_eq!(unit.0.get(), 0);
|
||||
assert_eq!(store.token.borrow().as_deref(), Some("as-store"));
|
||||
std::fs::remove_dir_all(&dir).ok();
|
||||
cleanup(&runtime);
|
||||
}
|
||||
|
||||
/// `restart_via` against a `systemctl` that exits with the given codes for the
|
||||
|
|
@ -939,7 +1090,8 @@ mod tests {
|
|||
#[test]
|
||||
fn every_variable_is_scoped_to_the_verb() {
|
||||
for var in [
|
||||
ENV_DIR,
|
||||
ENV_RUNTIME_DIR,
|
||||
ENV_LEGACY_DIR,
|
||||
ENV_SENDER,
|
||||
ENV_USER_REGEX,
|
||||
ENV_CERT_ROLE,
|
||||
|
|
|
|||
|
|
@ -7,8 +7,8 @@
|
|||
//! identity plumbing to add one action, so the next thing that has to run in
|
||||
//! here is a verb below, not a new crate.
|
||||
//!
|
||||
//! [`appservice`] mints the **swarm's** own appservice registration and
|
||||
//! keeps it in step with the store and publishes its token for `swarm-controller`.
|
||||
//! [`appservice`] renders the **swarm's** own appservice registration from
|
||||
//! the store and publishes its token for `swarm-controller`.
|
||||
//!
|
||||
//! 🩸 **A secret is a path, never a value.** The only identifier any verb here
|
||||
//! logs is the store path; see `swarm_matrix_client`'s module doc for the same rule
|
||||
|
|
@ -39,12 +39,13 @@ enum Command {
|
|||
|
||||
#[derive(Debug, Subcommand)]
|
||||
enum Appservice {
|
||||
/// Mint the tokens when absent and render the registration tuwunel loads.
|
||||
/// Local only: it runs before the homeserver and must not need a network.
|
||||
/// Render the registration tuwunel loads from the store, minting the tokens
|
||||
/// first when the store holds none. Runs before the homeserver, which stays
|
||||
/// down until the store answers.
|
||||
Render,
|
||||
/// Make the state dir hold the store's registration, seeding the store
|
||||
/// from it when the store holds none, and restart the homeserver when the
|
||||
/// registration is newer than its last start. Runs beside the homeserver, never before it.
|
||||
/// Make the rendered registration match the store's, restart the homeserver
|
||||
/// when the registration is newer than its last start, and publish the
|
||||
/// `as_token` for `swarm-controller`. Runs beside the homeserver.
|
||||
Sync,
|
||||
}
|
||||
|
||||
|
|
@ -58,7 +59,7 @@ async fn main() -> Result<()> {
|
|||
.init();
|
||||
|
||||
match Cli::parse().command {
|
||||
Command::Appservice(Appservice::Render) => appservice::render(),
|
||||
Command::Appservice(Appservice::Render) => appservice::render().await,
|
||||
Command::Appservice(Appservice::Sync) => appservice::sync().await,
|
||||
}
|
||||
}
|
||||
|
|
|
|||
Loading…
Reference in a new issue