diff --git a/CLAUDE.md b/CLAUDE.md index ecc18dcc..55fccb33 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -153,9 +153,9 @@ hand-maintained per-file tree drifts out of sync with the code. **root and acts directly** — no socket, no HTTP route, no priv helper; the crate's README records why the rootless shape was examined and rejected. Does not link `swarm-controller`, mirroring `hivectl` ÷ - `hive-c0re`. ⚠️ Its user store is **two files, one authoritative**: - `users.json` is canonical, authelia's `users.yml` is a *rendered - artifact* that is written and never read back. Full, always-current + `hive-c0re`. ⚠️ Its user store is authelia's own **`users.yml`, read and + written in place** — one file, shared with `swarm-authelia-bridge`; see + that crate's README for what both writers must uphold. Full, always-current verb reference (CI-enforced against the clap tree, same pattern as `hivectl`'s — see `docs/conventions.md`): [`docs/tools/swarmctl-cli.md`](docs/tools/swarmctl-cli.md). diff --git a/docs/swarm/secrets.md b/docs/swarm/secrets.md index 8c670efc..23547b7c 100644 --- a/docs/swarm/secrets.md +++ b/docs/swarm/secrets.md @@ -47,7 +47,7 @@ Every row below is read against one of these. | authelia OIDC issuer key (RSA) | same unit | `/var/lib/authelia-swarm/oidc-issuer.key` | same — relying parties verify against the **public** half at `/jwks.json` | | OIDC client secret, plaintext half | `authelia crypto hash generate --random` | `/var/lib/authelia-swarm/oidc-clients/.secret` | operator provides the file and names it in the service's `sso.clientSecretFile` | | OIDC client secret, digest half | the same mint | `oidc-clients/.digest` | authelia's own half; merged at runtime via `settingsFiles` | -| authelia subject store | `swarmctl` | `users.json` (canonical) → `users.yml` (rendered) | `swarmctl`, on the host that runs authelia | +| authelia subject store | `swarmctl` and `swarm-authelia-bridge` | `users.yml` — one file, read and written by both | `swarmctl`, on the host that runs authelia | | wireguard private key | **the operator** — `wg genkey` | whatever `swarm.wireguard.privateKeyFile` names | always operator-provided; nothing generates this for you | | queue auth-callout nkeys (user seed + account seed) | `swarm-nats-callout-keys` first-boot unit, when `nats.autoGenerateCallout` is set | `/var/lib/swarm-nats-callout/{callout-user,issuer}.seed`, `0600` | operator mints both with `nk` and names them in `nats.calloutUserSeedFile` / `nats.calloutIssuerSeedFile` | diff --git a/docs/swarm/sso.md b/docs/swarm/sso.md index c858428f..c78dab5e 100644 --- a/docs/swarm/sso.md +++ b/docs/swarm/sso.md @@ -48,9 +48,14 @@ this password is stored nowhere — record it now ``` The password is generated, hashed, and printed once; only the hash is -kept. `swarmctl` writes its canonical `users.json`, re-renders authelia's -`users.yml` from it, and restarts authelia. Full reference: -[`../tools/swarmctl-cli.md`](../tools/swarmctl-cli.md). +kept. `swarmctl` reads and writes authelia's `users.yml` directly — it is +the one user store, shared with `swarm-authelia-bridge`, which creates +agent identities in the same file. No restart: authelia watches it. Full +reference: [`../tools/swarmctl-cli.md`](../tools/swarmctl-cli.md). + +You can edit `users.yml` by hand, and `swarmctl` will read what you +wrote. ⚠️ It rewrites the whole file on every change, so **comments and +formatting do not survive**; values and unrecognised keys do. This step stays manual on purpose. Bootstrapping an identity provider non-interactively means a secret arriving from somewhere — a file, an @@ -125,8 +130,8 @@ presenting as "the forge is broken", several layers from its cause. ### 2. Swarm-managed services The controller side owns provisioning: `swarmctl` writes both halves, the -same way it already owns authelia's user store (`users.json` canonical, -`users.yml` a rendered artifact). +same way it already owns authelia's user store (`users.yml`, read and +written in place). ### 3. A hive elsewhere diff --git a/docs/swarm/ui.md b/docs/swarm/ui.md index 006e5b22..f292993e 100644 --- a/docs/swarm/ui.md +++ b/docs/swarm/ui.md @@ -45,8 +45,8 @@ would have meant those accounts silently failing a check they were supposed to pass. An account created without any group needs re-adding with the flag — -`swarmctl` treats the existing entry as the canonical store, so the group -is what changes. +`swarmctl` reads the existing entry out of `users.yml`, so the group is +what changes. Why a group and not a list of usernames: agents are getting authelia accounts of their own (matrix SSO), and *authenticated* would then diff --git a/docs/tools/swarmctl-cli.md b/docs/tools/swarmctl-cli.md index 68acc6ce..1da3e791 100644 --- a/docs/tools/swarmctl-cli.md +++ b/docs/tools/swarmctl-cli.md @@ -25,8 +25,9 @@ swarm-level operator CLI ###### **Options:** * `--authelia-bin ` — authelia binary used to hash passwords. The argon2 parameters must match the verifier's, so this has to be the *configured* package rather than whatever is on `PATH` -* `--users-file ` — Host-side path of authelia's users database — i.e. the path inside the container, prefixed with the container's root -* `--store ` — Canonical user store +* `--users-file ` — Host-side path of authelia's users database — i.e. the path inside the container, prefixed with the container's root. + + This is the only user store: it is read before every change and written in place, and `swarm-authelia-bridge` writes the same file. @@ -40,7 +41,7 @@ Manage subjects in the swarm's SSO provider * `add` — Add a user, generating a password for them * `update` — Change an existing user's attributes -* `list` — List every user in the canonical store +* `list` — List every user in authelia's users database @@ -85,9 +86,9 @@ Every flag is optional and they compose, so one call can set several things at o ## `swarmctl user list` -List every user in the canonical store. +List every user in authelia's users database. -Reads `store` only — never authelia's rendered `users.yml`, which is a derived artifact this crate writes and never reads back (see the crate doc comment). One line per user: username, display name, email (if set), groups (if any). +Read-only: it never writes the file. Shows every subject in it, including agent identities `swarm-authelia-bridge` created — one line per user: username, display name, email (if set), groups (if any). **Usage:** `swarmctl user list` diff --git a/swarm-authelia-bridge/README.md b/swarm-authelia-bridge/README.md index c9d040e6..cb80a825 100644 --- a/swarm-authelia-bridge/README.md +++ b/swarm-authelia-bridge/README.md @@ -32,10 +32,29 @@ simply owns the file it writes. No elevated privilege anywhere. `swarm-controller` (loopback) and a split-host one (bind wider + firewall) with no separate transport. -## Known limitation +## One store, two writers -`swarmctl` still writes an independent `users.json`/`users.yml` for human -accounts, assuming co-location with `swarm-controller`'s host. Two -canonical stores for the same physical file is a real seam, not solved -here — routing `swarmctl` through this bridge too is a plausible follow-up, -out of scope for the agent-identity slice this crate shipped with. +This bridge and `swarmctl` both read and write authelia's `users.yml` +directly. That is the whole arrangement — neither keeps a private copy it +considers canonical. + +It used to be otherwise, and the seam was real: each side had its own +`users.json` treated as authoritative, rendering the *same* physical +`users.yml`. A writer whose own JSON was missing could not tell "nothing +here yet" from "someone else's users", so it refused to write at all — +which is exactly what a hive with existing users hit (#3422). + +What still has to hold, since two processes share the file: + +- **Read before write.** Both do, so neither can drop a user the other + added between operations. +- **Unknown keys survive.** Both carry unmodelled top-level and per-user + fields through a round-trip, or whichever writes second would silently + delete what the first added. +- **Every user gets an email.** A relying party asking for the claim fails + rather than degrades, so both write paths fill in a synthetic address + when none was supplied. + +Neither restarts authelia: it watches the file. `swarmctl` *could* (it is +root); this bridge deliberately cannot, and a reload that depends on which +process wrote is not a reload. diff --git a/swarm-authelia-bridge/src/store.rs b/swarm-authelia-bridge/src/store.rs index 5f19f4b8..1d6ad3b4 100644 --- a/swarm-authelia-bridge/src/store.rs +++ b/swarm-authelia-bridge/src/store.rs @@ -1,37 +1,20 @@ -//! The canonical user store this bridge owns, and the authelia users -//! database rendered from it. -//! -//! Deliberately its own copy, not shared code with `swarmctl::users` even -//! though the YAML shape is identical (same "factor out later if -//! duplication actually bites" reasoning as `swarm-controller::forge` -//! mirroring `hive-c0re::forge` — see the agent-identity-at-swarm-level -//! design thread). One real difference from `swarmctl`'s copy: this -//! store lives **wherever this -//! bridge runs** (inside the `swarm-authelia` container, alongside -//! `authelia-swarm`'s own state), not under `swarm-controller`'s state -//! dir — the two are not guaranteed to be the same host once -//! `swarm-authelia` and `swarm-controller` split across hosts, and this -//! bridge only ever runs where `swarm-authelia` does. -//! -//! # `users.yml` is the store, not a rendering of one +//! Authelia's users database — **the** store, not a rendering of one. //! //! There used to be a private `users.json` here, canonical, with `users.yml` -//! rendered from it — and `swarmctl` had its own pair against the *same* -//! physical `users.yml`. Two canonical stores for one file is a seam, and it -//! bit: a bridge whose own JSON was absent refused to write at all, because -//! it could not tell "nothing here yet" from "someone else's users". +//! rendered from it, while `swarmctl` kept its own pair against the *same* +//! physical file. Two canonical stores for one file is a seam, and it bit: a +//! writer whose own JSON was absent refused to write at all, unable to tell +//! "nothing here yet" from "someone else's users". The JSON bought nothing — +//! it was read back on every load, so the round-trip it seemed to avoid was +//! already being paid. //! -//! The JSON bought nothing. It was read back on every load, so the -//! round-trip it seemed to avoid was already being paid — the two files -//! differed only in *format*. One file removes the class: there is no second -//! store to disagree with, and the overwrite guard that policed them has -//! nothing left to do. +//! ⚠️ The file is **round-tripped**, so comments and hand-formatting do not +//! survive a write; values and unmodelled keys do (see `extra`). //! -//! ⚠️ Consequence, deliberate: this file is now **round-tripped**, so -//! comments and hand-formatting in it do not survive a write. An operator -//! editing it directly gets their *values* kept and their *comments* dropped. -//! Unknown keys are preserved (see `extra` below) so a field this binary does -//! not know about is not deleted by it. +//! Deliberately its own copy of the shape rather than code shared with +//! `swarmctl::users` — same "factor out if the duplication actually bites" +//! reasoning as `swarm-controller::forge` mirroring `hive-c0re::forge`. What +//! both copies must uphold is in `../README.md`. use std::collections::BTreeMap; use std::fs::{self, File, Permissions}; @@ -188,13 +171,14 @@ const SYNTHETIC_EMAIL_DOMAIN: &str = "hyperhive.local"; /// Give every user an address before the file is written. /// -/// 🩸 **This bridge did not do it, and `swarmctl` did.** #3414 fixed the -/// missing-email defect in `swarmctl` only, so every identity created *here* -/// landed in `users.yml` with no `email` — and a relying party that asks for -/// the claim does not degrade, it fails (grafana's OIDC login is the -/// measured case, #3393). Two writers of one file disagreeing about a -/// required field is not a difference worth keeping, so the rule now lives -/// on both write paths. +/// 🩸 **This bridge did not do it, and `swarmctl` did.** The fix for the +/// missing-email defect landed in `swarmctl` only, so every identity created +/// *here* landed in `users.yml` with no `email` — and a relying party that +/// asks for the claim does not degrade, it fails (grafana's OIDC login is +/// the measured case: it falls through to an endpoint authelia does not +/// implement and dies with an internal error). Two writers of one file +/// disagreeing about a required field is not a difference worth keeping, so +/// the rule now lives on both write paths. fn fill_missing_emails(store: &mut UserStore) { for (name, user) in &mut store.users { if user.email.is_none() { @@ -342,11 +326,11 @@ some_future_top_level_key: 7 /// `write_atomic` round-trips through a real temp dir — the property /// under test is the rename-into-place, not just the render. - /// 🩸 The asymmetry this fixes: #3414 gave `swarmctl`-created humans a - /// synthetic address and left bridge-created **agents** with none, so - /// two writers of one file disagreed about a field a relying party - /// treats as required — grafana's OIDC login fails outright without it - /// (#3393) rather than degrading. + /// 🩸 The asymmetry this fixes: the earlier missing-email fix gave + /// `swarmctl`-created humans a synthetic address and left bridge-created + /// **agents** with none, so two writers of one file disagreed about a + /// field a relying party treats as required — grafana's OIDC login fails + /// outright without it rather than degrading. /// /// Asserted through the real write path, because that is where the rule /// lives: `handle` inserts a user with `email: None` and nothing between diff --git a/swarmctl/README.md b/swarmctl/README.md index 18d64166..604a8e65 100644 --- a/swarmctl/README.md +++ b/swarmctl/README.md @@ -23,21 +23,24 @@ a verb has to run as a non-root user or from another host, the answer is a **group-gated admin socket**, separate from the controller's `0666` gateway-facing one — not a widening of what root does here. -## Two files, one of them authoritative +## One file, two writers -- `users.json` — canonical, ours, JSON. -- `users.yml` — a **rendered artifact** for authelia. Written, never read - back. +`users.yml` — authelia's own users database — is read and written +directly. There is no second store. -The split is what lets this crate work without a YAML parser: the -workspace has none, and adding one costs a crates.io fetch, a lock update -and a vendor hash for a schema we fully control and only ever emit. +There used to be: a private `users.json` here, canonical, with `users.yml` +rendered from it, while `swarm-authelia-bridge` kept its own pair against +the *same* physical file. Two canonical stores for one file is a seam, and +it bit — a writer whose own JSON was missing could not tell "nothing here +yet" from "someone else's users", and refused to write (#3422). -The shortcut of writing JSON into the `.yml` (JSON being a subset of -YAML) is deliberately not taken: authelia refuses to start on a users file -it cannot parse, so that file fronts the whole SSO provider's boot, and -"almost certainly parses" is not a claim worth betting a boot on without -running it. +The argument for the split was that it let this crate work without a YAML +parser. It didn't: the JSON was read back on every run, so the round-trip +was already being paid — the two files differed only in *format*. + +⚠️ The file is round-tripped, so **comments and hand-formatting do not +survive a write**. Values do, and so do keys this binary does not model. +See `swarm-authelia-bridge/README.md` for what both writers must uphold. ## Configuration @@ -53,7 +56,6 @@ an error. | `SWARMCTL_AUTHELIA_USERS_FILE` | host-side path of the users database | | `SWARMCTL_AUTHELIA_MACHINE` | container name, for `systemctl -M` | | `SWARMCTL_AUTHELIA_UNIT` | authelia's unit inside that container | -| `SWARMCTL_STORE` | canonical store (defaults to the controller's state dir) | ## Usage diff --git a/swarmctl/src/main.rs b/swarmctl/src/main.rs index 7630a59f..515dc78b 100644 --- a/swarmctl/src/main.rs +++ b/swarmctl/src/main.rs @@ -61,12 +61,14 @@ struct PathArgs { /// Host-side path of authelia's users database — i.e. the path inside /// the container, prefixed with the container's root. /// - /// ⚠️ This is the **only** user store. There used to be a `--store` - /// flag naming a private canonical JSON that this file was rendered - /// from; it is gone rather than deprecated, because a flag whose only - /// remaining effect would be nothing is worse than an unknown-argument - /// error — the operator sets it, sees success, and gets none of what - /// they asked for. + /// This is the only user store: it is read before every change and + /// written in place, and `swarm-authelia-bridge` writes the same file. + // + // The `--store` flag that named a second, private JSON store is gone + // rather than deprecated — a flag whose only remaining effect would be + // nothing reads as accepted and does nothing, where an unknown-argument + // error is loud. Not in the doc comment: `--help` is an operator + // surface, and the removal's reasoning belongs in the README. #[arg(long, value_name = "PATH")] users_file: Option, } @@ -149,12 +151,12 @@ enum UserVerb { /// editing an attribute, and folded together an attribute edit can /// invalidate a login by accident. Update(UpdateArgs), - /// List every user in the canonical store. + /// List every user in authelia's users database. /// - /// Reads `store` only — never authelia's rendered `users.yml`, which - /// is a derived artifact this crate writes and never reads back (see - /// the crate doc comment). One line per user: username, display name, - /// email (if set), groups (if any). + /// Read-only: it never writes the file. Shows every subject in it, + /// including agent identities `swarm-authelia-bridge` created — one + /// line per user: username, display name, email (if set), groups (if + /// any). List, } @@ -294,8 +296,8 @@ fn user_update(paths: &Paths, args: UpdateArgs) -> Result<()> { Ok(()) } -/// `swarmctl user list` — read-only, never touches authelia's users file -/// or restarts it. Prints one line per user from the canonical store. +/// `swarmctl user list` — read-only: it loads authelia's users file and +/// writes nothing. One line per user, agent identities included. fn user_list(paths: &Paths) -> Result<()> { use std::fmt::Write as _; @@ -325,8 +327,9 @@ fn publish(paths: &Paths, store: &mut UserStore) -> Result<()> { // Before rendering, not at creation: a user can also arrive by being // *read* — from a file the bridge wrote, or one an operator edited — // and an authelia subject with no `email` breaks any relying party that - // asks for the claim (grafana's OIDC login is the measured case, - // #3393). Filling it here is the only place no entry point can skip. + // asks for the claim (grafana's OIDC login is the measured case: it + // fails outright rather than degrading). Filling it here is the only + // place no entry point can skip. for name in users::fill_missing_emails(store) { println!("note: {name} had no email; set to a synthetic address"); } @@ -570,7 +573,10 @@ mod tests { fs::write(&users_file, "users: {}\n").expect("seed"); assert!( - load_store(&users_file).expect("the seed loads").users.is_empty(), + load_store(&users_file) + .expect("the seed loads") + .users + .is_empty(), "the first-boot seed is an empty store, with no special case" ); diff --git a/swarmctl/src/users.rs b/swarmctl/src/users.rs index 0b36270c..e6a3b99b 100644 --- a/swarmctl/src/users.rs +++ b/swarmctl/src/users.rs @@ -406,9 +406,9 @@ users: ); } - /// The #3414 property, moved from the renderer to the write path along - /// with the synthesis itself: a user who supplied an address keeps it, - /// and no invented one appears beside it. + /// The earlier missing-email property, moved from the renderer to the + /// write path along with the synthesis itself: a user who supplied an + /// address keeps it, and no invented one appears beside it. #[test] fn a_supplied_email_is_never_replaced_by_the_synthetic_one() { let mut u = user("$argon2id$x");