d9d6d37951
The scope-parameter addition pushed the doc comment on token_request to 39 lines. Moved the HTTP-Basic incident story and the audience/scope rationale to the crate README's new "Token request shape" section (docs/ is markdown, exempt from the lint); the code comment keeps the pointer plus the one-line summary of the invariant. Refs #4464
107 lines
5.6 KiB
Markdown
107 lines
5.6 KiB
Markdown
# swarm-queue-client
|
|
|
|
Connecting to the swarm message queue as an authenticated client. Shared by
|
|
every process that participates: the swarm controller reads hive status out of
|
|
the queue, a hive publishes its own status into it.
|
|
|
|
## Why a crate and not a module per binary
|
|
|
|
The _connect_ is identical for every participant — mint an authelia token,
|
|
present it at CONNECT for the `auth_callout` responder to introspect, let
|
|
`async-nats` re-run the callback on each connection attempt. Only the **use**
|
|
differs.
|
|
|
|
Two copies of that would be two copies of credential handling, and a
|
|
token-refresh fix would have to be found twice. The same reasoning already put
|
|
`hive-sock-client` in its own crate rather than in each daemon that speaks to a
|
|
unix socket.
|
|
|
|
## The two properties that constrain the code
|
|
|
|
**A token expires.** Authelia issues `client_credentials` access tokens with
|
|
`expires_in: 3599`. Authentication happens at CONNECT, so a long-lived
|
|
connection is fine — but a _reconnect_ an hour later needs a token minted an
|
|
hour later.
|
|
|
|
**The refresh therefore lives in the auth callback, not in a timer.**
|
|
`async-nats` invokes it per connection attempt, so there is no window in which
|
|
the client holds a token it minted for a previous connection. The alternative —
|
|
mint once, own the reconnect loop — fails in the way this subsystem exists to
|
|
prevent: the process keeps serving while its data quietly stops moving, and
|
|
nothing says so until someone reads a dashboard.
|
|
|
|
## Configuration
|
|
|
|
`QueueConfig::from_env(prefix)` reads `<prefix>_NATS_URL`,
|
|
`<prefix>_OIDC_TOKEN_ENDPOINT`, `<prefix>_OIDC_CLIENT_ID` and
|
|
`<prefix>_OIDC_CLIENT_SECRET_FILE`.
|
|
|
|
The prefix is a parameter because the variables belong to the consuming unit —
|
|
a NixOS module sets them alongside its other options. What is shared is the
|
|
rule, not the spelling: **all four together or none at all.** A half-set
|
|
environment is a hard error, because the failure it would otherwise produce is
|
|
the expensive kind — the process comes up "fine", never connects, and the data
|
|
it was supposed to move silently stops.
|
|
|
|
The client secret is a **path, not a value**: putting it in the environment
|
|
would publish it to anything that can read `/proc/<pid>/environ`. It is read
|
|
per token request rather than cached, so a rotation the operator believes took
|
|
effect actually did.
|
|
|
|
## Token request shape: HTTP Basic, `audience`, `scope`
|
|
|
|
`token_request` builds a `client_credentials` grant, authenticated with HTTP
|
|
Basic — **never** the form body. Both are legal OAuth 2.0
|
|
(`client_secret_basic` vs `client_secret_post`), but a client registration
|
|
names one, and authelia's default (and ours) is Basic. Sending the
|
|
credentials in the body once got every token request refused with `Client
|
|
authentication failed … the registered client is configured to only support
|
|
'client_secret_basic'`, which reached the operator as an endless `429`
|
|
because the retries tripped a rate limiter whose penalty grew faster than the
|
|
retry interval — the 429 arrived _before_ the credentials were ever
|
|
evaluated, so the line naming the real cause appeared once an hour. RFC 6749
|
|
§2.3.1 says clients SHOULD use Basic, both introspection callers in this
|
|
workspace already do, and a secret in a header is one fewer place for a proxy
|
|
to log it.
|
|
|
|
`audience` (RFC 8707 resource indicators) and `scope` are both opt-in;
|
|
`None` for either reproduces the request this crate sent before the
|
|
parameter existed. Omitted, `audience` gets whatever authelia defaults a
|
|
scopeless `client_credentials` grant to — the queue connection's own case,
|
|
which has always worked without asking. A caller proving this identity to a
|
|
specifically audience-checked receiver (the swarm-otel `oidc/swarm`
|
|
authenticator being the first one) has to ask by name, the same way
|
|
`swarm-otel.nix`'s own prometheus scrape config already does per target
|
|
(`endpoint_params.audience`): a token minted without asking carries `aud:
|
|
[]`, and an audience-checked receiver refuses that just as readily as the
|
|
wrong one. `scope` follows the same rule and travels with `audience`:
|
|
registration is not issuance, and the authorisation server grants no scope
|
|
the client never requested. A caller reaching a destination behind
|
|
authelia's `/api/authz/auth-request` needs `authelia.bearer.authz` here
|
|
however completely the client is registered for it — the failure
|
|
`swarm-otel.nix` records against its own client (a scopeless token refused
|
|
at introspection with "the requested scope is invalid, unknown, or
|
|
malformed") is the same one, one layer down.
|
|
|
|
## What this crate does not do
|
|
|
|
It ends at a connected client. `jetstream`/`kv` are **off by default** — what a
|
|
consumer does with the connection is its own business, and its `Cargo.toml` is
|
|
where that requirement should be visible. The auth-callout responder speaks the
|
|
connect and nothing else, and pays for nothing else.
|
|
|
|
## The one exception: the `kv` feature
|
|
|
|
`kv` adds `status`, which holds the name and the creation config of the
|
|
`hive-status` bucket — nothing more.
|
|
|
|
It is here because that bucket has **two ends in two crates**: a hive writes its
|
|
own key, the controller reads every key. The name being a repeated literal is
|
|
the mild half of the problem; the sharp half is that either end may arrive first
|
|
on a fresh swarm, so both create the bucket if it is missing. Two `Config`s that
|
|
drift means whichever end created it wins and the other opens a bucket it did
|
|
not ask for — no error, no log, just a retention policy nobody chose.
|
|
|
|
An agreement between two crates has to live in one of them, and neither end of
|
|
this bucket is senior to the other. Behind a default-off feature, the consumer
|
|
that needs none of it still pays nothing.
|