d8f6d99bf9
`swarm-logs query` got a bare nginx 401 from the swarm log store on every query. The agent OIDC client is registered for `authelia.bearer.authz` (`swarm-authelia.nix`'s `agentClients` sets `bearerAuthz`), but registration is not issuance: the token request asked for no scope, so the token came back carrying none, and authelia's `/api/authz/auth-request` refuses that exactly as it refuses an unauthenticated caller. The same failure is already recorded in `swarm-otel.nix` against the collector's client, on the same scope string — prometheus asks for no scopes unless told to, and every scrape was refused at introspection. This is that bug one layer down, so it gets the same shape of fix. `scope` becomes an opt-in parameter alongside `audience`, not a hardcoded value or a config field: the two travel together (registered ≠ requested applies to both) and only the destination decides whether either is needed. `None` keeps every other caller byte-identical — the NATS connect callback, `auth.rs`'s bridge client and the OTLP push client all pass it. Refs #4464
97 lines
4.9 KiB
Markdown
97 lines
4.9 KiB
Markdown
# swarm-logs
|
|
|
|
An agent's CLI for the swarm log store. One verb — `swarm-logs query
|
|
'<LogsQL>'` — so reading logs pipes and greps like any other command
|
|
instead of being a hand-rolled token request plus a curl, per query.
|
|
|
|
Read-only and query-only. There is no ingest path here (that is the
|
|
collector's), and no `tail`: streaming is a different LogsQL endpoint with a
|
|
different response shape, left for a follow-up rather than folded in.
|
|
|
|
Distinct from `hive-metric`, the other half of the same surface from an
|
|
agent's seat: that one _writes_ a custom metric, this one _reads_ logs.
|
|
|
|
## ⚠️ There is no scoping
|
|
|
|
The gateway forwards the query unmodified, so **any authenticated caller
|
|
reads the whole swarm's logs** — every hive's, not only its own. That is the
|
|
rule in force rather than an omission here; when read permissions exist they
|
|
attach at the gateway location, not in this binary.
|
|
|
|
## How it authenticates
|
|
|
|
The agent container already holds one identity, the per-hive machine client
|
|
`hive-<name>-agent`, delivered as a systemd credential pair. This binary
|
|
presents a `client_credentials` access token minted from it as
|
|
`Authorization: Bearer`.
|
|
|
|
Minting is **not** implemented here — `swarm_queue_client::mint_token_for_blocking`
|
|
already owns the request, its error type and its CA handling, and a
|
|
token-endpoint fix has to be findable in one place. What this crate adds is
|
|
the agent-shaped half: the client id arrives as a _file_ beside the secret, so
|
|
nothing outside `nix/agent-modules/queue.nix` spells `hive-<name>-agent`
|
|
twice. That is the same problem `hive-agent/src/swarm_queue.rs` solves, and
|
|
`src/auth.rs` is its `decide` restated over this binary's inputs.
|
|
|
|
🩸 Every credential here is a **path**. The secret's contents are read at the
|
|
moment of the request, inside `swarm-queue-client`, and are never bound to a
|
|
name in this crate, logged, or rendered into an error.
|
|
|
|
### The audience is the URL
|
|
|
|
`swarm-logs` mints its token with the query URL as the requested audience,
|
|
because that is what authelia checks at the authz endpoint the gateway's
|
|
`auth_request` calls. The same rule `swarm-otel.nix` states over its own push
|
|
targets: one binding for the address and the audience, since two spellings of
|
|
one address present as a valid token refused at the store.
|
|
|
|
⚠️ That requires the agent client to be _registered_ for that audience and to
|
|
hold `authelia.bearer.authz`. Both are set in `swarm-authelia.nix`'s
|
|
`agentClients`; without them authelia answers `invalid_target` at the token
|
|
endpoint, or the gateway answers 401 with no explanation.
|
|
|
|
⚠️ Registration is not issuance, so the token request _asks_ for both: the
|
|
audience and the scope are named in the `client_credentials` form, because a
|
|
client that is registered for a scope it does not request is handed a token
|
|
carrying none, and the gateway refuses that with the same bare 401.
|
|
|
|
## Configuration
|
|
|
|
Supplied by `nix/agent-modules/logs.nix`, which wraps the binary — the same
|
|
shape `swarmctl` is configured in, and for the same reason: every value is
|
|
derived from an option that module owns, so a default here would be an address
|
|
we _hope_ points at something.
|
|
|
|
| variable | what |
|
|
| ------------------------------------ | ---------------------------------------------------- |
|
|
| `HIVE_AGENT_LOGS_QUERY_URL` | the LogsQL query endpoint; also the token's audience |
|
|
| `HIVE_AGENT_OIDC_TOKEN_ENDPOINT` | authelia's token endpoint |
|
|
| `HIVE_AGENT_OIDC_CLIENT_ID_FILE` | path of the delivered client-id credential |
|
|
| `HIVE_AGENT_OIDC_CLIENT_SECRET_FILE` | path of the delivered secret credential |
|
|
| `HIVE_AGENT_OIDC_CA_FILE` | optional extra trust anchor; unset in this tree |
|
|
|
|
`HIVE_AGENT`, the same prefix the harness reads, because it is the same
|
|
identity — an agent authenticates as its hive's agent client whether the
|
|
caller is the harness or a CLI the agent typed. The first four are all-or-none:
|
|
a half-set environment is a deployment bug, and this binary says so by name
|
|
rather than behaving like an unconfigured one.
|
|
|
|
## Output
|
|
|
|
```console
|
|
$ swarm-logs query 'atlas-otel-probe' --limit 5
|
|
```
|
|
|
|
One log message per line by default (the record's `_msg`), which is what a
|
|
grep pattern is written against. `--format json` passes the store's NDJSON
|
|
through unmodified, for `_time`, `_stream` and `jq`.
|
|
|
|
A record the projection cannot understand — no string `_msg`, or not JSON at
|
|
all — is printed whole rather than dropped. Silently losing a row would make
|
|
an incomplete answer indistinguishable from a complete one.
|
|
|
|
A non-200 is reported with its body and a non-zero exit, never as an empty
|
|
result. That is the whole reason the gateway grew a separate machine route:
|
|
the operator's browser route answers an unauthenticated caller with authelia's
|
|
login page as a **200 with an HTML body**, and a client that reads only the
|
|
status code records a query that succeeded and matched nothing.
|