8be4030d03
Any request on logs.<swarm> carrying a well-formed swarm-agent bearer made swarm-logs-auth do a bao cert login and KV read before it could refuse, with no cap on the rate. nginx now applies limit_req (zone swarm_logs_agent_auth, keyed by client address, 5r/s burst 20 nodelay by default) on the internal location that forwards to swarm-logs-auth, answering 403 when over, since auth_request passes through only 401/403 and swarm-logs retries a 401. The authelia path and the query location carry no limit. swarm-logs-auth remembers bearers it has granted, keyed by SHA-256 of the bearer, for --verdict-ttl seconds (deploy.victorialogs.authVerdictTtl, 60 by default), holding at most 4096 and evicting the oldest. Denials are never remembered. A replaced or deleted queue secret therefore keeps working at the log store for up to the TTL after its last check. Closes #5190
78 lines
3.8 KiB
Markdown
78 lines
3.8 KiB
Markdown
# swarm-logs
|
|
|
|
An agent's CLI for the swarm log store. One verb — `swarm-logs query
|
|
'<LogsQL>'` — so reading logs pipes and greps like any other command
|
|
instead of being a hand-rolled token request plus a curl, per query.
|
|
|
|
Read-only and query-only. There is no ingest path here (that is the
|
|
collector's), and no `tail`: streaming is a different LogsQL endpoint with a
|
|
different response shape, left for a follow-up rather than folded in.
|
|
|
|
Distinct from `hive-metric`, the other half of the same surface from an
|
|
agent's seat: that one _writes_ a custom metric, this one _reads_ logs.
|
|
|
|
## ⚠️ There is no scoping
|
|
|
|
The gateway forwards the query unmodified, so **any authenticated caller
|
|
reads the whole swarm's logs** — every hive's, not only its own. That is the
|
|
rule in force rather than an omission here; when read permissions exist they
|
|
attach at the gateway location, not in this binary.
|
|
|
|
## How it authenticates
|
|
|
|
As the agent itself. On each query it reads the agent's own queue secret,
|
|
`swarm/agents/<agent>/queue`, from the swarm secret store with the agent's
|
|
store certificate (`nix/agent-modules/bao.nix`), the same read `hive-agent`
|
|
makes for its queue connection. It sends the secret as
|
|
`Authorization: Bearer swarm-agent.<agent>.<secret>`, spelled by
|
|
`swarm_queue_client::agent_token`.
|
|
|
|
At the gateway, nginx hands a bearer of that shape to `swarm-logs-auth`, which
|
|
compares it with the stored secret (`swarm-agent-verify`, the check the queue's
|
|
auth callout also uses). A 401 makes `swarm-logs` read the secret again and
|
|
retry once, which covers a secret re-minted between the read and the check.
|
|
`swarm-logs-auth` grants a secret it has verified for 60 s by default without asking
|
|
the store again, so a replaced secret also keeps working for up to that long.
|
|
A 403 is the gateway's per-address rate limit on these checks; it is not
|
|
retried.
|
|
|
|
🩸 The secret is never logged or rendered into an error.
|
|
|
|
## Configuration
|
|
|
|
Supplied by `nix/agent-modules/logs.nix`, which wraps the binary — the same
|
|
shape `swarmctl` is configured in, and for the same reason: every value is
|
|
derived from an option that module owns, so a default here would be an address
|
|
we _hope_ points at something.
|
|
|
|
| variable | what |
|
|
| --------------------------- | ------------------------------------------------ |
|
|
| `HIVE_AGENT_LOGS_QUERY_URL` | the LogsQL query endpoint |
|
|
| `HIVE_AGENT_NAME` | this agent's name; the store path and role |
|
|
| `BAO_ADDR` | the secret store's address |
|
|
| `BAO_CLIENT_CERT` | path of the agent's live store certificate |
|
|
| `BAO_CLIENT_KEY` | path of its key |
|
|
| `BAO_CACERT` | path of the store's CA; missing or empty is fine |
|
|
|
|
With no `HIVE_AGENT_LOGS_QUERY_URL` the binary says there is no log store;
|
|
with the URL and no store address it reports a deployment bug by name.
|
|
|
|
## Output
|
|
|
|
```console
|
|
$ swarm-logs query 'atlas-otel-probe' --limit 5
|
|
```
|
|
|
|
One log message per line by default (the record's `_msg`), which is what a
|
|
grep pattern is written against. `--format json` passes the store's NDJSON
|
|
through unmodified, for `_time`, `_stream` and `jq`.
|
|
|
|
A record the projection cannot understand — no string `_msg`, or not JSON at
|
|
all — is printed whole rather than dropped. Silently losing a row would make
|
|
an incomplete answer indistinguishable from a complete one.
|
|
|
|
A non-200 is reported with its body and a non-zero exit, never as an empty
|
|
result. That is the whole reason the gateway grew a separate machine route:
|
|
the operator's browser route answers an unauthenticated caller with authelia's
|
|
login page as a **200 with an HTML body**, and a client that reads only the
|
|
status code records a query that succeeded and matched nothing.
|