8be4030d03
Any request on logs.<swarm> carrying a well-formed swarm-agent bearer made swarm-logs-auth do a bao cert login and KV read before it could refuse, with no cap on the rate. nginx now applies limit_req (zone swarm_logs_agent_auth, keyed by client address, 5r/s burst 20 nodelay by default) on the internal location that forwards to swarm-logs-auth, answering 403 when over, since auth_request passes through only 401/403 and swarm-logs retries a 401. The authelia path and the query location carry no limit. swarm-logs-auth remembers bearers it has granted, keyed by SHA-256 of the bearer, for --verdict-ttl seconds (deploy.victorialogs.authVerdictTtl, 60 by default), holding at most 4096 and evicting the oldest. Denials are never remembered. A replaced or deleted queue secret therefore keeps working at the log store for up to the TTL after its last check. Closes #5190
3.8 KiB
swarm-logs
An agent's CLI for the swarm log store. One verb — swarm-logs query '<LogsQL>' — so reading logs pipes and greps like any other command
instead of being a hand-rolled token request plus a curl, per query.
Read-only and query-only. There is no ingest path here (that is the
collector's), and no tail: streaming is a different LogsQL endpoint with a
different response shape, left for a follow-up rather than folded in.
Distinct from hive-metric, the other half of the same surface from an
agent's seat: that one writes a custom metric, this one reads logs.
⚠️ There is no scoping
The gateway forwards the query unmodified, so any authenticated caller reads the whole swarm's logs — every hive's, not only its own. That is the rule in force rather than an omission here; when read permissions exist they attach at the gateway location, not in this binary.
How it authenticates
As the agent itself. On each query it reads the agent's own queue secret,
swarm/agents/<agent>/queue, from the swarm secret store with the agent's
store certificate (nix/agent-modules/bao.nix), the same read hive-agent
makes for its queue connection. It sends the secret as
Authorization: Bearer swarm-agent.<agent>.<secret>, spelled by
swarm_queue_client::agent_token.
At the gateway, nginx hands a bearer of that shape to swarm-logs-auth, which
compares it with the stored secret (swarm-agent-verify, the check the queue's
auth callout also uses). A 401 makes swarm-logs read the secret again and
retry once, which covers a secret re-minted between the read and the check.
swarm-logs-auth grants a secret it has verified for 60 s by default without asking
the store again, so a replaced secret also keeps working for up to that long.
A 403 is the gateway's per-address rate limit on these checks; it is not
retried.
🩸 The secret is never logged or rendered into an error.
Configuration
Supplied by nix/agent-modules/logs.nix, which wraps the binary — the same
shape swarmctl is configured in, and for the same reason: every value is
derived from an option that module owns, so a default here would be an address
we hope points at something.
| variable | what |
|---|---|
HIVE_AGENT_LOGS_QUERY_URL |
the LogsQL query endpoint |
HIVE_AGENT_NAME |
this agent's name; the store path and role |
BAO_ADDR |
the secret store's address |
BAO_CLIENT_CERT |
path of the agent's live store certificate |
BAO_CLIENT_KEY |
path of its key |
BAO_CACERT |
path of the store's CA; missing or empty is fine |
With no HIVE_AGENT_LOGS_QUERY_URL the binary says there is no log store;
with the URL and no store address it reports a deployment bug by name.
Output
$ swarm-logs query 'atlas-otel-probe' --limit 5
One log message per line by default (the record's _msg), which is what a
grep pattern is written against. --format json passes the store's NDJSON
through unmodified, for _time, _stream and jq.
A record the projection cannot understand — no string _msg, or not JSON at
all — is printed whole rather than dropped. Silently losing a row would make
an incomplete answer indistinguishable from a complete one.
A non-200 is reported with its body and a non-zero exit, never as an empty result. That is the whole reason the gateway grew a separate machine route: the operator's browser route answers an unauthenticated caller with authelia's login page as a 200 with an HTML body, and a client that reads only the status code records a query that succeeded and matched nothing.