egress dashboard: tabs like the agents board, with totals on the first
The board was one page of would-deny panels. It now has three tabs: overview (requests, agents active, refused, credential challenges, data transferred, status over time, busiest agents and destinations), refused (the hosts each agent was refused) and would-deny (the old panels). The module-eval check pins the tab layout and both squid log layouts the queries parse.
This commit is contained in:
parent
9193f37963
commit
6b0c06800b
5 changed files with 1860 additions and 276 deletions
|
|
@ -301,7 +301,7 @@ host. It never decrypts TLS, so it holds no signing key. It caches nothing.
|
|||
rule lets through gets a second log line,
|
||||
`would_deny user=<agent> method=<m> host=<h> port=<p> result=<r>`, from the
|
||||
`would_denied` `access_log` in `hive-squid.nix`; the usual request line keeps
|
||||
its format. The `egress` dashboard in Grafana lists the hosts per agent. Only
|
||||
its format. The **would-deny** tab of the `egress` dashboard in Grafana lists the hosts per agent. Only
|
||||
the operator changes the mode: nothing moves an agent to `restricted` after a
|
||||
time or a count. The self-check's users aren't agents,
|
||||
so `deny-canary.invalid` stays a 403.
|
||||
|
|
|
|||
|
|
@ -203,7 +203,7 @@ kept.
|
|||
- `agents`: per-agent resource usage and Claude Code activity.
|
||||
- `authelia`, `bao`, `queue`: the identity provider, the secret store and the NATS queue.
|
||||
- `claude-usage`: Claude use by model, effort, token type and source. Filter by hive, service (agents or the review classifier) and agent. The service filter separates the classifier's API-key spend from the agents.
|
||||
- `egress`: the requests that agents in `log` mode made to hosts restricted mode would refuse, by agent and host. Filter by agent.
|
||||
- `egress`: what agents send through the egress proxy, in three tabs. **overview** has the totals (requests, agents active, refused, credential challenges, data transferred), requests over time by status, and the busiest agents and destinations. **refused** lists the hosts the proxy refused for each agent. **would-deny** lists what agents in `log` mode reached that restricted mode would refuse. Filter by agent.
|
||||
- `controller`: the swarm controller's job graph by state, and the push and commit rates it sees on the forge.
|
||||
- `review-classifier`: review requests by outcome, the verdict mix, results per check, retried asks, and run duration p50 and p95.
|
||||
- `forge`: what the forge holds, and the forge process's CPU, memory and descriptors.
|
||||
|
|
|
|||
|
|
@ -592,8 +592,8 @@ modes (**allow unrestricted**, **log**, **restrict**) set the mode with a
|
|||
`PUT` without the list resets a stored list to the default. An agent with no
|
||||
stored list sends the flags alone. `log` sends `unrestricted_internet: false`
|
||||
with `internet_log: true`; the other two leave `internet_log` out, which ends
|
||||
logging. The **egress** dashboard in Grafana shows the hosts a `log` agent
|
||||
reached.
|
||||
logging. The **would-deny** tab of the **egress** dashboard in Grafana shows the hosts a
|
||||
`log` agent reached.
|
||||
The hive's egress proxy applies the flag to each agent's proxy credential (see
|
||||
[the egress proxy](../networking/network.md#egress-proxy-squid)). An agent sends its
|
||||
credential from its next start after it has fetched it, so a flag change applies
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
|
|
@ -453,25 +453,43 @@ let
|
|||
&& lib.all (m: lib.hasInfix "\"${m}\"" recorded) queried;
|
||||
}
|
||||
{
|
||||
# The egress board ships, and the `extract` pattern its queries use is the
|
||||
# field layout of the `would_deny` line squid logs: a changed `logformat`
|
||||
# is a board of empty panels.
|
||||
name = "the egress dashboard ships, and reads the field layout of squid's would_deny log line";
|
||||
# The egress board ships, and its queries read the two squid log lines as
|
||||
# squid writes them: the would_deny line's field layout, and the default
|
||||
# `squid` access format the overview and refused tabs parse. A changed
|
||||
# `logformat` or `access_log` is a board of empty panels.
|
||||
name = "the egress dashboard ships, and reads the field layout of squid's log lines";
|
||||
ok =
|
||||
let
|
||||
provider = lib.head grafanaWithAuthelia.containers.swarm-grafana.config.services.grafana.provision.dashboards.settings.providers;
|
||||
shipped = lib.attrNames provider.options.path.entries;
|
||||
board = builtins.fromJSON (builtins.readFile ../host-modules/swarm-grafana/dashboards/egress.json);
|
||||
exprs = lib.concatMap (p: map (t: t.expr or "") (p.targets or [ ])) board.panels;
|
||||
pattern = "would_deny user=<agent> method=<method> host=<dest> port=<port> result=<result>";
|
||||
agentVar = lib.findFirst (v: v.name == "agent") null board.templating.list;
|
||||
exprs = map (e: (lib.head e.spec.data.spec.queries).spec.query.spec.expr) (
|
||||
lib.attrValues board.spec.elements
|
||||
);
|
||||
tabs = map (t: t.spec.title) board.spec.layout.spec.tabs;
|
||||
wouldDeny = "would_deny user=<agent> method=<method> host=<dest> port=<port> result=<result>";
|
||||
access = "(?P<result>TCP_[A-Z_]+)/(?P<status>[0-9]{3}) (?P<bytes>[0-9]+) (?P<method>[A-Z]+) (?P<url>\\\\S+) (?P<agent>\\\\S+)";
|
||||
agentVar = lib.findFirst (v: v.spec.name == "agent") null board.spec.variables;
|
||||
squid = builtins.readFile ../host-modules/hive-squid.nix;
|
||||
in
|
||||
lib.elem "egress.json" shipped
|
||||
&& lib.length (lib.filter (e: lib.hasInfix pattern e) exprs) == 5
|
||||
&&
|
||||
tabs == [
|
||||
"overview"
|
||||
"refused"
|
||||
"would-deny"
|
||||
]
|
||||
# every panel sits in a tab, and every tab item names a panel
|
||||
&&
|
||||
lib.sort (a: b: a < b) (
|
||||
lib.concatMap (t: map (i: i.spec.element.name) t.spec.layout.spec.items) board.spec.layout.spec.tabs
|
||||
) == lib.sort (a: b: a < b) (lib.attrNames board.spec.elements)
|
||||
&& lib.length (lib.filter (e: lib.hasInfix wouldDeny e) exprs) == 6
|
||||
&& lib.length (lib.filter (e: lib.hasInfix access e) exprs) == 14
|
||||
&& agentVar != null
|
||||
&& lib.hasInfix pattern agentVar.query.query
|
||||
&& lib.hasInfix "logformat would_deny would_deny user=%[un method=%rm host=%>rd port=%>rP result=" squid;
|
||||
&& lib.hasInfix access agentVar.spec.query.spec.query
|
||||
&& lib.hasInfix "logformat would_deny would_deny user=%[un method=%rm host=%>rd port=%>rP result=" squid
|
||||
&& lib.hasInfix "access_log syslog:daemon.info squid !manager" squid;
|
||||
}
|
||||
{
|
||||
# The standing check on the collector's PRIORITY mapping — whether a line
|
||||
|
|
|
|||
Loading…
Reference in a new issue