Watch
0
0
Fork
You've already forked hyperhive
0

egress dashboard: tabs like the agents board, with totals on the first

The board was one page of would-deny panels. It now has three tabs: overview
(requests, agents active, refused, credential challenges, data transferred,
status over time, busiest agents and destinations), refused (the hosts each
agent was refused) and would-deny (the old panels). The module-eval check pins
the tab layout and both squid log layouts the queries parse.
This commit is contained in:
iris 2026-10-30 16:10:02 +01:00
commit 6b0c06800b
5 changed files with 1860 additions and 276 deletions

View file

@ -301,7 +301,7 @@ host. It never decrypts TLS, so it holds no signing key. It caches nothing.
rule lets through gets a second log line,
`would_deny user=<agent> method=<m> host=<h> port=<p> result=<r>`, from the
`would_denied` `access_log` in `hive-squid.nix`; the usual request line keeps
its format. The `egress` dashboard in Grafana lists the hosts per agent. Only
its format. The **would-deny** tab of the `egress` dashboard in Grafana lists the hosts per agent. Only
the operator changes the mode: nothing moves an agent to `restricted` after a
time or a count. The self-check's users aren't agents,
so `deny-canary.invalid` stays a 403.

View file

@ -203,7 +203,7 @@ kept.
- `agents`: per-agent resource usage and Claude Code activity.
- `authelia`, `bao`, `queue`: the identity provider, the secret store and the NATS queue.
- `claude-usage`: Claude use by model, effort, token type and source. Filter by hive, service (agents or the review classifier) and agent. The service filter separates the classifier's API-key spend from the agents.
- `egress`: the requests that agents in `log` mode made to hosts restricted mode would refuse, by agent and host. Filter by agent.
- `egress`: what agents send through the egress proxy, in three tabs. **overview** has the totals (requests, agents active, refused, credential challenges, data transferred), requests over time by status, and the busiest agents and destinations. **refused** lists the hosts the proxy refused for each agent. **would-deny** lists what agents in `log` mode reached that restricted mode would refuse. Filter by agent.
- `controller`: the swarm controller's job graph by state, and the push and commit rates it sees on the forge.
- `review-classifier`: review requests by outcome, the verdict mix, results per check, retried asks, and run duration p50 and p95.
- `forge`: what the forge holds, and the forge process's CPU, memory and descriptors.

View file

@ -592,8 +592,8 @@ modes (**allow unrestricted**, **log**, **restrict**) set the mode with a
`PUT` without the list resets a stored list to the default. An agent with no
stored list sends the flags alone. `log` sends `unrestricted_internet: false`
with `internet_log: true`; the other two leave `internet_log` out, which ends
logging. The **egress** dashboard in Grafana shows the hosts a `log` agent
reached.
logging. The **would-deny** tab of the **egress** dashboard in Grafana shows the hosts a
`log` agent reached.
The hive's egress proxy applies the flag to each agent's proxy credential (see
[the egress proxy](../networking/network.md#egress-proxy-squid)). An agent sends its
credential from its next start after it has fetched it, so a flag change applies

File diff suppressed because it is too large Load diff

View file

@ -453,25 +453,43 @@ let
&& lib.all (m: lib.hasInfix "\"${m}\"" recorded) queried;
}
{
# The egress board ships, and the `extract` pattern its queries use is the
# field layout of the `would_deny` line squid logs: a changed `logformat`
# is a board of empty panels.
name = "the egress dashboard ships, and reads the field layout of squid's would_deny log line";
# The egress board ships, and its queries read the two squid log lines as
# squid writes them: the would_deny line's field layout, and the default
# `squid` access format the overview and refused tabs parse. A changed
# `logformat` or `access_log` is a board of empty panels.
name = "the egress dashboard ships, and reads the field layout of squid's log lines";
ok =
let
provider = lib.head grafanaWithAuthelia.containers.swarm-grafana.config.services.grafana.provision.dashboards.settings.providers;
shipped = lib.attrNames provider.options.path.entries;
board = builtins.fromJSON (builtins.readFile ../host-modules/swarm-grafana/dashboards/egress.json);
exprs = lib.concatMap (p: map (t: t.expr or "") (p.targets or [ ])) board.panels;
pattern = "would_deny user=<agent> method=<method> host=<dest> port=<port> result=<result>";
agentVar = lib.findFirst (v: v.name == "agent") null board.templating.list;
exprs = map (e: (lib.head e.spec.data.spec.queries).spec.query.spec.expr) (
lib.attrValues board.spec.elements
);
tabs = map (t: t.spec.title) board.spec.layout.spec.tabs;
wouldDeny = "would_deny user=<agent> method=<method> host=<dest> port=<port> result=<result>";
access = "(?P<result>TCP_[A-Z_]+)/(?P<status>[0-9]{3}) (?P<bytes>[0-9]+) (?P<method>[A-Z]+) (?P<url>\\\\S+) (?P<agent>\\\\S+)";
agentVar = lib.findFirst (v: v.spec.name == "agent") null board.spec.variables;
squid = builtins.readFile ../host-modules/hive-squid.nix;
in
lib.elem "egress.json" shipped
&& lib.length (lib.filter (e: lib.hasInfix pattern e) exprs) == 5
&&
tabs == [
"overview"
"refused"
"would-deny"
]
# every panel sits in a tab, and every tab item names a panel
&&
lib.sort (a: b: a < b) (
lib.concatMap (t: map (i: i.spec.element.name) t.spec.layout.spec.items) board.spec.layout.spec.tabs
) == lib.sort (a: b: a < b) (lib.attrNames board.spec.elements)
&& lib.length (lib.filter (e: lib.hasInfix wouldDeny e) exprs) == 6
&& lib.length (lib.filter (e: lib.hasInfix access e) exprs) == 14
&& agentVar != null
&& lib.hasInfix pattern agentVar.query.query
&& lib.hasInfix "logformat would_deny would_deny user=%[un method=%rm host=%>rd port=%>rP result=" squid;
&& lib.hasInfix access agentVar.spec.query.spec.query
&& lib.hasInfix "logformat would_deny would_deny user=%[un method=%rm host=%>rd port=%>rP result=" squid
&& lib.hasInfix "access_log syslog:daemon.info squid !manager" squid;
}
{
# The standing check on the collector's PRIORITY mapping — whether a line