vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

Compare commits

base: vinzenz:638e1ac2c6863c49427caf4e8a6cfd491b3d9fef
Branches Tags
vinzenz:main
...
compare: vinzenz:20135962deba8c9267c1e7ade7f9aefb27223f68
Branches Tags
vinzenz:main
Author SHA1 Message Date
atlas
20135962de grafana: rank the by-label panel as bars, and stop calling it open issues 
Two findings on one panel, both from mara.

Form: 24 label series drawn as overlapping timeseries lines is
unreadable by construction. The panel is a current-count ranking, not a
time evolution, so it becomes a horizontal bar gauge on an instant
sort_desc query -- identity on the axis, magnitude in the bar, and a
fixed colour because colour was carrying no information that the axis
was not already carrying.

Title: the metric counts open AND closed issues. Measured, not assumed
-- the largest single label reads 1619 while gitea_issues_open reads
117, and no label can outnumber the set it is a subset of. The panel had
been asserting the opposite since it was written.
 2026-08-28 13:23:25 +02:00
atlas
21d2554d32 grafana: stop describing this hive's habits on a board every hive gets 
mara on the PR: "many hint texts are incorrect, most hives dont work on
hyperhive itself". Four panel descriptions asserted local practice as if
it were general:

  Releases            "expected to sit at zero on a hive that ships from
                      main" -- a claim about how THIS project releases
  Automation surface  "every hive registering the same webhook has bitten
                      this forge before" -- a war story from this
                      deployment, meaningless on any other
  Organizations       "near-constant by design" -- assumes our org layout
  Issues open/closed  "usually a bulk triage, not a productive afternoon"
                      -- a claim about our working rhythm

Each now describes what the metric IS and what a CHANGE means, and says
outright where the panel cannot know: whether zero releases is normal
depends on how a hive's projects ship, and a vertical step in closed is
some batch operation that this panel cannot distinguish from a burst of
real work.

The board title is NOT an instance of this and is unchanged -- all seven
dashboards use the 'hyperhive - <thing>' product namespace, checked
rather than assumed.
 2026-08-28 13:23:25 +02:00
atlas
05552e5115 grafana: the by-label panel has data, so drop its empty-panel warning 
The forge restarted (new forgejo PID at 12:36), so it re-read [metrics]
and gitea_issues_by_label now exists -- 24 series carrying real label
names, read back out of the store rather than inferred from the metric
name appearing in a label-values list.

The panel shipped deliberately empty with EMPTY UNTIL THE FORGE RESTARTS
in its title and a matching noValue, because an empty graph and an
undeployed metric look identical. That warning is now false, and a stale
warning is worse than none: it tells a reader the panel cannot have data
at the moment it does.

Keeps the operational half rather than deleting it -- the restart
requirement is a fact about the NEXT metric someone enables, so it moves
into the description as a note and into noValue, where it fires only when
the series really is missing.
 2026-08-28 13:23:25 +02:00
atlas
6f248a12be grafana: a forge board, honest about being an inventory board 
Adds the fourth per-service board. Unlike its three siblings it answers
"what does the forge CONTAIN", not "is the forge healthy" -- forgejo's
exporter publishes a census of stored objects and no request, error or
latency series at all.

That distinction is the board's own description rather than something a
reader has to infer, because a fourth per-service board that silently
answers a different question than the other three is worse than no board:
someone checks it during an incident and reads a flat line as calm.

Panels: repositories, users, orgs, issues open/closed, releases, the
open-vs-closed pair over time, repos-and-users together (they move in
lockstep when an agent is created, so divergence is the signal),
discussion volume, and the webhook/hooktask pair.

One panel ships deliberately EMPTY: issues-by-label. The metric is
enabled in the forge's config but forgejo reads its [metrics] section at
startup, so the series does not exist until that process restarts -- and
a config merge alone does not restart it. The panel title says so, the
description says how to tell "not deployed" from "no labelled issues",
and `noValue` says it again in the graph itself. An empty panel with no
explanation is the failure this board is trying not to be.

Gated: every panel's metric confirmed to exist in the store, with a
bogus name checked absent in the same run so the check can fail, plus a
known-positive control against the label-values query itself -- a
download that silently returned nothing would otherwise report every
metric missing. Gate kept as verify-3494-forge-board.sh.
 2026-08-28 13:23:25 +02:00
2 changed files with 415 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

1
nix/host-modules/swarm-grafana.nix
View file

@ -61,6 +61,7 @@ let
"agents.json" "agents.json"
"authelia.json" "authelia.json"
"claude-usage.json" "claude-usage.json"
"forge.json"
"logstore.json" "logstore.json"
"metricstore.json" "metricstore.json"
"queue.json" "queue.json"

414
nix/host-modules/swarm-grafana/dashboards/forge.json Normal file
View file

@ -0,0 +1,414 @@
{
"title": "hyperhive · forge (content)",
"uid": "hyperhive-swarm-forge",
"description": "An INVENTORY board, not a service-health board — and the difference is the first thing to know about it. Its three siblings (metrics store, log store, authelia) answer 'is this service healthy': request rates, error rates, latency. Forgejo's exporter publishes no such series; what it publishes is a census of what the forge CONTAINS. So every number here is a gauge of stored things, sampled at scrape time. Nothing on this board can tell you the forge is slow or erroring, and a flat line means nobody created anything, not that scraping stopped — check the metrics-store board for that. Series are labelled service.name, NOT job; there is no job label in this store.",
"editable": true,
"refresh": "5m",
"schemaVersion": 39,
"tags": ["hyperhive", "swarm", "forge", "content"],
"time": { "from": "now-7d", "to": "now" },
"timezone": "utc",
"version": 1,
"templating": { "list": [] },
"panels": [
{
"id": 1,
"type": "stat",
"title": "Repositories",
"description": "Every repo the forge holds, including agent config repos — which is most of them on a busy hive, so this tracks agent count more closely than it tracks project count.",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"gridPos": { "h": 4, "w": 4, "x": 0, "y": 0 },
"targets": [
{
"refId": "A",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"expr": "gitea_repositories",
"instant": true,
"legendFormat": "repos"
}
],
"fieldConfig": {
"defaults": {
"unit": "short",
"decimals": 0,
"color": { "mode": "fixed", "fixedColor": "text" },
"mappings": []
},
"overrides": []
},
"options": {
"graphMode": "none",
"colorMode": "value",
"textMode": "value",
"justifyMode": "auto",
"reduceOptions": { "calcs": ["lastNotNull"], "fields": "", "values": false }
}
},
{
"id": 2,
"type": "stat",
"title": "Users",
"description": "Forge accounts. Agents hold one each, so this is operators plus the live agent roster — a drop without a matching agent teardown is worth a look.",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"gridPos": { "h": 4, "w": 4, "x": 4, "y": 0 },
"targets": [
{
"refId": "A",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"expr": "gitea_users",
"instant": true,
"legendFormat": "users"
}
],
"fieldConfig": {
"defaults": {
"unit": "short",
"decimals": 0,
"color": { "mode": "fixed", "fixedColor": "text" },
"mappings": []
},
"overrides": []
},
"options": {
"graphMode": "none",
"colorMode": "value",
"textMode": "value",
"justifyMode": "auto",
"reduceOptions": { "calcs": ["lastNotNull"], "fields": "", "values": false }
}
},
{
"id": 3,
"type": "stat",
"title": "Organizations",
"description": "Orgs — the forge's coarsest grouping, and on a hyperhive forge the line between agent-owned repos and everything else. A change is a structural decision someone made rather than activity; how often that happens is a fact about this hive's projects, not something the panel can assume.",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"gridPos": { "h": 4, "w": 4, "x": 8, "y": 0 },
"targets": [
{
"refId": "A",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"expr": "gitea_organizations",
"instant": true,
"legendFormat": "orgs"
}
],
"fieldConfig": {
"defaults": {
"unit": "short",
"decimals": 0,
"color": { "mode": "fixed", "fixedColor": "text" },
"mappings": []
},
"overrides": []
},
"options": {
"graphMode": "none",
"colorMode": "value",
"textMode": "value",
"justifyMode": "auto",
"reduceOptions": { "calcs": ["lastNotNull"], "fields": "", "values": false }
}
},
{
"id": 4,
"type": "stat",
"title": "Issues open",
"description": "Open issues across every repo. The one number on this board that behaves like a backlog gauge — rising means work is being filed faster than it is closed.",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"gridPos": { "h": 4, "w": 4, "x": 12, "y": 0 },
"targets": [
{
"refId": "A",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"expr": "gitea_issues_open",
"instant": true,
"legendFormat": "open"
}
],
"fieldConfig": {
"defaults": {
"unit": "short",
"decimals": 0,
"color": { "mode": "fixed", "fixedColor": "text" },
"mappings": []
},
"overrides": []
},
"options": {
"graphMode": "none",
"colorMode": "value",
"textMode": "value",
"justifyMode": "auto",
"reduceOptions": { "calcs": ["lastNotNull"], "fields": "", "values": false }
}
},
{
"id": 5,
"type": "stat",
"title": "Issues closed",
"description": "Cumulative closed issues. Only ever rises, so read it as throughput-to-date rather than as a current state.",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"gridPos": { "h": 4, "w": 4, "x": 16, "y": 0 },
"targets": [
{
"refId": "A",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"expr": "gitea_issues_closed",
"instant": true,
"legendFormat": "closed"
}
],
"fieldConfig": {
"defaults": {
"unit": "short",
"decimals": 0,
"color": { "mode": "fixed", "fixedColor": "text" },
"mappings": []
},
"overrides": []
},
"options": {
"graphMode": "none",
"colorMode": "value",
"textMode": "value",
"justifyMode": "auto",
"reduceOptions": { "calcs": ["lastNotNull"], "fields": "", "values": false }
}
},
{
"id": 6,
"type": "stat",
"title": "Releases",
"description": "Tagged releases across every repo. Cumulative, so it only rises; the rate is the signal, not the level. Whether zero is normal here depends entirely on how this hive's projects ship — a board shipped to every hive cannot know that, so read a CHANGE rather than a value.",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"gridPos": { "h": 4, "w": 4, "x": 20, "y": 0 },
"targets": [
{
"refId": "A",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"expr": "gitea_releases",
"instant": true,
"legendFormat": "releases"
}
],
"fieldConfig": {
"defaults": {
"unit": "short",
"decimals": 0,
"color": { "mode": "fixed", "fixedColor": "text" },
"mappings": []
},
"overrides": []
},
"options": {
"graphMode": "none",
"colorMode": "value",
"textMode": "value",
"justifyMode": "auto",
"reduceOptions": { "calcs": ["lastNotNull"], "fields": "", "values": false }
}
},
{
"id": 10,
"type": "timeseries",
"title": "Issues: open vs closed",
"description": "The two together, because either alone misleads. Open flat while closed climbs is a backlog being worked down; both climbing together is a backlog growing while work happens. ⚠️ A vertical step in closed is a batch operation of some kind — a bulk close, an import, a repo removal — and is not distinguishable here from a burst of real work; the issue list is what tells them apart.",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 4 },
"targets": [
{
"refId": "A",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"expr": "gitea_issues_open",
"legendFormat": "open"
},
{
"refId": "B",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"expr": "gitea_issues_closed",
"legendFormat": "closed"
}
],
"fieldConfig": {
"defaults": {
"unit": "short",
"decimals": 0,
"color": { "mode": "palette-classic" },
"custom": {
"drawStyle": "line",
"lineWidth": 1,
"fillOpacity": 10,
"showPoints": "never",
"spanNulls": false,
"axisSoftMin": 0
}
},
"overrides": []
},
"options": {
"legend": { "displayMode": "list", "placement": "bottom", "showLegend": true, "calcs": [] },
"tooltip": { "mode": "multi", "sort": "desc" }
}
},
{
"id": 11,
"type": "timeseries",
"title": "Repositories and users",
"description": "Both climb when an agent is created — a config repo and an account arrive together. So the interesting reading is when they DIVERGE: repos without users means repos created by hand, users without repos means accounts provisioned for something that never got one.",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 4 },
"targets": [
{
"refId": "A",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"expr": "gitea_repositories",
"legendFormat": "repos"
},
{
"refId": "B",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"expr": "gitea_users",
"legendFormat": "users"
}
],
"fieldConfig": {
"defaults": {
"unit": "short",
"decimals": 0,
"color": { "mode": "palette-classic" },
"custom": {
"drawStyle": "line",
"lineWidth": 1,
"fillOpacity": 0,
"showPoints": "never",
"spanNulls": false,
"axisSoftMin": 0
}
},
"overrides": []
},
"options": {
"legend": { "displayMode": "list", "placement": "bottom", "showLegend": true, "calcs": [] },
"tooltip": { "mode": "multi", "sort": "desc" }
}
},
{
"id": 20,
"type": "timeseries",
"title": "Discussion volume",
"description": "Comments and review comments across the forge. The closest thing here to an activity signal, and still a cumulative count rather than a rate — a steepening slope is more conversation, a flat stretch is a quiet period, and neither says anything about whether the forge is up.",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"gridPos": { "h": 7, "w": 12, "x": 0, "y": 12 },
"targets": [
{
"refId": "A",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"expr": "gitea_comments",
"legendFormat": "comments"
},
{
"refId": "B",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"expr": "gitea_attachments",
"legendFormat": "attachments"
}
],
"fieldConfig": {
"defaults": {
"unit": "short",
"decimals": 0,
"color": { "mode": "palette-classic" },
"custom": {
"drawStyle": "line",
"lineWidth": 1,
"fillOpacity": 10,
"showPoints": "never",
"spanNulls": false,
"axisSoftMin": 0
}
},
"overrides": []
},
"options": {
"legend": { "displayMode": "list", "placement": "bottom", "showLegend": true, "calcs": [] },
"tooltip": { "mode": "multi", "sort": "desc" }
}
},
{
"id": 21,
"type": "timeseries",
"title": "Automation surface",
"description": "Webhooks and their queued delivery tasks. Worth a glance when something that should have triggered did not: a hooktask count that climbs and never settles is deliveries backing up rather than being consumed. The webhook count itself is whatever this hive has wired — it says nothing on its own, and a jump in it is someone's integration change.",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"gridPos": { "h": 7, "w": 12, "x": 12, "y": 12 },
"targets": [
{
"refId": "A",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"expr": "gitea_webhooks",
"legendFormat": "webhooks"
},
{
"refId": "B",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"expr": "gitea_hooktasks",
"legendFormat": "hooktasks"
}
],
"fieldConfig": {
"defaults": {
"unit": "short",
"decimals": 0,
"color": { "mode": "palette-classic" },
"custom": {
"drawStyle": "line",
"lineWidth": 1,
"fillOpacity": 0,
"showPoints": "never",
"spanNulls": false,
"axisSoftMin": 0
}
},
"overrides": []
},
"options": {
"legend": { "displayMode": "list", "placement": "bottom", "showLegend": true, "calcs": [] },
"tooltip": { "mode": "multi", "sort": "desc" }
}
},
{
"id": 30,
"type": "bargauge",
"title": "Issues by label — every issue ever, ranked",
"description": "How the whole issue corpus is distributed across labels, biggest first. NOT the open backlog: this metric counts open and closed issues alike, which is checkable from this board alone — the top label here is far larger than the Issues open stat beside it, and no single label can outnumber the issues it is a subset of. So read it as the shape of what the project has worked on, not as what is left. It is also a RANKING of one moment rather than a time series: the label set is as wide as the tracker's, so one line per label is unreadable by construction, and the history of any single label is an ad-hoc query, not a panel. Colour carries no meaning here; the axis carries the identity. If the panel is empty, check whether the forgejo process restarted since [metrics] last changed — a merged config alone does not enable a series, and an absent series never says why it is absent.",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"gridPos": { "h": 16, "w": 24, "x": 0, "y": 19 },
"targets": [
{
"refId": "A",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"expr": "sort_desc(gitea_issues_by_label)",
"legendFormat": "{{label}}",
"instant": true
}
],
"fieldConfig": {
"defaults": {
"unit": "short",
"decimals": 0,
"min": 0,
"color": { "mode": "fixed", "fixedColor": "semi-dark-blue" },
"noValue": "no series — check whether the forgejo process restarted since [metrics] last changed; an absent series never says why it is absent"
},
"overrides": []
},
"options": {
"orientation": "horizontal",
"displayMode": "basic",
"showUnfilled": true,
"reduceOptions": { "calcs": ["lastNotNull"], "fields": "", "values": false }
}
}
]
}