vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

grafana: a forge board, honest about being an inventory board

Browse source 
Adds the fourth per-service board. Unlike its three siblings it answers
"what does the forge CONTAIN", not "is the forge healthy" -- forgejo's
exporter publishes a census of stored objects and no request, error or
latency series at all.

That distinction is the board's own description rather than something a
reader has to infer, because a fourth per-service board that silently
answers a different question than the other three is worse than no board:
someone checks it during an incident and reads a flat line as calm.

Panels: repositories, users, orgs, issues open/closed, releases, the
open-vs-closed pair over time, repos-and-users together (they move in
lockstep when an agent is created, so divergence is the signal),
discussion volume, and the webhook/hooktask pair.

One panel ships deliberately EMPTY: issues-by-label. The metric is
enabled in the forge's config but forgejo reads its [metrics] section at
startup, so the series does not exist until that process restarts -- and
a config merge alone does not restart it. The panel title says so, the
description says how to tell "not deployed" from "no labelled issues",
and `noValue` says it again in the graph itself. An empty panel with no
explanation is the failure this board is trying not to be.

Gated: every panel's metric confirmed to exist in the store, with a
bogus name checked absent in the same run so the check can fail, plus a
known-positive control against the label-values query itself -- a
download that silently returned nothing would otherwise report every
metric missing. Gate kept as verify-3494-forge-board.sh.
This commit is contained in:
atlas 2026-08-27 11:45:52 +02:00 • committed by mara
commit 6f248a12be
2 changed files with 419 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

1
nix/host-modules/swarm-grafana.nix
View file

@ -61,6 +61,7 @@ let
"agents.json"
"authelia.json"
"claude-usage.json"
"forge.json"
"logstore.json"
"metricstore.json"
"queue.json"

418
nix/host-modules/swarm-grafana/dashboards/forge.json Normal file
View file

@ -0,0 +1,418 @@
{
"title": "hyperhive · forge (content)",
"uid": "hyperhive-swarm-forge",
"description": "An INVENTORY board, not a service-health board — and the difference is the first thing to know about it. Its three siblings (metrics store, log store, authelia) answer 'is this service healthy': request rates, error rates, latency. Forgejo's exporter publishes no such series; what it publishes is a census of what the forge CONTAINS. So every number here is a gauge of stored things, sampled at scrape time. Nothing on this board can tell you the forge is slow or erroring, and a flat line means nobody created anything, not that scraping stopped — check the metrics-store board for that. Series are labelled service.name, NOT job; there is no job label in this store.",
"editable": true,
"refresh": "5m",
"schemaVersion": 39,
"tags": ["hyperhive", "swarm", "forge", "content"],
"time": { "from": "now-7d", "to": "now" },
"timezone": "utc",
"version": 1,
"templating": { "list": [] },
"panels": [
{
"id": 1,
"type": "stat",
"title": "Repositories",
"description": "Every repo the forge holds, including agent config repos — which is most of them on a busy hive, so this tracks agent count more closely than it tracks project count.",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"gridPos": { "h": 4, "w": 4, "x": 0, "y": 0 },
"targets": [
{
"refId": "A",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"expr": "gitea_repositories",
"instant": true,
"legendFormat": "repos"
}
],
"fieldConfig": {
"defaults": {
"unit": "short",
"decimals": 0,
"color": { "mode": "fixed", "fixedColor": "text" },
"mappings": []
},
"overrides": []
},
"options": {
"graphMode": "none",
"colorMode": "value",
"textMode": "value",
"justifyMode": "auto",
"reduceOptions": { "calcs": ["lastNotNull"], "fields": "", "values": false }
}
},
{
"id": 2,
"type": "stat",
"title": "Users",
"description": "Forge accounts. Agents hold one each, so this is operators plus the live agent roster — a drop without a matching agent teardown is worth a look.",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"gridPos": { "h": 4, "w": 4, "x": 4, "y": 0 },
"targets": [
{
"refId": "A",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"expr": "gitea_users",
"instant": true,
"legendFormat": "users"
}
],
"fieldConfig": {
"defaults": {
"unit": "short",
"decimals": 0,
"color": { "mode": "fixed", "fixedColor": "text" },
"mappings": []
},
"overrides": []
},
"options": {
"graphMode": "none",
"colorMode": "value",
"textMode": "value",
"justifyMode": "auto",
"reduceOptions": { "calcs": ["lastNotNull"], "fields": "", "values": false }
}
},
{
"id": 3,
"type": "stat",
"title": "Organizations",
"description": "Orgs, which is how the forge separates agent-owned repos from the shared ones. Near-constant by design; a change here is a structural decision someone made, not activity.",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"gridPos": { "h": 4, "w": 4, "x": 8, "y": 0 },
"targets": [
{
"refId": "A",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"expr": "gitea_organizations",
"instant": true,
"legendFormat": "orgs"
}
],
"fieldConfig": {
"defaults": {
"unit": "short",
"decimals": 0,
"color": { "mode": "fixed", "fixedColor": "text" },
"mappings": []
},
"overrides": []
},
"options": {
"graphMode": "none",
"colorMode": "value",
"textMode": "value",
"justifyMode": "auto",
"reduceOptions": { "calcs": ["lastNotNull"], "fields": "", "values": false }
}
},
{
"id": 4,
"type": "stat",
"title": "Issues open",
"description": "Open issues across every repo. The one number on this board that behaves like a backlog gauge — rising means work is being filed faster than it is closed.",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"gridPos": { "h": 4, "w": 4, "x": 12, "y": 0 },
"targets": [
{
"refId": "A",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"expr": "gitea_issues_open",
"instant": true,
"legendFormat": "open"
}
],
"fieldConfig": {
"defaults": {
"unit": "short",
"decimals": 0,
"color": { "mode": "fixed", "fixedColor": "text" },
"mappings": []
},
"overrides": []
},
"options": {
"graphMode": "none",
"colorMode": "value",
"textMode": "value",
"justifyMode": "auto",
"reduceOptions": { "calcs": ["lastNotNull"], "fields": "", "values": false }
}
},
{
"id": 5,
"type": "stat",
"title": "Issues closed",
"description": "Cumulative closed issues. Only ever rises, so read it as throughput-to-date rather than as a current state.",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"gridPos": { "h": 4, "w": 4, "x": 16, "y": 0 },
"targets": [
{
"refId": "A",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"expr": "gitea_issues_closed",
"instant": true,
"legendFormat": "closed"
}
],
"fieldConfig": {
"defaults": {
"unit": "short",
"decimals": 0,
"color": { "mode": "fixed", "fixedColor": "text" },
"mappings": []
},
"overrides": []
},
"options": {
"graphMode": "none",
"colorMode": "value",
"textMode": "value",
"justifyMode": "auto",
"reduceOptions": { "calcs": ["lastNotNull"], "fields": "", "values": false }
}
},
{
"id": 6,
"type": "stat",
"title": "Releases",
"description": "Tagged releases. Expected to sit at zero on a hive that ships from main — a non-zero value here means someone started cutting them, which is a change worth noticing.",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"gridPos": { "h": 4, "w": 4, "x": 20, "y": 0 },
"targets": [
{
"refId": "A",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"expr": "gitea_releases",
"instant": true,
"legendFormat": "releases"
}
],
"fieldConfig": {
"defaults": {
"unit": "short",
"decimals": 0,
"color": { "mode": "fixed", "fixedColor": "text" },
"mappings": []
},
"overrides": []
},
"options": {
"graphMode": "none",
"colorMode": "value",
"textMode": "value",
"justifyMode": "auto",
"reduceOptions": { "calcs": ["lastNotNull"], "fields": "", "values": false }
}
},
{
"id": 10,
"type": "timeseries",
"title": "Issues: open vs closed",
"description": "The two together, because either alone misleads. Open flat while closed climbs is a backlog being worked down; both climbing together is a backlog growing while work happens. A vertical step in closed is usually a bulk triage, not a productive afternoon.",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 4 },
"targets": [
{
"refId": "A",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"expr": "gitea_issues_open",
"legendFormat": "open"
},
{
"refId": "B",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"expr": "gitea_issues_closed",
"legendFormat": "closed"
}
],
"fieldConfig": {
"defaults": {
"unit": "short",
"decimals": 0,
"color": { "mode": "palette-classic" },
"custom": {
"drawStyle": "line",
"lineWidth": 1,
"fillOpacity": 10,
"showPoints": "never",
"spanNulls": false,
"axisSoftMin": 0
}
},
"overrides": []
},
"options": {
"legend": { "displayMode": "list", "placement": "bottom", "showLegend": true, "calcs": [] },
"tooltip": { "mode": "multi", "sort": "desc" }
}
},
{
"id": 11,
"type": "timeseries",
"title": "Repositories and users",
"description": "Both climb when an agent is created — a config repo and an account arrive together. So the interesting reading is when they DIVERGE: repos without users means repos created by hand, users without repos means accounts provisioned for something that never got one.",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 4 },
"targets": [
{
"refId": "A",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"expr": "gitea_repositories",
"legendFormat": "repos"
},
{
"refId": "B",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"expr": "gitea_users",
"legendFormat": "users"
}
],
"fieldConfig": {
"defaults": {
"unit": "short",
"decimals": 0,
"color": { "mode": "palette-classic" },
"custom": {
"drawStyle": "line",
"lineWidth": 1,
"fillOpacity": 0,
"showPoints": "never",
"spanNulls": false,
"axisSoftMin": 0
}
},
"overrides": []
},
"options": {
"legend": { "displayMode": "list", "placement": "bottom", "showLegend": true, "calcs": [] },
"tooltip": { "mode": "multi", "sort": "desc" }
}
},
{
"id": 20,
"type": "timeseries",
"title": "Discussion volume",
"description": "Comments and review comments across the forge. The closest thing here to an activity signal, and still a cumulative count rather than a rate — a steepening slope is more conversation, a flat stretch is a quiet period, and neither says anything about whether the forge is up.",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"gridPos": { "h": 7, "w": 12, "x": 0, "y": 12 },
"targets": [
{
"refId": "A",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"expr": "gitea_comments",
"legendFormat": "comments"
},
{
"refId": "B",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"expr": "gitea_attachments",
"legendFormat": "attachments"
}
],
"fieldConfig": {
"defaults": {
"unit": "short",
"decimals": 0,
"color": { "mode": "palette-classic" },
"custom": {
"drawStyle": "line",
"lineWidth": 1,
"fillOpacity": 10,
"showPoints": "never",
"spanNulls": false,
"axisSoftMin": 0
}
},
"overrides": []
},
"options": {
"legend": { "displayMode": "list", "placement": "bottom", "showLegend": true, "calcs": [] },
"tooltip": { "mode": "multi", "sort": "desc" }
}
},
{
"id": 21,
"type": "timeseries",
"title": "Automation surface",
"description": "Webhooks and their queued delivery tasks. Worth a glance when something that should have triggered did not: a hooktask count that climbs and never settles is deliveries backing up, and every hive registering the same webhook has bitten this forge before.",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"gridPos": { "h": 7, "w": 12, "x": 12, "y": 12 },
"targets": [
{
"refId": "A",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"expr": "gitea_webhooks",
"legendFormat": "webhooks"
},
{
"refId": "B",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"expr": "gitea_hooktasks",
"legendFormat": "hooktasks"
}
],
"fieldConfig": {
"defaults": {
"unit": "short",
"decimals": 0,
"color": { "mode": "palette-classic" },
"custom": {
"drawStyle": "line",
"lineWidth": 1,
"fillOpacity": 0,
"showPoints": "never",
"spanNulls": false,
"axisSoftMin": 0
}
},
"overrides": []
},
"options": {
"legend": { "displayMode": "list", "placement": "bottom", "showLegend": true, "calcs": [] },
"tooltip": { "mode": "multi", "sort": "desc" }
}
},
{
"id": 30,
"type": "timeseries",
"title": "Open issues by label — EMPTY UNTIL THE FORGE RESTARTS",
"description": "Deliberately shipped empty, and this text is the panel's real content. The per-label and per-repository breakdowns are enabled in the forge's config, but forgejo reads its [metrics] section at STARTUP — so the series do not exist until that process restarts, which a config merge alone does not do. An empty panel here means 'not deployed yet', NOT 'no issues have labels'. Confirm which you are looking at by checking whether the forgejo process has restarted, not by re-querying the metric: an absent series says absent and never says why. Once it fills, this becomes the per-label backlog view and this warning should be deleted.",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"gridPos": { "h": 7, "w": 24, "x": 0, "y": 19 },
"targets": [
{
"refId": "A",
"datasource": { "type": "prometheus", "uid": "@datasourceUid@" },
"expr": "gitea_issues_by_label",
"legendFormat": "{{label}}"
}
],
"fieldConfig": {
"defaults": {
"unit": "short",
"decimals": 0,
"color": { "mode": "palette-classic" },
"custom": {
"drawStyle": "line",
"lineWidth": 1,
"fillOpacity": 0,
"showPoints": "never",
"spanNulls": false,
"axisSoftMin": 0
},
"noValue": "no data yet — the forge process has not restarted since the metric was enabled"
},
"overrides": []
},
"options": {
"legend": { "displayMode": "list", "placement": "bottom", "showLegend": true, "calcs": [] },
"tooltip": { "mode": "multi", "sort": "desc" }
}
}
]
}