docs: fix genuine passive-voice hits in docs/scheduler
Seventh batch of the ongoing write-good.Passive pass (hyperhive#4042):
read all 65 hits across jobq.md/ci.md/observability.md/coordinator.md
in context and rewrote 41 with a clearly nameable actor -- mostly
hive-c0re, nix/the nix module, the harness, or a specific fn/type
named right there or nearby (coordinator.md's node-inventory table
and DAG-shape descriptions name concrete Rust items constantly, so
the actor is almost always sitting in the same sentence).
Left 24 alone: predicate-adjective-copula state descriptions ("is
stuck", "is gone", "is unaffected", "is done", etc. -- the largest
recurring bucket this batch, especially in observability.md's
scope/status descriptions), negative-capability idioms ("no X is
needed/left", "X can't be written down"), the established "is
tracked as a follow-up" idiom, a firewall-shorthand notation
("bridge->127.0.0.0/8 is dropped") where rewriting would break the
compact rule-like format, a CLI-flag "(repeatable)" annotation ("May
be repeated"), a Rust type-signature fact ("`moves` is typed ..."),
a hypothetical/counterfactual maintenance-burden clause, a
readiness-condition list ("a node is ready when ... every dep is
satisfied"), and one deliberately-parallel idiom pair
("When OTEL is enabled" used identically twice as a section-opening
convention -- fixing one would break the parallelism, not the
opposite).
One self-caught regression: an early attempt to fix "used by every
`Reconcile` node's start action" (a reduced participial clause, not
flagged) into "is used by every `Reconcile` node's start action"
introduced a brand-new flagged passive. Caught by the post-edit vale
count (expected 65->24, got 65->25) not matching, same discipline as
the docs/turn-loop batch's tail-truncation catch -- re-ran with
active voice instead ("Every `Reconcile` node's start action uses
this fallback").
Verified via vale before/after: 65 -> 24 write-good.Passive hits,
exactly the 24 left alone above; error count and other warning
categories unchanged. Re-read every changed line in full surrounding
context after editing before running the final vale check.
This commit is contained in:
iris
mara
parent
ec433a07dd
commit
09e4e2f5e9
4 changed files with 68 additions and 68 deletions
|
|
@ -56,8 +56,8 @@ never writes to that status slot. Reproduced with three dispatches on one
|
|||
PR over 20+ minutes, all genuinely green, the PR's own status unmoved
|
||||
throughout. **Always re-check `pr-status` after a dispatch and believe
|
||||
what it says — don't push a commit just to unwedge it, that's the
|
||||
empty-commit anti-pattern this verb exists to avoid.** If the code is
|
||||
verified green (`hive-forge ci-log --run <n> --job 0`, verdict is the
|
||||
empty-commit anti-pattern this verb exists to avoid.** If `hive-forge
|
||||
ci-log --run <n> --job 0` shows the code green (verdict is the
|
||||
last line) but the status is stuck, ask the operator to select "rerun" in
|
||||
the forge web UI — it's CSRF-gated, so only they can do it.
|
||||
|
||||
|
|
@ -74,8 +74,8 @@ sh scripts/check-issue-refs.sh # tracker-tag lint
|
|||
sh scripts/check-comment-blocks.sh # comment-block lint
|
||||
```
|
||||
|
||||
A git pre-push hook that automates the two lint checks is provided at
|
||||
`scripts/pre-push`. Install it once per clone:
|
||||
`scripts/pre-push` provides a git pre-push hook that automates the two
|
||||
lint checks. Install it once per clone:
|
||||
|
||||
```sh
|
||||
ln -sf ../../scripts/pre-push .git/hooks/pre-push
|
||||
|
|
@ -112,13 +112,13 @@ it and runs it out of band, so a slow forge or core-token never delays the
|
|||
container's start. Gotcha: don't gate the container's own start on a forge
|
||||
round-trip (a host-side unit that did this could exceed the nspawn start
|
||||
timeout and trip a restart loop) — registration must stay something
|
||||
hive-c0re drives after the container is already up. The core admin token is
|
||||
held only by hive-c0re on the host; only the runner registration token
|
||||
hive-c0re drives after the container is already up. hive-c0re holds the
|
||||
core admin token only on the host; only the runner registration token
|
||||
reaches the container.
|
||||
|
||||
### hive-c0re side (`forge/ci_runner.rs`, run during the startup sweep)
|
||||
|
||||
Gated on `HYPERHIVE_FORGE_CI_ENABLED` (the nix module sets it on `hive-c0re.service` when `deploy.forgejo.ci.enable`). Best-effort — failures are logged and never abort the sweep; a healthy runner is never restarted.
|
||||
Gated on `HYPERHIVE_FORGE_CI_ENABLED` (the nix module sets it on `hive-c0re.service` when `deploy.forgejo.ci.enable`). Best-effort — hive-c0re logs failures and never aborts the sweep; a healthy runner is never restarted.
|
||||
|
||||
1. If `.runner` exists at `/var/lib/nixos-containers/hive-ci/var/lib/gitea-runner/hive/.runner`, validate its id against `GET /api/v1/admin/runners/{id}` with the core admin token:
|
||||
- **200**: still registered — done, no restart.
|
||||
|
|
@ -141,7 +141,7 @@ When `deploy.forgejo.ci.enable` is set, hive-c0re autoseeds an
|
|||
`uses: actions/checkout@vN` steps resolve entirely on loopback — no
|
||||
external DNS on the CI critical path.
|
||||
|
||||
The mirror is seeded by **hive-c0re** itself during its forge
|
||||
**hive-c0re** itself seeds the mirror during its forge
|
||||
provisioning sweep (`forge/repos.rs::ensure_mirrors`). The nix module
|
||||
forwards the effective mirror list as `HYPERHIVE_FORGE_MIRRORS` in the
|
||||
`hive-c0re` service environment (JSON-encoded `[{upstream, dest}]`
|
||||
|
|
@ -159,11 +159,11 @@ services.hyperhive.deploy.forgejo.mirrors = [
|
|||
];
|
||||
```
|
||||
|
||||
Each entry is created as a real Forgejo pull-mirror — not a one-off
|
||||
hive-c0re creates each entry as a real Forgejo pull-mirror — not a one-off
|
||||
clone. Forgejo re-syncs the mirror on every pull (`git-upload-pack`
|
||||
request), so a DNS blip during that sync will propagate back to the
|
||||
runner as a hard `git clone` failure. The
|
||||
`<owner>` org in `dest` is autocreated. Keep mirror dests out of the
|
||||
runner as a hard `git clone` failure. hive-c0re
|
||||
autocreates the `<owner>` org in `dest`. Keep mirror dests out of the
|
||||
hive-c0re-managed namespaces (`config/`, `shared/`, `agents/`, `core/`)
|
||||
to avoid provisioning collisions.
|
||||
|
||||
|
|
@ -173,7 +173,7 @@ to avoid provisioning collisions.
|
|||
|
||||
### What unsandboxed builds mean
|
||||
|
||||
nspawn containers can't create user-namespaces, so `nix.settings.sandbox-fallback = true` is set in the container. This means every `nix build` (and `nix flake check`) runs **without a build sandbox** — the build process has full access to the container filesystem, network, and any bind-mounts during the build phase.
|
||||
nspawn containers can't create user-namespaces, so the module sets `nix.settings.sandbox-fallback = true` in the container. This means every `nix build` (and `nix flake check`) runs **without a build sandbox** — the build process has full access to the container filesystem, network, and any bind-mounts during the build phase.
|
||||
|
||||
A malicious `default.nix` or build script in a PR can therefore:
|
||||
|
||||
|
|
@ -235,15 +235,15 @@ following to your host config:
|
|||
```
|
||||
|
||||
**Measure a collection on your own hosts before relying on that stagger.**
|
||||
`randomizedDelaySec` spreads the _start_, not the run, and the run is dominated
|
||||
by the sweep over unused `.links` entries — which scales with the size of the
|
||||
`randomizedDelaySec` spreads the _start_, not the run, and the sweep over
|
||||
unused `.links` entries dominates the run — which scales with the size of the
|
||||
store, not with what this collection deletes. On a large store it can take most
|
||||
of an hour, so two hosts both set to `daily` may overlap however much jitter you
|
||||
give them. `journalctl -u nix-gc.service` reports each run's wall clock.
|
||||
|
||||
**Remote builders:** if CI dispatches builds to a remote builder (for example via
|
||||
`nix.buildMachines` / `ssh-ng://`), the build outputs land in _that host's_
|
||||
store, so the same GC config should be applied wherever the builder runs —
|
||||
store, so apply the same GC config wherever the builder runs —
|
||||
GC on the coordinator host won't reclaim space on the builder.
|
||||
|
||||
⚠️ Two stores also means the temp-root protection above **doesn't span
|
||||
|
|
|
|||
|
|
@ -14,8 +14,8 @@ section covers the concrete node inventory hive-c0re builds on top of that
|
|||
engine, plus the internals (module layout, scheduler mechanics,
|
||||
resource/lease semantics).
|
||||
|
||||
Every container/meta operation (rebuild, meta-update, first-spawn, power
|
||||
changes) is submitted to the global job-DAG queue (`hive-c0re/src/job_queue/`)
|
||||
hive-c0re submits every container/meta operation (rebuild, meta-update, first-spawn, power
|
||||
changes) to the global job-DAG queue (`hive-c0re/src/job_queue/`)
|
||||
as a **DAG of primitive nodes**. One scheduler task drives all DAGs;
|
||||
concurrency comes from the resource classes below, not from multiple workers.
|
||||
Special cases like graceful stop, deferred starts, and the meta-update
|
||||
|
|
@ -63,7 +63,7 @@ Cheap — no build slot:
|
|||
| `Start` | mechanical container start — runtime dir + drop-ins, `start_with_fallback`, MCP listener registration, the manager kick. Fanned out by a `Reconcile` that observed `wanted = Up` and the container down |
|
||||
| `Stop` | mechanical container stop — `nixos-container` kill, MCP listener unregister, the `Killed` manager notify. Fanned out by a `Reconcile` that observed `wanted = Offline` and up |
|
||||
| `StopForUpdate` | mechanical `nixos-container stop` for the profile swap; never touches `wanted`; noop if already stopped |
|
||||
| `RebuildBookkeeping` | the swap's Ok-only bookkeeping tail — rev marker, forge/matrix sync, manager kick, rescan, meta-inputs snapshot; `AfterOk(Swap)` so it runs only on a successful swap (the `Rebuilt` manager event is emitted by the DAG's `EmitRebuilt` tail node, not here). Split out of `Swap` for dashboard visibility + retry granularity, declares no resources of its own — a coordinated child of the `AgentWindow` brace |
|
||||
| `RebuildBookkeeping` | the swap's Ok-only bookkeeping tail — rev marker, forge/matrix sync, manager kick, rescan, meta-inputs snapshot; `AfterOk(Swap)` so it runs only on a successful swap (the DAG's `EmitRebuilt` tail node emits the `Rebuilt` manager event, not here). Split out of `Swap` for dashboard visibility + retry granularity, declares no resources of its own — a coordinated child of the `AgentWindow` brace |
|
||||
| `AgentWindow` | pure resource holder — the brace for one agent's rebuild. Declares the build slot + agent lease atomically and holds both for its whole subtree, so `Prebuild` and the `Signal`→`Drain` quiesce window run concurrently instead of one nested under the other. Performs no work; see _Braces_ |
|
||||
| `Signal` | set the graceful fence + kick, so the harness runs one stop-checkpoint turn |
|
||||
| `Drain` | await the harness clearing the fence, bounded by the 3-min graceful-stop timeout; resolves ok either way |
|
||||
|
|
@ -101,7 +101,7 @@ container build:
|
|||
phase below it (it declares `Resource::MetaWindow`). Two meta
|
||||
mutations can therefore never interleave, so no commit lands inside another
|
||||
node's staged window. It's a queue resource rather than a runtime mutex
|
||||
because a resource is held by a subtree root across its whole subtree, which
|
||||
because a subtree root holds a resource across its whole subtree, which
|
||||
a `MutexGuard` (bounded by one executor fn) can't — that's what lets a
|
||||
multi-node deploy own one window. For the same reason the window must stay
|
||||
_off_ long store-only work: the rebuild's meta preamble is its own
|
||||
|
|
@ -127,9 +127,9 @@ a hive-wide `hivectl restart` / `stop` / `start` is ONE DAG with a per-agent
|
|||
subgraph each (independent roots, run concurrently on their own leases), not
|
||||
N separate DAGs.
|
||||
|
||||
**These are built dynamically from each agent's live running state** (an
|
||||
async `lifecycle::is_running` read), so they live in `job_queue/power.rs`,
|
||||
not the pure/sync `templates.rs`. Per-agent shape rule: `stop`/`start` carry
|
||||
**`job_queue/power.rs` builds these dynamically from each agent's live
|
||||
running state** (an async `lifecycle::is_running` read) — that's why they
|
||||
live there rather than in the pure/sync `templates.rs`. Per-agent shape rule: `stop`/`start` carry
|
||||
a head `SetWanted` (intent) — `restart` doesn't; the tail `Reconcile`
|
||||
(convergence guarantee — cheap, noops when already converged) is ALWAYS
|
||||
present; only the _mechanical_ nodes (`Signal`/`Drain`/`StopForUpdate`) are
|
||||
|
|
@ -192,8 +192,8 @@ rides the queue through that power layer, so intent, lease serialization,
|
|||
and crash-watch suppression can't drift per surface; the only direct starts
|
||||
left are the root-agent bootstrap and infra containers (no lease, no
|
||||
harness). Cancelling a still-queued power DAG reverts `wanted` to the
|
||||
observed state — a cancel means "don't do it," not "do it later." Agents
|
||||
without a row are seeded from observed state on first touch (running ⇒
|
||||
observed state — a cancel means "don't do it," not "do it later." hive-c0re
|
||||
seeds agents without a row from observed state on first touch (running ⇒
|
||||
`Up`); destroy removes the row.
|
||||
|
||||
The admin-socket responses carry the submitted DAG ids; `hivectl` polls
|
||||
|
|
@ -275,7 +275,7 @@ reconcile_), so there is no durable-recovery machinery to go wrong.
|
|||
|
||||
### Cancel, history
|
||||
|
||||
Submit-time dedup was removed with the agent-per-node move (a multi-agent DAG
|
||||
The agent-per-node move removed submit-time dedup (a multi-agent DAG
|
||||
has no single agent to key a dedup on), so every submit enqueues a fresh DAG;
|
||||
whether any dedup needs reintroducing is tracked as a follow-up.
|
||||
|
||||
|
|
@ -297,13 +297,13 @@ DeployWindow (root — build slot + lease + meta window, no work of its own)
|
|||
└── DeployTail AfterAny(apply) compensate, mirror to forge
|
||||
```
|
||||
|
||||
The root's resources are held across the whole subtree, so the two-phase
|
||||
The root holds its resources across the whole subtree, so the two-phase
|
||||
`prepare_deploy` / `finalize_deploy` span keeps its staged `flake.lock`
|
||||
protected even though the phases are separate nodes. Splitting them buys
|
||||
three things a single opaque node couldn't have: per-phase visibility on the
|
||||
dashboard, a `MergeVerify` failure that provably mutated nothing, and a
|
||||
compensation step that survives a hive-c0re restart — the pre-merge
|
||||
`applied/main` is parked in `refs/hyperhive/rollback/<approval-id>`, not in a
|
||||
compensation step that survives a hive-c0re restart — `DeployApply` parks the pre-merge
|
||||
`applied/main` in `refs/hyperhive/rollback/<approval-id>`, not in a
|
||||
local variable, so `DeployTail` can still undo a half-finished deploy after a
|
||||
crash.
|
||||
|
||||
|
|
@ -378,9 +378,9 @@ vs the current flake path) and persisted `wanted` intent, then:
|
|||
|
||||
Booting with no config change performs **no meta commit** — only reconciles.
|
||||
The sweep reason records the rebuild / deferred / up-to-date counts so the
|
||||
operator sees at a glance how much work the boot triggered. Agents without an
|
||||
`agent_power` row are seeded from observed state during classification (the
|
||||
one-time migration; thereafter the DB is authoritative).
|
||||
operator sees at a glance how much work the boot triggered. `auto_update::run`
|
||||
seeds agents without an `agent_power` row from observed state during
|
||||
classification (the one-time migration; thereafter the DB is authoritative).
|
||||
|
||||
## Meta flake
|
||||
|
||||
|
|
@ -435,7 +435,7 @@ Sequence for a rebuild DAG (each step is its own queue node):
|
|||
The container keeps serving the previous generation while eval + fetch +
|
||||
build happen out-of-band. `nixos-container update` then finds the result
|
||||
cached and skips straight to the profile-swap. Build failures surface
|
||||
here, before the running container is touched. Skipped when the
|
||||
here, before `Swap` touches the running container. Skipped when the
|
||||
container is already down — a stopped agent has no uptime to preserve,
|
||||
so `Swap` builds inline instead of paying the double eval.
|
||||
3. `StopForUpdate` — bring the container down (noop when already stopped).
|
||||
|
|
@ -449,9 +449,9 @@ The approval deploy uses this same chain rather than a rebuild path of its
|
|||
own. Its `DeployApply` node doesn't build: it merges, opens the two-phase
|
||||
meta deploy, and returns the chain above as a subgraph the scheduler grafts
|
||||
into the live DAG under that node. A `FinalizeDeploy` node gated on the
|
||||
graft's completion then plants the deploy tag — so "did the agent come back
|
||||
up?" is answered by `Reconcile` succeeding, the same way it's for every
|
||||
other rebuild, instead of by a fused inline start.
|
||||
graft's completion then plants the deploy tag — so `Reconcile`'s success
|
||||
answers "did the agent come back up?", the same way it does for every
|
||||
other rebuild, instead of a fused inline start.
|
||||
|
||||
The grafted nodes land _inside_ `DeployWindow`'s subtree, so they re-enter
|
||||
the meta window and build slot it already holds rather than deadlocking
|
||||
|
|
@ -466,9 +466,9 @@ half-started at that point.
|
|||
|
||||
Fallback: `stop` (graceful SIGTERM drain) → `kill` (SIGKILL any lingering processes)
|
||||
→ `start` (clean cold-start, no generation transition, new activation runs cleanly).
|
||||
Both errors are preserved and surfaced if the cold-start also fails. The fallback
|
||||
lives in `lifecycle::start_with_fallback`, used by every `Reconcile` node's
|
||||
start action.
|
||||
`lifecycle::start_with_fallback` preserves and surfaces both errors if the
|
||||
cold-start also fails. Every `Reconcile` node's start action uses this
|
||||
fallback.
|
||||
|
||||
### Spawn path (new container)
|
||||
|
||||
|
|
@ -480,8 +480,8 @@ flags → `systemctl daemon-reload` → `start`.
|
|||
### Prebuild attr path
|
||||
|
||||
`nix build` doesn't autoresolve `meta#<name>` against `nixosConfigurations` the
|
||||
way `nixos-container` does internally. The explicit attr path
|
||||
`<flake-root>#nixosConfigurations.<name>.config.system.build.toplevel` is required;
|
||||
way `nixos-container` does internally. nix requires the explicit attr path
|
||||
`<flake-root>#nixosConfigurations.<name>.config.system.build.toplevel`;
|
||||
using the bare `meta#<name>` ref would make nix look in `packages`, `legacyPackages`,
|
||||
or the flake root directly — none of which exist in the rendered meta flake.
|
||||
|
||||
|
|
@ -556,14 +556,14 @@ trade-off:
|
|||
- **On** (recommended for x86_64 hosts that care about first-spawn
|
||||
latency): the first `nixos-container start` for any new agent
|
||||
completes in seconds because nothing is left to fetch. Cost:
|
||||
the full nixpkgs runtime closure + claude-code + the harness
|
||||
binary are added to the host system closure (low single-digit GB
|
||||
`preBuildAgentTemplates` adds the full nixpkgs runtime closure + claude-code + the harness
|
||||
binary to the host system closure (low single-digit GB
|
||||
additional).
|
||||
- **Off** (default): the host closure stays lean; the first spawn
|
||||
does all the eval + fetch work at runtime (can take a few
|
||||
minutes on a fresh store).
|
||||
|
||||
**Note**: toplevels are pinned to `x86_64-linux`. Enabling on an
|
||||
**Note**: the meta flake pins toplevels to `x86_64-linux`. Enabling on an
|
||||
`aarch64` host forces a cross-compilation or remote-builder build,
|
||||
which is almost never desired. Leave off on non-x86 hosts.
|
||||
|
||||
|
|
|
|||
|
|
@ -47,6 +47,6 @@ side by side). A step shows one of:
|
|||
| `⊘` | cancelled |
|
||||
| `·` | skipped (not needed for this run) |
|
||||
|
||||
A step that isn't needed for a given run shows as `·` rather than being
|
||||
left out of the tree entirely, so the same kind of operation keeps a
|
||||
A step that isn't needed for a given run shows as `·` rather than
|
||||
dropping out of the tree entirely, so the same kind of operation keeps a
|
||||
recognizable shape run to run, whichever steps it actually needed.
|
||||
|
|
|
|||
|
|
@ -87,10 +87,10 @@ turn-stats rather than from a metric label.
|
|||
|
||||
## Options reference
|
||||
|
||||
Every `services.hyperhive.otel.*` option's full type/default/description/
|
||||
example is generated straight from the nix module (`nix/host-modules/
|
||||
otel.nix`) into [`/options/`](/options/) (host options — `nix build
|
||||
.#docs-host` for a local render). That page is kept honest by the build in a
|
||||
The nix module (`nix/host-modules/otel.nix`) generates every
|
||||
`services.hyperhive.otel.*` option's full type/default/description/
|
||||
example straight into [`/options/`](/options/) (host options — `nix build
|
||||
.#docs-host` for a local render). The build keeps that page honest in a
|
||||
way a hand-copied version here can't be, so it's the reference, not this
|
||||
doc. What follows is what a flat per-option listing can't express: the
|
||||
two-tier architecture, the security model, and how the options interact.
|
||||
|
|
@ -130,7 +130,7 @@ the local deployment stops testing.
|
|||
**`endpoint` keeps meaning "where telemetry goes upstream."** Neither tier
|
||||
redefines it — the agent-facing value is _derived_
|
||||
(`http://<bridgeIp>:<collector.port>`), so an existing deployment's `endpoint`
|
||||
keeps working unchanged. The bridge port is contributed to `exposeHostPorts`
|
||||
keeps working unchanged. The otel module contributes the bridge port to `exposeHostPorts`
|
||||
automatically; there is nothing to open by hand.
|
||||
|
||||
### Authenticated ingest
|
||||
|
|
@ -162,7 +162,7 @@ swarm tier and no metrics appearing for that hive.
|
|||
`otel.protocol` (`grpc` → the gRPC exporter, otherwise OTLP/HTTP), and the gRPC
|
||||
exporter takes an _address_: `https://host/path` is a legal
|
||||
`OTEL_EXPORTER_OTLP_ENDPOINT` for HTTP but fails as gRPC with _"missing port in
|
||||
address"_. The collector's config is validated at build time, so a mismatch is
|
||||
address"_. nix validates the collector's config at build time, so a mismatch is
|
||||
a build error naming the reason rather than telemetry silently going nowhere.
|
||||
|
||||
## Network access
|
||||
|
|
@ -186,8 +186,8 @@ contributes the collector's port and derives the agent-facing endpoint itself.
|
|||
|
||||
## Built-in resource labels
|
||||
|
||||
The OTLP variables (`OTEL_EXPORTER_OTLP_ENDPOINT`, `_PROTOCOL`,
|
||||
`OTEL_RESOURCE_ATTRIBUTES`, the temporality preference) are set **container
|
||||
The harness sets the OTLP variables (`OTEL_EXPORTER_OTLP_ENDPOINT`, `_PROTOCOL`,
|
||||
`OTEL_RESOURCE_ATTRIBUTES`, the temporality preference) **container
|
||||
wide** — in systemd's `DefaultEnvironment` and in `/etc/profile` — so every
|
||||
process in an agent container exports to the hive's collector without any
|
||||
per-tool wiring. That covers Claude Code, `hive-metric`, and anything you run
|
||||
|
|
@ -203,8 +203,8 @@ automatically:
|
|||
| `hive` | hive display name (`services.hyperhive.hiveName`) |
|
||||
| `swarm` | swarm display name (`services.hyperhive.swarm.name`, if set) |
|
||||
|
||||
Additional labels can be appended via `extraResourceAttributes` (see option
|
||||
reference above); custom per-data-point labels can be passed with
|
||||
Append additional labels via `extraResourceAttributes` (see option
|
||||
reference above); pass custom per-data-point labels with
|
||||
`hive-metric --labels` (see below).
|
||||
|
||||
## Host-emitted container-resource metrics (hive-c0re)
|
||||
|
|
@ -219,7 +219,7 @@ Emitted via the OpenTelemetry Rust SDK, using the
|
|||
[semconv `container.*`](https://opentelemetry.io/docs/specs/semconv/system/container-metrics/)
|
||||
metric names + the standard `container.name` attribute where a spec metric
|
||||
exists, so off-the-shelf OTEL/Grafana container dashboards work. Resource
|
||||
`service.name = hyperhive-c0re`; each data point is tagged `container.name`
|
||||
`service.name = hyperhive-c0re`; hive-c0re tags each data point `container.name`
|
||||
(= the `h-<agent>` machine) and the hive `agent` label:
|
||||
|
||||
| Metric | Unit | Kind | Source |
|
||||
|
|
@ -266,8 +266,8 @@ same container-wide `OTEL_RESOURCE_ATTRIBUTES` as everything else in this
|
|||
section — nothing extra to configure. Cadence follows
|
||||
`HYPERHIVE_OTEL_METRIC_INTERVAL_MS` (default 60s, same variable + default as
|
||||
`hive-c0re`'s container-resource export above) — that only controls how often
|
||||
the batched points are flushed to the collector, not how often they're
|
||||
recorded (every turn, always).
|
||||
the harness flushes the batched points to the collector, not how often it
|
||||
records them (every turn, always).
|
||||
|
||||
## Hive-scoped metrics (hive-c0re)
|
||||
|
||||
|
|
@ -283,7 +283,7 @@ distinguishable from "this hive is gone." Select them with
|
|||
| `hyperhive.hive.degraded` | `1` | gauge | `1` while the hive reports itself unhealthy — the same verdict `/health/ready` gives and the swarm status view shows |
|
||||
| `hyperhive.hive.warnings` | `1` | gauge | how many warnings are currently raised, split by a `level` attribute (`warn`, `crit`) |
|
||||
|
||||
Both levels are reported every cycle, `0` included, so a healthy hive is
|
||||
hive-c0re reports both levels every cycle, `0` included, so a healthy hive is
|
||||
visible as zeros rather than as missing series.
|
||||
|
||||
`hyperhive.hive.degraded` is what a dashboard should alert on: it's
|
||||
|
|
@ -300,8 +300,8 @@ Same cadence, transport and resource labels as the container metrics above.
|
|||
"global/system" hook, not scoped to any one org or repo) and counts commit
|
||||
and push activity as deliveries arrive — occurrence-driven, not polled.
|
||||
Forgejo's own native `/metrics` endpoint has no equivalent: it exposes
|
||||
counts of durable rows (issues, comments, repos), and neither a commit nor a
|
||||
push is stored anywhere as a row to count.
|
||||
counts of durable rows (issues, comments, repos), and Forgejo doesn't
|
||||
store either a commit or a push anywhere as a row to count.
|
||||
|
||||
| Metric | Unit | Kind | Attributes |
|
||||
| ---------------------------- | ---- | ------- | ------------------- |
|
||||
|
|
@ -314,9 +314,9 @@ when the delivery actually carries commits. Same enable signal (`OTEL_EXPORTER_O
|
|||
(`HYPERHIVE_OTEL_METRIC_INTERVAL_MS`) and `HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES`
|
||||
resource-attribute channel as `swarm-controller`'s other OTEL exporter (its
|
||||
`hive-jobq-metrics`-backed job-graph rollup, undocumented here — see that
|
||||
crate's own doc comment) — `service.name = swarm-controller` is set
|
||||
directly rather than read from the container environment, since
|
||||
`swarm-controller` is a standalone daemon, not a per-agent harness process.
|
||||
crate's own doc comment) — `swarm-controller` sets `service.name =
|
||||
swarm-controller` directly rather than reading it from the container
|
||||
environment, since it's a standalone daemon, not a per-agent harness process.
|
||||
|
||||
## Agent-emitted custom metrics (`hive-metric`)
|
||||
|
||||
|
|
@ -340,7 +340,7 @@ hive-metric <name> <value> [--type counter|gauge] [--temporality delta|cumulativ
|
|||
reports the running total, which a stateless one-shot CLI can't track
|
||||
itself).
|
||||
- `--labels key=value` — extra per-data-point labels. May be repeated.
|
||||
The resource labels (agent, hive, swarm, service.name) are inherited
|
||||
`hive-metric` inherits the resource labels (agent, hive, swarm, service.name)
|
||||
automatically from `OTEL_RESOURCE_ATTRIBUTES` — don't re-specify them.
|
||||
|
||||
### Examples
|
||||
|
|
@ -366,12 +366,12 @@ with an informative error message. No silently dropped metrics.
|
|||
|
||||
`hive-metric` always uses **OTLP HTTP/JSON** (`application/json` POST to
|
||||
`$OTEL_EXPORTER_OTLP_ENDPOINT/v1/metrics`), regardless of the
|
||||
`OTEL_EXPORTER_OTLP_PROTOCOL` setting. Auth headers from
|
||||
`OTEL_EXPORTER_OTLP_HEADERS` are forwarded verbatim.
|
||||
`OTEL_EXPORTER_OTLP_PROTOCOL` setting. `hive-metric` forwards auth headers from
|
||||
`OTEL_EXPORTER_OTLP_HEADERS` verbatim.
|
||||
|
||||
## Metrics temporality
|
||||
|
||||
OTEL export is configured with **cumulative** temporality by default
|
||||
The harness configures OTEL export with **cumulative** temporality by default
|
||||
(`OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE=cumulative`),
|
||||
overriding Claude Code's default of DELTA. This avoids silent metric drops in
|
||||
Prometheus-family backends (including Grafana LGTM / Mimir) that don't ship a
|
||||
|
|
|
|||
Loading…
Reference in a new issue