hive-c0re's container-resource exporter has POSTed to a 404 for as long as
it has existed, silently: it passed the collector's base address to
`with_endpoint`, which the SDK takes verbatim, so every export went to `/`
instead of `/v1/metrics`. Nothing reported it — OTLP export failures go to
an error handler no binary here installs — so the daemon logged "exporter
enabled" and delivered nothing. VictoriaMetrics has never held a sample
under `service.name=hyperhive-c0re`.
Fix the way the rest of the repo already resolves an endpoint: an endpoint
option names a BASE, and the layer that knows the signal appends to it.
`hive-metric` — same SDK, same collector — never calls `with_endpoint`, and
`docs/observability.md` documents the append as system behaviour; the one
place a full path is spelled out is the VictoriaMetrics exporter, because
its far end is not a standard OTLP path.
So drop the call. The builder is now byte-identical to hive-metric's, and
hive-c0re's unit carries the standard `OTEL_EXPORTER_OTLP_ENDPOINT` for the
SDK to read. The address is bound once in nix and consumed twice, so what a
hive hands its agents and what it exports to itself cannot drift.
The enable signal moves to that same standard variable: "configured" and
"where it actually goes" become one string rather than two that agree by
convention. `HYPERHIVE_OTEL_*` keeps its own job, the agent-config
transport meta.rs reads — a name the SDK has never known, which is the bug.
The test changes shape with the fix. The old one asserted a URL this module
built; the new one pins that the exporter is gated on the variable the SDK
itself reads, because the fix is now an absence and an absence is what a
later "the endpoint is right there, just pass it" edit puts back.
Refs #3402
hive-c0re's container-resource exporter already targets this hive's own
collector (environment.nix derives the bridge address), so the upstream
header it was loaded with has nowhere to be presented: that hop is
unauthenticated for every producer on the host, and the credential
belongs to the swarm tier, which is the one that leaves the swarm.
Drop the LoadCredential entry and the auth_headers() reader with it.
The option itself stays -- swarm-otel.nix is its real consumer, via
EnvironmentFile on the collector unit.
Also corrects three descriptions that this makes false, or that were
already false: the module doc claimed to reuse the config "Claude Code's
in-container SDK export uses", which stopped being true when agents
moved off that path; the nix comment claimed the secret is "the same one
the agent containers get, forwarded via nspawn --load-credential", which
lost its last producer earlier; and docs/observability.md described an
Authorization header on a hop that will no longer send one. The
headersCredential option's own docs already said it reaches "neither an
agent container nor a hive's own collector" -- this makes that true
rather than aspirational.
environment.nix explains why HYPERHIVE_OTEL_HEADERS_CREDENTIAL is not
emitted by naming the machinery that used to consume it -- machinery this
PR deletes. Left as written it would describe a removed nspawn credential
and a removed oneshot as if both still ran, in the same paragraph that
justifies withholding the variable.
That is this PR's own defect inverted: it exists so an auditor asking
'can an agent obtain the OTEL token?' finds nothing misleading, and a
comment claiming live delivery machinery is exactly the kind of thing
that costs an auditor a reconstruction.
Keeps the rule rather than the history -- one holder, on the host -- since
that is the part still worth reading.
hive-c0re is a host unit that wired no CA trust at all. Harmless today
-- every endpoint it is handed defaults to plain http or loopback, so
nothing verifies a certificate -- and latent for the split-host shape the
options invite: `swarm.matrix.apiUrl`'s own example is an https URL, and
pointing it or `statusPublish.natsUrl` at another hive's gateway means
verifying a leaf signed by a CA generated at runtime.
Unlike the other consumers of this helper, hive-c0re already makes a
public-TLS call -- the OTEL exporter reaches whatever `otel.endpoint`
names. `SSL_CERT_FILE` replaces the trust store rather than adding to it,
so the bundle being system CAs + hive CA is what keeps that path working;
narrowing it to the hive CA alone would fix a case nobody hits yet and
break one that runs today.
Three options, all three derived from ONE predicate — this host runs both
the queue and the IdP — so a defaulted set is all or nothing. Deriving
them per-service looks equivalent and is not: `enableRequiredServices`
turns on matrix and authelia but not nats, so an ordinary all-local hive
would resolve two of three and trip the assertion below. Making the
partial state unrepresentable is what keeps that assertion honest.
Deliberately not the shape swarm-controller uses. That module emits its
queue coordinates only when authelia and NATS are local, which is right
for a service that *is* a swarm-host service — but a hive is the one thing
in a swarm that routinely is not on the swarm host, so the same rule would
make status publishing work on exactly the deployment that needs it least.
There is no `enable`: three coordinates that are all set is the enable. An
extra flag would allow configured-but-off, which is one more state to
explain and one more way to be silently quiet.
A half-set trio is an eval error rather than a silent no-op, because its
runtime failure mode is the expensive kind — the daemon comes up fine,
never connects, and the hive reads never_reported on a dashboard nobody is
watching yet. With the defaults all-or-nothing, the assertion only ever
judges what an operator typed by hand.
The secret arrives by LoadCredential, not a copy: hive-c0re is a host
unit, so systemd hands it the file directly and the secret never gains a
second on-disk copy. The client id is not chosen here either — it is
`hive-<hiveName>`, the identity swarm-authelia.nix already declares for
every entry in the roster.
Mara wanted the underlying plumbing gone too, not just the dashboard
display. Traced every consumer before cutting:
- certFingerprint (services.hyperhive.swarm.hives.<name>.certFingerprint):
removed the nix option entirely. Its only consumer was the dashboard
code removed in the previous commits.
- HYPERHIVE_PEERS: removed entirely — the env var itself, the whole
block that built it in hive-c0re/environment.nix, and its entry in
meta.rs's FORWARDED_VARS (which forwarded it into every agent
container). Turned out to have zero real consumers, not just one:
the docs claimed hive-agent::identity::peers() read it for qualified
agent labels, but no such function exists — identity.rs only
qualifies THIS agent's own label with HYPERHIVE_HIVE_DOMAIN, nothing
peer-list-related. Grepped the whole hive-agent crate to confirm
before removing.
services.hyperhive.swarm.peerHives (the nix option HYPERHIVE_PEERS was
built from) is untouched — swarm-wireguard.nix reads it directly for
the wg-hive mesh, a real and unrelated consumer.
Verified: cargo build/clippy/test -p hive-c0re -p swarm-controller all
clean (needed nix develop -c per the usual -lsqlite3 gap), all touched
nix files pass nix-instantiate --parse, and a throwaway nixosSystem
eval confirms the wireguard mesh still configures a peer's
wireguardAddress into wg-hive correctly with certFingerprint gone.
Follow-up to the peer-hives removal: docs/web-ui/dashboard.md drops
the whole "P33R H1V3S" section + its stale #peers hash and tab-strip
mentions; docs/swarm/README.md's runtime-effects list and
cross-references point at swarm-ui's roster page instead. Nix comments
in hive-c0re/environment.nix, swarm.nix and swarm-wireguard.nix that
described HYPERHIVE_PEERS/certFingerprint as dashboard-consumed are
updated to say what's actually still true (agent identity only).
Deliberately NOT removed: the `certFingerprint` nix option itself.
It has no remaining Rust consumer, but deleting a documented,
potentially already-set option is a bigger, breaking change than
removing dead display code — flagged in docs/swarm/README.md with a
note instead of silently dropped, so an operator or reviewer can make
that call explicitly rather than have it made for them here.
mara: 'there should only be the one via otel-collector'. Collapses
collector.enable away - enabling otel runs the collector, and agents
always export to it.
The argument for it is the same one the collector exists for: as long
as the direct path stays selectable, the credential-in-settings.json
hole stays selectable. An option that can reintroduce a hole is a hole
with extra steps. HYPERHIVE_OTEL_HEADERS_CREDENTIAL is now never
forwarded to containers at all rather than conditionally.
Two consequences, both deliberate:
- the enable option's own promise changes. It said each harness exports
directly so telemetry survives anything host-side being down; that is
now false for every hive rather than for opted-in ones. Stated in the
option text rather than left for a reader to discover.
- the collector-requires-headersCredential assertion is gone. It was
correct only under the shape it was written for: when the collector
was opt-in FOR the credential, opting in without one was pure
indirection. With one path it isn't - an upstream needing no auth
header is legitimate, and the assertion would have rejected it. The
upstream header block is now omitted entirely in that case, rather
than rendering an ${env:...} reference nothing sets.
The agent-facing endpoint becomes derived rather than `otel.endpoint`
re-defined: with the collector on, agents and hive-c0re's own exporter
send to the bridge address; with it off the expression is the old value
character for character. `otel.endpoint` keeps meaning "where telemetry
ultimately goes" on every hive that already sets it.
Suppressing HYPERHIVE_OTEL_HEADERS_CREDENTIAL is the half that actually
closes the hole this issue is about. The credential path host -> container
is sound (/run/credentials is dr-x------ root), but the hive-otel-header
oneshot then writes the value into the agent's own settings.json, mode
0600 and owned by the agent — so today every agent can read the hive's
upstream telemetry credential. With a collector there is nothing to
forward.
The protocol is derived for the same reason the endpoint is: the first
hop is now the collector's OTLP/HTTP receiver, while otel.protocol
describes the upstream link.
Moving agents.conf/gateway.htpasswd out of StateDirectory (0e1a975f)
left hive-c0re unable to write them at all: ProtectSystem=strict only
keeps StateDirectory and RuntimeDirectory writable, so every write to
/var/lib/hive-gateway/conf failed with EROFS. agents.conf never got
past the tmpfiles pre-seed, so nginx had no location block for any
agent.
Prose-only sweep of every remaining claim that nginx or dnsmasq lives
in a container: the port comment (root in a container -> root on the
host), upstreamHost's netns rationale, the ACME state dir, the store
path reachability note, the vhost tree header, dnsmasq's resolv.conf
paragraph (there is no copy and no path unit watching it any more),
the two hive-network bridge comments, and swarm-controller's socket
access-control note, which described a bind-mount that no longer
exists.
No behaviour change; all of it was describing a mechanism that was
deleted.
core_avatar_png() and config_org_avatar_png() were hive-sh4re
functions used only by hive-c0re::forge::users. Move them there as
private path-resolution helpers (core_avatar_png_path,
config_org_avatar_png_path), following the existing
std::env::var(...).expect(...) required-env-var style used elsewhere
in hive-c0re.
The org avatar is now independently configurable:
services.hyperhive.c0re.orgAvatarPng (nullable path, default null)
lets an operator override just the agent-configs org avatar PNG
without replacing the whole assets package. Wired via a new
HIVE_ORG_AVATAR_PNG env var that falls back to the bundled PNG when
the option is unset. The core avatar stays under HIVE_ASSETS_DIR,
unchanged.
One attrset describing every hive in the swarm including this one,
identical on every host, with hiveName selecting which entry is us.
"My peers" is derived (swarm.peerHives) rather than declared.
Every field in the old per-host peer list was intrinsic to the hive it
described, never to the pair -- so the list was a directory each host
kept its own copy of. Beyond the deduplication it removes a bug class:
two hosts could hold different endpoints for the same third hive with
nothing to detect the disagreement.
Drops the per-hive caCert. Trust inside a swarm derives from the swarm
root, which every hive chains to. What that genuinely removes is
trusting a hive whose root this swarm does not own -- a cross-swarm
problem that wants a mechanism of its own, not a field that happened to
work.
The matrix container's certificateFiles block goes with it and could
NOT be migrated: that list is read at build time and the swarm root is
a runtime file (its key must never enter the store), so there is no
build-time name to put there. caCert being a nix path was precisely
what made it the build-time distribution channel. Agents are unaffected
-- hive-tls folds the root into the hive trust bundle and the meta
renderer embeds that one file. Tracked separately.
Migration is an assertion plus warnings, not a rename: hives is peers
union {self}, and the set gains a member no existing config has written
down. A rename migrates a name and a default can re-root a meaning;
neither can conjure a new member. The warning explains, the self-entry
assertion stops the build.
Second slice of the swarm-global service consolidation, same shape as
the forge move: the operator-facing host options become
services.hyperhive.swarm.matrix.*, and one mkRenamedOptionModule on the
namespace carries the whole subtree (nested gui.* included), so there is
no leaf list to forget an entry from.
The rename lives in hive-matrix.nix, the module that declares the
options, so each service's migration stays independent of its siblings.
The per-agent hyperhive.matrix.{enable,url} and hyperhive.matrixAccounts
are a different namespace -- a client pointer at the service, not the
service -- and deliberately do not move.
services.hyperhive.swarmName becomes services.hyperhive.swarm.name, with
one mkRenamedOptionModule in hyperhive.nix -- the module that declares
it, same convention as the forge and matrix renames, so each migration
stays independent of its siblings.
hiveName deliberately stays where it is. It names this hive; swarm.name
names the group the hive belongs to, and that they now sit one level
apart is the distinction rather than an inconsistency.
The per-agent hyperhive.swarmName is an internal mirror rendered from
the host value and does not move, same split as forge and matrix.
Forgejo is a swarm-global service, so its operator-facing host options
move to services.hyperhive.swarm.forge (and .swarm.forge.ci) as the
first of the namespace consolidation.
Existing hive configs keep evaluating: swarm-renames.nix maps every
moved leaf with mkRenamedOptionModule, which also emits a deprecation
warning naming both the old and new path, so an operator is told what to
rename rather than discovering it from a failed eval.
The per-agent hyperhive.forge.url does NOT move. It is a client pointer
at whatever forge an agent talks to - it shares a word with the service
and nothing else, and the two are already documented as separate option
surfaces.
Verified by evaluating the host module, since no Rust gate evaluates
nix: setting the old paths and reading the new ones yields the values
(httpPort 3999, ci.concurrency 7), and config.warnings carries the
rename notice.
Nix half of the 4th layer mara found (47535: core cannot assume matrix
is on localhost). Rust half (matrix.rs MATRIX_HTTP) NOT done.
Parked here rather than left dirty: she has redirected me to jobq as
prio 1, and uncommitted files migrate across a checkout.
Third and last of #2860's agent-facing URL fallbacks. The operator's
ruling was "any special casing is done on the nix side - same binaries,
no hard coded fallback", so the default is deleted rather than replaced.
Every layer guessed the same wrong thing, and each guess was only ever
correct for a process sharing the host netns:
- nix/agent-modules/matrix.nix: matrixUrlDefault = localhost:8008, both
as the option's default and as a sentinel the daemon unit compared
against to decide whether to write HIVE_MATRIX_URL. Now nullOr str,
default null, the guard is != null, and the doc says what forge.url's
already says: null means "no matrix", not "guess one".
- nix/host-modules/hive-c0re/environment.nix: forwarded
http://127.0.0.1:<port> when no gatewayHost was set. hive-c0re shares
the host netns so it reads as harmless, but the value is handed to
agents, which do not -- there it names the agent itself. Now forwarded
only when there is a gateway vhost to name, matching the guard
HIVE_MATRIX_PUBLIC_URL already uses twelve lines below.
- hive-matrix-mcp: paths::DEFAULT_HOMESERVER was the same address
compiled in, so dropping the nix defaults alone would have left the
daemon dialling loopback inside the agent's own netns -- the very bug,
one layer down. homeserver_url() is now Option, and an account with no
homeserver is skipped with a log, exactly as one with no token is.
discover_token_accounts already refused to guess for the same reason.
Two comments taught the assumption back to the next reader ("shared host
netns means every agent container resolves localhost to the same
machine"); both now say which side of the netns boundary they describe.
MATRIX_HTTP keeps its value -- hive-c0re really does share the host
netns -- but no longer claims agents do.
Gated with nix eval against the extended agent-base config, as a pair:
with no url set the daemon unit carries no HIVE_MATRIX_URL, and with one
set it carries exactly that. Either check alone passes on a broken guard.
Adds services.hyperhive.forge.publicUrl (defaults to the gateway vhost
URL when behindGateway=true, null otherwise). HIVE_FORGE_PUBLIC_URL is
now sourced from it instead of hardcoding https://${forge.domain}
whenever behindGateway is on.
The 4 frontend call sites that built a forge link from
state.forge_public_url now hide the link when that's absent, rather
than guessing http://<browser-hostname>:3000 — a guess that's only
correct by accident once the operator isn't on plain localhost. Fixes
the dashboard H0M3 tile, per-agent-row forge links + agent menu, the
approval-queue PR link, and the per-agent page's own meta-nav forge
link (found during this pass, same defect, not in the original
3-site inventory).
Docs + doc-comments updated to match.
services.hyperhive.c0re.swaggerUiTheme -> services.hyperhive.gateway.swaggerUiTheme.
hive-c0re has no relationship to this option (it's consumed entirely
by the gateway's nginx config) since the previous two commits moved
the theme files out of hive-c0re/ and gateway to hosting the whole
dist itself — the option shouldn't live in c0re's namespace either.
Pure option move: default wiring in flake.nix now sets
services.hyperhive.gateway.swaggerUiTheme instead of the c0re one;
hive-gateway/default.nix reads it via the module's own already-bound
cfg (services.hyperhive.gateway) instead of reaching into c0re's.
Verified: nix build .#swagger-ui-theme unaffected (same store path,
pure option-plumbing change, no content change); a full nixosSystem
eval of nixosModules.default resolves the option at its new path and
confirms the gateway's /api/docs/ location still aliases the correct
store path.
Extends the theme-only alias into the full shape mara asked for on the
PR thread:
1. nix/packages/swagger-ui-dist.nix — plain vendored Swagger UI 5.17.14
dist, sourced directly from the swagger-ui-dist npm package (same
release the Rust utoipa-swagger-ui-vendored crate ships, verified
via matching gitHead commit) rather than through Cargo.lock/cargo.
2. nix/packages/swagger-ui-theme.nix — overlays our 3 override files
(index.html, hyperhive-theme.css, and now swagger-initializer.js)
onto (1).
3. vhosts.nix's swaggerUiLocations now prefix-matches the whole
/api/docs/ tree (not just 2 exact-match files) straight from (2),
plus a `= /api/docs` redirect shim since hive-c0re's own redirect
is going away too. /api/openapi.json (outside this prefix) keeps
proxying to c0re unchanged — that's the one thing that stays
dynamic.
New file swagger-initializer.js needed hand-verification: the plain
vendored copy hardcodes the swagger.io petstore demo URL.
utoipa-swagger-ui normally rewrites it per-request from a {{config}}
placeholder its own build.rs injects — since hive-c0re won't be
serving this file at all once its SwaggerUi mount is removed, that
rewrite has to be baked in statically here instead. Derived by
actually running build.rs's own two transforms (strip the default
layout: line, splice the Config JSON in place of the url/deepLinking
block) against the real vendored file, not typed from scratch —
verified byte-for-byte against what format_config() would produce for
hive-c0re's actual single-URL config, and checked with node --check.
Coordinated with damocles: he's taking the corresponding hive-c0re
side (drop the utoipa-swagger-ui dependency + SwaggerUi::new(...)
mount, keep only the plain /api/openapi.json route) once this lands.
Verified: nix fmt clean; nix build .#swagger-ui-theme succeeds, output
byte-matches the checked-in override files and node --check passes on
swagger-initializer.js; a full nixosSystem eval of nixosModules.default
resolves both new locations (/api/docs/ aliased to the right store
path, = /api/docs redirecting) with auth threaded through.
The push side modelled a store per peer hive: a --peer argument, a
swarm.peers.<domain>.snapshotStorePort option, and a swarm_peers module
whose entire job was answering "which peer". A swarm has exactly one
store, so none of that had anything to select between.
The receiver already proved it. It keys destination directories by
agent, not by sending hive, precisely so an agent that migrates keeps
one unbroken incremental chain -- which only makes sense if every hive
pushes to the same place. Per-hive stores would split the chain in two,
the case that keying exists to prevent.
So the destination moves to services.hyperhive.swarm.snapshotStore,
rendered into HYPERHIVE_SNAPSHOT_STORE, and swarm_peers is deleted
rather than adapted. address has no default because it is a
deployment fact this host cannot derive; port defaults because it is a
convention both ends read from the same option docs. An unset or empty
address fails naming the option instead of connecting somewhere
arbitrary, and a test asserts the message suggests no value.
Adds the caller the fd-passing machinery existed for: hivectl agent
<name> subvol snapshot push --peer <hive> resolves the peer, connects
to its snapshot store, writes the agent header, and hands the connected
socket to hive-priv, which runs btrfs send straight into it.
The split keeps the root helper ignorant. Everything that involves
knowing where a peer is, what the wire protocol looks like, and which
hive to trust happens in the unprivileged daemon; hive-priv only ever
receives an already-open descriptor. Once btrfs send starts, neither
process is in the data path, so a multi-gigabyte transfer costs no
per-byte work and survives a hive-c0re restart.
call_with_fd takes the descriptor by value and closes it as soon as the
kernel has it. A socket stays open until every copy closes, so holding
one back would leave the receiver waiting for an EOF that never comes:
btrfs receive blocks and this side reports success for a transfer the
peer never committed. Ownership makes that unrepresentable.
The peer's store port is a new swarm.peers.<domain>.snapshotStorePort
option rather than a constant matching the module default. A pushing
hive cannot read the receiver's configuration, so assuming 51821 would
push at a port nobody promised to listen on; absent, the push fails
naming the option. swarm_peers parses the mesh address the host module
has always rendered into HYPERHIVE_PEERS but nothing read.
`builtins.toJSON` already serialises a derivation as its out path and
`null` as `null`, so the `if … then null else "${…}"` binding was doing
by hand what the serialiser does anyway. Hand the package in whole and
drop the intermediate.
The gc-root property is unchanged and re-measured on the real module:
`environment.etc."hyperhive/serve.json".text` still `hasContext`, so the
host system closure still holds the package alive. Verified both ways —
with the package set the rendered JSON is byte-identical to the
interpolated version, and unset still emits `null` — with all module
assertions passing in each case.
The assertion now checks `builtins.toJSON cfg.claudeCodePackage`, which
is the value that actually has to carry the context, rather than an
intermediate that no longer exists.
mara on PR #2769: "make the default null instead of special casing """.
`claude_code_path` was a `String` whose empty value meant "no host-level
pin". That is a sentinel doing an `Option`'s job — the same shape argus
and mara already rejected on #2755's weights, and the same
empty-field cruft mara called out on #2756.
So it is `Option<String>` end to end:
- host module: `claudeCodePath` evaluates to `null` when
`claudeCodePackage` is unset, so `serve.json` carries JSON `null`
rather than `""`.
- `Coordinator` + `HiveEnv`: `Option<String>`, defaulting to `None`.
- `render_flake`/`render_flake_with_lookup`: `Option<&str>`, and the
emission is an `if let Some(path)` instead of an `is_empty()` guard.
- agent module: `hyperhive.claudeCodePath` is `nullOr str`, default
`null`.
Behaviour is unchanged in both directions; only the way "unset" is
spelled moves. The `builtins.hasContext` assertion still guards the
pinned case (short-circuited by the null check, so an unpinned hive
never evaluates it).
16/16 `meta::` tests, clippy clean, `nix fmt` no-op, `nix build .#docs`
green.
Agents run whatever `claude-code` the meta flake's `nixpkgs` resolves
to, and that is normally a release channel. This one package moves fast
enough that stable trails unstable by weeks — 26.05 is on 2.1.187 while
unstable carries 2.1.220 — and an agent cannot fix it for itself: it
only ever sees the single nixpkgs hive-c0re injects, so an `agent.nix`
has no other tree to reach for.
New host option `services.hyperhive.c0re.claudeCodePackage` takes the
package directly and rides the existing `hyperhiveDocs` threading path —
serveConfigJson -> HiveEnv -> render_flake — to reach each agent as
`hyperhive.claudeCodePath`. Null (the default) is today's behaviour.
What travels is the store *path*, as a plain string literal, not a flake
input: containers share the host's `/nix/store`, so the build is already
reachable inside them with its whole closure and has nothing to travel.
An input would be worse than useless — a `path:/nix/store/<pkg>` input
is re-copied as a reference-less `-source`, which strips exactly the
closure the binary needs.
The catch is that a path written into a generated flake is text, so
nothing in the container's closure keeps the binary alive. The host does
that instead, and gets it for free: the package is interpolated into
`/etc/hyperhive/serve.json`, `builtins.toJSON` preserves string context,
so the /etc entry references it and the system closure gc-roots it for
as long as that generation is the one the agents were rendered from. An
assertion pins that property, because losing the context is invisible at
eval and at deploy — it would surface only as every agent failing to
spawn `claude` whenever the next gc ran.
Container side wraps the path in a symlink farm rather than putting it
on PATH directly: `systemd.services.<name>.path` and
`environment.systemPackages` both coerce a store-path *string* through
`lib.toDerivation`, i.e. `builtins.storePath`, which pure evaluation
rejects. Interpolating the path into a builder is just text and
evaluates anywhere. `claude-code` drops out of systemPackages when a
pin is set, so there is exactly one claude in the container.
Refs #2693
Encoding "not configured" as weight 0 worked (the writer omitted the
line) but the type lied: 0 is not a legal cgroup v2 weight, and every
reader had to know the sentinel. Use Option<u32> end to end instead —
wire type, priv_client, HiveEnv, drop-in writer — so "unset" is a state
of the type rather than a magic value.
The nix options become nullOr, keeping their default of 80; null now
expresses "leave the setting out of the drop-in entirely" declaratively,
which is the useful shape on a host whose IO scheduler ignores
io.weight anyway.
Backward compat is unchanged: the fields stay #[serde(default)], so a
request from an older hive-c0re deserialises to None and reproduces the
pre-weights drop-in byte for byte. The test that pins that now passes
None instead of 0.
`CPUQuota=`/`MemoryMax=` are hard caps: they throttle an agent even when
the host is idle, so they are the wrong tool for "be polite under
contention". The cgroup v2 relative shares are, and neither was wired.
Adds `services.hyperhive.{agentCpuWeight,agentIoWeight}` (1..=10000,
default 80) threaded through the existing drop-in path: HiveEnv ->
write_dropins -> WriteResourceLimits -> hyperhive-limits.conf, next to
the caps already there. Hive-wide only, as the operator scoped it on the
issue: no per-agent override, no resource-limits.json field, no
dashboard form.
The default of 80 is below the kernel's 100, so agent containers yield
to everything *not* on this drop-in path -- host services and the infra
containers (hive-ci, hive-forge, hive-gateway, hive-matrix). It does not
rank agents against each other; they all carry the same weight.
`WriteResourceLimits` gains two `#[serde(default)]` fields, and the
writer treats weight 0 as "not configured" and omits the line, so an
older hive-c0re talking to a newer hive-priv still produces the exact
pre-weights drop-in. The body is extracted into `limits_dropin_body` so
that is covered by a test rather than asserted by eye.
The host admin socket `/run/hyperhive/host.sock` was `0660 root:root` (no
SocketGroup), so hivectl needed sudo. Group-own it by a new `hive-admin`
group and add a `services.hyperhive.c0re.adminUsers` allowlist: listed users
join `hive-admin` and drive hivectl without root.
- `SocketGroup = "hive-admin"`, `SocketMode = "0660"` on the hive-c0re.socket
unit.
- `/run/hyperhive` -> `0751` (traverse-only, no listing) so the group can reach
the socket path; the socket's own `0660 hive-admin` mode gates the
connection, and the per-agent subdirs keep their own restrictive perms.
- Empty `adminUsers` (the default) leaves `hive-admin` memberless -> root-only,
as before.
The admin socket is full hive control (spawn/kill/destroy/deploy), so
`adminUsers` is an explicit, opt-in trust grant. Documented in
docs/boundary.md (host admin socket access) + docs/tools/hivectl.md.
hive-core now fetches each agent's config as a forge-hosted flake input
(git+http://<forge>/agent-configs/<n>.git). Add a git credential helper
(git-credential-hive-forge) that reads the live forge-core admin token on
every invocation and authenticates as the forge core user, wired via the
[credential] stanza in hive-core's $HOME/.gitconfig + on the service PATH.
Reading the token file live means zero stale copies and no resync on
rotation; no token ever lands in a flake URL or lock. Mirrors the existing
github.nix credential-helper pattern.
hive-c0re already samples each agent container's cgroup load for the
dashboard (stats/container_stats.rs); this rides those gauges out to the
configured OTLP endpoint, reusing the existing services.hyperhive.otel
config (endpoint + auth header) — no new toggle.
- New stats/otel_metrics.rs: exports via the OpenTelemetry Rust SDK (same
crates as hive-metric) with the semconv container.* metric names +
container.name attribute so off-the-shelf OTel/Grafana dashboards work,
plus the hive agent label. container.cpu.time (counter, s, from cumulative
cpu.stat usage_usec), container.memory.usage, container.memory.usage.limit;
memory peak / on-disk storage / instantaneous cpu percent stay hyperhive.*
custom (no semconv equivalent). Observable instruments read a shared
snapshot an async task refreshes (gather() is async; SDK callbacks sync).
- container_stats: expose cpu_time_usec (cumulative) on ContainerResource.
- The OTLP auth header is loaded onto hive-c0re's own unit via systemd
LoadCredential and read from $CREDENTIALS_DIRECTORY/otel-headers.
- docs/observability.md documents the host-emitted semconv metrics.
Host-side export, so it covers containers even when their agent is idle.
