A node no longer hands back a recipe for the scheduler to replay later. It
declares straight onto a builder it was given, and that builder is inserted
as part of completing the node.
Deleted: `pub type Declare`, `struct NodeOutput` (+ its hand-written `Debug`),
`JobQueue::append_subgraph`. Nothing added to `Dag` / `DagView`.
jobq gains `Scheduler::new_job()` (the only way to obtain a `JobBuilder`) and
`complete_growing(id, outcome, grown)`, which inserts under `id` and *then*
completes it, so a DAG cannot roll terminal while grown work is still pending.
`complete()` and `complete_growing()` share a private `finish()` rather than
one redirecting through the other. The DAG-gone guard lives beside the graph
now, where it cannot be skipped, instead of being a caller-side lookup.
The growth executors return data (`run_meta_lock -> (Vec<String>, RebuildOpts)`,
`run_reconcile -> Option<NodeKind>`) rather than taking the builder: a `&Job`
parameter is live for the whole function body, and `&RefCell<T>` is never
`Send`, so an async fn taking one cannot be spawned. `run_node` threads the
builder by value and hands it back.
A node can now declare work and then fail, which was previously inexpressible.
`grown` is dropped in that case — failure cancel-cascades downstream, so
inserting it would only add nodes to immediately cancel — and the log line
carries `grown_nodes` so the drop is visible.
Follow-up from #2872: the comment said 'the rest of the (much larger)
route table below is undocumented for now' -- that was true when the
annotation sweep started, not anymore. Trimmed to state the current
fact plainly instead of narrating the sweep's history (mara: is that
level of detail relevant here) -- this is a plain rustdoc comment on
ApiDoc, not part of the actual OpenAPI JSON (that comes from the
separate #[openapi(info(description = ...))] attribute below it), so
keeping it terse and mechanical is the right call.
Swagger UI's endpoint-list row already shows the HTTP method badge +
path for every row, so restating `METHOD /path` at the start of a
handler's own summary is pure duplication. Strips that self-referential
prefix from every summary that has it and re-capitalizes what follows
as a standalone sentence.
Left two false positives untouched: misc_api.rs's operator-inbox
summary cross-references a *different* sibling endpoint
(mark-all-read) for context, and topology.rs's SetParentForm struct
doc happens to mention its endpoint's path but isn't a handler summary
line. Both are legitimate, not redundant.
utoipa splits a handler's doc comment on the first blank `///` line:
everything before it becomes the OpenAPI `summary` (shown in Swagger
UI's collapsed endpoint-list row), everything after becomes the
`description` (only shown once that row is expanded). With no blank
line, the whole doc comment becomes the summary and the description is
empty — which is what every handler in hive-c0re/src/dashboard/ was
doing, so the all-endpoints list showed full multi-sentence prose next
to every route instead of a short one-liner.
For every `#[utoipa::path(...)]`-annotated handler across the 19 files
in that module:
- Inserted a blank `///` line after the first short sentence/clause so
utoipa's split produces a real summary + description, where the doc
comment had more to say. Left already-short single-clause docs alone
(nothing to split).
- Where a query struct derives `IntoParams`, moved param prose that
duplicated a field's own doc comment out of the handler doc (the
field already documents itself in the generated spec), or added a
field doc where the handler explained a param that had none.
No behavior changes — doc comments and `params()` description text
only. Verified `cargo build -p hive-c0re` (clean) and `nix fmt` (zero
changes) after.
Closes#2969
Swagger UI itself is nginx-hosted now (iris's 86a39c4c), so c0re
carrying its own vendored copy via utoipa-swagger-ui was a straight
duplicate — dropped the dependency (root Cargo.toml + hive-c0re's),
swapped the SwaggerUi::new(...).url(...) mount for a plain
/api/openapi.json GET route serving the same OpenApi doc as JSON.
Verified: cargo build/clippy/test -p hive-c0re clean, Cargo.lock
dropped utoipa-swagger-ui + utoipa-swagger-ui-vendored with no other
changes, nix fmt clean.
hive-c0re doesn't touch these files anymore (nginx hosts them
directly, see the previous commit) — hive-c0re/swagger-ui-theme/ was
a leftover from the original SWAGGER_UI_OVERWRITE_FOLDER build-hook
design, which this PR moved away from. New location matches the
existing top-level-directory-for-nix-packaged-assets convention
(branding/, claude-plugins/).
Pure rename, no content change: nix/packages/swagger-ui-theme.nix's
three file references updated, everything else picks it up from the
same content hash (nix build .#swagger-ui-theme resolves to the exact
same store path as before the move).
Extends the theme-only alias into the full shape mara asked for on the
PR thread:
1. nix/packages/swagger-ui-dist.nix — plain vendored Swagger UI 5.17.14
dist, sourced directly from the swagger-ui-dist npm package (same
release the Rust utoipa-swagger-ui-vendored crate ships, verified
via matching gitHead commit) rather than through Cargo.lock/cargo.
2. nix/packages/swagger-ui-theme.nix — overlays our 3 override files
(index.html, hyperhive-theme.css, and now swagger-initializer.js)
onto (1).
3. vhosts.nix's swaggerUiLocations now prefix-matches the whole
/api/docs/ tree (not just 2 exact-match files) straight from (2),
plus a `= /api/docs` redirect shim since hive-c0re's own redirect
is going away too. /api/openapi.json (outside this prefix) keeps
proxying to c0re unchanged — that's the one thing that stays
dynamic.
New file swagger-initializer.js needed hand-verification: the plain
vendored copy hardcodes the swagger.io petstore demo URL.
utoipa-swagger-ui normally rewrites it per-request from a {{config}}
placeholder its own build.rs injects — since hive-c0re won't be
serving this file at all once its SwaggerUi mount is removed, that
rewrite has to be baked in statically here instead. Derived by
actually running build.rs's own two transforms (strip the default
layout: line, splice the Config JSON in place of the url/deepLinking
block) against the real vendored file, not typed from scratch —
verified byte-for-byte against what format_config() would produce for
hive-c0re's actual single-URL config, and checked with node --check.
Coordinated with damocles: he's taking the corresponding hive-c0re
side (drop the utoipa-swagger-ui dependency + SwaggerUi::new(...)
mount, keep only the plain /api/openapi.json route) once this lands.
Verified: nix fmt clean; nix build .#swagger-ui-theme succeeds, output
byte-matches the checked-in override files and node --check passes on
swagger-initializer.js; a full nixosSystem eval of nixosModules.default
resolves both new locations (/api/docs/ aliased to the right store
path, = /api/docs redirecting) with auth threaded through.
Cut the process-narrative framing (operator quote, backstory) and
shortened per-section labels — the CSS was carrying more explanation
than declaration. Keep only what a future reader actually needs: the
override mechanism, the stylix-follows-live fact, and the two
non-obvious calls (method-colour exception, the one !important).
Loads the dashboard's own themed /static/colors.css directly (same
origin, see vhosts.nix dashboardProxyLocation) instead of redeclaring
the Catppuccin Mocha hex values locally. The theme now re-themes live
with the rest of the dashboard instead of drifting out of sync.
Addresses review feedback on PR.
Not yet wired up -- these are the SWAGGER_UI_OVERWRITE_FOLDER payload
files (utoipa-swagger-ui's build-time overwrite hook), inert until
something points that env var at hive-c0re/swagger-ui-theme/ during
the crate's cargo build. See message to damocles for the nix-side ask.
hive-c0re/swagger-ui-theme/index.html: byte-identical to the vendored
swagger-ui-5.17.14 dist's own index.html (utoipa-swagger-ui-vendored
0.1.2), plus one added <link> to hyperhive-theme.css.
hive-c0re/swagger-ui-theme/hyperhive-theme.css: partial reskin per
mara's steer on hyperhive#2966 ("apply theme colors only", not a full
redesign) -- page background, topbar, borders, links, headings, form
controls recoloured to the Catppuccin Mocha palette already used
elsewhere (frontend/packages/shared/src/colors.css); Swagger's own
semantic HTTP-method badge colours and execute/cancel action-button
colours left untouched, they carry real meaning independent of
hyperhive branding.
Two review findings on the resources-at-construction change.
argus: `workers::auto_update`'s boot sweep constructs nodes through
`templates::node` too, and it was not converted. With the kind-derived
declaration gone, its sweep `MetaLock` and its per-agent `Reconcile`
silently declared no resources at all — so a boot reconcile no longer
held the agent lease and could race another DAG's container ops, and the
sweep's meta commit could land inside another node's staged deploy
window. Nothing failed to compile: removing an implicit behaviour from a
helper is invisible at every call site that relied on it.
The declarations now live in a pure `boot_nodes`, split out of
`submit_boot_tree` so they can be exercised without a `Coordinator`.
That path is the only place job nodes are built outside `job_queue/`,
which is exactly why it had no coverage; `boot_sweep_nodes_declare_
their_own_resources` closes that, asserting against declared graph edges
rather than against the kind.
mara: `templates::node` is a redundant redirect now that it no longer
derives resources — deleted, and its 43 call sites use `Job::node`
directly. The reasoning it documented moved to the module docs of
`templates.rs` and `resource.rs`, which is where it stays true.
Resources were derived from the node's kind: `templates::node` called
`NodeKind::resource_deps()`, which fanned out to `needs_build_slot` /
`needs_lease` / `needs_meta_window`. That made the requirement a property
of the *kind*, so a kind that happened to run under an ancestor already
holding the resource could get away with declaring nothing.
Three did. `Start`, `Stop` and `PostSwap` appear in none of the three
predicates, and that was only safe because one construction site fans
them out from inside a lease-holding `Reconcile` — a fact about today's
DAG shape, not about the nodes.
Each of the 41 construction sites now says what it holds. `Start` /
`Stop` / `PostSwap` declare the agent lease; per the contract that is a
re-entrant borrow, which a new test pins rather than argues.
`running_transients` reads the node's declared deps instead of
re-deriving from the kind. That closes the blank-pill gap: the pill went
blank during container start, stop and the post-swap tail because the
declaration was missing, not because the filter was wrong.
The deleted predicates carried the only written record of three design
decisions; each moved to the `Resource` variant it constrains rather than
dying with its function.
Three findings from the operator's review, all correct.
1. Two insert_job's. Graph::insert_job had no caller outside hive-jobq's
own tests -- production only ever went through Scheduler::insert_job.
It existed because the graph-level one got written first. Deleted; the
tests moved onto a Scheduler, which is where insertion belongs anyway.
2. insert_job was not atomic, and the previous commit made that worse: a
forward edge or forward parent surfaced mid-loop, leaving the nodes
before it in the graph, and resolve_wanted ran after every insert, so
an unknown handle failed once the whole job was already committed.
The module documented this under "Partial insertion" instead of fixing
it -- prose describing a hole is not a design.
All three are decidable from what the builder holds, so
check_declaration_order now runs before the first insert and the loop
indexes ids directly. A malformed job leaves the graph untouched.
What remains mid-insert is the graph's own rejection (out-of-group
dep, empty DepWhen); closing that needs a dry-run validate on Graph,
which is a separate change.
3. DagSpec no longer boxes its recipe: it is generic over the closure,
which travels from the template that built it straight into submit.
The box bought type inference, and paying for it costs annotations --
`|b: &Job|` at each declaration site (the field needs an HRTB, and an
unannotated closure binds one lifetime) and `+ use<>` on each
returning signature (or the opaque type captures the caller's borrows).
Erasure is still needed where several recipe shapes share one type:
the boxed Declare stays for the executor's append_subgraph, and a test
table uses an erase() helper.
The operator's instruction on the issue was "the closure returns an array
of guids, and enqueue_job returns the node ids in that order". What was
here instead returned a HashMap of everything inserted, and no caller used
the keys: submit dropped the return, insert_group did into_values(), and
the scheduler ignored what append_subgraph handed back. The guid-keyed
lookup was dead weight, and into_values() made that Vec arbitrarily
ordered -- harmless only because nothing read it.
insert_job now takes FnOnce(&JobBuilder) -> Vec<NodeGuid> and returns the
matching ids positionally. A handle from another job is UnknownNode rather
than a silent omission: the return is positional, so a short vector would
misalign every id after it.
c0re's Declare stays FnOnce(&Job) and the wrapper names no handles in one
place, rather than ending seven templates in an empty vector -- a DAG is
addressed by its container node, which submit inserts itself. That frees
insert_group from needing every id, so the node_rt pre-seeding goes too:
NodeRuntime is one Option field and every reader already tolerated a
missing entry (entry().or_default(), get().and_then(), iter().find()).
The tests are the argument for the shape: capturing a handle through a
mutable binding to look it up in the map afterwards collapses into
returning it and destructuring the result.
Follows the jobq change: a builder can no longer be constructed or
inserted outside `hive_jobq`, so `DagSpec` cannot hold one. It carries a
`Declare` — `Box<dyn FnOnce(&Job) + Send>` — and the queue runs it
against a builder jobq owns, at the moment it inserts.
`NodeOutput.append_subgraph` becomes `Vec<Declare>` for the same reason,
and this is where the shape was always heading: that field's doc already
said an executor "cannot reach the queue, so it hands the declaration
back", while its type was a `Vec<Job>` the executor had built itself.
The rejected `build_nodes -> Vec<NodeSpec>` was the first version of that
escape hatch; a recipe is the last one, because there is no job-shaped
value to hand over at all.
Templates and the power-op assemblers move their owned data into the
closure and are otherwise unchanged — `rebuild_nodes`, `node` and the
tail helpers already took `&Job` and returned handles, so only each
template's outermost frame moved.
Two `Debug` impls are hand-written: a closure has nothing to show, and
its nodes do not exist until the queue runs it. `NodeOutput` reports how
many subgraphs were emitted, `DagSpec` its source and reason.
`append_subgraph`'s `is_empty()` early-return is gone — you cannot ask a
recipe whether it will declare anything without running it. It now
inserts and returns an empty id list if nothing was declared, which
takes the queue lock in a case that previously skipped it.
The two in-DAG-growth tests build `Declare`s now, so they exercise the
shape an executor actually produces rather than one only a test could
construct. 45 job-queue tests unchanged and passing.
The submit-time petgraph `toposort` this described is gone — a cycle
needs an edge pointing at a node declared later, and a handle only
exists for a node already declared. Say why the validation pass is
absent rather than leaving a description of one that isn't there.
`templates.rs`'s module doc was 35 lines and over the comment-block
lint's max; it now points here for the reasoning instead of restating
it, and drops the power-op paragraph that `submit.rs` already owns.
Every template built a `Vec<NodeSpec>` whose edges and parents were
positional indices into that vector, so a shape was expressed as
arithmetic: `base + 1`, `stop_root + 2`, `sfu + 1`, and a
`reconcile_index()` helper that read the emitted vector's length to find
out where its own last node had landed. `concat_subgraphs` existed
solely to rebase one per-agent subgraph's indices onto another's.
Templates now declare into a `hive_jobq::JobBuilder` and hold the
handles they get back, so an edge names the node it waits on. The
arithmetic is gone, and with it:
- `NodeSpec` and the job-queue's own index-based `Dep`.
- `insert_group`'s index resolution — it wraps `Scheduler::insert_job`.
- `concat_subgraphs` — per-agent chains share one builder and each keeps
its own root, so independence is structural rather than computed.
- `reconcile_index` and `dep_index`.
- `templates::validate` and its petgraph toposort. It rejected dangling
deps and cycles; both are now unrepresentable, since a handle only
exists for an already-declared node and every edge therefore points
backwards. (petgraph stays in the tree for `agent_config::topology`.)
`NodeOutput.append_subgraph` becomes `Vec<Job>`: an executor cannot
reach the queue, so it hands back declarations and the scheduler inserts
them under its own lock. That is what the in-DAG growth path always
wanted — a transferable declaration, not a vector of specs.
Resource declaration is unchanged in behaviour: the `templates::node`
helper applies `NodeKind::resource_deps()` at the construction site, so
every node still declares what its kind needs. Moving that declaration
to the call sites is #2818's job; this leaves it one place to delete.
Three tests went with the guard they covered — they hand-built malformed
specs out of indices, which is the representation that made those shapes
possible. Two more now read a DAG's shape off the queue rather than out
of a spec vector, which is where it is observable. The remaining 45
job-queue tests are unchanged and still pass: lease serialization,
roll-up, cancel-cascade, in-DAG growth and per-agent concurrency all
behave as before.
It never produced a `Dep::Node`, so returning `Vec<Dep<Resource>>` made
every caller match a variant that cannot occur. `running_transients`
paid for it with a two-arm match to pull the agent out of a lease edge.
`Vec<(Resource, u32)>` says the same thing in the type, and is what the
job builder's `.needs_units(name, count)` takes — the insertion path
wraps it back into a `Dep::Resource` at the one place that still speaks
in edges.
`forge_git_url` spliced `core:<token>@` between scheme and authority, and
that URL is a process argument. `/proc/<pid>/cmdline` is mode 0444 —
world-readable — so the core admin token, which provisions every agent's
forge account, was published to any local user for the lifetime of each
git child. Seven call sites built such a URL.
The credential now travels in the environment instead:
`git_command_authed` sets `http.extraHeader` via `GIT_CONFIG_*`, which
git reads exactly like a config file, and `/proc/<pid>/environ` is 0400 —
owner-only. Same credential, materially smaller audience. The remote is a
plain `http://forge/<org>/<repo>.git`, and `forge_git_url` no longer takes
a token, so the old shape cannot be rebuilt by accident.
`knowledge`'s clone was the one place a credentialed URL was stored as a
named remote — git persists the clone URL into `.git/config`, so the
token sat on disk and every later `pull` authenticated from there. That
is the case `forge::repos::push_config` documents as forbidden ("the
tokenised URL ... deliberately never stored as a named remote"). `pull`
now rewrites `origin` to the plain URL first, which also scrubs the
persisted token from existing deployments, and authenticates from the
environment when a token is available. The repo is public, so the pull
still works without one.
Three call sites also stopped spawning `Command::new("git")` directly,
so they honour the `HYPERHIVE_GIT` path the NixOS module bakes in and
the `kill_on_drop` every other git spawn gets.
The two URL-shape tests now assert the *absence* of a credential, and a
new one decodes the header back to `core:<token>` — without that, a
malformed header would leave every forge operation silently anonymous
with the other assertions still green.
forge_admin interpolated its whole argument vector into the error
context, and two callers pass a live operator password in that vector
(user create --password, user change-password). Any failure of those
commands wrote the password to hive-c0re's log in cleartext -- and the
likeliest trigger is forgejo rejecting a weak password, so the secret
got logged precisely because forgejo refused it.
Redacting the value after --password would repeat the bug the issue is
about: redact_password_line matched one keyword and a differently named
secret walked past it. A denylist fails open, silently, and the next
secret-bearing flag would leak until someone extended the list.
describe_forge_admin keeps the leading verb path and stops at the first
flag, so "user create --username iris --password ..." is reported as
"forgejo admin user create". The verbs are a closed set this crate
chooses itself; argument values never are, so a new flag is excluded by
construction. Nothing useful is lost -- the context says which operation
failed, and the underlying error already carries forgejo's own message
about why.
The same pattern in hive-priv is deliberately untouched: that crate runs
as root and the redactor's shape is still an open question on the issue.
This change holds under either answer.
mint_token interpolated forgejo's raw stdout into its anyhow context on
the parse-failure path, and on that call stdout carries the access token
that was just created. The happy path below it is careful to log only
the user and token names; the error path handed the secret over whole.
It fires exactly when forgejo's output format drifts, which is the same
drift that breaks extract_token in the first place -- so the "help me
debug this" context printed the secret it had failed to find.
Report the shape of the output (bytes, lines) instead of its contents.
That is what diagnoses a version drift anyway: you want to know forgejo
printed something with no token-shaped word in it, not the bytes.
Redaction at a logging call site does not cover the error path.
with_context and bail! are output channels too.
Phase 4 (repoint every container onto `meta#<n>`) and phase 5 (rename
the `root` container to `h-root`) were marker-guarded one-shots for
layouts no live hive still has: containers are rendered onto `meta#<n>`
at creation, and the `h-` prefix has been the naming for far longer than
any deployment predates. A one-shot nobody can still trigger is dead
weight, so both are gone along with `repoint_container`,
`rename_manager_container`, `CONTAINER_TIMEOUT` and the two marker paths.
Phase 6 was not obsolete, only misplaced. Ruth's tool groups are now
seeded by `ensure_root_agent` on the one path that creates her, rather
than re-asserted on every hive-c0re boot. The skip-if-already-set guard
survives the move: a destroy+recreate under the same name must not reset
an operator's chosen group set back to MANAGER_DEFAULT.
That also settles a latent bug. Phase 4's marker check was a `return`,
not a skip, so on any hive carrying the marker phases 5 and 6 never ran
at all — the tool-group backfill, whose whole job was preventing a silent
privilege downgrade, has not executed here in a long time. Moving it to
create-time removes the question rather than answering it.
What stays is convergence: three unguarded, idempotent phases that re-run
each boot and no-op once their state is right. The module doc now names
the three categories so the next person can tell which kind they're
adding.
The doc still described a `label`/`deliberate_stop` parameter pair
inherited from `transient_guard`, which this function replaced and whose
signature it does not share — it only takes `name`.
Rewritten to say what it does and, more usefully, what must not come
through it: the queue answers the same question from the node itself via
`NodeKind::takes_container_down`, so this path is only for the two
operations that have no node behind them yet.
mara on !2910: "also remove imperative path for the things that are not
nodes yet, file follow up issue to fix that".
`TransientGuard`, the stored map and both manual set/clear are gone.
`transient_snapshot()` is derived and nothing else — destroy and
migration show no pill, because there is no node to derive one from. The
pill returns for free when they become nodes.
What those three guards were actually doing, though, was suppressing the
crash watcher, not drawing a pill. `migrate.rs` said so in its own
comment: without it, `crash_watch` fires `ContainerCrash` for every
migrated agent and the manager tries to recover containers that were
stopped on purpose. Destroy is the same — the container disappears
deliberately and nothing in the graph says so.
Deleting them outright would therefore have traded a dashboard pill for
false crash alerts on every destroy and every migration. So the
suppression survives as its own thing, `suppress_crash_watch`, with a
name that says what it is. It is still RAII, and still held for the
operation rather than stamped once, because the crash watcher's grace
window is finite and a destroy is not — a single tombstone would expire
mid-operation. The drop stamps the tombstone, covering the poll that
lands just after.
That leaves RAII in the codebase for exactly one purpose instead of two.
Untangling the pill from the suppression is what made the transient layer
deletable at all.
Follow-up issue for making destroy + migration real queue nodes to
follow; at that point this guard goes too.
Checked with clippy (`--all-targets -D warnings`), `cargo test -p
hive-c0re -p hive-jobq` (322 + 41 passed) and `nix fmt`.
mara on !2910: "why is set_transient still a thing if it completely
derives from nodes?"
It was still a thing because the scheduler mirrored the derived set into
a stored map that every consumer read — derived state computed once and
then cached, with the reconciliation loop existing only to keep the cache
honest. `transient_snapshot()` now derives: `running_transients()` off the
live graph, with the handful of entries that have no node behind them
(destroy, migration) overlaid on top. There is no cached copy left to go
stale or disagree with what is running.
`set_transient` / `clear_transient` split by what they actually do:
`set_manual_transient` / `clear_manual_transient` own the stored map for
the no-node callers, and `emit_transient_set` / `emit_transient_cleared`
publish the edges both paths need.
Two things had to survive, and both are edges rather than state:
- The dashboard's `TransientSet` / `TransientCleared` events. The
scheduler carries the previous derived value and emits the diff.
- The crash watcher's grace window. `recent_transient_within` answers
"was a transient cleared just now?", which is what stops a deliberate
stop from reading as a crash on the next 10s poll — a derived read of
current state cannot answer it, so the clear still stamps. The
scheduler keeps `deliberate_stop` alongside the label precisely so it
is available at clear time: the node it came from is, by definition, no
longer running to be asked.
`TransientState::since` becomes wall-clock and, for derived entries, is
the node's own `started_at` — the true start of the operation rather than
the moment a watcher first noticed it, which is what the old
guard-creation timestamp actually measured.
`running_transients` returns a named `RunningTransient` rather than a
4-tuple; two of its fields are strings and one is a bool whose meaning is
not guessable at a call site.
Note for anyone reaching for a timestamp here: chrono is vendored with
`default-features = false`, so there is no `Utc::now()`. The workspace
convention is `wire_time::now_unix()` / `from_secs()`.
Checked with clippy (`--all-targets -D warnings`), `cargo test -p
hive-c0re -p hive-jobq` (322 + 41 passed) and `nix fmt`.
mara on !2910: "transient guard as well - should be removable now?" — for
the queue path, yes.
`set_transient`'s own doc explained why the RAII guard existed: a
cancelled future must not leak an imperatively-set transient and pin the
dashboard on "rebuilding…" forever. That cannot happen to a derived set.
`running_transients()` is recomputed from the graph every loop, so a node
that stops running stops appearing — there is nothing to own and nothing
to leak.
So the scheduler no longer holds a guard per pill. It keeps the previous
derived value and publishes the transitions, which is the one thing a
derived read cannot express: the dashboard wants `TransientSet` /
`TransientCleared` edges, and the crash watcher wants the *moment* a pill
cleared, since its grace window is what stops an operator stop from
reading as a crash.
That also retires a hazard rather than restating it. The old code carried
a warning that stale guards had to be dropped before new ones were
created, because `TransientGuard::drop` clears by agent with no notion of
which label it was clearing — so a same-agent label change could clear
the pill it had just set. With no guards there is no ordering to get
wrong; clears are emitted before sets so a relabel reads as
clear-then-set rather than two overlapping pills.
`set_transient` / `clear_transient` become `pub(crate)`. The guard stays
for destroy and migration, which have no node behind them and where the
cancellation concern is real.
Checked with clippy (`--all-targets -D warnings`), `cargo test -p
hive-c0re -p hive-jobq` (322 + 41 passed) and `nix fmt`.
mara on !2910: "rename now, we will see if we can remove it later when
some of the users have been removed or work differently."
Nothing is held. The old name described a transient the DAG declared and
kept for its whole lifetime — precisely the thing this PR replaces — so
it outlived its own meaning the moment the derivation landed. The value
is recomputed from the running set on every call.
Kept as a function rather than inlined at its single call site, per the
above: removing it is a later step that depends on its users changing,
not something this PR should force.
Rename plus its two references (the call in `reconcile_transients` and
the module doc link). No behaviour change; the doc comment records what
the old name meant so the rename doesn't erase the reason for it.
Checked with clippy (`--all-targets -D warnings`), `cargo test -p
hive-c0re -p hive-jobq` (322 + 41 passed) and `nix fmt`.
The dashboard pill was declared once per DAG at submit time, so a rebuild
reported `rebuilding` for its entire life — through the prebuild, the
stop, the swap, the tail and the reconcile. It named the intent of the
request, not what was happening.
It is now read off the nodes actually running. A node lights a pill when
it is `Running` and declares the agent's own resource. Declaring is the
test, not targeting: `Prebuild` and `MetaSync` name an agent but are
lease-exempt on purpose (the container keeps serving), so they must not
light one. It is also not the lease *owner* — `resource_state()` answers
"who holds the slot", which is a different question from "what is
running", and a descendant that borrows an ancestor's grant never
appears in that map.
`TransientKind` is gone entirely rather than being re-derived. The label
is the node's own wire tag (`NodeKind::as_str`) — the same vocabulary
`NodeView.kind` already ships, so a pill and a DAG node name an operation
identically and there is no second taxonomy to keep in step. Work with no
node behind it (destroy, migration) supplies its own literal.
`DagSpec::transient`, `Claim::transient`, `DagMeta::transient` and
`NodeKind::Dag`'s `transient` field all go with it.
## the safety half, which is deliberately not the display half
`crash_watch::is_deliberate_stop` used to match a `TransientKind` to
decide whether a vanished container was intentional or a crash. That made
a pill's display vocabulary decide an alerting question, so renaming or
adding a label would silently move the alerting boundary.
`TransientState` now carries two independent fields: `label` (rendered,
nothing branches on it) and `deliberate_stop` (read only by the crash
watcher). The producer sets the second, because the producer is the only
thing that knows — it is not recoverable from the first.
For queue work that value is `NodeKind::takes_container_down()`, and it
is emphatically not "holds a lease": `Create` and `Start` hold the
agent's lease exactly like `Stop` does, and a container dying *while
starting* is a real crash that must keep reporting as one. The default is
`false` on purpose — a wrong `false` costs a spurious crash event, a
wrong `true` swallows a real crash silently.
## known cost, accepted on the issue
A restart no longer reads `restarting`. No `NodeKind` is unique to a
restart — `restart_chain` reuses `Signal` / `StopForUpdate` / `Drain` /
`Reconcile` — because "restart" is a property of the DAG's shape, not of
any node. A restart now reads `signal` / `stop_for_update`, then the
agent returns.
`Start` / `Stop` / `PostSwap` run inside a lease-holding ancestor and
re-declare nothing, so they light no pill and the agent reads idle for
those windows. Closing that is the resources-where-constructed work
(#2818), not this change.
Checked with clippy (`--all-targets -D warnings`), `cargo test -p
hive-c0re -p hive-jobq` (321 + 40 passed) and `nix fmt`.
mara on !2909: "shouldnt node_by_id be part of jobq?" — yes. Resolving a
raw value to a `NodeId` is the exact inverse of `NodeId::get`, which
already lives in hive-jobq, and it is only a search because the graph
owns the counter that makes ids unfabricable. Both halves of that
round-trip belong on the same side of the crate boundary.
Placing it in `QueueInner` also put it in a layer slated for removal, so
the c0re-side helper would have had to move later anyway — and it was
private there, leaving any other caller needing the same resolution to
write the same `nodes().find_map(…)` by hand.
`Graph::resolve_id` replaces it, with a unit test covering the
round-trip and the rejection of a value that was never an id.
While re-reading the diff for that question: the doc comment added in
the previous commit landed *between* `container`'s doc comment and its
signature, silently reattaching "The container node of `dag_id`" to the
new helper and leaving `container` undocumented. Restored.
No behaviour change and no wire change — same search, same call site.
Checked with clippy (`--all-targets -D warnings`), `cargo test -p
hive-jobq -p hive-c0re` (41 + 322 passed) and `nix fmt`.
`JobQueue::cancel(dag_id)` resolved the id to a `NodeKind::Dag` container
and cancelled that. But the container lookup was the only DAG-specific
part — everything that makes cancel work already lives in the scheduler:
`cancel_node` marks the node `Cancelled` and cascades to its pending
descendants, sparing any node whose edge accepts `Cancelled` (which is
what keeps a dropped approval DAG from dangling its row).
So `cancel` now takes any node id. A group root cancels the whole group,
which is what the dashboard's button does today and why nothing about
its behaviour changes: a DAG id *is* its root node's id. An interior
node cancels just that branch — a capability the DAG-scoped version
could not express, covered by the new test (a hive-wide restart drops
one agent's subgraph while the other keeps running).
`QueueInner::node_by_id` replaces `container()` here: same search, same
cost, without asserting the node is a DAG container. `container()` stays
for `first_error` and the append-subgraph guard, which are genuinely
DAG-scoped.
No wire change. The route is `POST /api/rebuild-queue/{id}/cancel` with
a `u64` path param — same type, same route, and the client keeps sending
the same number. Only the param's documented meaning moves from "DAG id"
to "node id".
Checked with clippy (`--all-targets -D warnings`), `cargo test -p
hive-c0re` (322 passed) and `nix fmt`. No option surface touched, so no
nix-eval gate.
handle_list_descendants derived a container's running state from
membership in nixos-container list output, but that command returns
every registered container - stopped and ghost-registered (machined
still holds the name after the process died) ones included - not just
running ones. use lifecycle::is_running (systemctl is-active on the
container's systemd unit) per container instead, same source
container_view::build_all already uses for the dashboard.
fixeshyperhive/hyperhive#2846
The container-resource exporter identified the samples it sent by
container and by hive, but not by machine or by build — so a sample
could not be attributed to the host it came from or the deploy that
produced it.
Both go on the OTEL resource rather than on each data point, for the
same reason `hive` / `swarm` already do: they are constant across one
hive-c0re. Only per-container facts stay per-data-point.
- `host.arch` — mapped to the semconv spelling, not forwarded from
rust's. The two disagree on exactly the architectures this runs on
(`x86_64` / `aarch64` vs `amd64` / `arm64`), and the failure mode is
silent: a dashboard filtering the standard value matches nothing.
A test pins this, since nothing else would catch it.
- `service.version` — the running flake rev, via
`auto_update::current_flake_rev`, the same source the dashboard
snapshot and `get_agent_meta` already use. NOT the crate version:
that's a workspace constant that never moves between deploys, so it
could not answer "which build produced this sample?". Omitted rather
than guessed when the flake ref carries no rev.
`spawn_exporter` takes the flake ref to reach the rev — the string it
needs, not the whole `Coordinator`, so the module's coupling doesn't
widen for one attribute.
The issue's third item, `container`, needs no change: `attrs()` has
emitted `container.name` per data point since this exporter landed.
Checked with clippy (`--all-targets -D warnings`), `cargo test -p
hive-c0re` (321 passed) and `nix fmt`. No option surface is touched, so
no nix-eval gate.
Second of the `Dag` field removals, and the same shape as the first:
`DagSpec`/`NodeKind::Dag` carried an `inputs: Vec<String>` that exactly
one node ever read. Both reads live inside `run_meta_lock` — the
`meta::lock_update` call and the `meta_update_cascade_agents` fan-out —
so the list now rides `NodeKind::MetaLock` itself.
The executor stops touching `Claim` for this node entirely: its dispatch
arm already destructured `MetaLock { sweep, fanout }`, so `inputs` joins
them and the `claim` parameter, which had no other use, is gone.
Falls out of that:
- `Claim::inputs` and `DagMeta::inputs` delete.
- `dag_view`'s DAG-level projection onto the `MetaLock` node reads the
payload instead. The wire `NodeView::inputs` is unchanged: still
populated on the `meta_lock` node alone.
- the boot sweep names no inputs (it bumps `hyperhive` alone via
`lock_update_hyperhive`), which the construction site now says out loud
rather than leaving implicit in an empty DAG-level field.
Checked with clippy (`--all-targets -D warnings`), `cargo test -p
hive-c0re` (320 passed) and `nix fmt`. No option surface is touched, so
no nix-eval gate.
`DagSpec`/`NodeKind::Dag` carried an `Option<i64>` approval id that four
deploy phases read back out through `Claim`, via a fallible helper whose
error ("approval deploy dag N has no approval_id") described a state the
type system should have forbidden. Two other templates (`spawn`,
`meta_update`) set the field for nothing: their approval is resolved by
the `ResolveApproval` tails, which already carry the id themselves.
So the id moves onto the nodes that actually need it —
`DeployWindow` / `MergeVerify` / `DeployApply` / `FinalizeDeploy` /
`DeployTail` each take an `i64`, the same way `ResolveApproval` always
has. `templates::approval_deploy` builds all of them in one place with
the value in hand, and `deploy_rebuild_nodes` takes it as a parameter so
the `FinalizeDeploy` it appends at runtime is constructed the same way.
Falls out of that:
- `deploy_approval_id` and its runtime error path delete; each executor
takes the id from its own node payload at dispatch.
- `run_deploy_window` had nothing left to do but validate that id, so the
node joins `Dag` on the shared no-op arm.
- `Claim::approval_id` and `DagMeta::approval_id` delete.
- `dag_view`'s DAG-level projection onto `DeployWindow` reads the payload
instead. The wire `NodeView::approval_id` is unchanged: still set on
the deploy root alone, so the dashboard still renders one approval link
per DAG rather than one per phase.
No option surface is touched, so there is no nix-eval gate here; checked
with clippy (`--all-targets -D warnings`), `cargo test -p hive-c0re`
(320 passed) and `nix fmt`.
The OTEL resource attributes are baked into every agent's
managed-settings.json at evaluation time, but claude-settings.nix read
the names from the container's environment.variables - where they are
never set. meta.rs forwards them as runtime env only, so the reader hit
its "unknown" fallback and every agent shipped
service.name=hyperhive-agent,agent=<a>,hive=unknown,swarm=unknown
on every metric, while the same process's env held pr1ma/constellation.
Measured in this container's /etc/claude-code/managed-settings.json.
The map from forwarded env var to agent option already existed for the
service URLs, with a doc comment naming this exact hazard: "setting only
one leaves the other on its default". The names were simply never added
to it. They are now, and the constant is renamed FORWARDED_VAR_OPTIONS
since it no longer holds only URLs.
hyperhive.hiveName / hyperhive.swarmName follow the forge.url shape:
nullOr str defaulting to null, where null means the hive did not name
itself and "unknown" is an honest label rather than a guess baked at
eval time.
Also fixes, unasked: CLAUDE_REMOTE_CONTROL_SESSION_NAME_PREFIX reads the
same value, so remote-control sessions were named "unknown-<agent>".
Gated with nix, not just cargo - the blast radius here is module eval,
which fmt/clippy/test cannot see:
agent-base assertions -> [] (no failures)
extendModules with both options -> hive=pr1ma,swarm=constellation
Note the value is baked, so every agent needs a rebuild before the new
label appears on its metrics.
Per mara's review on #2896: has_log: bool was fully redundant once
build_log_id: Option<i64> existed alongside it (has_log was always
just build_log_id.is_some()). Dropped has_log, threading the single
Option<i64> field through job_queue::mod.rs, the hivectl NodeView
test-helper literal, and the one remaining frontend consumer
(findLiveBuild's live-log-panel gate, which now checks
build_log_id != null instead of the separate bool).
Also fixed a now-stale doc comment on GET /api/build-log/{node_id}
that claimed the dashboard used on-demand node-id fetches "instead
of an inline build_log_id on the wire" -- no longer true after this
PR put one there for the BUILD L0GS deep-link.
cargo build/clippy/test clean across the three touched crates; nix
fmt clean; frontend build verified (0 has_log references, 3
build_log_id references in the built builds.js bundle).
Fixes hyperhive#2895. The rebuild-queue tree's per-node log icon (the
printer-glyph "open" affordance next to each node in the R3BU1LD
QU3U3 tab) linked directly to the raw-text download endpoint
(/api/build-log/<node_id>/raw, which sets Content-Disposition:
attachment server-side) -- surprising, since nothing about that icon
signals "this leaves the app", unlike the other two explicit
"download raw"/"raw" links elsewhere on the page.
Point it at the existing ?id=N#buildlogs deep-link into the BUILD
L0GS tab instead (builds.js's fetchBuild already auto-expands +
scrolls to the matching row there). That deep-link's id is the
build-log history row id -- a different id space than the queue
tree's NodeId, and wasn't exposed to the frontend before (only a
derived has_log bool was). Added NodeView.build_log_id: Option<i64>
to the wire type alongside the existing has_log (kept, since
findLiveBuild's separate live-log-panel gate still needs a plain
bool), threaded through job_queue::mod.rs, updated hivectl's NodeView
test-helper literal.
The raw download is still one click away once on that row's BUILD
L0GS detail (the two already-explicit raw-download links are
untouched). cargo build/clippy/test clean across the three touched
crates (hive-c0re, hive-host-sock, hivectl); nix fmt clean; frontend
build verified (grep for build_log_id in the built builds.js bundle).
With the agent option nullable, a missing `HIVE_FORGE_URL` would no
longer fail anything — it would deploy a whole fleet of agents that
silently never log into the forge. The forge is not optional on a
running hive, so the hive asserts that itself rather than leaning on a
module that legitimately allows "no forge" when evaluated standalone.
`sync_agents` checks it before writing anything. That is the moment the
hive commits to a flake, and it keeps `render_flake` a pure string
operation: the renderer is exercised directly by a dozen tests, so
making *it* env-dependent would force each of them to either set a
process-wide var — the parallel-test race this module already avoids —
or fail for reasons unrelated to what they assert.
`require_service_urls` is pure over the already-collected pairs, so its
two tests need no process env at all.
Refs #2860
