The bootstrap block still described a one-shot unit "inside the store's
container". That stopped being true when the unit moved, and a merged
change that leaves the docs asserting the old topology is worse than one
that never moved it.
Also retires the "only run once, didn't go all the way through" note. It
has run twice against a live store now and failed both times — but for
its own reasons, not the grants': it could not reach the store from where
it ran, so neither attempt exercised a single one of them. The grants are
still derived from `-output-policy` rather than proven, and the warning
now says that directly instead of implying a partial test.
`swarm-bao-controller-policy` creates the `swarm-controller` policy and
cert-auth role — the credential every hive logs in with. It has never
succeeded on any deployment, and the reason is where it ran.
Inside the container it had neither of the two things the store demands.
Its `BAO_ADDR` was the public DNS name, which from that netns resolves to
the hive bridge: `dial tcp 10.42.0.1:8200: connect: connection refused`.
And every API listener carries `tls_require_and_verify_client_cert`, while
`tlsDir` holds the server's leaf and the CA that signs clients — no client
identity at all. Fixing only the address moves the failure one hop.
The comment above the unit asserted the opposite — that in there the store
is "reachable without a client certificate at all, which is the point". The
listener config decides that, and says otherwise. That belief is what put
the unit in the container, so it goes with it.
On the host all four coordinates already exist: `baoCli` carries the
address, the CA, the certificate and the key, so the unit needs no
`environment` block at all. `bootstrapTokenFile` was always a host path —
the container only saw it through a bind mount. Nothing new crosses the
boundary; the mount gets no wider.
The retry bound is resized with it. 10 attempts at 30s is five minutes,
and under `seal = "shamir"` an operator unseals by hand, so it would give
up before a human arrived — permanently, because `start-limit-hit` does
not self-heal. That is the same silent no-bootstrap this issue is about.
2880 × 30s covers a day, inside a 25h window.
module-eval follows the unit to the host and gains an arm asserting it is
NOT rendered inside the container: the move is the fix, so the side it
landed on is worth pinning.
`StartLimitBurst` sat in `serviceConfig`, so it rendered into `[Service]`,
where systemd silently ignores it — the unit retried every 30s forever.
Measured on a live store: the journal reports `restart counter is at 18`
against a burst of 10.
This repo already states the rule and pins it with a test:
`hive-priv/src/main.rs` renders its drop-in with `StartLimit*` under
`[Unit]` and says why — "systemd silently ignores them under `[Service]`,
so a bound that moved sections would look configured and do nothing".
That is exactly what happened here, in another module.
Moving the burst alone would not have fixed it. systemd's default window
is 10s while `RestartSec = 30`, so at most one restart falls inside it and
a burst of 10 is unreachable; the interval has to exceed `RestartSec` times
the burst. 600 matches the value hive-priv already uses.
Uses the NixOS service-level options rather than a hand-written
`unitConfig`: nixpkgs renders `startLimitBurst` / `startLimitIntervalSec`
into `unitConfig` itself (`nixos/lib/systemd-lib.nix`), and `hive-ci.nix`
already sets `startLimitIntervalSec` that way.
The module-eval case asserts placement where nixpkgs puts it, and that
`serviceConfig` does not carry it — so moving it back fails the build.
The host now has a wrapped bao (#4154, deployed) carrying the store's
address, CA and client cert, so the init step is a bare command there
instead of a root-login hop with an SNI override. The container route
stays as one sentence explaining why it needs the two extra pieces.
The delete step said "once that has run" without saying what -- the
rebuild, or the unit succeeding. It now names the check that settles it,
because deleting the token before the role exists leaves the unit
skipping forever and looks exactly like a store nobody bootstrapped.
Closes#4162
The doc-comment sweep changed --help text, so the generated reference went
stale and hive-forge-docs-fresh caught it. Regenerated with the command the
check itself prints; the diff is five lines, all propagations of the sweep.
Kept as its own commit rather than an amend: argus approved d090a9fb, and a
force-push clears both the verdict and the review request.
Refs #4155
docs/tools/forge.md already listed the nine renamed verbs as removed, then
used them ~30 more times in pasteable blocks. Sweeps every occurrence a
reader would type, including three runtime messages that told the user to
run a verb the same binary rejects.
The renamed-verb list itself keeps the old names; it is what documents them.
Refs #4155
mcp.md called it `hivectl choom`; the verb is `hivectl agent <name>
choom`. observability.md cited `hivectl shell`, which has never existed
at any level — the sentence only needs a shell, so it names one instead of
guessing which command an operator reaches for.
Found by subtracting the generated hivectl-cli.md's command headings from
every hivectl citation in docs/. Re-running it after the fix drops the
candidate list from 16 to 14, losing exactly these two.
Closes#4159
Reading a role out of the store took four round-trips of environment
guessing: the certificate has no IP SAN so loopback cannot verify, the
DNS name resolves to the bridge from inside the container, and the CA
the client needs lives on the host, which had no `bao` at all.
The wrapper carries the address, the server CA and — where the PKI glue
minted one — the reader's own leaf, so cert-auth login needs nothing
typed. `--set-default` throughout: an operator pointing BAO_ADDR
elsewhere still wins.
Only the wrapper reaches PATH. `wrapProgram` renames the real binary,
so there is no unwrapped `bao` to reach by accident, and the module-eval
case asserting the package's absence is what keeps a later "install it
too" from undoing that.
"On the host that RUNS the store" is ambiguous in the deployment we
actually ship: openbao runs inside the `swarm-bao` container, so the
host and the container are different vantages with different working
addresses, and the instruction reads as either.
From inside the container neither obvious address works — the server
certificate is issued for the store's DNS name and carries no IP SAN,
while that name resolves to the bridge rather than to the container. The
fix is an SNI override: verify the name, connect on loopback. The host's
own address is stated beside it so the reader can pick a vantage instead
of discovering the constraint from a TLS error.
`clientCn` fell back to `cfg.domain` when `hiveName` was unset. That branch
cannot run: `hive-network.nix` asserts `hiveName != null` under
`mkIf services.hyperhive.enable`, and this file's `config` is gated on the same
predicate, so any host that evaluates the conditional has already failed the
assertion.
Worse than dead, it read as a second supported spelling of a hive's identity —
which is what a cert-auth role matches on. It was not even the hive's own
domain: `cfg` here is `services.hyperhive.swarm.bao`, so the fallback resolved
to the store's address, one string shared by every hive in the swarm and the
same CN the server leaf carries.
Reading the option directly matches what other modules needing the name already
do (`hive-c0re/environment.nix`).
A hive holds an mTLS pair and a policy naming what it may read, and still
cannot log in: nothing creates the role that maps its certificate to that
policy. The one pre-shared credential in the system therefore buys no
access.
Minting happens here rather than in nix, which was the first plan. Nix
mints from the store's own container, and that path is gated on the
bootstrap token -- so onboarding a hive later would mean placing the one
genuinely pre-shared secret again. Doing it from the controller costs a
public certificate authority as an input and makes the bootstrap token
one-time.
A startup pass, not a hook: the hive list is loaded once and a config
change means a redeploy, so the roles are as static as the list. Only the
policy is derived from something that moves.
The subject is the hive's name because glue-bao-tls.nix mints a hive's
client leaf with its name as the CN, and cert auth matches on that.
Per-hive failures are logged and skipped, matching the queue, bridge and
forge connects above it: a controller whose store is unreachable still
serves everything else, and the next start retries.
Not covered by a test: ensure_hive_roles is IO from end to end, and the
seam that would make it assertable is the one the read-grant sink already
has. Said here rather than implied by a green suite.
Creating a hive's cert-auth role means writing the authority into the role
by value -- the store matches a presented certificate against the role's own
copy -- and nothing gave this daemon that file.
Named separately from deploy.bao.clientCaFile rather than read off it: that
option is the store's, saying which readers the store trusts on the host
that runs it, while a controller runs anywhere. The glue module supplies it
where the two are co-located, which is the same split baoClientCertFile
already makes against the hive reader's leaf.
Gated on the identity as well as the CA. Without a leaf there is nothing to
write a role with, so the file would reach a daemon that cannot act on it.
The module-eval arm needed a fixture of its own: a deployment that
self-signs both ends points clientCaFile and serverCaFile at one file, so on
the existing fixture the two authorities are the same string and wiring
either into the other's slot passes. controllerTwoCas is where they differ.
Per mara's screenshot report (agent-terminal icons not aligned in the
first column) — and her explicit follow-up steer on the first version of
this fix: 'dont do it by offsets at all, it should be part of the layout
that they align.'
Root cause traced first, not guessed: .row-glyph relied on inheriting
text-indent: -1.4em from .live .row to paint its glyph in the reserved
prefix slot; details.row > summary separately reset text-indent: 0 for
its own hanging-caret needs, which also zeroed the inherited value for
the icon nested inside it (indent inherits by computed value, not by
rule). Two independently-computed offsets that had to agree by hand,
and silently didn't.
Replaced the whole offset scheme with a real structural fix: every row
is a 2-column CSS grid (icon column, width from one shared
--row-icon-col custom property, then content column), and a details
row grids its own <summary> with the exact same grid-template-columns
value instead of griding itself (so its <pre> body still stacks full
width below, not squeezed into column 2). Icon and content are placed
by explicit grid-column, not auto-placement inference, so an icon-less
row's lone child still lands in the content column. .row-content is a
new wrapper class (Row.tsx, terminal.js's row()/mutableRow()/
placeholder()) giving that content an element the grid can target by
class - a DocumentFragment (what linkify() returns) doesn't persist as
a node once appended, so without an explicit wrapper there was nothing
for the grid to place.
Caught and fixed a second real bug while building this for real (not
just reasoning about the CSS): the first draft's details.row { display:
block } had lower specificity than .live .row's display: grid and never
actually applied, squeezing <summary> into the row's own 1.4em icon
column and wrapping its text one character per line. Needed
.live details.row to out-specify it.
Verified with a headless-chromium render of all 6 row shapes (flat
icon / flat icon-less / flat markdown-body / details icon / details
icon-less / a long-wrapping flat row) - all align and wrap correctly.
Also verified via the earlier Range.getBoundingClientRect() measurement
(glyph paint position, not just the element's own box): diff 0.00 for
both this and the previous fix, but only this one is structural rather
than two numbers that happen to still agree today.
Per mara's issue: tables should remember their filters (with a reset
button), the labels filter should be multi-select, and any filter should
support negation (search vs exclude).
All in the shared Table component (ui/table/Table.tsx), used by
AgentsPage/HivesPage/IssueReportPage:
- filters now persist via the same useLocalSetting hook IssueReportPage
already used for its own state, keyed by a new required storageKey
prop (required, not optional, so no caller can forget it and every
table gets persistence for free)
- a small 'reset filters' button clears every column's filter at once,
shown only when at least one is active
- new filterMode: "multiselect" (+ a filterValues extractor, alongside
the existing single-value filterValue) renders a checkbox list and
matches on any overlap - IssueReportPage's own bespoke label-checkbox
sidebar is folded into this instead of staying a second, separate
filter mechanism
- a negate toggle ('exclude') sits under every filter mode's control,
applying uniformly to text/select/multiselect
Verified: tsc --noEmit and the esbuild bundle both clean.
hive-rules.md: no #NNN tags in code, prose only. The two mentions in
the new prose-lint-errors job's comments were exactly that -- reworded
to describe the split without a ticket number.
mara, on #4128: add a CI step (separate from the existing one) that
only flags error-level vale hits, so it can be made a required check
without also gating merges on the pre-existing warning/suggestion
backlog. The full job stays as-is, advisory, for that backlog.
Also brought docs/scheduler/ci.md's CI-checks table back in sync with
reality (it was missing the doc-pointer lint job already; now lists
both vale jobs and their required-check status).
mara disagreed with exempting this section from #4041's ruling.
'Backend' here meant 'which LLM API provider', and the doc already
uses 'provider' for the same concept a few lines down ('provider-
specific model string') - so reword to that instead of carving out
an exception. Drops the inline vale suppression comment entirely.
Per #4128 (mara: allow-everywhere false positives go in a central
list, otherwise fix in source). Testing surfaced better fixes than
the plan posted on the issue:
- 5x Microsoft.Contractions 'that is' idiom false positives: adding
the missing comma ("that is, ...") both reads better and satisfies
the rule's own negative-lookahead, so no suppression is needed at
all. Fixed in docs/integrations/forge.md, docs/tools/forge.md,
docs/tools/hivectl.md, docs/web-ui/dashboard.md, and
swarmctl-cli.md's generated source (swarmctl/src/main.rs, doc
comment regenerated via markdown-docs).
- persistence.md's 'is not' matching inside 'is nothing': reworded to
'there'\''s nothing' rather than add any exception -- dodges the trap
and is a genuine contraction besides.
- ca.md's 'it is' matching inside the already-correct 'it isn'\''t':
tried a central .vale.ini TokenIgnores entry first per the
allow-everywhere framing, but testing against the real file (not
just a synthetic snippet) found it silently fails to suppress
whenever markdown emphasis syntax appears earlier in the same
file -- an offset-drift bug in how Vale applies TokenIgnores, not
a config mistake. Reworded to "it'\''s not" instead, same fix
shape as persistence.md.
- config.md's 3 genuine Microsoft.Avoid 'backend' exceptions (already
flagged and accepted on #4139 -- an actually-pluggable LLM API
provider, matching the nix option's own name, not one internal
system to name): scoped inline vale suppression around just that
section, since this one really is context-specific rather than a
rule bug.
Verified: fresh 'vale docs/ --minAlertLevel=error' is 0 errors AND
0 warnings (was 10 errors). nix fmt 0 changed beyond the edits
themselves. pre-push lints (tracker-tag/comment-block/doc-pointer)
clean. cargo clippy -p swarmctl -- -D warnings clean. Diffed the
regenerated swarmctl-cli.md against the old copy to confirm only
the intended line moved.
Per mara's ruling on hyperhive#4041 (Microsoft.We): keep the rule enabled,
same treatment as the 'backend' rewrites. Traced all 21 genuine hits (4
gateway.md 'Let's Encrypt' hits are a substring-match false positive, left
alone) to their actual referent: some name a specific component already
established nearby in the same doc (forge_notify, hive-github-notify's
poller, hive-forge, hive-agent/the harness, colors.css, the dashboard),
others were pure filler that adds nothing once dropped.
The section this branch adds was the only source of `Microsoft.Contractions`
errors in the file — 9 of them, and vale over the pre-change bytes still
reports exactly those 9, so the count is attributable rather than inferred
from a repo-wide total that was already red.
One of the nine is not a contraction: vale reads `it is` in "Narrowing it is
tracked in ..." as a subject-verb pair, but it is `it` (the object of
narrowing) plus `is tracked`. Rephrased so the sentence leads with the issue
instead, which drops the false hit and the passive-voice warning together.
A hive reads its agents' credentials with its own certificate, and nothing
said which paths that certificate may read, so the read half of a delivery
answered 403.
The grant is wide on purpose. An agent's path does not name the hive
hosting it -- agents move -- so a per-hive grant has to be an enumeration
the controller re-emits whenever the roster changes, and an enumeration
that can drift or land out of order advertises a boundary it does not
hold. A wide grant that says what it is beats a narrow one that only looks
narrow. mara's call, on the PR: rather a too-lax scope than one that
pretends to be strict.
What that buys, beyond honesty: the document is identical for every hive
and depends on nothing, so it is written once at startup beside the rest of
a hive's provisioning instead of on every declaration. No derived state, no
re-emission, and the ordering hazard that came with one stops existing.
What still holds is read-only. A hive cannot write an agent's credential,
so it cannot hand itself an agent's identity, and the grant reaches nothing
in the store outside the agent-credential prefix.
The fact is documented where someone meets the boundary rather than only in
this message, and the two ways to narrow it later -- scope per hive, or
give agents their own store identity -- are tracked.
The controller writes an agent's credential; the hive fetches it back with
its own certificate. Nothing said which paths that certificate may read, so
the read half of a delivery answers 403 with no way to tell why.
The grant is derived from the declaration, so it is re-rendered at the one
place the declaration changes -- WantedWriter::set -- rather than at its
caller, which would work today and break on the second caller.
Emitted before the KV write: a grant that lands late is a 403 on an agent's
first fetch, while one that shrinks early only affects an agent already
being torn down. A failed write then leaves a superset the next declaration
re-renders.
Destroyed agents are filtered out. The declared set is a hive's whole
history -- a destroyed entry stays so that redeclaring it Up is refused as
the terminal transition it is -- so granting every declared agent would
leave a torn-down agent's credentials readable forever.
The sink is a trait because a missed emission is that same untraceable 403:
the double pins which agents were published, and the no-sink and refusing
arms pin the two deployments that are not a happy path. Not covered: the
call site inside set(), which needs a live queue.
The cert role moves to its own module on the way past. It is the
controller's identity at the store, not something the matrix route owns,
and the policy writer needs the same login.
The renderer beside this produces a hive's read policy; nothing could put
one in the store. `read`/`write` are KV v2 verbs and a policy is not a
secret, so this is a second endpoint rather than another path.
A whole-document write, because the store has no merge verb: the caller
renders from the current agent set, so a stanza missing from the render
is a grant that is meant to be gone.
The controller writes an agent's credential; the hive reads it back with its
own token. Nothing says which paths that token may read, so the read half of
a delivery has no identity at all and answers 403.
This is the pure half of the fix: the policy text is a function of a hive name
and its agent set, so the shape can be asserted with no store to talk to.
One stanza per hosted agent rather than a prefix grant. An agent's credential
path does not name the hive hosting it -- deliberately, since agents migrate --
so "this hive's agents" has no prefix expression and must be enumerated. The
grant is read-only: the controller mints these and never reads one back.
The paths come from MOUNT and AGENT_PREFIX rather than literals, so the policy
cannot drift from the module that builds the paths it grants.
checked_segment runs before any name reaches HCL. That is policy injection
rather than path traversal -- a name can close a stanza and open a wider one --
so the test carries a real injection string, paired with the reachable-charset
control that stops it passing by refusing everything.
An empty agent set renders an empty policy, which grants nothing.
Per mara's ruling on hyperhive#4041 (Microsoft.Avoid): 'backend' never says
which system it means. Fixed all 21 fixable hits to name the actual daemon
(mostly hive-c0re; one hive-agent instance and one hive-matrix-daemon one,
both confirmed against surrounding context, not guessed). Also tightened
Weston's own 'VNC backend'/'RDP backend' plugin references to their literal
upstream names (vnc-backend/rdp-backend) rather than dropping the accurate
term entirely, and reworded one unrelated Microsoft.Avoid hit ('and so on').
Left alone (flagged on the issue, not guessed): docs/turn-loop/config.md's
3 remaining hits match the actual nix option name (backendEnvironmentFile)
and describe a genuinely pluggable concept (which LLM API provider), not
one specific internal system.
cc8fb0ee44 (swarm-bao controller-write-credentials
work) added a fresh 'did not' warning callout in the bao-provision section,
right after #4129 zeroed out the genuine error-level Microsoft.Contractions
backlog, and its passive 'has been exercised' phrasing put getting-started/
back above its documented Passive-pass accepted-residual baseline (10) too.
Contracted the 'did not' and reworded the passive opener to an active one
('This has only run once...') in the same sentence, since both hits were on
the same line.
16 of the 23 error-level hits from a full-repo vale run were real:
'is/are/does/do NOT' and bold 'not' expanded per Microsoft.Contractions,
one hyphen-as-negative-number fixed to an en dash per Microsoft.Negative
(docs/web-ui/dashboard.md's -45 deg column-header tilt).
Where the caps/bold was there for emphasis (contrasting X does this,
Y does not), kept the emphasis by bolding the contraction rather than
dropping it silently -- 'is not' -> 'isn't', 'does NOT' -> '**doesn't**',
etc.
docs/tools/swarmctl-cli.md's one genuine hit is generated -- fixed the
source doc comment in swarmctl/src/main.rs and regenerated via
'cargo run --bin swarmctl -- markdown-docs', diffed against the old
copy first to confirm only the intended line changed.
Left 7 hits unfixed, false positives from the rule matching a
substring rather than the actual pattern it names:
- 4x 'that's instead of that is': every one is the explanatory 'that
is' idiom (~= i.e.), not a contractible subject+verb -- 'that's'
would misparse as a demonstrative-pronoun contraction and change the
sentence's meaning. docs/integrations/forge.md:282,
docs/tools/forge.md:260, docs/tools/hivectl.md:243,
docs/web-ui/dashboard.md:457 (and the swarmctl-cli.md generated
twin of the same category, source left alone for the same reason).
- docs/agent-lifecycle/persistence.md:482 -- 'is not' matched inside
'is nothing' (the same has-not/has-nothing substring trap noted
before in this doc's own git history).
- docs/swarm/ca.md:182 -- 'it is' matched inside the already-correct
'it isn't'.
Per #4128. Remaining errors on main after this: Microsoft.Avoid's 26
hits, already tracked + awaiting a house-style ruling on #4041.
- resolve_credential (+ token_credential / password_fields helpers)
extracted out of put_matrix_account, which was tripping
clippy::too_many_lines (101/100). Both helpers surface plain &str
errors rather than ProblemDetails to dodge clippy::result_large_err on
a private fn (put_matrix_account itself is exempt only via clippy's
avoid-breaking-exported-api default, which does not cover these);
resolve_credential boxes its own ProblemDetails Err for the same
reason, unboxed at its one call site.
- Added unit tests for the extracted logic: main-reserved, token-mode
missing-token / pass-through, password-mode missing-fields / trailing-
slash trim, and two async resolve_credential end-to-end checks
(token mode, unknown mode) that need no network access.
- Dropped PutMatrixAccountRequest's Debug derive to match
hive-c0re::dashboard::matrix_accounts::MatrixLoginForm's existing
precedent of not deriving Debug on a struct carrying a password field.
Extends PutMatrixAccountRequest with a mode field (token, the existing
behavior and default; or password). Password mode has swarm-controller
itself perform m.login.password against the caller-given homeserver
(mirrors hive-c0re's own /api/matrix-account-login for the hive-local
case) and stores the resulting token instead of a caller-supplied one
-- the password is used once, over this PUT, and never stored. Also
adds the 'main is reserved' guard hive-c0re's login form already has,
which swarm-controller had no equivalent of before this.
swarm-ui's link-matrix-account form gets a credential-mode toggle
wired to the same contract: token mode is unchanged, password mode
swaps the token field for user-id + password fields and makes
homeserver required (no hive-side fallback to resolve it against, per
PutMatrixAccountRequest::homeserver's own doc).
Per #4122.
Closes#4124.
The controller's policy granted only the bootstrap paths -- hive cert-auth
roles and hive ACLs. #4113 then made it a secret WRITER, and nothing related
the grants to the paths the code writes, so every matrix token provision
answered 403. The two halves landed on different issues and neither looked
wrong on its own.
`secret/data/` is KV v2's ACL prefix and is absent from the path the code
passes, so matching `swarm-secret-client`'s spelling literally would have
granted nothing. Write-only: the controller mints these and never reads one
back, and a read capability would let it recover every agent's credentials
rather than only replace them.
The gate is the point. Two module-eval arms -- the grant exists and is not
a broader wildcard, and its capability list is pinned whole, because an
ADDED capability is what a presence check misses -- plus a test in path.rs
pinning MOUNT/AGENT_PREFIX and naming the nix file, since renaming either
constant is a silent 403 rather than a compile error.
setup.md carried two warnings this makes false: that nothing in the tree had
ever authenticated to the store, and that no deployment shape mints a leaf
whose CN reads swarm-controller. glue-bao-tls.nix has minted one since #3726
item 1.
The link-matrix-account form's placeholder read 'e.g. primary', which
sounds like it names the hive-provided default account. That account is
actually named main (reserved in hyperhive.matrixAccounts's schema) and
this route can neither create nor touch it. Swapped to an example with
no relationship to the real default, per #4122.
nix fmt table-column-width drift from the earlier comma fix (f3fe0604/82236952)
- shortening that cell by one char left the row under-padded relative to the
table's other rows. Pure whitespace, no content change.
