On a hive fronting no swarm name, swarm-services-cert's token-less
GET pki/ca/pem presents baoHiveIdentity{Cert,Key}File again. The fetch
never logs in and grants nothing; the client certificate only completes
the store listener's mTLS. A hive host without the store therefore needs
no copy of the services-issuer pair.
The issue path, which logs in to pki/issue/swarm-services, still presents
only deploy.bao.servicesIssuerClient{Cert,Key}File.
module-eval-hive-tls: the root-fetch cases pin the hive identity and the
absence of the services-issuer pair (a fixture holding both, and one
holding only the hive identity); the issue-path case pins the pair on a
hive host and on a host without a hive; a gateway without the pair
evaluates and its issue path names both options.
Root-fetch identity: option (a), mara on PR #5002 (comment 92591).
Refs #4921
Drop the eval assertion b78190e1 added. It refused every self-signed
gateway host that does not run the store, which is the default hive
shape, and contradicts docs/swarm/secrets.md's supported no-store
deployment. The approved #4921 plan has such a host evaluate and fail
at runtime; `requireServicesIssuer` already exits 1 naming both
options, on the root-fetch branch and before issuing.
The module-eval case that pinned `hiveWithController { }` as refused
now pins it, and a hive-less gateway with no pair, as evaluating with
the unit's refusal in the script. The fixture pairs the assertion
forced into forge-placement, nats-tls and swarm-otel-identity are
reverted.
Refs #4921
swarm-services-cert issues the swarm-services leaf a self-signed gateway
serves, and on a host fronting no swarm name fetches the store's pki root.
It is gated on the gateway, not on a hive, yet its leaf pair was named
under deploy.hive-controller.tls and defaulted from the hive reader's cert
directory, and the root fetch presented the hive's own identity. A gateway
host with no hive had no such identity, so that branch was red by
construction.
- The pair moves to deploy.bao.servicesIssuerClient{Cert,Key}File, beside
the store's other host-unit readers, and glue-bao-tls.nix defaults it on
the store's host from the leaf it already mints. The old
deploy.hive-controller.tls.baoClient{Cert,Key}File paths are
mkRenamedOptionModule aliases. glue-services-issuer-bao-identity.nix,
which derived the default from deploy.bao.clientCertFile, is gone.
- The no-swarm-name root fetch presents the services-issuer pair. The hive
identity is now only hive-tls-resign's.
- A self-signed gateway with no pair fails evaluation, naming both options.
A pair named but unreadable still fails the unit, naming the paths.
Refs #4921
The CA-file assertion compared the option's raw string, so
`/var/lib//swarm-ca/root.pem` or `/var/lib/hive-tls/./ca.pem` got past it.
Both sides are normalised lexically now: empty and `.` components drop out,
`..` removes the one before it, a trailing `/` goes. A symlink is not
resolved at eval time, so a path through one still passes; docs/swarm/ca.md
says so.
A switch that changed `hive-tls-ca` stopped and started it, and the stop
propagated through `Requires=` to the store's container (sealed again under
the shamir seal), every container binding the bundle, and every
`<name>-ca-bundle` unit with its consumers. `reloadIfChanged` with the same
script as `ExecReload` rewrites the bundle in place and restarts none of them.
`Requires=` stays, so a failed assembly still keeps them from starting.
With `swarm-services-root.pem` missing, `hive-tls-ca` writes an empty bundle
and every `<name>-ca-bundle` unit fails. Its log line now names the root's
path. docs/swarm/ca.md describes both.
- swarm-services-cert (and its daily renewal) treats a services leaf that
`openssl verify` rejects against swarm-services-root.pem as invalid and
asks the store for a new one, as hive-tls-resign already does for the
gateway leaf. The trust bundle is that root alone, so a leaf under any
other issuer verifies for no consumer. With the store unreachable the
unit fails, and where the gateway waits for the leaf, the import fails
with it.
- Every host also deletes the old hive CA's ca.srl and .swarm-ca-adopted
in deploy.hive-controller.tls.stateDir, with the same tmpfiles `r`
rules as ca.pem/ca-previous.pem/ca-key.pem. Nothing reads either.
- docs/swarm/ca.md: the services leaf's re-issue conditions, the two
extra deleted files, and that the deletion is irreversible: rolling back
to a generation with swarm-ca.service mints a new, unrelated root that
nothing issues under.
module-eval hive-tls: a case for the services-leaf check in the boot
issuance and the renewal (red with the line removed), and the deletion
case covers the two extra files.
Refs #4588
The gateway leaf comes from the store's pki mount, so a hive CA's
certificates anchor nothing and its key is a secret outside the store.
- hive-tls-ca writes trust-bundle.pem from swarm-services-root.pem alone;
the ca.pem and ca-previous.pem slots are gone.
- Every host deletes ca.pem, ca-previous.pem and ca-key.pem in
deploy.hive-controller.tls.stateDir with tmpfiles `r` rules that follow
the option. The removal shim, now host-ca-removed.nix, holds these
beside the /var/lib/swarm-ca rule and the swarm.ca.* removals.
- deploy.bao.clientCaFile, deploy.bao.serverCaFile and
deploy.swarm-controller.hiveClientCaFile naming /var/lib/swarm-ca or one
of the deleted hive CA files fail evaluation, naming the store's mTLS CA
(/var/lib/swarm-bao-pki/ca.pem on the store's host) as the replacement.
module-eval hive-tls covers the single-anchor bundle, the deletion (incl.
a relocated stateDir) and the assertion on all three options.
Refs #4588
The secret store's pki root issues every hive's gateway leaf and the
swarm's service names, so the root swarm-ca.service generated on the
host anchors nothing and its key is a secret outside the store.
- swarm-ca.nix is gone: swarm-ca.service, root.pem and root-key.pem,
and the swarm.ca.{autoConfigure,stateDir,validityDays} options. A
definition of any of them fails evaluation with a removal message
(swarm-ca-removed.nix), and a tmpfiles rule deletes /var/lib/swarm-ca
on every host. singleHostSwarm no longer sets a swarm.* option.
- hive-tls-ca drops the $root slot: trust-bundle.pem holds the store's
pki root and a hive CA's public ca.pem/ca-previous.pem only.
- hivectl peer-config no longer prints the scp line for the swarm root;
the store's pki root is the one step.
- The gateway's cert import no longer stands a hive leaf or a throwaway
in for a missing swarm-services leaf where it Requires= the issuing
unit: it fails, naming swarm-services-cert.service. The stand-in stays
where the gateway does not wait (no swarm name, or the store's UI
only, whose passthrough must start with the store sealed).
Comments and docs that named swarm-ca.nix, swarm.ca.* or the host root
now name the store's pki root.
Refs #4588
The shared `hive` fixture turned `deploy.hive-controller.enable` on for
every eval test. It is now off by default; `hiveWithController` turns it
on, and the fixtures whose claims read the hive tier (hive-c0re, the
queue-agent credential, the hive leaf, `deploy.hive-controller.*`, or an
absence that is only meaningful with a hive present) use it.
glue-nats-auth-bao-identity.nix still read the renamed
`services.hyperhive.enable`, which printed the "Obsolete option" trace
six times per CI run. The term is dropped: the pairing is gated on the
queue and the minted PKI, as its sibling glue-nats-bao-identity.nix is.
The hive-domain vhost renders only where a hive runs. On a host without
one the only certificate is the swarm-services leaf, which does not name
the hive domain, so a homeserver host without a hive serves matrix on the
homeserver's own name only; apex matrix discovery is #4878.
Comments and test names describe the code's current behaviour.
Refs #4588
hive-tls-ca no longer deletes a ca-key.pem an earlier generation left:
nothing reads it, and leaving it means a rollback finds the old key instead
of minting a new anchor. Deleting it moves to PR 3, beside dropping ca.pem
from the trust bundle.
hive-tls-resign now also re-issues when the leaf's public key differs from
the key file's. The leaf and key are moved into place by two renames, and an
interrupted pair left a certificate beside the wrong key that the validity
check accepted.
Refs #4588
Every hive gets its own store identity (CN
<deploy.bao.hiveLeafCommonNamePrefix>-<hive>) and its own role on the
existing pki mount (<deploy.bao.hiveLeafPkiRolePrefix>-<hive>), whose
allowed_domains is the hive's domain and *.<domain>. The policy grants
update on that one issue path. swarm-bao-hive-leaf-policy writes role,
policy and cert-auth role for every hive in swarm.hives; the store's host
mints its own hive's identity in glue-bao-tls.nix and a new glue file
points deploy.hive-controller.tls.baoHiveIdentity{Cert,Key}File at it.
Hives on other hosts place it by hand. swarm.nix reserves the composed CN.
hive-tls-ca no longer generates a hive CA or signs a leaf: caGenScript,
signLeafScript/signHiveLeaf and the adoption block are gone, ca-key.pem
is deleted where it exists, and the unit only assembles trust-bundle.pem.
The retired ca.pem/ca-previous.pem stay in the bundle so leaves they
signed keep verifying.
hive-tls-resign becomes the bao issuance: at boot, on deploy and from a
daily timer it re-issues when the leaf is missing, past half the role's
lifetime, missing a name, or not chained to the store's pki root. Nothing
orders the gateway after it, so a valid leaf on disk never waits on the
store; a failed renewal leaves the leaf in place. A missing identity
fails the unit naming both options, as does the root fetch on a hive
fronting no swarm name, which now presents this identity instead of the
reader cert.
caValidityDays and leafValidityDays are removed (both spellings), since
nothing reads them.
Refs #4588
A hive fronting no swarm name fetches the store's pki root presenting its
reader certificate. When deploy.bao.clientCertFile or clientKeyFile is unset,
or either named file is not readable, the unit now exits 1 with a message
naming both options and their paths, instead of exiting 0 with the root
missing from the trust bundle.
The module-eval cases that asserted the quiet skip assert the failure
instead, and a new case covers the named-but-absent files.
The `want_svc=0` root fetch presents `deploy.bao.clientCertFile`/
`clientKeyFile` to a listener that demands a client certificate. On a
hive with neither set, the handshake is refused every time, and
`swarm-services-cert` went from green to failed-and-retrying for 24h.
The branch is now chosen at eval: with no reader identity, the unit
logs that the store root was not staged and names the option, then
exits 0, and no fetch is rendered. A hive with an identity fetches as
before. `hivectl peer-config` already handles the absent file.
Refs #4588 (review c88774, point 2).
A self-signed hive fronting no swarm service name never filled
`$svcroot` (/var/lib/hive-tls/swarm-services-root.pem): the only writer
was `swarm-services-cert`'s issuance, which exits early on
`want_svc=0`. So those hives' trust-bundle.pem lacked the store's `pki`
root, even though every other hive's swarm-service names chain to it.
On `want_svc=0`, `swarm-services-cert` now reads the root from the
store's unauthenticated `GET /v1/pki/ca/pem`, over the hive's existing
BAO_ADDR, `deploy.bao.serverCaFile` and reader identity
(`deploy.bao.clientCertFile`/`clientKeyFile`) at the TLS layer only; no
login. It writes `$svcroot` and restarts `hive-tls-ca` when the root
changed, as the issuing path already does. The daily
`swarm-services-cert-renew` timer runs the same script, so the copy
follows a root rotation.
The gateway no longer `Requires=` `swarm-services-cert` on such a host:
nginx reads nothing that unit writes there, and the fetch retries
against a sealed store.
`hivectl peer-config` prints the root as step 2 of the self-signed
instructions: the swarm's service names are issued under it, not the
swarm root.
No new option. Refs #4588.
The swarm collector's journald receiver read only the units listed in
`services.hyperhive.swarm.otel.journaldUnits`. A unit nobody listed
never reached the store, and a misspelt entry shipped nothing without
an error. The list existed to keep an operator's desktop session out of
a store every swarm operator can read, but the receiver can only match
positively, so the only way to express "not user sessions" was to name
every service instead.
The receiver now reads the whole host journal, and a new
`filter/exclude-user-sessions` processor in the `logs/<swarm>` pipeline
drops records whose `_SYSTEMD_SLICE` is `user-<uid>.slice` (session
scopes and `user@<uid>.service`). The per-hive `logs/<hive>` pipelines
carry agent-container journals only and get no filter.
`journaldUnits` is removed with `mkRemovedOptionModule`, together with
its non-empty assertion and the entry each host module added. The four
module-eval membership checks go with it, replaced by one structural
case in swarm-otel-core.
Closes#3646
Every unit that writes a bao policy or cert-auth role ran only while the
operator-placed bootstrap token existed, and skipped silently otherwise.
The token lives 24h, so on any real swarm a PR adding or changing a grant
deployed with its unit skipped, and each one needed a manual token refresh
(plus a root `bao policy write` when it added a path).
A `bao-granter` principal now writes them. Its leaf is minted by
swarm-bao-pki on the store host (0600 root, never copied off it), and its
policy covers `swarm-*` policies, `swarm-*` cert-auth roles and
`pki/roles/swarm-*` by glob, plus the mount and services-root paths the
controller's unit already used. All ten granting units
(controller, secret-publisher, matrix-ctl, matrix-token, queue-agent,
grafana-oidc, otel-oidc, forwarder-oidc, services-issuer, nats-tls) log in
with it instead of reading the token. They keep the 2880 x 30s retry, now
require swarm-bao-pki, and when the store refuses the granter they fail
and print the one-time step instead of skipping.
swarm-bao-granter-role is the one unit left on the token. It enables the
auth mounts (moved out of the controller's unit) and writes the granter's
own policy and role. The bootstrap policy is renamed `bao-bootstrap` and
shrinks to those five stanzas; it is shipped at
/etc/hyperhive/bao-bootstrap-policy.hcl. The old name `swarm-bootstrap`
matched the granter's own `swarm-*` glob.
The granter's CN joins certAuthCns, so no hive can be named into its role.
An assertion keeps both pki role names under `swarm-`. With no client CA
the granting units no longer render, and a warning says so.
module-eval pins the granter's policy stanza by stanza, what it cannot
reach, that every call a granting unit makes is granted, and that only
swarm-bao-granter-role reads the token.
Refs #4704
The store's `swarm-services` role issues the services leaf for 720h, and
`swarm-services-cert` only ever ran at boot or rebuild: it is a
`RemainAfterExit` oneshot wanted by `multi-user.target` and no timer
targeted it. A hive not rebuilt within 30 days served an expired leaf.
`swarm-services-cert-renew` runs the same script from a daily timer. It
is a unit of its own because a timer starting the `RemainAfterExit` unit
is a no-op, and restarting that unit instead would propagate through
`hive-gateway-self-signed-cert`'s `Requires=` to nginx, so a sealed store
would take the gateway down over a still-valid leaf. Nothing requires or
orders against the new unit; it has no `Restart=`, so a failure stays in
`systemctl --failed` until the next tick, and the script only moves files
into place after the store has answered.
The re-issue threshold was `checkend 2592000`, the whole 30-day
lifetime, so every run re-issued. It is now half the role's lifetime,
read from a new internal option `deploy.bao.servicesPkiLeafTtlHours`
that the role's `ttl`/`max_ttl` also read. Boot and timer share the
script and so the threshold. The services-root re-check reads the
same option, at the store's own replacement threshold (hours × 3600),
so the hive asks for a new leaf when the store replaces its root. A
`flock` keeps the two runs from interleaving one issuance's key with another's leaf.
`checks.module-eval-hive-tls` pins the timer, that the unit it starts
re-runs the issuance without `RemainAfterExit`, that nothing depends on
it, and that both the leaf and root thresholds move with the option.
Closes#4587
