document the controller tcp listener and its two leaves
This commit is contained in:
parent
187ed82de0
commit
cd786ef401
3 changed files with 83 additions and 1 deletions
|
|
@ -206,6 +206,36 @@ and the next day's run tries again: about fifteen tries before the leaf
|
|||
lapses. `systemctl
|
||||
list-timers swarm-services-cert-renew` shows the next run.
|
||||
|
||||
## The controller's TCP listener has two leaves of its own
|
||||
|
||||
`swarm.controller.tcp.enable` adds a TLS listener to the swarm controller (default off). The
|
||||
controller checks nobody per request, so the listener finishes a handshake for one client only:
|
||||
the gateway. Both ends hold a leaf from the store's `pki` mount, each through a role of its own on
|
||||
the same mount and the same root:
|
||||
|
||||
- **Server leaf.** The `swarm-controller-tls` role issues for `swarm.controller.tcp.domain` alone,
|
||||
`serverAuth` only. `swarm-controller-tls.service` on the controller's host requests it under the
|
||||
controller's own store leaf, whose policy gains that one issue path.
|
||||
- **Gateway client leaf.** The `swarm-controller-gateway-client` role issues for
|
||||
`gateway-client.<swarm.domain>` alone, `clientAuth` only. `swarm-controller-gateway-client.service`
|
||||
on the host that runs the controller requests it under the services-issuer identity, whose policy
|
||||
gains that one issue path.
|
||||
|
||||
The controller accepts a client certificate that chains to the services root, carries the
|
||||
`clientAuth` usage and names `gateway-client.<swarm.domain>`. The swarm-services leaf chains to
|
||||
the same root but is `serverAuth` only and names other hosts, so the listener refuses it at the
|
||||
handshake. Nginx verifies the controller's server leaf against the trust bundle, by `domain`.
|
||||
|
||||
Every host that holds the services-issuer identity can obtain the gateway client leaf, which is
|
||||
every gateway host. The controller trusts exactly that identity: a gateway of this swarm.
|
||||
|
||||
Set `swarm.controller.tcp.enable` and `swarm.controller.tcp.domain` identically on every host. The
|
||||
store's host writes both roles from them. A store host that leaves the option off has no role, and
|
||||
the issuing unit fails with the store's refusal naming it.
|
||||
|
||||
The listener reads the services root once at start. Replacing the root (about 30 years out) needs a
|
||||
controller restart as well as new leaves.
|
||||
|
||||
## Constraints on the material
|
||||
|
||||
No CA private key exists on any hive. The `pki` root's key stays inside the
|
||||
|
|
|
|||
|
|
@ -498,6 +498,33 @@ The controller registers the hooks within five minutes of having both the forge
|
|||
token and the secret. It keeps trying, at most five minutes apart, for as long
|
||||
as the forge or the secret store is down. A restart isn't needed to recover.
|
||||
|
||||
The URL hooks register against is `deploy.swarm-controller.publicUrl`, default
|
||||
`https://<swarm.domain>`. It names the vhost that fronts the controller, so it stays right
|
||||
whether the gateway reaches the controller by socket or by its TCP listener. Set it when another
|
||||
URL fronts the controller.
|
||||
|
||||
## Reaching the controller over TCP (optional)
|
||||
|
||||
By default the gateway reaches the controller through a unix socket on the same host. Setting
|
||||
`swarm.controller.tcp.enable` and `swarm.controller.tcp.domain` on **every host of the swarm**
|
||||
adds a TLS listener beside the socket and points the gateway's `/api/` and `/webhook/forge/`
|
||||
locations at it by that name. Nothing else changes: the socket stays, and the operator gate on
|
||||
`/api/` stays.
|
||||
|
||||
What you provide:
|
||||
|
||||
- `swarm.controller.tcp.domain`, resolvable from the gateway's host. Nginx resolves it when it
|
||||
loads its config.
|
||||
- `deploy.swarm-controller.tcp.openFirewall = true` on the controller's host only when a gateway
|
||||
on another host dials it. The port (`swarm.controller.tcp.port`, default 9443) stays closed
|
||||
until you do, and `deploy.swarm-controller.tcp.listenAddress` picks the bound address.
|
||||
|
||||
What you don't: both certificates come from the secret store with no manual step
|
||||
([`ca.md`](ca.md#the-controllers-tcp-listener-has-two-leaves-of-its-own)). The listener accepts
|
||||
the gateway's client certificate and nothing else, so reaching the port gets a stranger a refused
|
||||
handshake. The swarm UI's vhost is still declared on the host that runs the controller, so the
|
||||
gateway that dials the listener runs there too.
|
||||
|
||||
## Agent identity minting
|
||||
|
||||
A store identity that fails to mint logs `swarm jobq: mint_agent_identity
|
||||
|
|
|
|||
|
|
@ -152,6 +152,8 @@ pieces fit, and the reasoning behind them, is in [`credentials.md`](../../2-hand
|
|||
| queue auth-callout nkeys (user seed + account seed) | `swarm-nats-auth ensure-callout-keys`, run by `swarm-bao-nats-callout-xkey` on the queue's host when the store holds no seed; the verb stores a seed it finds at `/var/lib/swarm-nats-callout/{callout-user,issuer}.seed` as is, then removes the file | the swarm secret store: the seeds at `swarm/services/swarm-nats/callout/{user,issuer}-seed`, read by the `swarm-nats-auth` role alone; the public halves at `swarm/public/swarm-nats/callout-{user,issuer}` | nothing to place beyond the responder's store leaf (`deploy.nats.authBaoClientCertFile`) |
|
||||
| queue auth-callout xkey (curve seed) | `swarm-nats-auth ensure-callout-keys`, run by `swarm-bao-nats-callout-xkey` on the queue's host when the store holds no seed | the swarm secret store: the seed at `swarm/services/swarm-nats/callout/xkey-seed`, read by the `swarm-nats-auth` role alone; the public half at `swarm/public/swarm-nats/callout-xkey` | nothing to place beyond the responder's store leaf (`deploy.nats.authBaoClientCertFile`) |
|
||||
| the queue's TLS key, copied for the gateway's `/metrics` vhost | `swarm-nats-gateway-cert.service` on the queue's host, where the gateway uses self-signed TLS: a copy of the queue's bao-PKI TLS leaf (`/var/lib/swarm-nats-tls/`) once `swarm-bao-nats-tls` has issued it, else a throwaway 30-day self-signed pair for the queue's name until then | `/var/lib/swarm-nats-gateway-tls/{cert,key}.pem`; the key is `0640` group `nginx`, read by nginx for the queue's `/metrics` vhost. The queue's own key stays `0600` root | the copy is redone on each leaf renewal (`swarm-bao-nats-tls` starts the unit, which queues a non-blocking nginx reload when nginx is running); a restart before any leaf exists keeps the pair already there. Until `swarm-bao-nats-tls` issues the leaf the vhost serves the throwaway cert, so the collector's scrape fails TLS. Nothing for an operator to place |
|
||||
| the controller's TCP listener certificate and key | `swarm-controller-tls.service` on the controller's host, only where you set `swarm.controller.tcp.enable`. It logs in under the controller's own store leaf and issues from the store's `swarm-controller-tls` role: one name, `swarm.controller.tcp.domain`, `serverAuth` only. A daily timer renews it past half its life | `<deploy.swarm-controller.tcp.tlsDir>/{cert,key}.pem` (`/var/lib/swarm-controller-tls`), the key `0640` group `swarm-controller`; the daemon re-reads both when they change, so a rotation restarts nothing. A store host that leaves the option off has no role to issue from, and the unit fails with the store's refusal naming it | Nothing to place: the controller's own store leaf is the one it already holds |
|
||||
| the gateway's client certificate for the controller's TCP listener | `swarm-controller-gateway-client.service` on the host that runs the controller, only where you set `swarm.controller.tcp.enable`. It logs in under the services-issuer identity and issues from the store's `swarm-controller-gateway-client` role: one name, `gateway-client.<swarm.domain>`, `clientAuth` only, so the services leaf can't stand in for it. A daily timer renews it | `<deploy.hive-controller.tls.stateDir>/controller-client.pem` and `controller-client-key.pem`, the key `0640` group `nginx`; the gateway's controller locations name them in `proxy_ssl_certificate`. Until the store answers, a throwaway self-signed pair keeps nginx's config loadable. The listener refuses that pair, so those locations answer 502. Every holder of the services-issuer identity can obtain this leaf: the listener trusts "a gateway of this swarm" | Nothing to place: the services-issuer identity is the one every gateway host already holds |
|
||||
| the secret store's own contents | openbao, on first `bao operator init` — **an operator action, not a unit** | inside the `swarm-bao` container, at its own `/var/lib/openbao`, kept across rebuilds by `ephemeral = false`. ⚠️ **Not a host path**: `nixos-container destroy swarm-bao` takes the raft data with it, so back up the container's tree, not `/var/lib/`. Only the store's TLS material (`/var/lib/swarm-bao-tls`) and its PKCS11 token (`/var/lib/swarm-bao-token`) are host-level | n/a — there is one store; a hive elsewhere is a _client_ of it and holds none of this |
|
||||
| the secret store's unseal material | the HSM/TPM under `deploy.bao.seal = "pkcs11"`; openbao itself under `"shamir"` | in the token; or held by whoever ran `bao operator init`, which is what `"shamir"` means and why it's stated rather than inferred | n/a — only the host running the store seals anything |
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue