swarm: mint, publish and login-verify an agent's store identity at create
`swarm/agents/<agent>/bao-mtls` did not exist, and neither did any per-agent identity at the secret store: `policy::agent_object_name`, `render_agent` and `render_agent_with_queue` had been written and never called outside their own tests. An agent's only "per-agent" secret today is read under the HIVE's certificate, through a wide grant on `swarm/agents/*` — so "per-agent" was presentational. The swarm now mints the certificate, so no hive ever needs the capability to mint one. `swarm-controller` is the service that does it: it already logs in to the store, and its existing grant already covers exactly the three objects written here (`create/update` on `secret/data/swarm/agents/*`, `sys/policies/acl/hive-*` and `auth/cert/certs/hive-*`). No new bao grant, and nothing co-located — a cert-auth role pins its authority by value, per role, so the controller issues from its own CA on its own host and pins that CA in the role it writes. No existing role changes. The mint node does not report success on a write. After publishing it connects again, with the leaf it just issued and under the role it just wrote, and reads the path back — so the policy, the role, the common name and the leaf are exercised in production on every agent creation. A certificate this code mints that the role this code writes will not accept turns the job node red at creation time instead of surfacing later as an agent container that cannot start. `TriggerDeploy` gains an `after_any` edge on the mint, not `after_ok`: a hive cannot pass down a certificate the swarm has not published, but a host with no authority configured must still create agents exactly as it does today. The private key is generated in memory and never written to disk on the controller — `SecretStore::connect_with_identity` takes the PEM the minter is already holding, so nothing is written out purely to be logged in with. Refs #4137
This commit is contained in:
atlas
parent
992468dccc
commit
676c45bc93
10 changed files with 1262 additions and 71 deletions
|
|
@ -37,6 +37,33 @@ let
|
|||
# file would be handed to a daemon that cannot use it.
|
||||
haveHiveClientCa = haveBaoIdentity && deployCfg.swarm-controller.hiveClientCaFile != null;
|
||||
|
||||
# The authority this daemon issues AGENT client leaves from — a different
|
||||
# question from `hiveClientCaFile` above, which is the authority it *trusts*
|
||||
# hives by. This one it signs with, so it needs the private key too.
|
||||
#
|
||||
# ⚠️ Deliberately NOT the store's own PKI (`glue-bao-tls.nix`'s
|
||||
# `/var/lib/swarm-bao-pki`). A cert-auth role pins its authority by value,
|
||||
# per role, so a role this daemon writes carries whatever authority this
|
||||
# daemon hands it — which is what lets the controller mint from its own CA
|
||||
# on its own host without anything being co-located and without any
|
||||
# existing role changing. `swarm-controller/src/agent_identity.rs`'s module
|
||||
# doc is the long form.
|
||||
agentCaDir = "/var/lib/swarm-controller-agent-ca";
|
||||
|
||||
# No authority named means mint one here. The alternative — leaving agent
|
||||
# identities off until an operator places a CA by hand — is the state where
|
||||
# the whole path is configured and silently does nothing, which is the
|
||||
# failure mode `glue-bao-tls.nix` avoids the same way.
|
||||
selfSignAgentCa = deployCfg.swarm-controller.agentCaFile == null;
|
||||
agentCaCert =
|
||||
if selfSignAgentCa then "${agentCaDir}/ca.pem" else deployCfg.swarm-controller.agentCaFile;
|
||||
agentCaKey =
|
||||
if selfSignAgentCa then "${agentCaDir}/ca-key.pem" else deployCfg.swarm-controller.agentCaKeyFile;
|
||||
|
||||
# Minting an agent's identity means publishing it to the store, so the
|
||||
# authority alone is not enough — same rule `haveHiveClientCa` states.
|
||||
haveAgentCa = haveBaoIdentity && agentCaKey != null;
|
||||
|
||||
# `swarm_secret_client` reads these spellings explicitly rather than
|
||||
# vaultrs's `VAULT_*` defaults — falling through to those builds a client
|
||||
# with no identity and fails at the TLS handshake, naming neither. `%d` and
|
||||
|
|
@ -59,6 +86,13 @@ let
|
|||
# store should trust hives by, which the controller reads as a value
|
||||
# to put in each hive's cert-auth role.
|
||||
SWARM_CONTROLLER_HIVE_CLIENT_CA_FILE = "%d/hive-client-ca.pem";
|
||||
}
|
||||
// lib.optionalAttrs haveAgentCa {
|
||||
# The authority agent leaves are ISSUED FROM, so unlike every other
|
||||
# `*_CA_FILE` here it comes with a key. `%d` for both: the key is
|
||||
# `0600` and root-owned, and this daemon runs unprivileged.
|
||||
SWARM_CONTROLLER_AGENT_CA_FILE = "%d/agent-ca.pem";
|
||||
SWARM_CONTROLLER_AGENT_CA_KEY_FILE = "%d/agent-ca-key.pem";
|
||||
};
|
||||
|
||||
# What `swarmctl` needs in order to act on authelia from the host.
|
||||
|
|
@ -604,6 +638,47 @@ in
|
|||
'';
|
||||
};
|
||||
|
||||
agentCaFile = lib.mkOption {
|
||||
type = lib.types.nullOr lib.types.str;
|
||||
default = null;
|
||||
example = "/var/lib/swarm-agent-ca/ca.pem";
|
||||
description = ''
|
||||
Authority this daemon **issues** agent client certificates from, so
|
||||
that an agent container can authenticate to the swarm secret store
|
||||
under its own name. Read together with
|
||||
{option}`services.hyperhive.deploy.swarm-controller.agentCaKeyFile`,
|
||||
which is the private key it signs with.
|
||||
|
||||
The mirror image of
|
||||
{option}`services.hyperhive.deploy.swarm-controller.hiveClientCaFile`:
|
||||
that one is an authority this daemon only *trusts by value*, so it is
|
||||
public material and needs no key. This one signs, so it does.
|
||||
|
||||
Leaving this `null` — the default — makes the module mint a
|
||||
self-signed authority in `${agentCaDir}` on first boot and use that.
|
||||
That is the ordinary shape: the store pins an authority per cert-auth
|
||||
role, by value, so the authority agents are issued from does not have
|
||||
to be the store's own PKI and does not have to live on the store's
|
||||
host. Name a file here only when an operator issues agent leaves from
|
||||
somewhere else; doing so turns the self-signing unit off.
|
||||
'';
|
||||
};
|
||||
|
||||
agentCaKeyFile = lib.mkOption {
|
||||
type = lib.types.nullOr lib.types.str;
|
||||
default = null;
|
||||
example = "/var/lib/swarm-agent-ca/ca-key.pem";
|
||||
description = ''
|
||||
Private key for
|
||||
{option}`services.hyperhive.deploy.swarm-controller.agentCaFile`.
|
||||
Both or neither — an authority with no key signs nothing, and the
|
||||
daemon refuses to start half-configured rather than looking ready.
|
||||
|
||||
A path, never a value: the key's bytes in a nix expression land in
|
||||
the world-readable nix store, permanently.
|
||||
'';
|
||||
};
|
||||
|
||||
queue = {
|
||||
clientSecretFile = lib.mkOption {
|
||||
type = lib.types.str;
|
||||
|
|
@ -634,7 +709,10 @@ in
|
|||
services.hyperhive.swarm.otel.journaldUnits = [
|
||||
"swarm-controller"
|
||||
"swarm-controller-credential"
|
||||
];
|
||||
]
|
||||
# Declared only where the unit exists — an entry for a unit that was
|
||||
# never defined is a collector waiting on a journal that never speaks.
|
||||
++ lib.optional (haveAgentCa && selfSignAgentCa) "swarm-controller-agent-ca";
|
||||
|
||||
users.users.swarm-controller = {
|
||||
isSystemUser = true;
|
||||
|
|
@ -721,6 +799,18 @@ in
|
|||
state directory.
|
||||
'';
|
||||
}
|
||||
{
|
||||
assertion =
|
||||
deployCfg.swarm-controller.agentCaFile == null || deployCfg.swarm-controller.agentCaKeyFile != null;
|
||||
message = ''
|
||||
services.hyperhive.deploy.swarm-controller.agentCaFile names an
|
||||
authority but agentCaKeyFile is unset.
|
||||
|
||||
The controller does not merely trust this authority, it issues
|
||||
agent client certificates from it, so it needs the private key.
|
||||
Set both, or set neither and let the module mint its own.
|
||||
'';
|
||||
}
|
||||
];
|
||||
|
||||
systemd.services.swarm-controller = {
|
||||
|
|
@ -762,7 +852,16 @@ in
|
|||
++ lib.optional (
|
||||
haveBaoIdentity && deployCfg.bao.serverCaFile != null
|
||||
) "bao-ca.pem:${deployCfg.bao.serverCaFile}"
|
||||
++ lib.optional haveHiveClientCa "hive-client-ca.pem:${deployCfg.swarm-controller.hiveClientCaFile}";
|
||||
++ lib.optional haveHiveClientCa "hive-client-ca.pem:${deployCfg.swarm-controller.hiveClientCaFile}"
|
||||
# The agent authority, key included — same shape and same reason as
|
||||
# the store identity above: the key is root-owned `0600` and this
|
||||
# daemon runs as `swarm-controller`. `swarm-controller-agent-ca`
|
||||
# below is `requiredBy` this unit, so the files exist by the time
|
||||
# systemd resolves these.
|
||||
++ lib.optionals haveAgentCa [
|
||||
"agent-ca.pem:${agentCaCert}"
|
||||
"agent-ca-key.pem:${agentCaKey}"
|
||||
];
|
||||
|
||||
# The placeholder default that makes the above non-fatal.
|
||||
# `LoadCredential=` takes priority over `SetCredential=`, so this is
|
||||
|
|
@ -935,5 +1034,50 @@ in
|
|||
ExecStart = "${pkgs.systemd}/bin/systemctl try-restart swarm-controller.service";
|
||||
};
|
||||
};
|
||||
|
||||
# The authority agent client leaves are issued from, minted here when the
|
||||
# operator named none. Shape copied from ./glue-bao-tls.nix's
|
||||
# `swarm-bao-pki`, including the rule that matters most:
|
||||
#
|
||||
# 🩸 Idempotent on ABSENCE, never on content. Re-issuing this CA would
|
||||
# invalidate every agent leaf already published to the store AND every
|
||||
# cert-auth role that pinned it by value, locking every agent container
|
||||
# in the swarm out at once — on a rebuild that changed nothing an
|
||||
# operator asked for.
|
||||
#
|
||||
# `before` + `requiredBy` rather than `after`: the daemon's
|
||||
# `LoadCredential=` names these files by absolute path, and a
|
||||
# `LoadCredential=` pointing at a file that is not there yet is fatal
|
||||
# (`243/CREDENTIALS`), not a slow start.
|
||||
systemd.services.swarm-controller-agent-ca = lib.mkIf (haveAgentCa && selfSignAgentCa) {
|
||||
description = "mint the authority swarm agents' store certificates are issued from";
|
||||
before = [ "swarm-controller.service" ];
|
||||
requiredBy = [ "swarm-controller.service" ];
|
||||
path = [
|
||||
pkgs.openssl
|
||||
pkgs.coreutils
|
||||
];
|
||||
serviceConfig = {
|
||||
Type = "oneshot";
|
||||
RemainAfterExit = true;
|
||||
};
|
||||
script = ''
|
||||
set -euo pipefail
|
||||
install -d -m 0700 ${agentCaDir}
|
||||
|
||||
if [ ! -s ${agentCaCert} ]; then
|
||||
# `pathlen:0` — this authority signs leaves and nothing else. An
|
||||
# intermediate under it would be a second issuer for the one name
|
||||
# space the store matches agents by.
|
||||
openssl req -x509 -newkey rsa:4096 -nodes -sha256 -days 3650 \
|
||||
-keyout ${agentCaKey} -out ${agentCaCert} \
|
||||
-subj "/CN=swarm-agent-ca ${swarmDomain}" \
|
||||
-addext "basicConstraints=critical,CA:TRUE,pathlen:0" \
|
||||
-addext "keyUsage=critical,keyCertSign,cRLSign"
|
||||
chmod 0600 ${agentCaKey}
|
||||
chmod 0644 ${agentCaCert}
|
||||
fi
|
||||
'';
|
||||
};
|
||||
};
|
||||
}
|
||||
|
|
|
|||
Loading…
Reference in a new issue