add an ssh ca to bao, agent ssh certificates and a target module
This commit is contained in:
parent
ac01ef1bdb
commit
0afb111862
16 changed files with 1056 additions and 17 deletions
|
|
@ -14,3 +14,5 @@ homeserver are in [`swarm/`](../swarm/README.md).
|
|||
`hive-forge -f <label>`.
|
||||
- **[Matrix](matrix.md)** — an account on another homeserver, beside the
|
||||
agent's own.
|
||||
- **[SSH](ssh.md)** — let an agent SSH into a machine with short-lived
|
||||
certificates from the swarm's SSH CA, and make that machine trust them.
|
||||
|
|
|
|||
92
docs/2-handbook/external-accounts/ssh.md
Normal file
92
docs/2-handbook/external-accounts/ssh.md
Normal file
|
|
@ -0,0 +1,92 @@
|
|||
# SSH access to a machine
|
||||
|
||||
Let an agent SSH into a machine you name, such as a test VM, without copying a
|
||||
key onto it. The agent holds no standing SSH key. It asks the swarm's secret
|
||||
store for a certificate that lasts an hour, and the machine trusts the swarm's
|
||||
SSH certificate authority (CA) instead of a list of keys.
|
||||
|
||||
Nothing changes until you do both of these:
|
||||
|
||||
1. Switch SSH on for the agent and name the machines it may reach.
|
||||
2. Make each machine trust the CA and list the agents allowed in.
|
||||
|
||||
## 1. Switch it on for an agent
|
||||
|
||||
In the agent's config:
|
||||
|
||||
```nix
|
||||
services.hyperhive.agent.ssh = {
|
||||
enable = true;
|
||||
hosts.testvm = {
|
||||
hostName = "10.0.0.7";
|
||||
user = "test";
|
||||
port = 22; # the default
|
||||
hostKey = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAA…";
|
||||
};
|
||||
};
|
||||
```
|
||||
|
||||
`hostKey` is the machine's own host public key, the one line in
|
||||
`/etc/ssh/ssh_host_ed25519_key.pub` on it. The agent pins it: it refuses a
|
||||
machine that presents any other key, and never trusts a key on first sight.
|
||||
|
||||
The agent then runs `ssh testvm` with no other flag. The agent's SSH client
|
||||
offers its certificate and nothing else. The option needs the agent to have a
|
||||
store identity, the same as for its other store credentials.
|
||||
|
||||
This doesn't open the network. If you set the hive's `egress.forward` to
|
||||
`reject`, the hive refuses the agent's direct connection to the machine's port
|
||||
until a separate allowance exists; the default `log` lets it through. See
|
||||
[the network page](../../4-implementation/networking/network.md).
|
||||
|
||||
## 2. Make a machine trust the CA
|
||||
|
||||
On the swarm's store host, read the CA's public key:
|
||||
|
||||
```bash
|
||||
bao read -field=public_key ssh-agents/config/ca
|
||||
```
|
||||
|
||||
The store also serves the same key without a token at
|
||||
`<store address>/v1/ssh-agents/public_key`. The key is public and you set it
|
||||
once. Then import the module on the machine, which can be any NixOS machine,
|
||||
and not only a hive host:
|
||||
|
||||
```nix
|
||||
{
|
||||
imports = [ hyperhive.nixosModules.agent-ssh-target ];
|
||||
|
||||
services.hyperhive.agentSshTarget = {
|
||||
enable = true;
|
||||
caPublicKey = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAA…";
|
||||
principals.test = [ "atlas" "maia" ];
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
`principals.<user>` lists the agents that may log in as that local account. The
|
||||
account has to exist. An account with no entry accepts no certificate. The
|
||||
module writes no `authorized_keys`.
|
||||
|
||||
## What an agent can and can't do
|
||||
|
||||
- It can only get a certificate for **its own name**. The store signs for the
|
||||
name its identity carries, so atlas can't obtain one that says maia.
|
||||
- The certificate lasts an hour, and the agent signs a new one every 20
|
||||
minutes. The key lives in the agent's `/run` (memory) and never reaches disk.
|
||||
A new container start makes a new key.
|
||||
- It signs ed25519 user keys only, and the certificate allows a terminal
|
||||
(`permit-pty`) and nothing else.
|
||||
|
||||
## Revoking access
|
||||
|
||||
- **One agent from one machine:** remove its name from that user's
|
||||
`principals` and deploy the machine. The next login attempt fails.
|
||||
- **One agent from every machine:** delete its cert-auth role
|
||||
`hive-agent-<name>` in the store. It can no longer log in to the store, so
|
||||
nothing signs for it, and its last certificate lapses within three hours.
|
||||
- **The CA itself:** if the CA key changes, every machine needs the
|
||||
new public key. Nothing replaces the CA by itself.
|
||||
|
||||
For the bao paths, field names and units, see
|
||||
[the SSH CA implementation page](../../4-implementation/swarm/ssh-ca.md).
|
||||
|
|
@ -38,6 +38,7 @@ the rest of the cell says how.
|
|||
| `swarm/agents/<agent>/claude-credentials` | `swarm-claude-login`, when you log the agent in from its card; one version per login | the agent, while it waits for a login | ❌ log in again | ✅ every 30 seconds while waiting |
|
||||
| `swarm/hives/<hive>/forge/token` | `swarm-controller`, every 5 minutes | `swarm-controller` and the hive | ✅ when missing or no longer matching the forge | ✅ read on every forge call |
|
||||
| `swarm/agents/<agent>/proxy-credential` | `swarm-controller`: 32 random bytes, at start and every 5 minutes for every agent that holds a store identity | the hive (`hive-c0re`) and the agent, which fetches it | ❌ never replaced: written only while the path holds nothing | ✅ `hive-c0re` re-reads it every 30 seconds |
|
||||
| `ssh-agents/sign/swarm-agent` (a signing call, nothing is stored) | the store's SSH CA mount (`ssh-agents`), when the agent's `hive-agent-ssh-cert` unit asks it to sign the public key the unit generated in its own `/run`; the CA key is generated inside the store by `swarm-bao-agent-ssh-policy` | the agent, which keeps the key and the signed certificate in `/run` only; a target machine trusts the CA's public key by value | ✅ the certificate lasts 60 minutes; the CA is never replaced, and a target that holds a different CA key refuses every certificate | ✅ the agent signs again every 20 minutes |
|
||||
| `swarm/hives/<hive>/squid-selfcheck` | `swarm-controller`, in the same pass as the agents' credentials | the hive: `hive-c0re` and `squid-selfcheck-credential` (root) | ❌ never replaced: written only while the path holds nothing | ✅ `hive-c0re` every 30 seconds, the root unit every 10 minutes |
|
||||
| `swarm/hives/<hive>/oidc/client` | authelia, first boot, straight into the store | the hive | ❌ minted once, straight into the store; minted again only when the store holds none | ✅ re-read when authelia refuses it |
|
||||
| `swarm/services/<clientId>/oidc/client` | authelia | that service | must be stated | must be stated |
|
||||
|
|
|
|||
83
docs/4-implementation/swarm/ssh-ca.md
Normal file
83
docs/4-implementation/swarm/ssh-ca.md
Normal file
|
|
@ -0,0 +1,83 @@
|
|||
# The SSH CA for agents
|
||||
|
||||
Agents SSH out with certificates signed by an OpenBao SSH secrets engine mount.
|
||||
The operator view, how to enable it and how to trust it on a machine, is
|
||||
[SSH access to a machine](../../2-handbook/external-accounts/ssh.md).
|
||||
|
||||
## The mount and its role
|
||||
|
||||
`swarm-bao-agent-ssh-policy.service` runs on the store's host under the
|
||||
granter's identity, like the agent PKI unit in
|
||||
[the store page](bao.md). Each run:
|
||||
|
||||
1. enables the `ssh-agents` mount (type `ssh`) when it isn't there;
|
||||
2. generates the CA key inside the mount, once, with
|
||||
`config/ca generate_signing_key=true key_type=ssh-ed25519`. It does that
|
||||
only when `ssh-agents/issuers` lists none. Nothing replaces or deletes the
|
||||
CA, and the granter holds no read on it;
|
||||
3. writes the role `ssh-agents/roles/swarm-agent`.
|
||||
|
||||
The mount and role names come from `nix/host-modules/lib/ssh-agent-ca.nix` and
|
||||
are `SSH_MOUNT` and `SSH_ROLE` in `swarm-secret-client/src/policy.rs`; the
|
||||
`bao-grants` module-eval suite fails when they differ.
|
||||
|
||||
The role signs for every agent:
|
||||
|
||||
| field | value |
|
||||
| ----------------------------------------------------- | ---------------------------------------------------------------------------------------- |
|
||||
| `key_type` | `ca` |
|
||||
| `allow_user_certificates` / `allow_host_certificates` | `true` / `false` |
|
||||
| `allowed_users`, `default_user` | `{{identity.entity.name}}`, with `allowed_users_template` and `default_user_template` on |
|
||||
| `allowed_user_key_lengths` | `{"ssh-ed25519": 0}`: ed25519 keys only |
|
||||
| `allowed_extensions`, `default_extensions` | `permit-pty` |
|
||||
| `allow_user_key_ids` | `false` |
|
||||
| `ttl`, `max_ttl` | 60 minutes, 180 minutes |
|
||||
|
||||
The entity name is the agent's name: `swarm-controller` names the entity of
|
||||
each agent's cert login after the agent
|
||||
(`swarm-controller/src/agent_entity.rs`). A request for any other principal,
|
||||
a glob, host certificates, other key types or another extension is a refusal
|
||||
from the store.
|
||||
|
||||
## What an agent may call
|
||||
|
||||
The shared agent policy `hive-agent` (`render_shared_agent`) holds one more
|
||||
stanza: `update` on `ssh-agents/sign/swarm-agent`, and nothing on the rest of
|
||||
the mount. `swarm-controller` writes the policy at every start and at every
|
||||
agent mint, so a deploy that adds the stanza reaches existing agents when the
|
||||
controller restarts. A token already issued picks the change up on its next
|
||||
request, because bao reads policies per request.
|
||||
|
||||
## The agent's unit
|
||||
|
||||
With `services.hyperhive.agent.ssh.enable`, `nix/agent-modules/ssh.nix` adds
|
||||
`hive-agent-ssh-cert.service` and its timer, as the agent user:
|
||||
|
||||
- the key is `/run/hive-agent-ssh-cert/key` (directory `0700`, files `0400`,
|
||||
`UMask=0377`), generated when absent; the certificate is
|
||||
`key-cert.pub` beside it;
|
||||
- each run logs in to the store with the cert identity `hive-agent-bao-identity`
|
||||
keeps, signs `key.pub` through the role, checks the answer parses as a
|
||||
certificate and renames it into place;
|
||||
- the timer fires 20 minutes after each run, a third of the 60-minute lifetime,
|
||||
from the same file as the role's TTL;
|
||||
- the generated `/etc/ssh/ssh_config` has one `Host <name>` block per entry
|
||||
(`HostName`, `User`, `Port`, `IdentityFile`, `CertificateFile`,
|
||||
`IdentitiesOnly yes`, `StrictHostKeyChecking yes`), and
|
||||
`/etc/ssh/ssh_known_hosts` one pinned line per `hostKey` (`[host]:port` for a
|
||||
port other than 22).
|
||||
|
||||
## The target module
|
||||
|
||||
`nixosModules.agent-ssh-target` (`nix/target-modules/agent-ssh.nix`) sets
|
||||
`services.hyperhive.agentSshTarget`: `caPublicKey`, written to
|
||||
`/etc/ssh/hyperhive-agent-ssh-ca.pub` and named by `TrustedUserCAKeys`, and
|
||||
`principals.<user>`, which becomes `users.users.<user>.openssh.authorizedPrincipals`.
|
||||
NixOS then writes `/etc/ssh/authorized_principals.d/<user>` and sets
|
||||
`AuthorizedPrincipalsFile` to it.
|
||||
|
||||
## Not covered
|
||||
|
||||
Egress. The default `egress.forward = "log"` lets a direct connection from an
|
||||
agent to a machine's SSH port through, and `reject` blocks it; there is no
|
||||
per-agent allowance for it yet.
|
||||
|
|
@ -100,6 +100,10 @@ Accounts outside the swarm that an agent can hold. Start at
|
|||
the PAT injected?** → [`2-handbook/external-accounts/github.md`](2-handbook/external-accounts/github.md).
|
||||
- **How to give an agent GitLab accounts (`glab` + `git push`, more than one
|
||||
instance)?** → [`2-handbook/external-accounts/gitlab.md`](2-handbook/external-accounts/gitlab.md).
|
||||
- **How to let an agent SSH into a machine (short-lived certificates, a target
|
||||
module)?** → [`2-handbook/external-accounts/ssh.md`](2-handbook/external-accounts/ssh.md);
|
||||
the bao role and units in
|
||||
[`4-implementation/swarm/ssh-ca.md`](4-implementation/swarm/ssh-ca.md).
|
||||
|
||||
## 3. Options reference
|
||||
|
||||
|
|
|
|||
|
|
@ -197,6 +197,10 @@
|
|||
self.packages.${pkgs.stdenv.hostPlatform.system}.swagger-ui-theme;
|
||||
};
|
||||
hive-forge = ./nix/host-modules/hive-forge;
|
||||
# An arbitrary NixOS machine agents may SSH into: trusts the swarm's SSH
|
||||
# CA and lists which agent may log in as which user. Not part of
|
||||
# `default`: it needs no hive.
|
||||
agent-ssh-target = ./nix/target-modules/agent-ssh.nix;
|
||||
};
|
||||
|
||||
nixosConfigurations =
|
||||
|
|
|
|||
|
|
@ -103,6 +103,7 @@ in
|
|||
./packages.nix
|
||||
./queue-identity.nix
|
||||
./rea.nix
|
||||
./ssh.nix
|
||||
./user.nix
|
||||
./screen.nix
|
||||
./weston-vnc.nix
|
||||
|
|
|
|||
264
nix/agent-modules/ssh.nix
Normal file
264
nix/agent-modules/ssh.nix
Normal file
|
|
@ -0,0 +1,264 @@
|
|||
# Outbound SSH for this agent: a certificate for a key it generates itself,
|
||||
# signed by the swarm's SSH CA, and a client config for the hosts it may reach.
|
||||
#
|
||||
# Nothing is configured unless `services.hyperhive.agent.ssh.enable` is set. The
|
||||
# agent has no standing SSH credential: `${unitName}.service` generates an
|
||||
# ed25519 key in its own `/run` directory (tmpfs, never on disk), logs in to the
|
||||
# swarm secret store with the identity ./bao.nix keeps current, and has the
|
||||
# public half signed by the store's SSH engine
|
||||
# (`swarm_secret_client::policy::SSH_MOUNT`). The role it signs through issues a
|
||||
# user certificate with this agent's name as its only principal, valid for an
|
||||
# hour; a timer signs again well before that runs out.
|
||||
#
|
||||
# A target machine trusts the CA, not this key: ../target-modules/agent-ssh.nix.
|
||||
{
|
||||
pkgs,
|
||||
lib,
|
||||
config,
|
||||
...
|
||||
}:
|
||||
let
|
||||
cfg = config.services.hyperhive.agent.ssh;
|
||||
baoCfg = config.services.hyperhive.agent.bao;
|
||||
|
||||
# The unix user is named for the agent, so this is also the principal the
|
||||
# role puts on the certificate (the identity entity swarm-controller names
|
||||
# after the agent).
|
||||
agentName = config.services.hyperhive.agent.user.name;
|
||||
|
||||
ca = import ../host-modules/lib/ssh-agent-ca.nix { };
|
||||
|
||||
# The server CA id ./bao.nix loads.
|
||||
serverCaCredential = "hive-agent-bao-server-ca";
|
||||
|
||||
unitName = "hive-agent-ssh-cert";
|
||||
runtimeDir = unitName;
|
||||
keyFile = "/run/${runtimeDir}/key";
|
||||
pubFile = "${keyFile}.pub";
|
||||
# `ssh` reads `<IdentityFile>-cert.pub` by itself, but the config names it so
|
||||
# that the file a Host block uses is visible in the config.
|
||||
certFile = "${keyFile}-cert.pub";
|
||||
stagingFile = "/run/${runtimeDir}/cert.new";
|
||||
errFile = "/run/${runtimeDir}/bao.err";
|
||||
|
||||
signPath = "${ca.mountPath}/sign/${ca.roleName}";
|
||||
|
||||
storeRetry = import ../host-modules/lib/store-retry.nix { };
|
||||
|
||||
# `[host]:port` is how a known_hosts line names a host on a non-default port.
|
||||
knownHostName = h: if h.port == 22 then h.hostName else "[${h.hostName}]:${toString h.port}";
|
||||
|
||||
hostBlock = name: h: ''
|
||||
Host ${name}
|
||||
HostName ${h.hostName}
|
||||
User ${h.user}
|
||||
Port ${toString h.port}
|
||||
IdentityFile ${keyFile}
|
||||
CertificateFile ${certFile}
|
||||
IdentitiesOnly yes
|
||||
StrictHostKeyChecking yes
|
||||
'';
|
||||
|
||||
hostType = lib.types.submodule {
|
||||
options = {
|
||||
hostName = lib.mkOption {
|
||||
type = lib.types.strMatching "[A-Za-z0-9._:-]+";
|
||||
example = "vm1.pc2.example.com";
|
||||
description = ''
|
||||
Name or address the agent connects to. The host key below is pinned to
|
||||
exactly this spelling.
|
||||
'';
|
||||
};
|
||||
user = lib.mkOption {
|
||||
type = lib.types.strMatching "[A-Za-z0-9._-]+";
|
||||
example = "test";
|
||||
description = ''
|
||||
Account to log in as on the target. The target's
|
||||
{option}`principals` for that account must list this agent.
|
||||
'';
|
||||
};
|
||||
port = lib.mkOption {
|
||||
type = lib.types.port;
|
||||
default = 22;
|
||||
description = "TCP port of the target's sshd.";
|
||||
};
|
||||
hostKey = lib.mkOption {
|
||||
type = lib.types.strMatching "(ssh-ed25519|ssh-rsa|ecdsa-sha2-nistp(256|384|521)) [A-Za-z0-9+/]+=*( [A-Za-z0-9@._-]+)?";
|
||||
example = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAA…";
|
||||
description = ''
|
||||
The target's SSH host public key, one line as in
|
||||
{file}`/etc/ssh/ssh_host_ed25519_key.pub` on the target. It is pinned:
|
||||
the agent refuses a target that presents any other key, and there is
|
||||
no trust on first use.
|
||||
'';
|
||||
};
|
||||
};
|
||||
};
|
||||
in
|
||||
{
|
||||
options.services.hyperhive.agent.ssh = {
|
||||
enable = lib.mkEnableOption ''
|
||||
outbound SSH for this agent with short-lived certificates from the swarm
|
||||
secret store's SSH CA. Needs {option}`services.hyperhive.agent.bao.addr`
|
||||
'';
|
||||
|
||||
hosts = lib.mkOption {
|
||||
type = lib.types.attrsOf hostType;
|
||||
default = { };
|
||||
example = lib.literalExpression ''
|
||||
{
|
||||
testvm = {
|
||||
hostName = "10.0.0.7";
|
||||
user = "test";
|
||||
hostKey = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAA…";
|
||||
};
|
||||
}
|
||||
'';
|
||||
description = ''
|
||||
Hosts this agent may SSH to, by the name it types: `ssh testvm`. Each
|
||||
entry becomes a `Host` block in the system {file}`ssh_config` that
|
||||
offers this agent's certificate and nothing else, and a pinned line in
|
||||
the system {file}`ssh_known_hosts`.
|
||||
|
||||
This does not open the network path: with the hive's egress set to
|
||||
reject, direct TCP out of the container is refused whatever is
|
||||
configured here.
|
||||
'';
|
||||
};
|
||||
};
|
||||
|
||||
config = lib.mkIf cfg.enable {
|
||||
assertions = [
|
||||
{
|
||||
assertion = baoCfg.addr != null;
|
||||
message = "services.hyperhive.agent.ssh.enable needs services.hyperhive.agent.bao.addr: the certificate is signed by the swarm secret store, and this agent has no store address";
|
||||
}
|
||||
{
|
||||
assertion = lib.all (n: builtins.match "[A-Za-z0-9._-]+" n != null) (lib.attrNames cfg.hosts);
|
||||
message = "services.hyperhive.agent.ssh.hosts names are written into ssh_config and may hold only letters, digits, `.`, `_` and `-`";
|
||||
}
|
||||
];
|
||||
|
||||
programs.ssh.extraConfig = lib.concatStrings (lib.mapAttrsToList hostBlock cfg.hosts);
|
||||
programs.ssh.knownHosts = lib.mapAttrs' (
|
||||
name: h:
|
||||
lib.nameValuePair "hive-agent-ssh-${name}" {
|
||||
hostNames = [ (knownHostName h) ];
|
||||
publicKey = h.hostKey;
|
||||
}
|
||||
) cfg.hosts;
|
||||
|
||||
systemd.services.${unitName} = {
|
||||
description = "sign this agent's SSH key with the swarm's SSH CA";
|
||||
after = [
|
||||
"network.target"
|
||||
# Ordering only: that unit reports a store this agent cannot reach as
|
||||
# itself, and should get to say so first.
|
||||
"hive-agent-bao-identity.service"
|
||||
];
|
||||
wantedBy = [ "multi-user.target" ];
|
||||
path = [
|
||||
pkgs.openbao
|
||||
pkgs.openssh
|
||||
pkgs.coreutils
|
||||
];
|
||||
# ../host-modules/lib/store-retry.nix.
|
||||
inherit (storeRetry) startLimitBurst startLimitIntervalSec;
|
||||
serviceConfig = storeRetry.serviceConfig // {
|
||||
Type = "oneshot";
|
||||
# Not `RemainAfterExit`: the timer has to be able to start this unit
|
||||
# again. `RuntimeDirectoryPreserve` keeps the key between runs.
|
||||
RemainAfterExit = false;
|
||||
TimeoutStartSec = 30;
|
||||
User = agentName;
|
||||
Group = agentName;
|
||||
RuntimeDirectory = runtimeDir;
|
||||
RuntimeDirectoryMode = "0700";
|
||||
RuntimeDirectoryPreserve = "yes";
|
||||
UMask = "0377";
|
||||
LoadCredential = [ serverCaCredential ];
|
||||
};
|
||||
environment = {
|
||||
BAO_ADDR = toString baoCfg.addr;
|
||||
BAO_CLIENT_CERT = baoCfg.clientCertFile;
|
||||
BAO_CLIENT_KEY = baoCfg.clientKeyFile;
|
||||
};
|
||||
script = ''
|
||||
set -euo pipefail
|
||||
|
||||
# No identity delivered: ./bao.nix's check reports that; saying it
|
||||
# twice adds nothing.
|
||||
for f in "$BAO_CLIENT_CERT" "$BAO_CLIENT_KEY"; do
|
||||
if [ ! -s "$f" ]; then
|
||||
echo "this agent has no store identity, so it cannot get an SSH certificate." >&2
|
||||
exit 0
|
||||
fi
|
||||
if [ ! -r "$f" ]; then
|
||||
echo "cannot read $f, so this agent cannot present its store identity." >&2
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
if [ -s "$CREDENTIALS_DIRECTORY/${serverCaCredential}" ]; then
|
||||
export BAO_CACERT="$CREDENTIALS_DIRECTORY/${serverCaCredential}"
|
||||
fi
|
||||
|
||||
# `UMask=0377` makes every file this script creates `0400`, so each one
|
||||
# is removed before the command that creates it.
|
||||
key=${lib.escapeShellArg keyFile}
|
||||
pub=${lib.escapeShellArg pubFile}
|
||||
cert=${lib.escapeShellArg certFile}
|
||||
staged=${lib.escapeShellArg stagingFile}
|
||||
err=${lib.escapeShellArg errFile}
|
||||
trap 'rm -f "$err" "$staged"' EXIT
|
||||
|
||||
# The key lives as long as /run does, which is as long as this
|
||||
# container: a restart is a new key. Only a pair with both halves is
|
||||
# kept.
|
||||
if [ ! -s "$key" ] || [ ! -s "$pub" ]; then
|
||||
rm -f "$key" "$pub" "$cert"
|
||||
ssh-keygen -q -t ed25519 -N "" -C ${lib.escapeShellArg "hive-agent-${agentName}"} -f "$key"
|
||||
chmod 0400 "$key"
|
||||
fi
|
||||
|
||||
rm -f "$err"
|
||||
if ! BAO_TOKEN="$(bao login -method=cert -token-only 2>"$err")"; then
|
||||
echo "the swarm secret store at $BAO_ADDR refused or could not answer this agent's certificate login:" >&2
|
||||
if [ -s "$err" ]; then cat "$err" >&2; fi
|
||||
exit 1
|
||||
fi
|
||||
export BAO_TOKEN
|
||||
|
||||
# No principal is requested: the role's default is this agent's own
|
||||
# name, and it refuses any other.
|
||||
rm -f "$err" "$staged"
|
||||
if ! bao write -field=signed_key ${lib.escapeShellArg signPath} public_key=@"$pub" >"$staged" 2>"$err"; then
|
||||
echo "the swarm secret store would not sign this agent's SSH key at ${signPath}:" >&2
|
||||
if [ -s "$err" ]; then cat "$err" >&2; fi
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# A reply that is not a certificate must not replace the one in use.
|
||||
if ! ssh-keygen -L -f "$staged" >/dev/null 2>&1; then
|
||||
echo "what the store returned at ${signPath} is not an SSH certificate; keeping the current one." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# One rename: ssh never reads a half-written certificate.
|
||||
mv -f "$staged" "$cert"
|
||||
echo "signed this agent's SSH key as ${agentName}, valid for ${toString ca.certTtlMinutes} minutes."
|
||||
'';
|
||||
};
|
||||
|
||||
# Every third of the certificate's lifetime, so a store that is down for
|
||||
# a run or two does not outlast it.
|
||||
systemd.timers.${unitName} = {
|
||||
description = "sign this agent's SSH key again before its certificate expires";
|
||||
wantedBy = [ "timers.target" ];
|
||||
timerConfig = {
|
||||
OnUnitInactiveSec = "${toString ca.renewEveryMinutes}min";
|
||||
RandomizedDelaySec = "1min";
|
||||
};
|
||||
};
|
||||
};
|
||||
}
|
||||
|
|
@ -225,6 +225,10 @@ in
|
|||
inherit pkgs self nixosSystem;
|
||||
inherit (pkgs) lib;
|
||||
};
|
||||
module-eval-agent-ssh-target = import ./module-eval/agent-ssh-target.nix {
|
||||
inherit pkgs self nixosSystem;
|
||||
inherit (pkgs) lib;
|
||||
};
|
||||
# Executes what the two `module-eval-agent-*-bao` suites above only
|
||||
# evaluate: the rendered scripts of the forge-token, identity and
|
||||
# proxy-credential fetch units, against a stub `bao`.
|
||||
|
|
|
|||
23
nix/host-modules/lib/ssh-agent-ca.nix
Normal file
23
nix/host-modules/lib/ssh-agent-ca.nix
Normal file
|
|
@ -0,0 +1,23 @@
|
|||
# What the swarm's SSH CA and its agent role fix, spelled once for the two
|
||||
# files that must agree: ../swarm-bao.nix writes the role, and
|
||||
# ../../agent-modules/ssh.nix signs through it and renews on a schedule
|
||||
# derived from the certificate's lifetime.
|
||||
#
|
||||
# `mountPath` and `roleName` are also `swarm_secret_client::policy::SSH_MOUNT`
|
||||
# and `SSH_ROLE`, which the shared agent policy names; the bao-grants module-eval
|
||||
# suite reads that file and fails when the two disagree.
|
||||
#
|
||||
# Pure attrset, not a NixOS module.
|
||||
{ }:
|
||||
rec {
|
||||
mountPath = "ssh-agents";
|
||||
roleName = "swarm-agent";
|
||||
|
||||
# A certificate is valid this long, and the role refuses a longer request.
|
||||
certTtlMinutes = 60;
|
||||
certMaxTtlMinutes = 180;
|
||||
|
||||
# A third of the lifetime: a run that fails once still leaves the old
|
||||
# certificate valid for the retry, and a few failed runs still do.
|
||||
renewEveryMinutes = certTtlMinutes / 3;
|
||||
}
|
||||
|
|
@ -195,10 +195,13 @@ let
|
|||
# the services root. The next six set up the agent PKI mount: the mount, its
|
||||
# root and the `swarm-*` role agent certificates are issued through. No
|
||||
# `root` delete there: every agent's cert-auth role pins that root by value,
|
||||
# so replacing it would lock every agent out. The last three configure the
|
||||
# `oidc` auth method: its config, its `swarm-*` roles, and a read on the one
|
||||
# client secret that config carries. No `sys/auth`: the auth mounts are
|
||||
# created with the bootstrap token by `swarm-bao-granter-role`.
|
||||
# so replacing it would lock every agent out. The four after those set up the
|
||||
# agent SSH CA mount: the mount, its issuer list, the CA write and its
|
||||
# `swarm-*` role. No read on the CA and no delete: the CA is generated once.
|
||||
# The last three configure the `oidc` auth method: its config, its `swarm-*`
|
||||
# roles, and a read on the one client secret that config carries. No
|
||||
# `sys/auth`: the auth mounts are created with the bootstrap token by
|
||||
# `swarm-bao-granter-role`.
|
||||
#
|
||||
# Piped as a shell-quoted argument like `controllerPolicyText`, so
|
||||
# ../module-eval/bao-grants.nix can read it out of the unit script.
|
||||
|
|
@ -271,6 +274,22 @@ let
|
|||
capabilities = ["create", "update"]
|
||||
}
|
||||
|
||||
path "sys/mounts/${sshAgentsMountPath}" {
|
||||
capabilities = ["create", "update"]
|
||||
}
|
||||
|
||||
path "${sshAgentsMountPath}/issuers" {
|
||||
capabilities = ["list"]
|
||||
}
|
||||
|
||||
path "${sshAgentsMountPath}/config/ca" {
|
||||
capabilities = ["create", "update"]
|
||||
}
|
||||
|
||||
path "${sshAgentsMountPath}/roles/swarm-*" {
|
||||
capabilities = ["create", "update"]
|
||||
}
|
||||
|
||||
path "auth/oidc/config" {
|
||||
capabilities = ["create", "update"]
|
||||
}
|
||||
|
|
@ -1082,6 +1101,43 @@ let
|
|||
# live agent's leaf once it is past half of it (`agent_renewal.rs`).
|
||||
agentPkiLeafTtl = "168h";
|
||||
|
||||
# The SSH secrets engine mount whose CA signs the agents' user certificates,
|
||||
# and the one role they sign through, spelled once for this file and
|
||||
# ../agent-modules/ssh.nix in ./lib/ssh-agent-ca.nix.
|
||||
sshAgent = import ./lib/ssh-agent-ca.nix { };
|
||||
sshAgentsMountPath = sshAgent.mountPath;
|
||||
sshAgentRoleName = sshAgent.roleName;
|
||||
|
||||
# What the role pins. `{{identity.entity.name}}` is the name of the entity the
|
||||
# agent's cert login lands on, which swarm-controller sets to the agent's name
|
||||
# (`swarm-controller/src/agent_entity.rs`); with `allowed_users_template` the
|
||||
# engine checks every requested principal against it, and with
|
||||
# `default_user_template` an unrequested one defaults to it, so a certificate
|
||||
# names its holder's own agent and no other. `allowed_user_key_lengths` limits
|
||||
# the keys it signs to ed25519, whatever the length. The only extension is
|
||||
# `permit-pty`; `allow_host_certificates = false` keeps the CA from signing a
|
||||
# host key. A certificate is valid for an hour and never longer than three, and
|
||||
# `ssh-agent-ca.nix` derives the agent's renewal interval from the hour.
|
||||
sshAgentRoleBody = builtins.toJSON {
|
||||
key_type = "ca";
|
||||
allow_user_certificates = true;
|
||||
allow_host_certificates = false;
|
||||
allowed_users = "{{identity.entity.name}}";
|
||||
allowed_users_template = true;
|
||||
default_user = "{{identity.entity.name}}";
|
||||
default_user_template = true;
|
||||
allow_user_key_ids = false;
|
||||
allowed_user_key_lengths = {
|
||||
"ssh-ed25519" = 0;
|
||||
};
|
||||
allowed_extensions = "permit-pty";
|
||||
default_extensions = {
|
||||
"permit-pty" = "";
|
||||
};
|
||||
ttl = "${toString sshAgent.certTtlMinutes}m";
|
||||
max_ttl = "${toString sshAgent.certMaxTtlMinutes}m";
|
||||
};
|
||||
|
||||
# Lifetime of a token from a cert login, on every cert-auth role this file
|
||||
# writes except the batch ones. The store keeps a lease per token until it
|
||||
# expires; a login per operation at the mount default (32 days) leaves one
|
||||
|
|
@ -4181,6 +4237,40 @@ in
|
|||
'') hiveLeafIssuers
|
||||
);
|
||||
|
||||
# The SSH CA agents get their short-lived user certificates from: the
|
||||
# mount, the CA key the engine generates inside itself, and the one role
|
||||
# every agent signs through. The same idempotence as the agent PKI unit
|
||||
# below, without its restart: nothing here changes what the listener trusts.
|
||||
systemd.services.swarm-bao-agent-ssh-policy = granterPolicyUnit "set up the swarm SSH CA mount and the role agents sign their user certificates through" ''
|
||||
|
||||
mounts="$(bao secrets list -format=json)"
|
||||
case "$mounts" in
|
||||
*'"${sshAgentsMountPath}/"'*) ;;
|
||||
*) bao secrets enable -path=${sshAgentsMountPath} ssh ;;
|
||||
esac
|
||||
|
||||
# Generated once, ever: every target machine trusts this CA's public key
|
||||
# by value, so a second `config/ca` write would replace the default
|
||||
# issuer and lock every agent out of every target. There is no replace
|
||||
# branch. The issuer list is the guard, and `{}` is the only answer that
|
||||
# means "none".
|
||||
if issuers="$(bao list -format=json ${lib.escapeShellArg "${sshAgentsMountPath}/issuers"})"; then
|
||||
echo "the ${sshAgentsMountPath} mount already has a CA — leaving it alone"
|
||||
elif [ "$issuers" = '{}' ]; then
|
||||
echo "generating the swarm agent SSH CA into the ${sshAgentsMountPath} mount"
|
||||
bao write -field=public_key ${lib.escapeShellArg "${sshAgentsMountPath}/config/ca"} \
|
||||
generate_signing_key=true \
|
||||
key_type=ssh-ed25519 >/dev/null
|
||||
else
|
||||
echo "could not list the ${sshAgentsMountPath} issuers — not generating a CA over one that may exist" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Upserted every run, so a change to the role applies on deploy.
|
||||
printf '%s' ${lib.escapeShellArg sshAgentRoleBody} |
|
||||
bao write ${lib.escapeShellArg "${sshAgentsMountPath}/roles/${sshAgentRoleName}"} -
|
||||
'';
|
||||
|
||||
# The agent PKI mount: its root, its one role, the host's copy of the
|
||||
# root's certificate, and the listener's trust in it.
|
||||
#
|
||||
|
|
|
|||
|
|
@ -6,6 +6,8 @@
|
|||
# fetches it, and ../agent-modules/forge.nix's readers find it. The same fixtures
|
||||
# cover ../agent-modules/egress-proxy.nix, the agent's proxy credential and the
|
||||
# environment built from it.
|
||||
# They also cover ../agent-modules/ssh.nix: the agent's SSH certificate unit and
|
||||
# client config, on the same store address.
|
||||
{
|
||||
pkgs,
|
||||
lib,
|
||||
|
|
@ -32,12 +34,30 @@ let
|
|||
# A forge and a store: the fetch exists and every reader points at it.
|
||||
agentForgeBao = agentWith {
|
||||
services.hyperhive.agent.bao.addr = baoAddr;
|
||||
services.hyperhive.agent.ssh = {
|
||||
enable = true;
|
||||
hosts.testvm = {
|
||||
hostName = "10.0.0.7";
|
||||
user = "test";
|
||||
hostKey = hostKeyA;
|
||||
};
|
||||
hosts.altvm = {
|
||||
hostName = "vm2.t.local";
|
||||
user = "ops";
|
||||
port = 2222;
|
||||
hostKey = hostKeyB;
|
||||
};
|
||||
};
|
||||
services.hyperhive.agent.forge.url = forgeUrl;
|
||||
# A second `EnvironmentFile=` on the harness, which the proxy file must join.
|
||||
services.hyperhive.agent.backendEnvironmentFile = "/var/lib/t/backend.env";
|
||||
services.hyperhive.agent.icon = pkgs.emptyFile;
|
||||
};
|
||||
|
||||
# Outbound SSH on the same agent: two hosts, one on a non-default port.
|
||||
hostKeyA = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOHfYeWemIBqSc5Ac51ecuhi0G9LTN+57DFRq6mFKKk0";
|
||||
hostKeyB = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMY1xboa8d2ncPDhTSPKsq5WMyYfWwIqLcCOnkHLO3ad";
|
||||
|
||||
# A forge and no store: the absence arm, and what makes the cases above able
|
||||
# to fail.
|
||||
agentForgeNoBao = agentWith {
|
||||
|
|
@ -45,6 +65,15 @@ let
|
|||
services.hyperhive.agent.icon = pkgs.emptyFile;
|
||||
};
|
||||
|
||||
# SSH asked for with no store: the certificate has nowhere to be signed.
|
||||
agentSshNoBao = agentWith {
|
||||
services.hyperhive.agent.ssh.enable = true;
|
||||
services.hyperhive.agent.icon = pkgs.emptyFile;
|
||||
};
|
||||
|
||||
sshUnit = machine: machine.systemd.services.hive-agent-ssh-cert;
|
||||
sshCa = import ../host-modules/lib/ssh-agent-ca.nix { };
|
||||
|
||||
fetchUnit = machine: machine.systemd.services.hive-agent-forge-token;
|
||||
proxyUnit = machine: machine.systemd.services.hive-agent-proxy-credential;
|
||||
accountsUnit = machine: machine.systemd.services.hive-agent-forge-accounts;
|
||||
|
|
@ -521,6 +550,88 @@ let
|
|||
&& lib.hasInfix "NODE_USE_ENV_PROXY=1" s
|
||||
&& lib.hasInfix "url=\"http://${name}:$secret@$proxy\"" s;
|
||||
}
|
||||
{
|
||||
# ../agent-modules/ssh.nix. Presence arm first, then the agent that
|
||||
# did not ask for SSH, which gets neither the unit nor a host block.
|
||||
name = "an agent that enables ssh gets the certificate unit and its timer, and one that does not gets neither";
|
||||
ok =
|
||||
agentForgeBao.systemd.services ? hive-agent-ssh-cert
|
||||
&& agentForgeBao.systemd.timers ? hive-agent-ssh-cert
|
||||
&& !(agentForgeNoBao.systemd.services ? hive-agent-ssh-cert)
|
||||
&& !(agentForgeNoBao.systemd.timers ? hive-agent-ssh-cert)
|
||||
&& !(lib.hasInfix "Host testvm" agentForgeNoBao.environment.etc."ssh/ssh_config".text);
|
||||
}
|
||||
{
|
||||
# The key and the certificate stay in the unit's own tmpfs directory,
|
||||
# owner-only, and the unit asks for no principal: the role's default is
|
||||
# the agent's own name, and a `valid_principals` here would be the one
|
||||
# place the unit could name another.
|
||||
name = "the ssh certificate unit keeps the key in a 0700 directory under /run as the agent user, asks for no principal, and renames the certificate in";
|
||||
ok =
|
||||
let
|
||||
u = sshUnit agentForgeBao;
|
||||
s = u.script;
|
||||
in
|
||||
u.serviceConfig.User == "a1"
|
||||
&& u.serviceConfig.RuntimeDirectory == "hive-agent-ssh-cert"
|
||||
&& u.serviceConfig.RuntimeDirectoryMode == "0700"
|
||||
&& u.serviceConfig.UMask == "0377"
|
||||
&& lib.hasInfix "key=/run/hive-agent-ssh-cert/key" s
|
||||
&& lib.hasInfix "ssh-keygen -q -t ed25519" s
|
||||
&& lib.hasInfix "bao write -field=signed_key ssh-agents/sign/swarm-agent" s
|
||||
&& lib.hasInfix "mv -f \"$staged\" \"$cert\"" s
|
||||
&& !(lib.hasInfix "valid_principals" s)
|
||||
&& !(lib.hasInfix ".ssh" s)
|
||||
&& lib.elem "hive-agent-bao-identity.service" u.after
|
||||
&& u.environment.BAO_CLIENT_CERT == agentForgeBao.services.hyperhive.agent.bao.clientCertFile;
|
||||
}
|
||||
{
|
||||
# The certificate lives an hour and the timer fires every third of
|
||||
# that, so one missed run still leaves the old certificate valid.
|
||||
name = "the ssh certificate is renewed at least twice within its lifetime";
|
||||
ok =
|
||||
let
|
||||
every = lib.toInt (
|
||||
lib.removeSuffix "min" agentForgeBao.systemd.timers.hive-agent-ssh-cert.timerConfig.OnUnitInactiveSec
|
||||
);
|
||||
in
|
||||
every * 2 < sshCa.certTtlMinutes && every > 0;
|
||||
}
|
||||
{
|
||||
# `ssh testvm` has to work with no other flag: the host block names the
|
||||
# target, the user, the port, the one identity and its certificate, and
|
||||
# refuses an unknown host key.
|
||||
name = "each ssh host gets a Host block offering only the agent's certificate, and a pinned known_hosts line";
|
||||
ok =
|
||||
let
|
||||
conf = agentForgeBao.environment.etc."ssh/ssh_config".text;
|
||||
known = agentForgeBao.environment.etc."ssh/ssh_known_hosts".text;
|
||||
block = lib.concatStringsSep "\n" [
|
||||
"Host testvm"
|
||||
" HostName 10.0.0.7"
|
||||
" User test"
|
||||
" Port 22"
|
||||
" IdentityFile /run/hive-agent-ssh-cert/key"
|
||||
" CertificateFile /run/hive-agent-ssh-cert/key-cert.pub"
|
||||
" IdentitiesOnly yes"
|
||||
" StrictHostKeyChecking yes"
|
||||
];
|
||||
in
|
||||
lib.hasInfix block conf
|
||||
&& lib.hasInfix "Host altvm\n HostName vm2.t.local\n User ops\n Port 2222\n" conf
|
||||
&& lib.hasInfix "10.0.0.7 ${hostKeyA}" known
|
||||
# A port other than 22 is spelled the way known_hosts keys it.
|
||||
&& lib.hasInfix "[vm2.t.local]:2222 ${hostKeyB}" known
|
||||
&& !(lib.hasInfix "\nvm2.t.local " "\n${known}");
|
||||
}
|
||||
{
|
||||
name = "enabling ssh on an agent with no store address is refused, naming the option";
|
||||
ok =
|
||||
let
|
||||
refuses = m: lib.any (a: !a.assertion && lib.hasInfix "agent.bao.addr" a.message) m.assertions;
|
||||
in
|
||||
refuses agentSshNoBao && !(refuses agentForgeBao);
|
||||
}
|
||||
];
|
||||
in
|
||||
runGroup "agent-forge-bao" cases
|
||||
|
|
|
|||
120
nix/module-eval/agent-ssh-target.nix
Normal file
120
nix/module-eval/agent-ssh-target.nix
Normal file
|
|
@ -0,0 +1,120 @@
|
|||
# `checks.module-eval-agent-ssh-target` — see ./lib.nix for the shared
|
||||
# rationale (why this suite exists, naming convention, "evaluates
|
||||
# not executes").
|
||||
#
|
||||
# ../target-modules/agent-ssh.nix, the module for a machine agents SSH into. It
|
||||
# needs a fixture of its own: it is not a hive host or an agent, and the
|
||||
# agent-side cases in ./agent-forge-bao.nix cannot evaluate it.
|
||||
{
|
||||
pkgs,
|
||||
lib,
|
||||
self,
|
||||
nixosSystem,
|
||||
}:
|
||||
let
|
||||
inherit
|
||||
(import ./lib.nix {
|
||||
inherit
|
||||
pkgs
|
||||
lib
|
||||
self
|
||||
nixosSystem
|
||||
;
|
||||
})
|
||||
runGroup
|
||||
;
|
||||
|
||||
caKey = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOHfYeWemIBqSc5Ac51ecuhi0G9LTN+57DFRq6mFKKk0";
|
||||
|
||||
machine =
|
||||
extra:
|
||||
(nixosSystem {
|
||||
system = pkgs.stdenv.hostPlatform.system;
|
||||
modules = [
|
||||
self.nixosModules.agent-ssh-target
|
||||
{
|
||||
fileSystems."/" = {
|
||||
device = "/dev/null";
|
||||
fsType = "tmpfs";
|
||||
};
|
||||
boot.loader.grub.enable = false;
|
||||
system.stateVersion = "25.11";
|
||||
users.users.test.isNormalUser = true;
|
||||
users.users.ops.isNormalUser = true;
|
||||
}
|
||||
extra
|
||||
];
|
||||
}).config;
|
||||
|
||||
trusting = machine {
|
||||
services.hyperhive.agentSshTarget = {
|
||||
enable = true;
|
||||
# The command's output ends in a newline.
|
||||
caPublicKey = "${caKey}\n";
|
||||
principals.test = [
|
||||
"atlas"
|
||||
"maia"
|
||||
];
|
||||
};
|
||||
};
|
||||
|
||||
# The module imported and left off.
|
||||
untouched = machine { };
|
||||
|
||||
# Enabled with nobody listed.
|
||||
nobody = machine {
|
||||
services.hyperhive.agentSshTarget = {
|
||||
enable = true;
|
||||
caPublicKey = caKey;
|
||||
};
|
||||
};
|
||||
|
||||
principalsOf = m: user: m.environment.etc."ssh/authorized_principals.d/${user}".text;
|
||||
|
||||
cases = [
|
||||
{
|
||||
name = "an enabled target trusts the CA's public key and ties sshd to the per-user principals file";
|
||||
ok =
|
||||
trusting.services.openssh.enable
|
||||
&& trusting.services.openssh.settings.TrustedUserCAKeys == "/etc/ssh/hyperhive-agent-ssh-ca.pub"
|
||||
&& trusting.environment.etc."ssh/hyperhive-agent-ssh-ca.pub".text == "${caKey}\n"
|
||||
&&
|
||||
trusting.services.openssh.settings.AuthorizedPrincipalsFile
|
||||
== "/etc/ssh/authorized_principals.d/%u";
|
||||
}
|
||||
{
|
||||
# `ops` is a real account on the machine that no agent was listed for.
|
||||
name = "only the listed agents are principals of the listed user, and an unlisted user has no principals file";
|
||||
ok =
|
||||
principalsOf trusting "test" == "atlas\nmaia"
|
||||
&& !(trusting.environment.etc ? "ssh/authorized_principals.d/ops");
|
||||
}
|
||||
{
|
||||
# No key is ever copied: the module writes none for any user.
|
||||
name = "the target module writes no authorized key";
|
||||
ok =
|
||||
trusting.users.users.test.openssh.authorizedKeys.keys == [ ]
|
||||
&& trusting.users.users.test.openssh.authorizedKeys.keyFiles == [ ];
|
||||
}
|
||||
{
|
||||
# The absence arm, and the control that makes the cases above able to
|
||||
# fail: the same module, imported and not enabled, changes nothing.
|
||||
name = "an imported but disabled target trusts no CA";
|
||||
ok =
|
||||
!(untouched.services.openssh.settings ? TrustedUserCAKeys)
|
||||
&& !(untouched.environment.etc ? "ssh/hyperhive-agent-ssh-ca.pub")
|
||||
&& untouched.services.openssh.settings.AuthorizedPrincipalsFile == "none"
|
||||
&& !untouched.services.openssh.enable;
|
||||
}
|
||||
{
|
||||
name = "an enabled target that lists no agent is refused";
|
||||
ok =
|
||||
let
|
||||
refuses =
|
||||
m: lib.any (a: !a.assertion && lib.hasInfix "agentSshTarget.principals" a.message) m.assertions;
|
||||
in
|
||||
refuses nobody && !(refuses trusting);
|
||||
}
|
||||
];
|
||||
in
|
||||
runGroup "agent-ssh-target" cases
|
||||
|
|
@ -182,6 +182,7 @@ let
|
|||
"swarm-bao-services-issuer-policy"
|
||||
"swarm-bao-nats-tls-policy"
|
||||
"swarm-bao-agent-pki"
|
||||
"swarm-bao-agent-ssh-policy"
|
||||
"swarm-bao-nats-auth-policy"
|
||||
"swarm-bao-operator-viewer-policy"
|
||||
"swarm-bao-hive-leaf-policy"
|
||||
|
|
@ -1340,7 +1341,7 @@ let
|
|||
{
|
||||
# The granter's grants, whole. Pinned as the full list, because an added
|
||||
# path or capability is exactly what a presence check misses.
|
||||
name = "the granter's policy is exactly these twenty stanzas";
|
||||
name = "the granter's policy is exactly these twenty-four stanzas";
|
||||
ok =
|
||||
let
|
||||
cu = [
|
||||
|
|
@ -1420,6 +1421,22 @@ let
|
|||
path = "pki-agents/roles/swarm-*";
|
||||
caps = cu;
|
||||
}
|
||||
{
|
||||
path = "sys/mounts/ssh-agents";
|
||||
caps = cu;
|
||||
}
|
||||
{
|
||||
path = "ssh-agents/issuers";
|
||||
caps = [ "list" ];
|
||||
}
|
||||
{
|
||||
path = "ssh-agents/config/ca";
|
||||
caps = cu;
|
||||
}
|
||||
{
|
||||
path = "ssh-agents/roles/swarm-*";
|
||||
caps = cu;
|
||||
}
|
||||
{
|
||||
path = "auth/oidc/config";
|
||||
caps = cu;
|
||||
|
|
@ -2806,6 +2823,94 @@ let
|
|||
&& !(lib.any (a: !a.assertion && names a) baoGrantHere.assertions);
|
||||
}
|
||||
|
||||
{
|
||||
# The SSH CA is the trust every target machine holds by value, so it is
|
||||
# made once, only when the mount lists no issuer, and the granter can
|
||||
# neither read it back nor delete it. The presence arms: the same unit
|
||||
# does write the mount, the CA and the role.
|
||||
name = "the agent SSH CA is generated once from an ed25519 key, only when the mount has no issuer, and nothing deletes it";
|
||||
ok =
|
||||
let
|
||||
s = baoGrantHere.systemd.services.swarm-bao-agent-ssh-policy.script;
|
||||
generate = "ssh-agents/config/ca";
|
||||
in
|
||||
lib.length (lib.splitString generate s) == 2
|
||||
&& lib.hasInfix "generate_signing_key=true" s
|
||||
&& lib.hasInfix "key_type=ssh-ed25519" s
|
||||
&& lib.hasInfix "bao secrets enable -path=ssh-agents ssh" s
|
||||
&& lib.hasInfix "elif [ \"$issuers\" = '{}' ]; then" s
|
||||
&& !(lib.hasInfix "bao delete" s)
|
||||
&&
|
||||
(grantFor granterGrants "ssh-agents/config/ca").caps == [
|
||||
"create"
|
||||
"update"
|
||||
]
|
||||
&& grantFor granterGrants "ssh-agents/sign/swarm-agent" == null
|
||||
&& grantFor granterGrants "ssh-agents/roles/other" == null;
|
||||
}
|
||||
{
|
||||
# What keeps an agent from a certificate for another agent: the
|
||||
# principal is the caller's entity name, requested or not, and the role
|
||||
# signs nothing but ed25519 user keys with one extension. The role is
|
||||
# a single JSON document, so each fragment below is one property of it.
|
||||
name = "the agent SSH role names the caller's own entity as the only principal, signs ed25519 user keys for an hour, and one unit writes it";
|
||||
ok =
|
||||
let
|
||||
s = baoGrantHere.systemd.services.swarm-bao-agent-ssh-policy.script;
|
||||
roleWriters = lib.filter (u: matches "bao write '?ssh-agents/roles/" (u.script or "") != [ ]) (
|
||||
lib.attrValues baoGrantHere.systemd.services
|
||||
);
|
||||
in
|
||||
lib.all (t: lib.hasInfix t s) [
|
||||
''"allowed_users":"{{identity.entity.name}}"''
|
||||
''"allowed_users_template":true''
|
||||
''"default_user":"{{identity.entity.name}}"''
|
||||
''"default_user_template":true''
|
||||
''"allow_user_certificates":true''
|
||||
''"allow_host_certificates":false''
|
||||
''"allow_user_key_ids":false''
|
||||
''"allowed_user_key_lengths":{"ssh-ed25519":0}''
|
||||
''"allowed_extensions":"permit-pty"''
|
||||
''"key_type":"ca"''
|
||||
''"ttl":"60m"''
|
||||
''"max_ttl":"180m"''
|
||||
"bao write ssh-agents/roles/swarm-agent -"
|
||||
]
|
||||
&& lib.length roleWriters == 1;
|
||||
}
|
||||
{
|
||||
# The agents' `update` on the sign path comes from the shared agent
|
||||
# policy (swarm-secret-client), not from a unit here. Of the policies
|
||||
# these units write, only the granter's names the mount: the controller,
|
||||
# which writes the shared one, holds nothing on it.
|
||||
name = "of the policies the host writes, only the granter's has a grant on the agent SSH mount";
|
||||
ok =
|
||||
let
|
||||
cg = grantsIn baoGrantHere.systemd.services.swarm-bao-controller-policy.script;
|
||||
onMount = lib.filter (g: lib.hasPrefix "ssh-agents" g.path);
|
||||
in
|
||||
onMount cg == [ ]
|
||||
# The control: the granter's grants do name the mount.
|
||||
&& onMount granterGrants != [ ]
|
||||
&& lib.all (u: onMount (grantsIn baoGrantHere.systemd.services.${u}.script) == [ ]) (
|
||||
lib.filter (u: u != "swarm-bao-agent-ssh-policy") grantingUnitNames
|
||||
);
|
||||
}
|
||||
{
|
||||
# The shared agent policy (swarm-secret-client) names the mount and the
|
||||
# role; the unit above and the agent module spell the same two strings
|
||||
# from ../host-modules/lib/ssh-agent-ca.nix.
|
||||
name = "the SSH mount and role the shared agent policy names are the ones the swarm writes";
|
||||
ok =
|
||||
let
|
||||
ca = import ../host-modules/lib/ssh-agent-ca.nix { };
|
||||
policy = builtins.readFile ../../swarm-secret-client/src/policy.rs;
|
||||
in
|
||||
lib.hasInfix ''pub const SSH_MOUNT: &str = "${ca.mountPath}";'' policy
|
||||
&& lib.hasInfix ''pub const SSH_ROLE: &str = "${ca.roleName}";'' policy
|
||||
&& lib.hasInfix "bao write ssh-agents/roles/${ca.roleName} -" baoGrantHere.systemd.services.swarm-bao-agent-ssh-policy.script;
|
||||
}
|
||||
|
||||
# ── the UI's OIDC login ─────────────────────────────────────────────────
|
||||
{
|
||||
# What an `admins` login through the UI holds: the key tree, KV
|
||||
|
|
|
|||
79
nix/target-modules/agent-ssh.nix
Normal file
79
nix/target-modules/agent-ssh.nix
Normal file
|
|
@ -0,0 +1,79 @@
|
|||
# A machine agents may SSH into: it trusts the swarm's SSH CA and lists which
|
||||
# agent may log in as which local user. Exported as
|
||||
# `hyperhive.nixosModules.agent-ssh-target`, and meant for any NixOS machine,
|
||||
# not only a hive host.
|
||||
#
|
||||
# sshd accepts a user certificate when the CA that signed it is in
|
||||
# `TrustedUserCAKeys` and one of its principals is in that user's
|
||||
# `AuthorizedPrincipalsFile`. The swarm's SSH role puts exactly one principal on
|
||||
# a certificate, the agent's name, so listing an agent here is what lets it in.
|
||||
# No `authorized_keys` entry is written or needed. Removing a name from
|
||||
# `principals` ends that agent's logins at the next attempt, whatever
|
||||
# certificate it holds; the swarm stops signing for an agent whose store
|
||||
# identity is gone, and the certificates it has lapse within three hours.
|
||||
#
|
||||
# The agent's side is ../agent-modules/ssh.nix.
|
||||
{
|
||||
lib,
|
||||
config,
|
||||
...
|
||||
}:
|
||||
let
|
||||
cfg = config.services.hyperhive.agentSshTarget;
|
||||
in
|
||||
{
|
||||
options.services.hyperhive.agentSshTarget = {
|
||||
enable = lib.mkEnableOption "logins by agents holding a certificate from the swarm's SSH CA";
|
||||
|
||||
caPublicKey = lib.mkOption {
|
||||
type = lib.types.strMatching "ssh-ed25519 [A-Za-z0-9+/]+=*[[:space:]]*";
|
||||
example = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAA…";
|
||||
description = ''
|
||||
The public key of the swarm's SSH CA: a public value, set here once. Read
|
||||
it on the swarm's store host, as an operator with a token that may read
|
||||
the mount:
|
||||
|
||||
```
|
||||
bao read -field=public_key ssh-agents/config/ca
|
||||
```
|
||||
|
||||
The same key is served without a token at `<store address>/v1/ssh-agents/public_key`.
|
||||
A CA that is regenerated has a new key, and every target must then be
|
||||
given it.
|
||||
'';
|
||||
};
|
||||
|
||||
principals = lib.mkOption {
|
||||
type = lib.types.attrsOf (lib.types.listOf (lib.types.strMatching "[A-Za-z0-9_-]+"));
|
||||
default = { };
|
||||
example = {
|
||||
test = [
|
||||
"atlas"
|
||||
"maia"
|
||||
];
|
||||
};
|
||||
description = ''
|
||||
For each local account, the names of the agents that may log in as it. The
|
||||
account has to exist on this machine. An account that is not listed
|
||||
accepts no certificate.
|
||||
'';
|
||||
};
|
||||
};
|
||||
|
||||
config = lib.mkIf cfg.enable {
|
||||
assertions = [
|
||||
{
|
||||
assertion = lib.any (agents: agents != [ ]) (lib.attrValues cfg.principals);
|
||||
message = "services.hyperhive.agentSshTarget.principals lists no agent, so this machine would trust the CA and admit nobody";
|
||||
}
|
||||
];
|
||||
|
||||
services.openssh.enable = lib.mkDefault true;
|
||||
services.openssh.settings.TrustedUserCAKeys = "/etc/ssh/hyperhive-agent-ssh-ca.pub";
|
||||
environment.etc."ssh/hyperhive-agent-ssh-ca.pub".text = "${lib.trim cfg.caPublicKey}\n";
|
||||
|
||||
# NixOS writes `/etc/ssh/authorized_principals.d/<user>` for each of these
|
||||
# and points `AuthorizedPrincipalsFile` at it.
|
||||
users.users = lib.mapAttrs (_: agents: { openssh.authorizedPrincipals = agents; }) cfg.principals;
|
||||
};
|
||||
}
|
||||
|
|
@ -111,9 +111,9 @@ pub fn agents_from_role_names(names: &[String]) -> Vec<String> {
|
|||
.collect()
|
||||
}
|
||||
|
||||
/// One read stanza. With [`list_stanza`] and [`create_stanza`], the only shapes
|
||||
/// this module emits, so what an agent may write is a property of the renderers
|
||||
/// rather than of each call site.
|
||||
/// One read stanza. With [`list_stanza`], [`create_stanza`] and [`sign_stanza`],
|
||||
/// the only shapes this module emits, so what an agent may write is a property
|
||||
/// of the renderers rather than of each call site.
|
||||
fn read_stanza(path: &str) -> String {
|
||||
format!("path \"{path}\" {{\n capabilities = [\"read\"]\n}}\n")
|
||||
}
|
||||
|
|
@ -131,6 +131,12 @@ fn create_stanza(path: &str) -> String {
|
|||
format!("path \"{path}\" {{\n capabilities = [\"create\", \"read\"]\n}}\n")
|
||||
}
|
||||
|
||||
/// One sign stanza: `update`, which is what a signing endpoint takes, and
|
||||
/// nothing that reads or rewrites the role behind it.
|
||||
fn sign_stanza(path: &str) -> String {
|
||||
format!("path \"{path}\" {{\n capabilities = [\"update\"]\n}}\n")
|
||||
}
|
||||
|
||||
/// The one policy every hive's cert-auth role attaches.
|
||||
///
|
||||
/// Inside [`HIVE_PREFIX`] so that the controller may write it. It is the object
|
||||
|
|
@ -193,6 +199,16 @@ pub const SHARED_AGENT_POLICY: &str = "hive-agent";
|
|||
/// agent's name.
|
||||
const ENTITY_NAME_TEMPLATE: &str = "{{identity.entity.name}}";
|
||||
|
||||
/// The SSH secrets engine mount whose CA signs the agents' short-lived user
|
||||
/// certificates. `nix/host-modules/swarm-bao.nix` enables it and writes the one
|
||||
/// role below; `nix/agent-modules/ssh.nix` signs through it.
|
||||
pub const SSH_MOUNT: &str = "ssh-agents";
|
||||
|
||||
/// The one role on [`SSH_MOUNT`] an agent signs through. It templates the
|
||||
/// certificate's principal from the identity entity's name, so the role is the
|
||||
/// same for every agent and each can only be signed for its own name.
|
||||
pub const SSH_ROLE: &str = "swarm-agent";
|
||||
|
||||
/// Render the policy [`SHARED_AGENT_POLICY`] holds: read on one agent's
|
||||
/// credentials and list on their names (how `hive-matrix-daemon` finds its
|
||||
/// linked accounts), with the agent's name taken from the identity entity of
|
||||
|
|
@ -216,6 +232,12 @@ const ENTITY_NAME_TEMPLATE: &str = "{{identity.entity.name}}";
|
|||
/// another agent's credentials, a hive's, a service's, or the controller's, so
|
||||
/// none of [`render_shared_hives`]'s breadth applies. A hive still reads this
|
||||
/// agent's secrets, and closing that is its own decision.
|
||||
///
|
||||
/// The last stanza is `update` on [`SSH_MOUNT`]'s sign endpoint for [`SSH_ROLE`]:
|
||||
/// the agent submits a public key it generated and gets a short-lived user
|
||||
/// certificate back. It names no agent. What a certificate may say is the
|
||||
/// role's: the principal is the identity entity's name, so an agent can be
|
||||
/// signed for its own name and no other.
|
||||
#[must_use]
|
||||
pub fn render_shared_agent() -> String {
|
||||
let kind = <&str>::from(Kind::Agent);
|
||||
|
|
@ -229,7 +251,8 @@ pub fn render_shared_agent() -> String {
|
|||
"{MOUNT}/data/{ROOT}/{kind}/{ENTITY_NAME_TEMPLATE}/{}",
|
||||
backend_env::LEAF
|
||||
));
|
||||
format!("{read}{list}{seed}")
|
||||
let sign = sign_stanza(&format!("{SSH_MOUNT}/sign/{SSH_ROLE}"));
|
||||
format!("{read}{list}{seed}{sign}")
|
||||
}
|
||||
|
||||
/// What an agent's token grants under [`SHARED_AGENT_POLICY`]: the document
|
||||
|
|
@ -261,7 +284,8 @@ mod tests {
|
|||
render_shared_agent(),
|
||||
"path \"secret/data/swarm/agents/{{identity.entity.name}}/*\" {\n capabilities = [\"read\"]\n}\n\
|
||||
path \"secret/metadata/swarm/agents/{{identity.entity.name}}/*\" {\n capabilities = [\"list\"]\n}\n\
|
||||
path \"secret/data/swarm/agents/{{identity.entity.name}}/backend-env\" {\n capabilities = [\"create\", \"read\"]\n}\n"
|
||||
path \"secret/data/swarm/agents/{{identity.entity.name}}/backend-env\" {\n capabilities = [\"create\", \"read\"]\n}\n\
|
||||
path \"ssh-agents/sign/swarm-agent\" {\n capabilities = [\"update\"]\n}\n"
|
||||
);
|
||||
}
|
||||
|
||||
|
|
@ -270,8 +294,33 @@ mod tests {
|
|||
let shared = render_shared_agent();
|
||||
assert!(!shared.contains("atlas"));
|
||||
assert!(!shared.contains("agents/*"), "{shared}");
|
||||
assert!(!shared.contains("update"));
|
||||
assert!(!shared.contains("delete"));
|
||||
// `update` is the signing stanza's and no KV path's: a KV leaf the
|
||||
// agent could rewrite would let it replace a credential it only seeds.
|
||||
let secret_only = shared
|
||||
.split("path \"ssh-agents/")
|
||||
.next()
|
||||
.unwrap_or_default();
|
||||
assert!(!secret_only.contains("update"), "{shared}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_agent_may_sign_through_one_role_and_do_nothing_else_on_the_ssh_mount() {
|
||||
let shared = render_shared_agent();
|
||||
let ssh: Vec<&str> = shared
|
||||
.split("path \"")
|
||||
.filter(|s| s.starts_with(SSH_MOUNT))
|
||||
.collect();
|
||||
assert_eq!(
|
||||
ssh,
|
||||
["ssh-agents/sign/swarm-agent\" {\n capabilities = [\"update\"]\n}\n"]
|
||||
);
|
||||
// The control: the filter finds a stanza that is there.
|
||||
assert!(shared.split("path \"").any(|s| s.starts_with(MOUNT)));
|
||||
assert_eq!(
|
||||
format!("{SSH_MOUNT}/sign/{SSH_ROLE}"),
|
||||
"ssh-agents/sign/swarm-agent"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
|
@ -476,7 +525,8 @@ mod tests {
|
|||
/// What `effective_agent("atlas")` must be, byte for byte.
|
||||
const ATLAS_DOCUMENT: &str = "path \"secret/data/swarm/agents/atlas/*\" {\n capabilities = [\"read\"]\n}\n\
|
||||
path \"secret/metadata/swarm/agents/atlas/*\" {\n capabilities = [\"list\"]\n}\n\
|
||||
path \"secret/data/swarm/agents/atlas/backend-env\" {\n capabilities = [\"create\", \"read\"]\n}\n";
|
||||
path \"secret/data/swarm/agents/atlas/backend-env\" {\n capabilities = [\"create\", \"read\"]\n}\n\
|
||||
path \"ssh-agents/sign/swarm-agent\" {\n capabilities = [\"update\"]\n}\n";
|
||||
|
||||
#[test]
|
||||
fn an_agents_document_is_its_stanzas_naming_that_agent() {
|
||||
|
|
@ -515,8 +565,8 @@ mod tests {
|
|||
assert!(!p.contains("argus"), "no other principal is named");
|
||||
assert_eq!(
|
||||
p.matches("path \"").count(),
|
||||
3,
|
||||
"three stanzas, or the document grants something unaccounted for: {p}"
|
||||
4,
|
||||
"four stanzas, or the document grants something unaccounted for: {p}"
|
||||
);
|
||||
}
|
||||
|
||||
|
|
@ -560,7 +610,12 @@ mod tests {
|
|||
["secret/data/swarm/agents/atlas/*", leaf.as_str()]
|
||||
);
|
||||
assert_eq!(paths_granting(&p, "create"), [leaf.as_str()]);
|
||||
for capability in ["update", "delete", "sudo", "patch"] {
|
||||
// The one `update` is the SSH signing endpoint's: no KV path takes it.
|
||||
assert_eq!(
|
||||
paths_granting(&p, "update"),
|
||||
["ssh-agents/sign/swarm-agent"]
|
||||
);
|
||||
for capability in ["delete", "sudo", "patch"] {
|
||||
assert!(
|
||||
!p.contains(capability),
|
||||
"an agent's document must not grant {capability}: {p}"
|
||||
|
|
@ -581,8 +636,9 @@ mod tests {
|
|||
// granted, nor is `list` anywhere but `metadata/`.
|
||||
let p = effective_agent("atlas");
|
||||
let stanzas: Vec<&str> = p.split_inclusive("}\n").collect();
|
||||
assert_eq!(stanzas.len(), 3, "{p}");
|
||||
// The third is the create stanza, pinned by the test above.
|
||||
assert_eq!(stanzas.len(), 4, "{p}");
|
||||
// The third is the create stanza and the fourth the signing stanza,
|
||||
// each pinned by a test above.
|
||||
for stanza in &stanzas[..2] {
|
||||
let on_data = stanza.starts_with("path \"secret/data/");
|
||||
let on_metadata = stanza.starts_with("path \"secret/metadata/");
|
||||
|
|
|
|||
Loading…
Reference in a new issue