vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

fix(#3882): pin the gid that owns the TPM node, on both sides of the boundary

Browse source 
#3880 bound /dev/tpmrm0 into the store's container, which was necessary and
not sufficient: a bind mount preserves the host's ownership, openbao runs as a
DynamicUser with an empty CapabilityBoundingSet (so no CAP_DAC_OVERRIDE), and
the host applies no tpm udev rule at all — measured, 0 hits across all 41 host
closures against 62 rule files as a control. The node therefore keeps the
kernel default and the seal cannot open it.

A name cannot fix this. NixOS allocates system groups at activation, per
machine, so `tss` — or any group declared on both sides — gets two different
ids, and the device node carries the number. mara picked pinning a gid with an
overridable default (deploy.bao.tpmGid).

The default sits above the range NixOS auto-assigns system groups from
(400-999, measured in update-users-groups.pl) and above the normal-user range,
and below systemd's DynamicUser range (61184-65519), so it collides with
nothing any of those allocate.

The module-eval case compares the two sides rather than checking each against
a literal: the property is that they AGREE, not what they agree on. Its
absence arm is a shamir store, which never opens a TPM and must not claim a
device node's group — without it, pinning unconditionally would look identical.
This commit is contained in:
atlas 2026-08-31 23:03:19 +02:00
commit c044a33be9
2 changed files with 88 additions and 4 deletions
Download patch file Download diff file Expand all files Collapse all files

65
nix/host-modules/swarm-bao.nix
View file

@ -89,6 +89,12 @@ let
# hands the store over by group rather than by owner.
tokenGroup = "swarm-bao-token";
# The store's half of the TPM device's ownership. Declared on BOTH sides of
# the container boundary with the same pinned gid: the node is the host's and
# carries a number, while the unit that opens it lives in the container, so a
# name alone resolves to two different groups. See `deploy.bao.tpmGid`.
tpmGroup = "swarm-bao-tpm";
sealSettings = lib.optionalAttrs (baoDeploy.seal == "pkcs11") {
seal.pkcs11 = {
lib = "${pkgs.tpm2-pkcs11}/lib/libtpm2_pkcs11.so";
@ -278,6 +284,32 @@ in
'';
};
tpmGid = lib.mkOption {
type = lib.types.int;
default = 31337;
example = 4242;
description = ''
Numeric group id owning `/dev/tpmrm0`, so the store's seal can open it.
⚠️ A **number**, and it has a default, because a name cannot do this
job. The store runs as a `DynamicUser` whose uid is allocated inside
its container, and NixOS allocates `tss` — and every other system
group — at *activation*, per machine. A group declared on the host and
a group of the same name declared in the container therefore get
different ids, and the device node carries the number. Pinning one
value is what makes the two sides agree.
The default sits above the range NixOS auto-assigns system groups from
(400–999) and above the normal-user range (1000–29999), and below the
range systemd allocates `DynamicUser` ids from (61184–65519), so it
collides with nothing those allocate. Override it if it collides with
something this module cannot see.
Only read when {option}`services.hyperhive.deploy.bao.seal` is
`pkcs11`; a shamir store never touches the TPM.
'';
};
serverCertFile = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
@ -607,6 +639,21 @@ in
'';
};
# The device node is the host's, so its ownership is set here — the
# container can only be given a group that already matches.
#
# ⚠️ If the host also sets `security.tpm2.tssGroup`, both rules match
# `tpmrm*` and the later one wins. That is survivable (either group
# reaches the device) but worth knowing before debugging a 0660 node the
# store still cannot open.
users.groups.${tpmGroup} = lib.mkIf (baoDeploy.seal == "pkcs11") {
gid = baoDeploy.tpmGid;
};
services.udev.extraRules = lib.mkIf (baoDeploy.seal == "pkcs11") ''
KERNEL=="tpmrm[0-9]*", MODE="0660", GROUP="${tpmGroup}"
'';
# The token store is a HOST path (see the bind mount below), so the
# directory has to exist before the container starts — `bindMounts` only
# adds a `RequiresMountsFor`, and nothing in nixos-containers creates a
@ -697,7 +744,14 @@ in
# resolvconf on would let host-tracking regenerate it empty.
networking.resolvconf.enable = lib.mkForce false;
users.groups = lib.mkIf (baoDeploy.seal == "pkcs11") { ${tokenGroup} = { }; };
# `${tokenGroup}` takes whatever gid this container allocates —
# nothing outside reads it. `${tpmGroup}` must take the PINNED one,
# because the device node it names is the host's and matching is by
# number; letting this side auto-allocate is the whole bug.
users.groups = lib.mkIf (baoDeploy.seal == "pkcs11") {
${tokenGroup} = { };
${tpmGroup}.gid = baoDeploy.tpmGid;
};
# Provisioned here rather than on the host because the store has to
# end up reachable by openbao's `DynamicUser`, and that identity
@ -800,7 +854,14 @@ in
# bind mount read-only to this unit however it is owned, and the
# pkcs11 library opens its sqlite store read-write.
ReadWritePaths = [ tokenStoreDir ];
SupplementaryGroups = [ tokenGroup ];
# Two groups, two different jobs: the store's sqlite file, and
# the TPM device the seal opens through it. `CapabilityBounding
# Set=` is empty upstream, so there is no `CAP_DAC_OVERRIDE` to
# fall back on — group membership is the only way in.
SupplementaryGroups = [
tokenGroup
tpmGroup
];
};
# ⚠️ Upstream sets `restartIfChanged = false` on this unit, on