From ed53e9abccf912b81a257ed9f76077c162daa8c9 Mon Sep 17 00:00:00 2001 From: atlas Date: Sat, 26 Sep 2026 15:16:42 +0200 Subject: [PATCH 1/5] host-modules: write credential files atomically; agent-modules: retry a failed .claude migration MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Four host glue units fetched a secret from swarm-bao and rendered it with `> path; chmod`: a reader racing the write could see a truncated file, and briefly one at the wrong mode before the chmod landed. glue-matrix-bao-token.nix, glue-queue-agent-credential.nix (both files), swarm-grafana.nix and swarm-otel.nix now write to a same- directory temp file, set its final mode/owner, then `mv -f` it over the target — a shared `atomic_write_secret` helper (nix/host-modules/lib/atomic-write-secret.nix) so the five call sites share one implementation. The first-boot `/root/.claude` migration in nix/agent-modules/user.nix wrote its done-marker unconditionally, so a failed `cp` (disk full, permission error) left the marker behind and no boot ever retried the copy. The marker is now written only when there was nothing to migrate or the copy succeeded; `cp -an`'s no-clobber semantics already make a retry after a partial copy safe. Refs #4723 --- docs/agent-lifecycle/persistence.md | 7 +++- nix/agent-modules/user.nix | 27 ++++++++++---- nix/host-modules/glue-matrix-bao-token.nix | 8 ++-- .../glue-queue-agent-credential.nix | 11 +++--- nix/host-modules/lib/atomic-write-secret.nix | 37 +++++++++++++++++++ nix/host-modules/swarm-grafana.nix | 9 +++-- nix/host-modules/swarm-otel.nix | 9 +++-- 7 files changed, 83 insertions(+), 25 deletions(-) create mode 100644 nix/host-modules/lib/atomic-write-secret.nix diff --git a/docs/agent-lifecycle/persistence.md b/docs/agent-lifecycle/persistence.md index 6f697e73..ac536424 100644 --- a/docs/agent-lifecycle/persistence.md +++ b/docs/agent-lifecycle/persistence.md @@ -535,9 +535,12 @@ container lifetime: 2. **Migrate any leftover `/root/.claude` content into `${homeDir}/.claude`** — legacy `claude` wrote to root's empty home; the bind mount didn't exist yet. Marker - (`/var/lib/hive-agent-user-migrated`) guards single-shot. + (`/var/lib/hive-agent-user-migrated`) guards single-shot; it is + only written once there is nothing left to migrate, so a `cp` + failure leaves it absent and the next boot retries. `cp -an` (no-clobber) so any pre-existing files at the new - location win — never blow over data already there. + location win — never blow over data already there, and so a + retry after a partial copy is as safe as the first attempt. 3. **Chown the bind-mounted state dir** (`/agents/*/state`) recursively so the agent user can read/write it. Wildcard matches the single agent that container sees; `-h` skips diff --git a/nix/agent-modules/user.nix b/nix/agent-modules/user.nix index 52424e00..8d620cce 100644 --- a/nix/agent-modules/user.nix +++ b/nix/agent-modules/user.nix @@ -172,16 +172,29 @@ in userName=${lib.escapeShellArg userName} mkdir -p "$homeDir" chown "$userName:$userName" "$homeDir" + # The marker is only written once nothing is left to migrate — either + # there was nothing under /root/.claude, or `cp` copied it all. A + # failed `cp` (disk full, permission error) leaves the marker absent, + # so the next boot's activation retries; `-an` never clobbers a file + # this attempt already placed, so a retry after a partial copy is + # exactly as safe as the first attempt. marker=/var/lib/hive-agent-user-migrated - if [ ! -e "$marker" ] && [ -d /root/.claude ] && [ "$(ls -A /root/.claude 2>/dev/null)" ]; then - mkdir -p "$homeDir/.claude" - if cp -an /root/.claude/. "$homeDir/.claude/" 2>/dev/null; then - rm -rf /root/.claude - echo "hive-agent-user-migrate: moved /root/.claude → $homeDir/.claude" + if [ ! -e "$marker" ]; then + if [ -d /root/.claude ] && [ "$(ls -A /root/.claude 2>/dev/null)" ]; then + mkdir -p "$homeDir/.claude" + if cp -an /root/.claude/. "$homeDir/.claude/" 2>/dev/null; then + rm -rf /root/.claude + echo "hive-agent-user-migrate: moved /root/.claude → $homeDir/.claude" + mkdir -p "$(dirname "$marker")" + : > "$marker" + else + echo "hive-agent-user-migrate: copying /root/.claude to $homeDir/.claude failed; will retry next boot" >&2 + fi + else + mkdir -p "$(dirname "$marker")" + : > "$marker" fi fi - mkdir -p "$(dirname "$marker")" - : > "$marker" # Scope state + harness chowns to THIS container's own dirs only. # The glob `/agents/*/state` also matches other agents' state dirs # bind-mounted into a `ManageRootAgent` holder's container, which diff --git a/nix/host-modules/glue-matrix-bao-token.nix b/nix/host-modules/glue-matrix-bao-token.nix index 3c1bdb62..d1e36de0 100644 --- a/nix/host-modules/glue-matrix-bao-token.nix +++ b/nix/host-modules/glue-matrix-bao-token.nix @@ -80,6 +80,8 @@ let # resolution, so this is the kind of mistake only reading the target module # catches. matrixMachine = "hive-matrix"; + + atomicWriteSecret = import ./lib/atomic-write-secret.nix { }; in { config = lib.mkMerge [ @@ -187,6 +189,8 @@ in script = '' set -euo pipefail + ${atomicWriteSecret} + # A sealed or uninitialised store answers on the port and never # answers the read, so "the store is up" is not the same as "the # store can answer". `TimeoutStartSec` above is the bound; the @@ -237,9 +241,7 @@ in exit 0 fi - umask 077 - printf '%s\n' "$token" > ${lib.escapeShellArg (toString deployCfg.matrix.appserviceTokenFile)} - chmod 0600 ${lib.escapeShellArg (toString deployCfg.matrix.appserviceTokenFile)} + printf '%s\n' "$token" | atomic_write_secret 0600 "" ${lib.escapeShellArg (toString deployCfg.matrix.appserviceTokenFile)} # Re-stamp the registration file from the token just written. The # token is half an agreement — the registration the homeserver loads diff --git a/nix/host-modules/glue-queue-agent-credential.nix b/nix/host-modules/glue-queue-agent-credential.nix index 21900692..33efb086 100644 --- a/nix/host-modules/glue-queue-agent-credential.nix +++ b/nix/host-modules/glue-queue-agent-credential.nix @@ -77,6 +77,8 @@ let # its own use of it: it is asserted set wherever # `deploy.hive-controller.enable` is. credentialPath = "secret/swarm/hives/${hyperhiveCfg.hiveName}/queue/agent"; + + atomicWriteSecret = import ./lib/atomic-write-secret.nix { }; in { options.services.hyperhive.deploy.hive-controller.queue = { @@ -217,6 +219,8 @@ in script = '' set -euo pipefail + ${atomicWriteSecret} + # `bao`'s own message is the only thing separating a missing value from # a refused identity from an unreachable host. This unit's degraded # mode is correct for all three, so it reports which one rather than @@ -279,14 +283,11 @@ in install -d -m 0755 ${lib.escapeShellArg credentialDir} - umask 077 - printf '%s\n' "$secret" > ${lib.escapeShellArg secretFile} - chmod 0600 ${lib.escapeShellArg secretFile} + printf '%s\n' "$secret" | atomic_write_secret 0600 "" ${lib.escapeShellArg secretFile} # `0644` on purpose: an OIDC client id is presented to the token # endpoint on every connection and is public by construction. - printf '%s\n' "$client_id" > ${lib.escapeShellArg clientIdFile} - chmod 0644 ${lib.escapeShellArg clientIdFile} + printf '%s\n' "$client_id" | atomic_write_secret 0644 "" ${lib.escapeShellArg clientIdFile} ''; }; }) diff --git a/nix/host-modules/lib/atomic-write-secret.nix b/nix/host-modules/lib/atomic-write-secret.nix new file mode 100644 index 00000000..dccf7e9d --- /dev/null +++ b/nix/host-modules/lib/atomic-write-secret.nix @@ -0,0 +1,37 @@ +# Shared shell step for a systemd oneshot that lands a freshly-fetched +# secret on disk: never `> path` the live file directly, because a reader +# racing the write can open it between the truncate and the write, or +# between the write and a `chmod` that follows it, and see an empty file +# or one at the wrong mode. `mktemp` always creates its file at 0600 +# regardless of umask, so the temp file is private for its whole life; only +# the `chmod`/`chown`/`mv -f` sequence below ever makes the target mode and +# owner visible, and only once the content is already final. +# +# Pure function — NOT a NixOS module. Call it from a module's `let`: +# +# atomicWriteSecret = import ./lib/atomic-write-secret.nix { }; +# ... +# script = '' +# ${atomicWriteSecret} +# printf '%s\n' "$secret" | atomic_write_secret 0600 "" "$path" +# printf '%s\n' "$secret" | atomic_write_secret 0400 "grafana:0" "$path" +# ''; +# +# Third argument to `atomic_write_secret` is the target path; the second is +# an owner for `chown` (`user:group` or a bare uid), or "" to leave the +# mktemp-created root:root ownership as it is. Reads its content from +# stdin. Requires `coreutils` on the caller's `path`. +{ }: +'' + atomic_write_secret() { + local mode="$1" owner="$2" target="$3" tmp + tmp="$(mktemp "$(dirname -- "$target")/.$(basename -- "$target").XXXXXX")" + trap 'rm -f "$tmp"' RETURN + cat > "$tmp" + chmod "$mode" "$tmp" + if [ -n "$owner" ]; then + chown "$owner" "$tmp" + fi + mv -f "$tmp" "$target" + } +'' diff --git a/nix/host-modules/swarm-grafana.nix b/nix/host-modules/swarm-grafana.nix index b447427a..355e8c6f 100644 --- a/nix/host-modules/swarm-grafana.nix +++ b/nix/host-modules/swarm-grafana.nix @@ -168,6 +168,8 @@ let nginxGid = config.ids.gids.nginx; grafanaUid = config.ids.uids.grafana; + atomicWriteSecret = import ./lib/atomic-write-secret.nix { }; + in { # `enable` moved to `services.hyperhive.deploy.grafana.enable` — see @@ -633,6 +635,8 @@ in script = '' set -euo pipefail + ${atomicWriteSecret} + # `bao`'s own message is the only thing separating a missing value from # a refused identity from an unreachable host. This unit's degraded # mode is correct for all three, so it reports which one rather than @@ -689,10 +693,7 @@ in # grants the group nothing; if this mode ever widens, the gid has to be # discovered at runtime rather than assumed. install -d -m 0755 ${lib.escapeShellArg hostSecretDir} - umask 077 - printf '%s\n' "$secret" > ${lib.escapeShellArg hostSecretPath} - chown ${toString config.ids.uids.grafana}:0 ${lib.escapeShellArg hostSecretPath} - chmod 0400 ${lib.escapeShellArg hostSecretPath} + printf '%s\n' "$secret" | atomic_write_secret 0400 ${lib.escapeShellArg "${toString config.ids.uids.grafana}:0"} ${lib.escapeShellArg hostSecretPath} ''; }; diff --git a/nix/host-modules/swarm-otel.nix b/nix/host-modules/swarm-otel.nix index edb36ff5..3b1f2e38 100644 --- a/nix/host-modules/swarm-otel.nix +++ b/nix/host-modules/swarm-otel.nix @@ -166,6 +166,8 @@ let # collector cannot open, discovered at runtime and nowhere else. collectorSecretPath = "/run/credentials/opentelemetry-collector.service/${collectorCredentialId}"; + atomicWriteSecret = import ./lib/atomic-write-secret.nix { }; + # `attrNames` is sorted, so this is a function of the hive SET and not of # the order anyone wrote it in. # @@ -817,6 +819,8 @@ in script = '' set -euo pipefail + ${atomicWriteSecret} + # `bao`'s own message is the only thing separating a missing value # from a refused identity from an unreachable host. This unit's # degraded mode is correct for all three, so it reports which one @@ -862,10 +866,7 @@ in # no uid to give this to — `LoadCredential` reads it as root before # the sandbox exists and re-exposes it to whichever uid the unit got. install -d -m 0755 ${lib.escapeShellArg collectorHostSecretDir} - umask 077 - printf '%s\n' "$secret" > ${lib.escapeShellArg collectorHostSecretPath} - chown root:root ${lib.escapeShellArg collectorHostSecretPath} - chmod 0400 ${lib.escapeShellArg collectorHostSecretPath} + printf '%s\n' "$secret" | atomic_write_secret 0400 root:root ${lib.escapeShellArg collectorHostSecretPath} ''; }; From 5466cec066b83c31c987327d5a31fd65ff7d67e8 Mon Sep 17 00:00:00 2001 From: atlas Date: Sat, 26 Sep 2026 15:42:30 +0200 Subject: [PATCH 2/5] host-modules: fix atomic_write_secret's leftover-tmp bug; convert swarm-bao's forwarder-oidc writer too MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit atomic_write_secret's cleanup trap used RETURN, which never fires when set -e aborts the function mid-body (a failing cat/chmod/chown), so a secret-bearing temp file was left behind instead of being removed. The write now runs in a subshell with its own EXIT trap, invoked via a named handler (so `local rc=$?` is a normal, shellcheck-visible assignment) that only removes the temp file when the subshell's exit status is nonzero — the subshell's trap table is private, so a calling unit's own EXIT trap is untouched. Reproduced the leftover-tmp bug against the prior commit, confirmed it's gone, and confirmed both the success path and a caller's own EXIT trap still work as before. swarm-bao.nix's swarm-bao-forwarder-oidc unit had the identical write-then-chmod-on-live-path defect as the four sites already fixed here (fetches an OIDC client secret from swarm-bao, printfs it to the live host path, chowns/chmods after) and was missed by the original sweep. Converted it to atomic_write_secret; content and final owner/mode (root:root, 0400) are unchanged. Refs #4723 --- nix/host-modules/lib/atomic-write-secret.nix | 28 +++++++++++++++----- nix/host-modules/swarm-bao.nix | 9 ++++--- 2 files changed, 27 insertions(+), 10 deletions(-) diff --git a/nix/host-modules/lib/atomic-write-secret.nix b/nix/host-modules/lib/atomic-write-secret.nix index dccf7e9d..cf50e12a 100644 --- a/nix/host-modules/lib/atomic-write-secret.nix +++ b/nix/host-modules/lib/atomic-write-secret.nix @@ -26,12 +26,28 @@ atomic_write_secret() { local mode="$1" owner="$2" target="$3" tmp tmp="$(mktemp "$(dirname -- "$target")/.$(basename -- "$target").XXXXXX")" - trap 'rm -f "$tmp"' RETURN - cat > "$tmp" - chmod "$mode" "$tmp" - if [ -n "$owner" ]; then - chown "$owner" "$tmp" - fi + # The write happens in a subshell so its own EXIT trap cleans up `$tmp` + # on failure (a `RETURN` trap does not fire when `set -e` aborts the + # function mid-body) without touching an EXIT trap the calling script's + # own `script` may already have — subshell traps are local to the + # subshell. A named handler, not an inline trap string, so `local rc=$?` + # is a normal function-local assignment shellcheck can see: it only + # removes `$tmp` when `$?` is nonzero — on the ordinary path the + # subshell also exits successfully, and `$tmp` has to survive that to + # reach the `mv` below. + ( + _atomic_write_secret_cleanup() { + local rc=$? + [ "$rc" -eq 0 ] || rm -f "$tmp" + exit "$rc" + } + trap _atomic_write_secret_cleanup EXIT + cat > "$tmp" + chmod "$mode" "$tmp" + if [ -n "$owner" ]; then + chown "$owner" "$tmp" + fi + ) mv -f "$tmp" "$target" } '' diff --git a/nix/host-modules/swarm-bao.nix b/nix/host-modules/swarm-bao.nix index c517e763..459934f3 100644 --- a/nix/host-modules/swarm-bao.nix +++ b/nix/host-modules/swarm-bao.nix @@ -764,6 +764,8 @@ let forwarderSecretInContainer = "/var/lib/swarm-bao-otel-oidc/${cfg.otel.clientId}.secret"; forwarderHostSecretPath = "/var/lib/nixos-containers/${cfg.machine}${forwarderSecretInContainer}"; forwarderHostSecretDir = builtins.dirOf forwarderHostSecretPath; + + atomicWriteSecret = import ./lib/atomic-write-secret.nix { }; # The collector runs under `DynamicUser` and opens `client_secret_file` # itself, so there is no uid to hand a 0400 file to. `LoadCredential` reads # it as root before the sandbox exists and re-exposes it under a path that @@ -1797,6 +1799,8 @@ in script = '' set -euo pipefail + ${atomicWriteSecret} + # `bao`'s own message is the only thing separating a missing value # from a refused identity from an unreachable store, and this unit # retries on all three — so it reports which one rather than @@ -1837,10 +1841,7 @@ in # an argument in /proc the way `install <<<"$secret"` or an `echo` # from `path` would. install -d -m 0755 ${lib.escapeShellArg forwarderHostSecretDir} - umask 077 - printf '%s\n' "$secret" > ${lib.escapeShellArg forwarderHostSecretPath} - chown root:root ${lib.escapeShellArg forwarderHostSecretPath} - chmod 0400 ${lib.escapeShellArg forwarderHostSecretPath} + printf '%s\n' "$secret" | atomic_write_secret 0400 root:root ${lib.escapeShellArg forwarderHostSecretPath} ''; }; }) From 7a9fadc21ad994c361219145886ddc62de31e207 Mon Sep 17 00:00:00 2001 From: atlas Date: Sat, 26 Sep 2026 18:05:40 +0200 Subject: [PATCH 3/5] host-modules: convert swarm-bao's HSM-PIN env writer to atomic_write_secret MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit swarm-bao-token's pin.env write (the BAO_HSM_PIN EnvironmentFile for openbao's pkcs11 seal) had the same write-then-chmod-on-live-path shape as the sites already converted: printf > path directly on the live file, chmod after. Same fix, same helper. Content ("BAO_HSM_PIN=\n") and final mode (0400, root-owned — no chown, same as before) are unchanged; pin.env stays at the same path, so openbao's EnvironmentFile= reference needs no change. Refs #4723 --- nix/host-modules/swarm-bao.nix | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/nix/host-modules/swarm-bao.nix b/nix/host-modules/swarm-bao.nix index 459934f3..05daa859 100644 --- a/nix/host-modules/swarm-bao.nix +++ b/nix/host-modules/swarm-bao.nix @@ -2595,6 +2595,9 @@ in }; script = '' set -euo pipefail + + ${atomicWriteSecret} + install -d -m 0770 -g ${tokenGroup} ${tokenStoreDir} # Required whenever the store is not at its default location, or the # library cannot find the token the seal asks for. @@ -2644,8 +2647,7 @@ in chgrp ${tokenGroup} ${tokenStoreDir}/tpm2_pkcs11.sqlite3 chmod 0660 ${tokenStoreDir}/tpm2_pkcs11.sqlite3 - ( umask 077; printf 'BAO_HSM_PIN=%s\n' "$(cat ${tokenStoreDir}/user-pin)" > ${pinEnvFile} ) - chmod 0400 ${pinEnvFile} + printf 'BAO_HSM_PIN=%s\n' "$(cat ${tokenStoreDir}/user-pin)" | atomic_write_secret 0400 "" ${pinEnvFile} ''; }; From 770f68c2725d72c2676bf8cc3c3a20212819ccce Mon Sep 17 00:00:00 2001 From: atlas Date: Sat, 26 Sep 2026 18:30:16 +0200 Subject: [PATCH 4/5] host-modules: atomic_write_secret takes the value as an argument, not stdin MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The pipe contract had a gap: if a producer piped into atomic_write_secret exited non-zero after writing partial output, cat still saw a clean EOF and wrote that partial content through to the live target via mv — pipefail only reported the failure afterward, once the bad write was already committed. The helper now takes the value as its 4th argument and writes it itself with printf (a shell builtin, so the value never touches an external process's own argv/environ, same as a function argument never does), so there is no pipe left to fail silently. Callers that compute the value with a command now capture it into a variable first (`value=$(cmd)`), which fails under `set -e` before atomic_write_secret is ever called — swarm-bao.nix's pin.env site is the one that needed this (`pin_env_value="BAO_HSM_PIN=$(cat ...)"`). All seven call sites converted; output is byte-identical (same printf '%s\n' framing, now applied inside the helper instead of by each caller). Refs #4723 --- nix/host-modules/glue-matrix-bao-token.nix | 2 +- .../glue-queue-agent-credential.nix | 4 +-- nix/host-modules/lib/atomic-write-secret.nix | 32 ++++++++++++++----- nix/host-modules/swarm-bao.nix | 9 ++++-- nix/host-modules/swarm-grafana.nix | 2 +- nix/host-modules/swarm-otel.nix | 2 +- 6 files changed, 36 insertions(+), 15 deletions(-) diff --git a/nix/host-modules/glue-matrix-bao-token.nix b/nix/host-modules/glue-matrix-bao-token.nix index d1e36de0..0bb4e716 100644 --- a/nix/host-modules/glue-matrix-bao-token.nix +++ b/nix/host-modules/glue-matrix-bao-token.nix @@ -241,7 +241,7 @@ in exit 0 fi - printf '%s\n' "$token" | atomic_write_secret 0600 "" ${lib.escapeShellArg (toString deployCfg.matrix.appserviceTokenFile)} + atomic_write_secret 0600 "" ${lib.escapeShellArg (toString deployCfg.matrix.appserviceTokenFile)} "$token" # Re-stamp the registration file from the token just written. The # token is half an agreement — the registration the homeserver loads diff --git a/nix/host-modules/glue-queue-agent-credential.nix b/nix/host-modules/glue-queue-agent-credential.nix index 33efb086..d2a48a89 100644 --- a/nix/host-modules/glue-queue-agent-credential.nix +++ b/nix/host-modules/glue-queue-agent-credential.nix @@ -283,11 +283,11 @@ in install -d -m 0755 ${lib.escapeShellArg credentialDir} - printf '%s\n' "$secret" | atomic_write_secret 0600 "" ${lib.escapeShellArg secretFile} + atomic_write_secret 0600 "" ${lib.escapeShellArg secretFile} "$secret" # `0644` on purpose: an OIDC client id is presented to the token # endpoint on every connection and is public by construction. - printf '%s\n' "$client_id" | atomic_write_secret 0644 "" ${lib.escapeShellArg clientIdFile} + atomic_write_secret 0644 "" ${lib.escapeShellArg clientIdFile} "$client_id" ''; }; }) diff --git a/nix/host-modules/lib/atomic-write-secret.nix b/nix/host-modules/lib/atomic-write-secret.nix index cf50e12a..1456775c 100644 --- a/nix/host-modules/lib/atomic-write-secret.nix +++ b/nix/host-modules/lib/atomic-write-secret.nix @@ -7,24 +7,40 @@ # the `chmod`/`chown`/`mv -f` sequence below ever makes the target mode and # owner visible, and only once the content is already final. # +# The value is a function ARGUMENT, not piped in on stdin: a producer that +# exits non-zero partway through a pipe still leaves `cat` a clean EOF, so +# `atomic_write_secret`'s own writer has no way to tell a truncated read from +# an intentionally short one — the partial content would still land at the +# live path, `mv` and all, with only `pipefail` reporting the failure +# afterwards. A caller that computes the value with a command captures it +# into a variable first (`value=$(cmd)`, which fails under `set -e` before +# this function is ever called) and passes the finished value in. +# +# `$4` is never handed to an external program — it's a shell function +# argument, not a process's `argv`/environment, so it never appears in that +# process's own `/proc//cmdline`; the write inside uses `printf`, a +# shell builtin, for the same reason. +# # Pure function — NOT a NixOS module. Call it from a module's `let`: # # atomicWriteSecret = import ./lib/atomic-write-secret.nix { }; # ... # script = '' # ${atomicWriteSecret} -# printf '%s\n' "$secret" | atomic_write_secret 0600 "" "$path" -# printf '%s\n' "$secret" | atomic_write_secret 0400 "grafana:0" "$path" +# atomic_write_secret 0600 "" "$path" "$secret" +# atomic_write_secret 0400 "grafana:0" "$path" "$secret" # ''; # -# Third argument to `atomic_write_secret` is the target path; the second is -# an owner for `chown` (`user:group` or a bare uid), or "" to leave the -# mktemp-created root:root ownership as it is. Reads its content from -# stdin. Requires `coreutils` on the caller's `path`. +# Args: mode, an owner for `chown` (`user:group` or a bare uid, or "" to +# leave the mktemp-created root:root ownership as it is), the target path, +# and the value. The value is written followed by a trailing newline (every +# call site's prior `printf '%s\n' "$value"` behaviour) — a caller building +# a multi-field value (e.g. `KEY=$value`) assembles the whole string first +# and passes that. Requires `coreutils` on the caller's `path`. { }: '' atomic_write_secret() { - local mode="$1" owner="$2" target="$3" tmp + local mode="$1" owner="$2" target="$3" value="$4" tmp tmp="$(mktemp "$(dirname -- "$target")/.$(basename -- "$target").XXXXXX")" # The write happens in a subshell so its own EXIT trap cleans up `$tmp` # on failure (a `RETURN` trap does not fire when `set -e` aborts the @@ -42,7 +58,7 @@ exit "$rc" } trap _atomic_write_secret_cleanup EXIT - cat > "$tmp" + printf '%s\n' "$value" > "$tmp" chmod "$mode" "$tmp" if [ -n "$owner" ]; then chown "$owner" "$tmp" diff --git a/nix/host-modules/swarm-bao.nix b/nix/host-modules/swarm-bao.nix index 05daa859..d44ddce2 100644 --- a/nix/host-modules/swarm-bao.nix +++ b/nix/host-modules/swarm-bao.nix @@ -1841,7 +1841,7 @@ in # an argument in /proc the way `install <<<"$secret"` or an `echo` # from `path` would. install -d -m 0755 ${lib.escapeShellArg forwarderHostSecretDir} - printf '%s\n' "$secret" | atomic_write_secret 0400 root:root ${lib.escapeShellArg forwarderHostSecretPath} + atomic_write_secret 0400 root:root ${lib.escapeShellArg forwarderHostSecretPath} "$secret" ''; }; }) @@ -2647,7 +2647,12 @@ in chgrp ${tokenGroup} ${tokenStoreDir}/tpm2_pkcs11.sqlite3 chmod 0660 ${tokenStoreDir}/tpm2_pkcs11.sqlite3 - printf 'BAO_HSM_PIN=%s\n' "$(cat ${tokenStoreDir}/user-pin)" | atomic_write_secret 0400 "" ${pinEnvFile} + # Captured first, not piped: a `$(cmd)` that fails aborts here + # under `set -e`, before the helper — which cannot see a + # producer's exit code once its output is inside the pipe — + # is ever called. + pin_env_value="BAO_HSM_PIN=$(cat ${tokenStoreDir}/user-pin)" + atomic_write_secret 0400 "" ${pinEnvFile} "$pin_env_value" ''; }; diff --git a/nix/host-modules/swarm-grafana.nix b/nix/host-modules/swarm-grafana.nix index 355e8c6f..026ad54d 100644 --- a/nix/host-modules/swarm-grafana.nix +++ b/nix/host-modules/swarm-grafana.nix @@ -693,7 +693,7 @@ in # grants the group nothing; if this mode ever widens, the gid has to be # discovered at runtime rather than assumed. install -d -m 0755 ${lib.escapeShellArg hostSecretDir} - printf '%s\n' "$secret" | atomic_write_secret 0400 ${lib.escapeShellArg "${toString config.ids.uids.grafana}:0"} ${lib.escapeShellArg hostSecretPath} + atomic_write_secret 0400 ${lib.escapeShellArg "${toString config.ids.uids.grafana}:0"} ${lib.escapeShellArg hostSecretPath} "$secret" ''; }; diff --git a/nix/host-modules/swarm-otel.nix b/nix/host-modules/swarm-otel.nix index 3b1f2e38..9782f3ed 100644 --- a/nix/host-modules/swarm-otel.nix +++ b/nix/host-modules/swarm-otel.nix @@ -866,7 +866,7 @@ in # no uid to give this to — `LoadCredential` reads it as root before # the sandbox exists and re-exposes it to whichever uid the unit got. install -d -m 0755 ${lib.escapeShellArg collectorHostSecretDir} - printf '%s\n' "$secret" | atomic_write_secret 0400 root:root ${lib.escapeShellArg collectorHostSecretPath} + atomic_write_secret 0400 root:root ${lib.escapeShellArg collectorHostSecretPath} "$secret" ''; }; From c6a778f66603f731dbfe4027319fa9c86397a865 Mon Sep 17 00:00:00 2001 From: atlas Date: Sat, 26 Sep 2026 19:19:35 +0200 Subject: [PATCH 5/5] lint: tighten atomic-write-secret.nix's header comment; fix vale contractions in persistence.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The header comment grew to 39 lines across two audit-driven rounds, over the 30-line comment-block-lint max. Trimmed to 30: merged the value-as-argument rationale with its /proc/cmdline justification into one paragraph, cut the usage example to one call instead of two, and condensed the args paragraph — no content dropped, just restatement. persistence.md's first-boot-migration marker paragraph used "it is" and "there is not" — vale's Microsoft.Contractions rule (this repo's config) wants the contracted forms, and "is not" also collides with "is nothing" as a literal substring, which is what actually tripped the error. Reworded to "it's" / "there's nothing", no meaning change. Lint-only: no script logic changed, gates re-run below are all lint checks (no module-eval, no cargo). Refs #4723 --- docs/agent-lifecycle/persistence.md | 4 +-- nix/host-modules/lib/atomic-write-secret.nix | 29 +++++++------------- 2 files changed, 12 insertions(+), 21 deletions(-) diff --git a/docs/agent-lifecycle/persistence.md b/docs/agent-lifecycle/persistence.md index ac536424..3f64dd44 100644 --- a/docs/agent-lifecycle/persistence.md +++ b/docs/agent-lifecycle/persistence.md @@ -535,8 +535,8 @@ container lifetime: 2. **Migrate any leftover `/root/.claude` content into `${homeDir}/.claude`** — legacy `claude` wrote to root's empty home; the bind mount didn't exist yet. Marker - (`/var/lib/hive-agent-user-migrated`) guards single-shot; it is - only written once there is nothing left to migrate, so a `cp` + (`/var/lib/hive-agent-user-migrated`) guards single-shot; it's + only written once there's nothing left to migrate, so a `cp` failure leaves it absent and the next boot retries. `cp -an` (no-clobber) so any pre-existing files at the new location win — never blow over data already there, and so a diff --git a/nix/host-modules/lib/atomic-write-secret.nix b/nix/host-modules/lib/atomic-write-secret.nix index 1456775c..6c6cb5a7 100644 --- a/nix/host-modules/lib/atomic-write-secret.nix +++ b/nix/host-modules/lib/atomic-write-secret.nix @@ -9,34 +9,25 @@ # # The value is a function ARGUMENT, not piped in on stdin: a producer that # exits non-zero partway through a pipe still leaves `cat` a clean EOF, so -# `atomic_write_secret`'s own writer has no way to tell a truncated read from -# an intentionally short one — the partial content would still land at the -# live path, `mv` and all, with only `pipefail` reporting the failure -# afterwards. A caller that computes the value with a command captures it -# into a variable first (`value=$(cmd)`, which fails under `set -e` before -# this function is ever called) and passes the finished value in. -# -# `$4` is never handed to an external program — it's a shell function -# argument, not a process's `argv`/environment, so it never appears in that -# process's own `/proc//cmdline`; the write inside uses `printf`, a -# shell builtin, for the same reason. +# a truncated read is indistinguishable from an intentionally short one, +# and the partial content would still land at the live path via `mv`. A +# caller that computes the value with a command captures it into a +# variable first (`value=$(cmd)`, which fails under `set -e` before this +# function is ever called) and passes the finished value in — never handed +# to an external program, so it never appears in a process's own +# `/proc//cmdline`, and written with `printf`, a shell builtin. # # Pure function — NOT a NixOS module. Call it from a module's `let`: # # atomicWriteSecret = import ./lib/atomic-write-secret.nix { }; -# ... # script = '' # ${atomicWriteSecret} # atomic_write_secret 0600 "" "$path" "$secret" -# atomic_write_secret 0400 "grafana:0" "$path" "$secret" # ''; # -# Args: mode, an owner for `chown` (`user:group` or a bare uid, or "" to -# leave the mktemp-created root:root ownership as it is), the target path, -# and the value. The value is written followed by a trailing newline (every -# call site's prior `printf '%s\n' "$value"` behaviour) — a caller building -# a multi-field value (e.g. `KEY=$value`) assembles the whole string first -# and passes that. Requires `coreutils` on the caller's `path`. +# Args: mode, an owner for `chown` ("" to keep mktemp's root:root), target +# path, value — written with a trailing newline like every call site's +# prior `printf '%s\n'`. Requires `coreutils` on the caller's `path`. { }: '' atomic_write_secret() {