nix trusts the swarm's root through its own CA bundle, so root's nix clients reach the cache
This commit is contained in:
atlas
parent
526cfbec1b
commit
9d62ef44b5
2 changed files with 299 additions and 79 deletions
|
|
@ -12,20 +12,18 @@
|
|||
# - the cache's public key and the pull token are read from the store by a root
|
||||
# oneshot, `hive-priv-cache-facts`, under this hive's own store identity. The
|
||||
# key lands in `/run/hyperhive-priv/attic/public-key` (directory `0700`, file
|
||||
# `0600`).
|
||||
# The cache's address, the key and a netrc also land in a nix.conf fragment,
|
||||
# `/run/hyperhive-nix/cache.conf`, as `extra-substituters`,
|
||||
# `extra-trusted-public-keys` and `netrc-file`, which the host's nix.conf
|
||||
# includes (`!include`, so a host whose unit has not run yet still starts).
|
||||
# Every client of the daemon (hive-c0re, agent containers on `nixBuilds =
|
||||
# host`) can have a swarm closure substituted; closures hold no secret;
|
||||
# - that netrc, `/run/hyperhive-nix/netrc` (`root:root 0600`), is the pull
|
||||
# token's entry plus the entries of the host's own netrc, which
|
||||
# `netrc-file` would otherwise replace. The host's netrc is
|
||||
# `nix.settings.netrc-file` when the host sets it, else nix's default
|
||||
# `/etc/nix/netrc`. The unit merges again on every run, so an edit reaches the
|
||||
# daemon within the timer's ten minutes. A netrc named only in
|
||||
# `nix.extraOptions` is not seen.
|
||||
# `0600`). The address, the key, a netrc and a CA bundle also land in a
|
||||
# nix.conf fragment, `/run/hyperhive-nix/cache.conf`, as `extra-substituters`,
|
||||
# `extra-trusted-public-keys`, `netrc-file` and `ssl-cert-file`, which the
|
||||
# host's nix.conf includes (`!include`, so a host whose unit has not run yet
|
||||
# still starts). Every nix client of the host can have a swarm closure
|
||||
# substituted; the fragment holds no secret;
|
||||
# - `netrc-file` and `ssl-cert-file` take one path and would replace the host's
|
||||
# own, so the files they name merge it in: the pull token's entry plus the
|
||||
# host's netrc entries, and the host's CA bundle plus the swarm's root, which
|
||||
# the cache's certificate chains to. The host's are `nix.settings.netrc-file`
|
||||
# else `/etc/nix/netrc`, and `nix.settings.ssl-cert-file` else
|
||||
# `security.pki.caBundle`; one named only in `nix.extraOptions` is not seen.
|
||||
#
|
||||
# hive-c0re keeps its own copies under `/run/hyperhive/attic/` for the probe.
|
||||
{
|
||||
|
|
@ -60,6 +58,13 @@ let
|
|||
netrcFile = "${confDir}/netrc";
|
||||
# `netrc-file` is not additive: the fragment's setting replaces the host's.
|
||||
hostNetrc = toString (config.nix.settings.netrc-file or "/etc/nix/netrc");
|
||||
# `ssl-cert-file` is not additive either. The host's bundle is the one nix
|
||||
# would use without the fragment: `nix.settings.ssl-cert-file` when the host
|
||||
# sets it, else the system bundle the nix daemon's `CURL_CA_BUNDLE` names.
|
||||
caBundleFile = "${confDir}/ca-bundle.crt";
|
||||
baseBundle = toString (config.nix.settings.ssl-cert-file or config.security.pki.caBundle);
|
||||
# Written by `swarm-services-cert` on a host whose gateway is self-signed.
|
||||
swarmCa = "${tls.stateDir}/swarm-services-root.pem";
|
||||
|
||||
# Where the store runs on this host, the first login waits for its container.
|
||||
# No `wants`: a pull would start the container mid-switch on the old closure.
|
||||
|
|
@ -118,6 +123,9 @@ in
|
|||
confFile
|
||||
netrcFile
|
||||
hostNetrc
|
||||
caBundleFile
|
||||
baseBundle
|
||||
swarmCa
|
||||
;
|
||||
url = cacheUrl;
|
||||
};
|
||||
|
|
|
|||
|
|
@ -1,32 +1,32 @@
|
|||
# The root-side copy of the cache facts the host's nix daemon and hive-priv
|
||||
# use: the key a downloaded closure must be signed by and the netrc the pull
|
||||
# token goes out in. The script runs as root on the hive host under the hive's
|
||||
# own store identity and writes:
|
||||
# The root-side copy of the cache facts the host's nix daemon and hive-priv use:
|
||||
# the key a closure must be signed by, the netrc the pull token goes out in and
|
||||
# the CA bundle that trusts the cache. The script runs as root on the hive host
|
||||
# under the hive's own store identity and writes:
|
||||
#
|
||||
# - the key into `dir`, which only root can read;
|
||||
# - the netrc at `netrcFile`, root 0600: the pull token's entry for `host`
|
||||
# followed by the entries of the host's own netrc (`hostNetrc`) for other
|
||||
# machines. `netrc-file` is one path, so a fragment that names this file must
|
||||
# carry what the host already keeps there;
|
||||
# - the netrc at `netrcFile`, root 0600: the pull token's entry for `host` then
|
||||
# the entries of the host's own netrc (`hostNetrc`), as `netrc-file` is one path;
|
||||
# - the CA bundle at `caBundleFile`, root 0644: the host's own bundle
|
||||
# (`baseBundle`) then the swarm's root (`swarmCa`), when the host holds it,
|
||||
# which the cache's certificate chains to; `ssl-cert-file` is one path too;
|
||||
# - a nix.conf fragment at `confFile` that the host's nix.conf includes, with
|
||||
# `extra-substituters` (the cache's address), `extra-trusted-public-keys` and
|
||||
# `netrc-file` (that merged netrc), so the daemon
|
||||
# substitutes from the cache and its other substituters. It holds no secret.
|
||||
# `extra-substituters` (the cache's address), `extra-trusted-public-keys`,
|
||||
# `netrc-file` and `ssl-cert-file`. It holds no secret.
|
||||
#
|
||||
# Nothing is read from a file hive-c0re (unprivileged) writes: the key and the
|
||||
# token come from the store.
|
||||
# token come from the store. A store that holds no facts yet, or cannot be
|
||||
# reached, leaves what is in `dir`, `netrcFile`, `caBundleFile` and `confFile`
|
||||
# as they are. Nothing here removes a file.
|
||||
#
|
||||
# A store that holds no facts yet, or cannot be reached, leaves what is in
|
||||
# `dir`, `netrcFile` and `confFile` as they are. Nothing here removes a file.
|
||||
#
|
||||
# Pure function — NOT a NixOS module. Call it from a unit's `script`, with
|
||||
# `bao`, `coreutils`, `gnugrep` and `systemd` on its `path` and the `BAO_*`
|
||||
# client environment set:
|
||||
# Pure function — NOT a NixOS module. Call it from a unit's `script`, with `bao`,
|
||||
# `coreutils`, `gnugrep`, `systemd` on its `path` and the `BAO_*` client set:
|
||||
#
|
||||
# script = import ./lib/attic-root-facts.nix { inherit lib; } {
|
||||
# hive = "alpha"; host = "attic.example.org"; dir = "/run/hyperhive-priv/attic";
|
||||
# url = "https://attic.example.org/swarm"; confFile = "/run/hyperhive-nix/cache.conf";
|
||||
# netrcFile = "/run/hyperhive-nix/netrc"; hostNetrc = "/etc/nix/netrc";
|
||||
# caBundleFile = "/run/hyperhive-nix/ca-bundle.crt"; baseBundle = "/etc/ssl/certs/ca-certificates.crt";
|
||||
# swarmCa = "/var/lib/hive-tls/swarm-services-root.pem";
|
||||
# };
|
||||
{ lib }:
|
||||
{
|
||||
|
|
@ -45,6 +45,15 @@
|
|||
# module, which is merged in and never written.
|
||||
netrcFile,
|
||||
hostNetrc,
|
||||
# Where the CA bundle goes.
|
||||
caBundleFile,
|
||||
# The CA bundle nix uses without this module, copied in and never written:
|
||||
# `ssl-cert-file` is one path, so a fragment that names the new bundle must
|
||||
# carry the host's CAs.
|
||||
baseBundle,
|
||||
# The swarm's root certificate. The file may be absent: the bundle then holds
|
||||
# the host's CAs alone until a later run finds it.
|
||||
swarmCa,
|
||||
}:
|
||||
''
|
||||
set -euo pipefail
|
||||
|
|
@ -56,8 +65,11 @@
|
|||
conf=${lib.escapeShellArg confFile}
|
||||
netrc_file=${lib.escapeShellArg netrcFile}
|
||||
host_netrc=${lib.escapeShellArg hostNetrc}
|
||||
ca_bundle_file=${lib.escapeShellArg caBundleFile}
|
||||
base_bundle=${lib.escapeShellArg baseBundle}
|
||||
swarm_ca=${lib.escapeShellArg swarmCa}
|
||||
install -d -m 0700 "$dir"
|
||||
install -d -m 0755 "$(dirname -- "$conf")" "$(dirname -- "$netrc_file")"
|
||||
install -d -m 0755 "$(dirname -- "$conf")" "$(dirname -- "$netrc_file")" "$(dirname -- "$ca_bundle_file")"
|
||||
|
||||
store_init
|
||||
store_login
|
||||
|
|
@ -128,14 +140,33 @@
|
|||
atomic_write_secret 0600 "" "$netrc_file" "$netrc"
|
||||
fi
|
||||
|
||||
# Last, so a daemon that reads the fragment finds the netrc in place. The
|
||||
# mtime moves only with the content. Lix re-reads nix.conf for every
|
||||
# connection, CppNix only at start, so a CppNix daemon that started before
|
||||
# this write is restarted.
|
||||
printf -v fragment '%s\n%s\n%s' \
|
||||
# The host's CAs, then the swarm's root. Nix reads the file for each
|
||||
# download, so a root that lands after the fragment reaches the next one
|
||||
# without a restart. A file without a certificate is not appended: the bundle
|
||||
# serves every https fetch of every nix client.
|
||||
if [ ! -r "$base_bundle" ]; then
|
||||
echo "cannot read the host's CA bundle $base_bundle; $ca_bundle_file stays as it is." >&2
|
||||
exit 1
|
||||
fi
|
||||
bundle="$(< "$base_bundle")"
|
||||
if [ -s "$swarm_ca" ] && grep -q -- '-----BEGIN CERTIFICATE-----' "$swarm_ca"; then
|
||||
bundle+=$'\n'"$(< "$swarm_ca")"
|
||||
else
|
||||
echo "the swarm's root $swarm_ca is not on this host yet; $ca_bundle_file holds the host's CAs only."
|
||||
fi
|
||||
if secret_differs "$ca_bundle_file" "$bundle" || [ "$(stat -c %a "$ca_bundle_file")" != 644 ]; then
|
||||
atomic_write_secret 0644 "" "$ca_bundle_file" "$bundle"
|
||||
fi
|
||||
|
||||
# Last, so a daemon that reads the fragment finds the netrc and the bundle in
|
||||
# place. The mtime moves only with the content. Lix re-reads nix.conf for
|
||||
# every connection, CppNix only at start, so a CppNix daemon that started
|
||||
# before this write is restarted.
|
||||
printf -v fragment '%s\n%s\n%s\n%s' \
|
||||
${lib.escapeShellArg "extra-substituters = ${url}"} \
|
||||
"extra-trusted-public-keys = $key" \
|
||||
"netrc-file = $netrc_file"
|
||||
"netrc-file = $netrc_file" \
|
||||
"ssl-cert-file = $ca_bundle_file"
|
||||
if secret_differs "$conf" "$fragment"; then
|
||||
atomic_write_secret 0644 "" "$conf" "$fragment"
|
||||
fi
|
||||
|
|
|
|||
Loading…
Reference in a new issue