2.7 KiB
Service images
The swarm daemons that run in the k8s cluster ship as OCI images built by nix.
nix/images/default.nix holds the one function that builds them,
mkServiceImage, and the set of daemons that use it.
What an image holds
One image runs one daemon. Its contents are:
- the daemon's per-bin package from
nix/packages/default.nix, - the CA roots from
pkgs.cacert, withSSL_CERT_FILEpointing at them, - the binary's own closure, which is glibc and libgcc.
The image has no shell and no init. The kubelet restarts a crashed container, so a
service manager inside it would only disagree with the kubelet. A missing
shell also leaves nothing to exec into. The entrypoint is /bin/<bin>, and
the process runs as the serviceUser uid and gid that nix/images/default.nix
defines. A pod's securityContext.runAsUser and runAsGroup read that binding
too, so the number lives in one place.
mkServiceImage takes name, package, and optionally bin (the binary
name, default name), extraContents and env (an attribute set added to
the container environment).
The image set
Each of these is a flake package named image-<daemon>, and the image name is
hyperhive/<daemon>:
swarm-controllerswarm-nats-authswarm-logs-authswarm-metrics-authswarm-notify-routerswarm-ci-metrics
nix flake check evaluates all six and builds none of them, so the workspace
compile stays out of CI. swarm-claude-login, swarm-stt and
swarm-review-classifier need claude-code, whisper or similar tools in their
image and aren't in the set yet.
Tags
An image has no fixed tag. The tag is the content hash, so a changed binary is
a changed reference. A manifest reads the full reference from the package's
imageName and imageTag attributes:
nix eval --raw .#image-swarm-logs-auth.imageName
nix eval --raw .#image-swarm-logs-auth.imageTag
Loading an image
nix build produces a .tar.gz that podman load, skopeo and the k3s image
directory all read. Use podman, not docker:
podman load -i "$(nix build --no-link --print-out-paths .#image-swarm-logs-auth)"
For a containerd node, stream the file into the node's image store:
k3s ctr images import - < "$(nix build --no-link --print-out-paths .#image-swarm-logs-auth)"
The image-fn-smoke check
checks.<system>.image-fn-smoke builds mkServiceImage around hello and
reads the tarball back. It asserts the entrypoint, the user, the CA env var, a
layer listing without bash or systemd and a size under 50 MB. The listing
must also contain hello and glibc, so an empty listing can't pass the
shell arm. The check runs without compiling any hyperhive binary.