Watch
0
0
Fork
You've already forked hyperhive
0
hyperhive/docs/4-implementation/k8s/images.md

2.7 KiB

Service images

The swarm daemons that run in the k8s cluster ship as OCI images built by nix. nix/images/default.nix holds the one function that builds them, mkServiceImage, and the set of daemons that use it.

What an image holds

One image runs one daemon. Its contents are:

  • the daemon's per-bin package from nix/packages/default.nix,
  • the CA roots from pkgs.cacert, with SSL_CERT_FILE pointing at them,
  • the binary's own closure, which is glibc and libgcc.

The image has no shell and no init. The kubelet restarts a crashed container, so a service manager inside it would only disagree with the kubelet. A missing shell also leaves nothing to exec into. The entrypoint is /bin/<bin>, and the process runs as the serviceUser uid and gid that nix/images/default.nix defines. A pod's securityContext.runAsUser and runAsGroup read that binding too, so the number lives in one place.

mkServiceImage takes name, package, and optionally bin (the binary name, default name), extraContents and env (an attribute set added to the container environment).

The image set

Each of these is a flake package named image-<daemon>, and the image name is hyperhive/<daemon>:

  • swarm-controller
  • swarm-nats-auth
  • swarm-logs-auth
  • swarm-metrics-auth
  • swarm-notify-router
  • swarm-ci-metrics

nix flake check evaluates all six and builds none of them, so the workspace compile stays out of CI. swarm-claude-login, swarm-stt and swarm-review-classifier need claude-code, whisper or similar tools in their image and aren't in the set yet.

Tags

An image has no fixed tag. The tag is the content hash, so a changed binary is a changed reference. A manifest reads the full reference from the package's imageName and imageTag attributes:

nix eval --raw .#image-swarm-logs-auth.imageName
nix eval --raw .#image-swarm-logs-auth.imageTag

Loading an image

nix build produces a .tar.gz that podman load, skopeo and the k3s image directory all read. Use podman, not docker:

podman load -i "$(nix build --no-link --print-out-paths .#image-swarm-logs-auth)"

For a containerd node, stream the file into the node's image store:

k3s ctr images import - < "$(nix build --no-link --print-out-paths .#image-swarm-logs-auth)"

The image-fn-smoke check

checks.<system>.image-fn-smoke builds mkServiceImage around hello and reads the tarball back. It asserts the entrypoint, the user, the CA env var, a layer listing without bash or systemd and a size under 50 MB. The listing must also contain hello and glibc, so an empty listing can't pass the shell arm. The check runs without compiling any hyperhive binary.