Watch
0
0
Fork
You've already forked hyperhive
0

swarm-ci-runner: load the registration token in a register unit of its own

systemd hands a unit's credentials to every process of that unit, and a
host job is a process of the runner daemon, so the `LoadCredential=` on
gitea-runner-swarm still let any workflow read the registration token
from `$CREDENTIALS_DIRECTORY`.

swarm-ci-runner-register.service, a oneshot under the daemon's dynamic
user and state dir, now holds the only `LoadCredential=` and runs the
precond and `act_runner register` (only when `.runner` is missing). The
daemon unit has no credential, no `EnvironmentFile` and no `TOKEN`; its
pre-start step starts the register unit, so a missing token fails the
daemon's start and `Restart=on-failure` retries it.

The precond compares `.runner`'s labels in the form the daemon rewrites
them to (`nix` -> `nix:host`); compared verbatim, a scheme-less label
re-registered the runner on every start.

tmpfiles removes the unused `registration-token.env` and upstream's
`.token-hash`/`.labels` markers.
This commit is contained in:
atlas 2026-10-05 16:33:24 +02:00
commit 8ffceea8e3
4 changed files with 184 additions and 100 deletions

View file

@ -130,7 +130,7 @@ can't hold the runner's single slot indefinitely).
- **Private netns, bridge-attached**: the container runs in its own network namespace (`privateNetwork = true`, `hostBridge`), takes its address from the gateway's DHCP, and resolves through the gateway's dnsmasq on the bridge IP. It can't reach host-loopback services — the core dashboard at `127.0.0.1:7000` and the raw forge port are unreachable from CI.
- **Non-ephemeral**: runner credentials (`.runner`) persist across restarts, written to the container's state dir on registration and reused after.
- **Sandbox fallback**: nspawn containers can't create user-namespaces, so nix's sandboxing would always fail. Module sets `nix.settings.sandbox-fallback = true` in the container — nix builds run unsandboxed (safe because the container is already isolated). See `docs/process/gotchas.md`.
- **Credential isolation**: no forge admin credential enters the container. swarm-controller holds the forge admin token; the runner holds only the registration token and its own `.runner`, and the host unit that fetches the token holds a store identity that reads that one path. Only the register step reads the registration token; the daemon and the jobs it runs never have it in their environment.
- **Credential isolation**: no forge admin credential enters the container. swarm-controller holds the forge admin token; the runner holds only the registration token and its own `.runner`, and the host unit that fetches the token holds a store identity that reads that one path. Only `swarm-ci-runner-register.service` loads the registration token; the daemon unit and the jobs it runs have it neither in their environment nor among their credentials.
## Registration
@ -146,13 +146,18 @@ can't hold the runner's single slot indefinitely).
writes the token to a root-only file inside the container, and restarts
the runner when the value changes. It runs at boot and on deploy, not on a
timer.
3. **The runner unit** (`gitea-runner-swarm` in the container) loads that
file with `LoadCredential=`, not as an environment variable: for a `host`
label, act_runner hands its whole environment to every job step. A
register step before the daemon reads the token from
`$CREDENTIALS_DIRECTORY` and runs `act_runner register` only when `.runner`
is missing; with a `.runner` in place the daemon starts without the token.
A pre-start step ahead of it removes `.runner` in three cases:
3. **`swarm-ci-runner-register.service`** in the container is the only unit
that loads that file, with `LoadCredential=`. systemd hands a unit's
credentials to all of its processes, and for a `host` label act_runner
runs every job step as a process of the daemon with the daemon's whole
environment, so the daemon unit carries the token in neither. The
register unit is a oneshot under the daemon's dynamic user and state
directory. It reads the token from `$CREDENTIALS_DIRECTORY` and runs
`act_runner register` only when `.runner` is missing, so the `.runner` it
writes is the daemon's own. **The runner unit** (`gitea-runner-swarm`)
starts it in a pre-start step and runs the daemon once it succeeds.
A root step of the register unit, ahead of registering, removes `.runner`
in three cases:
- The forge has no record of the runner. Forgejo answers every call from an
unknown runner with `unregistered runner`, and the daemon exits 1 when its
startup `Declare` gets that answer. On the next start the step finds the
@ -161,11 +166,14 @@ can't hold the runner's single slot indefinitely).
and logging the error without exiting; the next restart of the unit or
container registers it again.
- `.runner` names another forge address than the configured one.
- `.runner` names other labels than the configured ones.
- `.runner` names other labels than the configured ones, compared in the
form the daemon writes back: `<name>:<scheme>`, with `host` for a
label that names no scheme.
A changed token alone doesn't register again: the daemon authenticates with
the per-runner token in `.runner`. Without `.runner` and with no token yet
the register step fails, and `Restart=on-failure` retries every 15 seconds
the register unit fails and the runner unit's pre-start step with it;
`Restart=on-failure` on the runner unit starts both again every 15 seconds,
with no start limit.
Host jobs run as the runner's own user, so they can still read `.runner`
@ -231,7 +239,7 @@ A malicious `default.nix` or build script in a PR can therefore:
- **Make arbitrary network requests** to any address reachable from the container. The container runs in its own netns behind the hive bridge, so it reaches the forge by name at `https://<forge.domain>` (no admin credentials) and **can't** reach host-loopback services: the unauthenticated core dashboard at `127.0.0.1:7000` and the raw forge port are off-limits (a firewall rule drops bridge→127.0.0.0/8 traffic).
- **Write to the container filesystem**, including corrupting the runner's state dir or `.runner` credentials.
No forge admin token enters the container: swarm-controller holds it and publishes only the registration token, which only the runner's register step reads, so a job step doesn't inherit it. A build process can still reach the forge over the network, and it runs as the runner's own user, so it can read `.runner` and act as this runner; it can't issue admin API calls.
No forge admin token enters the container: swarm-controller holds it and publishes only the registration token, which only `swarm-ci-runner-register.service` loads, so a job step can't read it. A build process can still reach the forge over the network, and it runs as the runner's own user, so it can read `.runner` and act as this runner; it can't issue admin API calls.
Note: `nix flake check --no-build` (eval-only) reduces the attack surface but doesn't eliminate it — `builtins.fetchGit`, `builtins.fetchurl`, and import-from-derivation can reach the network and filesystem during evaluation. The default CI workflow runs full `nix flake check` (builds derivations), which is the higher-risk path.