swarm-ci-runner: load the registration token in a register unit of its own
systemd hands a unit's credentials to every process of that unit, and a host job is a process of the runner daemon, so the `LoadCredential=` on gitea-runner-swarm still let any workflow read the registration token from `$CREDENTIALS_DIRECTORY`. swarm-ci-runner-register.service, a oneshot under the daemon's dynamic user and state dir, now holds the only `LoadCredential=` and runs the precond and `act_runner register` (only when `.runner` is missing). The daemon unit has no credential, no `EnvironmentFile` and no `TOKEN`; its pre-start step starts the register unit, so a missing token fails the daemon's start and `Restart=on-failure` retries it. The precond compares `.runner`'s labels in the form the daemon rewrites them to (`nix` -> `nix:host`); compared verbatim, a scheme-less label re-registered the runner on every start. tmpfiles removes the unused `registration-token.env` and upstream's `.token-hash`/`.labels` markers.
This commit is contained in:
parent
0302b29ca4
commit
8ffceea8e3
4 changed files with 184 additions and 100 deletions
|
|
@ -130,7 +130,7 @@ can't hold the runner's single slot indefinitely).
|
|||
- **Private netns, bridge-attached**: the container runs in its own network namespace (`privateNetwork = true`, `hostBridge`), takes its address from the gateway's DHCP, and resolves through the gateway's dnsmasq on the bridge IP. It can't reach host-loopback services — the core dashboard at `127.0.0.1:7000` and the raw forge port are unreachable from CI.
|
||||
- **Non-ephemeral**: runner credentials (`.runner`) persist across restarts, written to the container's state dir on registration and reused after.
|
||||
- **Sandbox fallback**: nspawn containers can't create user-namespaces, so nix's sandboxing would always fail. Module sets `nix.settings.sandbox-fallback = true` in the container — nix builds run unsandboxed (safe because the container is already isolated). See `docs/process/gotchas.md`.
|
||||
- **Credential isolation**: no forge admin credential enters the container. swarm-controller holds the forge admin token; the runner holds only the registration token and its own `.runner`, and the host unit that fetches the token holds a store identity that reads that one path. Only the register step reads the registration token; the daemon and the jobs it runs never have it in their environment.
|
||||
- **Credential isolation**: no forge admin credential enters the container. swarm-controller holds the forge admin token; the runner holds only the registration token and its own `.runner`, and the host unit that fetches the token holds a store identity that reads that one path. Only `swarm-ci-runner-register.service` loads the registration token; the daemon unit and the jobs it runs have it neither in their environment nor among their credentials.
|
||||
|
||||
## Registration
|
||||
|
||||
|
|
@ -146,13 +146,18 @@ can't hold the runner's single slot indefinitely).
|
|||
writes the token to a root-only file inside the container, and restarts
|
||||
the runner when the value changes. It runs at boot and on deploy, not on a
|
||||
timer.
|
||||
3. **The runner unit** (`gitea-runner-swarm` in the container) loads that
|
||||
file with `LoadCredential=`, not as an environment variable: for a `host`
|
||||
label, act_runner hands its whole environment to every job step. A
|
||||
register step before the daemon reads the token from
|
||||
`$CREDENTIALS_DIRECTORY` and runs `act_runner register` only when `.runner`
|
||||
is missing; with a `.runner` in place the daemon starts without the token.
|
||||
A pre-start step ahead of it removes `.runner` in three cases:
|
||||
3. **`swarm-ci-runner-register.service`** in the container is the only unit
|
||||
that loads that file, with `LoadCredential=`. systemd hands a unit's
|
||||
credentials to all of its processes, and for a `host` label act_runner
|
||||
runs every job step as a process of the daemon with the daemon's whole
|
||||
environment, so the daemon unit carries the token in neither. The
|
||||
register unit is a oneshot under the daemon's dynamic user and state
|
||||
directory. It reads the token from `$CREDENTIALS_DIRECTORY` and runs
|
||||
`act_runner register` only when `.runner` is missing, so the `.runner` it
|
||||
writes is the daemon's own. **The runner unit** (`gitea-runner-swarm`)
|
||||
starts it in a pre-start step and runs the daemon once it succeeds.
|
||||
A root step of the register unit, ahead of registering, removes `.runner`
|
||||
in three cases:
|
||||
- The forge has no record of the runner. Forgejo answers every call from an
|
||||
unknown runner with `unregistered runner`, and the daemon exits 1 when its
|
||||
startup `Declare` gets that answer. On the next start the step finds the
|
||||
|
|
@ -161,11 +166,14 @@ can't hold the runner's single slot indefinitely).
|
|||
and logging the error without exiting; the next restart of the unit or
|
||||
container registers it again.
|
||||
- `.runner` names another forge address than the configured one.
|
||||
- `.runner` names other labels than the configured ones.
|
||||
- `.runner` names other labels than the configured ones, compared in the
|
||||
form the daemon writes back: `<name>:<scheme>`, with `host` for a
|
||||
label that names no scheme.
|
||||
|
||||
A changed token alone doesn't register again: the daemon authenticates with
|
||||
the per-runner token in `.runner`. Without `.runner` and with no token yet
|
||||
the register step fails, and `Restart=on-failure` retries every 15 seconds
|
||||
the register unit fails and the runner unit's pre-start step with it;
|
||||
`Restart=on-failure` on the runner unit starts both again every 15 seconds,
|
||||
with no start limit.
|
||||
|
||||
Host jobs run as the runner's own user, so they can still read `.runner`
|
||||
|
|
@ -231,7 +239,7 @@ A malicious `default.nix` or build script in a PR can therefore:
|
|||
- **Make arbitrary network requests** to any address reachable from the container. The container runs in its own netns behind the hive bridge, so it reaches the forge by name at `https://<forge.domain>` (no admin credentials) and **can't** reach host-loopback services: the unauthenticated core dashboard at `127.0.0.1:7000` and the raw forge port are off-limits (a firewall rule drops bridge→127.0.0.0/8 traffic).
|
||||
- **Write to the container filesystem**, including corrupting the runner's state dir or `.runner` credentials.
|
||||
|
||||
No forge admin token enters the container: swarm-controller holds it and publishes only the registration token, which only the runner's register step reads, so a job step doesn't inherit it. A build process can still reach the forge over the network, and it runs as the runner's own user, so it can read `.runner` and act as this runner; it can't issue admin API calls.
|
||||
No forge admin token enters the container: swarm-controller holds it and publishes only the registration token, which only `swarm-ci-runner-register.service` loads, so a job step can't read it. A build process can still reach the forge over the network, and it runs as the runner's own user, so it can read `.runner` and act as this runner; it can't issue admin API calls.
|
||||
|
||||
Note: `nix flake check --no-build` (eval-only) reduces the attack surface but doesn't eliminate it — `builtins.fetchGit`, `builtins.fetchurl`, and import-from-derivation can reach the network and filesystem during evaluation. The default CI workflow runs full `nix flake check` (builds derivations), which is the higher-risk path.
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue