swarm-ci-runner: give the registration token only to the register step
The runner got its registration token through an `EnvironmentFile` (`TOKEN=…`), so the daemon carried it for its whole life, and for a `host` label act_runner copies its environment into every job step: any workflow could read a token that registers runners on the forge. The host unit now writes the bare token, and the runner unit loads it with `LoadCredential=`. The upstream pre-start step, which registered from `$TOKEN`, is replaced by one register step that reads the token from `$CREDENTIALS_DIRECTORY` and runs `act_runner register` only when `.runner` is missing, with the instance URL, name, labels and config file upstream used. The precond now also removes `.runner` when its labels differ from the configured ones, which upstream's step covered. A changed token alone no longer re-registers. Host jobs can still read `.runner` itself (#4979). Closes #4974
This commit is contained in:
parent
84347f122c
commit
0302b29ca4
4 changed files with 158 additions and 71 deletions
|
|
@ -130,7 +130,7 @@ can't hold the runner's single slot indefinitely).
|
|||
- **Private netns, bridge-attached**: the container runs in its own network namespace (`privateNetwork = true`, `hostBridge`), takes its address from the gateway's DHCP, and resolves through the gateway's dnsmasq on the bridge IP. It can't reach host-loopback services — the core dashboard at `127.0.0.1:7000` and the raw forge port are unreachable from CI.
|
||||
- **Non-ephemeral**: runner credentials (`.runner`) persist across restarts, written to the container's state dir on registration and reused after.
|
||||
- **Sandbox fallback**: nspawn containers can't create user-namespaces, so nix's sandboxing would always fail. Module sets `nix.settings.sandbox-fallback = true` in the container — nix builds run unsandboxed (safe because the container is already isolated). See `docs/process/gotchas.md`.
|
||||
- **Credential isolation**: no forge admin credential enters the container. swarm-controller holds the forge admin token; the runner holds only the registration token and its own `.runner`, and the host unit that fetches the token holds a store identity that reads that one path.
|
||||
- **Credential isolation**: no forge admin credential enters the container. swarm-controller holds the forge admin token; the runner holds only the registration token and its own `.runner`, and the host unit that fetches the token holds a store identity that reads that one path. Only the register step reads the registration token; the daemon and the jobs it runs never have it in their environment.
|
||||
|
||||
## Registration
|
||||
|
||||
|
|
@ -143,13 +143,16 @@ can't hold the runner's single slot indefinitely).
|
|||
2. **`swarm-bao-ci-runner.service`** on the runner's host logs in to the store
|
||||
with the `ci-runner` leaf, whose `swarm-ci-runner` role reads that one path
|
||||
and nothing else, retrying every 30 seconds until the token is there. It
|
||||
writes `TOKEN=<value>` to the runner's `EnvironmentFile` inside the
|
||||
container, and restarts the runner when the value changes. It runs at boot
|
||||
and on deploy, not on a timer.
|
||||
3. **The runner unit** (`gitea-runner-swarm` in the container) registers in
|
||||
the upstream pre-start step whenever `.runner` is missing or the token or
|
||||
labels changed. A pre-start step of this module's, ahead of it, removes
|
||||
`.runner` in two cases upstream doesn't check:
|
||||
writes the token to a root-only file inside the container, and restarts
|
||||
the runner when the value changes. It runs at boot and on deploy, not on a
|
||||
timer.
|
||||
3. **The runner unit** (`gitea-runner-swarm` in the container) loads that
|
||||
file with `LoadCredential=`, not as an environment variable: for a `host`
|
||||
label, act_runner hands its whole environment to every job step. A
|
||||
register step before the daemon reads the token from
|
||||
`$CREDENTIALS_DIRECTORY` and runs `act_runner register` only when `.runner`
|
||||
is missing; with a `.runner` in place the daemon starts without the token.
|
||||
A pre-start step ahead of it removes `.runner` in three cases:
|
||||
- The forge has no record of the runner. Forgejo answers every call from an
|
||||
unknown runner with `unregistered runner`, and the daemon exits 1 when its
|
||||
startup `Declare` gets that answer. On the next start the step finds the
|
||||
|
|
@ -158,9 +161,15 @@ can't hold the runner's single slot indefinitely).
|
|||
and logging the error without exiting; the next restart of the unit or
|
||||
container registers it again.
|
||||
- `.runner` names another forge address than the configured one.
|
||||
- `.runner` names other labels than the configured ones.
|
||||
|
||||
Until a token arrives the step fails, and `Restart=on-failure` retries every
|
||||
15 seconds with no start limit.
|
||||
A changed token alone doesn't register again: the daemon authenticates with
|
||||
the per-runner token in `.runner`. Without `.runner` and with no token yet
|
||||
the register step fails, and `Restart=on-failure` retries every 15 seconds
|
||||
with no start limit.
|
||||
|
||||
Host jobs run as the runner's own user, so they can still read `.runner`
|
||||
and act as this runner; see the security section below.
|
||||
|
||||
A new registration is a new runner on the forge: the previous entry stays in
|
||||
the forge admin panel as offline until an admin deletes it.
|
||||
|
|
@ -222,7 +231,7 @@ A malicious `default.nix` or build script in a PR can therefore:
|
|||
- **Make arbitrary network requests** to any address reachable from the container. The container runs in its own netns behind the hive bridge, so it reaches the forge by name at `https://<forge.domain>` (no admin credentials) and **can't** reach host-loopback services: the unauthenticated core dashboard at `127.0.0.1:7000` and the raw forge port are off-limits (a firewall rule drops bridge→127.0.0.0/8 traffic).
|
||||
- **Write to the container filesystem**, including corrupting the runner's state dir or `.runner` credentials.
|
||||
|
||||
No forge admin token enters the container: swarm-controller holds it and publishes only the registration token, which reaches the container's env-file. A build process can still reach the forge over the network, and it runs as the runner's own user, so it can read `.runner` and act as this runner; it can't issue admin API calls.
|
||||
No forge admin token enters the container: swarm-controller holds it and publishes only the registration token, which only the runner's register step reads, so a job step doesn't inherit it. A build process can still reach the forge over the network, and it runs as the runner's own user, so it can read `.runner` and act as this runner; it can't issue admin API calls.
|
||||
|
||||
Note: `nix flake check --no-build` (eval-only) reduces the attack surface but doesn't eliminate it — `builtins.fetchGit`, `builtins.fetchurl`, and import-from-derivation can reach the network and filesystem during evaluation. The default CI workflow runs full `nix flake check` (builds derivations), which is the higher-risk path.
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue