diff --git a/CLAUDE.md b/CLAUDE.md index 50fdfc7b..f4eb91cc 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -381,6 +381,8 @@ read them à la carte. [`docs/swarm.md`](docs/swarm.md). - **"How does the rebuild queue work? What are queue kinds and sources?"** → [`docs/coordinator.md`](docs/coordinator.md). +- **"How does the CI runner work? What's the auto-registration flow?"** → + [`docs/ci.md`](docs/ci.md). ## Conventions & process diff --git a/docs/ci.md b/docs/ci.md index 6b92e07d..d2669fe1 100644 --- a/docs/ci.md +++ b/docs/ci.md @@ -1,81 +1,56 @@ -# hive-ci — Forgejo Actions runner +# hive-ci: Forgejo Actions Runner -The `hive-ci` module adds a Forgejo Actions runner to the hive so -workflows in the hyperhive repo (and any operator-managed repos on -the hive forge) can run on the same host. Disabled by default. +The `hive-ci` module runs a Forgejo Actions runner in a `hive-ci` nixos-container, executing CI jobs from `.forgejo/workflows/ci.yml` (e.g., `nix flake check` on every PR). -## Enabling +## Operator bootstrap -```nix -services.hyperhive.forge.ci = { - enable = true; - name = "prod-hive"; # shows in Forgejo admin panel; default = hostname -}; -``` +Set `services.hyperhive.ci.enable = true` in the host NixOS config. That's it — no manual token provisioning. -Requires `services.hyperhive.forge.enable = true` (the runner -registers against the hive-forge Forgejo instance). The module -asserts this at eval time. +**Requirements:** +- `services.hyperhive.forge.enable = true` must also be set (the runner registers against hive-forge). +- Optional: tune `services.hyperhive.ci.name` (runner name in forge admin panel), `concurrency` (parallel job capacity), `labels` (workflow targeting). -## Options +## Container design -| option | default | description | -|---|---|---| -| `forge.ci.enable` | `false` | opt-in; off by default | -| `forge.ci.name` | hostname | runner name in Forgejo admin panel | -| `forge.ci.concurrency` | `2` | maximum parallel jobs | -| `forge.ci.labels` | `["hive-ci:host"]` | runner labels (`:`) | -| `forge.ci.package` | `pkgs.gitea-actions-runner` | override the runner binary | +- **Shared host netns**: container reaches hive-forge at `http://127.0.0.1:` (same as hive-gateway). +- **Non-ephemeral**: runner credentials persist across restarts (written to container's stateDir on first registration, reused thereafter). +- **Sandbox fallback**: nspawn containers can't create user-namespaces, so nix's sandboxing would always fail. Module sets `nix.settings.sandbox-fallback = true` in the container — nix builds run unsandboxed (safe because the container is already isolated). See `docs/gotchas.md`. -The `host` scheme in labels means jobs run directly inside the -`hive-ci` container (no docker/podman). Workflow files target this -runner with: +## Auto-registration flow + +1. On first container boot, `systemd` runs the `ExecStartPre` script (from `hive-ci.nix:autoRegisterScript`). +2. Script checks for existing runner credentials (`/var/lib/gitea-runner/hive/.runner`): + - If found: container was previously registered — write dummy token to satisfy module's path check, exit. + - If not found: first boot — proceed to registration. +3. Script reads hive-c0re's admin token from `/run/hive-ci/core-token` (bind-mounted from `/var/lib/hyperhive/forge-core-token`). +4. Calls `POST /api/v1/admin/runners/registration-token` on the forge using `${pkgs.curl}/bin/curl` and `${pkgs.jq}/bin/jq` (absolute nix store paths — the container doesn't need these in systemPackages). Retries for 30s in case forge is still starting. +5. Writes real token to `/run/hive-ci/runner-token`. +6. `gitea-actions-runner` reads the token and registers itself, persisting credentials (`.runner` file) to stateDir. +7. On subsequent boots, runner reuses the `.runner` credentials — the preStart script detects the existing `.runner` file and writes a dummy token instead. The runner ignores the token file when `.runner` already exists. + +## CI workflow + +The single CI job is defined in `.forgejo/workflows/ci.yml`: ```yaml -runs-on: [hive-ci] +name: CI +on: + pull_request: + branches: ["**"] +jobs: + check: + name: nix flake check + runs-on: [hive-ci] + steps: + - uses: actions/checkout@v3 + - name: check + run: nix flake check ``` -## Container shape +This runs on every PR, executing all flake checks (treefmt, rustfmt, cargo test, cargo clippy, module evaluation). No `--no-build`: the checks' derivations are the canonical source of truth. -`hive-ci` is an ephemeral-false nixos-container sharing the host -network namespace (`privateNetwork = false`) so it reaches the -hive-forge Forgejo instance at `localhost:` without -needing a routed address. +## References -Bind mounts: -- `/run/hive-ci/core-token` ← `coreTokenPath` (host's forge - admin token, read-only). Used only on first boot for registration. - -The container has `git` in `systemPackages`; everything else a -workflow needs (rust tools, nix, etc.) is either pulled in by the -workflow itself via `nix flake check` or is part of the base NixOS -closure. - -Nix sandboxing is disabled in the container (`sandbox-fallback = -true`) because nspawn containers can't create the user namespaces -that the Nix sandbox requires. See `docs/gotchas.md` for the -general nspawn sandbox note. - -## Auto-registration - -On first boot an `ExecStartPre` script (`hive-ci-autoregister`) runs -before the runner service: - -1. Reads the hive-c0re admin token from `/run/hive-ci/core-token`. -2. Calls `GET /api/v1/admin/runners/registration-token` on the forge - (retrying for up to 30 s while forge starts). -3. Writes the registration token to `/run/hive-ci/runner-token`. -4. The runner service picks it up and registers, writing credentials - to `/var/lib/gitea-runner/hive/.runner`. - -On subsequent boots the `.runner` credentials file already exists; -the script writes a dummy token and exits — the runner ignores it -and reuses its stored credentials. - -No manual token handling is needed. The whole flow is automatic on -`nixos-container start`. - -## Cross-references - -- `docs/forge.md` — hive-forge setup (prerequisite) -- `docs/gotchas.md` — nspawn sandbox-fallback note +- `nix/modules/hive-ci.nix`: runner configuration, auto-registration script, container setup. +- `.forgejo/workflows/ci.yml`: workflow definition. +- `docs/gotchas.md`: nix sandboxing limitations in containers.