push green meta builds to the attic cache and record cached
This commit is contained in:
parent
c4e8d5b540
commit
0849afec80
16 changed files with 779 additions and 31 deletions
|
|
@ -103,9 +103,10 @@ status has no link.
|
||||||
hive's. A green check can therefore still fail on a hive, and the reverse.
|
hive's. A green check can therefore still fail on a hive, and the reverse.
|
||||||
- The agent base is the builder's host's, not the hive's. Hives on another
|
- The agent base is the builder's host's, not the hive's. Hives on another
|
||||||
hyperhive or nixpkgs pin build something else.
|
hyperhive or nixpkgs pin build something else.
|
||||||
- The builder pushes nothing. The closure stays in the builder
|
- The check pushes nothing. The closure stays in the builder
|
||||||
host's store until it's collected, so on a host that also runs the hive, the
|
host's store until it's collected, so on a host that also runs the hive, the
|
||||||
hive's own build can reuse whatever matches.
|
hive's own build can reuse whatever matches. A [meta build](#deploy-gate)
|
||||||
|
can push to [a binary cache](#binary-cache).
|
||||||
- The PR check gates nothing. A merge deploys whatever the status says. The
|
- The PR check gates nothing. A merge deploys whatever the status says. The
|
||||||
[deploy gate](#deploy-gate) is a separate build, from `internal/meta`.
|
[deploy gate](#deploy-gate) is a separate build, from `internal/meta`.
|
||||||
- A lost webhook delivery isn't retried: the head isn't built until the next
|
- A lost webhook delivery isn't retried: the head isn't built until the next
|
||||||
|
|
@ -247,13 +248,48 @@ On a built reply the controller writes the agent's key in the `agent-closures`
|
||||||
KV bucket: the `internal/meta` commit, the closure's store path, the builder's
|
KV bucket: the `internal/meta` commit, the closure's store path, the builder's
|
||||||
platform and the time. Only the controller writes the bucket, and the builder
|
platform and the time. Only the controller writes the bucket, and the builder
|
||||||
holds no grant on it. A failed build writes nothing and leaves the previous key.
|
holds no grant on it. A failed build writes nothing and leaves the previous key.
|
||||||
Nothing reads the bucket yet. The closure stays unrooted in the builder host's
|
The record's `cached` field says whether the builder pushed the closure to
|
||||||
store.
|
[the binary cache](#binary-cache). Nothing reads the bucket yet. The closure stays
|
||||||
|
unrooted in the builder host's store.
|
||||||
|
|
||||||
The controller gives the `builder` forge account read on `internal/meta` when it
|
The controller gives the `builder` forge account read on `internal/meta` when it
|
||||||
updates the flake, and again before each build, since the repo is private. A
|
updates the flake, and again before each build, since the repo is private. A
|
||||||
grant the forge refuses fails the build with no verdict.
|
grant the forge refuses fails the build with no verdict.
|
||||||
|
|
||||||
|
## Binary cache
|
||||||
|
|
||||||
|
With `services.hyperhive.swarm.attic.domain` and `cache` set, the builder pushes
|
||||||
|
each green meta build's closure to that cache, an attic server outside the swarm,
|
||||||
|
before it replies. It runs `attic push swarm:<cache> <out path>` once per build,
|
||||||
|
not `attic watch`, and the builder pushes again when the controller runs a build again. The
|
||||||
|
push has ten minutes, or less when the reply is due sooner, because it must end
|
||||||
|
inside the 61 minutes the controller waits. `attic` checks the server's certificate
|
||||||
|
against the system roots and pins nothing, so a Let's Encrypt certificate
|
||||||
|
works.
|
||||||
|
|
||||||
|
The push token is the operator's to write. The builder reads it on every push
|
||||||
|
from `secret/swarm/services/attic/push-token` (field `value`), writes an `attic`
|
||||||
|
config file holding it in the unit's private runtime directory (`/run/swarm-builder`,
|
||||||
|
mode `0700`, the file `0600`), points `attic` at it with `XDG_CONFIG_HOME` and
|
||||||
|
removes it after the push. The token is in no argument or variable, so no
|
||||||
|
process listing shows it.
|
||||||
|
|
||||||
|
The reply's `cached` says whether the push finished. A push that fails, times out
|
||||||
|
or can't read the token leaves the build green with `cached` false and logs a
|
||||||
|
warning with the reason; the deploy goes ahead and the next build of the agent
|
||||||
|
tries again. With no cache set, the builder pushes nothing and `cached` is always
|
||||||
|
false.
|
||||||
|
|
||||||
|
Mixed versions:
|
||||||
|
|
||||||
|
| controller | builder | what happens |
|
||||||
|
| ---------- | ------- | ------------------------------------------------------------- |
|
||||||
|
| new | new | `cached` in the reply and in the `agent-closures` record |
|
||||||
|
| new | old | the reply has no `cached`, which reads as `false` |
|
||||||
|
| old | new | the controller ignores `cached`; its record has no such field |
|
||||||
|
|
||||||
|
A record written before the field existed also reads as `false`.
|
||||||
|
|
||||||
## Credentials
|
## Credentials
|
||||||
|
|
||||||
| What | Where | Who reads it |
|
| What | Where | Who reads it |
|
||||||
|
|
@ -261,6 +297,7 @@ grant the forge refuses fails the build with no verdict.
|
||||||
| store leaf | `deploy.bao.builderClient{Cert,Key}File` | nspawn, into the service as a credential |
|
| store leaf | `deploy.bao.builderClient{Cert,Key}File` | nspawn, into the service as a credential |
|
||||||
| queue client secret | `secret/swarm/services/swarm-builder/oidc/client` | the builder, at start and when authelia refuses it |
|
| queue client secret | `secret/swarm/services/swarm-builder/oidc/client` | the builder, at start and when authelia refuses it |
|
||||||
| forge token (read-only) | `secret/swarm/services/builder/forge/token` | the builder, on every build |
|
| forge token (read-only) | `secret/swarm/services/builder/forge/token` | the builder, on every build |
|
||||||
|
| cache push token | `secret/swarm/services/attic/push-token` | the builder, on every push |
|
||||||
|
|
||||||
The token belongs to the `builder` forge account, which swarm-controller
|
The token belongs to the `builder` forge account, which swarm-controller
|
||||||
creates, puts in the `hives` team of `agent-configs` (read on every config
|
creates, puts in the `hives` team of `agent-configs` (read on every config
|
||||||
|
|
@ -270,7 +307,7 @@ gets the token through a credential helper reading the child's environment:
|
||||||
it's in no argument, URL or lock, so no line `nix` prints can carry it into a
|
it's in no argument, URL or lock, so no line `nix` prints can carry it into a
|
||||||
status. The same `git` fetches a config's own inputs, so the builder registers
|
status. The same `git` fetches a config's own inputs, so the builder registers
|
||||||
the helper for the forge's URL only, and the helper answers only an `https`
|
the helper for the forge's URL only, and the helper answers only an `https`
|
||||||
request for the forge's host: any other host gets nothing. The builder holds no write credential; swarm-controller writes the
|
request for the forge's host: any other host gets nothing. The builder holds no forge write credential; swarm-controller writes the
|
||||||
status with its own forge token. Details per path:
|
status with its own forge token. Details per path:
|
||||||
[`credentials-reference.md`](credentials-reference.md).
|
[`credentials-reference.md`](credentials-reference.md).
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -41,6 +41,7 @@ the rest of the cell says how.
|
||||||
| `swarm/services/review-classifier/acp-provider` | you, in the store's UI, field `api_key` | `swarm-review-classifier` | ❌ write a new key | ✅ read on every run |
|
| `swarm/services/review-classifier/acp-provider` | you, in the store's UI, field `api_key` | `swarm-review-classifier` | ❌ write a new key | ✅ read on every run |
|
||||||
| `swarm/services/review-classifier/forge/token` | `swarm-controller`, at start and every 5 minutes | `swarm-controller` | ✅ when missing or no longer matching the forge | ✅ read on every review |
|
| `swarm/services/review-classifier/forge/token` | `swarm-controller`, at start and every 5 minutes | `swarm-controller` | ✅ when missing or no longer matching the forge | ✅ read on every review |
|
||||||
| `swarm/services/builder/forge/token` | `swarm-controller`, at start and every 5 minutes | `swarm-builder`, `swarm-controller` | ✅ when missing or no longer matching the forge | ✅ read on every build |
|
| `swarm/services/builder/forge/token` | `swarm-controller`, at start and every 5 minutes | `swarm-builder`, `swarm-controller` | ✅ when missing or no longer matching the forge | ✅ read on every build |
|
||||||
|
| `swarm/services/attic/push-token` | you, in the store's UI (field `value`), or `bao kv put` with `value=-` and the token on stdin | `swarm-builder` | ❌ write a new token; a token the cache refuses fails the push, and the build stays green with `cached` false | ✅ read on every push; with no `swarm.attic` cache set, nothing reads it |
|
||||||
| `swarm/services/ci-metrics/forge/token` | `swarm-controller`, at start and every 5 minutes | `swarm-ci-metrics`, `swarm-controller` | ✅ when missing or no longer matching the forge | ✅ read on every pass |
|
| `swarm/services/ci-metrics/forge/token` | `swarm-controller`, at start and every 5 minutes | `swarm-ci-metrics`, `swarm-controller` | ✅ when missing or no longer matching the forge | ✅ read on every pass |
|
||||||
| _(not in the store)_ a hive's mTLS leaf | the store's PKI, or you by hand | the hive itself, from disk | must be stated | must be stated |
|
| _(not in the store)_ a hive's mTLS leaf | the store's PKI, or you by hand | the hive itself, from disk | must be stated | must be stated |
|
||||||
|
|
||||||
|
|
@ -158,7 +159,7 @@ login minter's, the forge host's and Grafana's, which reach more than one:
|
||||||
| `swarm-review-classifier`, in its container | `reviewClassifierClientCertFile` / `reviewClassifierClientKeyFile` | `swarm/services/review-classifier/anthropic-api-key`, `swarm/services/review-classifier/acp-provider` and `swarm/services/<classifier client id>/oidc/client` |
|
| `swarm-review-classifier`, in its container | `reviewClassifierClientCertFile` / `reviewClassifierClientKeyFile` | `swarm/services/review-classifier/anthropic-api-key`, `swarm/services/review-classifier/acp-provider` and `swarm/services/<classifier client id>/oidc/client` |
|
||||||
| `swarm-claude-login`, in its container | `claudeLoginClientCertFile` / `claudeLoginClientKeyFile` | creates and updates `swarm/agents/+/claude-credentials`; reads `swarm/services/<minter client id>/oidc/client` |
|
| `swarm-claude-login`, in its container | `claudeLoginClientCertFile` / `claudeLoginClientKeyFile` | creates and updates `swarm/agents/+/claude-credentials`; reads `swarm/services/<minter client id>/oidc/client` |
|
||||||
| `swarm-notify-router`, in its container | `notifyRouterClientCertFile` / `notifyRouterClientKeyFile` | `swarm/services/<router client id>/oidc/client` |
|
| `swarm-notify-router`, in its container | `notifyRouterClientCertFile` / `notifyRouterClientKeyFile` | `swarm/services/<router client id>/oidc/client` |
|
||||||
| `swarm-builder`, in its container | `builderClientCertFile` / `builderClientKeyFile` | `swarm/services/builder/forge/token` and `swarm/services/<builder client id>/oidc/client` |
|
| `swarm-builder`, in its container | `builderClientCertFile` / `builderClientKeyFile` | `swarm/services/builder/forge/token`, `swarm/services/attic/push-token` and `swarm/services/<builder client id>/oidc/client` |
|
||||||
| `swarm-ci-metrics` | `ciMetricsClientCertFile` / `ciMetricsClientKeyFile` | `swarm/services/ci-metrics/forge/token` and `swarm/services/<poller client id>/oidc/client` |
|
| `swarm-ci-metrics` | `ciMetricsClientCertFile` / `ciMetricsClientKeyFile` | `swarm/services/ci-metrics/forge/token` and `swarm/services/<poller client id>/oidc/client` |
|
||||||
| `swarm-bao-forge-oidc` and the forge's controller-token mint and its journal forwarder in its container | `forgePublisherClientCertFile` / `forgePublisherClientKeyFile` | writes `swarm/controller/swarm-controller/forge/token`; reads `swarm/services/<forge client id>/oidc/client` and `swarm/services/<forge forwarder client id>/oidc/client` |
|
| `swarm-bao-forge-oidc` and the forge's controller-token mint and its journal forwarder in its container | `forgePublisherClientCertFile` / `forgePublisherClientKeyFile` | writes `swarm/controller/swarm-controller/forge/token`; reads `swarm/services/<forge client id>/oidc/client` and `swarm/services/<forge forwarder client id>/oidc/client` |
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -62,6 +62,7 @@
|
||||||
./glue-swarm-otel-oidc-client.nix
|
./glue-swarm-otel-oidc-client.nix
|
||||||
./glue-victorialogs-otel-oidc-client.nix
|
./glue-victorialogs-otel-oidc-client.nix
|
||||||
./glue-victoriametrics-otel-oidc-client.nix
|
./glue-victoriametrics-otel-oidc-client.nix
|
||||||
|
./swarm-attic-options.nix
|
||||||
./swarm-authelia-service.nix
|
./swarm-authelia-service.nix
|
||||||
./swarm-authelia.nix
|
./swarm-authelia.nix
|
||||||
./swarm-bao-service.nix
|
./swarm-bao-service.nix
|
||||||
|
|
|
||||||
37
nix/host-modules/swarm-attic-options.nix
Normal file
37
nix/host-modules/swarm-attic-options.nix
Normal file
|
|
@ -0,0 +1,37 @@
|
||||||
|
# The binary cache the swarm pushes agent closures to, as every host sees it:
|
||||||
|
# which attic server, and which cache on it. The swarm runs no attic of its
|
||||||
|
# own yet; the instance is the operator's, outside the swarm.
|
||||||
|
{ lib, ... }:
|
||||||
|
{
|
||||||
|
options.services.hyperhive.swarm.attic = {
|
||||||
|
domain = lib.mkOption {
|
||||||
|
type = lib.types.nullOr lib.types.str;
|
||||||
|
default = null;
|
||||||
|
example = "cache.example.org";
|
||||||
|
description = ''
|
||||||
|
Name of the attic server, with no scheme: the swarm builder pushes to
|
||||||
|
`https://<domain>`, and trusts the certificate the system roots do, so
|
||||||
|
a Let's Encrypt certificate works and none is pinned. Set together with
|
||||||
|
{option}`services.hyperhive.swarm.attic.cache`.
|
||||||
|
|
||||||
|
Null pushes nothing: the swarm builder does not push, and the closure
|
||||||
|
record says it is not cached.
|
||||||
|
|
||||||
|
The push token is not an option. The operator writes it to
|
||||||
|
`secret/swarm/services/attic/push-token` in the swarm secret store
|
||||||
|
(field `value`), and the builder reads it on every push.
|
||||||
|
'';
|
||||||
|
};
|
||||||
|
|
||||||
|
cache = lib.mkOption {
|
||||||
|
type = lib.types.nullOr lib.types.str;
|
||||||
|
default = null;
|
||||||
|
example = "grid";
|
||||||
|
description = ''
|
||||||
|
Name of the cache on {option}`services.hyperhive.swarm.attic.domain`
|
||||||
|
that the swarm builder pushes closures to. The push token must be
|
||||||
|
allowed to push to it.
|
||||||
|
'';
|
||||||
|
};
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
@ -692,6 +692,12 @@ let
|
||||||
# to fetch config repos.
|
# to fetch config repos.
|
||||||
builderForgeTokenLeaf = "swarm/services/builder/forge/token";
|
builderForgeTokenLeaf = "swarm/services/builder/forge/token";
|
||||||
|
|
||||||
|
# The token the swarm builder pushes closures to the binary cache with, the nix
|
||||||
|
# half of `swarm_secret_client::builder::attic_push_token_path`. The operator
|
||||||
|
# writes it by hand; the builder reads it on every push, and nothing else in
|
||||||
|
# the swarm reaches it.
|
||||||
|
atticPushTokenLeaf = "swarm/services/attic/push-token";
|
||||||
|
|
||||||
# The CI run poller's forge token, the nix half of
|
# The CI run poller's forge token, the nix half of
|
||||||
# `swarm_secret_client::ci_metrics::forge_token_path`. The controller mints it
|
# `swarm_secret_client::ci_metrics::forge_token_path`. The controller mints it
|
||||||
# and reads it back before deciding whether to re-mint; the poller reads it
|
# and reads it back before deciding whether to re-mint; the poller reads it
|
||||||
|
|
@ -1146,14 +1152,16 @@ let
|
||||||
}
|
}
|
||||||
];
|
];
|
||||||
|
|
||||||
# The swarm builder: its forge token, read once per build, and its own queue
|
# The swarm builder: its forge token, read once per build, the cache push token,
|
||||||
# client secret, read at start. Both by the daemon inside its container.
|
# read once per push, and its own queue client secret, read at start. All by the
|
||||||
|
# daemon inside its container.
|
||||||
builderReaders = [
|
builderReaders = [
|
||||||
{
|
{
|
||||||
name = "swarm-builder";
|
name = "swarm-builder";
|
||||||
cn = baoDeploy.builderCommonName;
|
cn = baoDeploy.builderCommonName;
|
||||||
policyText =
|
policyText =
|
||||||
readStanza "${credentialMountPath}/data/${builderForgeTokenLeaf}"
|
readStanza "${credentialMountPath}/data/${builderForgeTokenLeaf}"
|
||||||
|
+ readStanza "${credentialMountPath}/data/${atticPushTokenLeaf}"
|
||||||
+ readStanza "${credentialMountPath}/data/swarm/services/${hyperhiveCfg.swarm.builder.clientId}/oidc/client";
|
+ readStanza "${credentialMountPath}/data/swarm/services/${hyperhiveCfg.swarm.builder.clientId}/oidc/client";
|
||||||
}
|
}
|
||||||
];
|
];
|
||||||
|
|
@ -2081,8 +2089,8 @@ in
|
||||||
description = ''
|
description = ''
|
||||||
Subject the store's `swarm-builder` cert-auth role accepts: the
|
Subject the store's `swarm-builder` cert-auth role accepts: the
|
||||||
identity the swarm builder presents from inside its container. Its
|
identity the swarm builder presents from inside its container. Its
|
||||||
grant is two paths, read only: `swarm/services/builder/forge/token`
|
grant is three paths, read only: `swarm/services/builder/forge/token`,
|
||||||
and its own queue client secret.
|
`swarm/services/attic/push-token` and its own queue client secret.
|
||||||
|
|
||||||
⚠️ Reserved as a hive name by ./swarm.nix, like its siblings.
|
⚠️ Reserved as a hive name by ./swarm.nix, like its siblings.
|
||||||
'';
|
'';
|
||||||
|
|
|
||||||
|
|
@ -17,8 +17,8 @@
|
||||||
#
|
#
|
||||||
# The container holds one credential, its own store leaf, which nspawn hands in
|
# The container holds one credential, its own store leaf, which nspawn hands in
|
||||||
# as a system credential with no copy on disk. The daemon reads everything else
|
# as a system credential with no copy on disk. The daemon reads everything else
|
||||||
# from the store with it: its queue client secret at start and its forge token
|
# from the store with it: its queue client secret at start, its forge token
|
||||||
# on every build.
|
# on every build and, where a cache is set, its cache push token on every push.
|
||||||
{
|
{
|
||||||
pkgs,
|
pkgs,
|
||||||
lib,
|
lib,
|
||||||
|
|
@ -33,6 +33,7 @@ let
|
||||||
natsCfg = hyperhiveCfg.swarm.nats;
|
natsCfg = hyperhiveCfg.swarm.nats;
|
||||||
autheliaCfg = hyperhiveCfg.swarm.authelia;
|
autheliaCfg = hyperhiveCfg.swarm.authelia;
|
||||||
forgeCfg = hyperhiveCfg.swarm.forge;
|
forgeCfg = hyperhiveCfg.swarm.forge;
|
||||||
|
atticCfg = hyperhiveCfg.swarm.attic;
|
||||||
gatewayCfg = hyperhiveCfg.gateway;
|
gatewayCfg = hyperhiveCfg.gateway;
|
||||||
networkCfg = hyperhiveCfg.network;
|
networkCfg = hyperhiveCfg.network;
|
||||||
tlsCfg = hyperhiveCfg.deploy.hive-controller.tls;
|
tlsCfg = hyperhiveCfg.deploy.hive-controller.tls;
|
||||||
|
|
@ -89,6 +90,9 @@ let
|
||||||
hyperhiveFlake = hyperhiveCfg.c0re.hyperhiveFlake;
|
hyperhiveFlake = hyperhiveCfg.c0re.hyperhiveFlake;
|
||||||
nixpkgsFlake = hyperhiveCfg.c0re.nixpkgsFlake;
|
nixpkgsFlake = hyperhiveCfg.c0re.nixpkgsFlake;
|
||||||
|
|
||||||
|
# Either half alone reaches the daemon too, which refuses to start on it.
|
||||||
|
pushesToCache = atticCfg.domain != null || atticCfg.cache != null;
|
||||||
|
|
||||||
privateNetwork = true;
|
privateNetwork = true;
|
||||||
in
|
in
|
||||||
{
|
{
|
||||||
|
|
@ -230,8 +234,9 @@ in
|
||||||
wantedBy = [ "multi-user.target" ];
|
wantedBy = [ "multi-user.target" ];
|
||||||
wants = [ "network-online.target" ];
|
wants = [ "network-online.target" ];
|
||||||
after = [ "network-online.target" ];
|
after = [ "network-online.target" ];
|
||||||
# nix fetches the config repo with `git`.
|
# nix fetches the config repo with `git`, and a green build is pushed
|
||||||
path = [ pkgs.git ];
|
# with `attic` where a cache is set.
|
||||||
|
path = [ pkgs.git ] ++ lib.optional pushesToCache pkgs.attic-client;
|
||||||
environment = {
|
environment = {
|
||||||
BAO_ADDR = "https://${baoCfg.domain}:${toString baoCfg.port}";
|
BAO_ADDR = "https://${baoCfg.domain}:${toString baoCfg.port}";
|
||||||
BAO_CLIENT_CERT = credential "builder-bao-cert";
|
BAO_CLIENT_CERT = credential "builder-bao-cert";
|
||||||
|
|
@ -252,6 +257,12 @@ in
|
||||||
SWARM_BUILDER_FORGE_URL = "https://${forgeCfg.domain}";
|
SWARM_BUILDER_FORGE_URL = "https://${forgeCfg.domain}";
|
||||||
SWARM_BUILDER_CONFIG_ORG = "agent-configs";
|
SWARM_BUILDER_CONFIG_ORG = "agent-configs";
|
||||||
}
|
}
|
||||||
|
// lib.optionalAttrs (atticCfg.domain != null) {
|
||||||
|
SWARM_BUILDER_ATTIC_DOMAIN = atticCfg.domain;
|
||||||
|
}
|
||||||
|
// lib.optionalAttrs (atticCfg.cache != null) {
|
||||||
|
SWARM_BUILDER_ATTIC_CACHE = atticCfg.cache;
|
||||||
|
}
|
||||||
// lib.optionalAttrs (storeFiles ? builder-bao-server-ca) {
|
// lib.optionalAttrs (storeFiles ? builder-bao-server-ca) {
|
||||||
BAO_CACERT = credential "builder-bao-server-ca";
|
BAO_CACERT = credential "builder-bao-server-ca";
|
||||||
}
|
}
|
||||||
|
|
@ -264,6 +275,10 @@ in
|
||||||
ExecStart = "${cfg.package}/bin/swarm-builder";
|
ExecStart = "${cfg.package}/bin/swarm-builder";
|
||||||
DynamicUser = true;
|
DynamicUser = true;
|
||||||
CacheDirectory = unit;
|
CacheDirectory = unit;
|
||||||
|
# /run/swarm-builder, mode 0700 and owned by the dynamic user: the
|
||||||
|
# push writes its token-bearing `attic` config under it.
|
||||||
|
RuntimeDirectory = unit;
|
||||||
|
RuntimeDirectoryMode = "0700";
|
||||||
# Bare ids: the system credentials nspawn handed in.
|
# Bare ids: the system credentials nspawn handed in.
|
||||||
LoadCredential = lib.attrNames storeFiles;
|
LoadCredential = lib.attrNames storeFiles;
|
||||||
Restart = "on-failure";
|
Restart = "on-failure";
|
||||||
|
|
|
||||||
|
|
@ -1743,17 +1743,18 @@ let
|
||||||
];
|
];
|
||||||
}
|
}
|
||||||
{
|
{
|
||||||
# The builder reads its forge token and its own queue client secret,
|
# The builder reads its forge token, the cache push token and its own queue
|
||||||
# each read-only and spelled to the leaf: `services/*` would reach every
|
# client secret, each read-only and spelled to the leaf: `services/*` would
|
||||||
# swarm service's secret.
|
# reach every swarm service's secret.
|
||||||
name = "the swarm builder's grant is its forge token and its queue client secret, and nothing else";
|
name = "the swarm builder's grant is its forge token, the cache push token and its queue client secret, and nothing else";
|
||||||
ok =
|
ok =
|
||||||
let
|
let
|
||||||
s = baoGrantHere.systemd.services.swarm-bao-builder-policy.script;
|
s = baoGrantHere.systemd.services.swarm-bao-builder-policy.script;
|
||||||
in
|
in
|
||||||
lib.hasInfix "path \"secret/data/swarm/services/builder/forge/token\" {\n capabilities = [\"read\"]\n}" s
|
lib.hasInfix "path \"secret/data/swarm/services/builder/forge/token\" {\n capabilities = [\"read\"]\n}" s
|
||||||
|
&& lib.hasInfix "path \"secret/data/swarm/services/attic/push-token\" {\n capabilities = [\"read\"]\n}" s
|
||||||
&& lib.hasInfix "path \"secret/data/swarm/services/swarm-builder/oidc/client\" {\n capabilities = [\"read\"]\n}" s
|
&& lib.hasInfix "path \"secret/data/swarm/services/swarm-builder/oidc/client\" {\n capabilities = [\"read\"]\n}" s
|
||||||
&& lib.length (lib.filter lib.isList (builtins.split "path \"" s)) == 2
|
&& lib.length (lib.filter lib.isList (builtins.split "path \"" s)) == 3
|
||||||
&& !(lib.hasInfix "secret/data/swarm/services/*" s)
|
&& !(lib.hasInfix "secret/data/swarm/services/*" s)
|
||||||
&& lib.hasInfix "auth/cert/certs/swarm-builder" s
|
&& lib.hasInfix "auth/cert/certs/swarm-builder" s
|
||||||
&& lib.hasInfix "allowed_common_names=swarm-builder" s
|
&& lib.hasInfix "allowed_common_names=swarm-builder" s
|
||||||
|
|
|
||||||
|
|
@ -29,6 +29,8 @@ let
|
||||||
withBuilder = hive {
|
withBuilder = hive {
|
||||||
deploy.bao.enable = true;
|
deploy.bao.enable = true;
|
||||||
deploy.builder.enable = true;
|
deploy.builder.enable = true;
|
||||||
|
swarm.attic.domain = "cache.example.org";
|
||||||
|
swarm.attic.cache = "grid";
|
||||||
};
|
};
|
||||||
|
|
||||||
servicesHere = shared.swarmServicesWithController;
|
servicesHere = shared.swarmServicesWithController;
|
||||||
|
|
@ -48,6 +50,8 @@ let
|
||||||
inner = container.config;
|
inner = container.config;
|
||||||
svc = inner.systemd.services.swarm-builder;
|
svc = inner.systemd.services.swarm-builder;
|
||||||
env = svc.environment;
|
env = svc.environment;
|
||||||
|
hasOnPath = name: s: lib.any (p: lib.getName p == name) s.path;
|
||||||
|
withoutCache = servicesHere.containers.swarm-builder.config.systemd.services.swarm-builder;
|
||||||
|
|
||||||
# The check flake on the inputs the builder's unit names, called as
|
# The check flake on the inputs the builder's unit names, called as
|
||||||
# `nix build` calls it. `hyperhive` is entered through the filtered source's
|
# `nix build` calls it. `hyperhive` is entered through the filtered source's
|
||||||
|
|
@ -116,6 +120,25 @@ let
|
||||||
&& svc.serviceConfig.DynamicUser
|
&& svc.serviceConfig.DynamicUser
|
||||||
&& !(lib.any (lib.hasSuffix "TOKEN") (lib.attrNames env));
|
&& !(lib.any (lib.hasSuffix "TOKEN") (lib.attrNames env));
|
||||||
}
|
}
|
||||||
|
{
|
||||||
|
# The closure is pushed with `attic` from the unit, whose config file holds
|
||||||
|
# the push token: it goes in a runtime directory only the dynamic user can
|
||||||
|
# enter, and the token is in no variable. The host without a cache is the
|
||||||
|
# control: same unit, none of it, with `git` still on the path to show
|
||||||
|
# the lookup finds what is there.
|
||||||
|
name = "the builder pushes to the configured cache with attic from a private runtime directory, and pushes nowhere without one";
|
||||||
|
ok =
|
||||||
|
env.SWARM_BUILDER_ATTIC_DOMAIN == "cache.example.org"
|
||||||
|
&& env.SWARM_BUILDER_ATTIC_CACHE == "grid"
|
||||||
|
&& hasOnPath "attic-client" svc
|
||||||
|
&& svc.serviceConfig.RuntimeDirectory == "swarm-builder"
|
||||||
|
&& svc.serviceConfig.RuntimeDirectoryMode == "0700"
|
||||||
|
&& !(lib.any (lib.hasInfix "TOKEN") (lib.attrNames env))
|
||||||
|
&& hasOnPath "git" withoutCache
|
||||||
|
&& !(hasOnPath "attic-client" withoutCache)
|
||||||
|
&& !(withoutCache.environment ? SWARM_BUILDER_ATTIC_DOMAIN)
|
||||||
|
&& !(withoutCache.environment ? SWARM_BUILDER_ATTIC_CACHE);
|
||||||
|
}
|
||||||
{
|
{
|
||||||
# The minting unit goes inactive after each run, so a start that pulls it
|
# The minting unit goes inactive after each run, so a start that pulls it
|
||||||
# runs it again and mints a leaf that is missing; an active
|
# runs it again and mints a leaf that is missing; an active
|
||||||
|
|
|
||||||
|
|
@ -19,7 +19,7 @@ the answer into a `swarm-builder` commit status on the PR's head.
|
||||||
as an untrusted client, so a config can neither set daemon options nor read
|
as an untrusted client, so a config can neither set daemon options nor read
|
||||||
this service's environment.
|
this service's environment.
|
||||||
- Nothing is pushed and nothing is recorded: the closure stays in the host's
|
- Nothing is pushed and nothing is recorded: the closure stays in the host's
|
||||||
store until it is collected.
|
store until it is collected. (A meta build can push, see below.)
|
||||||
|
|
||||||
## Meta builds
|
## Meta builds
|
||||||
|
|
||||||
|
|
@ -31,16 +31,21 @@ the platform it builds for. The controller records the answer; the service
|
||||||
writes nothing. It takes the same one-build slot as the PR checks. See
|
writes nothing. It takes the same one-build slot as the PR checks. See
|
||||||
[`docs/swarm/builder.md`](../docs/swarm/builder.md#deploy-gate).
|
[`docs/swarm/builder.md`](../docs/swarm/builder.md#deploy-gate).
|
||||||
|
|
||||||
|
Where `services.hyperhive.swarm.attic.{domain,cache}` are set, it first pushes a
|
||||||
|
green build's closure to that cache with `attic push`, and the answer says
|
||||||
|
whether that worked. A failed push leaves the build green. See
|
||||||
|
[`docs/swarm/builder.md`](../docs/swarm/builder.md#binary-cache).
|
||||||
|
|
||||||
## Credentials
|
## Credentials
|
||||||
|
|
||||||
The container holds one credential: its own store certificate, which nspawn
|
The container holds one credential: its own store certificate, which nspawn
|
||||||
hands in at start without a copy on disk. With it the service reads its
|
hands in at start without a copy on disk. With it the service reads its
|
||||||
queue client secret at start and again when authelia refuses it, and its
|
queue client secret at start and again when authelia refuses it, and its
|
||||||
read-only forge token at every build.
|
read-only forge token at every build, and the cache push token at every push.
|
||||||
`git` gets the token through a credential helper reading the child's
|
`git` gets the token through a credential helper reading the child's
|
||||||
environment, never in a URL or an argument, and the helper answers only an
|
environment, never in a URL or an argument, and the helper answers only an
|
||||||
`https` request for the forge's host. Its store role reads exactly
|
`https` request for the forge's host. Its store role reads exactly
|
||||||
those two paths.
|
those three paths.
|
||||||
|
|
||||||
## Archetypes
|
## Archetypes
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -16,6 +16,8 @@
|
||||||
//! and an `internal/meta` commit, and the answer is the store path of that
|
//! and an `internal/meta` commit, and the answer is the store path of that
|
||||||
//! agent's toplevel, built at the flake's own lock. They take the build slot
|
//! agent's toplevel, built at the flake's own lock. They take the build slot
|
||||||
//! PR checks take. The controller records the path; the builder writes nothing.
|
//! PR checks take. The controller records the path; the builder writes nothing.
|
||||||
|
//! When a cache is set (see [`push`]), a green meta build is pushed to it before
|
||||||
|
//! the reply, and the reply says whether that worked.
|
||||||
//!
|
//!
|
||||||
//! The store identity comes from the `BAO_*` environment. The queue client
|
//! The store identity comes from the `BAO_*` environment. The queue client
|
||||||
//! secret is read at start and again when authelia refuses it, the forge token
|
//! secret is read at start and again when authelia refuses it, the forge token
|
||||||
|
|
@ -39,6 +41,7 @@ use swarm_wire::builder::{
|
||||||
|
|
||||||
mod archetypes;
|
mod archetypes;
|
||||||
mod nix_build;
|
mod nix_build;
|
||||||
|
mod push;
|
||||||
|
|
||||||
/// The cert-auth role this service logs in to the store as. `swarm-bao.nix`
|
/// The cert-auth role this service logs in to the store as. `swarm-bao.nix`
|
||||||
/// writes it, matching the leaf's subject.
|
/// writes it, matching the leaf's subject.
|
||||||
|
|
@ -64,6 +67,17 @@ const MAX_SLOT_WAIT: Duration = REPLY_DEADLINE
|
||||||
.saturating_sub(BUILD_DEADLINE)
|
.saturating_sub(BUILD_DEADLINE)
|
||||||
.saturating_sub(SLOT_MARGIN);
|
.saturating_sub(SLOT_MARGIN);
|
||||||
|
|
||||||
|
/// What is held back from a meta build's reply deadline for publishing the reply
|
||||||
|
/// after its push.
|
||||||
|
const REPLY_MARGIN: Duration = Duration::from_secs(10);
|
||||||
|
|
||||||
|
/// How long the push of a meta build that ends at now may run: at most
|
||||||
|
/// [`push::PUSH_DEADLINE`], and never past `reply_by`, so a push cannot make the
|
||||||
|
/// reply later than the controller waits.
|
||||||
|
fn push_budget(reply_by: tokio::time::Instant) -> Duration {
|
||||||
|
push::PUSH_DEADLINE.min(reply_by.saturating_duration_since(tokio::time::Instant::now()))
|
||||||
|
}
|
||||||
|
|
||||||
#[tokio::main]
|
#[tokio::main]
|
||||||
async fn main() -> Result<()> {
|
async fn main() -> Result<()> {
|
||||||
hive_log::init();
|
hive_log::init();
|
||||||
|
|
@ -83,6 +97,7 @@ async fn main() -> Result<()> {
|
||||||
deadline: BUILD_DEADLINE,
|
deadline: BUILD_DEADLINE,
|
||||||
lock_deadline: LOCK_DEADLINE,
|
lock_deadline: LOCK_DEADLINE,
|
||||||
system: var("SYSTEM")?,
|
system: var("SYSTEM")?,
|
||||||
|
cache: push_target()?,
|
||||||
});
|
});
|
||||||
let secret =
|
let secret =
|
||||||
swarm_secret_client::read_with_startup_retry("reading the queue client secret", || {
|
swarm_secret_client::read_with_startup_retry("reading the queue client secret", || {
|
||||||
|
|
@ -300,7 +315,7 @@ async fn answer_meta_builds(
|
||||||
&slot,
|
&slot,
|
||||||
received,
|
received,
|
||||||
|| busy_meta(&job),
|
|| busy_meta(&job),
|
||||||
handle_meta(&job, &builder),
|
handle_meta(&job, &builder, received + REPLY_DEADLINE - REPLY_MARGIN),
|
||||||
)
|
)
|
||||||
.await
|
.await
|
||||||
else {
|
else {
|
||||||
|
|
@ -383,19 +398,36 @@ fn busy_meta(job: &BuildFromMeta) -> FromMetaReply {
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Build one meta request and log the build.
|
/// Build one meta request and log the build.
|
||||||
async fn handle_meta(job: &BuildFromMeta, builder: &nix_build::Builder) -> FromMetaReply {
|
async fn handle_meta(
|
||||||
|
job: &BuildFromMeta,
|
||||||
|
builder: &nix_build::Builder,
|
||||||
|
reply_by: tokio::time::Instant,
|
||||||
|
) -> FromMetaReply {
|
||||||
let reply = match within(TOKEN_DEADLINE, forge_token()).await {
|
let reply = match within(TOKEN_DEADLINE, forge_token()).await {
|
||||||
Ok(token) => builder.build_from_meta(job, &token).await,
|
Ok(token) => builder.build_from_meta(job, &token).await,
|
||||||
Err(e) => FromMetaReply::Unavailable {
|
Err(e) => FromMetaReply::Unavailable {
|
||||||
reason: format!("{e:#}"),
|
reason: format!("{e:#}"),
|
||||||
},
|
},
|
||||||
};
|
};
|
||||||
|
let reply = push::cache_built(
|
||||||
|
reply,
|
||||||
|
builder.cache.as_ref(),
|
||||||
|
attic_push_token(),
|
||||||
|
push_budget(reply_by),
|
||||||
|
&job.agent,
|
||||||
|
)
|
||||||
|
.await;
|
||||||
match &reply {
|
match &reply {
|
||||||
FromMetaReply::Built { out_path, system } => tracing::info!(
|
FromMetaReply::Built {
|
||||||
|
out_path,
|
||||||
|
system,
|
||||||
|
cached,
|
||||||
|
} => tracing::info!(
|
||||||
agent = %job.agent,
|
agent = %job.agent,
|
||||||
meta_rev = %job.meta_rev,
|
meta_rev = %job.meta_rev,
|
||||||
%out_path,
|
%out_path,
|
||||||
%system,
|
%system,
|
||||||
|
cached,
|
||||||
"agent meta build built"
|
"agent meta build built"
|
||||||
),
|
),
|
||||||
FromMetaReply::Failed { error_tail } => tracing::warn!(
|
FromMetaReply::Failed { error_tail } => tracing::warn!(
|
||||||
|
|
@ -458,6 +490,23 @@ fn log_build(job: &BuildAgent, reply: &Reply) {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The token the builder pushes closures to the cache with, read on every push so
|
||||||
|
/// a rotation takes effect on the next one.
|
||||||
|
async fn attic_push_token() -> Result<String> {
|
||||||
|
let path = swarm_secret_client::builder::attic_push_token_path()?;
|
||||||
|
let store = swarm_secret_client::SecretStore::from_env(CERT_ROLE)
|
||||||
|
.await
|
||||||
|
.context("logging in to the secret store")?;
|
||||||
|
let stored: Option<swarm_secret_client::builder::AtticPushToken> = store
|
||||||
|
.read_optional(&path)
|
||||||
|
.await
|
||||||
|
.with_context(|| format!("reading {path}"))?;
|
||||||
|
match stored {
|
||||||
|
Some(t) if !t.value.trim().is_empty() => Ok(t.value.trim().to_owned()),
|
||||||
|
_ => bail!("nothing at {path}: the cache push token has not been written"),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// The forge token the builder fetches config repos with, read on every
|
/// The forge token the builder fetches config repos with, read on every
|
||||||
/// build so a rotation takes effect on the next one.
|
/// build so a rotation takes effect on the next one.
|
||||||
async fn forge_token() -> Result<String> {
|
async fn forge_token() -> Result<String> {
|
||||||
|
|
@ -495,6 +544,17 @@ async fn client_secret(client_id: &str) -> Result<String> {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The cache the unit's environment names, if any. systemd sets
|
||||||
|
/// `RUNTIME_DIRECTORY` for the unit's runtime directory.
|
||||||
|
fn push_target() -> Result<Option<push::Remote>> {
|
||||||
|
let var = |name: &str| std::env::var(name).ok();
|
||||||
|
push::Remote::from_settings(
|
||||||
|
var(&format!("{PREFIX}_ATTIC_DOMAIN")),
|
||||||
|
var(&format!("{PREFIX}_ATTIC_CACHE")),
|
||||||
|
var("RUNTIME_DIRECTORY"),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
/// The forge's base URL, which must be `https`: the credential helper offers
|
/// The forge's base URL, which must be `https`: the credential helper offers
|
||||||
/// the token to `https` requests for its host alone.
|
/// the token to `https` requests for its host alone.
|
||||||
fn forge_url() -> Result<String> {
|
fn forge_url() -> Result<String> {
|
||||||
|
|
@ -580,6 +640,7 @@ mod tests {
|
||||||
FromMetaReply::Built {
|
FromMetaReply::Built {
|
||||||
out_path: "/nix/store/x".to_owned(),
|
out_path: "/nix/store/x".to_owned(),
|
||||||
system: "s".to_owned(),
|
system: "s".to_owned(),
|
||||||
|
cached: false,
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
.await;
|
.await;
|
||||||
|
|
@ -594,6 +655,7 @@ mod tests {
|
||||||
FromMetaReply::Built {
|
FromMetaReply::Built {
|
||||||
out_path: "/nix/store/x".to_owned(),
|
out_path: "/nix/store/x".to_owned(),
|
||||||
system: "s".to_owned(),
|
system: "s".to_owned(),
|
||||||
|
cached: false,
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
.await;
|
.await;
|
||||||
|
|
@ -616,6 +678,20 @@ mod tests {
|
||||||
assert_eq!(prompt.expect("read"), "t");
|
assert_eq!(prompt.expect("read"), "t");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[tokio::test(start_paused = true)]
|
||||||
|
async fn a_push_gets_the_deadline_unless_the_reply_is_due_sooner() {
|
||||||
|
let now = Instant::now();
|
||||||
|
assert_eq!(
|
||||||
|
push_budget(now + Duration::from_hours(1)),
|
||||||
|
push::PUSH_DEADLINE
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
push_budget(now + Duration::from_secs(90)),
|
||||||
|
Duration::from_secs(90)
|
||||||
|
);
|
||||||
|
assert_eq!(push_budget(now), Duration::ZERO);
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn a_build_started_at_the_slot_limit_ends_inside_the_reply_deadline() {
|
fn a_build_started_at_the_slot_limit_ends_inside_the_reply_deadline() {
|
||||||
assert!(MAX_SLOT_WAIT > Duration::ZERO);
|
assert!(MAX_SLOT_WAIT > Duration::ZERO);
|
||||||
|
|
|
||||||
|
|
@ -59,6 +59,9 @@ pub struct Builder {
|
||||||
pub lock_deadline: Duration,
|
pub lock_deadline: Duration,
|
||||||
/// The platform this host builds for, reported with each meta build.
|
/// The platform this host builds for, reported with each meta build.
|
||||||
pub system: String,
|
pub system: String,
|
||||||
|
/// Where a meta build's closure is pushed after it succeeds; `None` pushes
|
||||||
|
/// nothing.
|
||||||
|
pub cache: Option<crate::push::Remote>,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl Builder {
|
impl Builder {
|
||||||
|
|
@ -88,6 +91,7 @@ impl Builder {
|
||||||
Reply::Built { out_path } => FromMetaReply::Built {
|
Reply::Built { out_path } => FromMetaReply::Built {
|
||||||
out_path,
|
out_path,
|
||||||
system: self.system.clone(),
|
system: self.system.clone(),
|
||||||
|
cached: false,
|
||||||
},
|
},
|
||||||
Reply::Failed { error_tail } => FromMetaReply::Failed { error_tail },
|
Reply::Failed { error_tail } => FromMetaReply::Failed { error_tail },
|
||||||
},
|
},
|
||||||
|
|
@ -437,6 +441,7 @@ mod tests {
|
||||||
deadline: BUILD_DEADLINE,
|
deadline: BUILD_DEADLINE,
|
||||||
lock_deadline: LOCK_DEADLINE,
|
lock_deadline: LOCK_DEADLINE,
|
||||||
system: "x86_64-linux".to_owned(),
|
system: "x86_64-linux".to_owned(),
|
||||||
|
cache: None,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -514,6 +519,7 @@ mod tests {
|
||||||
FromMetaReply::Built {
|
FromMetaReply::Built {
|
||||||
out_path: format!("{} {}", BUILD_FLAGS.join(" "), b.meta_attr(&meta_job())),
|
out_path: format!("{} {}", BUILD_FLAGS.join(" "), b.meta_attr(&meta_job())),
|
||||||
system: "x86_64-linux".to_owned(),
|
system: "x86_64-linux".to_owned(),
|
||||||
|
cached: false,
|
||||||
}
|
}
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
|
||||||
437
swarm-builder/src/push.rs
Normal file
437
swarm-builder/src/push.rs
Normal file
|
|
@ -0,0 +1,437 @@
|
||||||
|
//! The push of a built closure to the binary cache, with `attic push`.
|
||||||
|
//!
|
||||||
|
//! The cache is an attic instance outside the swarm. Its push token is read
|
||||||
|
//! from the secret store per push and written to an `attic` config file in a
|
||||||
|
//! directory only the builder's unit can enter, which `attic` finds through
|
||||||
|
//! `XDG_CONFIG_HOME`. `attic login <name> <url> <token>` is not used: it puts
|
||||||
|
//! the token in the process list. `attic` verifies the server's certificate
|
||||||
|
//! against the system roots.
|
||||||
|
//!
|
||||||
|
//! A push that does not happen or does not finish leaves the build green with
|
||||||
|
//! `cached` false: the closure is still in the host's store, and the next build
|
||||||
|
//! of the agent pushes again.
|
||||||
|
|
||||||
|
use std::future::Future;
|
||||||
|
use std::path::{Path, PathBuf};
|
||||||
|
use std::process::Stdio;
|
||||||
|
use std::time::Duration;
|
||||||
|
|
||||||
|
use anyhow::{Result, bail};
|
||||||
|
use swarm_wire::builder::FromMetaReply;
|
||||||
|
|
||||||
|
/// The longest one push may run, when the reply deadline leaves that much.
|
||||||
|
pub const PUSH_DEADLINE: Duration = Duration::from_mins(10);
|
||||||
|
|
||||||
|
/// The server name in the config file, and in the cache argument of the push.
|
||||||
|
const SERVER: &str = "swarm";
|
||||||
|
|
||||||
|
/// Where the builder pushes: one cache on one attic server.
|
||||||
|
pub struct Remote {
|
||||||
|
/// The `attic` binary: a name found in the unit's path in production.
|
||||||
|
pub attic: PathBuf,
|
||||||
|
/// The server's name, without scheme: `host` or `host:port`.
|
||||||
|
pub domain: String,
|
||||||
|
/// The cache's name on that server.
|
||||||
|
pub cache: String,
|
||||||
|
/// A directory only the builder can enter, which the config files go in.
|
||||||
|
pub state_dir: PathBuf,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Remote {
|
||||||
|
/// The cache the environment names: `None` when neither value is set, so
|
||||||
|
/// the builder pushes nothing.
|
||||||
|
///
|
||||||
|
/// # Errors
|
||||||
|
/// When only one of the two is set, or a value would bend the config file
|
||||||
|
/// or the command.
|
||||||
|
pub fn from_settings(
|
||||||
|
domain: Option<String>,
|
||||||
|
cache: Option<String>,
|
||||||
|
state_dir: Option<String>,
|
||||||
|
) -> Result<Option<Self>> {
|
||||||
|
let (domain, cache) = match (domain, cache) {
|
||||||
|
(None, None) => return Ok(None),
|
||||||
|
(Some(domain), Some(cache)) => (domain, cache),
|
||||||
|
_ => bail!("the cache domain and the cache name must be set together"),
|
||||||
|
};
|
||||||
|
let plain = |s: &str| {
|
||||||
|
!s.is_empty()
|
||||||
|
&& s.bytes()
|
||||||
|
.all(|b| b.is_ascii_alphanumeric() || matches!(b, b'.' | b'-' | b'_' | b':'))
|
||||||
|
};
|
||||||
|
if !plain(&domain) || !plain(&cache) {
|
||||||
|
bail!("the cache domain {domain:?} and name {cache:?} must be plain names");
|
||||||
|
}
|
||||||
|
let Some(state_dir) = state_dir else {
|
||||||
|
bail!("a cache is set, but the unit has no runtime directory to hold its config");
|
||||||
|
};
|
||||||
|
Ok(Some(Self {
|
||||||
|
attic: PathBuf::from("attic"),
|
||||||
|
domain,
|
||||||
|
cache,
|
||||||
|
state_dir: PathBuf::from(state_dir),
|
||||||
|
}))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Push `out_path` with `token`. The result is what `attic` printed.
|
||||||
|
///
|
||||||
|
/// # Errors
|
||||||
|
/// Why the push did not finish: the config could not be written, `attic`
|
||||||
|
/// could not start, or it exited with an error.
|
||||||
|
pub async fn push(&self, token: &str, out_path: &str) -> Result<String, String> {
|
||||||
|
if !out_path.starts_with("/nix/store/") || out_path.contains(char::is_whitespace) {
|
||||||
|
return Err(format!("{out_path:?} is not a store path"));
|
||||||
|
}
|
||||||
|
let home = ConfigHome::write(&self.state_dir, &self.domain, token)
|
||||||
|
.map_err(|e| format!("writing the attic config: {e}"))?;
|
||||||
|
let output = self
|
||||||
|
.command(&home.0, out_path)
|
||||||
|
.output()
|
||||||
|
.await
|
||||||
|
.map_err(|e| format!("starting attic: {e}"))?;
|
||||||
|
let said = |bytes: &[u8]| {
|
||||||
|
crate::nix_build::last_lines(&String::from_utf8_lossy(bytes))
|
||||||
|
.unwrap_or_default()
|
||||||
|
.replace(token, "<redacted>")
|
||||||
|
};
|
||||||
|
if output.status.success() {
|
||||||
|
Ok(said(&output.stdout))
|
||||||
|
} else {
|
||||||
|
Err(format!(
|
||||||
|
"attic exited with {}: {}",
|
||||||
|
output.status,
|
||||||
|
said(&output.stderr)
|
||||||
|
))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// `attic push` of `out_path`, reading its config from `config_home`. The
|
||||||
|
/// token is in the file there and in no argument or variable.
|
||||||
|
fn command(&self, config_home: &Path, out_path: &str) -> tokio::process::Command {
|
||||||
|
let mut cmd = tokio::process::Command::new(&self.attic);
|
||||||
|
cmd.arg("push")
|
||||||
|
.arg(format!("{SERVER}:{}", self.cache))
|
||||||
|
.arg(out_path)
|
||||||
|
.env("XDG_CONFIG_HOME", config_home)
|
||||||
|
.stdin(Stdio::null())
|
||||||
|
.stdout(Stdio::piped())
|
||||||
|
.stderr(Stdio::piped())
|
||||||
|
.kill_on_drop(true);
|
||||||
|
cmd
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Numbers the config directories of one process's pushes.
|
||||||
|
static NEXT: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(0);
|
||||||
|
|
||||||
|
/// The config directory of one push, removed with the token in it on drop.
|
||||||
|
struct ConfigHome(PathBuf);
|
||||||
|
|
||||||
|
impl ConfigHome {
|
||||||
|
/// `<parent>/push-<n>/attic/config.toml`, readable by this user alone.
|
||||||
|
fn write(parent: &Path, domain: &str, token: &str) -> std::io::Result<Self> {
|
||||||
|
use std::io::Write as _;
|
||||||
|
use std::os::unix::fs::{DirBuilderExt as _, OpenOptionsExt as _};
|
||||||
|
// A quote or a control character would end the TOML string early.
|
||||||
|
if token.is_empty() || token.bytes().any(|b| b == b'"' || b == b'\\' || b < b' ') {
|
||||||
|
return Err(std::io::Error::other(
|
||||||
|
"the push token is empty or holds a character the config cannot",
|
||||||
|
));
|
||||||
|
}
|
||||||
|
let home = Self(parent.join(format!(
|
||||||
|
"push-{}-{}",
|
||||||
|
std::process::id(),
|
||||||
|
NEXT.fetch_add(1, std::sync::atomic::Ordering::Relaxed)
|
||||||
|
)));
|
||||||
|
let dir = home.0.join("attic");
|
||||||
|
std::fs::DirBuilder::new()
|
||||||
|
.recursive(true)
|
||||||
|
.mode(0o700)
|
||||||
|
.create(&dir)?;
|
||||||
|
let mut file = std::fs::OpenOptions::new()
|
||||||
|
.write(true)
|
||||||
|
.create_new(true)
|
||||||
|
.mode(0o600)
|
||||||
|
.open(dir.join("config.toml"))?;
|
||||||
|
write!(
|
||||||
|
file,
|
||||||
|
"default-server = \"{SERVER}\"\n\n[servers.{SERVER}]\n\
|
||||||
|
endpoint = \"https://{domain}\"\ntoken = \"{token}\"\n"
|
||||||
|
)?;
|
||||||
|
Ok(home)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Drop for ConfigHome {
|
||||||
|
fn drop(&mut self) {
|
||||||
|
let _ = std::fs::remove_dir_all(&self.0);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// `reply` with `cached` set by pushing its closure, when it is a built one and
|
||||||
|
/// a cache is set. `read_token` is not polled without a cache. The token read
|
||||||
|
/// and the push together get `budget`.
|
||||||
|
///
|
||||||
|
/// Whatever happens to the push, a built reply stays built: a failure is logged
|
||||||
|
/// and leaves `cached` false.
|
||||||
|
pub async fn cache_built(
|
||||||
|
reply: FromMetaReply,
|
||||||
|
cache: Option<&Remote>,
|
||||||
|
read_token: impl Future<Output = Result<String>>,
|
||||||
|
budget: Duration,
|
||||||
|
agent: &str,
|
||||||
|
) -> FromMetaReply {
|
||||||
|
let (
|
||||||
|
Some(cache),
|
||||||
|
FromMetaReply::Built {
|
||||||
|
out_path, system, ..
|
||||||
|
},
|
||||||
|
) = (cache, &reply)
|
||||||
|
else {
|
||||||
|
return reply;
|
||||||
|
};
|
||||||
|
let pushed = tokio::time::timeout(budget, async {
|
||||||
|
let token = read_token
|
||||||
|
.await
|
||||||
|
.map_err(|e| format!("reading the push token: {e:#}"))?;
|
||||||
|
cache.push(&token, out_path).await
|
||||||
|
})
|
||||||
|
.await
|
||||||
|
.unwrap_or_else(|_| {
|
||||||
|
Err(format!(
|
||||||
|
"the push did not finish within {} s",
|
||||||
|
budget.as_secs()
|
||||||
|
))
|
||||||
|
});
|
||||||
|
match pushed {
|
||||||
|
Ok(_) => {
|
||||||
|
tracing::info!(%agent, %out_path, cache = %cache.cache, "closure pushed to the cache");
|
||||||
|
FromMetaReply::Built {
|
||||||
|
out_path: out_path.clone(),
|
||||||
|
system: system.clone(),
|
||||||
|
cached: true,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Err(reason) => {
|
||||||
|
tracing::warn!(%agent, %out_path, cache = %cache.cache, %reason, "closure not pushed to the cache");
|
||||||
|
reply
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use std::os::unix::fs::PermissionsExt as _;
|
||||||
|
use std::sync::atomic::{AtomicBool, Ordering};
|
||||||
|
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
const TOKEN: &str = "eyJ0b2tlbiI6InNlY3JldCJ9.c2lnbmVk";
|
||||||
|
const OUT: &str = "/nix/store/0123456789abcdfghijklmnpqrsvwxyz-nixos-system-atlas";
|
||||||
|
|
||||||
|
fn built() -> FromMetaReply {
|
||||||
|
FromMetaReply::Built {
|
||||||
|
out_path: OUT.to_owned(),
|
||||||
|
system: "x86_64-linux".to_owned(),
|
||||||
|
cached: false,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn cached(reply: &FromMetaReply) -> bool {
|
||||||
|
match reply {
|
||||||
|
FromMetaReply::Built { cached, .. } => *cached,
|
||||||
|
other => panic!("not built: {other:?}"),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A cache whose `attic` is the program `attic`: `true` and `false` stand
|
||||||
|
/// in for a push that works and one that fails, and `echo` for one that
|
||||||
|
/// says what it was run with.
|
||||||
|
fn cache_with(attic: &str) -> Remote {
|
||||||
|
Remote {
|
||||||
|
attic: PathBuf::from(attic),
|
||||||
|
domain: "cache.example.org".to_owned(),
|
||||||
|
cache: "grid".to_owned(),
|
||||||
|
state_dir: std::env::temp_dir(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn token() -> impl Future<Output = Result<String>> {
|
||||||
|
std::future::ready(Ok(TOKEN.to_owned()))
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn a_push_that_works_marks_the_build_cached() {
|
||||||
|
let cache = cache_with("true");
|
||||||
|
let reply = cache_built(built(), Some(&cache), token(), PUSH_DEADLINE, "atlas").await;
|
||||||
|
assert!(cached(&reply), "{reply:?}");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn a_push_that_fails_leaves_the_build_green_and_not_cached() {
|
||||||
|
let cache = cache_with("false");
|
||||||
|
let reply = cache_built(built(), Some(&cache), token(), PUSH_DEADLINE, "atlas").await;
|
||||||
|
assert_eq!(reply, built());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn a_push_that_cannot_start_leaves_the_build_green_and_not_cached() {
|
||||||
|
let cache = cache_with("/nonexistent/attic");
|
||||||
|
let reply = cache_built(built(), Some(&cache), token(), PUSH_DEADLINE, "atlas").await;
|
||||||
|
assert_eq!(reply, built());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn an_unreadable_token_leaves_the_build_green_and_not_cached() {
|
||||||
|
let cache = cache_with("true");
|
||||||
|
let reply = cache_built(
|
||||||
|
built(),
|
||||||
|
Some(&cache),
|
||||||
|
async { anyhow::bail!("nothing at the path") },
|
||||||
|
PUSH_DEADLINE,
|
||||||
|
"atlas",
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
assert_eq!(reply, built());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test(start_paused = true)]
|
||||||
|
async fn a_token_read_that_outlasts_the_budget_leaves_the_build_green() {
|
||||||
|
let cache = cache_with("true");
|
||||||
|
let started = tokio::time::Instant::now();
|
||||||
|
let reply = cache_built(
|
||||||
|
built(),
|
||||||
|
Some(&cache),
|
||||||
|
std::future::pending(),
|
||||||
|
Duration::from_secs(5),
|
||||||
|
"atlas",
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
assert_eq!(reply, built());
|
||||||
|
assert_eq!(started.elapsed(), Duration::from_secs(5));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn without_a_cache_nothing_is_read_or_pushed() {
|
||||||
|
let polled = std::sync::Arc::new(AtomicBool::new(false));
|
||||||
|
let tracked = |polled: &std::sync::Arc<AtomicBool>| {
|
||||||
|
let polled = std::sync::Arc::clone(polled);
|
||||||
|
async move {
|
||||||
|
polled.store(true, Ordering::SeqCst);
|
||||||
|
token().await
|
||||||
|
}
|
||||||
|
};
|
||||||
|
let reply = cache_built(built(), None, tracked(&polled), PUSH_DEADLINE, "atlas").await;
|
||||||
|
assert_eq!(reply, built());
|
||||||
|
assert!(!polled.load(Ordering::SeqCst), "the token was read");
|
||||||
|
|
||||||
|
// Control: with a cache the same future runs.
|
||||||
|
let cache = cache_with("true");
|
||||||
|
cache_built(
|
||||||
|
built(),
|
||||||
|
Some(&cache),
|
||||||
|
tracked(&polled),
|
||||||
|
PUSH_DEADLINE,
|
||||||
|
"atlas",
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
assert!(polled.load(Ordering::SeqCst));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn a_build_that_is_not_built_is_not_pushed() {
|
||||||
|
let cache = cache_with("true");
|
||||||
|
let failed = FromMetaReply::Failed {
|
||||||
|
error_tail: "e".to_owned(),
|
||||||
|
};
|
||||||
|
let reply = cache_built(
|
||||||
|
failed.clone(),
|
||||||
|
Some(&cache),
|
||||||
|
token(),
|
||||||
|
PUSH_DEADLINE,
|
||||||
|
"atlas",
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
assert_eq!(reply, failed);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn the_push_command_has_no_token_in_it() {
|
||||||
|
// `echo` prints the arguments it is given.
|
||||||
|
let cache = cache_with("echo");
|
||||||
|
let said = cache.push(TOKEN, OUT).await.expect("echo succeeds");
|
||||||
|
assert_eq!(said, format!("push swarm:grid {OUT}"));
|
||||||
|
assert!(!said.contains(TOKEN));
|
||||||
|
|
||||||
|
let cmd = cache.command(Path::new("/run/home"), OUT);
|
||||||
|
let cmd = cmd.as_std();
|
||||||
|
let args: Vec<_> = cmd.get_args().collect();
|
||||||
|
assert_eq!(args, ["push", "swarm:grid", OUT]);
|
||||||
|
let env: Vec<_> = cmd.get_envs().collect();
|
||||||
|
assert_eq!(
|
||||||
|
env,
|
||||||
|
[(
|
||||||
|
std::ffi::OsStr::new("XDG_CONFIG_HOME"),
|
||||||
|
Some(std::ffi::OsStr::new("/run/home"))
|
||||||
|
)]
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn the_token_is_in_a_private_config_file_that_is_removed_after() {
|
||||||
|
let parent = std::env::temp_dir().join(format!("push-test-{}", std::process::id()));
|
||||||
|
std::fs::create_dir_all(&parent).expect("parent");
|
||||||
|
let home = ConfigHome::write(&parent, "cache.example.org", TOKEN).expect("written");
|
||||||
|
let file = home.0.join("attic/config.toml");
|
||||||
|
let text = std::fs::read_to_string(&file).expect("reads");
|
||||||
|
assert!(text.contains(&format!("token = \"{TOKEN}\"")), "{text}");
|
||||||
|
assert!(
|
||||||
|
text.contains("endpoint = \"https://cache.example.org\""),
|
||||||
|
"{text}"
|
||||||
|
);
|
||||||
|
let mode = |p: &Path| std::fs::metadata(p).expect("stat").permissions().mode() & 0o777;
|
||||||
|
assert_eq!(mode(&file), 0o600);
|
||||||
|
assert_eq!(mode(&home.0), 0o700);
|
||||||
|
let root = home.0.clone();
|
||||||
|
drop(home);
|
||||||
|
assert!(!root.exists());
|
||||||
|
std::fs::remove_dir_all(&parent).expect("cleanup");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_token_that_would_end_the_config_string_is_refused() {
|
||||||
|
let parent = std::env::temp_dir();
|
||||||
|
for bad in ["", "a\"b", "a\\b", "a\nb"] {
|
||||||
|
assert!(ConfigHome::write(&parent, "h", bad).is_err(), "{bad:?}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn a_path_that_is_not_a_store_path_is_not_pushed() {
|
||||||
|
let cache = cache_with("true");
|
||||||
|
assert!(cache.push(TOKEN, "--help").await.is_err());
|
||||||
|
assert!(cache.push(TOKEN, "/nix/store/x y").await.is_err());
|
||||||
|
assert!(cache.push(TOKEN, OUT).await.is_ok());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn the_settings_name_a_cache_only_when_both_are_set() {
|
||||||
|
let state = || Some("/run/swarm-builder".to_owned());
|
||||||
|
let some = |s: &str| Some(s.to_owned());
|
||||||
|
assert!(
|
||||||
|
Remote::from_settings(None, None, None)
|
||||||
|
.expect("ok")
|
||||||
|
.is_none()
|
||||||
|
);
|
||||||
|
let cache = Remote::from_settings(some("cache.example.org"), some("grid"), state())
|
||||||
|
.expect("ok")
|
||||||
|
.expect("set");
|
||||||
|
assert_eq!(cache.domain, "cache.example.org");
|
||||||
|
assert_eq!(cache.cache, "grid");
|
||||||
|
assert!(Remote::from_settings(some("cache.example.org"), None, state()).is_err());
|
||||||
|
assert!(Remote::from_settings(None, some("grid"), state()).is_err());
|
||||||
|
assert!(Remote::from_settings(some("https://cache"), some("grid"), state()).is_err());
|
||||||
|
assert!(Remote::from_settings(some("cache"), some("a\"b"), state()).is_err());
|
||||||
|
assert!(Remote::from_settings(some("cache"), some("grid"), None).is_err());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -306,13 +306,18 @@ pub(crate) async fn run(
|
||||||
}
|
}
|
||||||
Outcome::Failed(error_tail)
|
Outcome::Failed(error_tail)
|
||||||
}
|
}
|
||||||
Ok(FromMetaReply::Built { out_path, system }) => {
|
Ok(FromMetaReply::Built {
|
||||||
|
out_path,
|
||||||
|
system,
|
||||||
|
cached,
|
||||||
|
}) => {
|
||||||
let built_rev = meta_rev.clone();
|
let built_rev = meta_rev.clone();
|
||||||
let closure = AgentClosure {
|
let closure = AgentClosure {
|
||||||
meta_rev,
|
meta_rev,
|
||||||
store_path: out_path.clone(),
|
store_path: out_path.clone(),
|
||||||
system,
|
system,
|
||||||
built_unix: unix_now(),
|
built_unix: unix_now(),
|
||||||
|
cached,
|
||||||
};
|
};
|
||||||
match queue.record(&agent, &closure).await {
|
match queue.record(&agent, &closure).await {
|
||||||
Ok(()) => {
|
Ok(()) => {
|
||||||
|
|
@ -415,9 +420,14 @@ mod tests {
|
||||||
}
|
}
|
||||||
|
|
||||||
fn built() -> Self {
|
fn built() -> Self {
|
||||||
|
Self::built_cached(false)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn built_cached(cached: bool) -> Self {
|
||||||
Self::answering(Ok(FromMetaReply::Built {
|
Self::answering(Ok(FromMetaReply::Built {
|
||||||
out_path: "/nix/store/x-toplevel".to_owned(),
|
out_path: "/nix/store/x-toplevel".to_owned(),
|
||||||
system: "x86_64-linux".to_owned(),
|
system: "x86_64-linux".to_owned(),
|
||||||
|
cached,
|
||||||
}))
|
}))
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -470,6 +480,22 @@ mod tests {
|
||||||
assert_eq!(closure.store_path, "/nix/store/x-toplevel");
|
assert_eq!(closure.store_path, "/nix/store/x-toplevel");
|
||||||
assert_eq!(closure.system, "x86_64-linux");
|
assert_eq!(closure.system, "x86_64-linux");
|
||||||
assert!(closure.built_unix > 0);
|
assert!(closure.built_unix > 0);
|
||||||
|
assert!(!closure.cached);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The record says what the builder said about the cache, both ways.
|
||||||
|
#[tokio::test]
|
||||||
|
async fn the_record_carries_whether_the_builder_cached_the_closure() {
|
||||||
|
for cached in [true, false] {
|
||||||
|
let queue = FakeQueue::built_cached(cached);
|
||||||
|
let outcome = run_with(&FakeHead::reading(Ok(Some(REV))), &queue).await;
|
||||||
|
assert_eq!(outcome, Outcome::Done);
|
||||||
|
let recorded = queue.recorded.lock().unwrap();
|
||||||
|
let [(_, closure)] = recorded.as_slice() else {
|
||||||
|
panic!("one record: {recorded:?}")
|
||||||
|
};
|
||||||
|
assert_eq!(closure.cached, cached);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Any `nix` failure, a timeout included, fails the node at once with what
|
/// Any `nix` failure, a timeout included, fails the node at once with what
|
||||||
|
|
@ -749,6 +775,7 @@ mod tests {
|
||||||
store_path: "/nix/store/a".to_owned(),
|
store_path: "/nix/store/a".to_owned(),
|
||||||
system: "x86_64-linux".to_owned(),
|
system: "x86_64-linux".to_owned(),
|
||||||
built_unix: 1,
|
built_unix: 1,
|
||||||
|
cached: false,
|
||||||
};
|
};
|
||||||
client.record("atlas", &closure).await.expect("written");
|
client.record("atlas", &closure).await.expect("written");
|
||||||
closure.store_path = "/nix/store/b".to_owned();
|
closure.store_path = "/nix/store/b".to_owned();
|
||||||
|
|
|
||||||
|
|
@ -2,8 +2,9 @@
|
||||||
//! secret live in the store, and what each object holds.
|
//! secret live in the store, and what each object holds.
|
||||||
//!
|
//!
|
||||||
//! swarm-controller mints the forge token; the builder reads it, and its queue
|
//! swarm-controller mints the forge token; the builder reads it, and its queue
|
||||||
//! client secret, under its own certificate. Its read stanza in
|
//! client secret, under its own certificate. It also reads the cache push
|
||||||
//! `swarm-bao.nix` names exactly these two paths.
|
//! token, which the operator writes. Its read stanza in `swarm-bao.nix` names
|
||||||
|
//! exactly these three paths.
|
||||||
|
|
||||||
use serde::{Deserialize, Serialize};
|
use serde::{Deserialize, Serialize};
|
||||||
|
|
||||||
|
|
@ -26,6 +27,30 @@ pub fn forge_token_path() -> Result<String, Error> {
|
||||||
Ok(format!("{prefix}/forge/token"))
|
Ok(format!("{prefix}/forge/token"))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The path holding the token the builder pushes closures to the binary cache
|
||||||
|
/// with, as an [`AtticPushToken`]. The operator writes it by hand: the cache is
|
||||||
|
/// an instance outside the swarm, and no swarm unit mints the token.
|
||||||
|
///
|
||||||
|
/// Under the cache's own name segment rather than the builder's, since the
|
||||||
|
/// token belongs to the cache's tenant.
|
||||||
|
///
|
||||||
|
/// # Errors
|
||||||
|
/// Never in practice: the name segment is a constant. The `Result` is
|
||||||
|
/// [`principal_prefix`]'s.
|
||||||
|
pub fn attic_push_token_path() -> Result<String, Error> {
|
||||||
|
let prefix = principal_prefix(Kind::Service, "attic")?;
|
||||||
|
Ok(format!("{prefix}/push-token"))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What [`attic_push_token_path`] holds.
|
||||||
|
///
|
||||||
|
/// No `Debug`: `value` writes to the binary cache.
|
||||||
|
#[derive(Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
pub struct AtticPushToken {
|
||||||
|
/// The token, under the field name the operator writes.
|
||||||
|
pub value: String,
|
||||||
|
}
|
||||||
|
|
||||||
/// The path holding the queue client secret of the builder's OIDC client
|
/// The path holding the queue client secret of the builder's OIDC client
|
||||||
/// `client_id`, where `swarm-secret-publisher.nix` puts every swarm service's.
|
/// `client_id`, where `swarm-secret-publisher.nix` puts every swarm service's.
|
||||||
///
|
///
|
||||||
|
|
@ -60,5 +85,9 @@ mod tests {
|
||||||
client_secret_path("swarm-builder").expect("a plain id is legal"),
|
client_secret_path("swarm-builder").expect("a plain id is legal"),
|
||||||
"swarm/services/swarm-builder/oidc/client"
|
"swarm/services/swarm-builder/oidc/client"
|
||||||
);
|
);
|
||||||
|
assert_eq!(
|
||||||
|
attic_push_token_path().expect("a constant segment is legal"),
|
||||||
|
"swarm/services/attic/push-token"
|
||||||
|
);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -32,6 +32,10 @@ pub struct AgentClosure {
|
||||||
pub system: String,
|
pub system: String,
|
||||||
/// When the controller recorded the build, in unix seconds.
|
/// When the controller recorded the build, in unix seconds.
|
||||||
pub built_unix: i64,
|
pub built_unix: i64,
|
||||||
|
/// Whether the builder pushed the closure to the binary cache. A record
|
||||||
|
/// written before the field existed reads as false.
|
||||||
|
#[serde(default)]
|
||||||
|
pub cached: bool,
|
||||||
}
|
}
|
||||||
|
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
|
|
@ -51,12 +55,14 @@ mod tests {
|
||||||
store_path: "/nix/store/x".to_owned(),
|
store_path: "/nix/store/x".to_owned(),
|
||||||
system: "x86_64-linux".to_owned(),
|
system: "x86_64-linux".to_owned(),
|
||||||
built_unix: 7,
|
built_unix: 7,
|
||||||
|
cached: true,
|
||||||
};
|
};
|
||||||
let wire = serde_json::json!({
|
let wire = serde_json::json!({
|
||||||
"meta_rev": "abc",
|
"meta_rev": "abc",
|
||||||
"store_path": "/nix/store/x",
|
"store_path": "/nix/store/x",
|
||||||
"system": "x86_64-linux",
|
"system": "x86_64-linux",
|
||||||
"built_unix": 7,
|
"built_unix": 7,
|
||||||
|
"cached": true,
|
||||||
});
|
});
|
||||||
assert_eq!(serde_json::to_value(&value).expect("encodes"), wire);
|
assert_eq!(serde_json::to_value(&value).expect("encodes"), wire);
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
|
|
@ -64,4 +70,16 @@ mod tests {
|
||||||
value
|
value
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_record_without_the_field_reads_as_not_cached() {
|
||||||
|
let old = serde_json::json!({
|
||||||
|
"meta_rev": "abc",
|
||||||
|
"store_path": "/nix/store/x",
|
||||||
|
"system": "x86_64-linux",
|
||||||
|
"built_unix": 7,
|
||||||
|
});
|
||||||
|
let record = serde_json::from_value::<AgentClosure>(old).expect("decodes");
|
||||||
|
assert!(!record.cached);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -140,8 +140,16 @@ pub struct BuildFromMeta {
|
||||||
#[serde(tag = "state", rename_all = "kebab-case")]
|
#[serde(tag = "state", rename_all = "kebab-case")]
|
||||||
pub enum FromMetaReply {
|
pub enum FromMetaReply {
|
||||||
/// The toplevel built; `out_path` is its store path in the builder host's
|
/// The toplevel built; `out_path` is its store path in the builder host's
|
||||||
/// store and `system` the platform the builder host builds for.
|
/// store and `system` the platform the builder host builds for. `cached`
|
||||||
Built { out_path: String, system: String },
|
/// is whether the builder pushed the closure to the binary cache; a failed
|
||||||
|
/// push leaves the build green with `cached` false. A builder that predates
|
||||||
|
/// the field answers without it, which reads as false.
|
||||||
|
Built {
|
||||||
|
out_path: String,
|
||||||
|
system: String,
|
||||||
|
#[serde(default)]
|
||||||
|
cached: bool,
|
||||||
|
},
|
||||||
/// The build did not succeed: the last lines `nix` printed, or why it
|
/// The build did not succeed: the last lines `nix` printed, or why it
|
||||||
/// stopped (it timed out, or could not start). The controller holds the
|
/// stopped (it timed out, or could not start). The controller holds the
|
||||||
/// deploy back and builds again on its next round.
|
/// deploy back and builds again on its next round.
|
||||||
|
|
@ -253,8 +261,14 @@ mod tests {
|
||||||
FromMetaReply::Built {
|
FromMetaReply::Built {
|
||||||
out_path: "/nix/store/x".to_owned(),
|
out_path: "/nix/store/x".to_owned(),
|
||||||
system: "x86_64-linux".to_owned(),
|
system: "x86_64-linux".to_owned(),
|
||||||
|
cached: true,
|
||||||
},
|
},
|
||||||
json!({"state": "built", "out_path": "/nix/store/x", "system": "x86_64-linux"}),
|
json!({
|
||||||
|
"state": "built",
|
||||||
|
"out_path": "/nix/store/x",
|
||||||
|
"system": "x86_64-linux",
|
||||||
|
"cached": true,
|
||||||
|
}),
|
||||||
),
|
),
|
||||||
(
|
(
|
||||||
FromMetaReply::Failed {
|
FromMetaReply::Failed {
|
||||||
|
|
@ -277,6 +291,19 @@ mod tests {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_built_reply_from_a_builder_without_the_field_reads_as_not_cached() {
|
||||||
|
let old = json!({"state": "built", "out_path": "/nix/store/x", "system": "s"});
|
||||||
|
assert_eq!(
|
||||||
|
serde_json::from_value::<FromMetaReply>(old).expect("decodes"),
|
||||||
|
FromMetaReply::Built {
|
||||||
|
out_path: "/nix/store/x".to_owned(),
|
||||||
|
system: "s".to_owned(),
|
||||||
|
cached: false,
|
||||||
|
}
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn the_subjects_are_pinned() {
|
fn the_subjects_are_pinned() {
|
||||||
assert_eq!(SUBJECT, "$SWARM.builder.build");
|
assert_eq!(SUBJECT, "$SWARM.builder.build");
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue