hive-forge: mirror actions/upload-artifact alongside actions/checkout
With actionsFromSelf, Forgejo resolves bare `uses: actions/<name>` against
the forge itself, so every action a workflow names needs a mirror there.
Only actions/checkout was seeded, and coverage.yml's
`actions/upload-artifact@v3` has failed at job setup with "repository not
found" since the CI runner moved to swarm-ci-runner (e2da5509).
The single hardcoded checkout mirror becomes a list of action names both
share, mirrored from github.com/actions/* like checkout already was. v3
points at the same commit (ff15f03) on github.com and code.forgejo.org.
The controller's forge-objects pass creates any missing mirror on every
pass, and the changed SWARM_CONTROLLER_FORGE_MIRRORS env restarts the unit
on switch, so an existing forge picks the mirror up on deploy.
Closes #5040
This commit is contained in:
parent
b874d21cda
commit
9b711d5ffa
5 changed files with 51 additions and 37 deletions
|
|
@ -201,25 +201,27 @@ the forge admin panel as offline until an admin deletes it. The runner the old
|
||||||
under its own name and token. Delete it under Site administration, Actions,
|
under its own name and token. Delete it under Site administration, Actions,
|
||||||
Runners.
|
Runners.
|
||||||
|
|
||||||
## Actions checkout mirror
|
## Actions mirrors
|
||||||
|
|
||||||
<!-- vale write-good.Passive = NO -->
|
<!-- vale write-good.Passive = NO -->
|
||||||
|
|
||||||
With `services.hyperhive.deploy.forgejo.actionsFromSelf` enabled, the swarm
|
With `services.hyperhive.deploy.forgejo.actionsFromSelf` enabled, the swarm
|
||||||
controller seeds an `actions/checkout` pull-mirror on the forge and Forgejo's
|
controller seeds `actions/checkout` and `actions/upload-artifact` pull-mirrors
|
||||||
`DEFAULT_ACTIONS_URL` points at the forge itself. CI
|
(from `github.com/actions/*`) on the forge and Forgejo's `DEFAULT_ACTIONS_URL`
|
||||||
`uses: actions/checkout@vN` steps then resolve on the forge, with no external
|
points at the forge itself. CI `uses: actions/<name>@vN` steps then resolve on
|
||||||
DNS on the CI critical path.
|
the forge, with no external DNS on the CI critical path. A workflow that names
|
||||||
|
any other bare `actions/*` action fails at job setup with "repository not
|
||||||
|
found" until that action joins the list in `nix/host-modules/hive-forge/default.nix`.
|
||||||
|
|
||||||
A forge host that runs the swarm controller therefore resolves
|
A forge host that runs the swarm controller therefore resolves those actions
|
||||||
`uses: actions/checkout` from itself, with or without a runner on that host.
|
from itself, with or without a runner on that host.
|
||||||
Set `actionsFromSelf = false` there to resolve actions from `data.forgejo.org`
|
Set `actionsFromSelf = false` there to resolve actions from `data.forgejo.org`
|
||||||
and seed no mirror.
|
and seed no mirror.
|
||||||
|
|
||||||
The option defaults to whether the forge's host also runs the swarm controller
|
The option defaults to whether the forge's host also runs the swarm controller
|
||||||
(`services.hyperhive.deploy.swarm-controller.enable`): on when it does, off
|
(`services.hyperhive.deploy.swarm-controller.enable`): on when it does, off
|
||||||
otherwise. Setting it `true` on a forge host without the controller leaves
|
otherwise. Setting it `true` on a forge host without the controller leaves
|
||||||
`actions/checkout` unresolved, because nothing seeds the mirror; evaluation
|
the actions unresolved, because nothing seeds the mirrors; evaluation
|
||||||
warns about it.
|
warns about it.
|
||||||
|
|
||||||
<!-- vale write-good.Passive = YES -->
|
<!-- vale write-good.Passive = YES -->
|
||||||
|
|
|
||||||
|
|
@ -283,7 +283,8 @@ one per hive. It ensures them at start and every five minutes after
|
||||||
whitelists = the `operators` team, and no user. The controller leaves a
|
whitelists = the `operators` team, and no user. The controller leaves a
|
||||||
repo with no `main` rule alone;
|
repo with no `main` rule alone;
|
||||||
- the pull-mirrors from `deploy.forgejo.mirrors` on the controller's
|
- the pull-mirrors from `deploy.forgejo.mirrors` on the controller's
|
||||||
host (with the `actions/checkout` one `deploy.forgejo.actionsFromSelf` adds);
|
host (with the `actions/checkout` and `actions/upload-artifact` ones
|
||||||
|
`deploy.forgejo.actionsFromSelf` adds);
|
||||||
- `internal/docs` (private) and `internal/knowledge` (public, with a
|
- `internal/docs` (private) and `internal/knowledge` (public, with a
|
||||||
seed `README.md` while empty);
|
seed `README.md` while empty);
|
||||||
- the `agent-configs` org avatar
|
- the `agent-configs` org avatar
|
||||||
|
|
|
||||||
|
|
@ -88,18 +88,24 @@ let
|
||||||
effectiveRootUrl = if cfg.rootUrl != null then cfg.rootUrl else defaultRootUrl;
|
effectiveRootUrl = if cfg.rootUrl != null then cfg.rootUrl else defaultRootUrl;
|
||||||
|
|
||||||
# `actionsFromSelf` resolves `uses:` against this forge, which then has to
|
# `actionsFromSelf` resolves `uses:` against this forge, which then has to
|
||||||
# hold the action.
|
# hold every action a workflow in `.forgejo/workflows/` names.
|
||||||
actionsFromSelf = deployCfg.forgejo.actionsFromSelf;
|
actionsFromSelf = deployCfg.forgejo.actionsFromSelf;
|
||||||
actionCheckoutMirror = {
|
actionMirrors =
|
||||||
upstream = "https://github.com/actions/checkout";
|
map
|
||||||
dest = "actions/checkout";
|
(name: {
|
||||||
};
|
upstream = "https://github.com/actions/${name}";
|
||||||
# Unless the operator already declared that dest, which would duplicate it.
|
dest = "actions/${name}";
|
||||||
|
})
|
||||||
|
[
|
||||||
|
"checkout"
|
||||||
|
"upload-artifact"
|
||||||
|
];
|
||||||
|
# Minus any dest the operator already declared, which would duplicate it.
|
||||||
effectiveMirrors =
|
effectiveMirrors =
|
||||||
deployCfg.forgejo.mirrors
|
deployCfg.forgejo.mirrors
|
||||||
++ lib.optional (
|
++ lib.optionals actionsFromSelf (
|
||||||
actionsFromSelf && !(lib.any (m: m.dest == actionCheckoutMirror.dest) deployCfg.forgejo.mirrors)
|
lib.filter (a: !(lib.any (m: m.dest == a.dest) deployCfg.forgejo.mirrors)) actionMirrors
|
||||||
) actionCheckoutMirror;
|
);
|
||||||
|
|
||||||
# Share host netns — forgejo's HTTP / SSH listeners then look
|
# Share host netns — forgejo's HTTP / SSH listeners then look
|
||||||
# exactly like a host-side service, no port forwarding dance,
|
# exactly like a host-side service, no port forwarding dance,
|
||||||
|
|
@ -199,8 +205,9 @@ in
|
||||||
failure on whatever reads it.
|
failure on whatever reads it.
|
||||||
|
|
||||||
With {option}`services.hyperhive.deploy.forgejo.actionsFromSelf`
|
With {option}`services.hyperhive.deploy.forgejo.actionsFromSelf`
|
||||||
enabled (the default on a host that runs the swarm-controller), an
|
enabled (the default on a host that runs the swarm-controller),
|
||||||
`actions/checkout` mirror is appended to this list.
|
`actions/checkout` and `actions/upload-artifact` mirrors are appended
|
||||||
|
to this list.
|
||||||
'';
|
'';
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|
@ -211,17 +218,16 @@ in
|
||||||
example = false;
|
example = false;
|
||||||
description = ''
|
description = ''
|
||||||
Resolve a workflow's `uses: <org>/<action>@vN` against this forge
|
Resolve a workflow's `uses: <org>/<action>@vN` against this forge
|
||||||
instead of `data.forgejo.org`, and add an `actions/checkout`
|
instead of `data.forgejo.org`, and add `actions/checkout` and
|
||||||
pull-mirror to {option}`services.hyperhive.deploy.forgejo.mirrors`
|
`actions/upload-artifact` pull-mirrors to
|
||||||
so the checkout step has something to resolve to. A CI job's
|
{option}`services.hyperhive.deploy.forgejo.mirrors` so those steps
|
||||||
checkout then needs no external DNS; only the mirror's re-sync
|
have something to resolve to. A CI job's actions then need no
|
||||||
does.
|
external DNS; only the mirrors' re-sync does.
|
||||||
|
|
||||||
Defaults to on when this host runs the swarm-controller, which seeds
|
Defaults to on when this host runs the swarm-controller, which seeds
|
||||||
the `actions/checkout` mirror, and off otherwise. Setting it `true` on
|
the mirrors, and off otherwise. Setting it `true` on a host without
|
||||||
a host without the controller leaves `actions/checkout` unresolved:
|
the controller leaves both actions unresolved: nothing seeds the
|
||||||
nothing seeds the mirror, and the warning about unseeded mirrors
|
mirrors, and the warning about unseeded mirrors fires.
|
||||||
fires.
|
|
||||||
|
|
||||||
Any other action a workflow names has to be mirrored here too. Set to
|
Any other action a workflow names has to be mirrored here too. Set to
|
||||||
`false` to resolve actions from `data.forgejo.org` and seed no mirror.
|
`false` to resolve actions from `data.forgejo.org` and seed no mirror.
|
||||||
|
|
@ -663,7 +669,7 @@ in
|
||||||
# reads it.
|
# reads it.
|
||||||
actions.ENABLED = true;
|
actions.ENABLED = true;
|
||||||
# With `actionsFromSelf`, resolve `uses: <org>/<action>@vN` from
|
# With `actionsFromSelf`, resolve `uses: <org>/<action>@vN` from
|
||||||
# THIS instance (the seeded `actions/checkout` pull-mirror)
|
# THIS instance (the seeded `actions/*` pull-mirrors)
|
||||||
# instead of the upstream default `data.forgejo.org` — keeps
|
# instead of the upstream default `data.forgejo.org` — keeps
|
||||||
# the checkout step on loopback, immune to a host-resolver
|
# the checkout step on loopback, immune to a host-resolver
|
||||||
# blip. `self` = forgejo expands actions against its
|
# blip. `self` = forgejo expands actions against its
|
||||||
|
|
@ -1196,7 +1202,7 @@ in
|
||||||
systemd.services.swarm-controller.environment.SWARM_CONTROLLER_FORGE_MIRRORS =
|
systemd.services.swarm-controller.environment.SWARM_CONTROLLER_FORGE_MIRRORS =
|
||||||
lib.mkIf deployCfg.swarm-controller.enable (builtins.toJSON effectiveMirrors);
|
lib.mkIf deployCfg.swarm-controller.enable (builtins.toJSON effectiveMirrors);
|
||||||
warnings = lib.optional (effectiveMirrors != [ ] && !deployCfg.swarm-controller.enable) ''
|
warnings = lib.optional (effectiveMirrors != [ ] && !deployCfg.swarm-controller.enable) ''
|
||||||
services.hyperhive.deploy.forgejo.mirrors (or the actions/checkout mirror
|
services.hyperhive.deploy.forgejo.mirrors (or the actions/* mirrors
|
||||||
deploy.forgejo.actionsFromSelf adds) is set on a host that does not run the
|
deploy.forgejo.actionsFromSelf adds) is set on a host that does not run the
|
||||||
swarm-controller. The controller seeds mirrors from its own host's list,
|
swarm-controller. The controller seeds mirrors from its own host's list,
|
||||||
so these are not seeded: declare them on the swarm-controller's host.
|
so these are not seeded: declare them on the swarm-controller's host.
|
||||||
|
|
|
||||||
|
|
@ -479,9 +479,10 @@ let
|
||||||
warns mirrorsNoController && !(warns controllerWithMirrors);
|
warns mirrorsNoController && !(warns controllerWithMirrors);
|
||||||
}
|
}
|
||||||
{
|
{
|
||||||
# The mirror is what `self` resolves `actions/checkout` to, so the two
|
# The mirrors are what `self` resolves `actions/*` to, so the two come
|
||||||
# come together. The control is the same forge without the option.
|
# together: one per action `.forgejo/workflows/` names. The control is
|
||||||
name = "actionsFromSelf adds the checkout mirror and resolves actions from the forge";
|
# the same forge without the option.
|
||||||
|
name = "actionsFromSelf adds the actions mirrors and resolves actions from the forge";
|
||||||
ok =
|
ok =
|
||||||
let
|
let
|
||||||
mirrorsOf =
|
mirrorsOf =
|
||||||
|
|
@ -489,7 +490,11 @@ let
|
||||||
actionsUrlOf =
|
actionsUrlOf =
|
||||||
m: m.containers.hive-forge.config.services.forgejo.settings.actions.DEFAULT_ACTIONS_URL or null;
|
m: m.containers.hive-forge.config.services.forgejo.settings.actions.DEFAULT_ACTIONS_URL or null;
|
||||||
in
|
in
|
||||||
lib.elem "actions/checkout" (map (m: m.dest) (mirrorsOf controllerWithSelfActions))
|
map (m: m.dest) (mirrorsOf controllerWithSelfActions) == [
|
||||||
|
"mirrors/tool"
|
||||||
|
"actions/checkout"
|
||||||
|
"actions/upload-artifact"
|
||||||
|
]
|
||||||
&& actionsUrlOf controllerWithSelfActions == "self"
|
&& actionsUrlOf controllerWithSelfActions == "self"
|
||||||
&& mirrorsOf controllerWithMirrors == [ aMirror ]
|
&& mirrorsOf controllerWithMirrors == [ aMirror ]
|
||||||
&& actionsUrlOf controllerWithMirrors == null;
|
&& actionsUrlOf controllerWithMirrors == null;
|
||||||
|
|
|
||||||
|
|
@ -117,7 +117,7 @@ Do **not** push directly to `main` — agents have read-only access.
|
||||||
";
|
";
|
||||||
|
|
||||||
/// The operator-declared pull-mirrors (nix `deploy.forgejo.mirrors`, plus the
|
/// The operator-declared pull-mirrors (nix `deploy.forgejo.mirrors`, plus the
|
||||||
/// `actions/checkout` entry `deploy.forgejo.actionsFromSelf` adds), JSON-encoded by `hive-forge/default.nix`.
|
/// `actions/*` entries `deploy.forgejo.actionsFromSelf` adds), JSON-encoded by `hive-forge/default.nix`.
|
||||||
/// Absent or empty means no mirrors.
|
/// Absent or empty means no mirrors.
|
||||||
const MIRRORS_ENV: &str = "SWARM_CONTROLLER_FORGE_MIRRORS";
|
const MIRRORS_ENV: &str = "SWARM_CONTROLLER_FORGE_MIRRORS";
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue