forge: its own netns, HTTP on a unix socket, git over HTTPS only
The hive-forge container leaves the host netns and listens on no TCP port: Forgejo serves HTTP on a unix socket the gateway proxies to, and the built-in SSH server is off. The httpPort, sshPort and openFirewall options are removed with a message each.
This commit is contained in:
parent
e1434e677b
commit
fc70f538ac
6 changed files with 153 additions and 117 deletions
|
|
@ -157,31 +157,19 @@ Agents poll `HIVE_FORGE_URL` for Forgejo notifications and run every `hive-forge
|
|||
|
||||
## hive-forge container shape
|
||||
|
||||
Private Forgejo in a nixos-container named `hive-forge` (not `h-*`, so c0re's lifecycle scanner leaves it alone; manage it with the standard `nixos-container` CLI). The container keeps it from colliding with any `services.forgejo` the operator already runs on the host: separate systemd namespace, separate state dir, separate port.
|
||||
Private Forgejo in a nixos-container named `hive-forge` (not `h-*`, so c0re's lifecycle scanner leaves it alone; manage it with the standard `nixos-container` CLI). The container keeps it from colliding with any `services.forgejo` the operator already runs on the host: separate systemd namespace, separate state dir, no host port.
|
||||
|
||||
It shares the host network namespace (`privateNetwork = false`): the container is for state and unit isolation, not network isolation. Agent containers, by contrast, are network-isolated and reach the forge through the gateway ([`HIVE_FORGE_URL`](#hive_forge_url-agents-reach-the-forge-via-the-gateway-by-domain)).
|
||||
It has a network namespace of its own on the hive bridge (`privateNetwork = true`), like the other service containers. It listens on no TCP port at all: Forgejo serves HTTP on a unix socket, `/run/hive-forge/forgejo.sock`, in a directory the host bind-mounts into the container (mode `0770`, group `nginx`, so only the gateway and Forgejo itself get in). The vhost's `proxyPass` and the `= /metrics` location both name that socket. Agent containers reach the forge the same way an operator's browser does, through the gateway ([`HIVE_FORGE_URL`](#hive_forge_url-agents-reach-the-forge-via-the-gateway-by-domain)).
|
||||
|
||||
State lives at `/var/lib/nixos-containers/hive-forge/var/lib/forgejo/` and survives restarts and reboots; destroy the container to wipe it.
|
||||
|
||||
Only the swarm's forge host runs it (`services.hyperhive.deploy.forgejo.enable`, see [`2-handbook/swarm/services.md`](../../2-handbook/swarm/services.md)). Every other hive reaches that host's gateway by `forge.<swarm-domain>`.
|
||||
|
||||
### Network and port configuration
|
||||
### Git over HTTPS only
|
||||
|
||||
```nix
|
||||
# The SSH port is swarm-wide: every hive dials it. The HTTP port and whether
|
||||
# THIS host opens them in its firewall are this host's deployment decisions.
|
||||
services.hyperhive = {
|
||||
swarm.forge.sshPort = 2222; # default — git-over-SSH, off the host's openssh on 22
|
||||
deploy.forgejo = {
|
||||
httpPort = 3000; # default — outside hyperhive's 7000 / 8100-8999
|
||||
openFirewall = false; # default
|
||||
};
|
||||
};
|
||||
```
|
||||
The forge has no SSH server and no port options. Clone, push and pull over `https://<domain>/<owner>/<repo>.git`, through the gateway's vhost; no clone URL the forge shows offers SSH. Whether the gateway answers from outside the host is the gateway's own `openFirewall` option.
|
||||
|
||||
`sshPort` serves `git clone/push/pull` over SSH (`git@<domain>:owner/repo.git` with `-p 2222`). SSH goes straight to Forgejo, not through nginx.
|
||||
|
||||
`openFirewall` (default `false`) opens `httpPort` and `sshPort` on the host. Agents don't need it — they come through the gateway. Set it for a browser reaching `http://<host>:<httpPort>/` directly, or for external git clients pushing over SSH. Forge's gateway vhost doesn't need `openFirewall` — the gateway's own `openFirewall` option covers that path.
|
||||
The old `swarm.forge.sshPort`, `swarm.forge.httpPort`, `deploy.forgejo.httpPort` and `deploy.forgejo.openFirewall` options no longer exist. A configuration that still sets one fails at evaluation with a message saying so.
|
||||
|
||||
### `rootUrl` override
|
||||
|
||||
|
|
|
|||
|
|
@ -22,12 +22,11 @@ that touches it.
|
|||
│ uplink NIC — NAT MASQUERADE for the
|
||||
│ bridge subnet (10.42.0.0/24 default)
|
||||
┌──────────────────────────┴─────────────────────────────────────────┐
|
||||
│ host netns — the host itself plus gateway / forge │
|
||||
│ host netns — the host itself plus the gateway │
|
||||
│ │
|
||||
│ nginx :80/:443 [hive-gateway] │
|
||||
│ dnsmasq 10.42.0.1:53 (DNS) + :67 (DHCP) [hive-gateway] │
|
||||
│ squid 10.42.0.1:3128 (egress proxy) [hive-squid] │
|
||||
│ forgejo :3000 http, :2222 git-ssh [hive-forge] │
|
||||
│ hive-c0re dashboard 127.0.0.1:7000 (host service) │
|
||||
│ │
|
||||
│ hive-br0 10.42.0.1/24 │
|
||||
|
|
@ -45,7 +44,7 @@ that touches it.
|
|||
| ------------------------- | ----------------------- | -------------- | ----------------------------------------------------------------------------------------------- |
|
||||
| gateway (host) | host | host addresses | nginx `:80`/`:443` (every vhost); dnsmasq `bridgeIp:53` + DHCP `:67` on the bridge |
|
||||
| egress proxy (host) | host | `bridgeIp` | squid `bridgeIp:3128`; agents with a credential send HTTP and HTTPS through it |
|
||||
| `hive-forge` | host (shared) | host addresses | forgejo `:3000` http, `:2222` git-ssh; fronted by the `forge.<swarm-domain>` vhost |
|
||||
| `hive-forge` | private, veth on bridge | DHCP pool | forgejo on UDS `/run/hive-forge/forgejo.sock` → the `forge.<swarm-domain>` vhost; no SSH |
|
||||
| `hive-matrix` | private, veth on bridge | DHCP pool | tuwunel on UDS `/run/hive-matrix/tuwunel.sock` → the matrix vhost; out to store, SSO, peers |
|
||||
| `swarm-ci-runner` | private, veth on bridge | DHCP pool | outbound only (runner → forge); no inbound surface |
|
||||
| `swarm-review-classifier` | private, veth on bridge | DHCP pool | outbound only (classifier → queue, secret store, swarm collector); no inbound surface |
|
||||
|
|
@ -66,9 +65,8 @@ The flows, end to end:
|
|||
only on the bridge.
|
||||
- **HTTP** — `forge.` and `chat.` (under `swarm.domain`) plus the hive's
|
||||
own dashboard name resolve to the bridge IP, land on nginx
|
||||
`:80`/`:443`, and proxy to forgejo
|
||||
`:3000`, tuwunel's unix socket, hive-c0re `127.0.0.1:7000`, or a per-agent
|
||||
UI unix socket.
|
||||
`:80`/`:443`, and proxy to forgejo's and tuwunel's unix sockets, hive-c0re
|
||||
`127.0.0.1:7000`, or a per-agent UI unix socket.
|
||||
- **Internet egress** — agent default route points at the bridge IP;
|
||||
the host forwards + masquerades out its uplink.
|
||||
- **Swarm** — peer hives reach each other's gateway and forge over HTTPS, and
|
||||
|
|
@ -181,8 +179,8 @@ agent containers.
|
|||
UI proxies, and any other HTTP services.
|
||||
|
||||
The **host** firewall is the only firewall. The swarm service
|
||||
containers that share the host netns (forge, bao, the
|
||||
metrics and log stores) run with `networking.firewall.enable = false`
|
||||
containers that share the host netns (bao, the
|
||||
queue) run with `networking.firewall.enable = false`
|
||||
(`nix/container-modules/swarm-container.nix`): a NixOS firewall inside a
|
||||
shared-netns container runs against the _host_ ruleset — at container
|
||||
boot its `firewall-start` flushes the `nixos-fw` chains, rebuilds them
|
||||
|
|
|
|||
|
|
@ -527,10 +527,6 @@ in
|
|||
[ "services" "hyperhive" "swarm" "otel" "telemetryPort" ]
|
||||
[ "services" "hyperhive" "deploy" "swarm-otel" "telemetryPort" ]
|
||||
)
|
||||
(lib.mkRenamedOptionModule
|
||||
[ "services" "hyperhive" "swarm" "forge" "httpPort" ]
|
||||
[ "services" "hyperhive" "deploy" "forgejo" "httpPort" ]
|
||||
)
|
||||
(lib.mkRenamedOptionModule
|
||||
[ "services" "hyperhive" "swarm" "matrix" "allowEncryption" ]
|
||||
[ "services" "hyperhive" "deploy" "matrix" "allowEncryption" ]
|
||||
|
|
|
|||
|
|
@ -95,8 +95,7 @@ let
|
|||
forwarderClientId = cfg.otel.clientId;
|
||||
forwarderEndpoint = "https://${config.services.hyperhive.swarm.otel.domain}/hive-forge";
|
||||
forwarderUnit = "swarm-journal-forwarder-oidc";
|
||||
# In the host netns: distinct from every other collector's there, which the
|
||||
# assertion below checks.
|
||||
# Inside the forge's own netns, so no other collector's port can collide.
|
||||
forwarderTelemetryPort = 8898;
|
||||
|
||||
# The forge host's store identity, handed into the container as system
|
||||
|
|
@ -156,15 +155,22 @@ let
|
|||
lib.optionals actionsFromSelf actionMirrors ++ hyperhiveMirrors
|
||||
);
|
||||
|
||||
# Share host netns — forgejo's HTTP / SSH listeners then look
|
||||
# exactly like a host-side service, no port forwarding dance,
|
||||
# and agent containers (which also share host netns) reach it
|
||||
# via plain `localhost`.
|
||||
privateNetwork = false;
|
||||
# The forge's own netns, on the hive bridge. It listens on no TCP port at
|
||||
# all: the gateway reaches it through `socketPath`, and nothing else does
|
||||
# (git goes over HTTPS through the gateway; there is no SSH server).
|
||||
privateNetwork = true;
|
||||
|
||||
# The socket directory, shared with the host so nginx can reach in. Fixed,
|
||||
# not an option: the directory's mode is what keeps the socket private, so
|
||||
# it holds this one socket and nothing else. Same shape as
|
||||
# ../hive-matrix.nix.
|
||||
socketDir = "/run/hive-forge";
|
||||
socketPath = "${socketDir}/forgejo.sock";
|
||||
nginxGid = config.ids.gids.nginx;
|
||||
in
|
||||
{
|
||||
# Private Forgejo in a `hive-forge` nixos-container, shared host
|
||||
# netns, on the one host with `deploy.forgejo.enable`. Agents reach it
|
||||
# Private Forgejo in a `hive-forge` nixos-container on the hive bridge,
|
||||
# on the one host with `deploy.forgejo.enable`. Agents reach it
|
||||
# at `forge.<domain>` via the gateway. State at
|
||||
# `/var/lib/nixos-containers/hive-forge/var/lib/forgejo/` survives
|
||||
# restart. See `docs/4-implementation/networking/gateway.md::hive-forge container shape`.
|
||||
|
|
@ -174,6 +180,16 @@ in
|
|||
# Set on every host, not only the forge's: the controller's host forwards the
|
||||
# list whether or not the forge runs beside it.
|
||||
{ services.hyperhive.deploy.forgejo.effectiveMirrors = effectiveMirrors; }
|
||||
(lib.mkRemovedOptionModule [ "services" "hyperhive" "deploy" "forgejo" "httpPort" ] ''
|
||||
The forge listens on a unix socket inside its own network namespace, so
|
||||
it has no TCP port to name. Remove this definition.
|
||||
'')
|
||||
(lib.mkRemovedOptionModule [ "services" "hyperhive" "deploy" "forgejo" "openFirewall" ] ''
|
||||
The forge has no host port to open: the gateway reaches it through a
|
||||
unix socket, and git goes over HTTPS through the gateway's vhost, whose
|
||||
firewall switch is `services.hyperhive.gateway.openFirewall`. Remove
|
||||
this definition.
|
||||
'')
|
||||
(lib.mkRemovedOptionModule [ "services" "hyperhive" "deploy" "forgejo" "sso" "clientSecretFile" ] ''
|
||||
The forge's OIDC client secret is fetched from the swarm secret store
|
||||
(swarm-bao-forge-oidc) into a fixed runtime path. Remove this
|
||||
|
|
@ -197,17 +213,6 @@ in
|
|||
# where its host-local secrets sit. Same rule as ../swarm-victorialogs.nix,
|
||||
# and the renames are in ../deploy.nix with the rest.
|
||||
options.services.hyperhive.deploy.forgejo = {
|
||||
httpPort = lib.mkOption {
|
||||
type = lib.types.port;
|
||||
default = 3000;
|
||||
description = ''
|
||||
TCP port the forge serves HTTP on. Default 3000 sits outside
|
||||
hyperhive's claimed ranges (dashboard 7000, every agent in
|
||||
8100..8999 via FNV-1a hash). Change this if you already have
|
||||
another forgejo bound to 3000.
|
||||
'';
|
||||
};
|
||||
|
||||
package = lib.mkOption {
|
||||
type = lib.types.package;
|
||||
default = pkgs.forgejo;
|
||||
|
|
@ -223,21 +228,6 @@ in
|
|||
'';
|
||||
};
|
||||
|
||||
openFirewall = lib.mkOption {
|
||||
type = lib.types.bool;
|
||||
default = false;
|
||||
example = true;
|
||||
description = ''
|
||||
Open `httpPort` + `sshPort` in the host firewall. Off by
|
||||
default (secure-by-default): agent containers reach the forge
|
||||
at `forge.<domain>` via the gateway (not directly), and the
|
||||
host reaches it on loopback — so the firewall opens only
|
||||
matter for access from outside the host. Flip to `true` when
|
||||
you want the operator's browser or external git clients to
|
||||
hit the forge directly.
|
||||
'';
|
||||
};
|
||||
|
||||
mirrors = lib.mkOption {
|
||||
type = lib.types.listOf (
|
||||
lib.types.submodule {
|
||||
|
|
@ -349,13 +339,13 @@ in
|
|||
|
||||
# `server_name = forge.domain`, proxies all `/` → forgejo. Tuned for
|
||||
# git: `client_max_body_size 1G`, `proxy_read_timeout 1h` (multi-GB
|
||||
# clones). SSH stays direct on `forge.sshPort`. See
|
||||
# `docs/4-implementation/networking/gateway.md`.
|
||||
# clones). Git goes over HTTPS through this vhost; there is no SSH
|
||||
# server. See `docs/4-implementation/networking/gateway.md`.
|
||||
services.hyperhive.gateway.vhosts = {
|
||||
"${cfg.domain}" = {
|
||||
locations = {
|
||||
"/" = {
|
||||
proxyPass = "http://127.0.0.1:${toString deployCfg.forgejo.httpPort}/";
|
||||
proxyPass = "http://unix:${socketPath}:/";
|
||||
proxyWebsockets = true;
|
||||
extraConfig = ''
|
||||
proxy_buffering off;
|
||||
|
|
@ -376,7 +366,7 @@ in
|
|||
# `publishedScrapeTargets.forgejo` in ../swarm-otel-service.nix;
|
||||
# the two must agree, since that string is also the token's audience.
|
||||
"= /metrics" = {
|
||||
proxyPass = "http://127.0.0.1:${toString deployCfg.forgejo.httpPort}/metrics";
|
||||
proxyPass = "http://unix:${socketPath}:/metrics";
|
||||
extraConfig = ''
|
||||
auth_request /__forge_metrics_authz;
|
||||
'';
|
||||
|
|
@ -500,20 +490,6 @@ in
|
|||
Declare it in swarm-otel-service.nix.
|
||||
'';
|
||||
}
|
||||
{
|
||||
assertion =
|
||||
!(lib.elem forwarderTelemetryPort [
|
||||
config.services.hyperhive.otel.telemetryPort
|
||||
config.services.hyperhive.deploy.bao.otel.telemetryPort
|
||||
config.services.hyperhive.deploy.nats.otel.telemetryPort
|
||||
]);
|
||||
message = ''
|
||||
The forge's journal forwarder's telemetry port (${toString forwarderTelemetryPort})
|
||||
collides with another collector's in the host netns: the hive
|
||||
collector's, the store forwarder's or the
|
||||
queue forwarder's.
|
||||
'';
|
||||
}
|
||||
];
|
||||
|
||||
# `caTrust.containerOrdering` orders this unit after `hive-tls-ca.service`
|
||||
|
|
@ -538,6 +514,7 @@ in
|
|||
# container's own forwarder reads it (../../container-modules/swarm-journal-forwarder.nix).
|
||||
extraFlags = lib.mapAttrsToList (id: source: "--load-credential=${id}:${source}") storeFiles;
|
||||
inherit privateNetwork;
|
||||
hostBridge = networkCfg.bridgeName;
|
||||
# Self-signed mode: bind the hive's trust bundle read-only so forgejo
|
||||
# can trust the gateway's self-signed leaf for outbound webhook
|
||||
# delivery. The bind-mount, the `container@` ordering and the bundle
|
||||
|
|
@ -547,11 +524,19 @@ in
|
|||
# The OIDC secret's directory: a container's `/run` is an nspawn tmpfs
|
||||
# the host cannot see into. The host directory comes from the tmpfiles
|
||||
# rule below, so it exists before the container starts.
|
||||
#
|
||||
# The socket directory: a bind mount rather than a path both sides
|
||||
# happen to know, because a container's `/run` is an nspawn tmpfs the host
|
||||
# cannot see into. The tmpfiles rule below creates it host-side first.
|
||||
bindMounts = caTrust.bindMount // {
|
||||
${oidcSecretDir} = {
|
||||
hostPath = oidcSecretDir;
|
||||
isReadOnly = true;
|
||||
};
|
||||
${socketDir} = {
|
||||
hostPath = socketDir;
|
||||
isReadOnly = false;
|
||||
};
|
||||
};
|
||||
config =
|
||||
{ pkgs, ... }:
|
||||
|
|
@ -631,6 +616,19 @@ in
|
|||
system.stateVersion = "25.11";
|
||||
services.hyperhive.swarmContainer = { inherit privateNetwork; };
|
||||
|
||||
# The gateway's dnsmasq on the bridge answers every name in
|
||||
# `gateway.localNames` with the bridge IP and forwards the rest
|
||||
# upstream; the address itself comes from the gateway's DHCP.
|
||||
networking.nameservers = [ networkCfg.bridgeIp ];
|
||||
networking.interfaces.eth0.useDHCP = true;
|
||||
|
||||
# The group of the socket directory. A container has its own user
|
||||
# database, so only the gid crosses; the name is local. Forgejo is a
|
||||
# member, which lets it create its socket in the directory and keeps
|
||||
# every other uid out of it.
|
||||
users.groups.gateway-nginx.gid = nginxGid;
|
||||
users.users.forgejo.extraGroups = [ "gateway-nginx" ];
|
||||
|
||||
# Authenticated as the forwarder's own OIDC client, with the forge
|
||||
# host's store identity: one identity for the whole container. It is
|
||||
# a system credential, so the fetch unit loads it.
|
||||
|
|
@ -663,12 +661,18 @@ in
|
|||
server = {
|
||||
DOMAIN = cfg.domain;
|
||||
ROOT_URL = effectiveRootUrl;
|
||||
HTTP_PORT = deployCfg.forgejo.httpPort;
|
||||
START_SSH_SERVER = true;
|
||||
SSH_PORT = cfg.sshPort;
|
||||
SSH_LISTEN_PORT = cfg.sshPort;
|
||||
BUILTIN_SSH_SERVER_USER = "git";
|
||||
DISABLE_SSH = false;
|
||||
# A unix socket, not a port. `0666` is safe because the
|
||||
# directory it sits in is `0770` for the gateway's group only
|
||||
# (the tmpfiles rule on the host). Forgejo's primary group is
|
||||
# its own, so a narrower mode would shut nginx out.
|
||||
PROTOCOL = "http+unix";
|
||||
HTTP_ADDR = socketPath;
|
||||
UNIX_SOCKET_PERMISSION = "666";
|
||||
# No SSH: git goes over HTTPS through the gateway. With the
|
||||
# built-in server off and `DISABLE_SSH` set, no clone URL
|
||||
# offers it.
|
||||
START_SSH_SERVER = false;
|
||||
DISABLE_SSH = true;
|
||||
# Point forgejo at our extended static root that includes
|
||||
# the custom theme CSS baked straight into the nix store.
|
||||
STATIC_ROOT_PATH = staticRootWithTheme;
|
||||
|
|
@ -858,6 +862,10 @@ in
|
|||
# `ProtectProc = "invisible"` stays, so other processes remain hidden.
|
||||
systemd.services.forgejo.serviceConfig.ProcSubset = lib.mkForce "all";
|
||||
|
||||
# Upstream's unit mounts everything but the state directory
|
||||
# read-only, and the socket lives outside it.
|
||||
systemd.services.forgejo.serviceConfig.ReadWritePaths = [ socketDir ];
|
||||
|
||||
# Ensure Forgejo has a usable GPG signing key so UI merges / CRUD
|
||||
# commits are signed instead of erroring "does not have a signing
|
||||
# key". This service (a) generates a key in forgejo's persistent
|
||||
|
|
@ -1228,9 +1236,10 @@ in
|
|||
forge() {
|
||||
local method="$1" path="$2"
|
||||
curl --silent --show-error --connect-timeout 5 --max-time 15 \
|
||||
--unix-socket ${lib.escapeShellArg socketPath} \
|
||||
-H @<(printf 'Authorization: token %s\n' "$token") \
|
||||
-X "$method" -o "$body" -w '%{http_code}' \
|
||||
"http://127.0.0.1:${toString deployCfg.forgejo.httpPort}/api/v1$path"
|
||||
"http://localhost/api/v1$path"
|
||||
}
|
||||
|
||||
# Every other token this unit minted: one whose write to the
|
||||
|
|
@ -1385,6 +1394,13 @@ in
|
|||
systemd.timers.swarm-bao-forge-oidc = storeRefreshTimer "re-read the forge's OIDC client secret from the swarm secret store";
|
||||
|
||||
systemd.tmpfiles.rules = [
|
||||
# The socket directory, created host-side before the container starts:
|
||||
# nixos-container refuses to start when a bind source is missing.
|
||||
# Root-owned and group nginx, `0770`: forgejo creates its socket here
|
||||
# through the group (its uid is allocated inside the container, so the
|
||||
# host cannot name it), and the gateway traverses it the same way.
|
||||
# Everything else is kept out.
|
||||
"d ${socketDir} 0770 root ${toString nginxGid} - -"
|
||||
# The OIDC secret's directory, which the container binds at start.
|
||||
"d ${oidcSecretDir} 0700 root root -"
|
||||
# Unused: nothing writes or reads these host paths, and they may hold a
|
||||
|
|
@ -1393,13 +1409,6 @@ in
|
|||
"r ${oldOidcSecretPath}"
|
||||
];
|
||||
|
||||
networking.firewall = lib.mkIf deployCfg.forgejo.openFirewall {
|
||||
allowedTCPPorts = [
|
||||
deployCfg.forgejo.httpPort
|
||||
cfg.sshPort
|
||||
];
|
||||
};
|
||||
|
||||
# The mirrors reach a controller on THIS host only (see
|
||||
# ../glue-forge-controller.nix); a controller elsewhere reads its own
|
||||
# host's list, so the warning below says so at eval rather than leaving
|
||||
|
|
|
|||
|
|
@ -29,17 +29,19 @@ in
|
|||
# flake + every agent's config repo (and the `internal/*` repos) — but
|
||||
# only one host runs it: `deploy.forgejo.enable`, in ../deploy.nix. What
|
||||
# follows is what every hive needs to reach it, wherever it runs.
|
||||
options.services.hyperhive.swarm.forge = {
|
||||
sshPort = lib.mkOption {
|
||||
type = lib.types.port;
|
||||
default = 2222;
|
||||
description = ''
|
||||
TCP port the forge's built-in SSH server listens on. Kept off
|
||||
22 so it doesn't clash with the host's openssh. Agents push
|
||||
with `ssh -p <sshPort> git@<domain>:<owner>/<repo>.git`.
|
||||
'';
|
||||
};
|
||||
imports = [
|
||||
(lib.mkRemovedOptionModule [ "services" "hyperhive" "swarm" "forge" "sshPort" ] ''
|
||||
The forge runs no SSH server: git goes over HTTPS through the gateway
|
||||
(`https://<domain>/<owner>/<repo>.git`). Remove this definition, and
|
||||
point any remote that used `ssh -p <sshPort>` at the HTTPS URL.
|
||||
'')
|
||||
(lib.mkRemovedOptionModule [ "services" "hyperhive" "swarm" "forge" "httpPort" ] ''
|
||||
The forge listens on a unix socket inside its own network namespace, so
|
||||
it has no TCP port to name. Remove this definition.
|
||||
'')
|
||||
];
|
||||
|
||||
options.services.hyperhive.swarm.forge = {
|
||||
domain = lib.mkOption {
|
||||
type = lib.types.str;
|
||||
# Under the SWARM domain, not this hive's: a swarm runs one forge
|
||||
|
|
|
|||
|
|
@ -2414,9 +2414,9 @@ let
|
|||
# Each old spelling lands on the new path, with its own rename warning,
|
||||
# and its value reaches the unit or vhost that reads it. Ports are chosen
|
||||
# so two of them collide on purpose: the bao collector's telemetry
|
||||
# port equals the NATS monitor port and the forge forwarder's. Each of the
|
||||
# two colocation assertions must read the moved options to see it. (The log
|
||||
# store has a netns of its own, so its forwarder's port collides with none.)
|
||||
# port equals the NATS monitor port. The colocation assertion must read
|
||||
# the moved options to see it. (The log store and the forge have a netns
|
||||
# of their own, so their forwarders' ports collide with none.)
|
||||
name = "the per-service settings moved to deploy.* still apply from their old paths, with a rename warning each";
|
||||
ok =
|
||||
let
|
||||
|
|
@ -2436,7 +2436,6 @@ let
|
|||
nats.metricsPort = 17777;
|
||||
nats.otel.telemetryPort = 8897;
|
||||
otel.telemetryPort = 18889;
|
||||
forge.httpPort = 13000;
|
||||
matrix.allowEncryption = true;
|
||||
controller.links = [
|
||||
{
|
||||
|
|
@ -2472,7 +2471,6 @@ let
|
|||
&& d.nats.metricsPort == 17777
|
||||
&& d.nats.otel.telemetryPort == 8897
|
||||
&& d.swarm-otel.telemetryPort == 18889
|
||||
&& d.forgejo.httpPort == 13000
|
||||
&& d.matrix.allowEncryption
|
||||
&& d.builder.nixpkgsFlake == "path:/legacy/nixpkgs"
|
||||
&& lib.any (l: l.label == "Legacy" && l.url == "https://legacy.example/") d.swarm-controller.links
|
||||
|
|
@ -2492,7 +2490,6 @@ let
|
|||
old.containers.swarm-builder.config.systemd.services.swarm-builder.environment.SWARM_BUILDER_NIXPKGS_FLAKE
|
||||
== "path:/legacy/nixpkgs"
|
||||
&& (locs sw.nats.domain)."= /metrics".proxyPass == "http://127.0.0.1:17777/metrics"
|
||||
&& (locs sw.forge.domain)."/".proxyPass == "http://127.0.0.1:13000/"
|
||||
&& old.containers.hive-matrix.config.services.matrix-tuwunel.settings.global.allow_encryption
|
||||
&& lib.all warned [
|
||||
"swarm.controller.queue.natsUrl"
|
||||
|
|
@ -2502,7 +2499,6 @@ let
|
|||
"swarm.nats.metricsPort"
|
||||
"swarm.nats.otel.telemetryPort"
|
||||
"swarm.otel.telemetryPort"
|
||||
"swarm.forge.httpPort"
|
||||
"swarm.matrix.allowEncryption"
|
||||
"c0re.nixpkgsFlake"
|
||||
"swarm.controller.links"
|
||||
|
|
@ -2511,9 +2507,7 @@ let
|
|||
]
|
||||
&& !(lib.any (w: lib.hasInfix "swarm.controller.queue" w) allLocal.warnings)
|
||||
&& refused old "deploy.nats.otel.telemetryPort"
|
||||
&& refused old "The forge's journal forwarder's telemetry port"
|
||||
&& !(refused allLocal "deploy.nats.otel.telemetryPort")
|
||||
&& !(refused allLocal "The forge's journal forwarder's telemetry port");
|
||||
&& !(refused allLocal "deploy.nats.otel.telemetryPort");
|
||||
}
|
||||
{
|
||||
# The default render of the moved options, pinned so moving them changes
|
||||
|
|
@ -2537,7 +2531,7 @@ let
|
|||
&&
|
||||
(locs h.swarm.victorialogs.domain)."= /metrics".proxyPass
|
||||
== "http://unix:/run/swarm-victorialogs/victorialogs.sock:/metrics"
|
||||
&& (locs h.swarm.forge.domain)."/".proxyPass == "http://127.0.0.1:3000/"
|
||||
&& (locs h.swarm.forge.domain)."/".proxyPass == "http://unix:/run/hive-forge/forgejo.sock:/"
|
||||
&& !allLocal.containers.hive-matrix.config.services.matrix-tuwunel.settings.global.allow_encryption
|
||||
&& env.SWARM_CONTROLLER_MATRIX_ADMIN_SYNC_IGNORE_USERS == ""
|
||||
&& lib.any (l: l.label == "Forge" && l.url == "https://${h.swarm.forge.domain}/") (
|
||||
|
|
@ -2548,6 +2542,55 @@ let
|
|||
(otelSettings allLocal).receivers.prometheus.config.scrape_configs
|
||||
)).static_configs == [ { targets = [ "127.0.0.1:8889" ]; } ];
|
||||
}
|
||||
{
|
||||
# The forge has a netns of its own and no TCP port: Forgejo serves HTTP on
|
||||
# a unix socket in a directory bind-mounted from the host, the gateway's
|
||||
# `/` and `= /metrics` locations proxy to that socket, and git has no SSH
|
||||
# server. The host-side directory is the gateway group's and no one
|
||||
# else's, which is what makes the socket's `0666` safe.
|
||||
name = "the forge runs in its own netns and serves HTTP on a unix socket, with no SSH";
|
||||
ok =
|
||||
let
|
||||
h = allLocal.services.hyperhive;
|
||||
c = allLocal.containers.hive-forge;
|
||||
srv = c.config.services.forgejo.settings.server;
|
||||
locs = allLocal.services.nginx.virtualHosts.${h.swarm.forge.domain}.locations;
|
||||
socket = "/run/hive-forge/forgejo.sock";
|
||||
nginxGid = toString allLocal.ids.gids.nginx;
|
||||
in
|
||||
c.privateNetwork
|
||||
&& c.hostBridge == h.network.bridgeName
|
||||
&& c.config.services.hyperhive.swarmContainer.privateNetwork
|
||||
&& c.bindMounts."/run/hive-forge".hostPath == "/run/hive-forge"
|
||||
&& !c.bindMounts."/run/hive-forge".isReadOnly
|
||||
&& lib.elem "d /run/hive-forge 0770 root ${nginxGid} - -" allLocal.systemd.tmpfiles.rules
|
||||
&& srv.PROTOCOL == "http+unix"
|
||||
&& srv.HTTP_ADDR == socket
|
||||
&& !srv.START_SSH_SERVER
|
||||
&& srv.DISABLE_SSH
|
||||
&& locs."/".proxyPass == "http://unix:${socket}:/"
|
||||
&& locs."= /metrics".proxyPass == "http://unix:${socket}:/metrics"
|
||||
&& c.config.users.groups.gateway-nginx.gid == allLocal.ids.gids.nginx
|
||||
&& lib.elem "gateway-nginx" c.config.users.users.forgejo.extraGroups
|
||||
&& lib.elem "/run/hive-forge" c.config.systemd.services.forgejo.serviceConfig.ReadWritePaths
|
||||
&& !(lib.elem 3000 allLocal.networking.firewall.allowedTCPPorts)
|
||||
&& !(lib.elem 2222 allLocal.networking.firewall.allowedTCPPorts);
|
||||
}
|
||||
{
|
||||
# The options that named the forge's host ports are gone. A configuration
|
||||
# that still sets one fails at evaluation, and the message says why.
|
||||
name = "setting a removed forge port option is refused with the reason";
|
||||
ok =
|
||||
let
|
||||
refused = m: part: lib.any (a: !a.assertion && lib.hasInfix part a.message) m.assertions;
|
||||
in
|
||||
refused (hiveWithController { swarm.forge.sshPort = 2222; }) "runs no SSH server"
|
||||
&& refused (hiveWithController { swarm.forge.httpPort = 3000; }) "unix socket"
|
||||
&& refused (hiveWithController { deploy.forgejo.httpPort = 3000; }) "unix socket"
|
||||
&& refused (hiveWithController { deploy.forgejo.openFirewall = true; }) "no host port to open"
|
||||
&& !(refused allLocal "runs no SSH server")
|
||||
&& !(refused allLocal "unix socket");
|
||||
}
|
||||
{
|
||||
# Each old spelling lands on the new path, with its own rename warning;
|
||||
# the squid.conf it renders equals the one the new spelling renders, and
|
||||
|
|
|
|||
Loading…
Reference in a new issue