parent
ef156d9d23
commit
220b51d0a3
12 changed files with 204 additions and 105 deletions
|
|
@ -13,8 +13,8 @@ You rarely switch it on yourself. `gateway.enable` defaults to off, and every mo
|
||||||
| URL | upstream | declared by, when |
|
| URL | upstream | declared by, when |
|
||||||
| --- | --- | --- |
|
| --- | --- | --- |
|
||||||
| `<swarm>/` | swarm-ui dist (static), behind an authelia subrequest | `swarm-ui.nix`, `deploy.swarm-controller.enable` |
|
| `<swarm>/` | swarm-ui dist (static), behind an authelia subrequest | `swarm-ui.nix`, `deploy.swarm-controller.enable` |
|
||||||
| `auth.<swarm>/` | authelia (`9091`) | `swarm-authelia.nix`, `deploy.authelia.enable` |
|
| `auth.<swarm>/` | authelia (unix socket `/run/swarm-authelia/authelia.sock`) | `swarm-authelia.nix`, `deploy.authelia.enable` |
|
||||||
| `auth.<swarm>/bridge/*` | swarm-authelia-bridge (`9092`), prefix stripped; answers swarm-controller's client only | `swarm-authelia.nix`, `deploy.authelia.enable` |
|
| `auth.<swarm>/bridge/*` | swarm-authelia-bridge (unix socket `/run/swarm-authelia/bridge.sock`), prefix stripped; answers swarm-controller's client only | `swarm-authelia.nix`, `deploy.authelia.enable` |
|
||||||
| `forge.<swarm>/` | forgejo (`3000`) | `hive-forge/`, `deploy.forgejo.enable` |
|
| `forge.<swarm>/` | forgejo (`3000`) | `hive-forge/`, `deploy.forgejo.enable` |
|
||||||
| `chat.<swarm>/_matrix/*` | tuwunel (unix socket `/run/hive-matrix/tuwunel.sock`) | `hive-matrix.nix`, `swarm.matrix.gatewayHost != null` |
|
| `chat.<swarm>/_matrix/*` | tuwunel (unix socket `/run/hive-matrix/tuwunel.sock`) | `hive-matrix.nix`, `swarm.matrix.gatewayHost != null` |
|
||||||
| `chat.<swarm>/` | fluffychat-web static | `hive-matrix.nix`, `swarm.matrix.gatewayHost != null` |
|
| `chat.<swarm>/` | fluffychat-web static | `hive-matrix.nix`, `swarm.matrix.gatewayHost != null` |
|
||||||
|
|
|
||||||
|
|
@ -165,7 +165,7 @@ agent containers.
|
||||||
UI proxies, and any other HTTP services.
|
UI proxies, and any other HTTP services.
|
||||||
|
|
||||||
The **host** firewall is the only firewall. The swarm service
|
The **host** firewall is the only firewall. The swarm service
|
||||||
containers that share the host netns (forge, authelia, bao, the
|
containers that share the host netns (forge, bao, the
|
||||||
metrics and log stores) run with `networking.firewall.enable = false`
|
metrics and log stores) run with `networking.firewall.enable = false`
|
||||||
(`nix/container-modules/swarm-container.nix`): a NixOS firewall inside a
|
(`nix/container-modules/swarm-container.nix`): a NixOS firewall inside a
|
||||||
shared-netns container runs against the _host_ ruleset — at container
|
shared-netns container runs against the _host_ ruleset — at container
|
||||||
|
|
|
||||||
|
|
@ -106,6 +106,12 @@ One authelia per swarm, in a `swarm-authelia` container, at
|
||||||
provider, differentiated by roles and claims rather than by mechanism —
|
provider, differentiated by roles and claims rather than by mechanism —
|
||||||
there is one IdP and one auth path.
|
there is one IdP and one auth path.
|
||||||
|
|
||||||
|
The container sits on the hive bridge in a network namespace of its own and
|
||||||
|
dials the store, the collector and its own public name through it. It binds
|
||||||
|
no TCP port: the login server, the metrics listener and the bridge each
|
||||||
|
listen on a unix socket in `/run/swarm-authelia`, a directory only the
|
||||||
|
gateway's nginx and the container's processes can open.
|
||||||
|
|
||||||
- **`deploy.authelia`** — run the container here.
|
- **`deploy.authelia`** — run the container here.
|
||||||
- **`swarm.authelia.url`** — where clients go to authenticate. Present
|
- **`swarm.authelia.url`** — where clients go to authenticate. Present
|
||||||
on **every** hive and the same value on all of them:
|
on **every** hive and the same value on all of them:
|
||||||
|
|
@ -117,7 +123,7 @@ Agent subjects come from swarm-controller's agent-creation job, written
|
||||||
into the users database by `swarm-authelia-bridge`; human ones come from
|
into the users database by `swarm-authelia-bridge`; human ones come from
|
||||||
`swarmctl user add` → [setup.md § 2](../getting-started/setup.md#2--your-sso-account).
|
`swarmctl user add` → [setup.md § 2](../getting-started/setup.md#2--your-sso-account).
|
||||||
|
|
||||||
The bridge listens on loopback on the authelia host, and the gateway
|
The bridge listens on a unix socket on the authelia host, and the gateway
|
||||||
publishes it as
|
publishes it as
|
||||||
`https://<swarm.authelia.domain>/bridge/`, which is where
|
`https://<swarm.authelia.domain>/bridge/`, which is where
|
||||||
`deploy.swarm-controller.authBridgeUrl` points by default on every host.
|
`deploy.swarm-controller.authBridgeUrl` points by default on every host.
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@ separate `enable` would be a second fact free to disagree with the first.
|
||||||
|
|
||||||
## Getting in the first time
|
## Getting in the first time
|
||||||
|
|
||||||
authelia binds loopback only. The **gateway** on the host running it
|
authelia binds no TCP port: its listeners are unix sockets only the gateway can reach. The **gateway** on the host running it
|
||||||
publishes it as `auth.<swarm.domain>` — vhost, dnsmasq record and TLS
|
publishes it as `auth.<swarm.domain>` — vhost, dnsmasq record and TLS
|
||||||
name all follow `deploy.authelia`, so there is nothing to turn on
|
name all follow `deploy.authelia`, so there is nothing to turn on
|
||||||
separately. (Details, including why a client hive must not declare that
|
separately. (Details, including why a client hive must not declare that
|
||||||
|
|
|
||||||
|
|
@ -61,6 +61,19 @@ in
|
||||||
own, and the gateway reaches it there, so it binds no host port. Remove
|
own, and the gateway reaches it there, so it binds no host port. Remove
|
||||||
this definition.
|
this definition.
|
||||||
'')
|
'')
|
||||||
|
(lib.mkRemovedOptionModule [ "services" "hyperhive" "swarm" "authelia" "port" ] ''
|
||||||
|
authelia listens on a unix socket in a network namespace of its own, and
|
||||||
|
the gateway reaches it there, so it binds no port. Remove this
|
||||||
|
definition.
|
||||||
|
'')
|
||||||
|
(lib.mkRemovedOptionModule [ "services" "hyperhive" "swarm" "authelia" "metricsPort" ] ''
|
||||||
|
authelia's metrics listener is a unix socket the gateway scrapes, so it
|
||||||
|
binds no port. Remove this definition.
|
||||||
|
'')
|
||||||
|
(lib.mkRemovedOptionModule [ "services" "hyperhive" "swarm" "authelia" "bridgePort" ] ''
|
||||||
|
swarm-authelia-bridge listens on a unix socket the gateway proxies to, so
|
||||||
|
it binds no port. Remove this definition.
|
||||||
|
'')
|
||||||
(lib.mkRemovedOptionModule [
|
(lib.mkRemovedOptionModule [
|
||||||
"services"
|
"services"
|
||||||
"hyperhive"
|
"hyperhive"
|
||||||
|
|
|
||||||
|
|
@ -24,39 +24,6 @@ in
|
||||||
# Here is what authelia IS, including `url` and the OIDC client registry
|
# Here is what authelia IS, including `url` and the OIDC client registry
|
||||||
# every hive needs as a *client* whether or not it runs the container.
|
# every hive needs as a *client* whether or not it runs the container.
|
||||||
options.services.hyperhive.swarm.authelia = {
|
options.services.hyperhive.swarm.authelia = {
|
||||||
port = lib.mkOption {
|
|
||||||
type = lib.types.port;
|
|
||||||
default = 9091;
|
|
||||||
description = ''
|
|
||||||
TCP port authelia listens on. 9091 is upstream's default and
|
|
||||||
sits outside hyperhive's claimed ranges (dashboard 7000, forge
|
|
||||||
3000, every agent in 8100..8999 via FNV-1a hash).
|
|
||||||
'';
|
|
||||||
};
|
|
||||||
|
|
||||||
metricsPort = lib.mkOption {
|
|
||||||
type = lib.types.port;
|
|
||||||
default = 9959;
|
|
||||||
description = ''
|
|
||||||
TCP port authelia serves its Prometheus metrics on, bound to
|
|
||||||
loopback. Upstream's default, kept so an operator reading
|
|
||||||
authelia's documentation finds what they expect.
|
|
||||||
|
|
||||||
A separate port from {option}`port` because it is a separate
|
|
||||||
listener with a different audience: the main one serves logins and
|
|
||||||
is reachable from the swarm, this one carries no authentication of
|
|
||||||
its own. The swarm collector scrapes it, from whichever host it
|
|
||||||
runs on, as `https://''${domain}/metrics`: the gateway's
|
|
||||||
`= /metrics` location proxies to this port only after authelia
|
|
||||||
has accepted the collector's token for that URL.
|
|
||||||
|
|
||||||
⚠️ Every swarm container shares the host network namespace, so
|
|
||||||
two services defaulting to the same port do not conflict at build
|
|
||||||
time — one simply loses at runtime, with nothing in any log. Check
|
|
||||||
a new value against the others before changing this.
|
|
||||||
'';
|
|
||||||
};
|
|
||||||
|
|
||||||
domain = lib.mkOption {
|
domain = lib.mkOption {
|
||||||
type = lib.types.str;
|
type = lib.types.str;
|
||||||
# Under the SWARM domain, like the forge and matrix: a swarm has one
|
# Under the SWARM domain, like the forge and matrix: a swarm has one
|
||||||
|
|
@ -426,21 +393,6 @@ in
|
||||||
'';
|
'';
|
||||||
};
|
};
|
||||||
|
|
||||||
bridgePort = lib.mkOption {
|
|
||||||
type = lib.types.port;
|
|
||||||
default = 9092;
|
|
||||||
description = ''
|
|
||||||
TCP port `swarm-authelia-bridge` listens on, loopback-bound
|
|
||||||
(`127.0.0.1:''${bridgePort}`) — one above authelia's own default
|
|
||||||
`port` (9091), outside hyperhive's other claimed ranges.
|
|
||||||
|
|
||||||
Not published on its own: the authelia vhost proxies
|
|
||||||
`/bridge/` to it, and that is the address `swarm-controller`
|
|
||||||
uses from any host. The bridge answers only
|
|
||||||
`swarm-controller`'s OIDC client.
|
|
||||||
'';
|
|
||||||
};
|
|
||||||
|
|
||||||
otel.clientId = lib.mkOption {
|
otel.clientId = lib.mkOption {
|
||||||
type = lib.types.str;
|
type = lib.types.str;
|
||||||
readOnly = true;
|
readOnly = true;
|
||||||
|
|
|
||||||
|
|
@ -208,8 +208,8 @@ let
|
||||||
gatewayCfg.httpsPort != 443
|
gatewayCfg.httpsPort != 443
|
||||||
) ":${toString gatewayCfg.httpsPort}";
|
) ":${toString gatewayCfg.httpsPort}";
|
||||||
forwarderEndpoint = "https://${hyperhiveCfg.swarm.otel.domain}${gatewayPortSuffix}/${cfg.machine}";
|
forwarderEndpoint = "https://${hyperhiveCfg.swarm.otel.domain}${gatewayPortSuffix}/${cfg.machine}";
|
||||||
# In the host netns: distinct from every other collector's there, which the
|
# Loopback port of the forwarder's own metrics, inside the container's netns:
|
||||||
# assertion below checks.
|
# no other collector shares it. The module requires a value.
|
||||||
forwarderTelemetryPort = 8901;
|
forwarderTelemetryPort = 8901;
|
||||||
# A client this container registers and mints for itself. Its secret goes
|
# A client this container registers and mints for itself. Its secret goes
|
||||||
# through the store like every other forwarder's: the publisher carries it
|
# through the store like every other forwarder's: the publisher carries it
|
||||||
|
|
@ -603,13 +603,31 @@ let
|
||||||
mv ${lib.escapeShellArg "${clientsFile}.tmp"} ${lib.escapeShellArg clientsFile}
|
mv ${lib.escapeShellArg "${clientsFile}.tmp"} ${lib.escapeShellArg clientsFile}
|
||||||
'';
|
'';
|
||||||
|
|
||||||
# Shared host netns, like the forge container: the
|
# A netns of its own on the hive bridge: the container dials the store, the
|
||||||
# gateway reaches authelia at 127.0.0.1:<port>.
|
# swarm collector and its own public name through the bridge, and the
|
||||||
privateNetwork = false;
|
# gateway reaches the three listeners below over sockets, so authelia binds
|
||||||
|
# no TCP port on any interface.
|
||||||
|
privateNetwork = true;
|
||||||
|
|
||||||
|
# Fixed, not an option: the directory's mode is what keeps the sockets
|
||||||
|
# private, so it holds these three sockets and nothing else.
|
||||||
|
socketDir = "/run/swarm-authelia";
|
||||||
|
mainSocket = "${socketDir}/authelia.sock";
|
||||||
|
metricsSocket = "${socketDir}/metrics.sock";
|
||||||
|
bridgeSocket = "${socketDir}/bridge.sock";
|
||||||
|
# `upstream` forwards the request's own URI, `upstreamAt` replaces it.
|
||||||
|
upstream = socket: "http://unix:${socket}";
|
||||||
|
upstreamAt = socket: path: "http://unix:${socket}:${path}";
|
||||||
|
|
||||||
|
# The gid the gateway's nginx runs as. Only the number crosses into the
|
||||||
|
# container, which knows it as `gateway-nginx`; that group is the primary
|
||||||
|
# group of every process that creates a socket, so nginx can connect.
|
||||||
|
nginxGid = config.ids.gids.nginx;
|
||||||
|
socketGroup = "gateway-nginx";
|
||||||
in
|
in
|
||||||
{
|
{
|
||||||
# What authelia IS to every hive, where it answers (`url`), the OIDC
|
# What authelia IS to every hive, where it answers (`url`), the OIDC
|
||||||
# register every service checks itself against and its port, is
|
# register every service checks itself against, is
|
||||||
# `swarm.authelia` in ./swarm-authelia-service.nix. What the host running
|
# `swarm.authelia` in ./swarm-authelia-service.nix. What the host running
|
||||||
# the container decides is here — which two builds it
|
# the container decides is here — which two builds it
|
||||||
# runs, and two filesystem paths that only exist on the machine running
|
# runs, and two filesystem paths that only exist on the machine running
|
||||||
|
|
@ -854,21 +872,6 @@ in
|
||||||
it in swarm-otel-service.nix.
|
it in swarm-otel-service.nix.
|
||||||
'';
|
'';
|
||||||
}
|
}
|
||||||
{
|
|
||||||
assertion =
|
|
||||||
!(lib.elem forwarderTelemetryPort [
|
|
||||||
hyperhiveCfg.otel.telemetryPort
|
|
||||||
hyperhiveCfg.swarm.otel.telemetryPort
|
|
||||||
hyperhiveCfg.swarm.bao.otel.telemetryPort
|
|
||||||
hyperhiveCfg.swarm.nats.otel.telemetryPort
|
|
||||||
]);
|
|
||||||
message = ''
|
|
||||||
authelia's journal forwarder's telemetry port (${toString forwarderTelemetryPort})
|
|
||||||
collides with another collector's in the host netns: the hive
|
|
||||||
collector's, the swarm collector's, the store forwarder's or the
|
|
||||||
queue forwarder's.
|
|
||||||
'';
|
|
||||||
}
|
|
||||||
];
|
];
|
||||||
|
|
||||||
# Authelia's own gateway surface: the vhost that fronts it and the
|
# Authelia's own gateway surface: the vhost that fronts it and the
|
||||||
|
|
@ -899,14 +902,14 @@ in
|
||||||
# page behind the login it serves.
|
# page behind the login it serves.
|
||||||
services.hyperhive.gateway.vhosts."${cfg.domain}" = {
|
services.hyperhive.gateway.vhosts."${cfg.domain}" = {
|
||||||
locations."/" = {
|
locations."/" = {
|
||||||
proxyPass = "http://127.0.0.1:${toString cfg.port}/";
|
proxyPass = upstreamAt mainSocket "/";
|
||||||
proxyWebsockets = true;
|
proxyWebsockets = true;
|
||||||
extraConfig = ''
|
extraConfig = ''
|
||||||
proxy_buffering off;
|
proxy_buffering off;
|
||||||
# authelia decides by the ORIGINAL request, not by the hop it
|
# authelia decides by the ORIGINAL request, not by the hop it
|
||||||
# sees — the login redirect and the session cookie's domain
|
# sees — the login redirect and the session cookie's domain
|
||||||
# both derive from these. Without them every request looks
|
# both derive from these. Without them every request looks
|
||||||
# like it arrived at 127.0.0.1 over plain http.
|
# like it arrived over a local socket in plain http.
|
||||||
proxy_set_header X-Forwarded-Proto $scheme;
|
proxy_set_header X-Forwarded-Proto $scheme;
|
||||||
proxy_set_header X-Forwarded-Host $host;
|
proxy_set_header X-Forwarded-Host $host;
|
||||||
proxy_set_header X-Forwarded-Uri $request_uri;
|
proxy_set_header X-Forwarded-Uri $request_uri;
|
||||||
|
|
@ -939,21 +942,21 @@ in
|
||||||
# page's own XHR. Enumerating individual endpoints would leave the
|
# page's own XHR. Enumerating individual endpoints would leave the
|
||||||
# next one added silently intercepted.
|
# next one added silently intercepted.
|
||||||
locations."/api/" = {
|
locations."/api/" = {
|
||||||
proxyPass = "http://127.0.0.1:${toString cfg.port}";
|
proxyPass = upstream mainSocket;
|
||||||
};
|
};
|
||||||
locations."/.well-known/" = {
|
locations."/.well-known/" = {
|
||||||
proxyPass = "http://127.0.0.1:${toString cfg.port}";
|
proxyPass = upstream mainSocket;
|
||||||
};
|
};
|
||||||
# `swarm-authelia-bridge`, a separate process from authelia, also a
|
# `swarm-authelia-bridge`, a separate process from authelia, also a
|
||||||
# machine surface. The trailing slash on `proxyPass` strips the
|
# machine surface. The trailing slash on `proxyPass` strips the
|
||||||
# prefix, so `/bridge/requests` reaches the bridge as `/requests`.
|
# prefix, so `/bridge/requests` reaches the bridge as `/requests`.
|
||||||
# The bridge itself refuses every client but swarm-controller's.
|
# The bridge itself refuses every client but swarm-controller's.
|
||||||
locations."/bridge/" = {
|
locations."/bridge/" = {
|
||||||
proxyPass = "http://127.0.0.1:${toString cfg.bridgePort}/";
|
proxyPass = upstreamAt bridgeSocket "/";
|
||||||
};
|
};
|
||||||
|
|
||||||
# Authelia's own metrics, for the swarm collector wherever it runs.
|
# Authelia's own metrics, for the swarm collector wherever it runs.
|
||||||
# The listener (`metricsPort`) is loopback-only and unauthenticated;
|
# The listener (`metricsSocket`) is unauthenticated and serves at `/`;
|
||||||
# this location is the only way in from off the machine. `= /metrics`
|
# this location is the only way in from off the machine. `= /metrics`
|
||||||
# outranks the `/` prefix, so its auth is the one that runs.
|
# outranks the `/` prefix, so its auth is the one that runs.
|
||||||
#
|
#
|
||||||
|
|
@ -961,7 +964,7 @@ in
|
||||||
# `publishedScrapeTargets.authelia` in ./swarm-otel-service.nix; the two
|
# `publishedScrapeTargets.authelia` in ./swarm-otel-service.nix; the two
|
||||||
# must agree, since that string is also the token's audience.
|
# must agree, since that string is also the token's audience.
|
||||||
locations."= /metrics" = {
|
locations."= /metrics" = {
|
||||||
proxyPass = "http://127.0.0.1:${toString cfg.metricsPort}/metrics";
|
proxyPass = upstreamAt metricsSocket "/";
|
||||||
extraConfig = ''
|
extraConfig = ''
|
||||||
auth_request /__authelia_metrics_authz;
|
auth_request /__authelia_metrics_authz;
|
||||||
'';
|
'';
|
||||||
|
|
@ -1009,6 +1012,20 @@ in
|
||||||
})
|
})
|
||||||
];
|
];
|
||||||
|
|
||||||
|
# The socket directory, created host-side before the container starts:
|
||||||
|
# nixos-container refuses to start when a bind source is missing.
|
||||||
|
#
|
||||||
|
# Root-owned and group nginx, `0770`: the listeners create their sockets
|
||||||
|
# through the group (the authelia user's uid is allocated inside the
|
||||||
|
# container, so the host cannot name it), and the gateway traverses it the
|
||||||
|
# same way. Everything else is kept out.
|
||||||
|
systemd.tmpfiles.rules = [
|
||||||
|
"d ${socketDir} 0770 root ${toString nginxGid} - -"
|
||||||
|
];
|
||||||
|
|
||||||
|
# The hive bridge carries the container's DHCP lease and its resolver.
|
||||||
|
services.hyperhive.network.enable = lib.mkDefault true;
|
||||||
|
|
||||||
containers.${cfg.machine} = {
|
containers.${cfg.machine} = {
|
||||||
autoStart = true;
|
autoStart = true;
|
||||||
ephemeral = false;
|
ephemeral = false;
|
||||||
|
|
@ -1018,7 +1035,16 @@ in
|
||||||
inherit privateNetwork;
|
inherit privateNetwork;
|
||||||
# Public trust bundle only, read-only. Empty when the gateway is not
|
# Public trust bundle only, read-only. Empty when the gateway is not
|
||||||
# self-signed, so the whole trust path drops out cleanly.
|
# self-signed, so the whole trust path drops out cleanly.
|
||||||
bindMounts = caTrust.bindMount;
|
hostBridge = networkCfg.bridgeName;
|
||||||
|
bindMounts = {
|
||||||
|
# A bind mount rather than a path both sides happen to know: a
|
||||||
|
# container's `/run` is an nspawn tmpfs the host cannot see into.
|
||||||
|
${socketDir} = {
|
||||||
|
hostPath = socketDir;
|
||||||
|
isReadOnly = false;
|
||||||
|
};
|
||||||
|
}
|
||||||
|
// caTrust.bindMount;
|
||||||
|
|
||||||
config =
|
config =
|
||||||
{ ... }:
|
{ ... }:
|
||||||
|
|
@ -1045,6 +1071,16 @@ in
|
||||||
|
|
||||||
services.hyperhive.swarmContainer = { inherit privateNetwork; };
|
services.hyperhive.swarmContainer = { inherit privateNetwork; };
|
||||||
|
|
||||||
|
# The bridge address comes from the gateway's DHCP. `swarm-bridge-dns`
|
||||||
|
# owns resolv.conf, so dhcpcd must not write it over that unit.
|
||||||
|
networking.interfaces.eth0.useDHCP = true;
|
||||||
|
networking.dhcpcd.extraConfig = "nohook resolv.conf";
|
||||||
|
|
||||||
|
# The sockets' group is the primary group of the processes that create
|
||||||
|
# them, so that group is nginx's gid. A static group, so every unit
|
||||||
|
# that binds a socket can name it.
|
||||||
|
users.groups.${socketGroup}.gid = nginxGid;
|
||||||
|
|
||||||
# Authenticated as the forwarder's own OIDC client, its secret read
|
# Authenticated as the forwarder's own OIDC client, its secret read
|
||||||
# from the store with the publisher's identity: one identity for the
|
# from the store with the publisher's identity: one identity for the
|
||||||
# whole container. With no identity there is nothing to read it with,
|
# whole container. With no identity there is nothing to read it with,
|
||||||
|
|
@ -1119,7 +1155,7 @@ in
|
||||||
Type = "oneshot";
|
Type = "oneshot";
|
||||||
RemainAfterExit = true;
|
RemainAfterExit = true;
|
||||||
User = unitName;
|
User = unitName;
|
||||||
Group = unitName;
|
Group = socketGroup;
|
||||||
StateDirectory = unitName;
|
StateDirectory = unitName;
|
||||||
StateDirectoryMode = "0700";
|
StateDirectoryMode = "0700";
|
||||||
UMask = "0077";
|
UMask = "0077";
|
||||||
|
|
@ -1185,12 +1221,12 @@ in
|
||||||
serviceConfig = {
|
serviceConfig = {
|
||||||
ExecStart = "${deployCfg.authelia.bridgePackage}/bin/swarm-authelia-bridge";
|
ExecStart = "${deployCfg.authelia.bridgePackage}/bin/swarm-authelia-bridge";
|
||||||
User = unitName;
|
User = unitName;
|
||||||
Group = unitName;
|
Group = socketGroup;
|
||||||
Restart = "on-failure";
|
Restart = "on-failure";
|
||||||
RestartSec = "5s";
|
RestartSec = "5s";
|
||||||
};
|
};
|
||||||
environment = {
|
environment = {
|
||||||
SWARM_AUTHELIA_BRIDGE_BIND = "127.0.0.1:${toString cfg.bridgePort}";
|
SWARM_AUTHELIA_BRIDGE_SOCKET = bridgeSocket;
|
||||||
# ONE file, and this is it. The bridge keeps no second,
|
# ONE file, and this is it. The bridge keeps no second,
|
||||||
# private users store: a store of its own treated as canonical,
|
# private users store: a store of its own treated as canonical,
|
||||||
# with `users.yml` written as a rendering of it, would be two
|
# with `users.yml` written as a rendering of it, would be two
|
||||||
|
|
@ -1225,10 +1261,22 @@ in
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
|
|
||||||
|
# Upstream's unit is `ProtectSystem=strict`, so the socket directory
|
||||||
|
# is writable only by name. A socket left by a killed process would
|
||||||
|
# fail the next `bind`, so both are removed before each start.
|
||||||
|
systemd.services.${unitName}.serviceConfig = {
|
||||||
|
ReadWritePaths = [ socketDir ];
|
||||||
|
ExecStartPre = [ "${pkgs.coreutils}/bin/rm -f ${mainSocket} ${metricsSocket}" ];
|
||||||
|
};
|
||||||
|
|
||||||
services.authelia.instances.${instance} = {
|
services.authelia.instances.${instance} = {
|
||||||
enable = true;
|
enable = true;
|
||||||
package = deployCfg.authelia.package;
|
package = deployCfg.authelia.package;
|
||||||
|
|
||||||
|
# Primary group of the unit and its user: the group the sockets are
|
||||||
|
# created in (see `socketGroup`).
|
||||||
|
group = socketGroup;
|
||||||
|
|
||||||
# Merged at RUNTIME alongside the nix-generated config, which
|
# Merged at RUNTIME alongside the nix-generated config, which
|
||||||
# is the whole reason the client digests can exist at all:
|
# is the whole reason the client digests can exist at all:
|
||||||
# `settings` below is rendered into the world-readable nix
|
# `settings` below is rendered into the world-readable nix
|
||||||
|
|
@ -1272,7 +1320,7 @@ in
|
||||||
# where those messages go instead of implying a mail path
|
# where those messages go instead of implying a mail path
|
||||||
settings = {
|
settings = {
|
||||||
theme = "dark";
|
theme = "dark";
|
||||||
server.address = "tcp://127.0.0.1:${toString cfg.port}";
|
server.address = "unix://${mainSocket}?umask=0007";
|
||||||
|
|
||||||
# Let a machine present an OAuth2 access token to the same
|
# Let a machine present an OAuth2 access token to the same
|
||||||
# `auth_request` endpoint browsers use, so a scraper can be
|
# `auth_request` endpoint browsers use, so a scraper can be
|
||||||
|
|
@ -1305,14 +1353,14 @@ in
|
||||||
log.level = "info";
|
log.level = "info";
|
||||||
|
|
||||||
# Prometheus exposition for the swarm collector to scrape.
|
# Prometheus exposition for the swarm collector to scrape.
|
||||||
# Loopback only, like the main listener above and for a
|
# A socket only the gateway can reach, like the main listener above
|
||||||
# stronger reason: this endpoint has no authentication of its
|
# and for a stronger reason: this endpoint has no authentication of its
|
||||||
# own, and it reports request volumes and outcomes for every
|
# own, and it reports request volumes and outcomes for every
|
||||||
# SSO login on the swarm. The collector reaches it through the
|
# SSO login on the swarm. The collector reaches it through the
|
||||||
# vhost's gated `= /metrics` location.
|
# vhost's gated `= /metrics` location.
|
||||||
telemetry.metrics = {
|
telemetry.metrics = {
|
||||||
enabled = true;
|
enabled = true;
|
||||||
address = "tcp://127.0.0.1:${toString cfg.metricsPort}";
|
address = "unix://${metricsSocket}?umask=0007";
|
||||||
};
|
};
|
||||||
|
|
||||||
# `watch` is load-bearing, not a convenience: authelia reads
|
# `watch` is load-bearing, not a convenience: authelia reads
|
||||||
|
|
|
||||||
|
|
@ -692,7 +692,7 @@ let
|
||||||
in
|
in
|
||||||
url autheliaNotColocated == "${autheliaNotColocated.services.hyperhive.swarm.authelia.url}/bridge"
|
url autheliaNotColocated == "${autheliaNotColocated.services.hyperhive.swarm.authelia.url}/bridge"
|
||||||
&& url autheliaColocated == url autheliaNotColocated
|
&& url autheliaColocated == url autheliaNotColocated
|
||||||
&& location.proxyPass == "http://127.0.0.1:${toString authelia.bridgePort}/";
|
&& location.proxyPass == "http://unix:/run/swarm-authelia/bridge.sock:/";
|
||||||
}
|
}
|
||||||
{
|
{
|
||||||
# Config NAMES the IdP; it never computes where the IdP is. Both arms
|
# Config NAMES the IdP; it never computes where the IdP is. Both arms
|
||||||
|
|
|
||||||
|
|
@ -236,7 +236,7 @@ let
|
||||||
}
|
}
|
||||||
{
|
{
|
||||||
# The location the collector's job requests: gated, and proxying to the
|
# The location the collector's job requests: gated, and proxying to the
|
||||||
# loopback listener that has no authentication of its own.
|
# socket listener that has no authentication of its own.
|
||||||
name = "authelia's vhost serves /metrics only behind the collector's auth subrequest";
|
name = "authelia's vhost serves /metrics only behind the collector's auth subrequest";
|
||||||
ok =
|
ok =
|
||||||
let
|
let
|
||||||
|
|
@ -244,7 +244,7 @@ let
|
||||||
locs = autheliaOnly.services.nginx.virtualHosts.${h.swarm.authelia.domain}.locations;
|
locs = autheliaOnly.services.nginx.virtualHosts.${h.swarm.authelia.domain}.locations;
|
||||||
authz = locs."= /__authelia_metrics_authz";
|
authz = locs."= /__authelia_metrics_authz";
|
||||||
in
|
in
|
||||||
locs."= /metrics".proxyPass == "http://127.0.0.1:${toString h.swarm.authelia.metricsPort}/metrics"
|
locs."= /metrics".proxyPass == "http://unix:/run/swarm-authelia/metrics.sock:/"
|
||||||
&& lib.hasInfix "auth_request /__authelia_metrics_authz;" locs."= /metrics".extraConfig
|
&& lib.hasInfix "auth_request /__authelia_metrics_authz;" locs."= /metrics".extraConfig
|
||||||
&& authz.proxyPass == "https://${h.swarm.authelia.domain}/api/authz/auth-request"
|
&& authz.proxyPass == "https://${h.swarm.authelia.domain}/api/authz/auth-request"
|
||||||
&& lib.hasInfix "internal;" authz.extraConfig;
|
&& lib.hasInfix "internal;" authz.extraConfig;
|
||||||
|
|
|
||||||
|
|
@ -171,7 +171,7 @@ let
|
||||||
|
|
||||||
# Every service container at once: the services-only host plus the CI
|
# Every service container at once: the services-only host plus the CI
|
||||||
# runner. The CI runner, the review classifier, the builder, the claude login
|
# runner. The CI runner, the review classifier, the builder, the claude login
|
||||||
# minter, the metrics store and the homeserver are the six containers on a
|
# minter, the metrics store, the homeserver and authelia are the seven containers on a
|
||||||
# netns of their own.
|
# netns of their own.
|
||||||
serviceContainersWithCi = hive {
|
serviceContainersWithCi = hive {
|
||||||
deploy.hive-controller.enable = false;
|
deploy.hive-controller.enable = false;
|
||||||
|
|
@ -599,7 +599,7 @@ let
|
||||||
let
|
let
|
||||||
shared = lib.filter (c: !c.privateNetwork) (lib.attrValues serviceContainers);
|
shared = lib.filter (c: !c.privateNetwork) (lib.attrValues serviceContainers);
|
||||||
in
|
in
|
||||||
lib.length shared == 7
|
lib.length shared == 6
|
||||||
&& lib.all (
|
&& lib.all (
|
||||||
c: !c.config.networking.firewall.enable && !c.config.networking.resolvconf.enable
|
c: !c.config.networking.firewall.enable && !c.config.networking.resolvconf.enable
|
||||||
) shared;
|
) shared;
|
||||||
|
|
@ -694,6 +694,51 @@ let
|
||||||
&& unit.serviceConfig.Group == "gateway-nginx"
|
&& unit.serviceConfig.Group == "gateway-nginx"
|
||||||
&& lib.elem dir unit.serviceConfig.ReadWritePaths;
|
&& lib.elem dir unit.serviceConfig.ReadWritePaths;
|
||||||
}
|
}
|
||||||
|
{
|
||||||
|
# On the bridge by DHCP, with all three listeners on sockets: nothing
|
||||||
|
# the bridge can dial. Every vhost location that forwards goes through
|
||||||
|
# one of the sockets, except the internal authz subrequest, which is
|
||||||
|
# `https://`. The main, bridge and metrics locations are named so an
|
||||||
|
# empty filter can't pass.
|
||||||
|
name = "authelia runs on a netns of its own and the gateway reaches all three listeners over sockets";
|
||||||
|
ok =
|
||||||
|
let
|
||||||
|
host = serviceContainersWithCi;
|
||||||
|
net = host.services.hyperhive.network;
|
||||||
|
c = host.containers.swarm-authelia;
|
||||||
|
dir = "/run/swarm-authelia";
|
||||||
|
sock = n: "${dir}/${n}.sock";
|
||||||
|
locs = host.services.nginx.virtualHosts.${host.services.hyperhive.swarm.authelia.domain}.locations;
|
||||||
|
forwarded = lib.filter (p: p != null && !(lib.hasPrefix "https://" p)) (
|
||||||
|
map (l: l.proxyPass) (lib.attrValues locs)
|
||||||
|
);
|
||||||
|
inner = c.config;
|
||||||
|
settings = inner.services.authelia.instances.swarm.settings;
|
||||||
|
unit = inner.systemd.services.authelia-swarm;
|
||||||
|
bridge = inner.systemd.services.swarm-authelia-bridge;
|
||||||
|
nginxGid = host.ids.gids.nginx;
|
||||||
|
in
|
||||||
|
c.privateNetwork
|
||||||
|
&& c.hostBridge == net.bridgeName
|
||||||
|
&& c.hostAddress == null
|
||||||
|
&& c.localAddress == null
|
||||||
|
&& inner.networking.interfaces.eth0.useDHCP
|
||||||
|
&& inner.networking.firewall.enable
|
||||||
|
&& settings.server.address == "unix://${sock "authelia"}?umask=0007"
|
||||||
|
&& settings.telemetry.metrics.address == "unix://${sock "metrics"}?umask=0007"
|
||||||
|
&& bridge.environment.SWARM_AUTHELIA_BRIDGE_SOCKET == sock "bridge"
|
||||||
|
&& !(bridge.environment ? SWARM_AUTHELIA_BRIDGE_BIND)
|
||||||
|
&& locs."/".proxyPass == "http://unix:${sock "authelia"}:/"
|
||||||
|
&& locs."/bridge/".proxyPass == "http://unix:${sock "bridge"}:/"
|
||||||
|
&& locs."= /metrics".proxyPass == "http://unix:${sock "metrics"}:/"
|
||||||
|
&& lib.all (lib.hasPrefix "http://unix:${dir}/") forwarded
|
||||||
|
&& c.bindMounts.${dir}.hostPath == dir
|
||||||
|
&& builtins.elem "d ${dir} 0770 root ${toString nginxGid} - -" host.systemd.tmpfiles.rules
|
||||||
|
&& inner.users.groups.gateway-nginx.gid == nginxGid
|
||||||
|
&& unit.serviceConfig.Group == "gateway-nginx"
|
||||||
|
&& bridge.serviceConfig.Group == "gateway-nginx"
|
||||||
|
&& lib.elem dir unit.serviceConfig.ReadWritePaths;
|
||||||
|
}
|
||||||
{
|
{
|
||||||
# The DynamicUser layout keeps the history under /var/lib/private; the
|
# The DynamicUser layout keeps the history under /var/lib/private; the
|
||||||
# static user's StateDirectory is the plain path. The move runs as its
|
# static user's StateDirectory is the plain path. The move runs as its
|
||||||
|
|
|
||||||
|
|
@ -33,7 +33,8 @@ simply owns the file it writes. No elevated privilege anywhere.
|
||||||
- No restart of authelia after a write — relies on
|
- No restart of authelia after a write — relies on
|
||||||
`authentication_backend.file.watch`, confirmed working against the
|
`authentication_backend.file.watch`, confirmed working against the
|
||||||
pinned 4.39.20 build during this design work.
|
pinned 4.39.20 build during this design work.
|
||||||
- Listens on loopback. The authelia vhost proxies `/bridge/` to it with
|
- Listens on the unix socket named by `SWARM_AUTHELIA_BRIDGE_SOCKET` (mode
|
||||||
|
`0660`, in a directory shared with the host's nginx). The authelia vhost proxies `/bridge/` to it with
|
||||||
the prefix stripped, so `swarm-controller` on any host calls
|
the prefix stripped, so `swarm-controller` on any host calls
|
||||||
`https://<swarm.authelia.domain>/bridge/requests`.
|
`https://<swarm.authelia.domain>/bridge/requests`.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -11,7 +11,8 @@
|
||||||
//! with root/`CAP_CHOWN`/a shared group (all examined and rejected — see
|
//! with root/`CAP_CHOWN`/a shared group (all examined and rejected — see
|
||||||
//! `swarmctl/README.md`'s own identical analysis of this same file).
|
//! `swarmctl/README.md`'s own identical analysis of this same file).
|
||||||
//!
|
//!
|
||||||
//! Listens on loopback; the authelia vhost proxies `/bridge/` to it, so a
|
//! Listens on a unix socket (`SWARM_AUTHELIA_BRIDGE_SOCKET`) in a directory
|
||||||
|
//! the host's nginx shares; the authelia vhost proxies `/bridge/` to it, so a
|
||||||
//! `swarm-controller` on any host reaches it at the same address.
|
//! `swarm-controller` on any host reaches it at the same address.
|
||||||
//! Bearer-authenticated via authelia's own OIDC token introspection (RFC
|
//! Bearer-authenticated via authelia's own OIDC token introspection (RFC
|
||||||
//! 7662), and admits only `swarm-controller`'s **existing** machine-client
|
//! 7662), and admits only `swarm-controller`'s **existing** machine-client
|
||||||
|
|
@ -45,7 +46,7 @@ use swarm_authelia_bridge_sock::{AgentIdentity, BridgeRequest, BridgeResponse};
|
||||||
/// `swarm-controller`'s (the token being checked). Set by the nix module
|
/// `swarm-controller`'s (the token being checked). Set by the nix module
|
||||||
/// that provisions this bridge's own client alongside authelia.
|
/// that provisions this bridge's own client alongside authelia.
|
||||||
struct Config {
|
struct Config {
|
||||||
bind: String,
|
socket: std::path::PathBuf,
|
||||||
users_file: std::path::PathBuf,
|
users_file: std::path::PathBuf,
|
||||||
authelia_bin: std::path::PathBuf,
|
authelia_bin: std::path::PathBuf,
|
||||||
introspection_url: String,
|
introspection_url: String,
|
||||||
|
|
@ -60,7 +61,7 @@ struct Config {
|
||||||
impl Config {
|
impl Config {
|
||||||
fn from_env() -> Result<Self> {
|
fn from_env() -> Result<Self> {
|
||||||
Ok(Self {
|
Ok(Self {
|
||||||
bind: env_var("SWARM_AUTHELIA_BRIDGE_BIND")?,
|
socket: env_var("SWARM_AUTHELIA_BRIDGE_SOCKET")?.into(),
|
||||||
users_file: env_var("SWARM_AUTHELIA_BRIDGE_USERS_FILE")?.into(),
|
users_file: env_var("SWARM_AUTHELIA_BRIDGE_USERS_FILE")?.into(),
|
||||||
authelia_bin: env_var("SWARM_AUTHELIA_BRIDGE_AUTHELIA_BIN")?.into(),
|
authelia_bin: env_var("SWARM_AUTHELIA_BRIDGE_AUTHELIA_BIN")?.into(),
|
||||||
introspection_url: env_var("SWARM_AUTHELIA_BRIDGE_INTROSPECTION_URL")?,
|
introspection_url: env_var("SWARM_AUTHELIA_BRIDGE_INTROSPECTION_URL")?,
|
||||||
|
|
@ -116,7 +117,7 @@ async fn main() -> Result<()> {
|
||||||
.init();
|
.init();
|
||||||
|
|
||||||
let config = Config::from_env()?;
|
let config = Config::from_env()?;
|
||||||
let bind = config.bind.clone();
|
let socket = config.socket.clone();
|
||||||
let state = Arc::new(AppState {
|
let state = Arc::new(AppState {
|
||||||
config,
|
config,
|
||||||
http: reqwest::Client::new(),
|
http: reqwest::Client::new(),
|
||||||
|
|
@ -127,15 +128,32 @@ async fn main() -> Result<()> {
|
||||||
.route("/requests", post(handle_request))
|
.route("/requests", post(handle_request))
|
||||||
.with_state(state);
|
.with_state(state);
|
||||||
|
|
||||||
let listener = tokio::net::TcpListener::bind(&bind)
|
let listener = bind_socket(&socket)?;
|
||||||
.await
|
tracing::info!(socket = %socket.display(), "swarm-authelia-bridge listening");
|
||||||
.with_context(|| format!("binding {bind}"))?;
|
|
||||||
tracing::info!(%bind, "swarm-authelia-bridge listening");
|
|
||||||
axum::serve(listener, app)
|
axum::serve(listener, app)
|
||||||
.await
|
.await
|
||||||
.context("serving swarm-authelia-bridge")
|
.context("serving swarm-authelia-bridge")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Bind the listening socket at `path`, replacing a stale one a crash left
|
||||||
|
/// behind (it would otherwise fail `bind` with `EADDRINUSE`).
|
||||||
|
///
|
||||||
|
/// The mode is `0660`: `connect(2)` needs write permission, and the
|
||||||
|
/// directory's group decides who can reach the socket at all.
|
||||||
|
fn bind_socket(path: &std::path::Path) -> Result<tokio::net::UnixListener> {
|
||||||
|
use std::os::unix::fs::PermissionsExt as _;
|
||||||
|
match std::fs::remove_file(path) {
|
||||||
|
Ok(()) => {}
|
||||||
|
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {}
|
||||||
|
Err(e) => return Err(e).with_context(|| format!("removing stale {}", path.display())),
|
||||||
|
}
|
||||||
|
let listener = tokio::net::UnixListener::bind(path)
|
||||||
|
.with_context(|| format!("binding {}", path.display()))?;
|
||||||
|
std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o660))
|
||||||
|
.with_context(|| format!("setting the mode of {}", path.display()))?;
|
||||||
|
Ok(listener)
|
||||||
|
}
|
||||||
|
|
||||||
/// The single endpoint — body is the wire-crate's [`BridgeRequest`]
|
/// The single endpoint — body is the wire-crate's [`BridgeRequest`]
|
||||||
/// verbatim (JSON), not a per-operation REST route. That is what let the
|
/// verbatim (JSON), not a per-operation REST route. That is what let the
|
||||||
/// second operation arrive as a match arm rather than as another route
|
/// second operation arrive as a match arm rather than as another route
|
||||||
|
|
@ -448,7 +466,8 @@ async fn generate_password(bin: &std::path::Path) -> Result<String> {
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
mod tests {
|
mod tests {
|
||||||
use super::{
|
use super::{
|
||||||
BridgeRequest, Config, Refusal, StatusCode, classify, handle, introspect::Verdict, respond,
|
BridgeRequest, Config, Refusal, StatusCode, bind_socket, classify, handle,
|
||||||
|
introspect::Verdict, respond,
|
||||||
};
|
};
|
||||||
|
|
||||||
const CONTROLLER: &str = "swarm-controller";
|
const CONTROLLER: &str = "swarm-controller";
|
||||||
|
|
@ -565,7 +584,7 @@ mod tests {
|
||||||
)
|
)
|
||||||
.unwrap();
|
.unwrap();
|
||||||
Config {
|
Config {
|
||||||
bind: String::new(),
|
socket: std::path::PathBuf::new(),
|
||||||
users_file,
|
users_file,
|
||||||
authelia_bin: std::path::PathBuf::new(),
|
authelia_bin: std::path::PathBuf::new(),
|
||||||
introspection_url: String::new(),
|
introspection_url: String::new(),
|
||||||
|
|
@ -576,6 +595,21 @@ mod tests {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn the_socket_replaces_a_stale_one_and_is_group_only() {
|
||||||
|
use std::os::unix::fs::PermissionsExt as _;
|
||||||
|
let dir = std::env::temp_dir().join(format!("bridge-sock-{}", std::process::id()));
|
||||||
|
std::fs::create_dir_all(&dir).unwrap();
|
||||||
|
let path = dir.join("bridge.sock");
|
||||||
|
// Dropping a listener leaves its file behind: the state a crash leaves.
|
||||||
|
drop(bind_socket(&path).unwrap());
|
||||||
|
assert!(path.exists(), "control: the stale socket is on disk");
|
||||||
|
let _listener = bind_socket(&path).unwrap();
|
||||||
|
let mode = std::fs::metadata(&path).unwrap().permissions().mode() & 0o777;
|
||||||
|
assert_eq!(mode, 0o660);
|
||||||
|
let _ = std::fs::remove_dir_all(&dir);
|
||||||
|
}
|
||||||
|
|
||||||
#[tokio::test]
|
#[tokio::test]
|
||||||
async fn the_operator_list_is_the_operator_groups_members() {
|
async fn the_operator_list_is_the_operator_groups_members() {
|
||||||
let config = config_with_one_operator();
|
let config = config_with_one_operator();
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue