swarm: default every queue URL to the queue's name on every hive
A remote hive dialled nothing until an operator copied the queue's URL into it, though the URL is the same string everywhere. statusPublish.natsUrl, queue.agentNatsUrl and controller.queue.natsUrl now default to tls://<swarm.nats.domain>:<port> unconditionally. The statusPublish assertion treated a URL without a secret as a half config. With the URL a default on every hive, only the secret claims publishing: the assertion now refuses a secret without a URL or token endpoint, and hive-c0re's status environment is gated on the secret too, so a hive without one publishes nothing instead of reading a missing credential.
This commit is contained in:
atlas
mara
parent
0081d75c86
commit
a5259146dc
8 changed files with 220 additions and 129 deletions
|
|
@ -338,10 +338,8 @@ nothing has to re-publish.
|
|||
|
||||
### Making a hive report
|
||||
|
||||
Three options, on the **hive**, set together or not at all — a
|
||||
half-configured hive is an eval error rather than one that quietly never
|
||||
reports. They sit in two namespaces, because two of them are facts about
|
||||
_this machine_ and one is the swarm's single address:
|
||||
Three options on the **hive**. They sit in two namespaces, because two of them
|
||||
are facts about _this machine_ and one is the swarm's single address:
|
||||
|
||||
| option | what to set it to |
|
||||
| ------------------------------------------------------- | --------------------------------------------- |
|
||||
|
|
@ -349,10 +347,14 @@ _this machine_ and one is the swarm's single address:
|
|||
| `swarm.statusPublish.tokenEndpoint` | the swarm IdP's `/api/oidc/token` |
|
||||
| `deploy.hive-controller.statusPublish.clientSecretFile` | path to this hive's client secret, plaintext |
|
||||
|
||||
On a host that runs the queue and the IdP itself, all three default to
|
||||
the local ones and there is nothing to set. Any other hive needs them
|
||||
spelled out, and needs the secret to physically be there: the swarm does
|
||||
not distribute it. Copy `hive-<hiveName>.secret` out of the swarm host's
|
||||
The queue URL and the token endpoint default to the swarm's own addresses on
|
||||
every hive, so there is nothing to set for them. The secret is what turns
|
||||
publishing on: a hive without it doesn't publish. A secret without the other
|
||||
two is an eval error rather than a hive that quietly never reports.
|
||||
|
||||
On a host that runs the queue and the IdP, the secret defaults to the local one.
|
||||
Any other hive needs the secret to physically be there, because the swarm
|
||||
doesn't distribute it. Copy `hive-<hiveName>.secret` out of the swarm host's
|
||||
`deploy.authelia.hostClientSecretDir` with whatever secret management the
|
||||
deployment already uses.
|
||||
|
||||
|
|
@ -390,11 +392,12 @@ with the credential itself and aren't configurable.
|
|||
| `deploy.hive-controller.queue.agentNatsUrl` | where the queue listens, as an agent **container** reaches it |
|
||||
| `swarm.statusPublish.tokenEndpoint` | the swarm IdP's `/api/oidc/token` — the same one the hive uses |
|
||||
|
||||
On a host that runs the queue, `agentNatsUrl` defaults to the same
|
||||
`tls://<swarm.nats.domain>:<swarm.nats.port>` as the hive's own. Inside a
|
||||
container the name resolves to the bridge address, where the firewall opens the
|
||||
port. ⚠️ **Never a loopback address here**: an agent has its own network
|
||||
namespace, so `127.0.0.1` reaches the agent.
|
||||
On every hive, `agentNatsUrl` defaults to the same
|
||||
`tls://<swarm.nats.domain>:<swarm.nats.port>` as the hive's own. On the queue's
|
||||
host the name resolves inside a container to the bridge address, where the
|
||||
firewall opens the port; on any other hive it resolves through the host's DNS,
|
||||
like the store's name. ⚠️ **Never a loopback address here**: an agent has its
|
||||
own network namespace, so `127.0.0.1` reaches the agent.
|
||||
|
||||
The harness sees four variables, and treats them as all-or-none:
|
||||
`HIVE_AGENT_NATS_URL` and `HIVE_AGENT_OIDC_TOKEN_ENDPOINT` from the two options
|
||||
|
|
|
|||
|
|
@ -133,12 +133,12 @@ in
|
|||
./options.nix
|
||||
./theme.nix
|
||||
# Hive-CA trust for this daemon's outbound TLS. Nothing it is given by
|
||||
# default is https — the forge, matrix and queue URLs all resolve to
|
||||
# plain http or loopback — so this changes nothing on an all-local
|
||||
# hive. It matters for the split-host shape the options invite:
|
||||
# `swarm.matrix.apiUrl`'s own example is `https://matrix.example.com`,
|
||||
# and pointing it (or `deploy.hive-controller.statusPublish.natsUrl`)
|
||||
# at another hive's
|
||||
# default is https — the forge and matrix URLs resolve to plain http or
|
||||
# loopback, and the queue's TLS is verified against
|
||||
# `HIVE_C0RE_OIDC_CA_FILE` (../hive-tls.nix), not this bundle — so this
|
||||
# changes nothing on an all-local hive. It matters for the split-host
|
||||
# shape the options invite: `swarm.matrix.apiUrl`'s own example is
|
||||
# `https://matrix.example.com`, and pointing it at another hive's
|
||||
# gateway means verifying a leaf signed by a CA generated at runtime,
|
||||
# which no build-time trust store can contain.
|
||||
#
|
||||
|
|
|
|||
|
|
@ -257,24 +257,30 @@ in
|
|||
# environment is a deployment bug the daemon refuses to treat as
|
||||
# "no queue configured", because the failure it would otherwise
|
||||
# produce is a hive that comes up fine and silently never reports.
|
||||
# The three-option version of that same rule is asserted at eval in
|
||||
# ./../swarm.nix, so this can only ever emit a complete set.
|
||||
# The rule that a secret needs the other two options is asserted at
|
||||
# eval in ./../swarm.nix, so this can only ever emit a complete set.
|
||||
#
|
||||
# ⚠️ The guard and the value beside it read different namespaces on
|
||||
# purpose: where the queue is and where its secret sits are this
|
||||
# machine's, the token endpoint is the swarm's one address. The
|
||||
# assertion covers all three, which is what keeps the guard honest.
|
||||
lib.optionalAttrs (config.services.hyperhive.deploy.hive-controller.statusPublish.natsUrl != null) {
|
||||
HIVE_C0RE_NATS_URL = config.services.hyperhive.deploy.hive-controller.statusPublish.natsUrl;
|
||||
HIVE_C0RE_OIDC_TOKEN_ENDPOINT = config.services.hyperhive.swarm.statusPublish.tokenEndpoint;
|
||||
# The identity swarm-authelia.nix already declares for every entry in
|
||||
# `swarm.hives` — the hive does not choose its own name here, it uses
|
||||
# the one the roster gave it.
|
||||
HIVE_C0RE_OIDC_CLIENT_ID = "hive-${config.services.hyperhive.hiveName}";
|
||||
# `%d` is systemd's credentials directory — see the LoadCredential in
|
||||
# ./default.nix. The daemon reads a path, never a value.
|
||||
HIVE_C0RE_OIDC_CLIENT_SECRET_FILE = "%d/swarm-status-client.secret";
|
||||
}
|
||||
# ⚠️ Gated on the SECRET as well as the URL. The URL defaults to the
|
||||
# queue's name on every hive, so on its own it says where the queue is,
|
||||
# not that this hive publishes; the secret is that claim. Gated on the
|
||||
# URL alone, a hive with no secret would be handed a secret path that
|
||||
# `LoadCredential` in ./default.nix never fills.
|
||||
lib.optionalAttrs
|
||||
(
|
||||
config.services.hyperhive.deploy.hive-controller.statusPublish.natsUrl != null
|
||||
&& config.services.hyperhive.deploy.hive-controller.statusPublish.clientSecretFile != null
|
||||
)
|
||||
{
|
||||
HIVE_C0RE_NATS_URL = config.services.hyperhive.deploy.hive-controller.statusPublish.natsUrl;
|
||||
HIVE_C0RE_OIDC_TOKEN_ENDPOINT = config.services.hyperhive.swarm.statusPublish.tokenEndpoint;
|
||||
# The identity swarm-authelia.nix already declares for every entry in
|
||||
# `swarm.hives` — the hive does not choose its own name here, it uses
|
||||
# the one the roster gave it.
|
||||
HIVE_C0RE_OIDC_CLIENT_ID = "hive-${config.services.hyperhive.hiveName}";
|
||||
# `%d` is systemd's credentials directory — see the LoadCredential in
|
||||
# ./default.nix. The daemon reads a path, never a value.
|
||||
HIVE_C0RE_OIDC_CLIENT_SECRET_FILE = "%d/swarm-status-client.secret";
|
||||
}
|
||||
// {
|
||||
# Where ../glue-queue-agent-credential.nix lands the AGENTS' queue
|
||||
# credential. Read by `hive_c0re::lifecycle::host_config`, which stats the
|
||||
|
|
|
|||
|
|
@ -86,31 +86,13 @@ in
|
|||
config.services.hyperhive.deploy.nats.autoGenerateCallout =
|
||||
lib.mkDefault cfg.deploy.singleHostSwarm;
|
||||
|
||||
# ⚠️ Every `swarm.*` default this mode sets lives INSIDE this attrset, not
|
||||
# as a second `config.services.hyperhive.swarm.…` path beside it — written
|
||||
# that way the two definitions of `swarm` collide and the nested one is
|
||||
# silently lost. The gate caught exactly that on the controller's queue URL,
|
||||
# when this mode still set it: mode on, `natsUrl` still "".
|
||||
config.services.hyperhive.swarm = {
|
||||
ca.autoConfigure = lib.mkDefault cfg.deploy.singleHostSwarm;
|
||||
# The controller's queue coordinates. Kept with the mode, not in the
|
||||
# options' own `default`, because the controller's minted client secret
|
||||
# only exists on the host authelia ran its first boot on — so they belong
|
||||
# to the mode that asserts this box is the whole deployment.
|
||||
#
|
||||
# Deriving them from `deploy.nats` / `deploy.authelia`
|
||||
# inside those defaults is the mixing this file exists to prevent: the
|
||||
# option would be describing a deployment shape instead of describing
|
||||
# itself, and "what does all-local turn on?" would stop having one
|
||||
# answer.
|
||||
#
|
||||
# ⚠️ Must live INSIDE this attrset, not as a second
|
||||
# `config.services.hyperhive.swarm.…` path beside it — written that
|
||||
# way the two definitions of `swarm` collide and the nested one is
|
||||
# silently lost. The gate caught exactly that: mode on, `natsUrl`
|
||||
# still "".
|
||||
#
|
||||
# The *requirement* stays in `swarm-controller.nix` as an assertion:
|
||||
# needing a queue is the controller's own property in every topology,
|
||||
# and only the convenience is local.
|
||||
controller.queue.natsUrl = lib.mkIf cfg.deploy.singleHostSwarm (
|
||||
lib.mkDefault "tls://${config.services.hyperhive.swarm.nats.domain}:${toString config.services.hyperhive.swarm.nats.port}"
|
||||
);
|
||||
};
|
||||
|
||||
# The controller is asserted by the MODE and by nothing else. Its own
|
||||
|
|
|
|||
|
|
@ -17,6 +17,7 @@ let
|
|||
cfg = config.services.hyperhive.swarm.controller;
|
||||
deployCfg = config.services.hyperhive.deploy;
|
||||
autheliaCfg = config.services.hyperhive.swarm.authelia;
|
||||
natsCfg = config.services.hyperhive.swarm.nats;
|
||||
|
||||
# Where the secret store is, and whether this host holds the controller's
|
||||
# own leaf for it. ⚠️ The controller's pair, NOT `deploy.bao.clientCertFile`
|
||||
|
|
@ -364,18 +365,18 @@ in
|
|||
queue = {
|
||||
natsUrl = lib.mkOption {
|
||||
type = lib.types.str;
|
||||
default = "";
|
||||
default = "tls://${natsCfg.domain}:${toString natsCfg.port}";
|
||||
defaultText = lib.literalExpression ''"tls://''${swarm.nats.domain}:''${swarm.nats.port}"'';
|
||||
example = "tls://nats.example.com:4222";
|
||||
description = ''
|
||||
Where the controller reaches the swarm queue.
|
||||
|
||||
Defaults to the queue's name, which is the same URL on every
|
||||
host: the queue's own host resolves it locally, and any other
|
||||
through the operator's DNS.
|
||||
|
||||
Empty means unset, which the assertion below refuses — a
|
||||
controller with no queue is not a lighter controller.
|
||||
|
||||
`singleHostSwarm` fills this in with
|
||||
`tls://<swarm.nats.domain>:<port>`. That derivation lives with the
|
||||
mode rather than here, so this option describes itself rather than
|
||||
a deployment shape.
|
||||
'';
|
||||
};
|
||||
|
||||
|
|
@ -764,8 +765,8 @@ in
|
|||
message = ''
|
||||
services.hyperhive.swarm.controller.queue.natsUrl is unset.
|
||||
|
||||
`singleHostSwarm` fills it in with the queue's name. A controller
|
||||
in any other deployment has to be told the URL.
|
||||
It defaults to the queue's name; something in this configuration
|
||||
set it to "". Remove that setting, or set the URL.
|
||||
'';
|
||||
}
|
||||
{
|
||||
|
|
|
|||
|
|
@ -115,16 +115,15 @@ let
|
|||
lib.filterAttrs (_: hive: hive.certFingerprint != null) swarmCfg.hives
|
||||
);
|
||||
|
||||
# Whether the queue and its minted secret are on THIS host — the two
|
||||
# coordinates that are a statement about this machine's disk and
|
||||
# netns, from one condition so a partial set is unrepresentable rather
|
||||
# than merely detected.
|
||||
# Whether this hive's minted status secret is on THIS host, which is the
|
||||
# one status-publishing coordinate that is a statement about this
|
||||
# machine's disk.
|
||||
#
|
||||
# ⚠️ The token endpoint is NOT one of them any more. It is the swarm's
|
||||
# one address, derived from `swarm.authelia.url` like
|
||||
# ./swarm-controller.nix's own `queue.tokenEndpoint` already is, so it
|
||||
# is correct for a remote provider and does not ask where anything
|
||||
# runs.
|
||||
# ⚠️ Neither the token endpoint nor the queue's URL is gated on it. Both
|
||||
# are the swarm's one address: the endpoint derived from
|
||||
# `swarm.authelia.url`, the queue from its name, which resolves on every
|
||||
# hive. So they are correct wherever the queue and the IdP run, and do
|
||||
# not ask where anything runs.
|
||||
queueLocal = deployCfg.nats.enable && deployCfg.authelia.enable && cfg.hiveName != null;
|
||||
in
|
||||
{
|
||||
|
|
@ -373,18 +372,16 @@ in
|
|||
}
|
||||
{
|
||||
# Deliberately an assertion and not a silent "then publish
|
||||
# nothing": a half-set trio is a config an operator believes is
|
||||
# working, and its runtime failure mode is the expensive one —
|
||||
# the daemon comes up fine, never connects, and the hive reads
|
||||
# `never_reported` on a dashboard nobody is watching yet.
|
||||
# nothing": a secret without the coordinates it is presented at is
|
||||
# a config an operator believes is working, and its runtime failure
|
||||
# mode is the expensive one — the daemon comes up fine, never
|
||||
# connects, and the hive reads `never_reported` on a dashboard
|
||||
# nobody is watching yet.
|
||||
#
|
||||
# Safe to add to an existing deployment: every `statusPublish`
|
||||
# default is either all-local or all-null, so no config that
|
||||
# evaluates today can be caught by this. It also encodes a
|
||||
# property of the code rather than an intended shape — hive-c0re
|
||||
# genuinely cannot publish with two of three coordinates — which
|
||||
# is the distinction the `serviceDomains'` comment at the top of
|
||||
# this file was written about.
|
||||
# It encodes a property of the code rather than an intended shape —
|
||||
# hive-c0re genuinely cannot publish with two of three coordinates —
|
||||
# which is the distinction the `serviceDomains'` comment at the top
|
||||
# of this file was written about.
|
||||
#
|
||||
# The three coordinates live in two namespaces now: the token
|
||||
# endpoint is the swarm's one address, the other two are this
|
||||
|
|
@ -392,23 +389,21 @@ in
|
|||
# operator told only the option names would look for them under
|
||||
# one prefix and find one of them.
|
||||
#
|
||||
# ⚠️ Asymmetric on purpose. A token endpoint on its own is the
|
||||
# normal state of every hive in a swarm that has an IdP — it says
|
||||
# where the IdP is, not that this hive publishes. Only the two
|
||||
# per-host coordinates are a claim to be publishing, and they
|
||||
# need the endpoint to mean anything.
|
||||
# ⚠️ Asymmetric on purpose. The token endpoint and the queue's URL
|
||||
# both default to the swarm's one address on every hive: they say
|
||||
# where the IdP and the queue are, not that this hive publishes, so
|
||||
# either without the secret is publishing off rather than a half-set
|
||||
# trio. Only the client secret is a claim to be publishing, and it
|
||||
# needs the other two to mean anything.
|
||||
assertion =
|
||||
let
|
||||
local = lib.filter (v: v != null) [
|
||||
deployCfg.hive-controller.statusPublish.natsUrl
|
||||
deployCfg.hive-controller.statusPublish.clientSecretFile
|
||||
];
|
||||
in
|
||||
builtins.length local == 0
|
||||
|| (builtins.length local == 2 && swarmCfg.statusPublish.tokenEndpoint != null);
|
||||
deployCfg.hive-controller.statusPublish.clientSecretFile == null
|
||||
|| (
|
||||
deployCfg.hive-controller.statusPublish.natsUrl != null
|
||||
&& swarmCfg.statusPublish.tokenEndpoint != null
|
||||
);
|
||||
message = ''
|
||||
This hive's status-publishing coordinates have to be set
|
||||
together or not at all — it has only some of them.
|
||||
This hive has a status-publishing client secret but not the
|
||||
coordinates to publish with it.
|
||||
|
||||
Currently:
|
||||
deploy.hive-controller.statusPublish.natsUrl
|
||||
|
|
@ -419,7 +414,7 @@ in
|
|||
= ${toString deployCfg.hive-controller.statusPublish.clientSecretFile}
|
||||
|
||||
Set the missing ones to publish this hive's status to the
|
||||
swarm, or set all three to null to turn publishing off.
|
||||
swarm, or set clientSecretFile to null to turn publishing off.
|
||||
'';
|
||||
}
|
||||
(nameGuards.mustNotEqual {
|
||||
|
|
@ -543,21 +538,22 @@ in
|
|||
options.services.hyperhive.deploy.hive-controller.statusPublish = {
|
||||
natsUrl = lib.mkOption {
|
||||
type = lib.types.nullOr lib.types.str;
|
||||
default =
|
||||
if queueLocal then "tls://${swarmCfg.nats.domain}:${toString swarmCfg.nats.port}" else null;
|
||||
defaultText = lib.literalExpression ''"tls://''${swarm.nats.domain}:''${swarm.nats.port}" when this host runs the queue and the IdP, else null'';
|
||||
default = "tls://${swarmCfg.nats.domain}:${toString swarmCfg.nats.port}";
|
||||
defaultText = lib.literalExpression ''"tls://''${swarm.nats.domain}:''${swarm.nats.port}"'';
|
||||
example = "tls://nats.example.com:4222";
|
||||
description = ''
|
||||
Where the swarm queue listens, as seen from *this* hive.
|
||||
|
||||
Defaults to the queue's name when this host runs the queue and the
|
||||
IdP. A hive that is not the swarm host sets the same URL: the name
|
||||
resolves through the operator's DNS there. TLS only, and by name,
|
||||
since the queue's certificate carries the name and no address.
|
||||
Defaults to the queue's name on every hive. The queue's own host
|
||||
resolves it locally, and any other hive through the operator's
|
||||
DNS. TLS only, and by name, since the queue's certificate carries
|
||||
the name and no address.
|
||||
|
||||
Null disables status publishing: this hive computes its own
|
||||
readiness as always, and simply offers it to nobody. The swarm
|
||||
controller then reports it `never_reported`, which is the honest
|
||||
A URL alone does not publish.
|
||||
{option}`services.hyperhive.deploy.hive-controller.statusPublish.clientSecretFile`
|
||||
is what turns publishing on. Without it this hive computes its own
|
||||
readiness as always and offers it to nobody, and the swarm
|
||||
controller reports it `never_reported`, which is the honest
|
||||
reading.
|
||||
'';
|
||||
};
|
||||
|
|
@ -579,6 +575,9 @@ in
|
|||
readable, and one in the environment is readable by anything
|
||||
that can open {file}`/proc/<pid>/environ`.
|
||||
|
||||
Setting it is what makes this hive publish its status; null
|
||||
turns publishing off.
|
||||
|
||||
Defaults to authelia's own minted secret when the IdP runs on
|
||||
this host. On any other hive the secret has to get here somehow,
|
||||
and the swarm does not distribute it — copy it out of the swarm
|
||||
|
|
@ -598,25 +597,24 @@ in
|
|||
# owns `queue.agentCredentialDir` in the same namespace.
|
||||
options.services.hyperhive.deploy.hive-controller.queue.agentNatsUrl = lib.mkOption {
|
||||
type = lib.types.nullOr lib.types.str;
|
||||
default =
|
||||
if queueLocal then "tls://${swarmCfg.nats.domain}:${toString swarmCfg.nats.port}" else null;
|
||||
defaultText = lib.literalExpression ''"tls://''${swarm.nats.domain}:''${swarm.nats.port}" when this host runs the queue and the IdP, else null'';
|
||||
default = "tls://${swarmCfg.nats.domain}:${toString swarmCfg.nats.port}";
|
||||
defaultText = lib.literalExpression ''"tls://''${swarm.nats.domain}:''${swarm.nats.port}"'';
|
||||
example = "tls://nats.example.com:4222";
|
||||
description = ''
|
||||
Where the swarm queue listens, as an agent *container* on this host
|
||||
reaches it.
|
||||
|
||||
Defaults to the queue's name when this host runs the queue. The agent
|
||||
resolves it through the bridge, where dnsmasq answers it with the
|
||||
bridge address. ⚠️ Never a loopback address: inside an agent's network
|
||||
namespace `127.0.0.1` is the agent, not this host, and the queue's
|
||||
certificate names no address anyway.
|
||||
Defaults to the queue's name on every hive. The agent resolves it
|
||||
through the bridge: on the queue's own host dnsmasq answers it with
|
||||
the bridge address, and elsewhere it forwards to the operator's DNS,
|
||||
as for the store's name. ⚠️ Never a loopback address: inside an
|
||||
agent's network namespace `127.0.0.1` is the agent, not this host,
|
||||
and the queue's certificate names no address anyway.
|
||||
|
||||
Null means this hive's agents have not been given the queue's address.
|
||||
Together with
|
||||
{option}`services.hyperhive.swarm.statusPublish.tokenEndpoint` it is
|
||||
what decides whether the harness is handed queue coordinates at all; a
|
||||
hive whose queue is elsewhere names the address its containers route to.
|
||||
what decides whether the harness is handed queue coordinates at all.
|
||||
'';
|
||||
};
|
||||
|
||||
|
|
|
|||
|
|
@ -42,6 +42,10 @@ let
|
|||
# renders the same absences `bare` does.
|
||||
centralToggleOff = hive { enable = false; };
|
||||
|
||||
# Every hive defaults the agents' queue address to the queue's name, so the
|
||||
# only hive without one is a hive told to have none.
|
||||
noAgentQueue = hive { deploy.hive-controller.queue.agentNatsUrl = null; };
|
||||
|
||||
withCi = hive { deploy.forgejo.ci.enable = true; };
|
||||
|
||||
# A priority collision is a property of the *option*, not
|
||||
|
|
@ -149,13 +153,13 @@ let
|
|||
}
|
||||
{
|
||||
# The absence arm, and what makes the two above able to fail: a hive
|
||||
# with no queue address must forward neither coordinate, because half a
|
||||
# pair reaches the harness as a partial configuration rather than as
|
||||
# none.
|
||||
name = "a hive with no swarm queue forwards no agent queue coordinates";
|
||||
# with no queue address for its agents must forward neither coordinate,
|
||||
# because half a pair reaches the harness as a partial configuration
|
||||
# rather than as none.
|
||||
name = "a hive with no agent queue address forwards no agent queue coordinates";
|
||||
ok =
|
||||
let
|
||||
e = bare.systemd.services.hive-c0re.environment;
|
||||
e = noAgentQueue.systemd.services.hive-c0re.environment;
|
||||
in
|
||||
!(e ? HIVE_AGENT_NATS_URL) && !(e ? HIVE_AGENT_OIDC_TOKEN_ENDPOINT);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -47,6 +47,40 @@ let
|
|||
deploy.nats.autoGenerateCallout = true;
|
||||
};
|
||||
|
||||
# A hive that is not the queue's host, with nothing about the queue's
|
||||
# address set by hand: what every hive but one in a multi-host swarm looks
|
||||
# like. The controller is on too, since it may run away from the queue.
|
||||
#
|
||||
# The two secrets are the ones a hive away from authelia already has to be
|
||||
# handed, and neither is an address; without them this hive would fail
|
||||
# assertions that have nothing to do with the queue.
|
||||
remoteSecrets = {
|
||||
deploy.forgejo.sso.clientSecretFile = "/var/lib/forgejo-oidc/by-hand.secret";
|
||||
deploy.swarm-controller.queue.clientSecretFile = "/var/lib/secrets/swarm-controller.secret";
|
||||
};
|
||||
remote = hive (lib.recursiveUpdate remoteSecrets { deploy.swarm-controller.enable = true; });
|
||||
|
||||
# The same hive given its status secret by hand, which is what turns
|
||||
# publishing on away from the IdP's host.
|
||||
remotePublishing = hive (
|
||||
lib.recursiveUpdate remoteSecrets {
|
||||
deploy.hive-controller.statusPublish.clientSecretFile = "/var/lib/secrets/hive-h1.secret";
|
||||
}
|
||||
);
|
||||
|
||||
# A real half-config: the secret, with the URL it would be presented at
|
||||
# taken away.
|
||||
remoteSecretNoUrl = hive (
|
||||
lib.recursiveUpdate remoteSecrets {
|
||||
deploy.hive-controller.statusPublish.clientSecretFile = "/var/lib/secrets/hive-h1.secret";
|
||||
deploy.hive-controller.statusPublish.natsUrl = null;
|
||||
}
|
||||
);
|
||||
|
||||
failedAssertions = m: lib.filter (a: !a.assertion) m.assertions;
|
||||
refusedStatusSecret =
|
||||
m: lib.any (a: lib.hasInfix "status-publishing client secret" a.message) (failedAssertions m);
|
||||
|
||||
policyScript = allLocal.systemd.services.swarm-bao-nats-tls-policy.script;
|
||||
leafUnit = allLocal.systemd.services.swarm-bao-nats-tls;
|
||||
natsContainer = allLocal.containers.swarm-nats.config;
|
||||
|
|
@ -78,6 +112,8 @@ let
|
|||
containerUnits = lib.concatLists (
|
||||
lib.mapAttrsToList (c: cc: fromUnits c cc.config.systemd.services) machine.containers
|
||||
);
|
||||
# The responder runs beside the queue only, so a hive without one has
|
||||
# none to read.
|
||||
responder =
|
||||
map
|
||||
(u: {
|
||||
|
|
@ -85,7 +121,9 @@ let
|
|||
value = u;
|
||||
})
|
||||
(
|
||||
urlsIn machine.containers.swarm-nats.config.systemd.services.swarm-nats-auth.serviceConfig.ExecStart
|
||||
lib.optionals (machine.containers ? swarm-nats) (
|
||||
urlsIn machine.containers.swarm-nats.config.systemd.services.swarm-nats-auth.serviceConfig.ExecStart
|
||||
)
|
||||
);
|
||||
in
|
||||
lib.listToAttrs (fromUnits "host" machine.systemd.services ++ containerUnits ++ responder);
|
||||
|
|
@ -212,6 +250,65 @@ let
|
|||
!(lib.elem "swarm-bao-nats-tls-policy.service" u.after)
|
||||
&& !(lib.elem "swarm-bao-nats-tls-policy.service" u.wants);
|
||||
}
|
||||
{
|
||||
# Every hive dials the queue by the same name, so a hive away from it
|
||||
# needs no URL of its own. Control and property in one: the scan must
|
||||
# reach the controller and the agents' address here, and every URL it
|
||||
# finds, like the options it reads through, is the name.
|
||||
name = "a hive that is not the queue's host dials tls://<the queue's name>:4222 with nothing set";
|
||||
ok =
|
||||
let
|
||||
s = clientUrls remote;
|
||||
d = remote.services.hyperhive.deploy;
|
||||
in
|
||||
!d.nats.enable
|
||||
&& s ? "host/hive-c0re/HIVE_AGENT_NATS_URL"
|
||||
&& s ? "host/swarm-controller/SWARM_CONTROLLER_NATS_URL"
|
||||
&& lib.all (u: u == natsUrl) (lib.attrValues s)
|
||||
&& d.hive-controller.statusPublish.natsUrl == natsUrl
|
||||
&& d.hive-controller.queue.agentNatsUrl == natsUrl
|
||||
&& remote.services.hyperhive.swarm.controller.queue.natsUrl == natsUrl;
|
||||
}
|
||||
{
|
||||
# The URL is set and the secret is not, which is every such hive until
|
||||
# an operator places one: publishing is off, not misconfigured.
|
||||
name = "that hive evaluates without an assertion failure";
|
||||
ok = failedAssertions remote == [ ];
|
||||
}
|
||||
{
|
||||
# Off means off: hive-c0re is handed no status coordinates, rather
|
||||
# than a secret path nothing fills.
|
||||
name = "without its status secret, that hive's hive-c0re is given no status coordinates";
|
||||
ok =
|
||||
let
|
||||
s = remote.systemd.services.hive-c0re;
|
||||
in
|
||||
!(s.environment ? HIVE_C0RE_NATS_URL)
|
||||
&& !(s.environment ? HIVE_C0RE_OIDC_CLIENT_SECRET_FILE)
|
||||
&& !(lib.any (lib.hasPrefix "swarm-status-client.secret:") (
|
||||
lib.toList (s.serviceConfig.LoadCredential or [ ])
|
||||
));
|
||||
}
|
||||
{
|
||||
# The secret alone turns publishing on, at the default URL.
|
||||
name = "given its status secret, that hive publishes to the queue's name";
|
||||
ok =
|
||||
let
|
||||
s = remotePublishing.systemd.services.hive-c0re;
|
||||
in
|
||||
failedAssertions remotePublishing == [ ]
|
||||
&& s.environment.HIVE_C0RE_NATS_URL == natsUrl
|
||||
&& s.environment.HIVE_C0RE_OIDC_CLIENT_SECRET_FILE == "%d/swarm-status-client.secret"
|
||||
&& lib.elem "swarm-status-client.secret:/var/lib/secrets/hive-h1.secret" (
|
||||
lib.toList s.serviceConfig.LoadCredential
|
||||
);
|
||||
}
|
||||
{
|
||||
# What the assertion was written for is still refused: a secret with
|
||||
# nowhere to present it.
|
||||
name = "a status secret without a queue URL is refused at eval";
|
||||
ok = refusedStatusSecret remoteSecretNoUrl && !(refusedStatusSecret remote);
|
||||
}
|
||||
];
|
||||
in
|
||||
runGroup "nats-tls" cases
|
||||
|
|
|
|||
Loading…
Reference in a new issue