swarm-bao: set the cluster address raft refuses to start without
openbao has never started on a hive that deploys it. With raft storage
and no `cluster_addr`, `bao server` exits 1 before serving anything:
cluster address must be set when using raft storage
Reproduced against openbao 2.6.2 with the module's own config shape:
the same JSON minus `cluster_addr` exits 1 with that line, and with it
the server comes up and configures every listener.
Both addresses are built from `swarm.bao.domain` rather than a bind
address — that is the URL a reader already dials (`BAO_ADDR` in
glue-matrix-bao-token) and the name the server certificate carries.
Cluster traffic sits one port up, upstream's convention.
The `unknown or unsupported field loopback` warning in the same startup
log is unrelated and cosmetic: openbao's unknown-field check does not
know about named listener blocks, but the parser honours `type` and
configures each one (measured, two named listeners, both served).
Noted in the module so the next reader does not chase it; the JSON
array form that avoids the warning does not typecheck against nixpkgs'
`settings.listener` (`attrsOf`).
module-eval gains the regression gate plus a control that the settings
it reads vary per deployment.
Refs #3860
This commit is contained in:
parent
f3ce94b4f6
commit
75a6101f66
2 changed files with 39 additions and 0 deletions
|
|
@ -158,6 +158,10 @@ let
|
||||||
#
|
#
|
||||||
# Neither is a superset of the other, which is why this is not one address
|
# Neither is a superset of the other, which is why this is not one address
|
||||||
# with a conditional value.
|
# with a conditional value.
|
||||||
|
# ⚠️ openbao logs `unknown or unsupported field <name>` for each key here. Its
|
||||||
|
# unknown-field check does not know about named listener blocks; the parser
|
||||||
|
# does, honours `type`, and configures every one. Not the cause of a store
|
||||||
|
# that fails to start — look at `advertise` below.
|
||||||
listeners = {
|
listeners = {
|
||||||
loopback = {
|
loopback = {
|
||||||
type = "tcp";
|
type = "tcp";
|
||||||
|
|
@ -166,6 +170,18 @@ let
|
||||||
// listenerTls;
|
// listenerTls;
|
||||||
}
|
}
|
||||||
// extraListeners;
|
// extraListeners;
|
||||||
|
|
||||||
|
# Raft REFUSES TO START without `cluster_addr`, and the message names neither
|
||||||
|
# the setting nor the stanza: "cluster address must be set when using raft
|
||||||
|
# storage".
|
||||||
|
#
|
||||||
|
# By name and not by address: this is the URL a reader dials, and the name the
|
||||||
|
# server certificate has to carry anyway. Cluster traffic is one port up,
|
||||||
|
# upstream's own convention.
|
||||||
|
advertise = {
|
||||||
|
api_addr = "https://${cfg.domain}:${toString cfg.port}";
|
||||||
|
cluster_addr = "https://${cfg.domain}:${toString (cfg.port + 1)}";
|
||||||
|
};
|
||||||
in
|
in
|
||||||
{
|
{
|
||||||
# One service, two namespaces, and the split decides who may set what.
|
# One service, two namespaces, and the split decides who may set what.
|
||||||
|
|
@ -615,6 +631,7 @@ in
|
||||||
listener = listeners;
|
listener = listeners;
|
||||||
storage.raft.path = stateDir;
|
storage.raft.path = stateDir;
|
||||||
}
|
}
|
||||||
|
// advertise
|
||||||
// sealSettings;
|
// sealSettings;
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -105,6 +105,16 @@ let
|
||||||
|
|
||||||
baoNames = machine: machine.services.hyperhive.gateway.localNames;
|
baoNames = machine: machine.services.hyperhive.gateway.localNames;
|
||||||
|
|
||||||
|
baoTwoAddresses = hive {
|
||||||
|
deploy.bao.enable = true;
|
||||||
|
deploy.bao.extraListenAddresses = [ "10.0.0.1" ];
|
||||||
|
};
|
||||||
|
|
||||||
|
# The config file openbao parses, not the nix that produces it: a setting it
|
||||||
|
# requires is absent here without anything in the module system minding, so
|
||||||
|
# the daemon's own startup is otherwise the first reader.
|
||||||
|
baoSettings = machine: machine.containers.swarm-bao.config.services.openbao.settings;
|
||||||
|
|
||||||
# A priority collision is a property of the *option*, not
|
# A priority collision is a property of the *option*, not
|
||||||
# of the merged value's interior — nix throws the moment the value is
|
# of the merged value's interior — nix throws the moment the value is
|
||||||
# demanded at all, so `seq`-ing each `serviceConfig` value to WHNF is
|
# demanded at all, so `seq`-ing each `serviceConfig` value to WHNF is
|
||||||
|
|
@ -264,6 +274,18 @@ let
|
||||||
name = "a hive that does not run the store claims no name for it";
|
name = "a hive that does not run the store claims no name for it";
|
||||||
ok = !(builtins.elem "bao.t.local" (baoNames bare));
|
ok = !(builtins.elem "bao.t.local" (baoNames bare));
|
||||||
}
|
}
|
||||||
|
{
|
||||||
|
# Raft refuses to start without it, and says so in a message that names
|
||||||
|
# neither the setting nor the stanza.
|
||||||
|
name = "the store advertises a cluster address";
|
||||||
|
ok = lib.hasPrefix "https://" ((baoSettings baoPkcs11).cluster_addr or "");
|
||||||
|
}
|
||||||
|
{
|
||||||
|
# Control for the case above: these settings are rendered per deployment,
|
||||||
|
# not constants a passing case could be indifferent to.
|
||||||
|
name = "a declared extra address renders a second listener beside loopback";
|
||||||
|
ok = builtins.length (builtins.attrNames (baoSettings baoTwoAddresses).listener) == 2;
|
||||||
|
}
|
||||||
];
|
];
|
||||||
|
|
||||||
bad = builtins.filter (c: !c.ok) cases;
|
bad = builtins.filter (c: !c.ok) cases;
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue