hive status publishing: read the hive's OIDC secret from the store, retry until it is there
hive-c0re publishes this hive's status to the swarm queue as the OIDC client `hive-<name>`. Its secret came from `deploy.hive-controller.statusPublish.clientSecretFile`, which defaulted to authelia's minted file only on a host running both authelia and the queue. Every other hive had no secret and never published, so the controller showed it `never_reported`. Now `swarm-secret-publish`, beside authelia, also copies each hive's `hive-<name>` secret to `swarm/hives/<hive>/oidc/client`. Its policy already allows `swarm/hives/*`. hive-c0re reads that path under the hive's own store leaf, which the existing `swarm/hives/<hive>/*` grant covers, and holds the value in memory. The shared queue client no longer remembers a failed first connect for the life of the process. A failed attempt is retried with a backoff from 5s, doubling to a cap of 5 minutes; the dashboard banner stays up until an attempt succeeds. The two status publishers wait for the client instead of giving up on the first `None`, so a secret published after boot is picked up without a restart. Removed: the `statusPublish.clientSecretFile` option (both spellings, now removed options with a message), its same-host default, its LoadCredential, the assertion tying it to the other two coordinates, and `swarm_queue_client::QueueConfig::from_env`, whose only caller was this read. Publishing is now on wherever the queue URL and token endpoint are set and the hive holds a store leaf. Refs #5015
This commit is contained in:
parent
8d206f124f
commit
c2c61b6146
19 changed files with 643 additions and 394 deletions
|
|
@ -316,26 +316,18 @@ in
|
|||
# the daemon as well would put a secret on a process that has nowhere
|
||||
# to present it.
|
||||
LoadCredential =
|
||||
# The swarm-queue client secret this hive authenticates with to
|
||||
# publish its own status. `LoadCredential` and not a copy: root
|
||||
# reads the plaintext at unit start and hive-core sees it 0400
|
||||
# under `%d`, so the secret never gains a second on-disk copy
|
||||
# and the daemon never needs read access to wherever it lives.
|
||||
# (The callout responder copies instead only because it
|
||||
# delivers into a container, across a filesystem boundary.)
|
||||
lib.optional
|
||||
(config.services.hyperhive.deploy.hive-controller.statusPublish.clientSecretFile != null)
|
||||
"swarm-status-client.secret:${config.services.hyperhive.deploy.hive-controller.statusPublish.clientSecretFile}"
|
||||
# The secret store's client identity, on the same reasoning one
|
||||
# paragraph up — with a sharper edge: ./glue-bao-tls.nix mints the
|
||||
# key `0600` inside a `0700` directory, so hive-core cannot read it
|
||||
# at all. Loading it as a credential is what makes the store
|
||||
# reachable from an unprivileged daemon without widening either.
|
||||
# The secret store's client identity. `LoadCredential` and not a
|
||||
# copy: root reads it at unit start and hive-core sees it 0400
|
||||
# under `%d`, so it never gains a second on-disk copy. And
|
||||
# ./glue-bao-tls.nix mints the key `0600` inside a `0700`
|
||||
# directory, so hive-core cannot read the original at all.
|
||||
# Loading it as a credential is what makes the store reachable
|
||||
# from an unprivileged daemon without widening either.
|
||||
# The gate is the identity, never `deploy.bao.enable`: a hive that
|
||||
# reads a store on another machine holds one of these and runs no
|
||||
# store. ./glue-queue-agent-credential.nix gates its own reader the
|
||||
# same way.
|
||||
++ lib.optionals baoClientIdentity [
|
||||
lib.optionals baoClientIdentity [
|
||||
"bao-client.pem:${baoDeploy.clientCertFile}"
|
||||
"bao-client-key.pem:${baoDeploy.clientKeyFile}"
|
||||
]
|
||||
|
|
|
|||
|
|
@ -223,22 +223,20 @@ in
|
|||
}
|
||||
//
|
||||
# Swarm-queue coordinates for offering this hive's status upward
|
||||
# (hive-c0re::swarm_status). All four together or none: a half-set
|
||||
# (hive-c0re::swarm_status). All three together or none: a half-set
|
||||
# environment is a deployment bug the daemon refuses to treat as
|
||||
# "no queue configured", because the failure it would otherwise
|
||||
# produce is a hive that comes up fine and silently never reports.
|
||||
# The rule that a secret needs the other two options is asserted at
|
||||
# eval in ./../swarm.nix, so this can only ever emit a complete set.
|
||||
#
|
||||
# ⚠️ Gated on the SECRET as well as the URL. The URL defaults to the
|
||||
# queue's name on every hive, so on its own it says where the queue is,
|
||||
# not that this hive publishes; the secret is that claim. Gated on the
|
||||
# URL alone, a hive with no secret would be handed a secret path that
|
||||
# `LoadCredential` in ./default.nix never fills.
|
||||
# The client secret is not among them: the daemon reads it from the
|
||||
# swarm's secret store under this hive's own certificate
|
||||
# (`swarm_secret_client::queue::hive_client_path`), so a hive with no
|
||||
# store identity has no way to publish and is handed nothing.
|
||||
lib.optionalAttrs
|
||||
(
|
||||
config.services.hyperhive.deploy.hive-controller.statusPublish.natsUrl != null
|
||||
&& config.services.hyperhive.deploy.hive-controller.statusPublish.clientSecretFile != null
|
||||
&& config.services.hyperhive.swarm.statusPublish.tokenEndpoint != null
|
||||
&& haveBaoClientIdentity
|
||||
)
|
||||
{
|
||||
HIVE_C0RE_NATS_URL = config.services.hyperhive.deploy.hive-controller.statusPublish.natsUrl;
|
||||
|
|
@ -247,9 +245,6 @@ in
|
|||
# `swarm.hives` — the hive does not choose its own name here, it uses
|
||||
# the one the roster gave it.
|
||||
HIVE_C0RE_OIDC_CLIENT_ID = "hive-${config.services.hyperhive.hiveName}";
|
||||
# `%d` is systemd's credentials directory — see the LoadCredential in
|
||||
# ./default.nix. The daemon reads a path, never a value.
|
||||
HIVE_C0RE_OIDC_CLIENT_SECRET_FILE = "%d/swarm-status-client.secret";
|
||||
}
|
||||
// {
|
||||
# Where ../glue-queue-agent-credential.nix lands the AGENTS' queue
|
||||
|
|
|
|||
Loading…
Reference in a new issue