swarm: say "no queue coordinates", never "a hive with no queue"
The swarm always has exactly one queue; a hive can only lack its address. Reworded every prose site this PR added that stated or implied the opposite, to name what is actually absent (coordinates, credential, or address) instead of the queue itself. Refs #3805
This commit is contained in:
parent
86652f051a
commit
2989c5ccdb
8 changed files with 31 additions and 26 deletions
|
|
@ -385,9 +385,9 @@ above, plus `HIVE_AGENT_OIDC_CLIENT_SECRET_FILE` and
|
||||||
`HIVE_AGENT_OIDC_CLIENT_ID_FILE`, which point into the unit's own credentials
|
`HIVE_AGENT_OIDC_CLIENT_ID_FILE`, which point into the unit's own credentials
|
||||||
directory. The last two come from the delivered credential rather than from
|
directory. The last two come from the delivered credential rather than from
|
||||||
config — see [`secrets.md`](secrets.md#hive-level--one-of-each-per-hive) for
|
config — see [`secrets.md`](secrets.md#hive-level--one-of-each-per-hive) for
|
||||||
how it gets there. A hive with no queue sets none of the four and each agent
|
how it gets there. A hive that hasn't been given the queue's address for its
|
||||||
logs that it has none; a half-set environment logs an error and the harness
|
agents sets none of the four and each agent logs that it has none; a half-set
|
||||||
keeps serving.
|
environment logs an error and the harness keeps serving.
|
||||||
|
|
||||||
### Swarm-wide forge webhooks
|
### Swarm-wide forge webhooks
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -180,7 +180,7 @@ reads them out of its own `$CREDENTIALS_DIRECTORY`, owned by the agent user
|
||||||
and by nobody else. `hive-c0re` never reads the bytes at any point: it runs as
|
and by nobody else. `hive-c0re` never reads the bytes at any point: it runs as
|
||||||
`hive-core`, which is the reason a copy wasn't an option either. When the
|
`hive-core`, which is the reason a copy wasn't an option either. When the
|
||||||
files aren't there the daemon forwards nothing and says so in its journal, and
|
files aren't there the daemon forwards nothing and says so in its journal, and
|
||||||
the harness logs that it has no queue — the same absent-and-legal state, twice,
|
the harness logs that it has no queue coordinates — the same absent-and-legal state, twice,
|
||||||
rather than a container that refuses to start.
|
rather than a container that refuses to start.
|
||||||
|
|
||||||
⚠️ **Service↔store mTLS is its own trust domain.** A credential you must
|
⚠️ **Service↔store mTLS is its own trust domain.** A credential you must
|
||||||
|
|
|
||||||
|
|
@ -61,7 +61,7 @@ impl QueueEnv {
|
||||||
enum Resolution {
|
enum Resolution {
|
||||||
/// Everything is here; the agent can reach the queue.
|
/// Everything is here; the agent can reach the queue.
|
||||||
Configured(Box<QueueConfig>),
|
Configured(Box<QueueConfig>),
|
||||||
/// No queue for this agent, and that is a legal state — carries why.
|
/// No queue coordinates for this agent, and that is a legal state — carries why.
|
||||||
Absent(&'static str),
|
Absent(&'static str),
|
||||||
/// Some of the environment, not all of it. A deployment bug rather than
|
/// Some of the environment, not all of it. A deployment bug rather than
|
||||||
/// an absent integration, so it is reported and then survived.
|
/// an absent integration, so it is reported and then survived.
|
||||||
|
|
@ -141,7 +141,7 @@ pub fn init() {
|
||||||
Some(*cfg)
|
Some(*cfg)
|
||||||
}
|
}
|
||||||
Resolution::Absent(why) => {
|
Resolution::Absent(why) => {
|
||||||
tracing::info!(why, "no swarm queue for this agent");
|
tracing::info!(why, "no swarm queue coordinates for this agent");
|
||||||
None
|
None
|
||||||
}
|
}
|
||||||
Resolution::Partial => {
|
Resolution::Partial => {
|
||||||
|
|
@ -219,9 +219,10 @@ mod tests {
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
/// A hive with no swarm queue at all. Silence here is correct, and it has
|
/// A hive that has not been given its queue's coordinates at all. Silence
|
||||||
/// to be distinguishable from the half-set case below — that distinction
|
/// here is correct, and it has to be distinguishable from the half-set
|
||||||
/// is the only thing that makes the error branch worth logging.
|
/// case below — that distinction is the only thing that makes the error
|
||||||
|
/// branch worth logging.
|
||||||
#[test]
|
#[test]
|
||||||
fn an_empty_environment_is_no_queue_rather_than_an_error() {
|
fn an_empty_environment_is_no_queue_rather_than_an_error() {
|
||||||
let e = env([None, None, None, None]);
|
let e = env([None, None, None, None]);
|
||||||
|
|
|
||||||
|
|
@ -133,8 +133,8 @@ fn bind_child_agent_dirs(child: &str, binds: &mut Vec<BindMount>) {
|
||||||
/// Env var naming the host directory `swarm-bao-queue-agent.service` lands
|
/// Env var naming the host directory `swarm-bao-queue-agent.service` lands
|
||||||
/// this hive's agent queue credential in. Set by the hive-c0re NixOS module
|
/// this hive's agent queue credential in. Set by the hive-c0re NixOS module
|
||||||
/// from `deploy.hive-controller.queue.agentCredentialDir`; absent means this
|
/// from `deploy.hive-controller.queue.agentCredentialDir`; absent means this
|
||||||
/// daemon runs outside its unit, which is the same "no queue" answer as an
|
/// daemon runs outside its unit, which is the same "no queue credential"
|
||||||
/// empty directory.
|
/// answer as an empty directory.
|
||||||
const QUEUE_CREDENTIAL_DIR_ENV: &str = "HIVE_C0RE_AGENT_QUEUE_CREDENTIAL_DIR";
|
const QUEUE_CREDENTIAL_DIR_ENV: &str = "HIVE_C0RE_AGENT_QUEUE_CREDENTIAL_DIR";
|
||||||
|
|
||||||
/// systemd credential ids the two files arrive under inside the container.
|
/// systemd credential ids the two files arrive under inside the container.
|
||||||
|
|
@ -167,7 +167,7 @@ fn queue_agent_credentials(agent_name: &str, dir: Option<&Path>) -> Vec<Credenti
|
||||||
let Some(dir) = dir else {
|
let Some(dir) = dir else {
|
||||||
tracing::info!(
|
tracing::info!(
|
||||||
%agent_name,
|
%agent_name,
|
||||||
"no {QUEUE_CREDENTIAL_DIR_ENV} in this daemon's environment — agent gets no swarm queue"
|
"no {QUEUE_CREDENTIAL_DIR_ENV} in this daemon's environment — agent gets no swarm queue credential"
|
||||||
);
|
);
|
||||||
return Vec::new();
|
return Vec::new();
|
||||||
};
|
};
|
||||||
|
|
@ -179,7 +179,7 @@ fn queue_agent_credentials(agent_name: &str, dir: Option<&Path>) -> Vec<Credenti
|
||||||
if !secret.is_file() || !client_id.is_file() {
|
if !secret.is_file() || !client_id.is_file() {
|
||||||
tracing::info!(
|
tracing::info!(
|
||||||
%agent_name, dir = %dir.display(),
|
%agent_name, dir = %dir.display(),
|
||||||
"swarm queue credential not published yet — agent gets no swarm queue"
|
"swarm queue credential not published yet — agent gets no swarm queue credential"
|
||||||
);
|
);
|
||||||
return Vec::new();
|
return Vec::new();
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -732,7 +732,7 @@ const FORWARDED_VARS: &[&str] = &[
|
||||||
const FORWARDED_VAR_OPTIONS: &[(&str, &str)] = &[
|
const FORWARDED_VAR_OPTIONS: &[(&str, &str)] = &[
|
||||||
// Read at build time to decide whether the harness unit declares the
|
// Read at build time to decide whether the harness unit declares the
|
||||||
// queue credential at all — see `nix/agent-modules/queue.nix`. An agent
|
// queue credential at all — see `nix/agent-modules/queue.nix`. An agent
|
||||||
// whose option says no queue and whose env says otherwise logs the
|
// whose option says no queue coordinates and whose env says otherwise logs the
|
||||||
// partial-config error rather than half-connecting, which is the same
|
// partial-config error rather than half-connecting, which is the same
|
||||||
// failure this map's doc describes and the reason both halves are wired.
|
// failure this map's doc describes and the reason both halves are wired.
|
||||||
("HIVE_AGENT_NATS_URL", "hyperhive.queue.natsUrl"),
|
("HIVE_AGENT_NATS_URL", "hyperhive.queue.natsUrl"),
|
||||||
|
|
|
||||||
|
|
@ -38,10 +38,10 @@ in
|
||||||
which is the bridge address rather than a loopback one — inside this
|
which is the bridge address rather than a loopback one — inside this
|
||||||
container `127.0.0.1` is the agent itself.
|
container `127.0.0.1` is the agent itself.
|
||||||
|
|
||||||
`null` means this hive has no queue, and the harness then declares no
|
`null` means this hive has not been given the queue's address for its
|
||||||
credential and logs that it has none. It is deliberately not defaulted
|
agents, and the harness then declares no credential and logs that it
|
||||||
to anything: a guessed address builds fine and talks to the wrong
|
has none. It is deliberately not defaulted to anything: a guessed
|
||||||
machine.
|
address builds fine and talks to the wrong machine.
|
||||||
'';
|
'';
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|
@ -56,7 +56,7 @@ in
|
||||||
|
|
||||||
Set together with {option}`hyperhive.queue.natsUrl` or not at all —
|
Set together with {option}`hyperhive.queue.natsUrl` or not at all —
|
||||||
the harness treats a half-set pair as a deployment bug rather than as
|
the harness treats a half-set pair as a deployment bug rather than as
|
||||||
"no queue".
|
"no queue coordinates".
|
||||||
'';
|
'';
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
|
|
|
||||||
|
|
@ -611,7 +611,8 @@ in
|
||||||
interface alone. ⚠️ Never a loopback address — inside an agent's network
|
interface alone. ⚠️ Never a loopback address — inside an agent's network
|
||||||
namespace `127.0.0.1` is the agent, not this host.
|
namespace `127.0.0.1` is the agent, not this host.
|
||||||
|
|
||||||
Null means this hive's agents have no queue. Together with
|
Null means this hive's agents have not been given the queue's address.
|
||||||
|
Together with
|
||||||
{option}`services.hyperhive.swarm.statusPublish.tokenEndpoint` it is
|
{option}`services.hyperhive.swarm.statusPublish.tokenEndpoint` it is
|
||||||
what decides whether the harness is handed queue coordinates at all; a
|
what decides whether the harness is handed queue coordinates at all; a
|
||||||
hive whose queue is elsewhere names the address its containers route to.
|
hive whose queue is elsewhere names the address its containers route to.
|
||||||
|
|
|
||||||
|
|
@ -1278,7 +1278,8 @@ let
|
||||||
}
|
}
|
||||||
{
|
{
|
||||||
# The agents mint against the swarm's IdP, the same endpoint the hive's
|
# The agents mint against the swarm's IdP, the same endpoint the hive's
|
||||||
# own client uses — a hive-local guess would be a token no queue accepts.
|
# own client uses — a hive-local guess would produce a token the queue
|
||||||
|
# would not accept.
|
||||||
name = "the agents' token endpoint is the swarm IdP's";
|
name = "the agents' token endpoint is the swarm IdP's";
|
||||||
ok =
|
ok =
|
||||||
let
|
let
|
||||||
|
|
@ -1289,8 +1290,9 @@ let
|
||||||
}
|
}
|
||||||
{
|
{
|
||||||
# The absence arm, and what makes the two above able to fail: a hive
|
# The absence arm, and what makes the two above able to fail: a hive
|
||||||
# with no queue must forward neither coordinate, because half a pair
|
# with no queue address must forward neither coordinate, because half a
|
||||||
# reaches the harness as a partial configuration rather than as none.
|
# pair reaches the harness as a partial configuration rather than as
|
||||||
|
# none.
|
||||||
name = "a hive with no swarm queue forwards no agent queue coordinates";
|
name = "a hive with no swarm queue forwards no agent queue coordinates";
|
||||||
ok =
|
ok =
|
||||||
let
|
let
|
||||||
|
|
@ -1321,9 +1323,10 @@ let
|
||||||
&& e.HIVE_AGENT_OIDC_CLIENT_ID_FILE == "%d/hive-queue-agent-client-id";
|
&& e.HIVE_AGENT_OIDC_CLIENT_ID_FILE == "%d/hive-queue-agent-client-id";
|
||||||
}
|
}
|
||||||
{
|
{
|
||||||
# An agent built before its hive had a queue. It must declare nothing
|
# An agent built before its hive was handed the queue's address. It
|
||||||
# rather than name a credential that never arrives — and the harness
|
# must declare nothing rather than name a credential that never
|
||||||
# then reports "no queue" instead of a half-set environment.
|
# arrives — and the harness then reports "no queue coordinates"
|
||||||
|
# instead of a half-set environment.
|
||||||
name = "an agent with no queue coordinates declares no credential";
|
name = "an agent with no queue coordinates declares no credential";
|
||||||
ok =
|
ok =
|
||||||
let
|
let
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue