Watch
0
0
Fork
You've already forked hyperhive
0

swarm-controller: first credential-renewal pass waits for the queue connection

The renewal pass ran the moment swarm-controller started, before its
queue client had connected, so `WantedWriter::view` refused with
"client state Pending" and the pass logged
`agent credential renewal: pass failed; retrying next tick`. That fired
twice in 24h on muede-lpt2, both under a second after start, and would
trip a Grafana rule on that WARN on ordinary restarts.

The first pass now waits until the queue client is connected, polling
`swarm_queue_client::ensure_connected` every 5s the way
`AgentIconReader::create_when_connected` does. The wait is bounded by
one RECONCILE_INTERVAL (5 min): the old code's retry after a false start
also came one interval later, so a queue that never connects gets its
first pass no later than before. Hitting the bound logs one WARN and runs
the pass anyway. Only startup waits; a later disconnect still fails a
pass and logs the WARN.

Refs #4717
This commit is contained in:
atlas 2026-09-30 15:05:25 +02:00 • committed by mara
commit 9e06e191a3
3 changed files with 151 additions and 39 deletions

View file

@ -69,6 +69,12 @@ impl WantedWriter {
Self { client }
}
/// Whether the queue is connected, which [`Self::view`] and [`Self::set`]
/// require.
pub fn is_connected(&self) -> bool {
swarm_queue_client::ensure_connected(&self.client).is_ok()
}
/// The declaration currently published for `hive`, or `None`.
pub async fn view(&self, hive: &str) -> Result<Option<HiveWanted>> {
// An unconnected client does not fail a JetStream request, it hangs