swarm-controller: first credential-renewal pass waits for the queue connection
The renewal pass ran the moment swarm-controller started, before its queue client had connected, so `WantedWriter::view` refused with "client state Pending" and the pass logged `agent credential renewal: pass failed; retrying next tick`. That fired twice in 24h on muede-lpt2, both under a second after start, and would trip a Grafana rule on that WARN on ordinary restarts. The first pass now waits until the queue client is connected, polling `swarm_queue_client::ensure_connected` every 5s the way `AgentIconReader::create_when_connected` does. The wait is bounded by one RECONCILE_INTERVAL (5 min): the old code's retry after a false start also came one interval later, so a queue that never connects gets its first pass no later than before. Hitting the bound logs one WARN and runs the pass anyway. Only startup waits; a later disconnect still fails a pass and logs the WARN. Refs #4717
This commit is contained in:
parent
b9667d5975
commit
9e06e191a3
3 changed files with 151 additions and 39 deletions
|
|
@ -69,6 +69,12 @@ impl WantedWriter {
|
|||
Self { client }
|
||||
}
|
||||
|
||||
/// Whether the queue is connected, which [`Self::view`] and [`Self::set`]
|
||||
/// require.
|
||||
pub fn is_connected(&self) -> bool {
|
||||
swarm_queue_client::ensure_connected(&self.client).is_ok()
|
||||
}
|
||||
|
||||
/// The declaration currently published for `hive`, or `None`.
|
||||
pub async fn view(&self, hive: &str) -> Result<Option<HiveWanted>> {
|
||||
// An unconnected client does not fail a JetStream request, it hangs
|
||||
|
|
|
|||
Loading…
Reference in a new issue