hive-matrix-mcp: bound each secondary retry, warn once per outage
Three follow-ups to argus's review of #5237: - A down secondary's retry runs `restore` under RESTORE_TIMEOUT (2 min). Tries run one per tick on one future, so a restore that never resolved held back every other down account's retries for good; a try that outlasts the timeout now counts as a failure like any other. - `client::build_and_restore` warns about a rejected token only for the primary, which exits right after. For a secondary it logs at debug: the retry loop already warns once per outage (`DownAccounts::failed` is news on the first failure and on a changed reason, and an account that came up starts afresh), so the extra warn repeated on every try. - `a_secondary_that_comes_up_late_does_not_wipe_the_todos` runs the late bring-up in a child process of the test binary with HIVE_AGENT_SOCKET pointed at the test's listener. In-process the variable was unset, so a wipe placed back in the bring-up path was a no-op and the test passed. New tests: a_second_rejection_in_a_row_does_not_warn_again (captured logs, alone in a child process since tracing caches callsite interest process-wide; a stub whoami answering M_UNKNOWN_TOKEN, the real build_and_restore), a_failure_after_the_account_came_up_is_news_again, and an_account_whose_restore_hangs_does_not_hold_back_the_others (paused time).
This commit is contained in:
parent
8abea789fd
commit
7b1435eaea
3 changed files with 225 additions and 20 deletions
|
|
@ -108,13 +108,12 @@ pub async fn build_and_restore(
|
|||
// or a secondary one, because a single bad secondary token must
|
||||
// NOT take down the whole daemon (and with it every healthy
|
||||
// account).
|
||||
tracing::warn!(
|
||||
credential = %token.origin(),
|
||||
is_primary,
|
||||
"matrix token rejected (M_UNKNOWN_TOKEN); discarding stale token"
|
||||
);
|
||||
token.discard_stale().await;
|
||||
if is_primary {
|
||||
tracing::warn!(
|
||||
credential = %token.origin(),
|
||||
"matrix token rejected (M_UNKNOWN_TOKEN); discarding stale token"
|
||||
);
|
||||
// Primary: also drop the matrix-sdk sqlite state keyed to the
|
||||
// now-invalid session, then exit 0; the swarm's backfill
|
||||
// re-mints the token and systemd restarts us once it
|
||||
|
|
@ -137,6 +136,12 @@ pub async fn build_and_restore(
|
|||
// token for the same device.
|
||||
// Return a PermanentBringUpError so the caller can distinguish
|
||||
// "retrying now won't help" from a transient network/DNS failure.
|
||||
// Every try of a down secondary lands here, so the warning is the
|
||||
// caller's, once per outage.
|
||||
tracing::debug!(
|
||||
credential = %token.origin(),
|
||||
"matrix token rejected (M_UNKNOWN_TOKEN); discarded stale token"
|
||||
);
|
||||
return Err(anyhow::Error::new(PermanentBringUpError(
|
||||
"matrix token rejected (M_UNKNOWN_TOKEN); discarded stale token, leaving account down".into(),
|
||||
)));
|
||||
|
|
|
|||
Loading…
Reference in a new issue