mara/hyperhive
mara/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

hives report an agent being updated: its status shows updating from the update's stop until the container is back up

Browse source 
A hive marks an agent updating when StopForUpdate stops its running container and clears it when the tail Reconcile or Start settles it, success or failure. The flag rides in the agent status snapshot (absent when false, so older readers and documents are unchanged), the status publisher sends a change at once instead of at the next sweep, and the swarm UI card shows an updating badge.
This commit is contained in:
damocles 2026-11-11 15:41:19 +01:00 • committed by mara
commit 748ef2da08
1 changed files with 227 additions and 12 deletions
Download patch file Download diff file Expand all files Collapse all files

26
hive-c0re/src/swarm_agent_status.rs
View file

@ -37,12 +37,14 @@ const FAILURES_BEFORE_WARNING: u32 = 3;
/// `crate::swarm_queue::wait_for_client()`, one connection per process — so a
/// hive with no queue coordinates pays for this decision once, not twice.
///
/// Takes no `Coordinator` handle, unlike its sibling: this task only reads
/// agent state that already exists on disk / in the container runtime
/// (`lifecycle::agents_for_meta_listing`, `container_view::read_agent_status_bounded`),
/// it never touches the job queue or broker the way `swarm_status::spawn`'s
/// deploy-event listener does.
pub fn spawn(mut shutdown: tokio::sync::watch::Receiver<bool>) {
/// Takes the `Coordinator` only for the set of agents an update has down
/// (`Coordinator::is_updating`), which it publishes with the rest and wakes on.
/// Everything else it reads is agent state that already exists on disk / in the
/// container runtime (`lifecycle::agents_for_meta_listing`,
/// `container_view::read_agent_status_bounded`); it never touches the job queue
/// or broker the way `swarm_status::spawn`'s deploy-event listener does.
pub fn spawn(coord: std::sync::Arc<crate::coordinator::Coordinator>) {
let mut shutdown = coord.shutdown_rx();
let Some(hive) = crate::container_view::hive_swarm_names().0 else {
// Already warned once by `swarm_status::spawn`, which runs the
// identical check — a second warning for the same missing env var
@ -61,7 +63,7 @@ pub fn spawn(mut shutdown: tokio::sync::watch::Receiver<bool>) {
FAILURES_BEFORE_WARNING,
);
loop {
match publish_all(&client, &hive).await {
match publish_all(&client, &hive, &coord).await {
Ok(published) => {
health.record_ok();
tracing::debug!(published, "agent status: sweep complete");
@ -88,6 +90,9 @@ pub fn spawn(mut shutdown: tokio::sync::watch::Receiver<bool>) {
// one whose agents someone is looking at.
tokio::select! {
() = tokio::time::sleep(PUBLISH_INTERVAL) => {}
// An agent entering or leaving an update is the one change worth
// sending ahead of the sweep: the window can be shorter than it.
() = coord.updating_changed() => {}
_ = shutdown.changed() => {
tracing::info!("agent status: shutdown signal received");
break;
@ -104,7 +109,11 @@ pub fn spawn(mut shutdown: tokio::sync::watch::Receiver<bool>) {
/// all) is the one thing that fails the whole sweep; a single agent's
/// status read or publish failing is logged and skipped; skipping one
/// agent should not report every *other* agent on this hive as stale too.
async fn publish_all(client: &async_nats::Client, hive: &str) -> Result<usize> {
async fn publish_all(
client: &async_nats::Client,
hive: &str,
coord: &crate::coordinator::Coordinator,
) -> Result<usize> {
// Same hang risk `swarm_status::publish` guards against: an unconnected
// client does not fail a JetStream request, it hangs on it.
swarm_queue_client::ensure_connected(client)?;
@ -134,6 +143,7 @@ async fn publish_all(client: &async_nats::Client, hive: &str) -> Result<usize> {
status_text,
status_set_at,
running,
updating: coord.is_updating(&spec.name),
};
let bytes = match serde_json::to_vec(&payload) {
Ok(b) => b,