feat(#3139): tell a container that gave up from one stopped on purpose

is_running collapsed every non-active state into false, so a container that
exhausted its bounded restarts read as plain "down" -- indistinguishable
from one an operator stopped deliberately. Bounding the restarts made that
gap sharper: a slow-failing agent used to grind on visibly, now it can stop
quietly.

Adds UnitState + unit_state() beside is_running rather than widening it.
is_running has ~8 call sites and nearly all are reconcile/power logic asking
"is it up? if not, start it" -- a question with two answers. Only the view
builder needs more, and it gets both facts from one systemctl call, since
is-active prints the state when not passed --quiet.

Surfaces as a flat failed flag on ContainerView and AgentStatusRow, matching
the shape those types already document: independent, orthogonally-observed
facts rather than a state machine. serde(default) keeps it order-independent
with the frontend half.

No behaviour change: nothing acts on the flag, per the ruling.
This commit is contained in:
atlas 2026-08-10 23:18:43 +02:00 committed by mara
commit cae2cf8df6
6 changed files with 128 additions and 2 deletions

View file

@ -11,7 +11,7 @@ use std::path::Path;
use serde::Serialize;
use crate::coordinator::Coordinator;
use crate::lifecycle::{self, AGENT_PREFIX};
use crate::lifecycle::{self, AGENT_PREFIX, UnitState};
// Independent per-agent flags, each its own badge on the dashboard card
// and each diffed separately by `rescan_containers_and_emit`. Grouping
@ -32,6 +32,16 @@ pub struct ContainerView {
pub container: String,
pub port: u16,
pub running: bool,
/// The container's unit is in systemd's `failed` state — it exhausted
/// its bounded restarts and gave up, rather than being stopped
/// deliberately.
///
/// Orthogonal to `running` rather than a variant of it: a failed unit
/// is not running, but a not-running unit is usually just *off*. That
/// distinction is the whole point — without it a container that gave
/// up is indistinguishable from one an operator stopped on purpose.
#[serde(default)]
pub failed: bool,
pub needs_update: bool,
pub needs_login: bool,
/// First 12 chars of the sha the meta flake currently has locked
@ -110,7 +120,12 @@ pub async fn build_all(hive: &crate::coordinator::HiveEnv) -> Vec<ContainerView>
crate::auto_update::agent_config_pending(logical.as_str(), deployed_full).await;
let deployed_sha = deployed_full.map(|s| s[..s.len().min(12)].to_owned());
let parent = topology.get(logical.as_str()).cloned().flatten();
let running = lifecycle::is_running(logical.as_str()).await;
// One `systemctl` call for both facts: `unit_state` is the same
// shell-out `is_running` makes, minus `--quiet`. Asking twice would
// double the per-agent subprocess count on every SSE scan.
let state = lifecycle::unit_state(logical.as_str()).await;
let running = state == UnitState::Active;
let failed = state == UnitState::Failed;
// needs_login fires when EITHER the claude session dir is missing
// (boot-time / fresh container) OR the harness wrote the auth-failed
// sentinel because a turn hit 401. Cleared for stopped containers —
@ -136,6 +151,7 @@ pub async fn build_all(hive: &crate::coordinator::HiveEnv) -> Vec<ContainerView>
out.push(ContainerView {
port: lifecycle::agent_web_port(logical.as_str()),
running,
failed,
container: c.clone(),
name: logical.into_string(),
needs_update,