hive-c0re: remove the approvals subsystem
Nothing submits an approval since scheduled prompts left hive-c0re, so the
queue only acted on rows written before that.
- hive-c0re: the approvals store and its migrations, approve/deny/
resolve actions, the ResolveApproval tail node (and meta_update's
approval_id), POST /api/approve and /api/deny, the approval views on
/api/state, the approval_resolved dashboard event, the orphan GC,
destroy/purge failing pending rows, and the approval rows in
get_loose_ends. DestroyBookkeeping loses the purge flag, which only
worded the approval-failure note.
- The CancelLooseEnd request goes: its only hive-c0re arm was the
approval one. cancel_loose_end handles reminder and todo in the harness.
- Wire types: hive_sh4re::approvals (ReminderStats moves to inbox),
LooseEnd::Approval, CancelLooseEndKind, HelperEvent::ApprovalResolved,
ToolGroup::Approvals, HostRequest::{Pending,Approve,Deny} and
HostResponse.approvals. HOST_SOCK_VERSION goes to 2 for the removed
field.
- hivectl loses the approvals verbs; docs/tools/hivectl-cli.md is
regenerated.
- The system prompt drops approval_resolved.
- The approvals table stays in broker.sqlite, unread, the same way the
operator_questions table was left when its store was removed.
- An `approvals` entry still in tool-groups.json is skipped with a warning
by ToolGroup::parse_list, like any unknown group.
- docs/agent-lifecycle/approvals.md is renamed to config-changes.md so the
code's doc pointers resolve; its content follows in the docs commit.
This commit is contained in:
parent
99bd912ae0
commit
58ef299c3c
60 changed files with 153 additions and 1828 deletions
|
|
@ -14,7 +14,7 @@ docs/tools/swarm-logs-cli.md
|
||||||
# indentation normalisation, so these files need manual re-wrap before
|
# indentation normalisation, so these files need manual re-wrap before
|
||||||
# adding them back to the formatter scope.
|
# adding them back to the formatter scope.
|
||||||
CLAUDE.md
|
CLAUDE.md
|
||||||
docs/agent-lifecycle/approvals.md
|
docs/agent-lifecycle/config-changes.md
|
||||||
docs/process/conventions.md
|
docs/process/conventions.md
|
||||||
docs/networking/gateway.md
|
docs/networking/gateway.md
|
||||||
docs/integrations/matrix.md
|
docs/integrations/matrix.md
|
||||||
|
|
|
||||||
|
|
@ -37,10 +37,6 @@ This document contains the help content for the `hivectl` command-line program.
|
||||||
* [`hivectl agent subvol snapshot push`↴](#hivectl-agent-subvol-snapshot-push)
|
* [`hivectl agent subvol snapshot push`↴](#hivectl-agent-subvol-snapshot-push)
|
||||||
* [`hivectl list-agents`↴](#hivectl-list-agents)
|
* [`hivectl list-agents`↴](#hivectl-list-agents)
|
||||||
* [`hivectl quota-enable`↴](#hivectl-quota-enable)
|
* [`hivectl quota-enable`↴](#hivectl-quota-enable)
|
||||||
* [`hivectl approvals`↴](#hivectl-approvals)
|
|
||||||
* [`hivectl approvals pending`↴](#hivectl-approvals-pending)
|
|
||||||
* [`hivectl approvals approve`↴](#hivectl-approvals-approve)
|
|
||||||
* [`hivectl approvals deny`↴](#hivectl-approvals-deny)
|
|
||||||
* [`hivectl wg`↴](#hivectl-wg)
|
* [`hivectl wg`↴](#hivectl-wg)
|
||||||
* [`hivectl wg init`↴](#hivectl-wg-init)
|
* [`hivectl wg init`↴](#hivectl-wg-init)
|
||||||
* [`hivectl wg peer`↴](#hivectl-wg-peer)
|
* [`hivectl wg peer`↴](#hivectl-wg-peer)
|
||||||
|
|
@ -66,7 +62,6 @@ Sibling to the `hive-c0re` daemon binary. Covers host-side admin operations that
|
||||||
* `agent` — Lifecycle actions on ONE managed agent container. Needs the hive-c0re daemon running
|
* `agent` — Lifecycle actions on ONE managed agent container. Needs the hive-c0re daemon running
|
||||||
* `list-agents` — Show all managed agents with their status and technical state
|
* `list-agents` — Show all managed agents with their status and technical state
|
||||||
* `quota-enable` — Enable btrfs qgroup accounting on the agent-state filesystem
|
* `quota-enable` — Enable btrfs qgroup accounting on the agent-state filesystem
|
||||||
* `approvals` — Operator approval queue: list, approve, or deny pending requests
|
|
||||||
* `wg` — WireGuard inter-hive mesh setup helpers
|
* `wg` — WireGuard inter-hive mesh setup helpers
|
||||||
* `peer-config` — Generate the federation peer-config block for THIS hive
|
* `peer-config` — Generate the federation peer-config block for THIS hive
|
||||||
* `stop` — Stop containers hive-wide in one operator action
|
* `stop` — Stop containers hive-wide in one operator action
|
||||||
|
|
@ -524,54 +519,6 @@ Global one-shot toggle (not per-agent), hence top-level rather than under `hivec
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
## `hivectl approvals`
|
|
||||||
|
|
||||||
Operator approval queue: list, approve, or deny pending requests.
|
|
||||||
|
|
||||||
Needs the hive-c0re daemon running.
|
|
||||||
|
|
||||||
**Usage:** `hivectl approvals <COMMAND>`
|
|
||||||
|
|
||||||
###### **Subcommands:**
|
|
||||||
|
|
||||||
* `pending` — List pending approval requests submitted by agents
|
|
||||||
* `approve` — Approve a pending request by id; the action runs immediately
|
|
||||||
* `deny` — Deny a pending request by id
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
## `hivectl approvals pending`
|
|
||||||
|
|
||||||
List pending approval requests submitted by agents
|
|
||||||
|
|
||||||
**Usage:** `hivectl approvals pending`
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
## `hivectl approvals approve`
|
|
||||||
|
|
||||||
Approve a pending request by id; the action runs immediately
|
|
||||||
|
|
||||||
**Usage:** `hivectl approvals approve <ID>`
|
|
||||||
|
|
||||||
###### **Arguments:**
|
|
||||||
|
|
||||||
* `<ID>` — Approval id (from `hivectl approvals pending`)
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
## `hivectl approvals deny`
|
|
||||||
|
|
||||||
Deny a pending request by id
|
|
||||||
|
|
||||||
**Usage:** `hivectl approvals deny <ID>`
|
|
||||||
|
|
||||||
###### **Arguments:**
|
|
||||||
|
|
||||||
* `<ID>` — Approval id
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
## `hivectl wg`
|
## `hivectl wg`
|
||||||
|
|
||||||
WireGuard inter-hive mesh setup helpers.
|
WireGuard inter-hive mesh setup helpers.
|
||||||
|
|
|
||||||
|
|
@ -94,7 +94,7 @@ pub struct CompactArgs {
|
||||||
}
|
}
|
||||||
|
|
||||||
// -----------------------------------------------------------------------------
|
// -----------------------------------------------------------------------------
|
||||||
// Privileged tool arg types (lifecycle, approvals)
|
// Privileged tool arg types (lifecycle)
|
||||||
// -----------------------------------------------------------------------------
|
// -----------------------------------------------------------------------------
|
||||||
|
|
||||||
#[derive(Debug, serde::Deserialize, schemars::JsonSchema)]
|
#[derive(Debug, serde::Deserialize, schemars::JsonSchema)]
|
||||||
|
|
@ -115,8 +115,7 @@ pub struct GetAgentMetaArgs {
|
||||||
#[derive(Debug, serde::Deserialize, schemars::JsonSchema)]
|
#[derive(Debug, serde::Deserialize, schemars::JsonSchema)]
|
||||||
pub struct CancelLooseEndArgs {
|
pub struct CancelLooseEndArgs {
|
||||||
/// Which kind of thread to cancel — `"reminder"` for a scheduled
|
/// Which kind of thread to cancel — `"reminder"` for a scheduled
|
||||||
/// `remind` that hasn't fired yet, `"approval"` for a pending
|
/// `remind` that hasn't fired yet, or `"todo"` for a loose-ends-v2 todo
|
||||||
/// approval you submitted, or `"todo"` for a loose-ends-v2 todo
|
|
||||||
/// (bash/matrix/forge). Use the `kind` field straight off the
|
/// (bash/matrix/forge). Use the `kind` field straight off the
|
||||||
/// `get_loose_ends` row.
|
/// `get_loose_ends` row.
|
||||||
pub kind: String,
|
pub kind: String,
|
||||||
|
|
|
||||||
|
|
@ -30,9 +30,8 @@ pub use args::{
|
||||||
pub use render::{annotate_retries, format_ack, format_agent_meta, format_recv};
|
pub use render::{annotate_retries, format_ack, format_agent_meta, format_recv};
|
||||||
|
|
||||||
use render::{
|
use render::{
|
||||||
dial_agent_socket, format_matrix_summary, local_reminders, local_todos, loose_end_kind_label,
|
dial_agent_socket, format_matrix_summary, local_reminders, local_todos, mark_local_todo_done,
|
||||||
mark_local_todo_done, mark_local_todos_done, matrix_unread_summary, parse_loose_end_kind,
|
mark_local_todos_done, matrix_unread_summary, render_loose_ends, reply_err,
|
||||||
render_loose_ends, reply_err,
|
|
||||||
};
|
};
|
||||||
|
|
||||||
/// Write (or remove) the status file in the agent's own `state/` directory.
|
/// Write (or remove) the status file in the agent's own `state/` directory.
|
||||||
|
|
@ -231,9 +230,7 @@ impl AgentServer {
|
||||||
|
|
||||||
#[tool(
|
#[tool(
|
||||||
description = "List loose ends pending against this agent: pending reminders \
|
description = "List loose ends pending against this agent: pending reminders \
|
||||||
you scheduled, plus — for the manager only — pending \
|
you scheduled. Also lists active \
|
||||||
approvals you submitted that the operator hasn't acted on yet (agents with the \
|
|
||||||
`approvals` tool group also see their own pending approvals). Also lists active \
|
|
||||||
local tasks published by external MCP daemons (e.g. running bash tasks). Cheap sweep, no args. Useful \
|
local tasks published by external MCP daemons (e.g. running bash tasks). Cheap sweep, no args. Useful \
|
||||||
at turn start to remember what you owe / what's owed to you without scrolling \
|
at turn start to remember what you owe / what's owed to you without scrolling \
|
||||||
inbox history. Output is a short bulleted list with ids, ages in seconds, and \
|
inbox history. Output is a short bulleted list with ids, ages in seconds, and \
|
||||||
|
|
@ -346,9 +343,6 @@ impl AgentServer {
|
||||||
you scheduled (hard-deleted before it fires). `kind` is `\"reminder\"`; \
|
you scheduled (hard-deleted before it fires). `kind` is `\"reminder\"`; \
|
||||||
`id` is the row id from the matching `get_loose_ends` entry. Auth: you can only \
|
`id` is the row id from the matching `get_loose_ends` entry. Auth: you can only \
|
||||||
cancel rows where you're the owner. Returns `ok` or an error string.\n\
|
cancel rows where you're the owner. Returns `ok` or an error string.\n\
|
||||||
`kind` may also be `\"approval\"` to withdraw a pending approval you submitted \
|
|
||||||
(before the operator acts on it) — root agent (`ruth`) only; the server rejects \
|
|
||||||
`approval` kind for all other callers.\n\
|
|
||||||
`kind` may also be `\"todo\"` to clear one of your own loose-ends-v2 todos (the id \
|
`kind` may also be `\"todo\"` to clear one of your own loose-ends-v2 todos (the id \
|
||||||
from a `get_loose_ends` `todo #N` line) — dials the in-agent socket directly, safe \
|
from a `get_loose_ends` `todo #N` line) — dials the in-agent socket directly, safe \
|
||||||
to call repeatedly.")]
|
to call repeatedly.")]
|
||||||
|
|
@ -373,42 +367,26 @@ impl AgentServer {
|
||||||
.to_owned(),
|
.to_owned(),
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
let kind = match parse_loose_end_kind(&args.kind) {
|
let kind = args.kind.trim().to_ascii_lowercase();
|
||||||
Ok(k) => k,
|
if !matches!(kind.as_str(), "reminder" | "r") {
|
||||||
Err(e) => return e,
|
return format!(
|
||||||
};
|
"cancel_loose_end: unknown kind '{kind}' (expected \"reminder\" or \"todo\")"
|
||||||
let kind_label = loose_end_kind_label(kind);
|
);
|
||||||
// Reminders are harness-local — dial the in-agent socket
|
}
|
||||||
// directly instead of the broker; the other kind (approval)
|
// Reminders are harness-local — dial the in-agent socket directly.
|
||||||
// still lives in c0re.
|
match dial_agent_socket(&hive_agent_sock::Request::CancelReminder { id }).await {
|
||||||
if kind == hive_sh4re::inbox::CancelLooseEndKind::Reminder {
|
|
||||||
return match dial_agent_socket(&hive_agent_sock::Request::CancelReminder { id })
|
|
||||||
.await
|
|
||||||
{
|
|
||||||
Some(hive_agent_sock::Response::Acked { count }) if count > 0 => {
|
Some(hive_agent_sock::Response::Acked { count }) if count > 0 => {
|
||||||
format!("cancelled {kind_label} {id}")
|
format!("cancelled reminder {id}")
|
||||||
}
|
}
|
||||||
Some(hive_agent_sock::Response::Acked { .. }) => {
|
Some(hive_agent_sock::Response::Acked { .. }) => {
|
||||||
format!("cancel_loose_end failed: no pending {kind_label} {id}")
|
format!("cancel_loose_end failed: no pending reminder {id}")
|
||||||
}
|
}
|
||||||
Some(hive_agent_sock::Response::Err { message }) => {
|
Some(hive_agent_sock::Response::Err { message }) => {
|
||||||
format!("cancel_loose_end failed: {message}")
|
format!("cancel_loose_end failed: {message}")
|
||||||
}
|
}
|
||||||
Some(other) => format!("cancel_loose_end unexpected response: {other:?}"),
|
Some(other) => format!("cancel_loose_end unexpected response: {other:?}"),
|
||||||
None => "cancel_loose_end failed: in-agent socket unavailable".to_owned(),
|
None => "cancel_loose_end failed: in-agent socket unavailable".to_owned(),
|
||||||
};
|
|
||||||
}
|
}
|
||||||
let (resp, retries) = self
|
|
||||||
.dispatch(hive_core_agent_sock::Request::CancelLooseEnd { kind, id })
|
|
||||||
.await;
|
|
||||||
annotate_retries(
|
|
||||||
format_ack(
|
|
||||||
resp,
|
|
||||||
"cancel_loose_end",
|
|
||||||
format!("cancelled {kind_label} {id}"),
|
|
||||||
),
|
|
||||||
retries,
|
|
||||||
)
|
|
||||||
})
|
})
|
||||||
.await
|
.await
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -152,8 +152,8 @@ fn msg_id_tag(id: i64) -> String {
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Hard cap on how many individual `Todo` lines `render_loose_ends` will
|
/// Hard cap on how many individual `Todo` lines `render_loose_ends` will
|
||||||
/// emit. Approvals/reminders stay naturally bounded (they're
|
/// emit. Reminders stay naturally bounded (they're triaged interactively
|
||||||
/// triaged interactively and don't self-multiply), but todos are pushed by
|
/// and don't self-multiply), but todos are pushed by
|
||||||
/// unattended producers — matrix/bash/forge are the built-in ones, but any
|
/// unattended producers — matrix/bash/forge are the built-in ones, but any
|
||||||
/// user-configured MCP server can push its own too — an agent that goes a long
|
/// user-configured MCP server can push its own too — an agent that goes a long
|
||||||
/// stretch without calling `get_loose_ends`, or whose producers outpace its
|
/// stretch without calling `get_loose_ends`, or whose producers outpace its
|
||||||
|
|
@ -178,22 +178,6 @@ const MAX_RENDERED_TODOS: usize = 40;
|
||||||
fn render_one_loose_end(out: &mut String, t: &hive_sh4re::inbox::LooseEnd) {
|
fn render_one_loose_end(out: &mut String, t: &hive_sh4re::inbox::LooseEnd) {
|
||||||
use std::fmt::Write as _;
|
use std::fmt::Write as _;
|
||||||
match t {
|
match t {
|
||||||
hive_sh4re::inbox::LooseEnd::Approval {
|
|
||||||
id,
|
|
||||||
agent,
|
|
||||||
commit_ref,
|
|
||||||
description,
|
|
||||||
age_seconds,
|
|
||||||
} => {
|
|
||||||
let desc = description
|
|
||||||
.as_deref()
|
|
||||||
.map(|d| format!(" — {d}"))
|
|
||||||
.unwrap_or_default();
|
|
||||||
let _ = writeln!(
|
|
||||||
out,
|
|
||||||
"- approval #{id} ({agent} @ {commit_ref}, {age_seconds}s old){desc}"
|
|
||||||
);
|
|
||||||
}
|
|
||||||
hive_sh4re::inbox::LooseEnd::Reminder {
|
hive_sh4re::inbox::LooseEnd::Reminder {
|
||||||
id,
|
id,
|
||||||
owner,
|
owner,
|
||||||
|
|
@ -438,33 +422,6 @@ pub(super) fn format_matrix_summary(rooms: &[MatrixRoomUnread]) -> String {
|
||||||
out
|
out
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Parse the user-facing `kind` string for `cancel_loose_end` into the
|
|
||||||
/// wire enum. Accepts a small alias set so claude doesn't have to
|
|
||||||
/// remember the exact spelling (`"q"` / `"r"` shorthand falls out
|
|
||||||
/// for free).
|
|
||||||
pub(super) fn parse_loose_end_kind(
|
|
||||||
raw: &str,
|
|
||||||
) -> Result<hive_sh4re::inbox::CancelLooseEndKind, String> {
|
|
||||||
match raw.trim().to_ascii_lowercase().as_str() {
|
|
||||||
"reminder" | "r" => Ok(hive_sh4re::inbox::CancelLooseEndKind::Reminder),
|
|
||||||
"approval" | "a" => Ok(hive_sh4re::inbox::CancelLooseEndKind::Approval),
|
|
||||||
other => Err(format!(
|
|
||||||
"cancel_loose_end: unknown kind '{other}' \
|
|
||||||
(expected \"reminder\", \"approval\", or \"todo\")"
|
|
||||||
)),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Canonical user-facing label for a `CancelLooseEndKind` — used in
|
|
||||||
/// the success ack so the caller always sees `"reminder"` instead of
|
|
||||||
/// whatever alias they passed in (`"r"`).
|
|
||||||
pub(super) fn loose_end_kind_label(kind: hive_sh4re::inbox::CancelLooseEndKind) -> &'static str {
|
|
||||||
match kind {
|
|
||||||
hive_sh4re::inbox::CancelLooseEndKind::Reminder => "reminder",
|
|
||||||
hive_sh4re::inbox::CancelLooseEndKind::Approval => "approval",
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Format helper for `get_agent_meta`: renders an agent's identity +
|
/// Format helper for `get_agent_meta`: renders an agent's identity +
|
||||||
/// current status as a short human-readable block. `name`,
|
/// current status as a short human-readable block. `name`,
|
||||||
/// `hyperhive_rev`, and `running` are always shown; `status` only
|
/// `hyperhive_rev`, and `running` are always shown; `status` only
|
||||||
|
|
|
||||||
|
|
@ -149,7 +149,7 @@ pub enum Response {
|
||||||
PendingRemindersCount { count: u64 },
|
PendingRemindersCount { count: u64 },
|
||||||
/// `ReminderRollup` result.
|
/// `ReminderRollup` result.
|
||||||
ReminderRollup {
|
ReminderRollup {
|
||||||
stats: hive_sh4re::approvals::ReminderStats,
|
stats: hive_sh4re::inbox::ReminderStats,
|
||||||
},
|
},
|
||||||
/// Op failed; `message` is operator-facing.
|
/// Op failed; `message` is operator-facing.
|
||||||
Err { message: String },
|
Err { message: String },
|
||||||
|
|
|
||||||
|
|
@ -9,13 +9,12 @@ Need new packages, env vars, or other NixOS config for yourself? You can't edit
|
||||||
|
|
||||||
Your config repo is mounted **read-only** at `/agents/{label}/config/` — `agent.nix` plus whatever extra files define you (declared packages, env vars, MCP servers). Read it to see exactly what defines you before asking for a change, so you can point at the precise file and line.
|
Your config repo is mounted **read-only** at `/agents/{label}/config/` — `agent.nix` plus whatever extra files define you (declared packages, env vars, MCP servers). Read it to see exactly what defines you before asking for a change, so you can point at the precise file and line.
|
||||||
|
|
||||||
Approval boundary: starting, stopping and rebuilding containers is the operator's — ask them when any agent needs one. _Creating_ a new agent is not something you can do from here — ask the operator, who scaffolds the new agent's config repo and spawns it from the dashboard. _Changing_ any agent's config is not a tool call at all — it's a forge PR on the agent's `agent-configs/<name>` repo, which an operator reviews and merges on the forge; the merge deploys it. The operator only signs off on changes; you run the day-to-day.
|
Operator boundary: starting, stopping and rebuilding containers is the operator's — ask them when any agent needs one. _Creating_ a new agent is not something you can do from here — ask the operator, who scaffolds the new agent's config repo and spawns it from the dashboard. _Changing_ any agent's config is not a tool call at all — it's a forge PR on the agent's `agent-configs/<name>` repo, which an operator reviews and merges on the forge; the merge deploys it. The operator only signs off on changes; you run the day-to-day.
|
||||||
|
|
||||||
Messages from sender `system` are hyperhive helper events (JSON body, `event` field discriminates): `approval_resolved`, `container_crash`, `needs_update`. Use these to react to lifecycle changes:
|
Messages from sender `system` are hyperhive helper events (JSON body, `event` field discriminates): `container_crash`, `needs_update`. Use these to react to lifecycle changes:
|
||||||
|
|
||||||
- `needs_update` — agent's flake rev is stale. Ask the operator to rebuild it.
|
- `needs_update` — agent's flake rev is stale. Ask the operator to rebuild it.
|
||||||
- `container_crash` — ask the operator to start it again; if it keeps crashing, say so with what you saw.
|
- `container_crash` — ask the operator to start it again; if it keeps crashing, say so with what you saw.
|
||||||
- `approval_resolved` — one of your own submitted approvals was approved, denied, or failed; the body carries the resolution.
|
|
||||||
|
|
||||||
Lifecycle notices that don't need an immediate turn — a new agent spawned, a container rebuilt/killed/destroyed, or its login state changing — surface as todos instead of messages now. Call `get_loose_ends` to see them.
|
Lifecycle notices that don't need an immediate turn — a new agent spawned, a container rebuilt/killed/destroyed, or its login state changing — surface as todos instead of messages now. Call `get_loose_ends` to see them.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -18,7 +18,7 @@ use std::sync::Mutex;
|
||||||
|
|
||||||
use anyhow::{Context, Result};
|
use anyhow::{Context, Result};
|
||||||
use chrono::{DateTime, Utc};
|
use chrono::{DateTime, Utc};
|
||||||
use hive_sh4re::approvals::ReminderStats;
|
use hive_sh4re::inbox::ReminderStats;
|
||||||
use hive_sh4re::wire_time;
|
use hive_sh4re::wire_time;
|
||||||
use rusqlite::{Connection, params};
|
use rusqlite::{Connection, params};
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -15,7 +15,7 @@ use chrono::Utc;
|
||||||
use rusqlite::{Connection, OpenFlags};
|
use rusqlite::{Connection, OpenFlags};
|
||||||
use serde::Serialize;
|
use serde::Serialize;
|
||||||
|
|
||||||
use hive_sh4re::approvals::ReminderStats;
|
use hive_sh4re::inbox::ReminderStats;
|
||||||
|
|
||||||
/// Window param accepted by `/api/stats?window=`. Each maps to a
|
/// Window param accepted by `/api/stats?window=`. Each maps to a
|
||||||
/// total span + the bucket width used to roll up trend series.
|
/// total span + the bucket width used to roll up trend series.
|
||||||
|
|
|
||||||
|
|
@ -29,7 +29,7 @@ pub(super) async fn api_stats(
|
||||||
/// `HIVE_AGENT_SOCKET` — was a broker RPC before reminders moved
|
/// `HIVE_AGENT_SOCKET` — was a broker RPC before reminders moved
|
||||||
/// in-container. Returns `None` on any transport / decode failure or
|
/// in-container. Returns `None` on any transport / decode failure or
|
||||||
/// when the socket is unset — the stats are decorative, not authoritative.
|
/// when the socket is unset — the stats are decorative, not authoritative.
|
||||||
async fn fetch_reminder_stats(window_secs: u64) -> Option<hive_sh4re::approvals::ReminderStats> {
|
async fn fetch_reminder_stats(window_secs: u64) -> Option<hive_sh4re::inbox::ReminderStats> {
|
||||||
match crate::todo_server::dial(&hive_agent_sock::Request::ReminderRollup {
|
match crate::todo_server::dial(&hive_agent_sock::Request::ReminderRollup {
|
||||||
since_secs: window_secs,
|
since_secs: window_secs,
|
||||||
})
|
})
|
||||||
|
|
|
||||||
|
|
@ -1,7 +1,7 @@
|
||||||
# hive-c0re
|
# hive-c0re
|
||||||
|
|
||||||
The unprivileged host daemon (runs as `hive-core`). Owns the sqlite
|
The unprivileged host daemon (runs as `hive-core`). Owns the sqlite
|
||||||
broker, the approval/reminder queues, the generic job-DAG
|
broker, the reminder queue, the generic job-DAG
|
||||||
queue, container lifecycle, gateway vhost wiring, forge/matrix config
|
queue, container lifecycle, gateway vhost wiring, forge/matrix config
|
||||||
reconciliation, per-container stats, and the axum operator dashboard.
|
reconciliation, per-container stats, and the axum operator dashboard.
|
||||||
Largest crate in the workspace — bin-only, no separate lib.
|
Largest crate in the workspace — bin-only, no separate lib.
|
||||||
|
|
@ -9,7 +9,7 @@ Largest crate in the workspace — bin-only, no separate lib.
|
||||||
## When to use it
|
## When to use it
|
||||||
|
|
||||||
Host-level, cross-container orchestration: spawning/rebuilding/
|
Host-level, cross-container orchestration: spawning/rebuilding/
|
||||||
destroying agent containers, the approval flow, dashboard-visible
|
destroying agent containers, dashboard-visible
|
||||||
state, wiring each agent's gateway vhost and reconciling its forge
|
state, wiring each agent's gateway vhost and reconciling its forge
|
||||||
config-repo access and matrix room membership. The accounts themselves
|
config-repo access and matrix room membership. The accounts themselves
|
||||||
are swarm-controller's (`docs/integrations/forge.md`). Agent-side
|
are swarm-controller's (`docs/integrations/forge.md`). Agent-side
|
||||||
|
|
@ -25,7 +25,7 @@ subdirectory a module actually lives in). One line each — read the
|
||||||
module's own `//!` doc-comment for real detail, don't expect this file
|
module's own `//!` doc-comment for real detail, don't expect this file
|
||||||
to track it:
|
to track it:
|
||||||
|
|
||||||
- **`dashboard/`** — the operator dashboard (containers, approvals,
|
- **`dashboard/`** — the operator dashboard (containers,
|
||||||
schedules, logs, topology).
|
schedules, logs, topology).
|
||||||
- **`job_queue/`** — the job-DAG queue + desired-state reconciliation
|
- **`job_queue/`** — the job-DAG queue + desired-state reconciliation
|
||||||
(`docs/scheduler/coordinator.md`).
|
(`docs/scheduler/coordinator.md`).
|
||||||
|
|
|
||||||
|
|
@ -6,53 +6,10 @@
|
||||||
use std::sync::Arc;
|
use std::sync::Arc;
|
||||||
|
|
||||||
use anyhow::{Context as _, Result, bail};
|
use anyhow::{Context as _, Result, bail};
|
||||||
use hive_sh4re::approvals::{ApprovalKind, ApprovalStatus};
|
|
||||||
use hive_sh4re::manager::HelperEvent;
|
|
||||||
|
|
||||||
use crate::coordinator::Coordinator;
|
use crate::coordinator::Coordinator;
|
||||||
use crate::lifecycle;
|
use crate::lifecycle;
|
||||||
|
|
||||||
/// Approve a pending request. Marks the approval row durably, then submits
|
|
||||||
/// the work to the job queue so the dashboard POST returns immediately while
|
|
||||||
/// the long-running pipeline runs off-thread.
|
|
||||||
///
|
|
||||||
/// Dispatch:
|
|
||||||
/// - `UpdateMetaInputs` → a `MetaUpdate` DAG (fan-out on completion)
|
|
||||||
///
|
|
||||||
/// Every queued kind resolves its approval row via
|
|
||||||
/// [`resolve_approval_dag`] when the DAG settles terminal.
|
|
||||||
pub fn approve(coord: &Arc<Coordinator>, id: i64) -> Result<()> {
|
|
||||||
let approval = coord.approvals.mark_approved(id)?;
|
|
||||||
tracing::info!(
|
|
||||||
%approval.id,
|
|
||||||
%approval.agent,
|
|
||||||
kind = ?approval.kind,
|
|
||||||
%approval.commit_ref,
|
|
||||||
"approval: dispatching",
|
|
||||||
);
|
|
||||||
match approval.kind {
|
|
||||||
ApprovalKind::UpdateMetaInputs => {
|
|
||||||
// Inputs JSON-encoded into commit_ref by the manager's
|
|
||||||
// submit path — surface them on the DAG so the dashboard
|
|
||||||
// can show *which* inputs are about to bump. The cascade
|
|
||||||
// rebuilds fan out when the lock bump lands (so they build
|
|
||||||
// against the post-bump lock, and a failed bump fans out
|
|
||||||
// nothing).
|
|
||||||
let inputs: Vec<String> =
|
|
||||||
serde_json::from_str(&approval.commit_ref).unwrap_or_default();
|
|
||||||
let inserted = coord.job_queue.insert_job(|b| {
|
|
||||||
crate::job_queue::templates::meta_update(b, inputs, Some(id));
|
|
||||||
Vec::new()
|
|
||||||
});
|
|
||||||
if let Err(e) = inserted {
|
|
||||||
return Err(e.context("insert meta-update dag"));
|
|
||||||
}
|
|
||||||
coord.emit_rebuild_queue_snapshot();
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// `applied/main`, and whether `target` descends from it. Fast-forwarding
|
/// `applied/main`, and whether `target` descends from it. Fast-forwarding
|
||||||
/// `main` to a `target` that does not is a rewind: it drops every commit on
|
/// `main` to a `target` that does not is a rewind: it drops every commit on
|
||||||
/// `main` that `target` lacks.
|
/// `main` that `target` lacks.
|
||||||
|
|
@ -149,90 +106,6 @@ pub(crate) async fn post_rev_deploy_failure_to_pr(agent: &str, rev: &str, err: &
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Resolve an approval row from how its DAG's work ended — the body of the
|
|
||||||
/// [`NodeKind::ResolveApproval`] tail node. Every approval-carrying template
|
|
||||||
/// resolves here: the pipeline is ordinary queue nodes,
|
|
||||||
/// so the work's terminal state is the authoritative outcome and there's no
|
|
||||||
/// in-node resolution to skip around.
|
|
||||||
///
|
|
||||||
/// `outcome` is the one the calling node was built to report — a template emits
|
|
||||||
/// one tail per outcome, so success, failure and cancel each arrive here from
|
|
||||||
/// their own node rather than from one node branching.
|
|
||||||
///
|
|
||||||
/// [`NodeKind::ResolveApproval`]: crate::job_queue::NodeKind::ResolveApproval
|
|
||||||
pub(crate) fn resolve_approval_dag(
|
|
||||||
coord: &Arc<Coordinator>,
|
|
||||||
approval_id: i64,
|
|
||||||
outcome: crate::job_queue::TerminalState,
|
|
||||||
error: Option<&str>,
|
|
||||||
) {
|
|
||||||
use crate::job_queue::TerminalState;
|
|
||||||
let approval = match coord.approvals.get(approval_id) {
|
|
||||||
Ok(Some(a)) => a,
|
|
||||||
Ok(None) => {
|
|
||||||
tracing::warn!(approval_id, "approval dag terminal: row no longer exists");
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
Err(e) => {
|
|
||||||
tracing::warn!(approval_id, error = ?e, "approval dag terminal: row read failed");
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
};
|
|
||||||
let result: Result<()> = match outcome {
|
|
||||||
TerminalState::Done => Ok(()),
|
|
||||||
// `Skipped` never reaches here — no template emits a tail for it — but it
|
|
||||||
// reads the same to an operator either way: the work did not happen.
|
|
||||||
TerminalState::Cancelled | TerminalState::Skipped => {
|
|
||||||
Err(anyhow::anyhow!("cancelled before completion"))
|
|
||||||
}
|
|
||||||
TerminalState::Failed => Err(anyhow::anyhow!("{}", error.unwrap_or("job dag failed"))),
|
|
||||||
};
|
|
||||||
if let Err(e) = finish_approval(coord, &approval, result) {
|
|
||||||
tracing::warn!(approval_id, error = ?e, "approval dag resolved with failure");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
fn finish_approval(
|
|
||||||
coord: &Coordinator,
|
|
||||||
approval: &hive_sh4re::approvals::Approval,
|
|
||||||
result: Result<()>,
|
|
||||||
) -> Result<()> {
|
|
||||||
let (status, note, ok) = match &result {
|
|
||||||
Ok(()) => (ApprovalStatus::Approved, None, true),
|
|
||||||
Err(e) => {
|
|
||||||
let note = format!("{e:#}");
|
|
||||||
let _ = coord.approvals.mark_failed(approval.id, ¬e);
|
|
||||||
(ApprovalStatus::Failed, Some(note), false)
|
|
||||||
}
|
|
||||||
};
|
|
||||||
coord.notify_submitter(
|
|
||||||
approval.id,
|
|
||||||
&HelperEvent::ApprovalResolved {
|
|
||||||
id: approval.id,
|
|
||||||
agent: approval.agent.to_string(),
|
|
||||||
commit_ref: approval.commit_ref.clone(),
|
|
||||||
status,
|
|
||||||
note: note.clone(),
|
|
||||||
},
|
|
||||||
);
|
|
||||||
// Phase 5b: also fire on the dashboard event channel so the
|
|
||||||
// browser moves the row out of pending into history without a
|
|
||||||
// snapshot refetch. `approved` rows that succeed get the
|
|
||||||
// approval's logged resolved_at indirectly via `Utc::now()`;
|
|
||||||
// failures already wrote it via mark_failed above.
|
|
||||||
let approval_kind = <&str>::from(approval.kind);
|
|
||||||
let status_str = if ok { "approved" } else { "failed" };
|
|
||||||
coord.emit_approval_resolved(crate::coordinator::ApprovalResolved {
|
|
||||||
id: approval.id,
|
|
||||||
agent: approval.agent.as_str(),
|
|
||||||
approval_kind,
|
|
||||||
status: status_str,
|
|
||||||
note: note.clone(),
|
|
||||||
description: approval.description.clone(),
|
|
||||||
});
|
|
||||||
result
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Fast-forward `applied/main` to `target` and sync the working tree.
|
/// Fast-forward `applied/main` to `target` and sync the working tree.
|
||||||
///
|
///
|
||||||
/// Compare-and-swap, not a bare set: `main` must still be `expected_main`, the
|
/// Compare-and-swap, not a bare set: `main` must still be `expected_main`, the
|
||||||
|
|
@ -284,36 +157,6 @@ pub fn destroy(coord: &Arc<Coordinator>, name: &str, purge: bool) {
|
||||||
coord.emit_rebuild_queue_snapshot();
|
coord.emit_rebuild_queue_snapshot();
|
||||||
}
|
}
|
||||||
|
|
||||||
pub fn deny(coord: &Coordinator, id: i64, note: Option<&str>) -> Result<()> {
|
|
||||||
let approval = coord.approvals.get(id)?;
|
|
||||||
coord.approvals.mark_denied(id, note)?;
|
|
||||||
tracing::info!(%id, note, "approval denied");
|
|
||||||
if let Some(a) = approval {
|
|
||||||
let approval_kind = <&str>::from(a.kind);
|
|
||||||
let description = a.description.clone();
|
|
||||||
let agent_owned = a.agent.clone();
|
|
||||||
coord.notify_submitter(
|
|
||||||
a.id,
|
|
||||||
&HelperEvent::ApprovalResolved {
|
|
||||||
id: a.id,
|
|
||||||
agent: a.agent.to_string(),
|
|
||||||
commit_ref: a.commit_ref,
|
|
||||||
status: ApprovalStatus::Denied,
|
|
||||||
note: note.map(String::from),
|
|
||||||
},
|
|
||||||
);
|
|
||||||
coord.emit_approval_resolved(crate::coordinator::ApprovalResolved {
|
|
||||||
id,
|
|
||||||
agent: agent_owned.as_str(),
|
|
||||||
approval_kind,
|
|
||||||
status: "denied",
|
|
||||||
note: note.map(String::from),
|
|
||||||
description,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
mod tests {
|
mod tests {
|
||||||
use super::{RevAdvance, advance_applied_main};
|
use super::{RevAdvance, advance_applied_main};
|
||||||
|
|
|
||||||
|
|
@ -10,7 +10,6 @@ use std::sync::{Arc, Mutex};
|
||||||
use anyhow::{Context, Result};
|
use anyhow::{Context, Result};
|
||||||
use tokio::sync::{broadcast, watch};
|
use tokio::sync::{broadcast, watch};
|
||||||
|
|
||||||
use crate::approvals::Approvals;
|
|
||||||
use crate::broker::Broker;
|
use crate::broker::Broker;
|
||||||
use crate::container_view::{self, ContainerView};
|
use crate::container_view::{self, ContainerView};
|
||||||
use crate::dashboard_events::DashboardEvent;
|
use crate::dashboard_events::DashboardEvent;
|
||||||
|
|
@ -30,7 +29,6 @@ const LAST_STOPPED_RUNNING_KEY: &str = "last_stopped_running";
|
||||||
|
|
||||||
pub struct Coordinator {
|
pub struct Coordinator {
|
||||||
pub broker: Arc<Broker>,
|
pub broker: Arc<Broker>,
|
||||||
pub approvals: Arc<Approvals>,
|
|
||||||
/// Scheduled-prompt rows, read-only: the dashboard lists them so the
|
/// Scheduled-prompt rows, read-only: the dashboard lists them so the
|
||||||
/// operator can recreate them with `swarmctl schedule`. Nothing on
|
/// operator can recreate them with `swarmctl schedule`. Nothing on
|
||||||
/// the hive writes or fires them.
|
/// the hive writes or fires them.
|
||||||
|
|
@ -170,7 +168,7 @@ pub struct Coordinator {
|
||||||
/// Unified wire-facing event channel feeding the dashboard SSE
|
/// Unified wire-facing event channel feeding the dashboard SSE
|
||||||
/// stream. Carries broker messages (mirrored from `broker.subscribe`
|
/// stream. Carries broker messages (mirrored from `broker.subscribe`
|
||||||
/// by the forwarder task in `main.rs`) and dashboard-only mutation
|
/// by the forwarder task in `main.rs`) and dashboard-only mutation
|
||||||
/// events (approval added/resolved, etc.).
|
/// events (container state, rebuild queue, etc.).
|
||||||
/// Snapshot endpoints capture `event_seq` before reading state so
|
/// Snapshot endpoints capture `event_seq` before reading state so
|
||||||
/// the client can dedupe its buffered live traffic against the
|
/// the client can dedupe its buffered live traffic against the
|
||||||
/// snapshot.
|
/// snapshot.
|
||||||
|
|
@ -413,19 +411,6 @@ impl Drop for MetaUpdateGuard {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Field-named payload for [`Coordinator::emit_approval_resolved`].
|
|
||||||
/// Mirrors the `ApprovalResolved` dashboard-event fields. `agent`
|
|
||||||
/// borrows from the caller; `approval_kind` / `status` are
|
|
||||||
/// compile-time constants.
|
|
||||||
pub struct ApprovalResolved<'a> {
|
|
||||||
pub id: i64,
|
|
||||||
pub agent: &'a str,
|
|
||||||
pub approval_kind: &'static str,
|
|
||||||
pub status: &'static str,
|
|
||||||
pub note: Option<String>,
|
|
||||||
pub description: Option<String>,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// The two ways [`Coordinator::set_paused_by_name`] can fail. See its own
|
/// The two ways [`Coordinator::set_paused_by_name`] can fail. See its own
|
||||||
/// doc comment for why this is a distinct type rather than one
|
/// doc comment for why this is a distinct type rather than one
|
||||||
/// `anyhow::Error` (`Write` already wraps one, `BadName` never needs to).
|
/// `anyhow::Error` (`Write` already wraps one, `BadName` never needs to).
|
||||||
|
|
@ -467,7 +452,6 @@ impl Coordinator {
|
||||||
agent_io_weight,
|
agent_io_weight,
|
||||||
} = env;
|
} = env;
|
||||||
let broker = Broker::open(db_path).context("open broker")?;
|
let broker = Broker::open(db_path).context("open broker")?;
|
||||||
let approvals = Approvals::open(db_path).context("open approvals")?;
|
|
||||||
let scheduled_prompts = crate::scheduled_prompts::ScheduledPrompts::open(db_path)
|
let scheduled_prompts = crate::scheduled_prompts::ScheduledPrompts::open(db_path)
|
||||||
.context("open scheduled_prompts")?;
|
.context("open scheduled_prompts")?;
|
||||||
// BuildLogs wants a directory (it picks its own `build_logs.sqlite`
|
// BuildLogs wants a directory (it picks its own `build_logs.sqlite`
|
||||||
|
|
@ -488,7 +472,6 @@ impl Coordinator {
|
||||||
let (shutdown_tx, _) = watch::channel(false);
|
let (shutdown_tx, _) = watch::channel(false);
|
||||||
Ok(Self {
|
Ok(Self {
|
||||||
broker: Arc::new(broker),
|
broker: Arc::new(broker),
|
||||||
approvals: Arc::new(approvals),
|
|
||||||
scheduled_prompts: Arc::new(scheduled_prompts),
|
scheduled_prompts: Arc::new(scheduled_prompts),
|
||||||
build_logs,
|
build_logs,
|
||||||
hyperhive_flake,
|
hyperhive_flake,
|
||||||
|
|
@ -730,41 +713,6 @@ impl Coordinator {
|
||||||
self.meta_updates_active.load(Ordering::SeqCst) > 0
|
self.meta_updates_active.load(Ordering::SeqCst) > 0
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Emit `ApprovalResolved` after `mark_approved` / `mark_denied` /
|
|
||||||
/// `mark_failed` lands. `resolved_at` is stamped from the system
|
|
||||||
/// clock here so call sites don't repeat the conversion; if you
|
|
||||||
/// already have an authoritative timestamp from the db update,
|
|
||||||
/// the tiny skew between "row updated" and "event emitted" is
|
|
||||||
/// presentation-only and doesn't matter to clients.
|
|
||||||
///
|
|
||||||
/// Takes [`ApprovalResolved`] rather than a positional arg list so
|
|
||||||
/// the seven fields are named at every call site.
|
|
||||||
pub fn emit_approval_resolved(&self, ev: ApprovalResolved<'_>) {
|
|
||||||
let ApprovalResolved {
|
|
||||||
id,
|
|
||||||
agent,
|
|
||||||
approval_kind,
|
|
||||||
status,
|
|
||||||
note,
|
|
||||||
description,
|
|
||||||
} = ev;
|
|
||||||
let resolved_at = std::time::SystemTime::now()
|
|
||||||
.duration_since(std::time::UNIX_EPOCH)
|
|
||||||
.ok()
|
|
||||||
.and_then(|d| i64::try_from(d.as_secs()).ok())
|
|
||||||
.unwrap_or(0);
|
|
||||||
self.emit_dashboard_event(DashboardEvent::ApprovalResolved {
|
|
||||||
seq: self.next_seq(),
|
|
||||||
id,
|
|
||||||
agent: agent.to_owned(),
|
|
||||||
approval_kind,
|
|
||||||
status,
|
|
||||||
resolved_at: hive_sh4re::wire_time::from_secs(resolved_at),
|
|
||||||
note,
|
|
||||||
description,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Rebuild the per-container snapshot, diff it against the last
|
/// Rebuild the per-container snapshot, diff it against the last
|
||||||
/// one cached on `self`, and emit one
|
/// one cached on `self`, and emit one
|
||||||
/// `DashboardEvent::ContainerStateChanged` per added/changed row
|
/// `DashboardEvent::ContainerStateChanged` per added/changed row
|
||||||
|
|
@ -844,7 +792,7 @@ impl Coordinator {
|
||||||
.with_context(|| format!("create agent dir {}", agent_dir.display()))?;
|
.with_context(|| format!("create agent dir {}", agent_dir.display()))?;
|
||||||
let socket_path = Self::socket_path(name);
|
let socket_path = Self::socket_path(name);
|
||||||
// Hand the full Coordinator to the per-agent socket — it needs
|
// Hand the full Coordinator to the per-agent socket — it needs
|
||||||
// more than just the broker (approvals, ...).
|
// more than just the broker.
|
||||||
let socket = socket_server::start(name, &socket_path, self.clone())?;
|
let socket = socket_server::start(name, &socket_path, self.clone())?;
|
||||||
self.agents.lock().unwrap().insert(name.to_owned(), socket);
|
self.agents.lock().unwrap().insert(name.to_owned(), socket);
|
||||||
Ok(agent_dir)
|
Ok(agent_dir)
|
||||||
|
|
@ -1094,28 +1042,6 @@ impl Coordinator {
|
||||||
self.notify_agent(hive_sh4re::manager::MANAGER_AGENT, event);
|
self.notify_agent(hive_sh4re::manager::MANAGER_AGENT, event);
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Route an approval-scoped helper event to the agent that submitted
|
|
||||||
/// approval `approval_id` (the authenticated socket caller at submit
|
|
||||||
/// time). Legacy rows with no recorded submitter — and any lookup
|
|
||||||
/// failure — fall back to the root agent, preserving the prior
|
|
||||||
/// always-root behaviour.
|
|
||||||
pub fn notify_submitter(&self, approval_id: i64, event: &hive_sh4re::manager::HelperEvent) {
|
|
||||||
let target = self.submitter_or_manager(approval_id);
|
|
||||||
self.notify_agent(&target, event);
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Shared resolution for "which agent should hear about approval
|
|
||||||
/// `approval_id`" — the authenticated socket caller at submit time,
|
|
||||||
/// falling back to the manager for legacy rows with no recorded
|
|
||||||
/// submitter (or any lookup failure). Used by both `notify_submitter`
|
|
||||||
/// and `push_todo_submitter` so the fallback rule lives in one place.
|
|
||||||
fn submitter_or_manager(&self, approval_id: i64) -> String {
|
|
||||||
self.approvals
|
|
||||||
.submitter_of(approval_id)
|
|
||||||
.unwrap_or_default()
|
|
||||||
.unwrap_or_else(|| hive_sh4re::manager::MANAGER_AGENT.to_owned())
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Push a `HelperEvent` into an arbitrary agent's inbox. Encoded
|
/// Push a `HelperEvent` into an arbitrary agent's inbox. Encoded
|
||||||
/// the same way as `notify_manager` (sender = `SYSTEM_SENDER`,
|
/// the same way as `notify_manager` (sender = `SYSTEM_SENDER`,
|
||||||
/// body = JSON-encoded event) — e.g. `ContainerCrash`, `NeedsUpdate`.
|
/// body = JSON-encoded event) — e.g. `ContainerCrash`, `NeedsUpdate`.
|
||||||
|
|
|
||||||
|
|
@ -1,104 +0,0 @@
|
||||||
//! Approval endpoints for the dashboard.
|
|
||||||
//!
|
|
||||||
//! Approve/deny actions plus the orphan-approval GC sweep used by the
|
|
||||||
//! `/api/state` builder.
|
|
||||||
|
|
||||||
use axum::{
|
|
||||||
extract::{Form, Path as AxumPath, State},
|
|
||||||
http::StatusCode,
|
|
||||||
response::{IntoResponse, Response},
|
|
||||||
};
|
|
||||||
use hive_sh4re::approvals::Approval;
|
|
||||||
use serde::Deserialize;
|
|
||||||
use utoipa::ToSchema;
|
|
||||||
|
|
||||||
use super::{AppState, error_response};
|
|
||||||
use crate::actions;
|
|
||||||
use crate::coordinator::Coordinator;
|
|
||||||
|
|
||||||
/// Approve a pending approval row.
|
|
||||||
#[utoipa::path(
|
|
||||||
post,
|
|
||||||
path = "/api/approve/{id}",
|
|
||||||
params(("id" = i64, Path, description = "approval row id")),
|
|
||||||
responses(
|
|
||||||
(status = 200, description = "approved", body = String),
|
|
||||||
(status = 500, description = "approve failed"),
|
|
||||||
),
|
|
||||||
tag = "approvals"
|
|
||||||
)]
|
|
||||||
pub(super) async fn post_approve(
|
|
||||||
State(state): State<AppState>,
|
|
||||||
AxumPath(id): AxumPath<i64>,
|
|
||||||
) -> Response {
|
|
||||||
match actions::approve(&state.coord, id) {
|
|
||||||
// 200 instead of 303 — `actions::approve` fires
|
|
||||||
// `ApprovalResolved` (success path) or the eventual failure
|
|
||||||
// event, both of which the dashboard's derived store applies
|
|
||||||
// live. The matching form carries `data-no-refresh`.
|
|
||||||
Ok(()) => (StatusCode::OK, "ok").into_response(),
|
|
||||||
Err(e) => error_response(&format!("approve {id} failed: {e:#}")),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Deserialize, Default, ToSchema)]
|
|
||||||
pub(super) struct DenyForm {
|
|
||||||
#[serde(default)]
|
|
||||||
note: Option<String>,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Deny a pending approval row, with an optional
|
|
||||||
/// note (form field `note`).
|
|
||||||
#[utoipa::path(
|
|
||||||
post,
|
|
||||||
path = "/api/deny/{id}",
|
|
||||||
params(("id" = i64, Path, description = "approval row id")),
|
|
||||||
request_body(content = DenyForm, content_type = "application/x-www-form-urlencoded"),
|
|
||||||
responses(
|
|
||||||
(status = 200, description = "denied", body = String),
|
|
||||||
(status = 500, description = "deny failed"),
|
|
||||||
),
|
|
||||||
tag = "approvals"
|
|
||||||
)]
|
|
||||||
pub(super) async fn post_deny(
|
|
||||||
State(state): State<AppState>,
|
|
||||||
AxumPath(id): AxumPath<i64>,
|
|
||||||
Form(form): Form<DenyForm>,
|
|
||||||
) -> Response {
|
|
||||||
let note = form
|
|
||||||
.note
|
|
||||||
.as_deref()
|
|
||||||
.map(str::trim)
|
|
||||||
.filter(|s| !s.is_empty());
|
|
||||||
match actions::deny(&state.coord, id, note) {
|
|
||||||
Ok(()) => (StatusCode::OK, "ok").into_response(),
|
|
||||||
Err(e) => error_response(&format!("deny {id} failed: {e:#}")),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Filter out approvals whose agent state dir was wiped out from under us
|
|
||||||
/// (e.g. by a test script's cleanup). Marks them failed so they fall out of
|
|
||||||
/// `pending` on next render.
|
|
||||||
pub(super) fn gc_orphans(coord: &Coordinator, approvals: Vec<Approval>) -> Vec<Approval> {
|
|
||||||
approvals
|
|
||||||
.into_iter()
|
|
||||||
.filter(|a| {
|
|
||||||
if Coordinator::agent_proposed_dir(&a.agent).exists() {
|
|
||||||
true
|
|
||||||
} else {
|
|
||||||
let note = "agent state dir missing";
|
|
||||||
let _ = coord.approvals.mark_failed(a.id, note);
|
|
||||||
tracing::info!(id = a.id, agent = %a.agent, "auto-failed orphan approval");
|
|
||||||
coord.emit_approval_resolved(crate::coordinator::ApprovalResolved {
|
|
||||||
id: a.id,
|
|
||||||
agent: a.agent.as_str(),
|
|
||||||
approval_kind: <&str>::from(a.kind),
|
|
||||||
status: "failed",
|
|
||||||
note: Some(note.to_owned()),
|
|
||||||
description: a.description.clone(),
|
|
||||||
});
|
|
||||||
false
|
|
||||||
}
|
|
||||||
})
|
|
||||||
.collect()
|
|
||||||
}
|
|
||||||
|
|
@ -103,11 +103,8 @@ pub(super) async fn post_kill(
|
||||||
}
|
}
|
||||||
// Manager is stoppable from the dashboard like any other
|
// Manager is stoppable from the dashboard like any other
|
||||||
// agent. The host's dashboard server keeps running (it's
|
// agent. The host's dashboard server keeps running (it's
|
||||||
// hive-c0re, not the manager container), per-agent approvals
|
// hive-c0re, not the manager container), and operator-driven
|
||||||
// submitted by other sub-agents still process through the
|
// meta-input updates work from the dashboard either way.
|
||||||
// host-side approval queue without the manager up, and
|
|
||||||
// operator-driven meta-input updates work from the dashboard
|
|
||||||
// either way.
|
|
||||||
if let Err(e) =
|
if let Err(e) =
|
||||||
crate::job_queue::power::stop_many(&state.coord, std::slice::from_ref(&logical), false)
|
crate::job_queue::power::stop_many(&state.coord, std::slice::from_ref(&logical), false)
|
||||||
.await
|
.await
|
||||||
|
|
|
||||||
|
|
@ -215,7 +215,7 @@ pub(super) async fn post_meta_update(
|
||||||
.coord
|
.coord
|
||||||
.job_queue
|
.job_queue
|
||||||
.insert_job(|b| {
|
.insert_job(|b| {
|
||||||
crate::job_queue::templates::meta_update(b, inputs, None);
|
crate::job_queue::templates::meta_update(b, inputs);
|
||||||
Vec::new()
|
Vec::new()
|
||||||
})
|
})
|
||||||
.expect("template-declared shapes are acyclic");
|
.expect("template-declared shapes are acyclic");
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,5 @@
|
||||||
//! Hyperhive dashboard. Lists managed containers (with deep-links to each
|
//! Hyperhive dashboard. Lists managed containers (with deep-links to each
|
||||||
//! container's web UI), pending approvals (with unified diff vs the applied
|
//! container's web UI) and the manager.
|
||||||
//! repo, plus approve/deny buttons), and the manager.
|
|
||||||
|
|
||||||
use std::net::SocketAddr;
|
use std::net::SocketAddr;
|
||||||
use std::sync::Arc;
|
use std::sync::Arc;
|
||||||
|
|
@ -35,7 +34,6 @@ use crate::lifecycle;
|
||||||
tags(
|
tags(
|
||||||
(name = "health", description = "hive-wide liveness/readiness probes"),
|
(name = "health", description = "hive-wide liveness/readiness probes"),
|
||||||
(name = "journal", description = "container + host journal reads"),
|
(name = "journal", description = "container + host journal reads"),
|
||||||
(name = "approvals", description = "approve/deny pending approval rows"),
|
|
||||||
(name = "build_logs", description = "build log headers, full rows, and raw text downloads"),
|
(name = "build_logs", description = "build log headers, full rows, and raw text downloads"),
|
||||||
(name = "lifecycle_ops", description = "agent container lifecycle: rebuild/restart/start/stop/pause/limits"),
|
(name = "lifecycle_ops", description = "agent container lifecycle: rebuild/restart/start/stop/pause/limits"),
|
||||||
(name = "meta_inputs", description = "bulk flake-input update for the meta flake"),
|
(name = "meta_inputs", description = "bulk flake-input update for the meta flake"),
|
||||||
|
|
@ -49,7 +47,6 @@ use crate::lifecycle;
|
||||||
)]
|
)]
|
||||||
struct ApiDoc;
|
struct ApiDoc;
|
||||||
|
|
||||||
mod approvals;
|
|
||||||
mod build_logs;
|
mod build_logs;
|
||||||
// The single validated identifier type — homed in `hive-host-sock` (the crate
|
// The single validated identifier type — homed in `hive-host-sock` (the crate
|
||||||
// owning agent-path facts) so every dashboard path-param validates through the
|
// owning agent-path facts) so every dashboard path-param validates through the
|
||||||
|
|
@ -142,8 +139,6 @@ pub async fn serve(port: u16, coord: Arc<Coordinator>) -> Result<()> {
|
||||||
.routes(routes!(permissions::delete_agent_permissions))
|
.routes(routes!(permissions::delete_agent_permissions))
|
||||||
.routes(routes!(schedules::api_schedules))
|
.routes(routes!(schedules::api_schedules))
|
||||||
.routes(routes!(schedules::post_rebuild_queue_cancel))
|
.routes(routes!(schedules::post_rebuild_queue_cancel))
|
||||||
.routes(routes!(approvals::post_approve))
|
|
||||||
.routes(routes!(approvals::post_deny))
|
|
||||||
.routes(routes!(lifecycle_ops::post_destroy))
|
.routes(routes!(lifecycle_ops::post_destroy))
|
||||||
.routes(routes!(lifecycle_ops::post_kill))
|
.routes(routes!(lifecycle_ops::post_kill))
|
||||||
.routes(routes!(lifecycle_ops::post_restart))
|
.routes(routes!(lifecycle_ops::post_restart))
|
||||||
|
|
@ -360,8 +355,6 @@ mod router_build_probe {
|
||||||
.routes(routes!(permissions::delete_agent_permissions))
|
.routes(routes!(permissions::delete_agent_permissions))
|
||||||
.routes(routes!(schedules::api_schedules))
|
.routes(routes!(schedules::api_schedules))
|
||||||
.routes(routes!(schedules::post_rebuild_queue_cancel))
|
.routes(routes!(schedules::post_rebuild_queue_cancel))
|
||||||
.routes(routes!(approvals::post_approve))
|
|
||||||
.routes(routes!(approvals::post_deny))
|
|
||||||
.routes(routes!(lifecycle_ops::post_destroy))
|
.routes(routes!(lifecycle_ops::post_destroy))
|
||||||
.routes(routes!(lifecycle_ops::post_kill))
|
.routes(routes!(lifecycle_ops::post_kill))
|
||||||
.routes(routes!(lifecycle_ops::post_restart))
|
.routes(routes!(lifecycle_ops::post_restart))
|
||||||
|
|
|
||||||
|
|
@ -13,8 +13,6 @@ use axum::{
|
||||||
sse::{Event, KeepAlive, Sse},
|
sse::{Event, KeepAlive, Sse},
|
||||||
},
|
},
|
||||||
};
|
};
|
||||||
use chrono::{DateTime, Utc};
|
|
||||||
use hive_sh4re::approvals::Approval;
|
|
||||||
use serde::{Deserialize, Serialize};
|
use serde::{Deserialize, Serialize};
|
||||||
use tokio_stream::wrappers::BroadcastStream;
|
use tokio_stream::wrappers::BroadcastStream;
|
||||||
use tokio_stream::{Stream, StreamExt};
|
use tokio_stream::{Stream, StreamExt};
|
||||||
|
|
@ -24,7 +22,7 @@ use crate::container_view::ContainerView;
|
||||||
|
|
||||||
use super::meta_inputs::{MetaInputView, read_meta_inputs};
|
use super::meta_inputs::{MetaInputView, read_meta_inputs};
|
||||||
use super::tombstones::{TombstoneView, build_tombstone_views};
|
use super::tombstones::{TombstoneView, build_tombstone_views};
|
||||||
use super::{AppState, approvals, error_response, scan_validated_paths};
|
use super::{AppState, error_response, scan_validated_paths};
|
||||||
|
|
||||||
#[allow(clippy::struct_excessive_bools)]
|
#[allow(clippy::struct_excessive_bools)]
|
||||||
#[derive(Serialize)]
|
#[derive(Serialize)]
|
||||||
|
|
@ -48,10 +46,6 @@ pub(super) struct StateSnapshot {
|
||||||
any_stale: bool,
|
any_stale: bool,
|
||||||
containers: Vec<ContainerView>,
|
containers: Vec<ContainerView>,
|
||||||
transients: Vec<TransientView>,
|
transients: Vec<TransientView>,
|
||||||
approvals: Vec<ApprovalView>,
|
|
||||||
/// Last 30 resolved approvals (approved / denied / failed), newest-
|
|
||||||
/// first. Drives the "history" tab on the approvals section.
|
|
||||||
approval_history: Vec<ApprovalHistoryView>,
|
|
||||||
/// State dirs (config history + claude creds + /state/ notes) that
|
/// State dirs (config history + claude creds + /state/ notes) that
|
||||||
/// survive after a destroy-without-purge. The operator can re-spawn
|
/// survive after a destroy-without-purge. The operator can re-spawn
|
||||||
/// with the same name to resume, or PURG3 to wipe them.
|
/// with the same name to resume, or PURG3 to wipe them.
|
||||||
|
|
@ -70,8 +64,8 @@ pub(super) struct StateSnapshot {
|
||||||
/// `MetaUpdateRunning` event.
|
/// `MetaUpdateRunning` event.
|
||||||
meta_update_running: bool,
|
meta_update_running: bool,
|
||||||
/// Whether the hive-forge container is up. When true the dashboard
|
/// Whether the hive-forge container is up. When true the dashboard
|
||||||
/// links each container's config + each approval's commit into the
|
/// links each container's config into the forge's `agent-configs`
|
||||||
/// forge's `agent-configs` repos.
|
/// repos.
|
||||||
forge_present: bool,
|
forge_present: bool,
|
||||||
/// Whether `hive-gateway` is in front of this dashboard. Sourced
|
/// Whether `hive-gateway` is in front of this dashboard. Sourced
|
||||||
/// from the `HIVE_GATEWAY_ENABLED` env var, which the c0re NixOS
|
/// from the `HIVE_GATEWAY_ENABLED` env var, which the c0re NixOS
|
||||||
|
|
@ -126,60 +120,6 @@ struct TransientView {
|
||||||
secs: u64,
|
secs: u64,
|
||||||
}
|
}
|
||||||
|
|
||||||
#[derive(Serialize)]
|
|
||||||
struct ApprovalHistoryView {
|
|
||||||
id: i64,
|
|
||||||
agent: String,
|
|
||||||
kind: &'static str,
|
|
||||||
/// First 12 chars of the manager-supplied ref. None when it is empty.
|
|
||||||
sha_short: Option<String>,
|
|
||||||
/// `approved` / `denied` / `failed`.
|
|
||||||
status: &'static str,
|
|
||||||
/// RFC 3339 UTC. Renders as a relative time on the dashboard.
|
|
||||||
resolved_at: DateTime<Utc>,
|
|
||||||
/// Operator-supplied deny reason (for `denied`) or build error
|
|
||||||
/// (for `failed`). None on `approved`.
|
|
||||||
#[serde(skip_serializing_if = "Option::is_none")]
|
|
||||||
note: Option<String>,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Serialize)]
|
|
||||||
struct ApprovalView {
|
|
||||||
id: i64,
|
|
||||||
agent: String,
|
|
||||||
kind: &'static str,
|
|
||||||
/// Manager-supplied description shown on the approval card.
|
|
||||||
#[serde(skip_serializing_if = "Option::is_none")]
|
|
||||||
description: Option<String>,
|
|
||||||
/// Raw `commit_ref` payload for `UpdateMetaInputs` (JSON-encoded
|
|
||||||
/// `Vec<String>` of input names; `"[]"` = all inputs). The
|
|
||||||
/// frontend parses this to render a human-readable card body.
|
|
||||||
commit_ref: String,
|
|
||||||
/// RFC 3339 UTC time the approval was queued. Rendered as a
|
|
||||||
/// relative time on the card so the operator can spot a stale
|
|
||||||
/// request.
|
|
||||||
requested_at: DateTime<Utc>,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Replace silent `.unwrap_or_default()` on the data sources behind
|
|
||||||
/// `/api/state` so that whichever query degrades surfaces in journald
|
|
||||||
/// instead of leaving the operator staring at an empty list. The
|
|
||||||
/// dashboard still degrades to a sensible default value; the warn
|
|
||||||
/// is just the diagnostic breadcrumb the old code swallowed.
|
|
||||||
fn log_default<T, E>(what: &str, result: std::result::Result<T, E>) -> T
|
|
||||||
where
|
|
||||||
T: Default,
|
|
||||||
E: std::fmt::Debug,
|
|
||||||
{
|
|
||||||
match result {
|
|
||||||
Ok(v) => v,
|
|
||||||
Err(e) => {
|
|
||||||
tracing::warn!(target: "api_state", source = %what, error = ?e, "snapshot source failed; using default");
|
|
||||||
T::default()
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Window over which container crashes count toward the `agents_crashing`
|
/// Window over which container crashes count toward the `agents_crashing`
|
||||||
/// banner warning. Wide enough that a crash-looping container (restarted
|
/// banner warning. Wide enough that a crash-looping container (restarted
|
||||||
/// by `Restart=on-failure` every few seconds) keeps the warning lit
|
/// by `Restart=on-failure` every few seconds) keeps the warning lit
|
||||||
|
|
@ -189,12 +129,11 @@ const CRASH_WARNING_WINDOW: std::time::Duration = std::time::Duration::from_mins
|
||||||
|
|
||||||
/// Cold-load snapshot of the whole dashboard.
|
/// Cold-load snapshot of the whole dashboard.
|
||||||
///
|
///
|
||||||
/// Includes the roster, approvals (+ history),
|
/// Includes the roster, tombstones, job queue, meta inputs, and more. Live
|
||||||
/// tombstones, job queue, meta inputs, and more. Live clients then
|
/// clients then follow `/api/dashboard/stream` (SSE) for incremental updates
|
||||||
/// follow `/api/dashboard/stream` (SSE) for incremental updates keyed
|
/// keyed off `seq`.
|
||||||
/// off `seq`.
|
|
||||||
// `StateSnapshot` is a large tree of nested view types (`ContainerView`,
|
// `StateSnapshot` is a large tree of nested view types (`ContainerView`,
|
||||||
// `ApprovalView`, ...) with no `ToSchema` anywhere in that
|
// `TransientView`, ...) with no `ToSchema` anywhere in that
|
||||||
// graph; wiring it up is a schema-modelling project of its own, well past
|
// graph; wiring it up is a schema-modelling project of its own, well past
|
||||||
// "annotate what's reachable". `serde_json::Value` placeholder for now —
|
// "annotate what's reachable". `serde_json::Value` placeholder for now —
|
||||||
// see the batch report.
|
// see the batch report.
|
||||||
|
|
@ -252,19 +191,7 @@ pub(super) async fn api_state(
|
||||||
let containers = state.coord.containers_snapshot().await;
|
let containers = state.coord.containers_snapshot().await;
|
||||||
let any_stale = containers.iter().any(|c| c.needs_update);
|
let any_stale = containers.iter().any(|c| c.needs_update);
|
||||||
let transient_snapshot = state.coord.transient_snapshot();
|
let transient_snapshot = state.coord.transient_snapshot();
|
||||||
let pending_approvals = approvals::gc_orphans(
|
|
||||||
&state.coord,
|
|
||||||
log_default("approvals.pending", state.coord.approvals.pending()),
|
|
||||||
);
|
|
||||||
let transients = build_transient_views(&containers, &transient_snapshot);
|
let transients = build_transient_views(&containers, &transient_snapshot);
|
||||||
let approvals = build_approval_views(pending_approvals);
|
|
||||||
let approval_history = log_default(
|
|
||||||
"approvals.recent_resolved",
|
|
||||||
state.coord.approvals.recent_resolved(30),
|
|
||||||
)
|
|
||||||
.into_iter()
|
|
||||||
.map(history_view)
|
|
||||||
.collect();
|
|
||||||
let tombstones = build_tombstone_views(&state.coord, &containers);
|
let tombstones = build_tombstone_views(&state.coord, &containers);
|
||||||
let port_conflicts = build_port_conflicts(&containers);
|
let port_conflicts = build_port_conflicts(&containers);
|
||||||
|
|
||||||
|
|
@ -287,8 +214,6 @@ pub(super) async fn api_state(
|
||||||
any_stale,
|
any_stale,
|
||||||
containers,
|
containers,
|
||||||
transients,
|
transients,
|
||||||
approvals,
|
|
||||||
approval_history,
|
|
||||||
meta_inputs: read_meta_inputs(),
|
meta_inputs: read_meta_inputs(),
|
||||||
meta_update_running: state.coord.meta_update_in_progress(),
|
meta_update_running: state.coord.meta_update_in_progress(),
|
||||||
tombstones,
|
tombstones,
|
||||||
|
|
@ -375,50 +300,6 @@ fn build_transient_views(
|
||||||
out
|
out
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Project a resolved sqlite row into the lean shape the dashboard
|
|
||||||
/// history tab consumes — no `diff_html` (rendering 30 of them
|
|
||||||
/// per /api/state poll would mean 30 git diffs per refresh).
|
|
||||||
fn history_view(a: Approval) -> ApprovalHistoryView {
|
|
||||||
let displayed = a.commit_ref.as_str();
|
|
||||||
let sha_short = if displayed.is_empty() {
|
|
||||||
None
|
|
||||||
} else {
|
|
||||||
Some(displayed[..displayed.len().min(12)].to_owned())
|
|
||||||
};
|
|
||||||
let status = match a.status {
|
|
||||||
hive_sh4re::approvals::ApprovalStatus::Approved => "approved",
|
|
||||||
hive_sh4re::approvals::ApprovalStatus::Denied => "denied",
|
|
||||||
hive_sh4re::approvals::ApprovalStatus::Failed => "failed",
|
|
||||||
hive_sh4re::approvals::ApprovalStatus::Cancelled => "cancelled",
|
|
||||||
// Pending shouldn't appear in recent_resolved, but be defensive.
|
|
||||||
hive_sh4re::approvals::ApprovalStatus::Pending => "pending",
|
|
||||||
};
|
|
||||||
let kind = <&str>::from(a.kind);
|
|
||||||
ApprovalHistoryView {
|
|
||||||
id: a.id,
|
|
||||||
agent: a.agent.to_string(),
|
|
||||||
kind,
|
|
||||||
sha_short,
|
|
||||||
status,
|
|
||||||
resolved_at: a.resolved_at.unwrap_or_default(),
|
|
||||||
note: a.note,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
fn build_approval_views(approvals: Vec<Approval>) -> Vec<ApprovalView> {
|
|
||||||
approvals
|
|
||||||
.into_iter()
|
|
||||||
.map(|a| ApprovalView {
|
|
||||||
id: a.id,
|
|
||||||
agent: a.agent.to_string(),
|
|
||||||
kind: <&str>::from(a.kind),
|
|
||||||
description: a.description,
|
|
||||||
commit_ref: a.commit_ref,
|
|
||||||
requested_at: a.requested_at,
|
|
||||||
})
|
|
||||||
.collect()
|
|
||||||
}
|
|
||||||
|
|
||||||
/// `/api/jobq/graph` query string — the generic jobq/wire query shape (any
|
/// `/api/jobq/graph` query string — the generic jobq/wire query shape (any
|
||||||
/// host serving a `GraphWire` projection over HTTP takes the same one; see
|
/// host serving a `GraphWire` projection over HTTP takes the same one; see
|
||||||
/// `hive_jobq_wire::parse_states`, which does the actual parsing this side
|
/// `hive_jobq_wire::parse_states`, which does the actual parsing this side
|
||||||
|
|
|
||||||
|
|
@ -189,13 +189,6 @@ pub(super) async fn post_purge_tombstone(
|
||||||
errors.push(format!("{}: {e}", dir.display()));
|
errors.push(format!("{}: {e}", dir.display()));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if let Err(e) = state
|
|
||||||
.coord
|
|
||||||
.approvals
|
|
||||||
.fail_pending_for_agent(name.as_str(), "agent state purged")
|
|
||||||
{
|
|
||||||
tracing::warn!(%name, error = ?e, "approvals: failing pending on tombstone purge failed");
|
|
||||||
}
|
|
||||||
if errors.is_empty() {
|
if errors.is_empty() {
|
||||||
tracing::info!(%name, "tombstone purged");
|
tracing::info!(%name, "tombstone purged");
|
||||||
// Fire the post-purge tombstones snapshot so dashboards
|
// Fire the post-purge tombstones snapshot so dashboards
|
||||||
|
|
|
||||||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Reference in a new issue