feat: add infra container start/stop/restart tab to C0R3 page
New POST /api/infra-container/{name}/{action} dashboard route (start/
stop/restart on hive-ci/hive-forge/hive-gateway/hive-matrix), reusing
the existing priv_client::control_infra_container helper the
infra_admin agent path already uses, plus an audit_log entry per
attempt. Adds infra_containers to the /api/state StateSnapshot (name +
live running status via systemctl is-active). New 1NFR4 sub-tab on the
C0R3 dashboard page: one row per infra container with a running/
stopped badge and start/stop/restart buttons, polled every 5s while
the sub-tab is open.
This commit is contained in:
parent
79d4c345bb
commit
ef14641b94
7 changed files with 229 additions and 5 deletions
|
|
@ -127,7 +127,7 @@ Passive / rare-interaction state. No longer a dashboard tab — it's a
|
||||||
standalone page reached from the **Core** tile on the H0M3 hub (served at
|
standalone page reached from the **Core** tile on the H0M3 hub (served at
|
||||||
`/core.html`), with the same minimal chrome as `/logs.html`: a `← home`
|
`/core.html`), with the same minimal chrome as `/logs.html`: a `← home`
|
||||||
back-link + a `createTabStrip` sub-tab nav (**K3PT ST4T3** default,
|
back-link + a `createTabStrip` sub-tab nav (**K3PT ST4T3** default,
|
||||||
then **C0NT41N3R L04D**). The page is its own esbuild bundle (`core.js`)
|
then **C0NT41N3R L04D**, then **1NFR4**). The page is its own esbuild bundle (`core.js`)
|
||||||
that cold-loads `/api/state` and subscribes to `/api/dashboard/stream` for
|
that cold-loads `/api/state` and subscribes to `/api/dashboard/stream` for
|
||||||
`tombstones_changed`, `capabilities_changed`, and `tool_groups_changed`
|
`tombstones_changed`, `capabilities_changed`, and `tool_groups_changed`
|
||||||
(the latter two re-render the stale-perms sub-section when permission
|
(the latter two re-render the stale-perms sub-section when permission
|
||||||
|
|
@ -169,6 +169,26 @@ omitted — agents share the host netns, so there is no per-container net
|
||||||
counter (per-agent network needs the netns-isolation roadmap in
|
counter (per-agent network needs the netns-isolation roadmap in
|
||||||
`docs/network.md`).
|
`docs/network.md`).
|
||||||
|
|
||||||
|
**1NFR4** — start / stop / restart the four hive infrastructure
|
||||||
|
containers (`hive-ci`, `hive-forge`, `hive-gateway`, `hive-matrix`)
|
||||||
|
directly from the dashboard, without needing an `infra_admin` agent's
|
||||||
|
`restart` MCP tool. One row per container: name, a `badge-ok`/`badge-fail`
|
||||||
|
running/stopped dot, and `↺ R3ST4RT` + `■ ST0P` (running) or `▶ ST4RT`
|
||||||
|
(stopped) buttons, same themed-confirm pattern as the K3PT ST4T3
|
||||||
|
tombstone actions. Backed by
|
||||||
|
`POST /api/infra-container/{name}/{action}` (`action` ∈
|
||||||
|
`start|stop|restart`), which calls the same
|
||||||
|
`priv_client::control_infra_container` helper the agent-facing
|
||||||
|
`infra_admin` path uses — no new privileged-helper surface, no
|
||||||
|
capability check (the dashboard is already operator-authenticated).
|
||||||
|
Every attempt is written to the audit log (actor `"operator"`, action
|
||||||
|
`start_infra`/`stop_infra`/`restart_infra`) alongside agent-driven infra
|
||||||
|
restarts. Status rows ride the `infra_containers` field on
|
||||||
|
`GET /api/state`'s `StateSnapshot` (`{name, running}`, live
|
||||||
|
`systemctl is-active container@<unit>.service` read); `core.js` polls
|
||||||
|
`/api/state` every 5 s only while the 1NFR4 sub-tab is active, same
|
||||||
|
cadence/lifecycle as C0NT41N3R L04D's polling.
|
||||||
|
|
||||||
## BU1LDS page (`/builds.html`)
|
## BU1LDS page (`/builds.html`)
|
||||||
|
|
||||||
The build lifecycle hub — rebuild queue, live build log, meta inputs,
|
The build lifecycle hub — rebuild queue, live build log, meta inputs,
|
||||||
|
|
@ -1195,6 +1215,14 @@ that's a browser-level decision, not ours.
|
||||||
a background `du -sxb` of the agent's state dir + container writable
|
a background `du -sxb` of the agent's state dir + container writable
|
||||||
rootfs every ~5 min, `-x` excluding the shared read-only nix store.
|
rootfs every ~5 min, `-x` excluding the shared read-only nix store.
|
||||||
`null` until the first sample lands.
|
`null` until the first sample lands.
|
||||||
|
- `POST /api/infra-container/{name}/{action}` — start / stop / restart a
|
||||||
|
hive infra container (C0R3 › 1NFR4 panel). `name` parses into the
|
||||||
|
`InfraContainer` allowlist (`hive-ci`/`hive-forge`/`hive-gateway`/
|
||||||
|
`hive-matrix`, 400 on unknown), `action` ∈ `start|stop|restart`. Calls
|
||||||
|
the same `priv_client::control_infra_container` helper the
|
||||||
|
`infra_admin` agent path uses; records an `audit_log` entry
|
||||||
|
(`start_infra`/`stop_infra`/`restart_infra`, actor `"operator"`) either
|
||||||
|
way.
|
||||||
- `POST /api/cancel-reminder/{id}` — hard-delete a pending reminder.
|
- `POST /api/cancel-reminder/{id}` — hard-delete a pending reminder.
|
||||||
- `POST /api/retry-reminder/{id}` — re-arm a reminder whose delivery
|
- `POST /api/retry-reminder/{id}` — re-arm a reminder whose delivery
|
||||||
failed (clears the failure state so the scheduler retries).
|
failed (clears the failure state so the scheduler retries).
|
||||||
|
|
|
||||||
|
|
@ -16,11 +16,12 @@
|
||||||
/core.html) carved out of the dashboard's old SYST3M tab so the
|
/core.html) carved out of the dashboard's old SYST3M tab so the
|
||||||
dashboard tab strip stays lean. Same minimal chrome as
|
dashboard tab strip stays lean. Same minimal chrome as
|
||||||
/logs.html — a `← home` back-link to the H0M3 hub + a
|
/logs.html — a `← home` back-link to the H0M3 hub + a
|
||||||
createTabStrip sub-tab nav. Two sub-tabs: kept state (tombstones)
|
createTabStrip sub-tab nav. Three sub-tabs: kept state (tombstones),
|
||||||
and container load. Rebuild queue + meta inputs have moved to
|
container load, and infra containers (start/stop/restart the four
|
||||||
|
hive infra containers). Rebuild queue + meta inputs have moved to
|
||||||
/builds.html (the build lifecycle hub). Default tab: K3PT ST4T3.
|
/builds.html (the build lifecycle hub). Default tab: K3PT ST4T3.
|
||||||
The section <div> ids (tombstones-section, container-load-section)
|
The section <div> ids (tombstones-section, container-load-section,
|
||||||
match what core.js's renderers target. -->
|
infra-containers-section) match what core.js's renderers target. -->
|
||||||
<header class="page-header">
|
<header class="page-header">
|
||||||
<a class="page-back" href="/">← home</a>
|
<a class="page-back" href="/">← home</a>
|
||||||
<nav class="hive-tabbar core-tabbar" id="core-tabbar" role="tablist">
|
<nav class="hive-tabbar core-tabbar" id="core-tabbar" role="tablist">
|
||||||
|
|
@ -32,6 +33,10 @@
|
||||||
aria-controls="core-pane-load" data-tab="load">
|
aria-controls="core-pane-load" data-tab="load">
|
||||||
<span class="core-tab-label">C0NT41N3R L04D</span>
|
<span class="core-tab-label">C0NT41N3R L04D</span>
|
||||||
</a>
|
</a>
|
||||||
|
<a class="hive-tab" id="core-tab-infra" href="#infra" role="tab"
|
||||||
|
aria-controls="core-pane-infra" data-tab="infra">
|
||||||
|
<span class="core-tab-label">1NFR4</span>
|
||||||
|
</a>
|
||||||
</nav>
|
</nav>
|
||||||
</header>
|
</header>
|
||||||
|
|
||||||
|
|
@ -62,6 +67,19 @@
|
||||||
</div>
|
</div>
|
||||||
</section>
|
</section>
|
||||||
|
|
||||||
|
<!-- 1NFR4: start / stop / restart the four hive infrastructure
|
||||||
|
containers (hive-ci, hive-forge, hive-gateway, hive-matrix)
|
||||||
|
directly from the dashboard, mirroring what an infra_admin
|
||||||
|
agent's `restart` tool already reaches. Status polled every 5s
|
||||||
|
while this sub-tab is open, same cadence as C0NT41N3R L04D. -->
|
||||||
|
<section class="core-pane" id="core-pane-infra" data-tab-pane="infra"
|
||||||
|
role="tabpanel" aria-labelledby="core-tab-infra">
|
||||||
|
<p class="meta">hive infrastructure containers — ci, forge, gateway, matrix. actions are logged to the AUDIT log same as agent-driven restarts.</p>
|
||||||
|
<div id="infra-containers-section">
|
||||||
|
<p class="meta">loading…</p>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
|
||||||
</main>
|
</main>
|
||||||
|
|
||||||
<script type="module" src="/static/core.js" defer></script>
|
<script type="module" src="/static/core.js" defer></script>
|
||||||
|
|
|
||||||
|
|
@ -242,6 +242,68 @@ function stopContainerLoadPolling() {
|
||||||
if (containerLoadTimer) { clearInterval(containerLoadTimer); containerLoadTimer = null; }
|
if (containerLoadTimer) { clearInterval(containerLoadTimer); containerLoadTimer = null; }
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ─── infra containers (start/stop/restart the 4 hive infra containers) ────
|
||||||
|
let infraTimer = null;
|
||||||
|
|
||||||
|
function renderInfraContainers(rows) {
|
||||||
|
const root = $('infra-containers-section');
|
||||||
|
if (!root) return;
|
||||||
|
root.replaceChildren();
|
||||||
|
if (!Array.isArray(rows) || !rows.length) {
|
||||||
|
root.append(el('p', { class: 'meta' }, 'no infra container data'));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const ul = el('ul', { class: 'containers' });
|
||||||
|
for (const c of rows) {
|
||||||
|
const li = el('li', { class: 'container-row' });
|
||||||
|
const head = el('div', { class: 'head' });
|
||||||
|
head.append(
|
||||||
|
el('span', { class: 'name' }, c.name),
|
||||||
|
el('span', { class: 'badge ' + (c.running ? 'badge-ok' : 'badge-fail') },
|
||||||
|
c.running ? 'running' : 'stopped'),
|
||||||
|
);
|
||||||
|
li.append(head);
|
||||||
|
|
||||||
|
const actions = el('div', { class: 'actions' });
|
||||||
|
const base = '/api/infra-container/' + encodeURIComponent(c.name) + '/';
|
||||||
|
if (c.running) {
|
||||||
|
actions.append(form(base + 'restart', 'btn-restart', '↺ R3ST4RT',
|
||||||
|
'restart ' + c.name + '?'));
|
||||||
|
actions.append(form(base + 'stop', 'btn-stop', '■ ST0P',
|
||||||
|
'stop ' + c.name + '?'));
|
||||||
|
} else {
|
||||||
|
actions.append(form(base + 'start', 'btn-start', '▶ ST4RT',
|
||||||
|
'start ' + c.name + '?'));
|
||||||
|
}
|
||||||
|
li.append(actions);
|
||||||
|
ul.append(li);
|
||||||
|
}
|
||||||
|
root.append(ul);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function refreshInfraContainers() {
|
||||||
|
try {
|
||||||
|
const resp = await fetch('/api/state');
|
||||||
|
if (!resp.ok) throw new Error('http ' + resp.status);
|
||||||
|
const s = await resp.json();
|
||||||
|
renderInfraContainers(s.infra_containers || []);
|
||||||
|
} catch (e) {
|
||||||
|
const root = $('infra-containers-section');
|
||||||
|
if (root) {
|
||||||
|
root.replaceChildren();
|
||||||
|
root.append(el('p', { class: 'meta' }, 'infra container fetch failed: ' + e));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
function startInfraPolling() {
|
||||||
|
refreshInfraContainers();
|
||||||
|
if (infraTimer) return;
|
||||||
|
infraTimer = setInterval(refreshInfraContainers, 5000);
|
||||||
|
}
|
||||||
|
function stopInfraPolling() {
|
||||||
|
if (infraTimer) { clearInterval(infraTimer); infraTimer = null; }
|
||||||
|
}
|
||||||
|
|
||||||
// ─── render-all (cold load + any full re-render) ──────────────────────────
|
// ─── render-all (cold load + any full re-render) ──────────────────────────
|
||||||
function renderAll() {
|
function renderAll() {
|
||||||
renderTombstones({ tombstones: tombstonesState });
|
renderTombstones({ tombstones: tombstonesState });
|
||||||
|
|
@ -294,6 +356,8 @@ async function init() {
|
||||||
onShow: (id) => {
|
onShow: (id) => {
|
||||||
if (id === 'load') startContainerLoadPolling();
|
if (id === 'load') startContainerLoadPolling();
|
||||||
else stopContainerLoadPolling();
|
else stopContainerLoadPolling();
|
||||||
|
if (id === 'infra') startInfraPolling();
|
||||||
|
else stopInfraPolling();
|
||||||
// Lazy-load stale-perms on first K3PT ST4T3 activation; always
|
// Lazy-load stale-perms on first K3PT ST4T3 activation; always
|
||||||
// re-fetch on subsequent visits in case perms changed.
|
// re-fetch on subsequent visits in case perms changed.
|
||||||
if (id === 'kept') fetchAndRenderStalePerms();
|
if (id === 'kept') fetchAndRenderStalePerms();
|
||||||
|
|
|
||||||
63
hive-c0re/src/dashboard/infra_containers.rs
Normal file
63
hive-c0re/src/dashboard/infra_containers.rs
Normal file
|
|
@ -0,0 +1,63 @@
|
||||||
|
//! Dashboard endpoint for operator-driven infra-container lifecycle
|
||||||
|
//! (start / stop / restart on `hive-ci`, `hive-forge`, `hive-gateway`,
|
||||||
|
//! `hive-matrix`). Parallels the `infra_admin`-gated agent path in
|
||||||
|
//! `socket_server/lifecycle_handlers.rs::handle_restart_infra`, but this one
|
||||||
|
//! is reached from the dashboard — already fully operator-authenticated —
|
||||||
|
//! so no capability check is needed here, just the same audit trail.
|
||||||
|
|
||||||
|
use axum::{
|
||||||
|
extract::{Path as AxumPath, State},
|
||||||
|
http::StatusCode,
|
||||||
|
response::{IntoResponse, Response},
|
||||||
|
};
|
||||||
|
use hive_sh4re::priv_proto::{InfraAction, InfraContainer};
|
||||||
|
|
||||||
|
use super::{AppState, error_response};
|
||||||
|
|
||||||
|
/// `POST /api/infra-container/{name}/{action}` — start / stop / restart a
|
||||||
|
/// hive infrastructure container from the dashboard. `name` parses into
|
||||||
|
/// [`InfraContainer`] (the allowlist; unrecognised names 400), `action`
|
||||||
|
/// into `start` / `stop` / `restart`. Every attempt lands in the audit log
|
||||||
|
/// (actor `"operator"`, action `start_infra` / `stop_infra` /
|
||||||
|
/// `restart_infra`) and streams as an `AuditEntryAdded` event, so
|
||||||
|
/// operator-driven and agent-driven (`infra_admin`) infra actions show up
|
||||||
|
/// in the same AUDIT view.
|
||||||
|
pub(super) async fn post_infra_container(
|
||||||
|
State(state): State<AppState>,
|
||||||
|
AxumPath((name, action)): AxumPath<(String, String)>,
|
||||||
|
) -> Response {
|
||||||
|
let Ok(container) = name.parse::<InfraContainer>() else {
|
||||||
|
return error_response(&format!("unknown infra container: {name}"));
|
||||||
|
};
|
||||||
|
let (infra_action, action_label) = match action.as_str() {
|
||||||
|
"start" => (InfraAction::Start, "start_infra"),
|
||||||
|
"stop" => (InfraAction::Stop, "stop_infra"),
|
||||||
|
"restart" => (InfraAction::Restart, "restart_infra"),
|
||||||
|
other => {
|
||||||
|
return error_response(&format!(
|
||||||
|
"unknown action: {other} (want start|stop|restart)"
|
||||||
|
));
|
||||||
|
}
|
||||||
|
};
|
||||||
|
let unit = container.unit_name();
|
||||||
|
tracing::info!(%unit, %action, "dashboard: infra container action");
|
||||||
|
let result = crate::priv_client::control_infra_container(container, infra_action).await;
|
||||||
|
let outcome = if result.is_ok() {
|
||||||
|
crate::audit_log::AuditOutcome::Ok
|
||||||
|
} else {
|
||||||
|
crate::audit_log::AuditOutcome::Err
|
||||||
|
};
|
||||||
|
let detail = result.as_ref().err().map(|e| format!("{e:#}"));
|
||||||
|
if let Some(entry) =
|
||||||
|
state
|
||||||
|
.coord
|
||||||
|
.audit_log
|
||||||
|
.record("operator", action_label, unit, outcome, detail.as_deref())
|
||||||
|
{
|
||||||
|
state.coord.emit_audit_entry(entry);
|
||||||
|
}
|
||||||
|
match result {
|
||||||
|
Ok(()) => (StatusCode::OK, "ok").into_response(),
|
||||||
|
Err(e) => error_response(&format!("{unit}: {e:#}")),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -18,6 +18,7 @@ use crate::lifecycle;
|
||||||
|
|
||||||
mod approvals;
|
mod approvals;
|
||||||
mod build_logs;
|
mod build_logs;
|
||||||
|
mod infra_containers;
|
||||||
mod journal;
|
mod journal;
|
||||||
mod lifecycle_ops;
|
mod lifecycle_ops;
|
||||||
mod matrix_accounts;
|
mod matrix_accounts;
|
||||||
|
|
@ -182,6 +183,10 @@ pub async fn serve(
|
||||||
.route("/api/start/{name}", post(lifecycle_ops::post_start))
|
.route("/api/start/{name}", post(lifecycle_ops::post_start))
|
||||||
.route("/api/rebuild/{name}", post(lifecycle_ops::post_rebuild))
|
.route("/api/rebuild/{name}", post(lifecycle_ops::post_rebuild))
|
||||||
.route("/api/update-all", post(lifecycle_ops::post_update_all))
|
.route("/api/update-all", post(lifecycle_ops::post_update_all))
|
||||||
|
.route(
|
||||||
|
"/api/infra-container/{name}/{action}",
|
||||||
|
post(infra_containers::post_infra_container),
|
||||||
|
)
|
||||||
.route(
|
.route(
|
||||||
"/api/answer-question/{id}",
|
"/api/answer-question/{id}",
|
||||||
post(questions::post_answer_question),
|
post(questions::post_answer_question),
|
||||||
|
|
|
||||||
|
|
@ -126,6 +126,34 @@ pub(super) struct StateSnapshot {
|
||||||
/// `host_stats::server_warnings`; the frontend renders this list
|
/// `host_stats::server_warnings`; the frontend renders this list
|
||||||
/// generically, so new warning kinds need no frontend change.
|
/// generically, so new warning kinds need no frontend change.
|
||||||
server_warnings: Vec<crate::host_stats::ServerWarning>,
|
server_warnings: Vec<crate::host_stats::ServerWarning>,
|
||||||
|
/// Live running/stopped status for the four hive infra containers
|
||||||
|
/// (`hive-ci`, `hive-forge`, `hive-gateway`, `hive-matrix`). Feeds the
|
||||||
|
/// C0R3 page's 1NFR4 sub-tab so the operator can start/stop/restart
|
||||||
|
/// them without an `infra_admin` agent's `restart` tool.
|
||||||
|
infra_containers: Vec<InfraContainerView>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One row for the C0R3 page's 1NFR4 sub-tab.
|
||||||
|
#[derive(Serialize)]
|
||||||
|
struct InfraContainerView {
|
||||||
|
/// Container / systemd-unit name (e.g. `"hive-ci"`).
|
||||||
|
name: &'static str,
|
||||||
|
running: bool,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Live running/stopped status for all four hive infra containers.
|
||||||
|
/// Extracted out of [`api_state`] to keep it under clippy's
|
||||||
|
/// `too_many_lines` limit.
|
||||||
|
async fn infra_container_views() -> Vec<InfraContainerView> {
|
||||||
|
let mut infra_containers =
|
||||||
|
Vec::with_capacity(hive_sh4re::priv_proto::InfraContainer::ALL.len());
|
||||||
|
for container in hive_sh4re::priv_proto::InfraContainer::ALL {
|
||||||
|
infra_containers.push(InfraContainerView {
|
||||||
|
name: container.unit_name(),
|
||||||
|
running: crate::lifecycle::infra_is_running(container).await,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
infra_containers
|
||||||
}
|
}
|
||||||
|
|
||||||
/// One peer hive for the P33RS dashboard tab. Derived from
|
/// One peer hive for the P33RS dashboard tab. Derived from
|
||||||
|
|
@ -357,6 +385,8 @@ pub(super) async fn api_state(
|
||||||
w
|
w
|
||||||
};
|
};
|
||||||
|
|
||||||
|
let infra_containers = infra_container_views().await;
|
||||||
|
|
||||||
axum::Json(StateSnapshot {
|
axum::Json(StateSnapshot {
|
||||||
seq,
|
seq,
|
||||||
hostname,
|
hostname,
|
||||||
|
|
@ -398,6 +428,7 @@ pub(super) async fn api_state(
|
||||||
.filter(|s| !s.is_empty()),
|
.filter(|s| !s.is_empty()),
|
||||||
peer_hives: parse_peer_hives(),
|
peer_hives: parse_peer_hives(),
|
||||||
server_warnings,
|
server_warnings,
|
||||||
|
infra_containers,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -582,6 +582,21 @@ pub async fn is_running(name: &str) -> bool {
|
||||||
.is_ok_and(|s| s.success())
|
.is_ok_and(|s| s.success())
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// True when a hive infrastructure container's systemd unit is active.
|
||||||
|
/// Sibling of [`is_running`] for sub-agents, but infra container/unit names
|
||||||
|
/// (`hive-ci`, …) already have no `h-` prefix to strip, so this queries
|
||||||
|
/// `container@<unit_name>.service` directly rather than going through
|
||||||
|
/// [`container_name`]. Used by the dashboard C0R3 page's 1NFR4 sub-tab to
|
||||||
|
/// show each infra container's live status dot.
|
||||||
|
pub async fn infra_is_running(container: hive_sh4re::priv_proto::InfraContainer) -> bool {
|
||||||
|
let unit = format!("container@{}.service", container.unit_name());
|
||||||
|
Command::new("systemctl")
|
||||||
|
.args(["is-active", "--quiet", &unit])
|
||||||
|
.status()
|
||||||
|
.await
|
||||||
|
.is_ok_and(|s| s.success())
|
||||||
|
}
|
||||||
|
|
||||||
/// Fully tear down a sub-agent's container: stop + remove via `nixos-container
|
/// Fully tear down a sub-agent's container: stop + remove via `nixos-container
|
||||||
/// destroy`, then clean our own systemd drop-in. Leaves it to the caller to
|
/// destroy`, then clean our own systemd drop-in. Leaves it to the caller to
|
||||||
/// wipe `/var/lib/hyperhive/...` state and the per-agent runtime dir.
|
/// wipe `/var/lib/hyperhive/...` state and the per-agent runtime dir.
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue