hivectl: add agent <name> watch to follow live events from the CLI

This commit is contained in:
damocles 2026-07-27 20:16:26 +02:00 committed by mara
commit 9d5c7a7f7e
13 changed files with 230 additions and 16 deletions

View file

@ -34,10 +34,10 @@ hand-maintained per-file tree drifts out of sync with the code.
and the axum operator dashboard (`dashboard.rs`). Largest crate.
- **`hivectl/`** — standalone operator CLI (`hivectl` binary). Talks to
the `hive-c0re` daemon over the host admin socket (`hive-host-sock`
wire types) — does NOT link `hive-c0re`. Verbs: `agents <spawn|kill|
destroy|rebuild|restart|list|set-parent|…>`, `approvals <pending|
approve|deny>`, `forge`/`matrix`/`github`/`gateway` provisioning,
`choom`, `stop`/`start`, `wg`/`peer-config`.
wire types) — does NOT link `hive-c0re`. Verbs: `list-agents`,
`agent <name> <spawn|kill|destroy|rebuild|restart|set-parent|choom|
watch|…>`, `approvals <pending|approve|deny>`, `forge`/`matrix`/
`github`/`gateway` provisioning, `stop`/`start`, `wg`/`peer-config`.
- **`hive-agent/`**, **`hive-agent-mcp/`** —
in-container harness, two sibling crates for every agent (not a
single `hive-ag3nt/` dir — that's the runtime/binary-family nickname,

4
Cargo.lock generated
View file

@ -1834,12 +1834,16 @@ name = "hivectl"
version = "0.1.0"
dependencies = [
"anyhow",
"bytes",
"clap",
"clap-markdown",
"clap_complete",
"hive-host-sock",
"hive-sh4re",
"hive-types",
"http-body-util",
"hyper",
"hyper-util",
"indicatif",
"libc",
"serde_json",

View file

@ -249,9 +249,13 @@ gateway always terminates TLS, so discovery responses always advertise https.
`hive-c0re.nix` opens the per-agent web-port range
`8100..8999` in the host firewall **only when
`services.hyperhive.gateway.enable = false`**. With the gateway on
(default), it's the sole external entry point and proxies to
`127.0.0.1:<port>` internally — leaving the per-agent ports
firewall-open would defeat the single-front-door story.
(default) it's the sole external entry point and routes to agents over
the UDS upstream described above (see [Per-agent unix-socket
upstream](#per-agent-unix-socket-upstream)) — leaving the per-agent
ports firewall-open would defeat the single-front-door story. The
hashed TCP port (`lifecycle::agent_web_port`) still exists as a direct
host-loopback fallback for the pre-UDS/gateway-disabled case, but isn't
what the gateway itself proxies through.
`services.hyperhive.gateway.openFirewall = true` opens both `port` and
`httpsPort` — the gateway always terminates TLS (self-signed floor), so

View file

@ -32,6 +32,7 @@ This document contains the help content for the `hivectl` command-line program.
* [`hivectl agent set-parent`↴](#hivectl-agent-set-parent)
* [`hivectl agent set-limits`↴](#hivectl-agent-set-limits)
* [`hivectl agent choom`↴](#hivectl-agent-choom)
* [`hivectl agent watch`↴](#hivectl-agent-watch)
* [`hivectl agent quota`↴](#hivectl-agent-quota)
* [`hivectl agent quota show`↴](#hivectl-agent-quota-show)
* [`hivectl agent quota set`↴](#hivectl-agent-quota-set)
@ -348,6 +349,7 @@ Everything here targets a single named agent (`hivectl agent foo restart`, `hive
* `set-parent` — Move this agent in the topology tree — under a new parent, or to root
* `set-limits` — Declare this agent's CPU/memory limits, overriding the hive-wide defaults
* `choom` — Open an interactive Claude session inside this agent's container
* `watch` — Follow this agent's live turn/tool-call event stream from the CLI
* `quota` — This agent's disk accounting + optional quota via btrfs qgroups
* `subvol` — btrfs subvolume management for this agent's state dir
@ -476,6 +478,16 @@ A fresh session by default, or resume a prior one. Requires root and a running c
## `hivectl agent watch`
Follow this agent's live turn/tool-call event stream from the CLI.
Dials the same unix socket the gateway's nginx already proxies through (no gateway hop, no daemon round-trip for the stream itself) and prints one compact line per event. `Ctrl-C` to stop. Requires the agent to be running (its harness must have bound the web UI socket).
**Usage:** `hivectl agent watch`
## `hivectl agent quota`
This agent's disk accounting + optional quota via btrfs qgroups.

View file

@ -266,6 +266,32 @@ lands in a faithful copy of the agent's environment:
instead of the onboarding/trust dialog the headless harness never
completes. It's the single place hyperhive touches that file.
## Watch
Follow an agent's live turn/tool-call event stream from the CLI:
```bash
hivectl agent iris watch # tail iris's live events; Ctrl-C to stop
```
Dials the same unix socket the gateway's nginx `proxy_pass`es through
(`/run/hive-agent/<name>/web.sock` — see [Per-agent unix-socket
upstream](../gateway.md#per-agent-unix-socket-upstream)) directly and
speaks a bare HTTP/1.1 request for the agent's existing `/events/stream`
SSE endpoint over it. No gateway hop, no daemon round-trip for the
stream itself — the daemon socket is only used for the "does this agent
exist" pre-flight, same reasoning as `choom` above. Requires the agent
to be running with its web UI socket bound; a fresh spawn/rebuild that
hasn't come up yet gets a clear connection-refused hint rather than a
raw OS error.
Prints one compact line per event — reuses the `_icon`/`_summary` fields
the harness already stamps onto stream-json events for the web UI
(`stream_enrich.rs`), so tool calls and turn markers read as short
glyph-prefixed lines instead of raw JSON. Not an attempt at the web
UI's full collapsible-details rendering (`docs/terminal-rendering.md`)
— that's presentation for a browser, this is a `tail -f`.
## Open
Print (and best-effort open in a browser) one of the hive's web surfaces.

View file

@ -41,11 +41,6 @@ pub const STATE_ROOT: &str = "/var/lib/hyperhive";
// stay in sync; the privsep boundary prevents importing across the crate.
pub const RUNTIME_ROOT: &str = "/run/hyperhive";
/// `/run/hive-agent` — per-agent runtime socket dir root (web + bound
/// markers), one subdir per agent.
// nix: agent container bind-mount / `RuntimeDirectory` (the harness nix modules) — must match.
pub const AGENT_SOCKET_DIR: &str = "/run/hive-agent";
/// Default broker db path (`db/broker.sqlite`). Exposed as a `&str` for
/// the `--broker-db` clap `default_value`; `build_logs.sqlite` is placed
/// alongside it (the build-logs store keys off the broker db's parent).

View file

@ -17,9 +17,10 @@ use anyhow::{Context, Result};
/// gateway container bind-mounts this whole tree (read-only) so it
/// can `proxy_pass` to any agent. Each agent's container bind-mounts
/// only its own `<name>/` subdir — agents can only access their own
/// sockets. The literal lives in [`crate::paths`]; re-exported here
/// under the name this module's consumers have always used.
pub use crate::paths::AGENT_SOCKET_DIR;
/// sockets. The literal lives in `hive-host-sock` (shared with
/// `hivectl`); re-exported here under the name this module's consumers
/// have always used.
pub use hive_host_sock::AGENT_SOCKET_DIR;
/// Socket filename inside each per-agent subdir. Fixed so the path
/// derives entirely from `(AGENT_SOCKET_DIR, name)` — no second

View file

@ -52,6 +52,26 @@ pub fn agent_state_dir(name: &Ident) -> PathBuf {
// nix: read by the gateway container's nginx (hive-gateway.nix) — must match.
pub const GATEWAY_HTPASSWD: &str = "/var/lib/hyperhive/gateway/gateway.htpasswd";
/// `/run/hive-agent` — per-agent runtime socket dir root (web UI unix
/// socket + bound marker), one subdir per agent. The gateway's nginx
/// `proxy_pass`es to `agent_web_socket(name)` directly; `hivectl` dials
/// the same socket for host-side tooling that needs to talk to a running
/// agent's web UI without going through the gateway (e.g. `agent <name>
/// watch`).
// nix: agent container bind-mount / `RuntimeDirectory` (the harness nix modules) — must match.
pub const AGENT_SOCKET_DIR: &str = "/run/hive-agent";
/// Per-agent web UI unix socket path — `AGENT_SOCKET_DIR/<name>/web.sock`.
/// Same socket the gateway's nginx upstream and the harness's
/// `HIVE_WEB_SOCKET` bind both derive from; see
/// `docs/gateway.md::Per-agent unix-socket upstream`.
#[must_use]
pub fn agent_web_socket(name: &Ident) -> PathBuf {
PathBuf::from(AGENT_SOCKET_DIR)
.join(name.as_str())
.join("web.sock")
}
/// nspawn machine-name prefix for agent containers (`h-<name>`). A single
/// `starts_with(AGENT_PREFIX)` filter enumerates managed containers.
pub const AGENT_PREFIX: &str = "h-";

View file

@ -13,12 +13,19 @@ path = "src/main.rs"
[dependencies]
anyhow.workspace = true
# `agent <name> watch` — bare HTTP/1.1 client over the per-agent unix
# socket, same crate family `hive-agent/src/web_ui/proxy.rs` uses for the
# reverse direction.
bytes = "1"
clap.workspace = true
clap_complete.workspace = true
clap-markdown = "0.1"
hive-host-sock.workspace = true
hive-sh4re.workspace = true
hive-types.workspace = true
http-body-util.workspace = true
hyper.workspace = true
hyper-util.workspace = true
indicatif.workspace = true
libc.workspace = true
serde_json.workspace = true

View file

@ -1,6 +1,6 @@
//! `hivectl agent <name>` — everything scoped to ONE managed agent:
//! container lifecycle over the host admin socket (restart/pause/resume/
//! spawn/kill/destroy/rebuild/set-parent/set-limits/choom), plus the
//! spawn/kill/destroy/rebuild/set-parent/set-limits/choom/watch), plus the
//! `quota` and `subvol` groups, whose handlers live in their own modules.
//! `agents_list` (`hivectl list-agents`) is the one genuinely hive-wide
//! read that lives in this module too since it shares the same daemon
@ -210,6 +210,7 @@ pub(crate) async fn run_agent(socket: &Path, name: &str, cmd: AgentCmd) -> Resul
AgentCmd::Choom { resume_session } => {
crate::choom::choom(socket, name, resume_session.as_deref()).await
}
AgentCmd::Watch => crate::watch::watch(socket, name).await,
// `quota` and `subvol` keep their own modules — these arms are
// just the reparenting glue that hoists `name` in from the
// parent `agent <name>` command.

View file

@ -534,6 +534,14 @@ pub enum AgentCmd {
#[arg(long = "resume", value_name = "SESSION")]
resume_session: Option<String>,
},
/// Follow this agent's live turn/tool-call event stream from the CLI.
///
/// Dials the same unix socket the gateway's nginx already proxies
/// through (no gateway hop, no daemon round-trip for the stream
/// itself) and prints one compact line per event. `Ctrl-C` to stop.
/// Requires the agent to be running (its harness must have bound the
/// web UI socket).
Watch,
/// This agent's disk accounting + optional quota via btrfs qgroups.
///
/// Needs `hivectl quota-enable` run once hive-wide first. No-op on

View file

@ -42,6 +42,7 @@ mod wg;
use wg::{peer_config, require_hive_domain, wg_init, wg_peer, wg_status};
mod choom;
mod github;
mod watch;
use github::github_set_token;
mod forge;
use forge::{forge_create_user, forge_reconcile_config};

135
hivectl/src/watch.rs Normal file
View file

@ -0,0 +1,135 @@
//! `hivectl agent <name> watch` — follow an agent's live event stream from
//! the CLI. Dials the same unix socket the gateway's nginx already
//! `proxy_pass`es through (`hive_host_sock::agent_web_socket`) directly and
//! speaks a bare HTTP/1.1 request for the existing `/events/stream` SSE
//! endpoint over it — no gateway hop, no daemon round-trip for the stream
//! itself. Same client technique `hive-agent/src/web_ui/proxy.rs` uses for
//! the reverse direction (an agent proxying out to a unix-socket upstream).
//!
//! The daemon socket is only used for the "does this agent exist"
//! pre-flight — see `choom`'s doc comment for why that check has to be
//! remote rather than a local stat.
use std::path::Path;
use anyhow::{Context as _, Result, bail};
use http_body_util::{BodyExt, Empty};
use hyper_util::rt::TokioIo;
use tokio::net::UnixStream;
use crate::util::agent_exists;
pub(crate) async fn watch(socket: &Path, name: &str) -> Result<()> {
if !agent_exists(socket, name).await? {
bail!(
"no such agent: '{name}' (no state dir under {}/)",
hive_host_sock::AGENTS_ROOT
);
}
let ident = crate::util::parse_ident(name)?;
let sock_path = hive_host_sock::agent_web_socket(&ident);
let stream = UnixStream::connect(&sock_path).await.with_context(|| {
format!(
"connect to {name}'s web UI socket at {} — is the agent running? \
(`hivectl list-agents`; a container that just spawned or rebuilt may \
not have bound its socket yet)",
sock_path.display()
)
})?;
let io = TokioIo::new(stream);
let (mut sender, conn) = hyper::client::conn::http1::handshake(io)
.await
.with_context(|| format!("HTTP handshake with {name}'s web UI socket"))?;
// Drives the connection's I/O in the background; a one-shot GET has
// nothing useful to do with the join handle, and a closed connection
// surfaces to the caller anyway via the next `frame()` returning `None`
// or an error.
tokio::spawn(conn);
let req = hyper::Request::builder()
.method(hyper::Method::GET)
.uri("/events/stream")
.header(hyper::header::HOST, "localhost")
.header(hyper::header::ACCEPT, "text/event-stream")
.body(Empty::<bytes::Bytes>::new())
.context("build /events/stream request")?;
let resp = sender
.send_request(req)
.await
.with_context(|| format!("request {name}'s /events/stream"))?;
if !resp.status().is_success() {
bail!(
"{name}'s web UI returned {} for /events/stream",
resp.status()
);
}
println!("watching {name} — Ctrl-C to stop");
let mut body = resp.into_body();
// SSE frames are separated by a blank line; a chunk boundary from the
// socket has no relation to a frame boundary, so buffer raw text until
// a full frame (up to the `\n\n` separator) has arrived before parsing.
let mut pending = String::new();
while let Some(frame) = body.frame().await {
let frame = frame.with_context(|| format!("reading {name}'s event stream"))?;
let Some(chunk) = frame.data_ref() else {
continue;
};
pending.push_str(&String::from_utf8_lossy(chunk));
while let Some(pos) = pending.find("\n\n") {
let frame_text = pending[..pos].to_owned();
pending.drain(..=pos + 1);
render_frame(&frame_text);
}
}
println!("(stream closed by {name})");
Ok(())
}
/// Render one SSE frame (everything up to the blank-line separator,
/// already stripped) as a single compact terminal line. Not an attempt at
/// the web UI's full row taxonomy (`docs/terminal-rendering.md`) — that's
/// presentation for a browser, not a `tail -f`. Falls back to the raw
/// payload for anything unrecognised rather than silently dropping it, so
/// a shape this doesn't know about is still visible.
fn render_frame(frame: &str) {
let data: String = frame
.lines()
.filter_map(|l| l.strip_prefix("data:"))
.map(str::trim_start)
.collect::<Vec<_>>()
.join("\n");
if data.is_empty() {
return;
}
let Ok(v) = serde_json::from_str::<serde_json::Value>(&data) else {
println!("! {data}");
return;
};
let str_field = |k: &str| v.get(k).and_then(serde_json::Value::as_str);
let kind = str_field("kind").unwrap_or("?");
let line = match kind {
"turn_start" => format!("◆ TURN ← {}", str_field("from").unwrap_or("?")),
"turn_end" => {
if v.get("ok").and_then(serde_json::Value::as_bool) == Some(true) {
"✅ turn ok".to_owned()
} else {
format!("❌ turn fail — {}", str_field("note").unwrap_or(""))
}
}
"note" => format!("· {}", str_field("text").unwrap_or("")),
"status_changed" => format!("· status: {}", str_field("status").unwrap_or("")),
// Claude stream-json lines, enriched server-side
// (`hive-agent/src/stream_enrich.rs`) with `_icon`/`_summary` so
// clients don't need to reimplement the web UI's dispatch logic.
"stream" => match (str_field("_icon"), str_field("_summary")) {
(Some(icon), Some(summary)) => format!("{icon} {summary}"),
(None, Some(summary)) => summary.to_owned(),
_ => format!("· {}", str_field("type").unwrap_or("stream")),
},
other => format!("· {other}"),
};
println!("{line}");
}