vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

feat(#2112): make http-mcp the sole transport for the built-in surface

Browse source
This commit is contained in:
damocles 2026-07-11 00:32:51 +02:00
commit 1ba44b77ac
6 changed files with 110 additions and 103 deletions
Download patch file Download diff file Expand all files Collapse all files

13
docs/turn-loop.md
View file

@ -60,12 +60,13 @@ refused by the standard agent socket regardless of who sends them.)
- `hive-agent` — long-running harness loop (the inbox poll + - `hive-agent` — long-running harness loop (the inbox poll +
claude-pump + ack/requeue cycle described above). claude-pump + ack/requeue cycle described above).
- `hive-agent-mcp` — MCP server. Default: stdio child claude spawns - `hive-agent-mcp` — MCP server for the built-in `hyperhive` surface.
via `--mcp-config` per turn (the serve loop renders the config to Run with `--http <addr>` as a persistent streamable-HTTP daemon (the
point at this sibling of its own `/proc/self/exe`). With `hive-mcp-http` systemd unit, on `hyperhive.mcp.httpPort`, default
`--http <addr>`, runs as a persistent streamable-HTTP daemon 8790); claude connects to its URL via `--mcp-config`. HTTP is the sole
instead (used by the `hive-mcp-http` systemd unit when transport — no per-turn stdio child (eliminates the re-registration
`hyperhive.mcp.httpPort` is set). race). (A stdio mode still exists in the binary but isn't wired for
the built-in surface.)
- `hive-agent-wake --from <name> --body <body>` — push a message into - `hive-agent-wake --from <name> --body <body>` — push a message into
our own inbox so the next turn fires with the given body. Used by our own inbox so the next turn fires with the given body. Used by
co-process helpers (scrapers, webhook listeners) to nudge claude on co-process helpers (scrapers, webhook listeners) to nudge claude on

14
docs/turn-loop/claude-invocation.md
View file

@ -186,13 +186,13 @@ next turn picks it up like any other inbox message.
`hive_ag3nt::turn::write_*` writes two files next to the per-agent `hive_ag3nt::turn::write_*` writes two files next to the per-agent
socket at `/run/hive/` once at startup: socket at `/run/hive/` once at startup:
- `claude-mcp-config.json` — by default re-invokes the running binary - `claude-mcp-config.json` — points claude at the persistent
as `mcp` stdio child (so the same binary serves as harness + MCP `hive-mcp-http` daemon (`http://127.0.0.1:{port}/mcp`, port from
server per turn). When `hyperhive.mcp.httpPort` is set in the `hyperhive.mcp.httpPort`, default 8790) for the built-in `hyperhive`
agent's NixOS config, the config instead points claude at the surface. HTTP is the sole transport for it — no per-turn stdio child,
persistent `hive-mcp-http` daemon (`http://127.0.0.1:{port}/mcp`) so the URL survives the per-turn claude re-spawn (no re-registration
— no stdio child per turn; trades the per-turn re-registration race race), trading that for a hard dependency on the daemon's uptime
for a hard dependency on the daemon's uptime (`Restart=always`). (`Restart=always`, no stdio fallback). Extra servers stay stdio.
- `claude-system-prompt.md` — rendered from - `claude-system-prompt.md` — rendered from
`hive-ag3nt/prompts/system.md` by `hive_ag3nt::prompt::render`: `hive-ag3nt/prompts/system.md` by `hive_ag3nt::prompt::render`:
HTML-comment markers (`<!-- role:agent -->...<!-- /role:agent -->`, HTML-comment markers (`<!-- role:agent -->...<!-- /role:agent -->`,

14
docs/turn-loop/mcp.md
View file

@ -1,8 +1,16 @@
# MCP surface # MCP surface
The harness ships an embedded MCP server (rmcp 1.7). Claude launches The harness ships an embedded MCP server (rmcp 1.7). The built-in
it as a stdio child via `--mcp-config`. The hyperhive socket name is `hyperhive` surface is served over streamable HTTP by a persistent
`hyperhive`, so the tools land in claude as `mcp__hyperhive__<tool>`. `hive-mcp-http` daemon (loopback, `127.0.0.1:<hyperhive.mcp.httpPort>`,
per-container private netns). Claude connects to its stable URL via
`--mcp-config` rather than respawning a stdio child each turn, so the
URL survives the per-turn claude re-spawn (and a host-side hive-c0re
restart) — there is no per-turn MCP re-registration race for the
built-in surface. HTTP is the sole transport for it (no stdio fallback);
extra servers (`hyperhive.extraMcpServers`, e.g. matrix/bash) stay stdio
bridges. The server name is `hyperhive`, so the tools land in claude as
`mcp__hyperhive__<tool>`.
Tool access is gated by tool groups (`HIVE_TOOL_GROUPS`). The default Tool access is gated by tool groups (`HIVE_TOOL_GROUPS`). The default
preset (`AGENT_DEFAULT`) includes `messaging`, `meta`, `inbox`, and preset (`AGENT_DEFAULT`) includes `messaging`, `meta`, `inbox`, and

52
hive-ag3nt/src/mcp_config.rs
View file

@ -11,6 +11,14 @@
/// tools as `mcp__<this>__<tool>` (e.g. `mcp__hyperhive__send`). /// tools as `mcp__<this>__<tool>` (e.g. `mcp__hyperhive__send`).
pub const SERVER_NAME: &str = "hyperhive"; pub const SERVER_NAME: &str = "hyperhive";
/// Default loopback port the built-in hyperhive MCP surface is served on
/// (streamable HTTP, via the persistent `hive-mcp-http` daemon). Overridable
/// via `hyperhive.mcp.httpPort`; **must match that option's default** in
/// `nix/templates/harness-base.nix`. Safe as a single fixed value across all
/// agents because each container runs in its own private network namespace,
/// so `127.0.0.1:<port>` is per-container-private (no cross-agent collision).
pub const DEFAULT_MCP_HTTP_PORT: u16 = 8790;
/// Built-in claude tools always present in every session. Anything not /// Built-in claude tools always present in every session. Anything not
/// in this list (or added by `extra_builtin_tools`) literally doesn't /// in this list (or added by `extra_builtin_tools`) literally doesn't
/// exist in the session. Web egress (`WebFetch`/`WebSearch`) are /// exist in the session. Web egress (`WebFetch`/`WebSearch`) are
@ -330,34 +338,30 @@ fn load_extra_mcp() -> std::collections::BTreeMap<String, ExtraMcpServer> {
} }
/// Render the MCP config blob claude reads from `--mcp-config <path>`. /// Render the MCP config blob claude reads from `--mcp-config <path>`.
/// `mcp_binary` is the path (or PATH-resolvable name) of the /// The built-in `hyperhive` surface is an HTTP entry pointing at the
/// `hive-agent-mcp` bridge executable; `socket` is the hyperhive per-agent /// persistent `hive-mcp-http` daemon (see [`DEFAULT_MCP_HTTP_PORT`]); there
/// socket bind-mounted into the container (forwarded to the child as /// is no per-turn stdio child for it. Merges in any extra MCP servers
/// `--socket <path>`). Merges in any extra MCP servers declared via /// declared via `hyperhive.extraMcpServers` (those stay stdio bridges).
/// `hyperhive.extraMcpServers` in the agent's NixOS config.
#[must_use] #[must_use]
pub fn render_claude_config(mcp_binary: &str, socket: &std::path::Path) -> String { pub fn render_claude_config() -> String {
let mut servers = serde_json::Map::new(); let mut servers = serde_json::Map::new();
// When the harness is configured to run the built-in server as a // The built-in hyperhive surface is served exclusively over streamable
// persistent streamable-http daemon (loopback port in // HTTP by the persistent `hive-mcp-http` daemon (loopback, inside the
// `HYPERHIVE_MCP_HTTP_PORT`), point claude at the stable URL instead of // agent's private network namespace). Point claude at the stable URL
// respawning a fresh stdio child each turn. The URL survives the per-turn // rather than respawning a fresh stdio child each turn: the URL survives
// claude re-spawn, so there is no per-turn re-registration race for the // the per-turn claude re-spawn, so there is no per-turn re-registration
// hyperhive surface. Extra servers (matrix/bash) stay stdio bridges. // race for the hyperhive surface. Extra servers (matrix/bash) stay stdio
let hyperhive_entry = match std::env::var("HYPERHIVE_MCP_HTTP_PORT") // bridges. The port comes from `HYPERHIVE_MCP_HTTP_PORT` (always set by
// the harness); `DEFAULT_MCP_HTTP_PORT` is the fallback matching the nix
// default.
let port = std::env::var("HYPERHIVE_MCP_HTTP_PORT")
.ok() .ok()
.and_then(|p| p.trim().parse::<u16>().ok()) .and_then(|p| p.trim().parse::<u16>().ok())
{ .unwrap_or(DEFAULT_MCP_HTTP_PORT);
Some(port) => serde_json::json!({ let hyperhive_entry = serde_json::json!({
"type": "http", "type": "http",
"url": format!("http://127.0.0.1:{port}/mcp"), "url": format!("http://127.0.0.1:{port}/mcp"),
}), });
None => serde_json::json!({
"command": mcp_binary,
"args": ["--socket", socket.display().to_string()],
"env": {}
}),
};
servers.insert(SERVER_NAME.to_owned(), hyperhive_entry); servers.insert(SERVER_NAME.to_owned(), hyperhive_entry);
// Auto-inject HYPERHIVE_STATE_DIR so extra MCP servers can resolve the // Auto-inject HYPERHIVE_STATE_DIR so extra MCP servers can resolve the
// agent's durable state dir without the agent author hard-coding it. // agent's durable state dir without the agent author hard-coding it.

17
hive-ag3nt/src/turn.rs
View file

@ -113,30 +113,25 @@ impl TurnFiles {
/// Returns an error if any of the config files cannot be written to disk. /// Returns an error if any of the config files cannot be written to disk.
pub async fn prepare(socket: &Path, label: &str) -> Result<Self> { pub async fn prepare(socket: &Path, label: &str) -> Result<Self> {
Ok(Self { Ok(Self {
mcp_config: write_mcp_config(socket).await?, mcp_config: write_mcp_config().await?,
system_prompt: write_system_prompt(socket, label).await?, system_prompt: write_system_prompt(socket, label).await?,
}) })
} }
} }
/// Drop the MCP config blob claude reads from `--mcp-config <path>`. /// Drop the MCP config blob claude reads from `--mcp-config <path>`.
/// `socket` is the hyperhive per-container socket (forwarded to the child /// The built-in hyperhive surface is served over HTTP by the persistent
/// as `--socket <path>`). The MCP server is the `hive-agent-mcp` binary /// `hive-mcp-http` daemon, so no per-turn stdio child is spawned; extra
/// installed next to the running `hive-agent` (resolved as a sibling of /// servers declared via `hyperhive.extraMcpServers` are still stdio bridges.
/// `/proc/self/exe`; PATH-resolvable name as the fallback).
/// ///
/// # Errors /// # Errors
/// ///
/// Returns an error if the config file cannot be written. /// Returns an error if the config file cannot be written.
pub async fn write_mcp_config(socket: &Path) -> Result<PathBuf> { pub async fn write_mcp_config() -> Result<PathBuf> {
let parent = crate::paths::config_dir(); let parent = crate::paths::config_dir();
tokio::fs::create_dir_all(&parent).await.ok(); tokio::fs::create_dir_all(&parent).await.ok();
let path = parent.join("claude-mcp-config.json"); let path = parent.join("claude-mcp-config.json");
let exe = std::env::current_exe() let body = mcp_config::render_claude_config();
.ok()
.and_then(|p| Some(p.parent()?.join("hive-agent-mcp")))
.map_or_else(|| "hive-agent-mcp".into(), |p| p.display().to_string());
let body = mcp_config::render_claude_config(&exe, socket);
tokio::fs::write(&path, body).await?; tokio::fs::write(&path, body).await?;
tracing::info!(path = %path.display(), "wrote claude MCP config"); tracing::info!(path = %path.display(), "wrote claude MCP config");
Ok(path) Ok(path)

103
nix/templates/harness-base.nix
View file

@ -947,42 +947,41 @@ in
}; };
options.hyperhive.mcp.httpPort = lib.mkOption { options.hyperhive.mcp.httpPort = lib.mkOption {
type = lib.types.nullOr lib.types.port; type = lib.types.port;
default = null; default = 8790;
example = 8790; example = 8791;
description = '' description = ''
Serve the built-in hyperhive MCP surface as a persistent Loopback port the built-in hyperhive MCP surface is served on. HTTP
streamable-http daemon on this loopback port instead of the is the *sole* transport for the built-in surface: a
default per-turn stdio child. long-lived `hive-mcp-http` systemd unit runs
`hive-agent-mcp --http 127.0.0.1:<port>` and `render_claude_config`
When null (the default) claude spawns a fresh `hive-agent-mcp` points claude at the stable `http://127.0.0.1:<port>/mcp` URL. That
stdio subprocess every turn — the shape that carries the per-turn URL survives the per-turn claude re-spawn (and a host-side hive-c0re
MCP re-registration race (a resumed turn can emit its first tool restart — each tool call dials the control socket fresh), so there
call before that turn's async `initialize`/`tools-list` completes, is no per-turn MCP re-registration race (a resumed stdio child could
stranding the agent with `No such tool`). When set, a long-lived emit its first tool call before that turn's async
`hive-mcp-http` systemd unit runs `initialize`/`tools-list` completed, stranding the agent with `No
`hive-agent-mcp --http 127.0.0.1:<port>` such tool` — the http endpoint eliminates that). Extra MCP servers
and `render_claude_config` points claude at the stable (matrix/bash) stay stdio bridges regardless.
`http://127.0.0.1:<port>/mcp` URL, which survives the per-turn
claude re-spawn (and a host-side hive-c0re restart — each tool call
dials the control socket fresh). Extra MCP servers (matrix/bash)
stay stdio bridges regardless.
Bound loopback-only; the rmcp streamable-http transport's default Bound loopback-only; the rmcp streamable-http transport's default
`allowed_hosts` (`localhost` / `127.0.0.1` / `::1`) rejects Host `allowed_hosts` (`localhost` / `127.0.0.1` / `::1`) rejects Host
headers from anywhere else, so no auth token is required for a headers from anywhere else, so no auth token is required for a
container-local endpoint. container-local endpoint.
Failure-mode note: enabling this flips the MCP surface from a Failure-mode note: with no stdio fallback, if `hive-mcp-http` is
transient self-healing race (stdio child re-registers each turn) down claude hits a dead URL until the unit restarts (guarded by
to a hard dependency on the `hive-mcp-http` daemon's uptime — while `Restart=always`, `RestartSec=3`). Intended shape: no per-turn race
it's up there is no race ever, but while it's down claude hits a while up, a bounded self-healing gap while restarting.
dead URL with no stdio fallback until the unit restarts (guarded by
`Restart=always`, `RestartSec=3`). Net-better when up; operators Safe as a single fixed default across all agents: each container
should know the signature flips from "flaps + recovers" to "dead runs in its own private network namespace (isolation is always-on —
until the unit restarts". Also pick a port unique per host: two see docs/network.md), so `127.0.0.1:<port>` is per-container-private
agents sharing a host that both set the same `httpPort` collide on and cannot collide across agents. Override only if a container-local
bind and the loser Restart-loops. service already occupies this port.
Must match `mcp_config::DEFAULT_MCP_HTTP_PORT` (the harness always
exports `HYPERHIVE_MCP_HTTP_PORT`, so the const is only a fallback).
''; '';
}; };
@ -1940,18 +1939,17 @@ in
}; };
# Persistent streamable-http MCP daemon for the built-in hyperhive # Persistent streamable-http MCP daemon for the built-in hyperhive
# surface. Only wired when `hyperhive.mcp.httpPort` is set; otherwise # surface — the *sole* transport for that surface; always
# the surface stays the default per-turn stdio child (rendered by # wired. Long-lived so claude reconnects to the stable URL each turn
# `render_claude_config`). Long-lived so claude reconnects to the # instead of respawning + re-registering a stdio subprocess (the
# stable URL each turn instead of respawning + re-registering a stdio # per-turn MCP registration race). It dials the control socket
# subprocess (the per-turn MCP registration race). It dials the # (`/run/hive/mcp.sock`, the harness binaries' default) fresh on every
# control socket (`/run/hive/mcp.sock`, the harness binaries' default) # tool call, so a host-side hive-c0re restart is transparent.
# fresh on every tool call, so a host-side hive-c0re restart is # `before = hive-ag3nt` so the URL is already listening by the time
# transparent. `before = hive-ag3nt` so the URL is already listening # the harness renders the first turn's config; the harness/claude also
# by the time the harness renders the first turn's config; the # reconnect on their own, so ordering is a latency nicety not a hard
# harness/claude also reconnect on their own, so ordering is a # correctness dep.
# latency nicety not a hard correctness dep. systemd.services.hive-mcp-http = {
systemd.services.hive-mcp-http = lib.mkIf (config.hyperhive.mcp.httpPort != null) {
description = "persistent streamable-http MCP daemon for the hyperhive surface"; description = "persistent streamable-http MCP daemon for the hyperhive surface";
wantedBy = [ "multi-user.target" ]; wantedBy = [ "multi-user.target" ];
before = [ "hive-ag3nt.service" ]; before = [ "hive-ag3nt.service" ];
@ -1959,11 +1957,12 @@ in
serviceConfig = { serviceConfig = {
ExecStart = "${pkgs.hyperhive}/bin/hive-agent-mcp --http 127.0.0.1:${toString config.hyperhive.mcp.httpPort}"; ExecStart = "${pkgs.hyperhive}/bin/hive-agent-mcp --http 127.0.0.1:${toString config.hyperhive.mcp.httpPort}";
SyslogIdentifier = "hive-mcp-http"; SyslogIdentifier = "hive-mcp-http";
# `always` (not `on-failure`): this endpoint is load-bearing when # `always` (not `on-failure`): this endpoint is load-bearing — the
# `httpPort` is set — a down window is total hyperhive-MCP loss with # sole hyperhive-MCP transport, so a down window is total
# no stdio fallback and no per-turn self-heal (the URL just stays # hyperhive-MCP loss with no stdio fallback and no per-turn
# dead). `always` also covers any unforeseen clean-return path and # self-heal (the URL just stays dead). `always` also covers any
# restarts after a stray SIGTERM stops it out from under the harness. # unforeseen clean-return path and restarts after a stray SIGTERM
# stops it out from under the harness.
Restart = "always"; Restart = "always";
RestartSec = 3; RestartSec = 3;
User = userName; User = userName;
@ -2063,12 +2062,12 @@ in
# `hive_c0re::agent_sockets::socket_path_for(name)` so lifecycle # `hive_c0re::agent_sockets::socket_path_for(name)` so lifecycle
# bind-mounts and gateway upstream config stay in sync. # bind-mounts and gateway upstream config stay in sync.
HIVE_WEB_SOCKET = "/run/hive-agent/${userName}/web.sock"; HIVE_WEB_SOCKET = "/run/hive-agent/${userName}/web.sock";
} # Loopback URL of the persistent `hive-mcp-http` daemon that
// lib.optionalAttrs (config.hyperhive.mcp.httpPort != null) { # `render_claude_config` points claude at for the built-in
# Presence tells `render_claude_config` to point claude at the # surface (HTTP is the sole transport — no per-turn stdio child).
# persistent `hive-mcp-http` daemon's loopback URL instead of a # Kept in sync with the `hive-mcp-http` unit's `--http` port
# per-turn stdio child. Kept in sync with the `hive-mcp-http` # above via the same option. Always set — network isolation is
# unit's `--http` port above via the same option. # unconditional, so a fixed per-container port is collision-free.
HYPERHIVE_MCP_HTTP_PORT = toString config.hyperhive.mcp.httpPort; HYPERHIVE_MCP_HTTP_PORT = toString config.hyperhive.mcp.httpPort;
} }
// lib.optionalAttrs config.hyperhive.gui.enable { // lib.optionalAttrs config.hyperhive.gui.enable {