vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(#2112): make http-mcp the sole transport for the built-in surface

Browse source
This commit is contained in:
damocles 2026-07-11 00:32:51 +02:00
commit 1ba44b77ac
6 changed files with 110 additions and 103 deletions
Download patch file Download diff file Expand all files Collapse all files

13
docs/turn-loop.md
View file

@ -60,12 +60,13 @@ refused by the standard agent socket regardless of who sends them.)
- `hive-agent` — long-running harness loop (the inbox poll +
claude-pump + ack/requeue cycle described above).
- `hive-agent-mcp` — MCP server. Default: stdio child claude spawns
via `--mcp-config` per turn (the serve loop renders the config to
point at this sibling of its own `/proc/self/exe`). With
`--http <addr>`, runs as a persistent streamable-HTTP daemon
instead (used by the `hive-mcp-http` systemd unit when
`hyperhive.mcp.httpPort` is set).
- `hive-agent-mcp` — MCP server for the built-in `hyperhive` surface.
Run with `--http <addr>` as a persistent streamable-HTTP daemon (the
`hive-mcp-http` systemd unit, on `hyperhive.mcp.httpPort`, default
8790); claude connects to its URL via `--mcp-config`. HTTP is the sole
transport — no per-turn stdio child (eliminates the re-registration
race). (A stdio mode still exists in the binary but isn't wired for
the built-in surface.)
- `hive-agent-wake --from <name> --body <body>` — push a message into
our own inbox so the next turn fires with the given body. Used by
co-process helpers (scrapers, webhook listeners) to nudge claude on

14
docs/turn-loop/claude-invocation.md
View file

@ -186,13 +186,13 @@ next turn picks it up like any other inbox message.
`hive_ag3nt::turn::write_*` writes two files next to the per-agent
socket at `/run/hive/` once at startup:
- `claude-mcp-config.json` — by default re-invokes the running binary
as `mcp` stdio child (so the same binary serves as harness + MCP
server per turn). When `hyperhive.mcp.httpPort` is set in the
agent's NixOS config, the config instead points claude at the
persistent `hive-mcp-http` daemon (`http://127.0.0.1:{port}/mcp`)
— no stdio child per turn; trades the per-turn re-registration race
for a hard dependency on the daemon's uptime (`Restart=always`).
- `claude-mcp-config.json` — points claude at the persistent
`hive-mcp-http` daemon (`http://127.0.0.1:{port}/mcp`, port from
`hyperhive.mcp.httpPort`, default 8790) for the built-in `hyperhive`
surface. HTTP is the sole transport for it — no per-turn stdio child,
so the URL survives the per-turn claude re-spawn (no re-registration
race), trading that for a hard dependency on the daemon's uptime
(`Restart=always`, no stdio fallback). Extra servers stay stdio.
- `claude-system-prompt.md` — rendered from
`hive-ag3nt/prompts/system.md` by `hive_ag3nt::prompt::render`:
HTML-comment markers (`<!-- role:agent -->...<!-- /role:agent -->`,

14
docs/turn-loop/mcp.md
View file

@ -1,8 +1,16 @@
# MCP surface
The harness ships an embedded MCP server (rmcp 1.7). Claude launches
it as a stdio child via `--mcp-config`. The hyperhive socket name is
`hyperhive`, so the tools land in claude as `mcp__hyperhive__<tool>`.
The harness ships an embedded MCP server (rmcp 1.7). The built-in
`hyperhive` surface is served over streamable HTTP by a persistent
`hive-mcp-http` daemon (loopback, `127.0.0.1:<hyperhive.mcp.httpPort>`,
per-container private netns). Claude connects to its stable URL via
`--mcp-config` rather than respawning a stdio child each turn, so the
URL survives the per-turn claude re-spawn (and a host-side hive-c0re
restart) — there is no per-turn MCP re-registration race for the
built-in surface. HTTP is the sole transport for it (no stdio fallback);
extra servers (`hyperhive.extraMcpServers`, e.g. matrix/bash) stay stdio
bridges. The server name is `hyperhive`, so the tools land in claude as
`mcp__hyperhive__<tool>`.
Tool access is gated by tool groups (`HIVE_TOOL_GROUPS`). The default
preset (`AGENT_DEFAULT`) includes `messaging`, `meta`, `inbox`, and

52
hive-ag3nt/src/mcp_config.rs
View file

@ -11,6 +11,14 @@
/// tools as `mcp__<this>__<tool>` (e.g. `mcp__hyperhive__send`).
pub const SERVER_NAME: &str = "hyperhive";
/// Default loopback port the built-in hyperhive MCP surface is served on
/// (streamable HTTP, via the persistent `hive-mcp-http` daemon). Overridable
/// via `hyperhive.mcp.httpPort`; **must match that option's default** in
/// `nix/templates/harness-base.nix`. Safe as a single fixed value across all
/// agents because each container runs in its own private network namespace,
/// so `127.0.0.1:<port>` is per-container-private (no cross-agent collision).
pub const DEFAULT_MCP_HTTP_PORT: u16 = 8790;
/// Built-in claude tools always present in every session. Anything not
/// in this list (or added by `extra_builtin_tools`) literally doesn't
/// exist in the session. Web egress (`WebFetch`/`WebSearch`) are
@ -330,34 +338,30 @@ fn load_extra_mcp() -> std::collections::BTreeMap<String, ExtraMcpServer> {
}
/// Render the MCP config blob claude reads from `--mcp-config <path>`.
/// `mcp_binary` is the path (or PATH-resolvable name) of the
/// `hive-agent-mcp` bridge executable; `socket` is the hyperhive per-agent
/// socket bind-mounted into the container (forwarded to the child as
/// `--socket <path>`). Merges in any extra MCP servers declared via
/// `hyperhive.extraMcpServers` in the agent's NixOS config.
/// The built-in `hyperhive` surface is an HTTP entry pointing at the
/// persistent `hive-mcp-http` daemon (see [`DEFAULT_MCP_HTTP_PORT`]); there
/// is no per-turn stdio child for it. Merges in any extra MCP servers
/// declared via `hyperhive.extraMcpServers` (those stay stdio bridges).
#[must_use]
pub fn render_claude_config(mcp_binary: &str, socket: &std::path::Path) -> String {
pub fn render_claude_config() -> String {
let mut servers = serde_json::Map::new();
// When the harness is configured to run the built-in server as a
// persistent streamable-http daemon (loopback port in
// `HYPERHIVE_MCP_HTTP_PORT`), point claude at the stable URL instead of
// respawning a fresh stdio child each turn. The URL survives the per-turn
// claude re-spawn, so there is no per-turn re-registration race for the
// hyperhive surface. Extra servers (matrix/bash) stay stdio bridges.
let hyperhive_entry = match std::env::var("HYPERHIVE_MCP_HTTP_PORT")
// The built-in hyperhive surface is served exclusively over streamable
// HTTP by the persistent `hive-mcp-http` daemon (loopback, inside the
// agent's private network namespace). Point claude at the stable URL
// rather than respawning a fresh stdio child each turn: the URL survives
// the per-turn claude re-spawn, so there is no per-turn re-registration
// race for the hyperhive surface. Extra servers (matrix/bash) stay stdio
// bridges. The port comes from `HYPERHIVE_MCP_HTTP_PORT` (always set by
// the harness); `DEFAULT_MCP_HTTP_PORT` is the fallback matching the nix
// default.
let port = std::env::var("HYPERHIVE_MCP_HTTP_PORT")
.ok()
.and_then(|p| p.trim().parse::<u16>().ok())
{
Some(port) => serde_json::json!({
"type": "http",
"url": format!("http://127.0.0.1:{port}/mcp"),
}),
None => serde_json::json!({
"command": mcp_binary,
"args": ["--socket", socket.display().to_string()],
"env": {}
}),
};
.unwrap_or(DEFAULT_MCP_HTTP_PORT);
let hyperhive_entry = serde_json::json!({
"type": "http",
"url": format!("http://127.0.0.1:{port}/mcp"),
});
servers.insert(SERVER_NAME.to_owned(), hyperhive_entry);
// Auto-inject HYPERHIVE_STATE_DIR so extra MCP servers can resolve the
// agent's durable state dir without the agent author hard-coding it.

17
hive-ag3nt/src/turn.rs
View file

@ -113,30 +113,25 @@ impl TurnFiles {
/// Returns an error if any of the config files cannot be written to disk.
pub async fn prepare(socket: &Path, label: &str) -> Result<Self> {
Ok(Self {
mcp_config: write_mcp_config(socket).await?,
mcp_config: write_mcp_config().await?,
system_prompt: write_system_prompt(socket, label).await?,
})
}
}
/// Drop the MCP config blob claude reads from `--mcp-config <path>`.
/// `socket` is the hyperhive per-container socket (forwarded to the child
/// as `--socket <path>`). The MCP server is the `hive-agent-mcp` binary
/// installed next to the running `hive-agent` (resolved as a sibling of
/// `/proc/self/exe`; PATH-resolvable name as the fallback).
/// The built-in hyperhive surface is served over HTTP by the persistent
/// `hive-mcp-http` daemon, so no per-turn stdio child is spawned; extra
/// servers declared via `hyperhive.extraMcpServers` are still stdio bridges.
///
/// # Errors
///
/// Returns an error if the config file cannot be written.
pub async fn write_mcp_config(socket: &Path) -> Result<PathBuf> {
pub async fn write_mcp_config() -> Result<PathBuf> {
let parent = crate::paths::config_dir();
tokio::fs::create_dir_all(&parent).await.ok();
let path = parent.join("claude-mcp-config.json");
let exe = std::env::current_exe()
.ok()
.and_then(|p| Some(p.parent()?.join("hive-agent-mcp")))
.map_or_else(|| "hive-agent-mcp".into(), |p| p.display().to_string());
let body = mcp_config::render_claude_config(&exe, socket);
let body = mcp_config::render_claude_config();
tokio::fs::write(&path, body).await?;
tracing::info!(path = %path.display(), "wrote claude MCP config");
Ok(path)

103
nix/templates/harness-base.nix
View file

@ -947,42 +947,41 @@ in
};
options.hyperhive.mcp.httpPort = lib.mkOption {
type = lib.types.nullOr lib.types.port;
default = null;
example = 8790;
type = lib.types.port;
default = 8790;
example = 8791;
description = ''
Serve the built-in hyperhive MCP surface as a persistent
streamable-http daemon on this loopback port instead of the
default per-turn stdio child.
When null (the default) claude spawns a fresh `hive-agent-mcp`
stdio subprocess every turn — the shape that carries the per-turn
MCP re-registration race (a resumed turn can emit its first tool
call before that turn's async `initialize`/`tools-list` completes,
stranding the agent with `No such tool`). When set, a long-lived
`hive-mcp-http` systemd unit runs
`hive-agent-mcp --http 127.0.0.1:<port>`
and `render_claude_config` points claude at the stable
`http://127.0.0.1:<port>/mcp` URL, which survives the per-turn
claude re-spawn (and a host-side hive-c0re restart — each tool call
dials the control socket fresh). Extra MCP servers (matrix/bash)
stay stdio bridges regardless.
Loopback port the built-in hyperhive MCP surface is served on. HTTP
is the *sole* transport for the built-in surface: a
long-lived `hive-mcp-http` systemd unit runs
`hive-agent-mcp --http 127.0.0.1:<port>` and `render_claude_config`
points claude at the stable `http://127.0.0.1:<port>/mcp` URL. That
URL survives the per-turn claude re-spawn (and a host-side hive-c0re
restart — each tool call dials the control socket fresh), so there
is no per-turn MCP re-registration race (a resumed stdio child could
emit its first tool call before that turn's async
`initialize`/`tools-list` completed, stranding the agent with `No
such tool` — the http endpoint eliminates that). Extra MCP servers
(matrix/bash) stay stdio bridges regardless.
Bound loopback-only; the rmcp streamable-http transport's default
`allowed_hosts` (`localhost` / `127.0.0.1` / `::1`) rejects Host
headers from anywhere else, so no auth token is required for a
container-local endpoint.
Failure-mode note: enabling this flips the MCP surface from a
transient self-healing race (stdio child re-registers each turn)
to a hard dependency on the `hive-mcp-http` daemon's uptime — while
it's up there is no race ever, but while it's down claude hits a
dead URL with no stdio fallback until the unit restarts (guarded by
`Restart=always`, `RestartSec=3`). Net-better when up; operators
should know the signature flips from "flaps + recovers" to "dead
until the unit restarts". Also pick a port unique per host: two
agents sharing a host that both set the same `httpPort` collide on
bind and the loser Restart-loops.
Failure-mode note: with no stdio fallback, if `hive-mcp-http` is
down claude hits a dead URL until the unit restarts (guarded by
`Restart=always`, `RestartSec=3`). Intended shape: no per-turn race
while up, a bounded self-healing gap while restarting.
Safe as a single fixed default across all agents: each container
runs in its own private network namespace (isolation is always-on —
see docs/network.md), so `127.0.0.1:<port>` is per-container-private
and cannot collide across agents. Override only if a container-local
service already occupies this port.
Must match `mcp_config::DEFAULT_MCP_HTTP_PORT` (the harness always
exports `HYPERHIVE_MCP_HTTP_PORT`, so the const is only a fallback).
'';
};
@ -1940,18 +1939,17 @@ in
};
# Persistent streamable-http MCP daemon for the built-in hyperhive
# surface. Only wired when `hyperhive.mcp.httpPort` is set; otherwise
# the surface stays the default per-turn stdio child (rendered by
# `render_claude_config`). Long-lived so claude reconnects to the
# stable URL each turn instead of respawning + re-registering a stdio
# subprocess (the per-turn MCP registration race). It dials the
# control socket (`/run/hive/mcp.sock`, the harness binaries' default)
# fresh on every tool call, so a host-side hive-c0re restart is
# transparent. `before = hive-ag3nt` so the URL is already listening
# by the time the harness renders the first turn's config; the
# harness/claude also reconnect on their own, so ordering is a
# latency nicety not a hard correctness dep.
systemd.services.hive-mcp-http = lib.mkIf (config.hyperhive.mcp.httpPort != null) {
# surface — the *sole* transport for that surface; always
# wired. Long-lived so claude reconnects to the stable URL each turn
# instead of respawning + re-registering a stdio subprocess (the
# per-turn MCP registration race). It dials the control socket
# (`/run/hive/mcp.sock`, the harness binaries' default) fresh on every
# tool call, so a host-side hive-c0re restart is transparent.
# `before = hive-ag3nt` so the URL is already listening by the time
# the harness renders the first turn's config; the harness/claude also
# reconnect on their own, so ordering is a latency nicety not a hard
# correctness dep.
systemd.services.hive-mcp-http = {
description = "persistent streamable-http MCP daemon for the hyperhive surface";
wantedBy = [ "multi-user.target" ];
before = [ "hive-ag3nt.service" ];
@ -1959,11 +1957,12 @@ in
serviceConfig = {
ExecStart = "${pkgs.hyperhive}/bin/hive-agent-mcp --http 127.0.0.1:${toString config.hyperhive.mcp.httpPort}";
SyslogIdentifier = "hive-mcp-http";
# `always` (not `on-failure`): this endpoint is load-bearing when
# `httpPort` is set — a down window is total hyperhive-MCP loss with
# no stdio fallback and no per-turn self-heal (the URL just stays
# dead). `always` also covers any unforeseen clean-return path and
# restarts after a stray SIGTERM stops it out from under the harness.
# `always` (not `on-failure`): this endpoint is load-bearing — the
# sole hyperhive-MCP transport, so a down window is total
# hyperhive-MCP loss with no stdio fallback and no per-turn
# self-heal (the URL just stays dead). `always` also covers any
# unforeseen clean-return path and restarts after a stray SIGTERM
# stops it out from under the harness.
Restart = "always";
RestartSec = 3;
User = userName;
@ -2063,12 +2062,12 @@ in
# `hive_c0re::agent_sockets::socket_path_for(name)` so lifecycle
# bind-mounts and gateway upstream config stay in sync.
HIVE_WEB_SOCKET = "/run/hive-agent/${userName}/web.sock";
}
// lib.optionalAttrs (config.hyperhive.mcp.httpPort != null) {
# Presence tells `render_claude_config` to point claude at the
# persistent `hive-mcp-http` daemon's loopback URL instead of a
# per-turn stdio child. Kept in sync with the `hive-mcp-http`
# unit's `--http` port above via the same option.
# Loopback URL of the persistent `hive-mcp-http` daemon that
# `render_claude_config` points claude at for the built-in
# surface (HTTP is the sole transport — no per-turn stdio child).
# Kept in sync with the `hive-mcp-http` unit's `--http` port
# above via the same option. Always set — network isolation is
# unconditional, so a fixed per-container port is collision-free.
HYPERHIVE_MCP_HTTP_PORT = toString config.hyperhive.mcp.httpPort;
}
// lib.optionalAttrs config.hyperhive.gui.enable {