refactor(hive-agent): split the forge notification poller into its own crate

The poller was a `tokio::spawn` inside the `hive-agent` serve loop. It
never needed anything from that loop except a socket path, so being
in-process bought nothing and cost two things: a harness restart took
forge notifications down with it, and the whole forge/HTTP dependency
tree was linked into the serve-loop binary.

It is now `hive-forge-notify`, a per-agent daemon with its own systemd
unit, a sibling of `hive-bash-daemon` and `hive-matrix-daemon`. Same
contract as those two: it reaches the harness only by upserting todos on
the in-agent socket, and nowhere else.

The module moves verbatim (`notify.rs`) — the formatters, the activation
gates, the dedupe map and all 33 tests are unchanged. Only the socket
call sites are rewritten, onto a small local `todo_client` rather than
the harness's. That mirrors what both sibling daemons already do, and
the etiquette differs on purpose: the harness's client carries a 60s
backoff schedule sized to ride out a hive-c0re restart, which its
callers need because they have no retry of their own. This poller's two
call sites both sit inside the 30s poll loop and both treat a failure as
"leave the thread unread, try next tick", so the poll interval already
is the retry; a second backoff would only stack sleeps and delay the
rest of the batch.

The unit is `Restart=on-failure`, not `always`. An agent with no forge
account is a supported configuration and the poller reports it by
logging why and exiting 0 — under `always` that clean exit would be a
restart loop on every forge-less agent.

`forgejo-api`, `url` and `time` drop out of `hive-agent`'s dependencies
with the module.

Also corrects docs that outlived the code they described: the persisted
`forge_cursor` field is long gone (forge's own read-state is the durable
record of what has been delivered), but `docs/persistence.md` and the
`harness_state` module docs still documented it as live.
This commit is contained in:
atlas 2026-07-26 20:44:54 +02:00 committed by mara
commit 246c9471b1
18 changed files with 301 additions and 48 deletions

View file

@ -45,7 +45,7 @@ hand-maintained per-file tree drifts out of sync with the code.
- **`hive-agent/`** — the serve-loop binary: turn-loop *policy* layer
(`turn.rs`) over the `hive-claude` driver, per-agent web UI (`web_ui/`
module dir), event + turn-stats sqlite sinks, login flow, system-prompt
renderer, forge-notify subscriber.
renderer.
- **`hive-agent-mcp/`** — the embedded MCP server (long-lived
streamable-http listener, `hive-mcp-http` systemd unit) + its claude
launch-config layer (tool-group/capability → `--allowedTools`,
@ -62,6 +62,10 @@ hand-maintained per-file tree drifts out of sync with the code.
`docs/boundary.md`.
- **`hive-forge/`** — `hive-forge` Forgejo CLI wrapper; one module per
verb under `src/verbs/`.
- **`hive-forge-notify/`** — per-agent Forgejo notification poller daemon
(`hive-forge-notify`); turns unread notification threads into todos on
the harness's in-agent socket. Was a task inside the `hive-agent` serve
loop; own process since it needs nothing else from the harness.
- **`hive-matrix-mcp/`** — per-agent matrix-sdk daemon
(`hive-matrix-daemon`); serves its MCP tools (`send_message`,
`read_room`, …) directly over streamable-http (no stdio bridge),

20
Cargo.lock generated
View file

@ -1521,7 +1521,6 @@ dependencies = [
"anyhow",
"axum",
"clap",
"forgejo-api",
"futures-util",
"hive-agent-sock",
"hive-claude",
@ -1538,13 +1537,11 @@ dependencies = [
"serde",
"serde_json",
"tempfile",
"time",
"tokio",
"tokio-stream",
"tower-http 0.7.0",
"tracing",
"tracing-subscriber",
"url",
]
[[package]]
@ -1671,6 +1668,23 @@ dependencies = [
"url",
]
[[package]]
name = "hive-forge-notify"
version = "0.1.0"
dependencies = [
"anyhow",
"forgejo-api",
"hive-agent-sock",
"reqwest 0.13.1",
"serde",
"serde_json",
"time",
"tokio",
"tracing",
"tracing-subscriber",
"url",
]
[[package]]
name = "hive-host-sock"
version = "0.1.0"

View file

@ -10,6 +10,7 @@ members = [
"hive-claude",
"hive-screen-mcp",
"hive-forge",
"hive-forge-notify",
"hive-host-sock",
"hive-jobq",
"hive-matrix-mcp",

View file

@ -136,7 +136,7 @@ can't:
| `Kill` / `Start` / `Restart` / `Update` | container lifecycle on an existing agent | **topology** — descendants only |
| `RequestUpdateMetaInputs` | bump meta `flake.lock` | **per-agent cap** (root-only today; a future "let coder bump its own input" might grant it) |
| `GetLogs` | journalctl scrape of a sub-agent | **topology** — descendants only |
| `Wake` | inject a `from: <X>` message into self's inbox | **not really privileged** — the wire surface exists because daemon co-processes (e.g. `forge_notify`) need it. Sub-agents have the same via their own socket. |
| `Wake` | inject a `from: <X>` message into self's inbox | **not really privileged** — the wire surface exists because the per-agent daemons (e.g. `hive-forge-notify`) need it. Sub-agents have the same via their own socket. |
### C — storage / mounts (`hive-c0re::lifecycle`)

View file

@ -83,9 +83,11 @@ the typed `forgejo-api` crate; only non-`/api/v1` web-router routes
enrichment fetches of server-provided subject URLs stay on raw
reqwest.
## Notification poller (`hive-ag3nt/src/forge_notify.rs`)
## Notification poller (`hive-forge-notify/src/notify.rs`)
Background task spawned once per harness boot. Polls
Its own long-running per-agent daemon (`hive-forge-notify`, one systemd
unit, a sibling of `hive-bash-daemon` / `hive-matrix-daemon`) — it used
to be a background task inside the `hive-agent` serve loop. Polls
`GET /api/v1/notifications?all=false` every 30 seconds (Forgejo's
unread-only filter), formats each notification as a broker
`Wake { from: "forge" }` message, and delivers it to the agent's own

View file

@ -142,7 +142,7 @@ Consolidated harness state file written atomically (`.tmp` + rename) by
Shape:
```json
{ "rate_limited": false, "needs_login": false, "active_model": "…", "forge_cursor": { "42": "2026-07-01T18:00:00Z" } }
{ "rate_limited": false, "needs_login": false, "active_model": "…" }
```
- `rate_limited` — set when the harness detects a 429 from the Claude
@ -152,16 +152,16 @@ Shape:
cleared by `"online"` status (re-auth completed). Drives the
`needs_login` flag alongside the `claude_has_session` check.
- `active_model` — the resolved Claude model for the dashboard badge.
- `forge_cursor` — the `forge_notify` delivery-dedupe cursor
(notification thread id → last-delivered `updated_at`), so a
rebuild/restart doesn't re-deliver the whole currently-unread forge
backlog. See [`forge.md`](forge.md).
Multiple harness tasks write this file (the turn loop for the first
three fields, the `forge_notify` poller for `forge_cursor`), so every
writer goes read-modify-write under a shared in-process lock — each
preserves the fields it doesn't own rather than reconstructing the file
from scratch.
The turn loop is the only writer today, but it still goes
read-modify-write under a shared in-process lock and merges into the
existing object rather than reconstructing it — so a second writer
preserves fields it doesn't own, and the lock closes the lost-update
window between a writer's read and its rename. (The forge notification
poller used to be that second writer, for a delivery-dedupe cursor. It
persists nothing now — forge's own read-state is the durable record of
what has been delivered — and it is a separate process, which an
in-process lock could not have serialised anyway.)
hive-c0re reads this file on each `build_all` sweep (~10s) via
`container_view::read_harness_flags`. Falls back to the legacy individual
@ -258,7 +258,7 @@ Under `/var/lib/hyperhive/agents/<name>/`:
hourly and deletes terminal task trios older than 48 hours;
non-terminal (still-running) tasks are never deleted by vacuum.
- `hyperhive-todos.sqlite` — loose-ends-v2 todo store. In-container
MCP daemons (`hive-bash-daemon`, `hive-matrix-daemon`) and `forge_notify`
daemons (`hive-bash-daemon`, `hive-matrix-daemon`, `hive-forge-notify`)
upsert keyed todos here over the harness's in-agent socket
(`HIVE_AGENT_SOCKET`); the harness merges them into `get_loose_ends`
output and clears a row on `mark_todo_done`. Replaced the old

View file

@ -97,9 +97,11 @@ flake sets it unconditionally for any container-deployed agent;
see `docs/conventions.md::Hive identity` for the env stack),
opens turn-stats sqlite, prepares the on-boot files (see
[claude-invocation](turn-loop/claude-invocation.md#on-boot-files)),
installs claude plugins, spawns `forge_notify::run` + `web_ui::serve`,
installs claude plugins, spawns `web_ui::serve` + `vacuum::run`,
and either drops into `serve_loop` directly (`Online`) or parks on
the login flow first (`NeedsLogin`).
the login flow first (`NeedsLogin`). (The forge notification poller
used to be spawned here too; it is its own process now —
`hive-forge-notify`, see [`forge.md`](forge.md).)
`spawn_todo_socket` opens the todos store and, alongside
`todo_server::run` (the socket the out-of-process producers dial),

View file

@ -14,9 +14,6 @@ reqwest.workspace = true
hyper.workspace = true
hyper-util.workspace = true
http-body-util.workspace = true
forgejo-api.workspace = true
url.workspace = true
time.workspace = true
futures-util = "0.3"
clap.workspace = true
hive-claude.workspace = true

View file

@ -1,9 +1,8 @@
//! File-backed harness state, split out of [`crate::events`] (which owns
//! the live event bus + sqlite store). This module holds the runtime
//! model/effort selection, the consolidated `hyperhive-harness.json`
//! state file, and the `forge_notify` delivery-dedupe cursor — the
//! persisted knobs the harness reads/writes across turns, none of which
//! are about the live event stream.
//! model/effort selection and the consolidated `hyperhive-harness.json`
//! state file — the persisted knobs the harness reads/writes across
//! turns, none of which are about the live event stream.
use std::path::PathBuf;
@ -84,12 +83,14 @@ fn harness_json_path() -> PathBuf {
crate::paths::state_dir().join(HARNESS_JSON)
}
// Serialises the read-modify-write of `hyperhive-harness.json`. Two
// harness tasks touch it in the same process — the turn loop (rate-limit
// / needs-login / active-model) and the forge_notify poller (the
// delivery-dedupe cursor) — writing disjoint fields, so each writer must
// preserve the other's. The lock closes the lost-update window between a
// writer's read and its rename.
// Serialises the read-modify-write of `hyperhive-harness.json`. Every
// writer merges into the existing object rather than reconstructing it,
// so a future second writer preserves fields it doesn't own; the lock
// closes the lost-update window between a writer's read and its rename.
// (The forge poller used to be that second writer, for a delivery-dedupe
// cursor. It no longer persists one — forge's own read-state is the
// durable record — and it is a separate process now, which an
// in-process mutex could not have serialised anyway.)
static HARNESS_JSON_LOCK: std::sync::Mutex<()> = std::sync::Mutex::new(());
/// Read the consolidated state file as a JSON object, or an empty object

View file

@ -13,7 +13,6 @@ mod client;
mod db_migrate;
mod disk_watch;
mod events;
mod forge_notify;
mod harness_state;
mod identity;
mod login;
@ -550,15 +549,12 @@ async fn serve_main<S: Surface>(socket: &Path, poll_ms: u64) -> Result<()> {
for failure in plugins::install_configured().await {
S::send_to_parent(socket, failure).await;
}
// forge_notify pushes forge notifications as todos on the in-agent
// socket (loose-ends v2), not direct wakes — so it dials
// `HIVE_AGENT_SOCKET`, not the host mcp.sock.
tokio::spawn(crate::forge_notify::run(
std::env::var_os("HIVE_AGENT_SOCKET").map_or_else(
|| std::path::PathBuf::from(hive_agent_sock::DEFAULT_AGENT_SOCKET),
std::path::PathBuf::from,
),
));
// The forge notification poller used to be spawned here. It is its own
// process now (`hive-forge-notify`, its own systemd unit) so a harness
// restart doesn't take forge notifications down with it; it reaches the
// harness the same way the bash and matrix daemons do, by upserting
// todos on the in-agent socket.
//
// Agent-side cleanup of this agent's own harness artifacts (completed
// bash-task files + verbose event rows). Runs here, not host-side in
// hive-c0re, because the files are agent-owned — see `vacuum` module docs.

View file

@ -0,0 +1,31 @@
[package]
name = "hive-forge-notify"
edition.workspace = true
version.workspace = true
readme = "README.md"
[lints]
workspace = true
[dependencies]
anyhow.workspace = true
forgejo-api.workspace = true
hive-agent-sock.workspace = true
reqwest.workspace = true
serde.workspace = true
serde_json.workspace = true
time.workspace = true
tokio.workspace = true
tracing.workspace = true
tracing-subscriber.workspace = true
url.workspace = true
# `hive-forge-notify` — long-running per-agent Forgejo notification
# poller. Was a `tokio::spawn` inside the `hive-agent` serve loop; it is
# its own process now so a harness restart (or a claude-side stall) does
# not take forge notifications down with it, and so its dependency tree
# (forgejo-api, reqwest, time, url) is not linked into the serve-loop
# binary. Talks to the harness only over the in-agent todo socket.
[[bin]]
name = "hive-forge-notify"
path = "src/main.rs"

View file

@ -0,0 +1,44 @@
# hive-forge-notify
Per-agent Forgejo notification poller: a long-running daemon
(`hive-forge-notify`) that watches the agent's unread notification list
and turns each thread into a todo the harness surfaces in
`get_loose_ends`. This is why an agent wakes up when someone comments
on its issue or requests its review.
## When to use it
Look here when changing what a forge notification *says* when it
reaches an agent, or when it reaches one at all: the poll cadence, the
self-echo filter, the comment / review / new-item / state-change
wrapper formats, body-excerpt truncation, and the assigned-issue
rollup all live in `notify.rs`. The behaviour contract — activation
gates, filtering rules, the review-request override — is documented in
[`docs/forge.md`](../docs/forge.md), "Notification poller".
## Shape
One bin, three modules:
- **`main.rs`** — argument-free entry point. Reads `HIVE_AGENT_SOCKET`,
initialises tracing, hands off to `notify::run`.
- **`notify.rs`** — the poller: forge client setup, the 30s loop, the
formatters, mark-read, and the in-process delivery-dedupe map.
- **`todo_client.rs`** — one-shot JSON-line client for the harness's
in-agent socket. No retry schedule of its own; see the module doc.
## Why it is a separate process
It used to be a `tokio::spawn` inside the `hive-agent` serve loop. It
never needed anything from the serve loop except a socket path, and
running it in-process meant a harness restart also took forge
notifications down, and linked the whole forge/HTTP dependency tree
(`forgejo-api`, `reqwest`, `time`, `url`) into the serve-loop binary.
It is now a sibling daemon alongside `hive-bash-daemon` and
`hive-matrix-daemon`, with the same contract: it talks to the harness
over the in-agent todo socket and nowhere else.
Forge's own read-state is the durable, cross-rebuild record of what has
been delivered — there is no persisted cursor to migrate or corrupt. A
restarted poller re-scans only the genuinely-still-unread set, which is
tiny by construction because delivery marks the thread read.

View file

@ -0,0 +1,38 @@
//! `hive-forge-notify` binary — long-running per-agent Forgejo
//! notification poller. Polls the agent's unread notification list,
//! formats each thread into a short summary, and pushes it as a todo
//! (loose-ends v2) on the harness's in-agent socket so claude drives a
//! turn to handle it.
//!
//! Takes no arguments: everything comes from the environment the
//! per-agent systemd unit provides — `HIVE_FORGE_URL` (forwarded into
//! every container by the meta flake), `HYPERHIVE_STATE_DIR` (where the
//! agent's `forge-token` lives) and `HIVE_AGENT_SOCKET` (the harness's
//! todo socket). When the forge is not configured for this agent the
//! poller logs why and exits 0 — the unit is `Restart=on-failure`, so a
//! forge-less agent settles instead of restart-looping.
mod notify;
mod todo_client;
#[tokio::main]
async fn main() {
tracing_subscriber::fmt()
.with_env_filter(
tracing_subscriber::EnvFilter::try_from_env("RUST_LOG")
.unwrap_or_else(|_| tracing_subscriber::EnvFilter::new("info")),
)
.init();
let socket = std::env::var_os("HIVE_AGENT_SOCKET").map_or_else(
|| std::path::PathBuf::from(hive_agent_sock::DEFAULT_AGENT_SOCKET),
std::path::PathBuf::from,
);
tracing::info!(socket = %socket.display(), "hive-forge-notify starting");
// Returns only when the forge is not configured (or the token never
// arrives); otherwise loops forever. Either way there is nothing left
// for this process to do, so fall off the end and exit 0.
notify::run(socket).await;
}

View file

@ -1078,9 +1078,10 @@ async fn poll_once(
summary: body,
source: None,
};
let deliver_result = crate::client::request::<_, hive_agent_sock::Response>(socket, &req)
.await
.map(|_| ());
let deliver_result =
crate::todo_client::request::<_, hive_agent_sock::Response>(socket, &req)
.await
.map(|_| ());
match deliver_result {
Ok(()) => {
debug!(%id, "forge_notify: todo upserted");
@ -1244,7 +1245,7 @@ async fn update_assigned_rollup(
}
};
match crate::client::request::<_, hive_agent_sock::Response>(socket, &req).await {
match crate::todo_client::request::<_, hive_agent_sock::Response>(socket, &req).await {
Ok(_) => debug!(total, "forge_notify: assigned rollup todo updated"),
Err(e) => debug!("forge_notify: assigned rollup todo update failed: {e}"),
}

View file

@ -0,0 +1,86 @@
//! One-shot JSON-line client for the harness's in-agent socket
//! (`HIVE_AGENT_SOCKET`) — the only channel this daemon has back into the
//! harness.
//!
//! Deliberately has no retry/backoff schedule. The two callers are both
//! inside the 30s poll loop and both treat a failed request as "leave the
//! thread unread and try again next tick", so the poll interval *is* the
//! retry — a second, in-request backoff would only stack sleeps on top of
//! it and delay the rest of the batch. That is the opposite trade-off from
//! the serve loop's client, which rides out a hive-c0re restart because
//! its callers have no natural retry of their own.
//!
//! Each sibling per-agent daemon carries its own small helper like this
//! one rather than sharing the harness's, so a daemon's socket etiquette
//! stays visible in the crate that depends on it.
use std::path::Path;
use anyhow::{Context, Result, anyhow};
use serde::Serialize;
use serde::de::DeserializeOwned;
use tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader};
use tokio::net::UnixStream;
/// Write one JSON request line to `socket` and read the single JSON
/// response line back.
///
/// # Errors
///
/// Returns an error if the socket cannot be connected, if the request
/// cannot be serialised, if the write or read fails, if the server closes
/// without responding, or if the response does not deserialise into
/// `Resp`.
pub async fn request<Req, Resp>(socket: &Path, req: &Req) -> Result<Resp>
where
Req: Serialize + ?Sized,
Resp: DeserializeOwned,
{
let stream = UnixStream::connect(socket)
.await
.with_context(|| format!("connect to {}", socket.display()))?;
let (read, mut write) = stream.into_split();
let mut payload = serde_json::to_string(req)?;
payload.push('\n');
write
.write_all(payload.as_bytes())
.await
.with_context(|| format!("write to {}", socket.display()))?;
write
.flush()
.await
.with_context(|| format!("flush {}", socket.display()))?;
let mut reader = BufReader::new(read);
let mut line = String::new();
let read_bytes = reader
.read_line(&mut line)
.await
.with_context(|| format!("read from {}", socket.display()))?;
if read_bytes == 0 || line.is_empty() {
return Err(anyhow!(
"{} closed the connection without responding",
socket.display()
));
}
Ok(serde_json::from_str(line.trim())?)
}
#[cfg(test)]
mod tests {
use super::request;
/// Connecting to a path that does not exist is an error, not a hang —
/// the caller (a poll tick) needs it to come back promptly so the rest
/// of the batch still runs.
#[tokio::test]
async fn missing_socket_is_an_error() {
let bogus = std::path::Path::new("/nonexistent/hive/agent.sock");
let err = request::<(), serde_json::Value>(bogus, &())
.await
.expect_err("connect to a non-existent socket must fail");
let msg = format!("{err:#}");
assert!(msg.contains("connect to"), "missing connect context: {msg}");
}
}

View file

@ -57,6 +57,41 @@ in
config.hyperhive.packages.hive-forge
];
# Forge notification poller — a long-running sibling of
# `hive-bash-daemon` / `hive-matrix-daemon`. Polls the agent's unread
# notification list and upserts each thread as a todo on the harness's
# in-agent socket; it needs nothing else from the harness, which is why
# it is its own process rather than a task inside the serve loop.
systemd.services.hive-forge-notify = {
description = "Forgejo notification poller for this agent";
wantedBy = [ "multi-user.target" ];
after = [ "network.target" ];
environment = {
# In-agent todo socket the harness serves (loose-ends v2). Must
# match the harness's HIVE_AGENT_SOCKET (agent-service.nix) — the
# poller upserts one todo per forge thread here rather than firing
# a hive-c0re wake.
HIVE_AGENT_SOCKET = "/run/hive-agent/${userName}/agent.sock";
RUST_LOG = "info";
# HIVE_FORGE_URL and HYPERHIVE_STATE_DIR come from
# systemd.globalEnvironment (forwarded into every container by the
# meta flake) — the poller reads the forge base URL from the first
# and the agent's `forge-token` from under the second.
};
serviceConfig = {
ExecStart = "${config.hyperhive.packages.hive-forge-notify}/bin/hive-forge-notify";
SyslogIdentifier = "hive-forge-notify";
# `on-failure`, NOT `always`: an agent with no forge account is a
# supported configuration, and the poller reports that by logging
# why and exiting 0. Under `always` that clean exit would become a
# restart loop on every forge-less agent. A crash still restarts.
Restart = "on-failure";
RestartSec = 5;
User = userName;
Group = userName;
};
};
# One-shot: tea config.yml from the seeded forge token. Shape
# contract (always exit 0, no set -e, skip-silently, re-runnable):
# docs/conventions.md::Best-effort oneshot services.

View file

@ -12,7 +12,7 @@
hyperhive package outputs consumed by the harness modules: the
per-binary daemon/CLI packages (`hive-agent`, `hive-agent-mcp`,
`hive-bash-daemon`,
`hive-forge`, `hive-matrix-daemon`,
`hive-forge`, `hive-forge-notify`, `hive-matrix-daemon`,
`hive-metric`, `hive-screen-mcp`) plus the `assets`, `frontend` and
`reference-docs` trees. Wired by the flake's agent-base/ruth
nixosModules to `hyperhive.packages.<system>.*`; override an

View file

@ -31,6 +31,7 @@ let
hive-metric = "hyperhive agent-emitted custom metrics CLI";
hive-screen-mcp = "hyperhive screen MCP bridge (screenshot + input for GUI agents)";
hive-forge = "hyperhive Forgejo CLI";
hive-forge-notify = "hyperhive per-agent Forgejo notification poller daemon";
};
# ONE compile of the whole workspace (every bin, sharing the