Watch

atlas 0db83c40a0 feat(#2642 ): a github.com notification poller alongside the forge one

hive-forge-notify grows a second binary, hive-github-notify. The two
share the notification half of the job — tolerant parse, classification,
formatting, dedupe, todo delivery — and nothing else: each binary owns
its host's protocol outright.

Two binaries rather than one multi-source daemon, and rather than a
cargo feature. A feature would unify across the workspace and cost every
crate its build cache. Two binaries keep the decision in nix: forge.nix
installs the forge unit, github.nix installs the github one under
hyperhive.github.enable, so a hive built without that module has no
github poller in its closure at all — GitHub access is separable (a
tier, a policy boundary), not merely switched off. Both binaries ship
from the existing derivation, so packages.nix is untouched.

The split is real at the code level too, not just at the unit level.
source.rs is a trait; the impls live in the binaries that use them, so
neither binary links the other's protocol code and the library names no
host at all. The forge-only assigned-issue rollup moves into the forge
binary for the same reason: it asks the forge what is assigned to this
agent, which is not a notification-protocol concern.

At runtime the github unit needs a PAT at <state>/github-token, the same
dashboard-provisioned token the gh wrapper and the git credential helper
already use. No PAT: it logs why and exits 0, which is why the unit is
Restart=on-failure and not always.

Forgejo's notifications API is modelled on GitHub's, so one tolerant
parse serves both — the differences (string thread ids, PullRequest vs
Pull) are absorbed by lenient deserializers rather than a second parse
path. Thread ids normalise to String at the parse boundary; they are
only ever opaque keys. Todo keys gain a per-source prefix so the two
hosts cannot collide, and the forge's is deliberately empty to keep
existing forge todo keys stable across the deploy that lands this.

The github loop honours the server's X-Poll-Interval, re-arming only
when the server asks for a slower cadence than ours; the hint is read
before the status check, because it arrives on error and empty pages too
and that is exactly when it matters. Reading the notification stream
needs the notifications scope on the PAT, which a token minted for push
access typically lacks; the failure mode is silence, so docs/github.md
says so explicitly.

2026-07-31 17:23:18 +02:00

11 KiB

Raw Blame History

hyperhive — claude entry point

Hey claude. This is your starting page. The detailed docs live in docs/ and are written for humans + you both — read them when you need depth on a subsystem. This file is the index.

High-level project intro: README.md.
Open work + backlog: the forge issue tracker.
Operator/agent trust-boundary design: docs/boundary.md (area:ops issues for the deployment/gateway/privsep work).
Agent trust model (trust boundary, prompt-injection threat model, capability = accepted risk), credential isolation + sandbox threat model: docs/security.md.

Repo map

One line per crate / top-level dir. Each module's authoritative, always-current description lives in its own //! doc-comment — grep/read the module when you need detail. This index is kept deliberately lean: it auto-loads into every turn's context, and a hand-maintained per-file tree drifts out of sync with the code.

Rust workspace (`Cargo.toml` members)

hive-c0re/ — host daemon (runs as the unprivileged hive-core user). src/main.rs is the hive-c0re binary — daemon-only (serve + the periodic vacuum/sweep loops); the operator CLI lives in the separate hivectl crate, which talks to the daemon over the host admin socket. Owns the sqlite broker, approval + question + reminder + schedule queues, the meta flake, lifecycle (nixos-container shellouts), gateway / forge / matrix provisioning, per-container stats, and the axum operator dashboard (dashboard.rs). Largest crate.
hivectl/ — standalone operator CLI (hivectl binary). Talks to the hive-c0re daemon over the host admin socket (hive-host-sock wire types) — does NOT link hive-c0re. Verbs: list-agents, agent <name> <spawn|kill|destroy|rebuild|restart|set-parent|choom| watch|…>, approvals <pending|approve|deny>, forge/matrix/ github/gateway provisioning, stop/start, wg/peer-config.
hive-agent/, hive-agent-mcp/ — in-container harness, two sibling crates for every agent (not a single hive-ag3nt/ dir — that's the runtime/binary-family nickname, not a directory).
hive-agent/ — the serve-loop binary: turn-loop policy layer (turn.rs) over the hive-claude driver, per-agent web UI (web_ui/ module dir), event + turn-stats sqlite sinks, login flow, system-prompt renderer.
hive-agent-mcp/ — the embedded MCP server (long-lived streamable-http listener, hive-mcp-http systemd unit) + its claude launch-config layer (tool-group/capability → --allowedTools, --mcp-config render).
hive-claude/ — reusable, app-agnostic driver for headless claude --print: spawns the CLI, streams + classifies stream-json, parses per-turn Telemetry, and drives a durable self-compacting InfiniteSession (name + SessionStore + CompactionPolicy). Uses thiserror (it's a library); the hive-* binaries consume it with anyhow. See hive-claude/README.md.
hive-priv/ — minimal root privileged-helper, socket-activated at /run/hive/priv.sock; performs the few root operations (bind-mount edits, nsenter) the unprivileged hive-c0re delegates to it. See docs/boundary.md.
hive-forge/ — hive-forge Forgejo CLI wrapper; one module per verb under src/verbs/.
hive-forge-notify/ — per-agent notification poller daemons; turns unread notification threads into todos on the harness's in-agent socket. Two binaries from one crate: hive-forge-notify (the hive's Forgejo) and hive-github-notify (github.com, installed by nix/agent-modules/github.nix). Was a task inside the hive-agent serve loop; own process since it needs nothing else from the harness.
hive-matrix-mcp/ — per-agent matrix-sdk daemon (hive-matrix-daemon); serves its MCP tools (send_message, read_room, …) directly over streamable-http (no stdio bridge), same shape as hive-bash-mcp.
hive-bash-mcp/ — per-agent bash-task runner daemon (hive-bash-daemon); serves its MCP tools (run/status/kill) directly over streamable-http (no stdio bridge), writes task files under /harness/bash-tasks/, and records the favorite-tools bash_commands stat into turn-stats.sqlite.
hive-sh4re/ — shared wire types (Agent / Manager request + response, Message, Approval, HelperEvent) used across the unix sockets. Host-admin-socket and hive-priv-socket wire types have been split out into their own crates (below) so hivectl and hive-priv don't need to pull in the rest of hive-sh4re.
hive-host-sock/ — wire types for the host admin socket (/run/hyperhive/host.sock), the protocol hivectl speaks to hive-c0re. Split out of hive-sh4re so a standalone hivectl only depends on this protocol crate, not the whole daemon crate.
hive-priv-sock/ — wire types for the hive-priv privileged-helper socket (/run/hive/priv.sock), shared by hive-priv (server) and hive-c0re (client). Also split out of hive-sh4re.
hive-sock-client/ — the shared JSON-line-over-unix-socket client every daemon uses to talk to a hyperhive socket. Generic over the request/response types, so the host-served control socket and the harness's in-agent socket both use it with their own wire-type crates. Retry is a policy value (Retry::None for callers already inside a poll loop, Retry::RideOutRestart for callers with no natural retry), and the response is either decoded (request) or drained (notify). Deliberately separate from the *-sock crates — those stay dependency-free wire types.
hive-metric/ — small CLI to push a single labeled metric to the OTEL collector via the OpenTelemetry Rust SDK / OTLP HTTP exporter.

Other top-level dirs

frontend/ — npm workspaces → static dashboard + per-agent UI dist, built hermetically by nix/packages/frontend.nix. Packages: shared (terminal pane + Catppuccin palette), dashboard (the operator SPA), agent (the default per-container UI).
nix/ — host-modules/ (the host stack: hyperhive core options, hive-{c0re,priv,forge,gateway,matrix,network,tls,ci}, otel, swarm), agent-modules/ (the per-agent harness feature modules), templates/{agent,ruth}.nix (container entry points), packages/ (flake package outputs), docs/ (the options-doc derivation), plus sources.nix / rust.nix / checks.nix / devshell.nix / treefmt.nix behind the thin flake.nix.
docs/ — subsystem reference docs (see Reading paths below).
branding/, scripts/ — static assets + helper scripts.

Reading paths

Pick the doc that matches your task. None depend on the others — read them à la carte.

"How do I bring a fresh hive online (first-run hivectl bootstrap)?" → docs/setup.md.
"What does the dashboard look like?" → docs/web-ui.md (index; sub-pages: shape, dashboard, agent).
"How does the per-agent terminal classify + colour events?" → docs/terminal-rendering.md.
"How does claude get its prompt and what tools does it have?" → docs/turn-loop.md (index: the loop, binary shape, turn outcomes; sub-pages: claude-invocation, config, mcp).
"How do config changes flow from manager to operator to container?" → docs/approvals.md.
"What state survives destroy / purge / restart?" → docs/persistence.md.
"Naming, commit style, wire protocol, the data-async pattern." → docs/conventions.md.
"Why does the nspawn flag look like that?" → docs/gotchas.md.
"What nginx vhosts does the gateway serve? How does matrix discovery work?" → docs/gateway.md.
"How do per-agent forge accounts work? What does forge_notify poll + how does it format wake messages?" → docs/forge.md.
"What verbs does hive-forge support? How do I post a comment, upload an attachment, manage subscriptions?" → docs/tools/forge.md.
"What does hivectl do? How do I provision a forge/matrix account, manage gateway users, restart containers, or drop into an agent shell?" → docs/tools/hivectl.md.
"How does the matrix-tuwunel container work? What about fluffychat-web and per-agent matrix accounts?" → docs/matrix.md.
"How do I give an agent a GitHub account (gh + git push)? How is the PAT injected?" → docs/github.md.
"How does DNS resolution work in agent containers? What's the bridge network for?" → docs/network.md.
"How do I connect two hives into a swarm? How do I declare peer hives and configure TLS trust?" → docs/swarm.md.
"How does the rebuild queue work? What are queue kinds and sources?" → docs/coordinator.md.
"How does the CI runner work? What's the auto-registration flow?" → docs/ci.md.
"What is /knowledge? How does the hive-wide knowledge repo sync, and how do I contribute a document?" → docs/knowledge.md.
"How do I export Claude Code metrics (tokens, cost, tool calls) to a Prometheus/Grafana collector? What OTEL options are available?" → docs/observability.md.

Conventions & process

The docs below own the details — this section just points at them.

Commit style, naming, identity, reconcile verb: → docs/conventions.md.
Never add #[allow(clippy::…)] — fix the lint instead (extract a helper, add backticks, etc.). Details + worked examples: → docs/conventions.md.
NixOS / nspawn quirks (bind mounts, conf flags, etc.): → docs/gotchas.md.
Turn loop, sentinels (rate-limit, auth-failed), context window: → docs/turn-loop.md.
Two-step spawn, approval flow, flake.lock validation: → docs/approvals.md.
Pre-push lint hook (catches tracker-tag and comment-block failures before CI does — install once per clone): ln -sf ../../scripts/pre-push .git/hooks/pre-push

11 KiB Raw Blame History