vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

agents get a todo when assigned to an issue or PR, rewritten when unassigned and cleared on close

Browse source 
A new swarm service, swarm-notify-router, in its own container on the host
that sets deploy.notify-router.enable (defaulted on by allSwarmServices where
the host has the router's store leaf, like the claude login minter).

- swarm-controller registers instance-wide hooks for `issues` +
  `issue_assign` and `pull_request` + `pull_request_assign`, updates the
  event list of a hook already registered with fewer (leaving its active flag as it is), and keeps each
  verified body unchanged in the SWARM_EVENTS stream on
  $SWARM.events.forge.<X-Forgejo-Event-Type>, deduped by X-Forgejo-Delivery.
  A delivery the stream did not keep answers 503.
- swarm-notify-router drains its durable consumer on that stream, runs its
  flows (source -> match -> todo sink, stored as nodes plus edges and checked
  to be a tree) and queues each op on $SWARM.router.todo.<agent> in the
  work-queue stream ROUTER_TODOS. Built-in flows: assigned, unassigned (same
  key, rewritten) and closed (prefix-clears every holder's router todos). A
  per-issue snapshot in its notify-router-state bucket holds the last
  assignees and the todo holders; only agents in the agent-status bucket get
  ops, and an assignee that was not one is left out of the snapshot so the
  next assign event retries it. After the first event that must be retried,
  the rest of the batch is nakked so redelivery keeps stream order.
- hive-agent creates its own durable consumer todo-<agent> and applies each
  op to its todo store as subsystem `router`, then acks; a refused op stops
  the rest of its batch the same way. The store's (subsystem, key) dedup
  makes a redelivery harmless. New Todos::clear_prefix.
- swarm-nats-auth: a --notify-router-client arm; the controller may publish
  $SWARM.events.forge.*; each agent gets three {agent} templates (extended
  consumer create whose filter the server checks, MSG.NEXT, ACK).
- Credentials: a notify-router store leaf minted by glue-bao-tls, a
  swarm-notify-router bao policy (read on its own queue client secret), an
  authelia machine client, the secret publisher entry, a journal forwarder
  route. No manual step.

hive-forge-notify is unchanged, so agents also still get forge's own
notifications.

Refs #5571
This commit is contained in:
atlas 2026-10-22 03:03:26 +02:00 • committed by mara
commit 0f7b403803
1 changed files with 3587 additions and 96 deletions
Download patch file Download diff file Expand all files Collapse all files

518
swarm-notify-router/src/flow.rs Normal file
View file

@ -0,0 +1,518 @@
//! Flows: what the router does with an event.
//!
//! A flow is a graph of nodes joined by edges, rooted at one `source` node.
//! A `match` node passes when every one of its conditions holds, and then
//! each of its children runs, so one event can reach several branches. A
//! `sink` node is a leaf and emits one todo op per recipient.
//!
//! Flows are trees: one source, one parent for every other node, every node
//! reachable from the source. [`Flow::validate`] checks that, and the router
//! runs only flows that pass.
//!
//! The built-in flows ([`builtins`]) are ordinary flows defined in code.
use std::collections::{BTreeMap, BTreeSet};
use swarm_wire::router_todo::{Op, RouterTodoOp};
/// The most nodes one flow may have.
const MAX_NODES: usize = 64;
/// The deepest a node may sit below the source.
const MAX_DEPTH: usize = 16;
/// One event, as the fields a flow reads.
pub type Fields = BTreeMap<&'static str, Value>;
/// A field's value.
#[derive(Clone, Debug, PartialEq, Eq)]
pub enum Value {
Text(String),
List(Vec<String>),
}
#[derive(Clone, Debug)]
pub struct Flow {
pub id: String,
pub nodes: Vec<Node>,
pub edges: Vec<Edge>,
}
#[derive(Clone, Debug)]
pub struct Node {
pub id: String,
pub kind: NodeKind,
}
/// A directed edge, parent to child.
#[derive(Clone, Debug)]
pub struct Edge {
pub from: String,
pub to: String,
}
#[derive(Clone, Debug)]
pub enum NodeKind {
/// The root: the forge event types the flow starts from.
Source { events: Vec<String> },
/// Passes when every condition holds.
Match { when: Vec<Condition> },
/// Emits a todo op to each recipient.
Todo(TodoSink),
}
/// `field` equals `value`. A list field holds when one of its items does.
#[derive(Clone, Debug)]
pub struct Condition {
pub field: String,
pub equals: String,
}
#[derive(Clone, Debug)]
pub struct TodoSink {
/// The field naming the recipients, a list of agent names.
pub to: String,
pub op: Op,
/// The todo key, with `{field}` placeholders.
pub key: String,
/// For a clear: `key` is a prefix.
pub prefix: bool,
/// The todo text, with `{field}` placeholders. Unused for a clear.
pub text: String,
}
/// One op the walk emitted, for one recipient.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct Action {
pub flow: String,
pub node: String,
pub agent: String,
pub op: RouterTodoOp,
}
impl Flow {
/// Check the flow is a tree within the caps, or say why not.
pub fn validate(&self) -> Result<(), String> {
if self.nodes.len() > MAX_NODES {
return Err(format!("{} nodes, more than {MAX_NODES}", self.nodes.len()));
}
let mut ids = BTreeSet::new();
for node in &self.nodes {
if !ids.insert(node.id.as_str()) {
return Err(format!("node id {} is used twice", node.id));
}
}
for edge in &self.edges {
for end in [&edge.from, &edge.to] {
if !ids.contains(end.as_str()) {
return Err(format!("an edge names {end}, which is no node"));
}
}
}
let roots: Vec<&Node> = self
.nodes
.iter()
.filter(|n| matches!(n.kind, NodeKind::Source { .. }))
.collect();
let [root] = roots.as_slice() else {
return Err(format!("{} source nodes, not one", roots.len()));
};
for node in &self.nodes {
if matches!(node.kind, NodeKind::Todo(_)) && !self.children(&node.id).is_empty() {
return Err(format!("sink {} has children", node.id));
}
}
// From the one root, every node reached exactly once: a node reached
// twice has a second parent or sits on a cycle, and one not reached is
// cut off.
let mut reached = 0;
let mut stack = vec![(root.id.as_str(), 0)];
while let Some((id, depth)) = stack.pop() {
if depth > MAX_DEPTH {
return Err(format!("node {id} is deeper than {MAX_DEPTH}"));
}
reached += 1;
if reached > self.nodes.len() {
return Err(
"a node is reached twice: it has two parents or sits on a cycle".to_owned(),
);
}
stack.extend(self.children(id).into_iter().map(|c| (c, depth + 1)));
}
if reached != self.nodes.len() {
return Err(format!(
"{} of {} nodes are not reachable from the source",
self.nodes.len() - reached,
self.nodes.len()
));
}
Ok(())
}
fn children(&self, id: &str) -> Vec<&str> {
self.edges
.iter()
.filter(|e| e.from == id)
.map(|e| e.to.as_str())
.collect()
}
fn node(&self, id: &str) -> Option<&Node> {
self.nodes.iter().find(|n| n.id == id)
}
/// Walk the flow for one event of type `event_type` and return the ops it
/// emits. Assumes [`Self::validate`] passed.
pub fn evaluate(&self, event_type: &str, fields: &Fields) -> Vec<Action> {
let Some(root) = self
.nodes
.iter()
.find(|n| matches!(&n.kind, NodeKind::Source { events } if events.iter().any(|e| e == event_type)))
else {
return Vec::new();
};
let mut actions = Vec::new();
let mut stack = vec![root.id.as_str()];
while let Some(id) = stack.pop() {
let Some(node) = self.node(id) else { continue };
match &node.kind {
NodeKind::Source { .. } => stack.extend(self.children(id)),
NodeKind::Match { when } => {
if when.iter().all(|c| holds(c, fields)) {
stack.extend(self.children(id));
}
}
NodeKind::Todo(sink) => actions.extend(self.emit(node, sink, fields)),
}
}
actions
}
fn emit(&self, node: &Node, sink: &TodoSink, fields: &Fields) -> Vec<Action> {
let recipients = match fields.get(sink.to.as_str()) {
Some(Value::List(items)) => items.clone(),
Some(Value::Text(one)) => vec![one.clone()],
None => Vec::new(),
};
let op = RouterTodoOp {
op: sink.op,
key: expand(&sink.key, fields),
prefix: sink.op == Op::Clear && sink.prefix,
summary: match sink.op {
Op::Upsert => expand(&sink.text, fields),
Op::Clear => String::new(),
},
source: format!("router:{}", self.id),
};
recipients
.into_iter()
.map(|agent| Action {
flow: self.id.clone(),
node: node.id.clone(),
agent,
op: op.clone(),
})
.collect()
}
}
fn holds(condition: &Condition, fields: &Fields) -> bool {
match fields.get(condition.field.as_str()) {
Some(Value::Text(v)) => *v == condition.equals,
Some(Value::List(items)) => items.contains(&condition.equals),
None => false,
}
}
/// `template` with each `{field}` replaced by that text field's value. A list
/// field is joined with `, `; an unknown name is left as written.
pub fn expand(template: &str, fields: &Fields) -> String {
let mut out = String::with_capacity(template.len());
let mut rest = template;
while let Some(open) = rest.find('{') {
out.push_str(&rest[..open]);
let after = &rest[open + 1..];
let Some(close) = after.find('}') else {
out.push_str(&rest[open..]);
return out;
};
let name = &after[..close];
match fields.get(name) {
Some(Value::Text(v)) => out.push_str(v),
Some(Value::List(items)) => out.push_str(&items.join(", ")),
None => {
out.push('{');
out.push_str(name);
out.push('}');
}
}
rest = &after[close + 1..];
}
out.push_str(rest);
out
}
/// The flows the router ships with.
///
/// - `assigned`: a newly assigned agent gets a todo for the issue or PR.
/// - `unassigned`: an agent taken off it gets that todo rewritten to say so.
/// Both write the same key, so the second replaces the first.
/// - `closed`: closing it clears every router todo the router queued for it.
pub fn builtins() -> Vec<Flow> {
let assignment_key = "forge:{repo}#{number}:assignment";
vec![
chain(
"assigned",
&["issue_assign", "pull_request_assign"],
"assigned",
TodoSink {
to: "assignees_added".to_owned(),
op: Op::Upsert,
key: assignment_key.to_owned(),
prefix: false,
text: "you are assigned to {kind} {repo}#{number}: {title} ({url})".to_owned(),
},
),
chain(
"unassigned",
&["issue_assign", "pull_request_assign"],
"unassigned",
TodoSink {
to: "assignees_removed".to_owned(),
op: Op::Upsert,
key: assignment_key.to_owned(),
prefix: false,
text: "you are not assigned to {kind} {repo}#{number} anymore: {title} ({url})"
.to_owned(),
},
),
chain(
"closed",
&["issues", "pull_request"],
"closed",
TodoSink {
to: "todo_holders".to_owned(),
op: Op::Clear,
key: "forge:{repo}#{number}:".to_owned(),
prefix: true,
text: String::new(),
},
),
]
}
/// source(`events`) → match(`action` = `action`) → `sink`.
fn chain(id: &str, events: &[&str], action: &str, sink: TodoSink) -> Flow {
let node = |id: &str, kind| Node {
id: id.to_owned(),
kind,
};
let edge = |from: &str, to: &str| Edge {
from: from.to_owned(),
to: to.to_owned(),
};
Flow {
id: id.to_owned(),
nodes: vec![
node(
"source",
NodeKind::Source {
events: events.iter().map(|e| (*e).to_owned()).collect(),
},
),
node(
"match",
NodeKind::Match {
when: vec![Condition {
field: "action".to_owned(),
equals: action.to_owned(),
}],
},
),
node("sink", NodeKind::Todo(sink)),
],
edges: vec![edge("source", "match"), edge("match", "sink")],
}
}
#[cfg(test)]
mod tests {
use super::*;
fn text(v: &str) -> Value {
Value::Text(v.to_owned())
}
fn list(v: &[&str]) -> Value {
Value::List(v.iter().map(|s| (*s).to_owned()).collect())
}
fn event(action: &str, added: &[&str], removed: &[&str], holders: &[&str]) -> Fields {
Fields::from([
("action", text(action)),
("kind", text("issue")),
("repo", text("hyperhive/hyperhive")),
("number", text("7")),
("title", text("router")),
("url", text("https://forge/7")),
("assignees_added", list(added)),
("assignees_removed", list(removed)),
("todo_holders", list(holders)),
])
}
fn flow(id: &str) -> Flow {
builtins()
.into_iter()
.find(|f| f.id == id)
.expect("a built-in")
}
#[test]
fn every_builtin_is_a_valid_tree() {
for f in builtins() {
assert_eq!(f.validate(), Ok(()), "{}", f.id);
}
}
#[test]
fn assigning_queues_a_todo_for_each_newly_assigned_agent_only() {
let got = flow("assigned").evaluate(
"issue_assign",
&event("assigned", &["atlas", "iris"], &[], &[]),
);
let agents: Vec<&str> = got.iter().map(|a| a.agent.as_str()).collect();
assert_eq!(agents, ["atlas", "iris"]);
assert_eq!(got[0].op.op, Op::Upsert);
assert_eq!(got[0].op.key, "forge:hyperhive/hyperhive#7:assignment");
assert_eq!(
got[0].op.summary,
"you are assigned to issue hyperhive/hyperhive#7: router (https://forge/7)"
);
assert_eq!(got[0].op.source, "router:assigned");
// The match gates: an unassign event reaches no assigned sink.
assert!(
flow("assigned")
.evaluate("issue_assign", &event("unassigned", &["atlas"], &[], &[]))
.is_empty()
);
// The source gates: another event type starts nothing.
assert!(
flow("assigned")
.evaluate("issues", &event("assigned", &["atlas"], &[], &[]))
.is_empty()
);
}
#[test]
fn unassigning_rewrites_the_same_key_for_the_removed_agent() {
let got = flow("unassigned").evaluate(
"pull_request_assign",
&event("unassigned", &[], &["argus"], &[]),
);
assert_eq!(got.len(), 1);
assert_eq!(got[0].agent, "argus");
assert_eq!(got[0].op.key, "forge:hyperhive/hyperhive#7:assignment");
assert!(
got[0]
.op
.summary
.starts_with("you are not assigned to issue")
);
}
#[test]
fn closing_clears_by_prefix_for_every_holder() {
let got =
flow("closed").evaluate("issues", &event("closed", &[], &[], &["atlas", "argus"]));
assert_eq!(got.len(), 2);
for a in &got {
assert_eq!(a.op.op, Op::Clear);
assert!(a.op.prefix);
assert_eq!(a.op.key, "forge:hyperhive/hyperhive#7:");
assert_eq!(a.op.summary, "");
}
assert!(
flow("closed")
.evaluate("issues", &event("reopened", &[], &[], &["atlas"]))
.is_empty()
);
}
/// A match node with two children runs both: the branching the tree
/// allows.
#[test]
fn a_match_with_two_children_runs_both() {
let mut f = flow("assigned");
let NodeKind::Todo(sink) = f.nodes[2].kind.clone() else {
panic!("the third node is the sink")
};
f.nodes.push(Node {
id: "second".to_owned(),
kind: NodeKind::Todo(TodoSink {
to: "assignees_removed".to_owned(),
..sink
}),
});
f.edges.push(Edge {
from: "match".to_owned(),
to: "second".to_owned(),
});
assert_eq!(f.validate(), Ok(()));
let got = f.evaluate(
"issue_assign",
&event("assigned", &["atlas"], &["argus"], &[]),
);
let mut nodes: Vec<(&str, &str)> = got
.iter()
.map(|a| (a.node.as_str(), a.agent.as_str()))
.collect();
nodes.sort_unstable();
assert_eq!(nodes, [("second", "argus"), ("sink", "atlas")]);
}
#[test]
fn a_flow_that_is_not_a_tree_is_refused() {
let edge = |from: &str, to: &str| Edge {
from: from.to_owned(),
to: to.to_owned(),
};
let mut cycle = flow("assigned");
cycle.edges[0] = edge("sink", "match");
cycle.edges.push(edge("match", "sink"));
assert!(cycle.validate().is_err(), "a cycle");
let mut orphan = flow("assigned");
orphan.edges.pop();
assert!(orphan.validate().is_err(), "an unreachable sink");
let mut two_parents = flow("assigned");
two_parents.edges.push(edge("source", "sink"));
assert!(two_parents.validate().is_err(), "a sink with two parents");
let mut sink_child = flow("assigned");
sink_child.nodes.push(Node {
id: "m2".to_owned(),
kind: NodeKind::Match { when: Vec::new() },
});
sink_child.edges.push(edge("sink", "m2"));
assert!(sink_child.validate().is_err(), "a sink with a child");
let mut dangling = flow("assigned");
dangling.edges.push(edge("match", "nowhere"));
assert!(dangling.validate().is_err(), "an edge to no node");
let mut two_sources = flow("assigned");
two_sources.nodes[1].kind = NodeKind::Source { events: Vec::new() };
assert!(two_sources.validate().is_err(), "two sources");
}
#[test]
fn a_placeholder_expands_to_its_field_and_an_unknown_one_stays() {
let fields = event("assigned", &["a", "b"], &[], &[]);
assert_eq!(
expand("{repo}#{number} {assignees_added} {nope} {", &fields),
"hyperhive/hyperhive#7 a, b {nope} {"
);
}
}