Watch
0
0
Fork
You've already forked hyperhive
0

Compare commits

...
21 changed files with 2193 additions and 368 deletions

View file

@ -64,21 +64,60 @@ The router holds no forge token and no webhook secret, and registers no hook.
A flow is a graph of nodes joined by edges: one `source` node naming the forge A flow is a graph of nodes joined by edges: one `source` node naming the forge
event types it starts from, `match` nodes that pass when every condition event types it starts from, `match` nodes that pass when every condition
holds, and `sink` nodes that emit one todo op per recipient. A match node can holds, and `todo` nodes that emit one todo op per recipient. A match node can
have more than one child, and each one runs, so one event can reach several have more than one child, and each one runs, so one event can reach several
sinks. A flow must be a tree: one source, one parent for every other node, todos. A flow must be a tree: one source, one parent for every other node,
every node reachable, no children under a sink. every node reachable, no children under a `todo`. The most is 64 nodes, 16
levels deep, and 100 custom flows. A `todo` node sends to the agents an event
field holds (`assignees_added`, `todo_holders`, …) or to one named agent.
The router ships the three flows above, each a source → match on `action` → The router ships the three flows above, each a source → match on `action` →
todo sink. todo. The code defines them (`swarm_wire::notify_flows::builtins`), so a
build that changes one reaches every swarm. A custom flow has the same type and
the same evaluator, so it can do anything a built-in does.
**The stored document.** swarm-controller keeps the rest in one document, the
`all` key of the `notify-flows` bucket: the ids of the built-in flows you
switched off, and the custom flows. Only the controller writes it, through
`GET` and `PUT /api/notify/flows`. A `PUT` replaces the whole document in one
compare-and-set. The controller answers 400 unless every flow passes the tree
check above and reads only what exists: a source event from `issues`,
`issue_assign`, `pull_request` or `pull_request_assign`, and condition and
recipient fields from the event's field list. The same goes for the `{field}`
placeholders in a todo's key and text. A `PUT` must carry the `revision` it read, `0` for no
document, and gets 409 when the stored revision is another. Swarm UI's
[`/notifications`](ui.md#notification-flows) page shows the document and
switches built-ins.
**What the router runs.** It reads the key at most every 10 seconds, when an
event arrives, and runs the built-ins you left on, then the custom flows that
are on and pass the tree check. It logs a custom flow that fails the check,
leaves it out, and runs the rest.
| the key | the router runs |
| -------------------------------------- | ----------------------- |
| absent, deleted, or its bucket missing | the built-in flows |
| a document it reads | the above |
| a document it can't read | the flows it ran before |
| a read that fails | the flows it ran before |
"Can't read" means not JSON, not a flows document, or a `version` other than 1.
Before the router has read any document, "before" means the built-ins, so a bad
document never leaves it with no flows. The controller refuses to overwrite a
document of a version it doesn't know.
**Deploying in any order.** The controller, the router and the UI can be on
different hosts. A controller that predates this leaves the key absent, and the
router runs the built-ins as before. A router that predates this ignores the
document, so the page and the API show a state the router isn't yet running.
## Credentials and grants ## Credentials and grants
| Principal | Grant | | Principal | Grant |
| --------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | --------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `swarm-notify-router`, in its container | store: `read` on `swarm/services/<router client id>/oidc/client`. Queue: its own consumer on `SWARM_EVENTS` (create, pull, ack), publish on `$SWARM.router.todo.*`, create `ROUTER_TODOS`, its own `notify-router-state` bucket, and `STREAM.INFO` on the agent-status bucket, which lists the agents | | `swarm-notify-router`, in its container | store: `read` on `swarm/services/<router client id>/oidc/client`. Queue: its own consumer on `SWARM_EVENTS` (create, pull, ack), publish on `$SWARM.router.todo.*`, create `ROUTER_TODOS`, its own `notify-router-state` bucket, read on the `notify-flows` bucket, and `STREAM.INFO` on the agent-status bucket, which lists the agents |
| swarm-controller | publish on `$SWARM.events.forge.*` | | swarm-controller | publish on `$SWARM.events.forge.*`; creates and writes the `notify-flows` bucket (every bucket, by its `$KV.*.>` grant) |
| an agent, under its own credential | create its own consumer `todo-<agent>` in the form whose filter is `$SWARM.router.todo.<agent>`, pull from it, and ack what it delivered | | an agent, under its own credential | create its own consumer `todo-<agent>` in the form whose filter is `$SWARM.router.todo.<agent>`, pull from it, and ack what it delivered |
No other client may publish on `$SWARM.router.todo.*` or No other client may publish on `$SWARM.router.todo.*` or
`$SWARM.events.forge.*`, so each has one writer. The agent's create subject `$SWARM.events.forge.*`, so each has one writer. The agent's create subject

View file

@ -19,6 +19,7 @@ domain, covers host-level detail for one hive.
| `/schedules` | the scheduled prompts: add, edit, pause, resume and remove | | `/schedules` | the scheduled prompts: add, edit, pause, resume and remove |
| `/broadcast` | send one message to several agents at once, picked from a grid of their icons | | `/broadcast` | send one message to several agents at once, picked from a grid of their icons |
| `/messages` | the swarm-wide message feed: every message between agents, and to them from the operator, reminders and compaction, with a history window and a filter on sender or recipient | | `/messages` | the swarm-wide message feed: every message between agents, and to them from the operator, reminders and compaction, with a history window and a filter on sender or recipient |
| `/notifications` | the notification router's flows: a switch for each built-in, and every flow drawn as a tree |
| `/issues` | a cross-repo issue report | | `/issues` | a cross-repo issue report |
| `/review` | the review classifier: its recent runs, a try-out on a real PR, and its settings | | `/review` | the review classifier: its recent runs, a try-out on a real PR, and its settings |
| `/builds` | the swarm builder's recent builds of agent config PRs, with what nix printed when one failed | | `/builds` | the swarm builder's recent builds of agent config PRs, with what nix printed when one failed |
@ -216,6 +217,22 @@ Only an operator can change a schedule here or with `swarmctl schedule`; an
agent has no way to add one. The page needs the controller's `/api/schedules` agent has no way to add one. The page needs the controller's `/api/schedules`
routes. routes.
### Notification flows
`/notifications` lists the [notification router's flows](notify-router.md#flows)
in two groups. Each **built-in** flow has an on/off switch; switching one off
stops the router running it from its next event, and switching it on again
restores it. Each **custom** flow shows its name and whether it's on. Every
flow draws as an indented tree: the forge events it starts from, the
conditions under it, and the todo each branch ends in.
The page edits nothing else. Add custom flows with `PUT /api/notify/flows`.
A switch sends the whole flows document with the revision the page read, `0` if
it read no document. The controller refuses the switch when the stored revision
is another, including when someone wrote a document after the page read none, and
the page shows why. Reload to switch against the current document.
### Sending a message to an agent ### Sending a message to an agent
Under an agent's terminal, in the preview and in its own tab, is a **message** Under an agent's terminal, in the preview and in its own tab, is a **message**

View file

@ -8,6 +8,7 @@ import { AlertsPage } from "./pages/AlertsPage.js";
import { BuildsPage } from "./pages/builds/BuildsPage.js"; import { BuildsPage } from "./pages/builds/BuildsPage.js";
import { BroadcastPage } from "./pages/broadcast/BroadcastPage.js"; import { BroadcastPage } from "./pages/broadcast/BroadcastPage.js";
import { MessagesPage } from "./pages/messages/MessagesPage.js"; import { MessagesPage } from "./pages/messages/MessagesPage.js";
import { NotificationsPage } from "./pages/notifications/NotificationsPage.js";
import { ComponentsPage } from "./pages/ComponentsPage.js"; import { ComponentsPage } from "./pages/ComponentsPage.js";
import { JobsPage } from "./pages/JobsPage.js"; import { JobsPage } from "./pages/JobsPage.js";
import { HivesPage } from "./pages/HivesPage.js"; import { HivesPage } from "./pages/HivesPage.js";
@ -43,6 +44,7 @@ export function App() {
<Route path="/builds" component={BuildsPage} /> <Route path="/builds" component={BuildsPage} />
<Route path="/broadcast" component={BroadcastPage} /> <Route path="/broadcast" component={BroadcastPage} />
<Route path="/messages" component={MessagesPage} /> <Route path="/messages" component={MessagesPage} />
<Route path="/notifications" component={NotificationsPage} />
<Route path="/issues" component={IssueReportPage} /> <Route path="/issues" component={IssueReportPage} />
<Route path="/review" component={ReviewPage} /> <Route path="/review" component={ReviewPage} />
<Route path="/components" component={ComponentsPage} /> <Route path="/components" component={ComponentsPage} />

View file

@ -0,0 +1,95 @@
// Wire types of swarm-controller's `GET/PUT /api/notify/flows` and the small
// pure helpers the notifications page draws a flow with.
export type Recipients = { field: string } | { agent: string };
export interface Condition {
field: string;
equals: string;
}
// A node as the API sends it: `kind` sits beside `id`.
export type FlowNode =
| { id: string; kind: "source"; events: string[] }
| { id: string; kind: "match"; when: Condition[] }
| {
id: string;
kind: "todo";
to: Recipients;
op: "upsert" | "clear";
key: string;
prefix: boolean;
text: string;
};
export interface Flow {
id: string;
name: string;
enabled: boolean;
nodes: FlowNode[];
edges: { from: string; to: string }[];
}
export interface FlowsView {
revision: number;
disabled_builtins: string[];
builtins: Flow[];
flows: Flow[];
}
// One node of a flow with its depth below the source, in the order a reader
// walks the tree: a parent, then each child's subtree in edge order.
export interface TreeRow {
node: FlowNode;
depth: number;
}
export function treeRows(flow: Flow): TreeRow[] {
const byId = new Map(flow.nodes.map((n) => [n.id, n]));
const root = flow.nodes.find((n) => n.kind === "source");
if (!root) return [];
const rows: TreeRow[] = [];
const seen = new Set<string>();
const walk = (id: string, depth: number) => {
const node = byId.get(id);
// The controller refuses a flow that is not a tree; a repeat here would
// mean a cycle, which `seen` stops from looping.
if (!node || seen.has(id)) return;
seen.add(id);
rows.push({ node, depth });
for (const e of flow.edges) if (e.from === id) walk(e.to, depth + 1);
};
walk(root.id, 0);
return rows;
}
export function describeRecipients(to: Recipients): string {
return "field" in to ? `{${to.field}}` : to.agent;
}
// One line saying what a node is and does.
export function describeNode(node: FlowNode): string {
switch (node.kind) {
case "source":
return `forge ${node.events.join(", ")}`;
case "match":
return node.when.length === 0
? "match anything"
: node.when.map((c) => `${c.field} = ${c.equals}`).join(" and ");
case "todo":
return node.op === "clear"
? `clear todos ${node.key}${node.prefix ? "*" : ""} of ${describeRecipients(node.to)}`
: `todo to ${describeRecipients(node.to)}, key ${node.key}: ${node.text}`;
}
}
// `disabled` with built-in `id` switched on or off, without repeats and in a
// stable order.
export function withBuiltin(
disabled: string[],
id: string,
enabled: boolean,
): string[] {
const rest = disabled.filter((d) => d !== id);
return (enabled ? rest : [...rest, id]).sort();
}

View file

@ -0,0 +1,46 @@
/* NotificationsPage — the flows drawn as indented trees. */
.notify-flows {
display: flex;
flex-direction: column;
gap: 0.75em;
}
.notify-flow-head {
display: flex;
flex-wrap: wrap;
align-items: center;
justify-content: space-between;
gap: 0.5em;
}
.notify-flow-name {
font-weight: 600;
}
.notify-flow-tree {
margin: 0.4em 0 0;
padding: 0;
list-style: none;
font-size: 0.9em;
}
.notify-node {
display: flex;
gap: 0.5em;
padding: 0.1em 0;
}
.notify-node-kind {
flex: none;
min-width: 3.5em;
color: var(--muted);
}
.notify-node-text {
overflow-wrap: anywhere;
}
.notify-off {
opacity: 0.6;
}

View file

@ -0,0 +1,144 @@
// <NotificationsPage> — the notification router's flows. Backed by
// swarm-controller's `GET/PUT /api/notify/flows`; the controller is the only
// writer of the stored document and the router reads it every few seconds.
//
// Read-only except for one switch per built-in flow. A flow draws as an
// indented tree, the shape the API validates it to. Custom flows are not
// editable here: they arrive through the API.
import { useState } from "preact/hooks";
import { ApiErrorPanel } from "@hive/shared/api-error-panel.js";
import { readApiError, type ProblemDetails } from "@hive/shared/api-error.js";
import { Card } from "../../ui/card/Card.js";
import { CheckboxField } from "../../ui/checkbox-field/CheckboxField.js";
import { Panel } from "../../ui/panel/Panel.js";
import {
RefreshIntervalPicker,
type RefreshIntervalMs,
} from "../../ui/refresh-interval/RefreshInterval.js";
import { fetchJson, usePolled } from "../../ui/polled/usePolled.js";
import {
describeNode,
treeRows,
withBuiltin,
type Flow,
type FlowsView,
} from "./FlowTypes.js";
import "./NotificationsPage.css";
const DEFAULT_INTERVAL_MS: RefreshIntervalMs = 30_000;
function FlowTree({ flow }: { flow: Flow }) {
return (
<ul class="notify-flow-tree">
{treeRows(flow).map(({ node, depth }) => (
<li
class="notify-node"
key={node.id}
style={{ paddingLeft: `${depth * 1.5}em` }}
>
<span class="notify-node-kind">{node.kind}</span>
<span class="notify-node-text">{describeNode(node)}</span>
</li>
))}
</ul>
);
}
export function NotificationsPage() {
const [intervalMs, setIntervalMs] =
useState<RefreshIntervalMs>(DEFAULT_INTERVAL_MS);
const {
data: view,
error,
refresh,
} = usePolled(intervalMs, (signal) =>
fetchJson<FlowsView>("/api/notify/flows", signal),
);
const [pending, setPending] = useState(false);
const [actionError, setActionError] = useState<ProblemDetails | null>(null);
// Switch one built-in on or off: the whole document goes back with the
// revision this page read (0 for none), so a change made elsewhere since is refused
// rather than overwritten.
async function setBuiltin(id: string, enabled: boolean) {
if (!view) return;
setPending(true);
setActionError(null);
try {
const r = await fetch("/api/notify/flows", {
method: "PUT",
headers: { "content-type": "application/json" },
body: JSON.stringify({
revision: view.revision,
disabled_builtins: withBuiltin(view.disabled_builtins, id, enabled),
flows: view.flows,
}),
});
if (!r.ok) setActionError(await readApiError(r));
} catch (err) {
setActionError({ detail: String(err) });
} finally {
setPending(false);
refresh();
}
}
return (
<Panel
title="notifications"
icon="🔔"
actions={
<RefreshIntervalPicker
id="notifications-refresh"
value={intervalMs}
onChange={setIntervalMs}
/>
}
>
{error ? (
<ApiErrorPanel context="failed to load the flows" problem={error} />
) : null}
{actionError ? (
<ApiErrorPanel
context="the flows were not changed"
problem={actionError}
/>
) : null}
{!error && view === null ? <p>loading…</p> : null}
{view ? (
<div class="notify-flows">
<h3>built-in flows</h3>
{view.builtins.map((flow) => {
const on = !view.disabled_builtins.includes(flow.id);
return (
<Card key={flow.id} class={on ? "" : "notify-off"}>
<div class="notify-flow-head">
<span class="notify-flow-name">{flow.name}</span>
<CheckboxField
id={`builtin-${flow.id}`}
label={on ? "on" : "off"}
checked={on}
disabled={pending}
onChange={(checked) => setBuiltin(flow.id, checked)}
/>
</div>
<FlowTree flow={flow} />
</Card>
);
})}
<h3>custom flows</h3>
{view.flows.length === 0 ? <p>no custom flows</p> : null}
{view.flows.map((flow) => (
<Card key={flow.id} class={flow.enabled ? "" : "notify-off"}>
<div class="notify-flow-head">
<span class="notify-flow-name">{flow.name}</span>
<span>{flow.enabled ? "on" : "off"}</span>
</div>
<FlowTree flow={flow} />
</Card>
))}
</div>
) : null}
</Panel>
);
}

View file

@ -56,6 +56,7 @@ const NAV_ITEMS: { href: string; label: string; accent: string }[] = [
{ href: "/schedules", label: "schedules", accent: "var(--cyan)" }, { href: "/schedules", label: "schedules", accent: "var(--cyan)" },
{ href: "/broadcast", label: "broadcast", accent: "var(--blue)" }, { href: "/broadcast", label: "broadcast", accent: "var(--blue)" },
{ href: "/messages", label: "messages", accent: "var(--purple)" }, { href: "/messages", label: "messages", accent: "var(--purple)" },
{ href: "/notifications", label: "notifications", accent: "var(--yellow)" },
{ href: "/issues", label: "issues", accent: "var(--yellow)" }, { href: "/issues", label: "issues", accent: "var(--yellow)" },
{ href: "/review", label: "review", accent: "var(--amber)" }, { href: "/review", label: "review", accent: "var(--amber)" },
{ href: "/builds", label: "builds", accent: "var(--green)" }, { href: "/builds", label: "builds", accent: "var(--green)" },

View file

@ -0,0 +1,80 @@
// The notifications page's tree and wording helpers. Run by `npm test`.
import assert from "node:assert/strict";
import { test } from "node:test";
import {
describeNode,
treeRows,
withBuiltin,
} from "../src/pages/notifications/FlowTypes.ts";
const flow = {
id: "f",
name: "f",
enabled: true,
nodes: [
{ id: "s", kind: "source", events: ["issues", "issue_assign"] },
{ id: "m", kind: "match", when: [{ field: "action", equals: "closed" }] },
{ id: "m2", kind: "match", when: [] },
{
id: "t",
kind: "todo",
to: { field: "todo_holders" },
op: "clear",
key: "forge:{repo}#{number}:",
prefix: true,
text: "",
},
{
id: "u",
kind: "todo",
to: { agent: "triage" },
op: "upsert",
key: "k",
prefix: false,
text: "look",
},
],
edges: [
{ from: "s", to: "m" },
{ from: "m", to: "t" },
{ from: "s", to: "m2" },
{ from: "m2", to: "u" },
],
};
test("treeRows: each subtree follows its parent, with its depth", () => {
assert.deepEqual(
treeRows(flow).map((r) => [r.node.id, r.depth]),
[
["s", 0],
["m", 1],
["t", 2],
["m2", 1],
["u", 2],
],
);
});
test("treeRows: a cycle ends the walk and a flow with no source is empty", () => {
const cyclic = { ...flow, edges: [...flow.edges, { from: "t", to: "s" }] };
assert.equal(treeRows(cyclic).length, 5);
assert.deepEqual(treeRows({ ...flow, nodes: [] }), []);
});
test("describeNode: says what each kind does", () => {
assert.equal(describeNode(flow.nodes[0]), "forge issues, issue_assign");
assert.equal(describeNode(flow.nodes[1]), "action = closed");
assert.equal(describeNode(flow.nodes[2]), "match anything");
assert.equal(
describeNode(flow.nodes[3]),
"clear todos forge:{repo}#{number}:* of {todo_holders}",
);
assert.equal(describeNode(flow.nodes[4]), "todo to triage, key k: look");
});
test("withBuiltin: switching off adds once, switching on removes", () => {
assert.deepEqual(withBuiltin([], "b", false), ["b"]);
assert.deepEqual(withBuiltin(["b"], "b", false), ["b"]);
assert.deepEqual(withBuiltin(["c"], "a", false), ["a", "c"]);
assert.deepEqual(withBuiltin(["a", "b"], "a", true), ["b"]);
});

View file

@ -82,6 +82,11 @@ The swarm's control plane. The swarm UI and `swarmctl` are its clients.
`review-classifier-settings` KV bucket, which this daemon alone reads and `review-classifier-settings` KV bucket, which this daemon alone reads and
writes (`GET`/`PUT /api/review-classifier/settings`) → writes (`GET`/`PUT /api/review-classifier/settings`) →
[`docs/swarm/review-classifier.md`](../docs/swarm/review-classifier.md#swarm-level-settings). [`docs/swarm/review-classifier.md`](../docs/swarm/review-classifier.md#swarm-level-settings).
- **Notification flows** — which of the notification router's built-in flows are
switched off, and the custom flows, as one document in the `notify-flows`
KV bucket, which this daemon alone writes (`GET`/`PUT /api/notify/flows`);
the router reads it →
[`docs/swarm/notify-router.md`](../docs/swarm/notify-router.md#flows).
- **Operator profiles** — each operator's name and the pronouns on their - **Operator profiles** — each operator's name and the pronouns on their
forge profile, for every member of authelia's operator group, as one list forge profile, for every member of authelia's operator group, as one list
under the one key of the `operator-profiles` KV bucket. This daemon alone under the one key of the `operator-profiles` KV bucket. This daemon alone

View file

@ -2113,6 +2113,14 @@ mod tests {
events(DeliveryKind::PullRequest), events(DeliveryKind::PullRequest),
["pull_request", "pull_request_assign"] ["pull_request", "pull_request_assign"]
); );
// The events a stored flow's source may name are the ones the hooks
// deliver to the router.
let mut routed = events(DeliveryKind::Issues);
routed.extend(events(DeliveryKind::PullRequest));
routed.sort_unstable();
let mut sources = swarm_wire::notify_flows::SOURCE_EVENTS.to_vec();
sources.sort_unstable();
assert_eq!(routed, sources);
} }
#[test] #[test]

View file

@ -50,6 +50,7 @@ mod jobq;
mod linked_accounts; mod linked_accounts;
mod matrix_account; mod matrix_account;
mod message_stream; mod message_stream;
mod notify_flows;
mod otel_http_client; mod otel_http_client;
mod pulls; mod pulls;
mod queue_identity; mod queue_identity;

View file

@ -0,0 +1,446 @@
//! `GET` and `PUT /api/notify/flows`: which of the notification router's
//! built-in flows are switched off, and the custom flows.
//!
//! The flows are one document under one key ([`notify_flows::KEY`] in
//! [`notify_flows::BUCKET`]), which only this daemon writes, as the schedule
//! definitions are. A `PUT` replaces the whole document in one
//! compare-and-set, after [`Document::check`]. The router reads the key
//! every few seconds, so a change applies to the next event with no restart.
//! No document means the built-in flows alone.
//!
//! The built-in flows are defined in code, in `swarm_wire::notify_flows`, and
//! the document stores only their disabled ids, so a `GET` shows the ones
//! this build knows.
use anyhow::Context as _;
use async_nats::jetstream::kv::Store;
use axum::Json;
use axum::extract::State;
use axum::extract::rejection::JsonRejection;
use axum::http::StatusCode;
use problem_details::ProblemDetails;
use serde::{Deserialize, Serialize};
use swarm_wire::notify_flows::{self, DecodeError, Document, Flow};
use utoipa::ToSchema;
use crate::routes::{AppState, error_problem, wanted_error_status};
use crate::schedules::{queue, read_with, refused, write_revision};
/// How many times a losing writer re-reads and re-applies, as in
/// `wanted::MAX_ATTEMPTS`.
const MAX_ATTEMPTS: usize = 5;
/// What `GET` and a successful `PUT` answer.
#[derive(Debug, Clone, PartialEq, Serialize, ToSchema)]
pub(crate) struct FlowsView {
/// The revision of the stored document, to send back in a `PUT`; `0` when
/// none is stored.
revision: u64,
/// The ids of the built-in flows that are switched off.
disabled_builtins: Vec<String>,
/// The built-in flows this build ships, all of them, switched off or not.
builtins: Vec<Flow>,
/// The custom flows.
flows: Vec<Flow>,
}
/// The body of a `PUT`: the whole document.
#[derive(Debug, Clone, Deserialize, ToSchema)]
#[serde(deny_unknown_fields)]
pub(crate) struct PutFlows {
/// The `revision` the client read, `0` for "no document stored". The
/// `PUT` is refused when the stored revision is another, which includes a
/// document written since a client read `0`. A KV revision is never `0`.
revision: u64,
/// The ids of the built-in flows to switch off.
#[serde(default)]
disabled_builtins: Vec<String>,
/// The custom flows.
#[serde(default)]
flows: Vec<Flow>,
}
impl FlowsView {
fn of(doc: Document, revision: u64) -> Self {
Self {
revision,
disabled_builtins: doc.disabled_builtins,
builtins: notify_flows::builtins(),
flows: doc.flows,
}
}
}
/// What the key holds.
enum Stored {
/// No key, or a deleted one.
Nothing,
Document(Document),
/// A document this build cannot read, with why.
Unreadable(DecodeError),
}
/// The stored document, with the revision to write against.
async fn stored(store: &Store) -> anyhow::Result<(Stored, Option<u64>)> {
struct Read(Stored);
impl Default for Read {
fn default() -> Self {
Self(Stored::Nothing)
}
}
let (Read(stored), revision) = read_with(store, notify_flows::KEY, |bytes| {
Ok(Read(match Document::decode(bytes) {
Ok(doc) => Stored::Document(doc),
Err(e) => Stored::Unreadable(e),
}))
})
.await?;
Ok((stored, revision))
}
/// Open [`notify_flows::BUCKET`], creating it if nothing has yet.
async fn open(client: &async_nats::Client) -> anyhow::Result<Store> {
let js = async_nats::jetstream::new(client.clone());
if let Ok(store) = js.get_key_value(notify_flows::BUCKET).await {
return Ok(store);
}
js.create_key_value(async_nats::jetstream::kv::Config {
bucket: notify_flows::BUCKET.to_owned(),
description: "The notification router's flows: disabled built-ins and custom flows"
.to_owned(),
history: 1,
..Default::default()
})
.await
.with_context(|| format!("creating the {} bucket", notify_flows::BUCKET))
}
fn queue_problem(e: &anyhow::Error) -> ProblemDetails {
tracing::warn!(error = %format!("{e:#}"), "notify flows: queue failure");
error_problem(wanted_error_status(e), &format!("{e:#}"))
}
/// Read the flows. Creates the `notify-flows` bucket when it is missing, as a
/// `PUT` does, so the router finds it empty rather than absent.
#[utoipa::path(
get,
path = "/api/notify/flows",
responses(
(status = 200, description = "the stored document, with the built-in flows this \
build ships; with none stored, no disabled built-ins and no custom flows",
body = FlowsView),
(status = 409, description = "the stored document is not one this build reads, \
such as a newer build's version (problem+json)", body = String),
(status = 503, description = "no swarm queue is configured on this host, or it is \
not connected (problem+json)", body = String),
(status = 500, description = "the flows could not be read (problem+json)",
body = String),
),
tag = "notify"
)]
pub(crate) async fn get_flows(
State(state): State<AppState>,
) -> Result<Json<FlowsView>, ProblemDetails> {
let client = queue(&state).map_err(refused)?;
let store = open(&client).await.map_err(|e| queue_problem(&e))?;
let (stored, revision) = stored(&store).await.map_err(|e| queue_problem(&e))?;
match stored {
Stored::Nothing => Ok(Json(FlowsView::of(
Document::default(),
revision.unwrap_or(0),
))),
Stored::Document(doc) => Ok(Json(FlowsView::of(doc, revision.unwrap_or(0)))),
Stored::Unreadable(e) => Err(unreadable(&e)),
}
}
fn unreadable(why: &DecodeError) -> ProblemDetails {
error_problem(
StatusCode::CONFLICT,
&format!("the stored flows document is not one this build reads: {why}"),
)
}
/// Replace the flows.
#[utoipa::path(
put,
path = "/api/notify/flows",
request_body = PutFlows,
responses(
(status = 200, description = "the document as stored; the router applies it from \
its next event", body = FlowsView),
(status = 400, description = "not JSON or not this shape, or the document fails its \
check: a `disabled_builtins` entry that is no built-in, a custom flow whose id \
is another's or a built-in's, a flow that is not a tree (one `source`, one \
parent for every other node, every node reachable, no children under a \
`todo`), more than 64 nodes, 16 levels or 100 flows, an id that is not 1 to 64 \
letters, digits, `-` or `_`, a source event the router is not sent, a condition \
or recipient field that is no event field, or an upsert with no text \
(problem+json)", body = String),
(status = 409, description = "the stored document is not at `revision` (another write landed, or one now exists where the client read none), or the stored \
document is a version this build does not read, which a `PUT` would erase \
(problem+json)", body = String),
(status = 503, description = "no swarm queue is configured on this host, or it is \
not connected (problem+json)", body = String),
(status = 500, description = "the flows could not be written (problem+json)",
body = String),
),
tag = "notify"
)]
pub(crate) async fn put_flows(
State(state): State<AppState>,
body: Result<Json<PutFlows>, JsonRejection>,
) -> Result<Json<FlowsView>, ProblemDetails> {
let Json(put) = body.map_err(|e| error_problem(StatusCode::BAD_REQUEST, &e.body_text()))?;
let doc = Document {
version: notify_flows::VERSION,
disabled_builtins: put.disabled_builtins,
flows: put.flows,
};
doc.check()
.map_err(|why| error_problem(StatusCode::BAD_REQUEST, &why))?;
let client = queue(&state).map_err(refused)?;
let store = open(&client).await.map_err(|e| queue_problem(&e))?;
for _ in 0..MAX_ATTEMPTS {
let (current, revision) = stored(&store).await.map_err(|e| queue_problem(&e))?;
// An unreadable document is replaced only when it is not a version
// this build does not know: a newer build's document is kept for it.
if let Stored::Unreadable(why @ DecodeError::UnknownVersion(_)) = &current {
return Err(unreadable(why));
}
if put.revision != revision.unwrap_or(0) {
return Err(error_problem(
StatusCode::CONFLICT,
&format!(
"the flows are not at revision {}; read them again",
put.revision
),
));
}
if let Some(written) = write_revision(&store, notify_flows::KEY, &doc, revision)
.await
.map_err(|e| queue_problem(&e))?
{
tracing::info!(
disabled = ?doc.disabled_builtins,
custom = doc.flows.len(),
"set the notification router's flows"
);
return Ok(Json(FlowsView::of(doc, written)));
}
}
Err(queue_problem(&anyhow::anyhow!(
"gave up writing the notification flows after {MAX_ATTEMPTS} conflicting writes"
)))
}
#[cfg(test)]
mod tests {
use swarm_wire::notify_flows::{Edge, Node, NodeKind, Recipients, TodoSink};
use swarm_wire::router_todo::Op;
use super::*;
use crate::test_nats::TestNats;
fn custom(id: &str) -> Flow {
Flow {
id: id.to_owned(),
name: "new to triage".to_owned(),
enabled: true,
nodes: vec![
Node {
id: "source".to_owned(),
kind: NodeKind::Source {
events: vec!["issues".to_owned()],
},
},
Node {
id: "sink".to_owned(),
kind: NodeKind::Todo(TodoSink {
to: Recipients::Agent("triage".to_owned()),
op: Op::Upsert,
key: "forge:{repo}#{number}:triage".to_owned(),
prefix: false,
text: "triage {repo}#{number}".to_owned(),
}),
},
],
edges: vec![Edge {
from: "source".to_owned(),
to: "sink".to_owned(),
}],
}
}
fn body(revision: u64, disabled: &[&str], flows: Vec<Flow>) -> Json<PutFlows> {
Json(PutFlows {
revision,
disabled_builtins: disabled.iter().map(|s| (*s).to_owned()).collect(),
flows,
})
}
#[tokio::test]
async fn the_flows_are_read_replaced_and_checked_on_save() {
let nats = TestNats::start();
let client = nats.client().await;
let state = crate::agent_limits::tests::state(&client);
// Nothing stored: the built-ins alone, none off, no revision.
let Json(view) = get_flows(State(state.clone())).await.expect("get");
assert_eq!(view.revision, 0);
assert!(view.disabled_builtins.is_empty() && view.flows.is_empty());
assert_eq!(view.builtins.len(), 3);
let Json(saved) = put_flows(
State(state.clone()),
Ok(body(0, &["unassigned"], vec![custom("triage")])),
)
.await
.expect("put");
let revision = saved.revision;
assert_ne!(revision, 0);
let Json(read) = get_flows(State(state.clone())).await.expect("get");
assert_eq!(read, saved);
assert_eq!(read.disabled_builtins, ["unassigned"]);
assert_eq!(read.flows[0].id, "triage");
// The stored bytes are the document the router decodes.
let store = open(&client).await.expect("bucket");
let bytes = store
.get(notify_flows::KEY)
.await
.expect("read")
.expect("stored");
let doc = Document::decode(&bytes).expect("the router reads it");
assert_eq!(doc.active().0.len(), 3, "2 built-ins and the custom flow");
// A refused save changes nothing: an orphan node, and a built-in that does not exist.
let mut orphan = custom("orphan");
orphan.edges.clear();
let refused = put_flows(State(state.clone()), Ok(body(revision, &[], vec![orphan])))
.await
.expect_err("an orphan node");
assert_eq!(refused.status, Some(StatusCode::BAD_REQUEST), "{refused:?}");
let unknown = put_flows(State(state.clone()), Ok(body(revision, &["nope"], vec![])))
.await
.expect_err("no such built-in");
assert_eq!(unknown.status, Some(StatusCode::BAD_REQUEST), "{unknown:?}");
let Json(after) = get_flows(State(state.clone())).await.expect("get");
assert_eq!(after, saved);
// A stale revision is refused; the current one lands.
let outdated = put_flows(State(state.clone()), Ok(body(revision + 100, &[], vec![])))
.await
.expect_err("stale");
assert_eq!(outdated.status, Some(StatusCode::CONFLICT), "{outdated:?}");
let Json(next) = put_flows(State(state.clone()), Ok(body(revision, &[], vec![])))
.await
.expect("current revision");
assert!(next.revision > revision);
assert!(next.flows.is_empty());
}
/// A document of a version this build does not know is neither served
/// nor replaced, and a garbled one is replaced.
#[tokio::test]
async fn a_document_of_another_version_is_kept_and_a_garbled_one_is_replaced() {
let nats = TestNats::start();
let client = nats.client().await;
let state = crate::agent_limits::tests::state(&client);
let store = open(&client).await.expect("bucket");
store
.put(
notify_flows::KEY,
br#"{"version":2,"flows":[]}"#.to_vec().into(),
)
.await
.expect("seed");
let got = get_flows(State(state.clone()))
.await
.expect_err("version 2");
assert_eq!(got.status, Some(StatusCode::CONFLICT), "{got:?}");
let refused = put_flows(State(state.clone()), Ok(body(0, &[], vec![])))
.await
.expect_err("version 2 is kept");
assert_eq!(refused.status, Some(StatusCode::CONFLICT), "{refused:?}");
let kept = store
.get(notify_flows::KEY)
.await
.expect("read")
.expect("stored");
assert_eq!(&kept[..], br#"{"version":2,"flows":[]}"#);
// Control: garbage that names no version is replaced by a save.
let garbage = store
.put(notify_flows::KEY, b"garbage".to_vec().into())
.await
.expect("seed");
let _ = put_flows(State(state.clone()), Ok(body(garbage, &["closed"], vec![])))
.await
.expect("replaces garbage");
let Json(view) = get_flows(State(state)).await.expect("readable again");
assert_eq!(view.disabled_builtins, ["closed"]);
}
/// A page that read no document cannot erase one written since: its
/// `revision: 0` means "create", and the document exists by then.
#[tokio::test]
async fn a_page_that_read_nothing_cannot_erase_a_flow_written_after() {
let nats = TestNats::start();
let client = nats.client().await;
let state = crate::agent_limits::tests::state(&client);
let Json(page) = get_flows(State(state.clone())).await.expect("get");
assert_eq!(page.revision, 0);
// The API writes a custom flow while the page is open.
let Json(api) = put_flows(
State(state.clone()),
Ok(body(0, &[], vec![custom("triage")])),
)
.await
.expect("the API's write");
// The page's first switch carries what it read.
let erased = put_flows(
State(state.clone()),
Ok(body(page.revision, &["closed"], vec![])),
)
.await
.expect_err("the page read nothing, and a document exists");
assert_eq!(erased.status, Some(StatusCode::CONFLICT), "{erased:?}");
let Json(after) = get_flows(State(state.clone())).await.expect("get");
assert_eq!(after, api);
assert_eq!(after.flows[0].id, "triage");
// Control: re-read, the same switch lands and keeps the flow.
let Json(switched) = put_flows(
State(state),
Ok(body(after.revision, &["closed"], after.flows.clone())),
)
.await
.expect("the switch with the revision it read");
assert_eq!(switched.disabled_builtins, ["closed"]);
assert_eq!(switched.flows[0].id, "triage");
}
#[tokio::test]
async fn a_template_naming_no_event_field_is_refused_on_save() {
let nats = TestNats::start();
let client = nats.client().await;
let state = crate::agent_limits::tests::state(&client);
let mut typo = custom("typo");
if let NodeKind::Todo(sink) = &mut typo.nodes[1].kind {
sink.key = "forge:{repo}#{numbr}:triage".to_owned();
}
let refused = put_flows(State(state.clone()), Ok(body(0, &[], vec![typo])))
.await
.expect_err("{numbr} is no field");
assert_eq!(refused.status, Some(StatusCode::BAD_REQUEST), "{refused:?}");
// Control: the same flow spelled right is stored.
let _ = put_flows(State(state), Ok(body(0, &[], vec![custom("typo")])))
.await
.expect("{number} is a field");
}
}

View file

@ -31,8 +31,8 @@ use crate::{
agent_control, agent_icon, agent_input, agent_limits, agent_login, agent_permissions, agent_control, agent_icon, agent_input, agent_limits, agent_login, agent_permissions,
agent_renewal, agent_state_stream, agent_status, agents_md, alerts, archetypes, auth, builder, agent_renewal, agent_state_stream, agent_status, agents_md, alerts, archetypes, auth, builder,
claude_login, config_pr, forge, forge_account, github_account, inference, issue_report, claude_login, config_pr, forge, forge_account, github_account, inference, issue_report,
linked_accounts, matrix_account, message_stream, pulls, queue_identity, review_classifier, linked_accounts, matrix_account, message_stream, notify_flows, pulls, queue_identity,
schedules, status, subagent_term, term_stream, wanted, webhook, review_classifier, schedules, status, subagent_term, term_stream, wanted, webhook,
}; };
/// Where the daemon binds, overridable via `SWARM_CONTROLLER_SOCKET`. /// Where the daemon binds, overridable via `SWARM_CONTROLLER_SOCKET`.
@ -76,6 +76,7 @@ pub(crate) fn socket_path() -> PathBuf {
(name = "repos", description = "forge repo browsing (swarm-ui's issue-report page)"), (name = "repos", description = "forge repo browsing (swarm-ui's issue-report page)"),
(name = "forge", description = "forge site admins (`swarmctl forge make-admin`)"), (name = "forge", description = "forge site admins (`swarmctl forge make-admin`)"),
(name = "schedules", description = "scheduled prompts (`swarmctl schedule`)"), (name = "schedules", description = "scheduled prompts (`swarmctl schedule`)"),
(name = "notify", description = "the notification router's flows: built-ins switched off, and custom flows"),
(name = "review-classifier", description = "trying review classifications against a PR (swarm-ui's try-out page), and the classifier's swarm-wide settings"), (name = "review-classifier", description = "trying review classifications against a PR (swarm-ui's try-out page), and the classifier's swarm-wide settings"),
(name = "pulls", description = "open pull requests, for the try-out page's dropdowns"), (name = "pulls", description = "open pull requests, for the try-out page's dropdowns"),
(name = "alerts", description = "the swarm Grafana's current alerts (swarm-ui's alerts page)"), (name = "alerts", description = "the swarm Grafana's current alerts (swarm-ui's alerts page)"),
@ -635,6 +636,7 @@ pub(crate) fn build_app(state: AppState) -> axum::Router {
review_classifier::settings::get_settings, review_classifier::settings::get_settings,
review_classifier::settings::put_settings review_classifier::settings::put_settings
)) ))
.routes(routes!(notify_flows::get_flows, notify_flows::put_flows))
.routes(routes!(webhook::post_webhook_forge)) .routes(routes!(webhook::post_webhook_forge))
.split_for_parts(); .split_for_parts();
// Just the JSON, not the UI — Swagger UI itself is nginx-hosted from // Just the JSON, not the UI — Swagger UI itself is nginx-hosted from

View file

@ -317,7 +317,7 @@ impl Policy {
/// What the notification router may do: drain its own consumer on the /// What the notification router may do: drain its own consumer on the
/// forge events stream, queue todo ops for any agent, keep its own state /// forge events stream, queue todo ops for any agent, keep its own state
/// bucket, and list which agents exist. /// bucket, read the flows document, and list which agents exist.
/// ///
/// - **Forge events.** Open or create the stream, create its one durable /// - **Forge events.** Open or create the stream, create its one durable
/// in the extended form whose last tokens are the filter (the server /// in the extended form whose last tokens are the filter (the server
@ -330,14 +330,18 @@ impl Policy {
/// subjects, which is what makes the router their sole writer. /// subjects, which is what makes the router their sole writer.
/// - **Its state bucket.** Create, look up, read and write /// - **Its state bucket.** Create, look up, read and write
/// `notify-router-state`, and no other bucket. /// `notify-router-state`, and no other bucket.
/// - **The flows document.** `STREAM.INFO` and `DIRECT.GET` on
/// `notify-flows`, which it reads and never creates or writes: the
/// controller creates the bucket and is its one writer.
/// - **The roster.** `STREAM.INFO` on the agent-status bucket, which lists /// - **The roster.** `STREAM.INFO` on the agent-status bucket, which lists
/// its keys (`<hive>.<agent>`) and writes none. /// its keys (`<hive>.<agent>`) and writes none.
fn notify_router_permissions(client_id: &str) -> Permissions { fn notify_router_permissions(client_id: &str) -> Permissions {
use swarm_wire::{forge_events, notify_router, router_todo}; use swarm_wire::{forge_events, notify_flows, notify_router, router_todo};
let events = forge_events::STREAM; let events = forge_events::STREAM;
let consumer = forge_events::ROUTER_CONSUMER; let consumer = forge_events::ROUTER_CONSUMER;
let state = format!("KV_{}", notify_router::STATE_BUCKET); let state = format!("KV_{}", notify_router::STATE_BUCKET);
let flows = format!("KV_{}", notify_flows::BUCKET);
let mut publish = Self::jetstream_minimum().to_vec(); let mut publish = Self::jetstream_minimum().to_vec();
publish.extend([ publish.extend([
format!("$JS.API.STREAM.INFO.{events}"), format!("$JS.API.STREAM.INFO.{events}"),
@ -355,6 +359,8 @@ impl Policy {
format!("$JS.API.STREAM.CREATE.{state}"), format!("$JS.API.STREAM.CREATE.{state}"),
format!("$JS.API.DIRECT.GET.{state}.>"), format!("$JS.API.DIRECT.GET.{state}.>"),
format!("$KV.{}.>", notify_router::STATE_BUCKET), format!("$KV.{}.>", notify_router::STATE_BUCKET),
format!("$JS.API.STREAM.INFO.{flows}"),
format!("$JS.API.DIRECT.GET.{flows}.>"),
format!("$JS.API.STREAM.INFO.{}", Self::agent_status_stream()), format!("$JS.API.STREAM.INFO.{}", Self::agent_status_stream()),
Self::inbox(client_id), Self::inbox(client_id),
]); ]);
@ -1799,7 +1805,7 @@ mod tests {
#[test] #[test]
fn the_notify_router_drains_its_own_consumer_and_writes_only_its_own_state() { fn the_notify_router_drains_its_own_consumer_and_writes_only_its_own_state() {
use swarm_queue_client::client_inbox_prefix; use swarm_queue_client::client_inbox_prefix;
use swarm_wire::{forge_events, notify_router, router_todo}; use swarm_wire::{forge_events, notify_flows, notify_router, router_todo};
let p = with_router() let p = with_router()
.permissions(NOTIFY_ROUTER) .permissions(NOTIFY_ROUTER)
@ -1807,6 +1813,7 @@ mod tests {
let may = |subject: &str| p.publish.iter().any(|g| covers(g, subject)); let may = |subject: &str| p.publish.iter().any(|g| covers(g, subject));
let events = forge_events::STREAM; let events = forge_events::STREAM;
let state = notify_router::STATE_BUCKET; let state = notify_router::STATE_BUCKET;
let flows = notify_flows::BUCKET;
for allowed in [ for allowed in [
format!( format!(
"$JS.API.CONSUMER.CREATE.{events}.notify-router.{}", "$JS.API.CONSUMER.CREATE.{events}.notify-router.{}",
@ -1820,6 +1827,12 @@ mod tests {
format!("$KV.{state}.42.5571"), format!("$KV.{state}.42.5571"),
format!("$JS.API.DIRECT.GET.KV_{state}.$KV.{state}.42.5571"), format!("$JS.API.DIRECT.GET.KV_{state}.$KV.{state}.42.5571"),
"$JS.API.STREAM.INFO.KV_agent-status".to_owned(), "$JS.API.STREAM.INFO.KV_agent-status".to_owned(),
// The flows document: read it.
format!("$JS.API.STREAM.INFO.KV_{flows}"),
format!(
"$JS.API.DIRECT.GET.KV_{flows}.$KV.{flows}.{}",
notify_flows::KEY
),
] { ] {
assert!(may(&allowed), "the router may not reach {allowed}"); assert!(may(&allowed), "the router may not reach {allowed}");
} }
@ -1838,6 +1851,10 @@ mod tests {
forge_events::subject("issues"), forge_events::subject("issues"),
"$KV.agent-status.alpha.atlas".to_owned(), "$KV.agent-status.alpha.atlas".to_owned(),
"$KV.agent-schedules.atlas".to_owned(), "$KV.agent-schedules.atlas".to_owned(),
// The flows document: not write it, nor make the bucket.
format!("$KV.{flows}.{}", notify_flows::KEY),
format!("$JS.API.STREAM.CREATE.KV_{flows}"),
format!("$JS.API.STREAM.DELETE.KV_{flows}"),
swarm_queue_client::agent_input_subject("atlas"), swarm_queue_client::agent_input_subject("atlas"),
"$JS.API.STREAM.CREATE.KV_agent-status".to_owned(), "$JS.API.STREAM.CREATE.KV_agent-status".to_owned(),
] { ] {

View file

@ -163,6 +163,22 @@ mod tests {
} }
} }
/// The names a stored flow may read are the ones `fields` produces.
#[test]
fn the_fields_are_the_ones_a_stored_flow_may_name() {
let d = Delivery::parse(&issue_assign("assigned", &["atlas"])).expect("parses");
for event in ["issues", "issue_assign"] {
let mut got: Vec<&str> = fields(event, &d, &Snapshot::default())
.keys()
.copied()
.collect();
let mut want = swarm_wire::notify_flows::FIELDS.to_vec();
got.sort_unstable();
want.sort_unstable();
assert_eq!(got, want, "{event}");
}
}
#[test] #[test]
fn an_issue_delivery_reads_back_its_fields() { fn an_issue_delivery_reads_back_its_fields() {
let d = Delivery::parse(&issue_assign("assigned", &["atlas", "iris"])).expect("parses"); let d = Delivery::parse(&issue_assign("assigned", &["atlas", "iris"])).expect("parses");

View file

@ -1,26 +1,17 @@
//! Flows: what the router does with an event. //! Running a flow over one event.
//! //!
//! A flow is a graph of nodes joined by edges, rooted at one `source` node. //! The flow types, their checks and the built-in flows are
//! A `match` node passes when every one of its conditions holds, and then //! `swarm_wire::notify_flows`; the built-in and the custom flows are the same
//! each of its children runs, so one event can reach several branches. A //! type and this walk runs both. A `match` node passes when every one of its
//! `sink` node is a leaf and emits one todo op per recipient. //! conditions holds, and then each of its children runs, so one event can
//! //! reach several branches. A `todo` node emits one op per recipient. The
//! Flows are trees: one source, one parent for every other node, every node //! router runs only flows that pass [`Flow::validate`].
//! reachable from the source. [`Flow::validate`] checks that, and the router
//! runs only flows that pass.
//!
//! The built-in flows ([`builtins`]) are ordinary flows defined in code.
use std::collections::{BTreeMap, BTreeSet}; use std::collections::BTreeMap;
use swarm_wire::notify_flows::{Condition, Flow, Node, NodeKind, Recipients, TodoSink};
use swarm_wire::router_todo::{Op, RouterTodoOp}; use swarm_wire::router_todo::{Op, RouterTodoOp};
/// The most nodes one flow may have.
const MAX_NODES: usize = 64;
/// The deepest a node may sit below the source.
const MAX_DEPTH: usize = 16;
/// One event, as the fields a flow reads. /// One event, as the fields a flow reads.
pub type Fields = BTreeMap<&'static str, Value>; pub type Fields = BTreeMap<&'static str, Value>;
@ -31,56 +22,6 @@ pub enum Value {
List(Vec<String>), List(Vec<String>),
} }
#[derive(Clone, Debug)]
pub struct Flow {
pub id: String,
pub nodes: Vec<Node>,
pub edges: Vec<Edge>,
}
#[derive(Clone, Debug)]
pub struct Node {
pub id: String,
pub kind: NodeKind,
}
/// A directed edge, parent to child.
#[derive(Clone, Debug)]
pub struct Edge {
pub from: String,
pub to: String,
}
#[derive(Clone, Debug)]
pub enum NodeKind {
/// The root: the forge event types the flow starts from.
Source { events: Vec<String> },
/// Passes when every condition holds.
Match { when: Vec<Condition> },
/// Emits a todo op to each recipient.
Todo(TodoSink),
}
/// `field` equals `value`. A list field holds when one of its items does.
#[derive(Clone, Debug)]
pub struct Condition {
pub field: String,
pub equals: String,
}
#[derive(Clone, Debug)]
pub struct TodoSink {
/// The field naming the recipients, a list of agent names.
pub to: String,
pub op: Op,
/// The todo key, with `{field}` placeholders.
pub key: String,
/// For a clear: `key` is a prefix.
pub prefix: bool,
/// The todo text, with `{field}` placeholders. Unused for a clear.
pub text: String,
}
/// One op the walk emitted, for one recipient. /// One op the walk emitted, for one recipient.
#[derive(Clone, Debug, PartialEq, Eq)] #[derive(Clone, Debug, PartialEq, Eq)]
pub struct Action { pub struct Action {
@ -90,130 +31,72 @@ pub struct Action {
pub op: RouterTodoOp, pub op: RouterTodoOp,
} }
impl Flow { /// Walk `flow` for one event of type `event_type` and return the ops it
/// Check the flow is a tree within the caps, or say why not. /// emits. Assumes [`Flow::validate`] passed.
pub fn validate(&self) -> Result<(), String> { pub fn evaluate(flow: &Flow, event_type: &str, fields: &Fields) -> Vec<Action> {
if self.nodes.len() > MAX_NODES { let Some(root) = flow
return Err(format!("{} nodes, more than {MAX_NODES}", self.nodes.len())); .nodes
} .iter()
let mut ids = BTreeSet::new(); .find(|n| matches!(&n.kind, NodeKind::Source { events } if events.iter().any(|e| e == event_type)))
for node in &self.nodes { else {
if !ids.insert(node.id.as_str()) { return Vec::new();
return Err(format!("node id {} is used twice", node.id)); };
} let mut actions = Vec::new();
} let mut stack = vec![root.id.as_str()];
for edge in &self.edges { while let Some(id) = stack.pop() {
for end in [&edge.from, &edge.to] { let Some(node) = flow.node(id) else { continue };
if !ids.contains(end.as_str()) { match &node.kind {
return Err(format!("an edge names {end}, which is no node")); NodeKind::Source { .. } => stack.extend(flow.children(id)),
NodeKind::Match { when } => {
if when.iter().all(|c| holds(c, fields)) {
stack.extend(flow.children(id));
} }
} }
NodeKind::Todo(sink) => actions.extend(emit(flow, node, sink, fields)),
} }
let roots: Vec<&Node> = self
.nodes
.iter()
.filter(|n| matches!(n.kind, NodeKind::Source { .. }))
.collect();
let [root] = roots.as_slice() else {
return Err(format!("{} source nodes, not one", roots.len()));
};
for node in &self.nodes {
if matches!(node.kind, NodeKind::Todo(_)) && !self.children(&node.id).is_empty() {
return Err(format!("sink {} has children", node.id));
}
}
// From the one root, every node reached exactly once: a node reached
// twice has a second parent or sits on a cycle, and one not reached is
// cut off.
let mut reached = 0;
let mut stack = vec![(root.id.as_str(), 0)];
while let Some((id, depth)) = stack.pop() {
if depth > MAX_DEPTH {
return Err(format!("node {id} is deeper than {MAX_DEPTH}"));
}
reached += 1;
if reached > self.nodes.len() {
return Err(
"a node is reached twice: it has two parents or sits on a cycle".to_owned(),
);
}
stack.extend(self.children(id).into_iter().map(|c| (c, depth + 1)));
}
if reached != self.nodes.len() {
return Err(format!(
"{} of {} nodes are not reachable from the source",
self.nodes.len() - reached,
self.nodes.len()
));
}
Ok(())
} }
actions
}
fn children(&self, id: &str) -> Vec<&str> { fn emit(flow: &Flow, node: &Node, sink: &TodoSink, fields: &Fields) -> Vec<Action> {
self.edges let recipients = match &sink.to {
.iter() Recipients::Agent(agent) => vec![agent.clone()],
.filter(|e| e.from == id) Recipients::Field(name) => match fields.get(name.as_str()) {
.map(|e| e.to.as_str())
.collect()
}
fn node(&self, id: &str) -> Option<&Node> {
self.nodes.iter().find(|n| n.id == id)
}
/// Walk the flow for one event of type `event_type` and return the ops it
/// emits. Assumes [`Self::validate`] passed.
pub fn evaluate(&self, event_type: &str, fields: &Fields) -> Vec<Action> {
let Some(root) = self
.nodes
.iter()
.find(|n| matches!(&n.kind, NodeKind::Source { events } if events.iter().any(|e| e == event_type)))
else {
return Vec::new();
};
let mut actions = Vec::new();
let mut stack = vec![root.id.as_str()];
while let Some(id) = stack.pop() {
let Some(node) = self.node(id) else { continue };
match &node.kind {
NodeKind::Source { .. } => stack.extend(self.children(id)),
NodeKind::Match { when } => {
if when.iter().all(|c| holds(c, fields)) {
stack.extend(self.children(id));
}
}
NodeKind::Todo(sink) => actions.extend(self.emit(node, sink, fields)),
}
}
actions
}
fn emit(&self, node: &Node, sink: &TodoSink, fields: &Fields) -> Vec<Action> {
let recipients = match fields.get(sink.to.as_str()) {
Some(Value::List(items)) => items.clone(), Some(Value::List(items)) => items.clone(),
Some(Value::Text(one)) => vec![one.clone()], Some(Value::Text(one)) => vec![one.clone()],
None => Vec::new(), None => Vec::new(),
},
};
let rendered = expand(&sink.key, fields).and_then(|key| {
let summary = match sink.op {
Op::Upsert => expand(&sink.text, fields)?,
Op::Clear => String::new(),
}; };
let op = RouterTodoOp { Ok((key, summary))
op: sink.op, });
key: expand(&sink.key, fields), let (key, summary) = match rendered {
prefix: sink.op == Op::Clear && sink.prefix, Ok(done) => done,
summary: match sink.op { Err(why) => {
Op::Upsert => expand(&sink.text, fields), tracing::warn!(flow = %flow.id, node = %node.id, %why, "notify router: a todo node names a placeholder the event lacks; nothing sent");
Op::Clear => String::new(), return Vec::new();
}, }
source: format!("router:{}", self.id), };
}; let op = RouterTodoOp {
recipients op: sink.op,
.into_iter() key,
.map(|agent| Action { prefix: sink.op == Op::Clear && sink.prefix,
flow: self.id.clone(), summary,
node: node.id.clone(), source: format!("router:{}", flow.id),
agent, };
op: op.clone(), recipients
}) .into_iter()
.collect() .map(|agent| Action {
} flow: flow.id.clone(),
node: node.id.clone(),
agent,
op: op.clone(),
})
.collect()
} }
fn holds(condition: &Condition, fields: &Fields) -> bool { fn holds(condition: &Condition, fields: &Fields) -> bool {
@ -224,9 +107,9 @@ fn holds(condition: &Condition, fields: &Fields) -> bool {
} }
} }
/// `template` with each `{field}` replaced by that text field's value. A list /// `template` with each `{field}` replaced by that field's value, a list
/// field is joined with `, `; an unknown name is left as written. /// joined with `, `. A name that is no field is an error naming it.
pub fn expand(template: &str, fields: &Fields) -> String { pub fn expand(template: &str, fields: &Fields) -> Result<String, String> {
let mut out = String::with_capacity(template.len()); let mut out = String::with_capacity(template.len());
let mut rest = template; let mut rest = template;
while let Some(open) = rest.find('{') { while let Some(open) = rest.find('{') {
@ -234,109 +117,24 @@ pub fn expand(template: &str, fields: &Fields) -> String {
let after = &rest[open + 1..]; let after = &rest[open + 1..];
let Some(close) = after.find('}') else { let Some(close) = after.find('}') else {
out.push_str(&rest[open..]); out.push_str(&rest[open..]);
return out; return Ok(out);
}; };
let name = &after[..close]; let name = &after[..close];
match fields.get(name) { match fields.get(name) {
Some(Value::Text(v)) => out.push_str(v), Some(Value::Text(v)) => out.push_str(v),
Some(Value::List(items)) => out.push_str(&items.join(", ")), Some(Value::List(items)) => out.push_str(&items.join(", ")),
None => { None => return Err(format!("{{{name}}} is no event field")),
out.push('{');
out.push_str(name);
out.push('}');
}
} }
rest = &after[close + 1..]; rest = &after[close + 1..];
} }
out.push_str(rest); out.push_str(rest);
out Ok(out)
}
/// The flows the router ships with.
///
/// - `assigned`: a newly assigned agent gets a todo for the issue or PR.
/// - `unassigned`: an agent taken off it gets that todo rewritten to say so.
/// Both write the same key, so the second replaces the first.
/// - `closed`: closing it clears every router todo the router queued for it.
pub fn builtins() -> Vec<Flow> {
let assignment_key = "forge:{repo}#{number}:assignment";
vec![
chain(
"assigned",
&["issue_assign", "pull_request_assign"],
"assigned",
TodoSink {
to: "assignees_added".to_owned(),
op: Op::Upsert,
key: assignment_key.to_owned(),
prefix: false,
text: "you are assigned to {kind} {repo}#{number}: {title} ({url})".to_owned(),
},
),
chain(
"unassigned",
&["issue_assign", "pull_request_assign"],
"unassigned",
TodoSink {
to: "assignees_removed".to_owned(),
op: Op::Upsert,
key: assignment_key.to_owned(),
prefix: false,
text: "you are not assigned to {kind} {repo}#{number} anymore: {title} ({url})"
.to_owned(),
},
),
chain(
"closed",
&["issues", "pull_request"],
"closed",
TodoSink {
to: "todo_holders".to_owned(),
op: Op::Clear,
key: "forge:{repo}#{number}:".to_owned(),
prefix: true,
text: String::new(),
},
),
]
}
/// source(`events`) → match(`action` = `action`) → `sink`.
fn chain(id: &str, events: &[&str], action: &str, sink: TodoSink) -> Flow {
let node = |id: &str, kind| Node {
id: id.to_owned(),
kind,
};
let edge = |from: &str, to: &str| Edge {
from: from.to_owned(),
to: to.to_owned(),
};
Flow {
id: id.to_owned(),
nodes: vec![
node(
"source",
NodeKind::Source {
events: events.iter().map(|e| (*e).to_owned()).collect(),
},
),
node(
"match",
NodeKind::Match {
when: vec![Condition {
field: "action".to_owned(),
equals: action.to_owned(),
}],
},
),
node("sink", NodeKind::Todo(sink)),
],
edges: vec![edge("source", "match"), edge("match", "sink")],
}
} }
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use swarm_wire::notify_flows::{Edge, builtins};
use super::*; use super::*;
fn text(v: &str) -> Value { fn text(v: &str) -> Value {
@ -368,16 +166,11 @@ mod tests {
.expect("a built-in") .expect("a built-in")
} }
#[test]
fn every_builtin_is_a_valid_tree() {
for f in builtins() {
assert_eq!(f.validate(), Ok(()), "{}", f.id);
}
}
#[test] #[test]
fn assigning_queues_a_todo_for_each_newly_assigned_agent_only() { fn assigning_queues_a_todo_for_each_newly_assigned_agent_only() {
let got = flow("assigned").evaluate( let assigned = flow("assigned");
let got = evaluate(
&assigned,
"issue_assign", "issue_assign",
&event("assigned", &["atlas", "iris"], &[], &[]), &event("assigned", &["atlas", "iris"], &[], &[]),
); );
@ -392,21 +185,28 @@ mod tests {
assert_eq!(got[0].op.source, "router:assigned"); assert_eq!(got[0].op.source, "router:assigned");
// The match gates: an unassign event reaches no assigned sink. // The match gates: an unassign event reaches no assigned sink.
assert!( assert!(
flow("assigned") evaluate(
.evaluate("issue_assign", &event("unassigned", &["atlas"], &[], &[])) &assigned,
.is_empty() "issue_assign",
&event("unassigned", &["atlas"], &[], &[])
)
.is_empty()
); );
// The source gates: another event type starts nothing. // The source gates: another event type starts nothing.
assert!( assert!(
flow("assigned") evaluate(
.evaluate("issues", &event("assigned", &["atlas"], &[], &[])) &assigned,
.is_empty() "issues",
&event("assigned", &["atlas"], &[], &[])
)
.is_empty()
); );
} }
#[test] #[test]
fn unassigning_rewrites_the_same_key_for_the_removed_agent() { fn unassigning_rewrites_the_same_key_for_the_removed_agent() {
let got = flow("unassigned").evaluate( let got = evaluate(
&flow("unassigned"),
"pull_request_assign", "pull_request_assign",
&event("unassigned", &[], &["argus"], &[]), &event("unassigned", &[], &["argus"], &[]),
); );
@ -423,8 +223,12 @@ mod tests {
#[test] #[test]
fn closing_clears_by_prefix_for_every_holder() { fn closing_clears_by_prefix_for_every_holder() {
let got = let closed = flow("closed");
flow("closed").evaluate("issues", &event("closed", &[], &[], &["atlas", "argus"])); let got = evaluate(
&closed,
"issues",
&event("closed", &[], &[], &["atlas", "argus"]),
);
assert_eq!(got.len(), 2); assert_eq!(got.len(), 2);
for a in &got { for a in &got {
assert_eq!(a.op.op, Op::Clear); assert_eq!(a.op.op, Op::Clear);
@ -432,11 +236,7 @@ mod tests {
assert_eq!(a.op.key, "forge:hyperhive/hyperhive#7:"); assert_eq!(a.op.key, "forge:hyperhive/hyperhive#7:");
assert_eq!(a.op.summary, ""); assert_eq!(a.op.summary, "");
} }
assert!( assert!(evaluate(&closed, "issues", &event("reopened", &[], &[], &["atlas"])).is_empty());
flow("closed")
.evaluate("issues", &event("reopened", &[], &[], &["atlas"]))
.is_empty()
);
} }
/// A match node with two children runs both: the branching the tree /// A match node with two children runs both: the branching the tree
@ -450,7 +250,7 @@ mod tests {
f.nodes.push(Node { f.nodes.push(Node {
id: "second".to_owned(), id: "second".to_owned(),
kind: NodeKind::Todo(TodoSink { kind: NodeKind::Todo(TodoSink {
to: "assignees_removed".to_owned(), to: Recipients::Field("assignees_removed".to_owned()),
..sink ..sink
}), }),
}); });
@ -459,7 +259,8 @@ mod tests {
to: "second".to_owned(), to: "second".to_owned(),
}); });
assert_eq!(f.validate(), Ok(())); assert_eq!(f.validate(), Ok(()));
let got = f.evaluate( let got = evaluate(
&f,
"issue_assign", "issue_assign",
&event("assigned", &["atlas"], &["argus"], &[]), &event("assigned", &["atlas"], &["argus"], &[]),
); );
@ -471,48 +272,59 @@ mod tests {
assert_eq!(nodes, [("second", "argus"), ("sink", "atlas")]); assert_eq!(nodes, [("second", "argus"), ("sink", "atlas")]);
} }
/// A custom flow's sink can name one agent, whatever the event holds.
#[test] #[test]
fn a_flow_that_is_not_a_tree_is_refused() { fn a_sink_to_a_named_agent_reaches_it_once() {
let edge = |from: &str, to: &str| Edge { let mut f = flow("assigned");
from: from.to_owned(), f.nodes[2].kind = NodeKind::Todo(TodoSink {
to: to.to_owned(), to: Recipients::Agent("triage".to_owned()),
}; op: Op::Upsert,
let mut cycle = flow("assigned"); key: "forge:{repo}#{number}:triage".to_owned(),
cycle.edges[0] = edge("sink", "match"); prefix: false,
cycle.edges.push(edge("match", "sink")); text: "triage {repo}#{number}".to_owned(),
assert!(cycle.validate().is_err(), "a cycle");
let mut orphan = flow("assigned");
orphan.edges.pop();
assert!(orphan.validate().is_err(), "an unreachable sink");
let mut two_parents = flow("assigned");
two_parents.edges.push(edge("source", "sink"));
assert!(two_parents.validate().is_err(), "a sink with two parents");
let mut sink_child = flow("assigned");
sink_child.nodes.push(Node {
id: "m2".to_owned(),
kind: NodeKind::Match { when: Vec::new() },
}); });
sink_child.edges.push(edge("sink", "m2")); let got = evaluate(
assert!(sink_child.validate().is_err(), "a sink with a child"); &f,
"issue_assign",
let mut dangling = flow("assigned"); &event("assigned", &["atlas", "iris"], &[], &[]),
dangling.edges.push(edge("match", "nowhere")); );
assert!(dangling.validate().is_err(), "an edge to no node"); assert_eq!(got.len(), 1);
assert_eq!(got[0].agent, "triage");
let mut two_sources = flow("assigned"); assert_eq!(got[0].op.summary, "triage hyperhive/hyperhive#7");
two_sources.nodes[1].kind = NodeKind::Source { events: Vec::new() }; assert_eq!(got[0].op.source, "router:assigned");
assert!(two_sources.validate().is_err(), "two sources");
} }
#[test] #[test]
fn a_placeholder_expands_to_its_field_and_an_unknown_one_stays() { fn a_placeholder_expands_to_its_field_and_an_unknown_one_is_an_error() {
let fields = event("assigned", &["a", "b"], &[], &[]); let fields = event("assigned", &["a", "b"], &[], &[]);
assert_eq!( assert_eq!(
expand("{repo}#{number} {assignees_added} {nope} {", &fields), expand("{repo}#{number} {assignees_added} {", &fields),
"hyperhive/hyperhive#7 a, b {nope} {" Ok("hyperhive/hyperhive#7 a, b {".to_owned())
);
assert_eq!(
expand("x {nope}", &fields),
Err("{nope} is no event field".to_owned())
); );
} }
/// A sink naming a placeholder the event lacks sends nothing, rather
/// than a todo under a literal `{name}` key.
#[test]
fn a_sink_with_an_unknown_placeholder_emits_nothing() {
let mut f = flow("assigned");
f.nodes[2].kind = NodeKind::Todo(TodoSink {
to: Recipients::Agent("triage".to_owned()),
op: Op::Upsert,
key: "forge:{repo}#{numbr}:triage".to_owned(),
prefix: false,
text: "t".to_owned(),
});
let fields = event("assigned", &["atlas"], &[], &[]);
assert!(evaluate(&f, "issue_assign", &fields).is_empty());
// Control: spelled right, the same flow emits.
if let NodeKind::Todo(s) = &mut f.nodes[2].kind {
s.key = "forge:{repo}#{number}:triage".to_owned();
}
assert_eq!(evaluate(&f, "issue_assign", &fields).len(), 1);
}
} }

View file

@ -4,7 +4,8 @@
//! (`swarm_wire::forge_events`), runs every flow over each delivery //! (`swarm_wire::forge_events`), runs every flow over each delivery
//! ([`flow`], [`route`]), and queues the todo ops the flows emit on each //! ([`flow`], [`route`]), and queues the todo ops the flows emit on each
//! recipient agent's subject of `ROUTER_TODOS` (`swarm_wire::router_todo`). //! recipient agent's subject of `ROUTER_TODOS` (`swarm_wire::router_todo`).
//! Each agent's harness applies them to its own todo list. //! Each agent's harness applies them to its own todo list. The flows are the
//! built-in ones, changed by the `notify-flows` document ([`stored`]).
//! //!
//! Per issue it keeps a snapshot in its state bucket ([`event::Snapshot`]): //! Per issue it keeps a snapshot in its state bucket ([`event::Snapshot`]):
//! the assignees it last saw and the agents it queued a todo for. An event is //! the assignees it last saw and the agents it queued a todo for. An event is
@ -31,6 +32,7 @@ mod event;
mod flow; mod flow;
mod route; mod route;
mod settle; mod settle;
mod stored;
/// The cert-auth role this service logs in to the store as. `swarm-bao.nix` /// The cert-auth role this service logs in to the store as. `swarm-bao.nix`
/// writes it, matching the leaf's subject. /// writes it, matching the leaf's subject.
@ -83,17 +85,7 @@ async fn main() -> Result<()> {
}) })
.await?; .await?;
let flows: Vec<flow::Flow> = flow::builtins() let mut router = Router::open(&queue).await?;
.into_iter()
.filter(|f| match f.validate() {
Ok(()) => true,
Err(reason) => {
tracing::error!(flow = %f.id, %reason, "notify router: skipping a flow that is not a tree");
false
}
})
.collect();
let mut router = Router::open(&queue, flows).await?;
tracing::info!( tracing::info!(
stream = forge_events::STREAM, stream = forge_events::STREAM,
"notify router: routing forge events" "notify router: routing forge events"
@ -113,12 +105,12 @@ struct Router {
js: jetstream::Context, js: jetstream::Context,
events: jetstream::consumer::Consumer<pull::Config>, events: jetstream::consumer::Consumer<pull::Config>,
state: kv::Store, state: kv::Store,
flows: Vec<flow::Flow>, flows: stored::Flows,
roster: Option<(Instant, BTreeSet<String>)>, roster: Option<(Instant, BTreeSet<String>)>,
} }
impl Router { impl Router {
async fn open(queue: &async_nats::Client, flows: Vec<flow::Flow>) -> Result<Self> { async fn open(queue: &async_nats::Client) -> Result<Self> {
let js = jetstream::new(queue.clone()); let js = jetstream::new(queue.clone());
let events = swarm_queue_client::forge_events::open_or_create(queue) let events = swarm_queue_client::forge_events::open_or_create(queue)
.await .await
@ -154,7 +146,7 @@ impl Router {
js, js,
events, events,
state, state,
flows, flows: stored::Flows::builtins(),
roster: None, roster: None,
}) })
} }
@ -193,8 +185,9 @@ impl Router {
}), }),
None => event::Snapshot::default(), None => event::Snapshot::default(),
}; };
self.flows.refresh(&self.js).await;
let roster = self.roster().await.map_err(Skip::Retry)?; let roster = self.roster().await.map_err(Skip::Retry)?;
let routed = route::route(&event_type, &delivery, &snapshot, &self.flows, |a| { let routed = route::route(&event_type, &delivery, &snapshot, &self.flows.active, |a| {
roster.contains(a) roster.contains(a)
}); });
for action in &routed.actions { for action in &routed.actions {

View file

@ -6,7 +6,9 @@ use std::collections::BTreeSet;
use swarm_wire::router_todo::Op; use swarm_wire::router_todo::Op;
use crate::event::{Delivery, Snapshot, fields, is_assign}; use crate::event::{Delivery, Snapshot, fields, is_assign};
use crate::flow::{Action, Flow}; use swarm_wire::notify_flows::Flow;
use crate::flow::{Action, evaluate};
/// What one event comes to. /// What one event comes to.
#[derive(Debug, PartialEq, Eq)] #[derive(Debug, PartialEq, Eq)]
@ -30,7 +32,7 @@ pub fn route(
let fields = fields(event_type, delivery, snapshot); let fields = fields(event_type, delivery, snapshot);
let mut actions = Vec::new(); let mut actions = Vec::new();
let mut skipped = Vec::new(); let mut skipped = Vec::new();
for action in flows.iter().flat_map(|f| f.evaluate(event_type, &fields)) { for action in flows.iter().flat_map(|f| evaluate(f, event_type, &fields)) {
if is_agent(&action.agent) { if is_agent(&action.agent) {
actions.push(action); actions.push(action);
} else if !skipped.contains(&action.agent) { } else if !skipped.contains(&action.agent) {
@ -68,7 +70,7 @@ pub fn route(
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
use crate::flow::builtins; use swarm_wire::notify_flows::builtins;
fn delivery(action: &str, assignees: &[&str]) -> Delivery { fn delivery(action: &str, assignees: &[&str]) -> Delivery {
Delivery { Delivery {

View file

@ -0,0 +1,259 @@
//! The flows the router runs: the built-ins, changed by the document in the
//! `notify-flows` bucket.
//!
//! The router reads the key again every [`REFRESH`] and runs what
//! [`Document::active`] says. What it does with each thing it can read:
//!
//! | the key | the router runs |
//! | ------------------------------------ | ----------------------------------- |
//! | absent, deleted, or no such bucket | the built-ins |
//! | a document it reads | its built-ins and custom flows |
//! | a document it cannot read | what it ran before |
//! | the read itself failing | what it ran before |
//!
//! "Cannot read" is not JSON, not the document's shape, or a version this
//! build does not know. Before any document has been read, "what it ran
//! before" is the built-ins, so a bad document never leaves the router with
//! no flows.
use std::time::{Duration, Instant};
use async_nats::jetstream::ErrorCode;
use async_nats::jetstream::context::GetStreamErrorKind;
use async_nats::jetstream::{self, kv};
use swarm_wire::notify_flows::{self, Document, Flow};
/// How long a read of the key is reused.
const REFRESH: Duration = Duration::from_secs(10);
/// What a read of the key came to.
#[derive(Debug)]
pub enum Read {
/// No key, a deleted key, or no bucket.
Absent,
Document {
revision: u64,
bytes: Vec<u8>,
},
Failed(String),
}
/// The flows in force, and which revision of the key they came from.
pub struct Flows {
pub active: Vec<Flow>,
/// The revision `active` was read from; `None` for the built-ins alone.
revision: Option<u64>,
/// The last problem logged, so a standing one is logged once.
problem: Option<String>,
checked: Option<Instant>,
}
impl Flows {
/// The built-ins, which is what the router runs until it reads a document.
pub fn builtins() -> Self {
Self {
active: Document::default().active().0,
revision: None,
problem: None,
checked: None,
}
}
/// Take what a read came to.
pub fn apply(&mut self, read: Read) {
match read {
Read::Failed(why) => self.note(format!("reading the key failed: {why}")),
Read::Absent => {
if self.revision.take().is_some() {
tracing::info!(
"notify router: the flows document is gone; running the built-in flows"
);
}
self.active = Document::default().active().0;
self.problem = None;
}
Read::Document { revision, .. } if self.revision == Some(revision) => {}
Read::Document { revision, bytes } => match Document::decode(&bytes) {
Err(why) => {
// The revision is not taken, so the next read looks at
// the document again and the problem is logged once.
self.note(format!("revision {revision} cannot be read: {why}"));
}
Ok(doc) => {
let (active, skipped) = doc.active();
for why in &skipped {
tracing::error!(%why, "notify router: skipping a flow");
}
tracing::info!(
revision,
flows = active.len(),
disabled = doc.disabled_builtins.len(),
custom = doc.flows.len(),
"notify router: running the flows document"
);
self.active = active;
self.revision = Some(revision);
self.problem = None;
}
},
}
}
fn note(&mut self, problem: String) {
if self.problem.as_ref() != Some(&problem) {
tracing::error!(%problem, "notify router: keeping the flows it has");
self.problem = Some(problem);
}
}
/// Read the key if the last read is older than [`REFRESH`].
pub async fn refresh(&mut self, js: &jetstream::Context) {
if self.checked.is_some_and(|at| at.elapsed() < REFRESH) {
return;
}
self.checked = Some(Instant::now());
self.apply(read(js).await);
}
}
async fn read(js: &jetstream::Context) -> Read {
// A bucket that does not exist is the answer "no document". Any other
// failure to look, such as a timeout, says nothing about the document.
match js.get_stream(format!("KV_{}", notify_flows::BUCKET)).await {
Ok(_) => {}
Err(e)
if matches!(e.kind(), GetStreamErrorKind::JetStream(j)
if j.error_code() == ErrorCode::STREAM_NOT_FOUND) =>
{
return Read::Absent;
}
Err(e) => return Read::Failed(e.to_string()),
}
let store = match js.get_key_value(notify_flows::BUCKET).await {
Ok(store) => store,
Err(e) => return Read::Failed(e.to_string()),
};
match store.entry(notify_flows::KEY).await {
Ok(Some(entry)) if entry.operation == kv::Operation::Put => Read::Document {
revision: entry.revision,
bytes: entry.value.to_vec(),
},
Ok(_) => Read::Absent,
Err(e) => Read::Failed(e.to_string()),
}
}
#[cfg(test)]
mod tests {
use swarm_wire::notify_flows::{Document, VERSION};
use super::*;
fn ids(f: &Flows) -> Vec<&str> {
f.active.iter().map(|f| f.id.as_str()).collect()
}
fn doc(disabled: &[&str], custom: &[&str]) -> Read {
let mut flows = Vec::new();
for id in custom {
let mut f = notify_flows::builtins().remove(0);
f.id = (*id).to_owned();
flows.push(f);
}
let d = Document {
version: VERSION,
disabled_builtins: disabled.iter().map(|s| (*s).to_owned()).collect(),
flows,
};
Read::Document {
revision: 0,
bytes: serde_json::to_vec(&d).unwrap(),
}
}
fn at(revision: u64, read: Read) -> Read {
match read {
Read::Document { bytes, .. } => Read::Document { revision, bytes },
other => other,
}
}
fn junk(revision: u64, bytes: &[u8]) -> Read {
Read::Document {
revision,
bytes: bytes.to_vec(),
}
}
#[test]
fn with_no_document_the_router_runs_the_builtins() {
let mut f = Flows::builtins();
assert_eq!(ids(&f), ["assigned", "unassigned", "closed"]);
f.apply(Read::Absent);
assert_eq!(ids(&f), ["assigned", "unassigned", "closed"]);
// An empty document is the same.
f.apply(at(1, doc(&[], &[])));
assert_eq!(ids(&f), ["assigned", "unassigned", "closed"]);
}
#[test]
fn a_document_switches_builtins_off_and_adds_custom_flows() {
let mut f = Flows::builtins();
f.apply(at(1, doc(&["unassigned"], &["triage"])));
assert_eq!(ids(&f), ["assigned", "closed", "triage"]);
}
#[test]
fn a_document_it_cannot_read_leaves_the_flows_as_they_were() {
let mut f = Flows::builtins();
f.apply(at(1, doc(&["closed"], &["triage"])));
let before = ids(&f).join(",");
assert_eq!(before, "assigned,unassigned,triage");
for bad in [
junk(2, b"not json"),
junk(3, br#"{"version":2,"flows":[]}"#),
junk(4, br#"{"flows":[{"id":"x"}]}"#),
Read::Failed("timeout".to_owned()),
] {
f.apply(bad);
assert_eq!(ids(&f).join(","), before);
}
// The next good revision replaces them: the control that the bad
// ones were kept out by being bad, not by the loader being stuck.
f.apply(at(5, doc(&[], &[])));
assert_eq!(ids(&f), ["assigned", "unassigned", "closed"]);
}
#[test]
fn a_bad_first_document_is_the_builtins_not_nothing() {
let mut f = Flows::builtins();
f.apply(junk(1, br#"{"version":9}"#));
assert_eq!(ids(&f), ["assigned", "unassigned", "closed"]);
}
#[test]
fn a_deleted_document_goes_back_to_the_builtins() {
let mut f = Flows::builtins();
f.apply(at(1, doc(&["closed"], &["triage"])));
f.apply(Read::Absent);
assert_eq!(ids(&f), ["assigned", "unassigned", "closed"]);
}
#[test]
fn a_flow_that_is_not_valid_is_left_out_and_the_rest_run() {
let mut f = Flows::builtins();
let mut broken = notify_flows::builtins().remove(0);
broken.id = "broken".to_owned();
broken.edges.clear();
let d = Document {
version: VERSION,
disabled_builtins: Vec::new(),
flows: vec![broken],
};
f.apply(Read::Document {
revision: 1,
bytes: serde_json::to_vec(&d).unwrap(),
});
assert_eq!(ids(&f), ["assigned", "unassigned", "closed"]);
}
}

View file

@ -18,6 +18,7 @@ pub mod builder;
pub mod claude_login; pub mod claude_login;
pub mod forge_events; pub mod forge_events;
pub mod notices; pub mod notices;
pub mod notify_flows;
pub mod notify_router; pub mod notify_router;
pub mod review_classifier; pub mod review_classifier;
pub mod router_todo; pub mod router_todo;

Some files were not shown because too many files have changed in this diff Show more