Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
fe578c57fd | ||
|
|
ada364a482 | ||
|
|
7795da44f8 | ||
|
|
0512862b3d | ||
|
|
e2e124755d | ||
|
|
bb08fac33c |
21 changed files with 2193 additions and 368 deletions
|
|
@ -64,21 +64,60 @@ The router holds no forge token and no webhook secret, and registers no hook.
|
|||
|
||||
A flow is a graph of nodes joined by edges: one `source` node naming the forge
|
||||
event types it starts from, `match` nodes that pass when every condition
|
||||
holds, and `sink` nodes that emit one todo op per recipient. A match node can
|
||||
holds, and `todo` nodes that emit one todo op per recipient. A match node can
|
||||
have more than one child, and each one runs, so one event can reach several
|
||||
sinks. A flow must be a tree: one source, one parent for every other node,
|
||||
every node reachable, no children under a sink.
|
||||
todos. A flow must be a tree: one source, one parent for every other node,
|
||||
every node reachable, no children under a `todo`. The most is 64 nodes, 16
|
||||
levels deep, and 100 custom flows. A `todo` node sends to the agents an event
|
||||
field holds (`assignees_added`, `todo_holders`, …) or to one named agent.
|
||||
|
||||
The router ships the three flows above, each a source → match on `action` →
|
||||
todo sink.
|
||||
todo. The code defines them (`swarm_wire::notify_flows::builtins`), so a
|
||||
build that changes one reaches every swarm. A custom flow has the same type and
|
||||
the same evaluator, so it can do anything a built-in does.
|
||||
|
||||
**The stored document.** swarm-controller keeps the rest in one document, the
|
||||
`all` key of the `notify-flows` bucket: the ids of the built-in flows you
|
||||
switched off, and the custom flows. Only the controller writes it, through
|
||||
`GET` and `PUT /api/notify/flows`. A `PUT` replaces the whole document in one
|
||||
compare-and-set. The controller answers 400 unless every flow passes the tree
|
||||
check above and reads only what exists: a source event from `issues`,
|
||||
`issue_assign`, `pull_request` or `pull_request_assign`, and condition and
|
||||
recipient fields from the event's field list. The same goes for the `{field}`
|
||||
placeholders in a todo's key and text. A `PUT` must carry the `revision` it read, `0` for no
|
||||
document, and gets 409 when the stored revision is another. Swarm UI's
|
||||
[`/notifications`](ui.md#notification-flows) page shows the document and
|
||||
switches built-ins.
|
||||
|
||||
**What the router runs.** It reads the key at most every 10 seconds, when an
|
||||
event arrives, and runs the built-ins you left on, then the custom flows that
|
||||
are on and pass the tree check. It logs a custom flow that fails the check,
|
||||
leaves it out, and runs the rest.
|
||||
|
||||
| the key | the router runs |
|
||||
| -------------------------------------- | ----------------------- |
|
||||
| absent, deleted, or its bucket missing | the built-in flows |
|
||||
| a document it reads | the above |
|
||||
| a document it can't read | the flows it ran before |
|
||||
| a read that fails | the flows it ran before |
|
||||
|
||||
"Can't read" means not JSON, not a flows document, or a `version` other than 1.
|
||||
Before the router has read any document, "before" means the built-ins, so a bad
|
||||
document never leaves it with no flows. The controller refuses to overwrite a
|
||||
document of a version it doesn't know.
|
||||
|
||||
**Deploying in any order.** The controller, the router and the UI can be on
|
||||
different hosts. A controller that predates this leaves the key absent, and the
|
||||
router runs the built-ins as before. A router that predates this ignores the
|
||||
document, so the page and the API show a state the router isn't yet running.
|
||||
|
||||
## Credentials and grants
|
||||
|
||||
| Principal | Grant |
|
||||
| --------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `swarm-notify-router`, in its container | store: `read` on `swarm/services/<router client id>/oidc/client`. Queue: its own consumer on `SWARM_EVENTS` (create, pull, ack), publish on `$SWARM.router.todo.*`, create `ROUTER_TODOS`, its own `notify-router-state` bucket, and `STREAM.INFO` on the agent-status bucket, which lists the agents |
|
||||
| swarm-controller | publish on `$SWARM.events.forge.*` |
|
||||
| an agent, under its own credential | create its own consumer `todo-<agent>` in the form whose filter is `$SWARM.router.todo.<agent>`, pull from it, and ack what it delivered |
|
||||
| Principal | Grant |
|
||||
| --------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `swarm-notify-router`, in its container | store: `read` on `swarm/services/<router client id>/oidc/client`. Queue: its own consumer on `SWARM_EVENTS` (create, pull, ack), publish on `$SWARM.router.todo.*`, create `ROUTER_TODOS`, its own `notify-router-state` bucket, read on the `notify-flows` bucket, and `STREAM.INFO` on the agent-status bucket, which lists the agents |
|
||||
| swarm-controller | publish on `$SWARM.events.forge.*`; creates and writes the `notify-flows` bucket (every bucket, by its `$KV.*.>` grant) |
|
||||
| an agent, under its own credential | create its own consumer `todo-<agent>` in the form whose filter is `$SWARM.router.todo.<agent>`, pull from it, and ack what it delivered |
|
||||
|
||||
No other client may publish on `$SWARM.router.todo.*` or
|
||||
`$SWARM.events.forge.*`, so each has one writer. The agent's create subject
|
||||
|
|
|
|||
|
|
@ -19,6 +19,7 @@ domain, covers host-level detail for one hive.
|
|||
| `/schedules` | the scheduled prompts: add, edit, pause, resume and remove |
|
||||
| `/broadcast` | send one message to several agents at once, picked from a grid of their icons |
|
||||
| `/messages` | the swarm-wide message feed: every message between agents, and to them from the operator, reminders and compaction, with a history window and a filter on sender or recipient |
|
||||
| `/notifications` | the notification router's flows: a switch for each built-in, and every flow drawn as a tree |
|
||||
| `/issues` | a cross-repo issue report |
|
||||
| `/review` | the review classifier: its recent runs, a try-out on a real PR, and its settings |
|
||||
| `/builds` | the swarm builder's recent builds of agent config PRs, with what nix printed when one failed |
|
||||
|
|
@ -216,6 +217,22 @@ Only an operator can change a schedule here or with `swarmctl schedule`; an
|
|||
agent has no way to add one. The page needs the controller's `/api/schedules`
|
||||
routes.
|
||||
|
||||
### Notification flows
|
||||
|
||||
`/notifications` lists the [notification router's flows](notify-router.md#flows)
|
||||
in two groups. Each **built-in** flow has an on/off switch; switching one off
|
||||
stops the router running it from its next event, and switching it on again
|
||||
restores it. Each **custom** flow shows its name and whether it's on. Every
|
||||
flow draws as an indented tree: the forge events it starts from, the
|
||||
conditions under it, and the todo each branch ends in.
|
||||
|
||||
The page edits nothing else. Add custom flows with `PUT /api/notify/flows`.
|
||||
|
||||
A switch sends the whole flows document with the revision the page read, `0` if
|
||||
it read no document. The controller refuses the switch when the stored revision
|
||||
is another, including when someone wrote a document after the page read none, and
|
||||
the page shows why. Reload to switch against the current document.
|
||||
|
||||
### Sending a message to an agent
|
||||
|
||||
Under an agent's terminal, in the preview and in its own tab, is a **message**
|
||||
|
|
|
|||
|
|
@ -8,6 +8,7 @@ import { AlertsPage } from "./pages/AlertsPage.js";
|
|||
import { BuildsPage } from "./pages/builds/BuildsPage.js";
|
||||
import { BroadcastPage } from "./pages/broadcast/BroadcastPage.js";
|
||||
import { MessagesPage } from "./pages/messages/MessagesPage.js";
|
||||
import { NotificationsPage } from "./pages/notifications/NotificationsPage.js";
|
||||
import { ComponentsPage } from "./pages/ComponentsPage.js";
|
||||
import { JobsPage } from "./pages/JobsPage.js";
|
||||
import { HivesPage } from "./pages/HivesPage.js";
|
||||
|
|
@ -43,6 +44,7 @@ export function App() {
|
|||
<Route path="/builds" component={BuildsPage} />
|
||||
<Route path="/broadcast" component={BroadcastPage} />
|
||||
<Route path="/messages" component={MessagesPage} />
|
||||
<Route path="/notifications" component={NotificationsPage} />
|
||||
<Route path="/issues" component={IssueReportPage} />
|
||||
<Route path="/review" component={ReviewPage} />
|
||||
<Route path="/components" component={ComponentsPage} />
|
||||
|
|
|
|||
|
|
@ -0,0 +1,95 @@
|
|||
// Wire types of swarm-controller's `GET/PUT /api/notify/flows` and the small
|
||||
// pure helpers the notifications page draws a flow with.
|
||||
|
||||
export type Recipients = { field: string } | { agent: string };
|
||||
|
||||
export interface Condition {
|
||||
field: string;
|
||||
equals: string;
|
||||
}
|
||||
|
||||
// A node as the API sends it: `kind` sits beside `id`.
|
||||
export type FlowNode =
|
||||
| { id: string; kind: "source"; events: string[] }
|
||||
| { id: string; kind: "match"; when: Condition[] }
|
||||
| {
|
||||
id: string;
|
||||
kind: "todo";
|
||||
to: Recipients;
|
||||
op: "upsert" | "clear";
|
||||
key: string;
|
||||
prefix: boolean;
|
||||
text: string;
|
||||
};
|
||||
|
||||
export interface Flow {
|
||||
id: string;
|
||||
name: string;
|
||||
enabled: boolean;
|
||||
nodes: FlowNode[];
|
||||
edges: { from: string; to: string }[];
|
||||
}
|
||||
|
||||
export interface FlowsView {
|
||||
revision: number;
|
||||
disabled_builtins: string[];
|
||||
builtins: Flow[];
|
||||
flows: Flow[];
|
||||
}
|
||||
|
||||
// One node of a flow with its depth below the source, in the order a reader
|
||||
// walks the tree: a parent, then each child's subtree in edge order.
|
||||
export interface TreeRow {
|
||||
node: FlowNode;
|
||||
depth: number;
|
||||
}
|
||||
|
||||
export function treeRows(flow: Flow): TreeRow[] {
|
||||
const byId = new Map(flow.nodes.map((n) => [n.id, n]));
|
||||
const root = flow.nodes.find((n) => n.kind === "source");
|
||||
if (!root) return [];
|
||||
const rows: TreeRow[] = [];
|
||||
const seen = new Set<string>();
|
||||
const walk = (id: string, depth: number) => {
|
||||
const node = byId.get(id);
|
||||
// The controller refuses a flow that is not a tree; a repeat here would
|
||||
// mean a cycle, which `seen` stops from looping.
|
||||
if (!node || seen.has(id)) return;
|
||||
seen.add(id);
|
||||
rows.push({ node, depth });
|
||||
for (const e of flow.edges) if (e.from === id) walk(e.to, depth + 1);
|
||||
};
|
||||
walk(root.id, 0);
|
||||
return rows;
|
||||
}
|
||||
|
||||
export function describeRecipients(to: Recipients): string {
|
||||
return "field" in to ? `{${to.field}}` : to.agent;
|
||||
}
|
||||
|
||||
// One line saying what a node is and does.
|
||||
export function describeNode(node: FlowNode): string {
|
||||
switch (node.kind) {
|
||||
case "source":
|
||||
return `forge ${node.events.join(", ")}`;
|
||||
case "match":
|
||||
return node.when.length === 0
|
||||
? "match anything"
|
||||
: node.when.map((c) => `${c.field} = ${c.equals}`).join(" and ");
|
||||
case "todo":
|
||||
return node.op === "clear"
|
||||
? `clear todos ${node.key}${node.prefix ? "*" : ""} of ${describeRecipients(node.to)}`
|
||||
: `todo to ${describeRecipients(node.to)}, key ${node.key}: ${node.text}`;
|
||||
}
|
||||
}
|
||||
|
||||
// `disabled` with built-in `id` switched on or off, without repeats and in a
|
||||
// stable order.
|
||||
export function withBuiltin(
|
||||
disabled: string[],
|
||||
id: string,
|
||||
enabled: boolean,
|
||||
): string[] {
|
||||
const rest = disabled.filter((d) => d !== id);
|
||||
return (enabled ? rest : [...rest, id]).sort();
|
||||
}
|
||||
|
|
@ -0,0 +1,46 @@
|
|||
/* NotificationsPage — the flows drawn as indented trees. */
|
||||
|
||||
.notify-flows {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 0.75em;
|
||||
}
|
||||
|
||||
.notify-flow-head {
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
gap: 0.5em;
|
||||
}
|
||||
|
||||
.notify-flow-name {
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
.notify-flow-tree {
|
||||
margin: 0.4em 0 0;
|
||||
padding: 0;
|
||||
list-style: none;
|
||||
font-size: 0.9em;
|
||||
}
|
||||
|
||||
.notify-node {
|
||||
display: flex;
|
||||
gap: 0.5em;
|
||||
padding: 0.1em 0;
|
||||
}
|
||||
|
||||
.notify-node-kind {
|
||||
flex: none;
|
||||
min-width: 3.5em;
|
||||
color: var(--muted);
|
||||
}
|
||||
|
||||
.notify-node-text {
|
||||
overflow-wrap: anywhere;
|
||||
}
|
||||
|
||||
.notify-off {
|
||||
opacity: 0.6;
|
||||
}
|
||||
|
|
@ -0,0 +1,144 @@
|
|||
// <NotificationsPage> — the notification router's flows. Backed by
|
||||
// swarm-controller's `GET/PUT /api/notify/flows`; the controller is the only
|
||||
// writer of the stored document and the router reads it every few seconds.
|
||||
//
|
||||
// Read-only except for one switch per built-in flow. A flow draws as an
|
||||
// indented tree, the shape the API validates it to. Custom flows are not
|
||||
// editable here: they arrive through the API.
|
||||
import { useState } from "preact/hooks";
|
||||
import { ApiErrorPanel } from "@hive/shared/api-error-panel.js";
|
||||
import { readApiError, type ProblemDetails } from "@hive/shared/api-error.js";
|
||||
import { Card } from "../../ui/card/Card.js";
|
||||
import { CheckboxField } from "../../ui/checkbox-field/CheckboxField.js";
|
||||
import { Panel } from "../../ui/panel/Panel.js";
|
||||
import {
|
||||
RefreshIntervalPicker,
|
||||
type RefreshIntervalMs,
|
||||
} from "../../ui/refresh-interval/RefreshInterval.js";
|
||||
import { fetchJson, usePolled } from "../../ui/polled/usePolled.js";
|
||||
import {
|
||||
describeNode,
|
||||
treeRows,
|
||||
withBuiltin,
|
||||
type Flow,
|
||||
type FlowsView,
|
||||
} from "./FlowTypes.js";
|
||||
import "./NotificationsPage.css";
|
||||
|
||||
const DEFAULT_INTERVAL_MS: RefreshIntervalMs = 30_000;
|
||||
|
||||
function FlowTree({ flow }: { flow: Flow }) {
|
||||
return (
|
||||
<ul class="notify-flow-tree">
|
||||
{treeRows(flow).map(({ node, depth }) => (
|
||||
<li
|
||||
class="notify-node"
|
||||
key={node.id}
|
||||
style={{ paddingLeft: `${depth * 1.5}em` }}
|
||||
>
|
||||
<span class="notify-node-kind">{node.kind}</span>
|
||||
<span class="notify-node-text">{describeNode(node)}</span>
|
||||
</li>
|
||||
))}
|
||||
</ul>
|
||||
);
|
||||
}
|
||||
|
||||
export function NotificationsPage() {
|
||||
const [intervalMs, setIntervalMs] =
|
||||
useState<RefreshIntervalMs>(DEFAULT_INTERVAL_MS);
|
||||
const {
|
||||
data: view,
|
||||
error,
|
||||
refresh,
|
||||
} = usePolled(intervalMs, (signal) =>
|
||||
fetchJson<FlowsView>("/api/notify/flows", signal),
|
||||
);
|
||||
const [pending, setPending] = useState(false);
|
||||
const [actionError, setActionError] = useState<ProblemDetails | null>(null);
|
||||
|
||||
// Switch one built-in on or off: the whole document goes back with the
|
||||
// revision this page read (0 for none), so a change made elsewhere since is refused
|
||||
// rather than overwritten.
|
||||
async function setBuiltin(id: string, enabled: boolean) {
|
||||
if (!view) return;
|
||||
setPending(true);
|
||||
setActionError(null);
|
||||
try {
|
||||
const r = await fetch("/api/notify/flows", {
|
||||
method: "PUT",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
revision: view.revision,
|
||||
disabled_builtins: withBuiltin(view.disabled_builtins, id, enabled),
|
||||
flows: view.flows,
|
||||
}),
|
||||
});
|
||||
if (!r.ok) setActionError(await readApiError(r));
|
||||
} catch (err) {
|
||||
setActionError({ detail: String(err) });
|
||||
} finally {
|
||||
setPending(false);
|
||||
refresh();
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<Panel
|
||||
title="notifications"
|
||||
icon="🔔"
|
||||
actions={
|
||||
<RefreshIntervalPicker
|
||||
id="notifications-refresh"
|
||||
value={intervalMs}
|
||||
onChange={setIntervalMs}
|
||||
/>
|
||||
}
|
||||
>
|
||||
{error ? (
|
||||
<ApiErrorPanel context="failed to load the flows" problem={error} />
|
||||
) : null}
|
||||
{actionError ? (
|
||||
<ApiErrorPanel
|
||||
context="the flows were not changed"
|
||||
problem={actionError}
|
||||
/>
|
||||
) : null}
|
||||
{!error && view === null ? <p>loading…</p> : null}
|
||||
{view ? (
|
||||
<div class="notify-flows">
|
||||
<h3>built-in flows</h3>
|
||||
{view.builtins.map((flow) => {
|
||||
const on = !view.disabled_builtins.includes(flow.id);
|
||||
return (
|
||||
<Card key={flow.id} class={on ? "" : "notify-off"}>
|
||||
<div class="notify-flow-head">
|
||||
<span class="notify-flow-name">{flow.name}</span>
|
||||
<CheckboxField
|
||||
id={`builtin-${flow.id}`}
|
||||
label={on ? "on" : "off"}
|
||||
checked={on}
|
||||
disabled={pending}
|
||||
onChange={(checked) => setBuiltin(flow.id, checked)}
|
||||
/>
|
||||
</div>
|
||||
<FlowTree flow={flow} />
|
||||
</Card>
|
||||
);
|
||||
})}
|
||||
<h3>custom flows</h3>
|
||||
{view.flows.length === 0 ? <p>no custom flows</p> : null}
|
||||
{view.flows.map((flow) => (
|
||||
<Card key={flow.id} class={flow.enabled ? "" : "notify-off"}>
|
||||
<div class="notify-flow-head">
|
||||
<span class="notify-flow-name">{flow.name}</span>
|
||||
<span>{flow.enabled ? "on" : "off"}</span>
|
||||
</div>
|
||||
<FlowTree flow={flow} />
|
||||
</Card>
|
||||
))}
|
||||
</div>
|
||||
) : null}
|
||||
</Panel>
|
||||
);
|
||||
}
|
||||
|
|
@ -56,6 +56,7 @@ const NAV_ITEMS: { href: string; label: string; accent: string }[] = [
|
|||
{ href: "/schedules", label: "schedules", accent: "var(--cyan)" },
|
||||
{ href: "/broadcast", label: "broadcast", accent: "var(--blue)" },
|
||||
{ href: "/messages", label: "messages", accent: "var(--purple)" },
|
||||
{ href: "/notifications", label: "notifications", accent: "var(--yellow)" },
|
||||
{ href: "/issues", label: "issues", accent: "var(--yellow)" },
|
||||
{ href: "/review", label: "review", accent: "var(--amber)" },
|
||||
{ href: "/builds", label: "builds", accent: "var(--green)" },
|
||||
|
|
|
|||
80
frontend/packages/swarm-ui/test/flow-types.test.mjs
Normal file
80
frontend/packages/swarm-ui/test/flow-types.test.mjs
Normal file
|
|
@ -0,0 +1,80 @@
|
|||
// The notifications page's tree and wording helpers. Run by `npm test`.
|
||||
import assert from "node:assert/strict";
|
||||
import { test } from "node:test";
|
||||
import {
|
||||
describeNode,
|
||||
treeRows,
|
||||
withBuiltin,
|
||||
} from "../src/pages/notifications/FlowTypes.ts";
|
||||
|
||||
const flow = {
|
||||
id: "f",
|
||||
name: "f",
|
||||
enabled: true,
|
||||
nodes: [
|
||||
{ id: "s", kind: "source", events: ["issues", "issue_assign"] },
|
||||
{ id: "m", kind: "match", when: [{ field: "action", equals: "closed" }] },
|
||||
{ id: "m2", kind: "match", when: [] },
|
||||
{
|
||||
id: "t",
|
||||
kind: "todo",
|
||||
to: { field: "todo_holders" },
|
||||
op: "clear",
|
||||
key: "forge:{repo}#{number}:",
|
||||
prefix: true,
|
||||
text: "",
|
||||
},
|
||||
{
|
||||
id: "u",
|
||||
kind: "todo",
|
||||
to: { agent: "triage" },
|
||||
op: "upsert",
|
||||
key: "k",
|
||||
prefix: false,
|
||||
text: "look",
|
||||
},
|
||||
],
|
||||
edges: [
|
||||
{ from: "s", to: "m" },
|
||||
{ from: "m", to: "t" },
|
||||
{ from: "s", to: "m2" },
|
||||
{ from: "m2", to: "u" },
|
||||
],
|
||||
};
|
||||
|
||||
test("treeRows: each subtree follows its parent, with its depth", () => {
|
||||
assert.deepEqual(
|
||||
treeRows(flow).map((r) => [r.node.id, r.depth]),
|
||||
[
|
||||
["s", 0],
|
||||
["m", 1],
|
||||
["t", 2],
|
||||
["m2", 1],
|
||||
["u", 2],
|
||||
],
|
||||
);
|
||||
});
|
||||
|
||||
test("treeRows: a cycle ends the walk and a flow with no source is empty", () => {
|
||||
const cyclic = { ...flow, edges: [...flow.edges, { from: "t", to: "s" }] };
|
||||
assert.equal(treeRows(cyclic).length, 5);
|
||||
assert.deepEqual(treeRows({ ...flow, nodes: [] }), []);
|
||||
});
|
||||
|
||||
test("describeNode: says what each kind does", () => {
|
||||
assert.equal(describeNode(flow.nodes[0]), "forge issues, issue_assign");
|
||||
assert.equal(describeNode(flow.nodes[1]), "action = closed");
|
||||
assert.equal(describeNode(flow.nodes[2]), "match anything");
|
||||
assert.equal(
|
||||
describeNode(flow.nodes[3]),
|
||||
"clear todos forge:{repo}#{number}:* of {todo_holders}",
|
||||
);
|
||||
assert.equal(describeNode(flow.nodes[4]), "todo to triage, key k: look");
|
||||
});
|
||||
|
||||
test("withBuiltin: switching off adds once, switching on removes", () => {
|
||||
assert.deepEqual(withBuiltin([], "b", false), ["b"]);
|
||||
assert.deepEqual(withBuiltin(["b"], "b", false), ["b"]);
|
||||
assert.deepEqual(withBuiltin(["c"], "a", false), ["a", "c"]);
|
||||
assert.deepEqual(withBuiltin(["a", "b"], "a", true), ["b"]);
|
||||
});
|
||||
|
|
@ -82,6 +82,11 @@ The swarm's control plane. The swarm UI and `swarmctl` are its clients.
|
|||
`review-classifier-settings` KV bucket, which this daemon alone reads and
|
||||
writes (`GET`/`PUT /api/review-classifier/settings`) →
|
||||
[`docs/swarm/review-classifier.md`](../docs/swarm/review-classifier.md#swarm-level-settings).
|
||||
- **Notification flows** — which of the notification router's built-in flows are
|
||||
switched off, and the custom flows, as one document in the `notify-flows`
|
||||
KV bucket, which this daemon alone writes (`GET`/`PUT /api/notify/flows`);
|
||||
the router reads it →
|
||||
[`docs/swarm/notify-router.md`](../docs/swarm/notify-router.md#flows).
|
||||
- **Operator profiles** — each operator's name and the pronouns on their
|
||||
forge profile, for every member of authelia's operator group, as one list
|
||||
under the one key of the `operator-profiles` KV bucket. This daemon alone
|
||||
|
|
|
|||
|
|
@ -2113,6 +2113,14 @@ mod tests {
|
|||
events(DeliveryKind::PullRequest),
|
||||
["pull_request", "pull_request_assign"]
|
||||
);
|
||||
// The events a stored flow's source may name are the ones the hooks
|
||||
// deliver to the router.
|
||||
let mut routed = events(DeliveryKind::Issues);
|
||||
routed.extend(events(DeliveryKind::PullRequest));
|
||||
routed.sort_unstable();
|
||||
let mut sources = swarm_wire::notify_flows::SOURCE_EVENTS.to_vec();
|
||||
sources.sort_unstable();
|
||||
assert_eq!(routed, sources);
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
|
|
|||
|
|
@ -50,6 +50,7 @@ mod jobq;
|
|||
mod linked_accounts;
|
||||
mod matrix_account;
|
||||
mod message_stream;
|
||||
mod notify_flows;
|
||||
mod otel_http_client;
|
||||
mod pulls;
|
||||
mod queue_identity;
|
||||
|
|
|
|||
446
swarm-controller/src/notify_flows.rs
Normal file
446
swarm-controller/src/notify_flows.rs
Normal file
|
|
@ -0,0 +1,446 @@
|
|||
//! `GET` and `PUT /api/notify/flows`: which of the notification router's
|
||||
//! built-in flows are switched off, and the custom flows.
|
||||
//!
|
||||
//! The flows are one document under one key ([`notify_flows::KEY`] in
|
||||
//! [`notify_flows::BUCKET`]), which only this daemon writes, as the schedule
|
||||
//! definitions are. A `PUT` replaces the whole document in one
|
||||
//! compare-and-set, after [`Document::check`]. The router reads the key
|
||||
//! every few seconds, so a change applies to the next event with no restart.
|
||||
//! No document means the built-in flows alone.
|
||||
//!
|
||||
//! The built-in flows are defined in code, in `swarm_wire::notify_flows`, and
|
||||
//! the document stores only their disabled ids, so a `GET` shows the ones
|
||||
//! this build knows.
|
||||
|
||||
use anyhow::Context as _;
|
||||
use async_nats::jetstream::kv::Store;
|
||||
use axum::Json;
|
||||
use axum::extract::State;
|
||||
use axum::extract::rejection::JsonRejection;
|
||||
use axum::http::StatusCode;
|
||||
use problem_details::ProblemDetails;
|
||||
use serde::{Deserialize, Serialize};
|
||||
use swarm_wire::notify_flows::{self, DecodeError, Document, Flow};
|
||||
use utoipa::ToSchema;
|
||||
|
||||
use crate::routes::{AppState, error_problem, wanted_error_status};
|
||||
use crate::schedules::{queue, read_with, refused, write_revision};
|
||||
|
||||
/// How many times a losing writer re-reads and re-applies, as in
|
||||
/// `wanted::MAX_ATTEMPTS`.
|
||||
const MAX_ATTEMPTS: usize = 5;
|
||||
|
||||
/// What `GET` and a successful `PUT` answer.
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, ToSchema)]
|
||||
pub(crate) struct FlowsView {
|
||||
/// The revision of the stored document, to send back in a `PUT`; `0` when
|
||||
/// none is stored.
|
||||
revision: u64,
|
||||
/// The ids of the built-in flows that are switched off.
|
||||
disabled_builtins: Vec<String>,
|
||||
/// The built-in flows this build ships, all of them, switched off or not.
|
||||
builtins: Vec<Flow>,
|
||||
/// The custom flows.
|
||||
flows: Vec<Flow>,
|
||||
}
|
||||
|
||||
/// The body of a `PUT`: the whole document.
|
||||
#[derive(Debug, Clone, Deserialize, ToSchema)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
pub(crate) struct PutFlows {
|
||||
/// The `revision` the client read, `0` for "no document stored". The
|
||||
/// `PUT` is refused when the stored revision is another, which includes a
|
||||
/// document written since a client read `0`. A KV revision is never `0`.
|
||||
revision: u64,
|
||||
/// The ids of the built-in flows to switch off.
|
||||
#[serde(default)]
|
||||
disabled_builtins: Vec<String>,
|
||||
/// The custom flows.
|
||||
#[serde(default)]
|
||||
flows: Vec<Flow>,
|
||||
}
|
||||
|
||||
impl FlowsView {
|
||||
fn of(doc: Document, revision: u64) -> Self {
|
||||
Self {
|
||||
revision,
|
||||
disabled_builtins: doc.disabled_builtins,
|
||||
builtins: notify_flows::builtins(),
|
||||
flows: doc.flows,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// What the key holds.
|
||||
enum Stored {
|
||||
/// No key, or a deleted one.
|
||||
Nothing,
|
||||
Document(Document),
|
||||
/// A document this build cannot read, with why.
|
||||
Unreadable(DecodeError),
|
||||
}
|
||||
|
||||
/// The stored document, with the revision to write against.
|
||||
async fn stored(store: &Store) -> anyhow::Result<(Stored, Option<u64>)> {
|
||||
struct Read(Stored);
|
||||
impl Default for Read {
|
||||
fn default() -> Self {
|
||||
Self(Stored::Nothing)
|
||||
}
|
||||
}
|
||||
let (Read(stored), revision) = read_with(store, notify_flows::KEY, |bytes| {
|
||||
Ok(Read(match Document::decode(bytes) {
|
||||
Ok(doc) => Stored::Document(doc),
|
||||
Err(e) => Stored::Unreadable(e),
|
||||
}))
|
||||
})
|
||||
.await?;
|
||||
Ok((stored, revision))
|
||||
}
|
||||
|
||||
/// Open [`notify_flows::BUCKET`], creating it if nothing has yet.
|
||||
async fn open(client: &async_nats::Client) -> anyhow::Result<Store> {
|
||||
let js = async_nats::jetstream::new(client.clone());
|
||||
if let Ok(store) = js.get_key_value(notify_flows::BUCKET).await {
|
||||
return Ok(store);
|
||||
}
|
||||
js.create_key_value(async_nats::jetstream::kv::Config {
|
||||
bucket: notify_flows::BUCKET.to_owned(),
|
||||
description: "The notification router's flows: disabled built-ins and custom flows"
|
||||
.to_owned(),
|
||||
history: 1,
|
||||
..Default::default()
|
||||
})
|
||||
.await
|
||||
.with_context(|| format!("creating the {} bucket", notify_flows::BUCKET))
|
||||
}
|
||||
|
||||
fn queue_problem(e: &anyhow::Error) -> ProblemDetails {
|
||||
tracing::warn!(error = %format!("{e:#}"), "notify flows: queue failure");
|
||||
error_problem(wanted_error_status(e), &format!("{e:#}"))
|
||||
}
|
||||
|
||||
/// Read the flows. Creates the `notify-flows` bucket when it is missing, as a
|
||||
/// `PUT` does, so the router finds it empty rather than absent.
|
||||
#[utoipa::path(
|
||||
get,
|
||||
path = "/api/notify/flows",
|
||||
responses(
|
||||
(status = 200, description = "the stored document, with the built-in flows this \
|
||||
build ships; with none stored, no disabled built-ins and no custom flows",
|
||||
body = FlowsView),
|
||||
(status = 409, description = "the stored document is not one this build reads, \
|
||||
such as a newer build's version (problem+json)", body = String),
|
||||
(status = 503, description = "no swarm queue is configured on this host, or it is \
|
||||
not connected (problem+json)", body = String),
|
||||
(status = 500, description = "the flows could not be read (problem+json)",
|
||||
body = String),
|
||||
),
|
||||
tag = "notify"
|
||||
)]
|
||||
pub(crate) async fn get_flows(
|
||||
State(state): State<AppState>,
|
||||
) -> Result<Json<FlowsView>, ProblemDetails> {
|
||||
let client = queue(&state).map_err(refused)?;
|
||||
let store = open(&client).await.map_err(|e| queue_problem(&e))?;
|
||||
let (stored, revision) = stored(&store).await.map_err(|e| queue_problem(&e))?;
|
||||
match stored {
|
||||
Stored::Nothing => Ok(Json(FlowsView::of(
|
||||
Document::default(),
|
||||
revision.unwrap_or(0),
|
||||
))),
|
||||
Stored::Document(doc) => Ok(Json(FlowsView::of(doc, revision.unwrap_or(0)))),
|
||||
Stored::Unreadable(e) => Err(unreadable(&e)),
|
||||
}
|
||||
}
|
||||
|
||||
fn unreadable(why: &DecodeError) -> ProblemDetails {
|
||||
error_problem(
|
||||
StatusCode::CONFLICT,
|
||||
&format!("the stored flows document is not one this build reads: {why}"),
|
||||
)
|
||||
}
|
||||
|
||||
/// Replace the flows.
|
||||
#[utoipa::path(
|
||||
put,
|
||||
path = "/api/notify/flows",
|
||||
request_body = PutFlows,
|
||||
responses(
|
||||
(status = 200, description = "the document as stored; the router applies it from \
|
||||
its next event", body = FlowsView),
|
||||
(status = 400, description = "not JSON or not this shape, or the document fails its \
|
||||
check: a `disabled_builtins` entry that is no built-in, a custom flow whose id \
|
||||
is another's or a built-in's, a flow that is not a tree (one `source`, one \
|
||||
parent for every other node, every node reachable, no children under a \
|
||||
`todo`), more than 64 nodes, 16 levels or 100 flows, an id that is not 1 to 64 \
|
||||
letters, digits, `-` or `_`, a source event the router is not sent, a condition \
|
||||
or recipient field that is no event field, or an upsert with no text \
|
||||
(problem+json)", body = String),
|
||||
(status = 409, description = "the stored document is not at `revision` (another write landed, or one now exists where the client read none), or the stored \
|
||||
document is a version this build does not read, which a `PUT` would erase \
|
||||
(problem+json)", body = String),
|
||||
(status = 503, description = "no swarm queue is configured on this host, or it is \
|
||||
not connected (problem+json)", body = String),
|
||||
(status = 500, description = "the flows could not be written (problem+json)",
|
||||
body = String),
|
||||
),
|
||||
tag = "notify"
|
||||
)]
|
||||
pub(crate) async fn put_flows(
|
||||
State(state): State<AppState>,
|
||||
body: Result<Json<PutFlows>, JsonRejection>,
|
||||
) -> Result<Json<FlowsView>, ProblemDetails> {
|
||||
let Json(put) = body.map_err(|e| error_problem(StatusCode::BAD_REQUEST, &e.body_text()))?;
|
||||
let doc = Document {
|
||||
version: notify_flows::VERSION,
|
||||
disabled_builtins: put.disabled_builtins,
|
||||
flows: put.flows,
|
||||
};
|
||||
doc.check()
|
||||
.map_err(|why| error_problem(StatusCode::BAD_REQUEST, &why))?;
|
||||
let client = queue(&state).map_err(refused)?;
|
||||
let store = open(&client).await.map_err(|e| queue_problem(&e))?;
|
||||
for _ in 0..MAX_ATTEMPTS {
|
||||
let (current, revision) = stored(&store).await.map_err(|e| queue_problem(&e))?;
|
||||
// An unreadable document is replaced only when it is not a version
|
||||
// this build does not know: a newer build's document is kept for it.
|
||||
if let Stored::Unreadable(why @ DecodeError::UnknownVersion(_)) = ¤t {
|
||||
return Err(unreadable(why));
|
||||
}
|
||||
if put.revision != revision.unwrap_or(0) {
|
||||
return Err(error_problem(
|
||||
StatusCode::CONFLICT,
|
||||
&format!(
|
||||
"the flows are not at revision {}; read them again",
|
||||
put.revision
|
||||
),
|
||||
));
|
||||
}
|
||||
if let Some(written) = write_revision(&store, notify_flows::KEY, &doc, revision)
|
||||
.await
|
||||
.map_err(|e| queue_problem(&e))?
|
||||
{
|
||||
tracing::info!(
|
||||
disabled = ?doc.disabled_builtins,
|
||||
custom = doc.flows.len(),
|
||||
"set the notification router's flows"
|
||||
);
|
||||
return Ok(Json(FlowsView::of(doc, written)));
|
||||
}
|
||||
}
|
||||
Err(queue_problem(&anyhow::anyhow!(
|
||||
"gave up writing the notification flows after {MAX_ATTEMPTS} conflicting writes"
|
||||
)))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use swarm_wire::notify_flows::{Edge, Node, NodeKind, Recipients, TodoSink};
|
||||
use swarm_wire::router_todo::Op;
|
||||
|
||||
use super::*;
|
||||
use crate::test_nats::TestNats;
|
||||
|
||||
fn custom(id: &str) -> Flow {
|
||||
Flow {
|
||||
id: id.to_owned(),
|
||||
name: "new to triage".to_owned(),
|
||||
enabled: true,
|
||||
nodes: vec![
|
||||
Node {
|
||||
id: "source".to_owned(),
|
||||
kind: NodeKind::Source {
|
||||
events: vec!["issues".to_owned()],
|
||||
},
|
||||
},
|
||||
Node {
|
||||
id: "sink".to_owned(),
|
||||
kind: NodeKind::Todo(TodoSink {
|
||||
to: Recipients::Agent("triage".to_owned()),
|
||||
op: Op::Upsert,
|
||||
key: "forge:{repo}#{number}:triage".to_owned(),
|
||||
prefix: false,
|
||||
text: "triage {repo}#{number}".to_owned(),
|
||||
}),
|
||||
},
|
||||
],
|
||||
edges: vec![Edge {
|
||||
from: "source".to_owned(),
|
||||
to: "sink".to_owned(),
|
||||
}],
|
||||
}
|
||||
}
|
||||
|
||||
fn body(revision: u64, disabled: &[&str], flows: Vec<Flow>) -> Json<PutFlows> {
|
||||
Json(PutFlows {
|
||||
revision,
|
||||
disabled_builtins: disabled.iter().map(|s| (*s).to_owned()).collect(),
|
||||
flows,
|
||||
})
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn the_flows_are_read_replaced_and_checked_on_save() {
|
||||
let nats = TestNats::start();
|
||||
let client = nats.client().await;
|
||||
let state = crate::agent_limits::tests::state(&client);
|
||||
|
||||
// Nothing stored: the built-ins alone, none off, no revision.
|
||||
let Json(view) = get_flows(State(state.clone())).await.expect("get");
|
||||
assert_eq!(view.revision, 0);
|
||||
assert!(view.disabled_builtins.is_empty() && view.flows.is_empty());
|
||||
assert_eq!(view.builtins.len(), 3);
|
||||
|
||||
let Json(saved) = put_flows(
|
||||
State(state.clone()),
|
||||
Ok(body(0, &["unassigned"], vec![custom("triage")])),
|
||||
)
|
||||
.await
|
||||
.expect("put");
|
||||
let revision = saved.revision;
|
||||
assert_ne!(revision, 0);
|
||||
let Json(read) = get_flows(State(state.clone())).await.expect("get");
|
||||
assert_eq!(read, saved);
|
||||
assert_eq!(read.disabled_builtins, ["unassigned"]);
|
||||
assert_eq!(read.flows[0].id, "triage");
|
||||
|
||||
// The stored bytes are the document the router decodes.
|
||||
let store = open(&client).await.expect("bucket");
|
||||
let bytes = store
|
||||
.get(notify_flows::KEY)
|
||||
.await
|
||||
.expect("read")
|
||||
.expect("stored");
|
||||
let doc = Document::decode(&bytes).expect("the router reads it");
|
||||
assert_eq!(doc.active().0.len(), 3, "2 built-ins and the custom flow");
|
||||
|
||||
// A refused save changes nothing: an orphan node, and a built-in that does not exist.
|
||||
let mut orphan = custom("orphan");
|
||||
orphan.edges.clear();
|
||||
let refused = put_flows(State(state.clone()), Ok(body(revision, &[], vec![orphan])))
|
||||
.await
|
||||
.expect_err("an orphan node");
|
||||
assert_eq!(refused.status, Some(StatusCode::BAD_REQUEST), "{refused:?}");
|
||||
let unknown = put_flows(State(state.clone()), Ok(body(revision, &["nope"], vec![])))
|
||||
.await
|
||||
.expect_err("no such built-in");
|
||||
assert_eq!(unknown.status, Some(StatusCode::BAD_REQUEST), "{unknown:?}");
|
||||
let Json(after) = get_flows(State(state.clone())).await.expect("get");
|
||||
assert_eq!(after, saved);
|
||||
|
||||
// A stale revision is refused; the current one lands.
|
||||
let outdated = put_flows(State(state.clone()), Ok(body(revision + 100, &[], vec![])))
|
||||
.await
|
||||
.expect_err("stale");
|
||||
assert_eq!(outdated.status, Some(StatusCode::CONFLICT), "{outdated:?}");
|
||||
let Json(next) = put_flows(State(state.clone()), Ok(body(revision, &[], vec![])))
|
||||
.await
|
||||
.expect("current revision");
|
||||
assert!(next.revision > revision);
|
||||
assert!(next.flows.is_empty());
|
||||
}
|
||||
|
||||
/// A document of a version this build does not know is neither served
|
||||
/// nor replaced, and a garbled one is replaced.
|
||||
#[tokio::test]
|
||||
async fn a_document_of_another_version_is_kept_and_a_garbled_one_is_replaced() {
|
||||
let nats = TestNats::start();
|
||||
let client = nats.client().await;
|
||||
let state = crate::agent_limits::tests::state(&client);
|
||||
let store = open(&client).await.expect("bucket");
|
||||
|
||||
store
|
||||
.put(
|
||||
notify_flows::KEY,
|
||||
br#"{"version":2,"flows":[]}"#.to_vec().into(),
|
||||
)
|
||||
.await
|
||||
.expect("seed");
|
||||
let got = get_flows(State(state.clone()))
|
||||
.await
|
||||
.expect_err("version 2");
|
||||
assert_eq!(got.status, Some(StatusCode::CONFLICT), "{got:?}");
|
||||
let refused = put_flows(State(state.clone()), Ok(body(0, &[], vec![])))
|
||||
.await
|
||||
.expect_err("version 2 is kept");
|
||||
assert_eq!(refused.status, Some(StatusCode::CONFLICT), "{refused:?}");
|
||||
let kept = store
|
||||
.get(notify_flows::KEY)
|
||||
.await
|
||||
.expect("read")
|
||||
.expect("stored");
|
||||
assert_eq!(&kept[..], br#"{"version":2,"flows":[]}"#);
|
||||
|
||||
// Control: garbage that names no version is replaced by a save.
|
||||
let garbage = store
|
||||
.put(notify_flows::KEY, b"garbage".to_vec().into())
|
||||
.await
|
||||
.expect("seed");
|
||||
let _ = put_flows(State(state.clone()), Ok(body(garbage, &["closed"], vec![])))
|
||||
.await
|
||||
.expect("replaces garbage");
|
||||
let Json(view) = get_flows(State(state)).await.expect("readable again");
|
||||
assert_eq!(view.disabled_builtins, ["closed"]);
|
||||
}
|
||||
|
||||
/// A page that read no document cannot erase one written since: its
|
||||
/// `revision: 0` means "create", and the document exists by then.
|
||||
#[tokio::test]
|
||||
async fn a_page_that_read_nothing_cannot_erase_a_flow_written_after() {
|
||||
let nats = TestNats::start();
|
||||
let client = nats.client().await;
|
||||
let state = crate::agent_limits::tests::state(&client);
|
||||
|
||||
let Json(page) = get_flows(State(state.clone())).await.expect("get");
|
||||
assert_eq!(page.revision, 0);
|
||||
// The API writes a custom flow while the page is open.
|
||||
let Json(api) = put_flows(
|
||||
State(state.clone()),
|
||||
Ok(body(0, &[], vec![custom("triage")])),
|
||||
)
|
||||
.await
|
||||
.expect("the API's write");
|
||||
// The page's first switch carries what it read.
|
||||
let erased = put_flows(
|
||||
State(state.clone()),
|
||||
Ok(body(page.revision, &["closed"], vec![])),
|
||||
)
|
||||
.await
|
||||
.expect_err("the page read nothing, and a document exists");
|
||||
assert_eq!(erased.status, Some(StatusCode::CONFLICT), "{erased:?}");
|
||||
let Json(after) = get_flows(State(state.clone())).await.expect("get");
|
||||
assert_eq!(after, api);
|
||||
assert_eq!(after.flows[0].id, "triage");
|
||||
|
||||
// Control: re-read, the same switch lands and keeps the flow.
|
||||
let Json(switched) = put_flows(
|
||||
State(state),
|
||||
Ok(body(after.revision, &["closed"], after.flows.clone())),
|
||||
)
|
||||
.await
|
||||
.expect("the switch with the revision it read");
|
||||
assert_eq!(switched.disabled_builtins, ["closed"]);
|
||||
assert_eq!(switched.flows[0].id, "triage");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_template_naming_no_event_field_is_refused_on_save() {
|
||||
let nats = TestNats::start();
|
||||
let client = nats.client().await;
|
||||
let state = crate::agent_limits::tests::state(&client);
|
||||
|
||||
let mut typo = custom("typo");
|
||||
if let NodeKind::Todo(sink) = &mut typo.nodes[1].kind {
|
||||
sink.key = "forge:{repo}#{numbr}:triage".to_owned();
|
||||
}
|
||||
let refused = put_flows(State(state.clone()), Ok(body(0, &[], vec![typo])))
|
||||
.await
|
||||
.expect_err("{numbr} is no field");
|
||||
assert_eq!(refused.status, Some(StatusCode::BAD_REQUEST), "{refused:?}");
|
||||
// Control: the same flow spelled right is stored.
|
||||
let _ = put_flows(State(state), Ok(body(0, &[], vec![custom("typo")])))
|
||||
.await
|
||||
.expect("{number} is a field");
|
||||
}
|
||||
}
|
||||
|
|
@ -31,8 +31,8 @@ use crate::{
|
|||
agent_control, agent_icon, agent_input, agent_limits, agent_login, agent_permissions,
|
||||
agent_renewal, agent_state_stream, agent_status, agents_md, alerts, archetypes, auth, builder,
|
||||
claude_login, config_pr, forge, forge_account, github_account, inference, issue_report,
|
||||
linked_accounts, matrix_account, message_stream, pulls, queue_identity, review_classifier,
|
||||
schedules, status, subagent_term, term_stream, wanted, webhook,
|
||||
linked_accounts, matrix_account, message_stream, notify_flows, pulls, queue_identity,
|
||||
review_classifier, schedules, status, subagent_term, term_stream, wanted, webhook,
|
||||
};
|
||||
|
||||
/// Where the daemon binds, overridable via `SWARM_CONTROLLER_SOCKET`.
|
||||
|
|
@ -76,6 +76,7 @@ pub(crate) fn socket_path() -> PathBuf {
|
|||
(name = "repos", description = "forge repo browsing (swarm-ui's issue-report page)"),
|
||||
(name = "forge", description = "forge site admins (`swarmctl forge make-admin`)"),
|
||||
(name = "schedules", description = "scheduled prompts (`swarmctl schedule`)"),
|
||||
(name = "notify", description = "the notification router's flows: built-ins switched off, and custom flows"),
|
||||
(name = "review-classifier", description = "trying review classifications against a PR (swarm-ui's try-out page), and the classifier's swarm-wide settings"),
|
||||
(name = "pulls", description = "open pull requests, for the try-out page's dropdowns"),
|
||||
(name = "alerts", description = "the swarm Grafana's current alerts (swarm-ui's alerts page)"),
|
||||
|
|
@ -635,6 +636,7 @@ pub(crate) fn build_app(state: AppState) -> axum::Router {
|
|||
review_classifier::settings::get_settings,
|
||||
review_classifier::settings::put_settings
|
||||
))
|
||||
.routes(routes!(notify_flows::get_flows, notify_flows::put_flows))
|
||||
.routes(routes!(webhook::post_webhook_forge))
|
||||
.split_for_parts();
|
||||
// Just the JSON, not the UI — Swagger UI itself is nginx-hosted from
|
||||
|
|
|
|||
|
|
@ -317,7 +317,7 @@ impl Policy {
|
|||
|
||||
/// What the notification router may do: drain its own consumer on the
|
||||
/// forge events stream, queue todo ops for any agent, keep its own state
|
||||
/// bucket, and list which agents exist.
|
||||
/// bucket, read the flows document, and list which agents exist.
|
||||
///
|
||||
/// - **Forge events.** Open or create the stream, create its one durable
|
||||
/// in the extended form whose last tokens are the filter (the server
|
||||
|
|
@ -330,14 +330,18 @@ impl Policy {
|
|||
/// subjects, which is what makes the router their sole writer.
|
||||
/// - **Its state bucket.** Create, look up, read and write
|
||||
/// `notify-router-state`, and no other bucket.
|
||||
/// - **The flows document.** `STREAM.INFO` and `DIRECT.GET` on
|
||||
/// `notify-flows`, which it reads and never creates or writes: the
|
||||
/// controller creates the bucket and is its one writer.
|
||||
/// - **The roster.** `STREAM.INFO` on the agent-status bucket, which lists
|
||||
/// its keys (`<hive>.<agent>`) and writes none.
|
||||
fn notify_router_permissions(client_id: &str) -> Permissions {
|
||||
use swarm_wire::{forge_events, notify_router, router_todo};
|
||||
use swarm_wire::{forge_events, notify_flows, notify_router, router_todo};
|
||||
|
||||
let events = forge_events::STREAM;
|
||||
let consumer = forge_events::ROUTER_CONSUMER;
|
||||
let state = format!("KV_{}", notify_router::STATE_BUCKET);
|
||||
let flows = format!("KV_{}", notify_flows::BUCKET);
|
||||
let mut publish = Self::jetstream_minimum().to_vec();
|
||||
publish.extend([
|
||||
format!("$JS.API.STREAM.INFO.{events}"),
|
||||
|
|
@ -355,6 +359,8 @@ impl Policy {
|
|||
format!("$JS.API.STREAM.CREATE.{state}"),
|
||||
format!("$JS.API.DIRECT.GET.{state}.>"),
|
||||
format!("$KV.{}.>", notify_router::STATE_BUCKET),
|
||||
format!("$JS.API.STREAM.INFO.{flows}"),
|
||||
format!("$JS.API.DIRECT.GET.{flows}.>"),
|
||||
format!("$JS.API.STREAM.INFO.{}", Self::agent_status_stream()),
|
||||
Self::inbox(client_id),
|
||||
]);
|
||||
|
|
@ -1799,7 +1805,7 @@ mod tests {
|
|||
#[test]
|
||||
fn the_notify_router_drains_its_own_consumer_and_writes_only_its_own_state() {
|
||||
use swarm_queue_client::client_inbox_prefix;
|
||||
use swarm_wire::{forge_events, notify_router, router_todo};
|
||||
use swarm_wire::{forge_events, notify_flows, notify_router, router_todo};
|
||||
|
||||
let p = with_router()
|
||||
.permissions(NOTIFY_ROUTER)
|
||||
|
|
@ -1807,6 +1813,7 @@ mod tests {
|
|||
let may = |subject: &str| p.publish.iter().any(|g| covers(g, subject));
|
||||
let events = forge_events::STREAM;
|
||||
let state = notify_router::STATE_BUCKET;
|
||||
let flows = notify_flows::BUCKET;
|
||||
for allowed in [
|
||||
format!(
|
||||
"$JS.API.CONSUMER.CREATE.{events}.notify-router.{}",
|
||||
|
|
@ -1820,6 +1827,12 @@ mod tests {
|
|||
format!("$KV.{state}.42.5571"),
|
||||
format!("$JS.API.DIRECT.GET.KV_{state}.$KV.{state}.42.5571"),
|
||||
"$JS.API.STREAM.INFO.KV_agent-status".to_owned(),
|
||||
// The flows document: read it.
|
||||
format!("$JS.API.STREAM.INFO.KV_{flows}"),
|
||||
format!(
|
||||
"$JS.API.DIRECT.GET.KV_{flows}.$KV.{flows}.{}",
|
||||
notify_flows::KEY
|
||||
),
|
||||
] {
|
||||
assert!(may(&allowed), "the router may not reach {allowed}");
|
||||
}
|
||||
|
|
@ -1838,6 +1851,10 @@ mod tests {
|
|||
forge_events::subject("issues"),
|
||||
"$KV.agent-status.alpha.atlas".to_owned(),
|
||||
"$KV.agent-schedules.atlas".to_owned(),
|
||||
// The flows document: not write it, nor make the bucket.
|
||||
format!("$KV.{flows}.{}", notify_flows::KEY),
|
||||
format!("$JS.API.STREAM.CREATE.KV_{flows}"),
|
||||
format!("$JS.API.STREAM.DELETE.KV_{flows}"),
|
||||
swarm_queue_client::agent_input_subject("atlas"),
|
||||
"$JS.API.STREAM.CREATE.KV_agent-status".to_owned(),
|
||||
] {
|
||||
|
|
|
|||
|
|
@ -163,6 +163,22 @@ mod tests {
|
|||
}
|
||||
}
|
||||
|
||||
/// The names a stored flow may read are the ones `fields` produces.
|
||||
#[test]
|
||||
fn the_fields_are_the_ones_a_stored_flow_may_name() {
|
||||
let d = Delivery::parse(&issue_assign("assigned", &["atlas"])).expect("parses");
|
||||
for event in ["issues", "issue_assign"] {
|
||||
let mut got: Vec<&str> = fields(event, &d, &Snapshot::default())
|
||||
.keys()
|
||||
.copied()
|
||||
.collect();
|
||||
let mut want = swarm_wire::notify_flows::FIELDS.to_vec();
|
||||
got.sort_unstable();
|
||||
want.sort_unstable();
|
||||
assert_eq!(got, want, "{event}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_issue_delivery_reads_back_its_fields() {
|
||||
let d = Delivery::parse(&issue_assign("assigned", &["atlas", "iris"])).expect("parses");
|
||||
|
|
|
|||
|
|
@ -1,26 +1,17 @@
|
|||
//! Flows: what the router does with an event.
|
||||
//! Running a flow over one event.
|
||||
//!
|
||||
//! A flow is a graph of nodes joined by edges, rooted at one `source` node.
|
||||
//! A `match` node passes when every one of its conditions holds, and then
|
||||
//! each of its children runs, so one event can reach several branches. A
|
||||
//! `sink` node is a leaf and emits one todo op per recipient.
|
||||
//!
|
||||
//! Flows are trees: one source, one parent for every other node, every node
|
||||
//! reachable from the source. [`Flow::validate`] checks that, and the router
|
||||
//! runs only flows that pass.
|
||||
//!
|
||||
//! The built-in flows ([`builtins`]) are ordinary flows defined in code.
|
||||
//! The flow types, their checks and the built-in flows are
|
||||
//! `swarm_wire::notify_flows`; the built-in and the custom flows are the same
|
||||
//! type and this walk runs both. A `match` node passes when every one of its
|
||||
//! conditions holds, and then each of its children runs, so one event can
|
||||
//! reach several branches. A `todo` node emits one op per recipient. The
|
||||
//! router runs only flows that pass [`Flow::validate`].
|
||||
|
||||
use std::collections::{BTreeMap, BTreeSet};
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
use swarm_wire::notify_flows::{Condition, Flow, Node, NodeKind, Recipients, TodoSink};
|
||||
use swarm_wire::router_todo::{Op, RouterTodoOp};
|
||||
|
||||
/// The most nodes one flow may have.
|
||||
const MAX_NODES: usize = 64;
|
||||
|
||||
/// The deepest a node may sit below the source.
|
||||
const MAX_DEPTH: usize = 16;
|
||||
|
||||
/// One event, as the fields a flow reads.
|
||||
pub type Fields = BTreeMap<&'static str, Value>;
|
||||
|
||||
|
|
@ -31,56 +22,6 @@ pub enum Value {
|
|||
List(Vec<String>),
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct Flow {
|
||||
pub id: String,
|
||||
pub nodes: Vec<Node>,
|
||||
pub edges: Vec<Edge>,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct Node {
|
||||
pub id: String,
|
||||
pub kind: NodeKind,
|
||||
}
|
||||
|
||||
/// A directed edge, parent to child.
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct Edge {
|
||||
pub from: String,
|
||||
pub to: String,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug)]
|
||||
pub enum NodeKind {
|
||||
/// The root: the forge event types the flow starts from.
|
||||
Source { events: Vec<String> },
|
||||
/// Passes when every condition holds.
|
||||
Match { when: Vec<Condition> },
|
||||
/// Emits a todo op to each recipient.
|
||||
Todo(TodoSink),
|
||||
}
|
||||
|
||||
/// `field` equals `value`. A list field holds when one of its items does.
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct Condition {
|
||||
pub field: String,
|
||||
pub equals: String,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct TodoSink {
|
||||
/// The field naming the recipients, a list of agent names.
|
||||
pub to: String,
|
||||
pub op: Op,
|
||||
/// The todo key, with `{field}` placeholders.
|
||||
pub key: String,
|
||||
/// For a clear: `key` is a prefix.
|
||||
pub prefix: bool,
|
||||
/// The todo text, with `{field}` placeholders. Unused for a clear.
|
||||
pub text: String,
|
||||
}
|
||||
|
||||
/// One op the walk emitted, for one recipient.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct Action {
|
||||
|
|
@ -90,130 +31,72 @@ pub struct Action {
|
|||
pub op: RouterTodoOp,
|
||||
}
|
||||
|
||||
impl Flow {
|
||||
/// Check the flow is a tree within the caps, or say why not.
|
||||
pub fn validate(&self) -> Result<(), String> {
|
||||
if self.nodes.len() > MAX_NODES {
|
||||
return Err(format!("{} nodes, more than {MAX_NODES}", self.nodes.len()));
|
||||
}
|
||||
let mut ids = BTreeSet::new();
|
||||
for node in &self.nodes {
|
||||
if !ids.insert(node.id.as_str()) {
|
||||
return Err(format!("node id {} is used twice", node.id));
|
||||
}
|
||||
}
|
||||
for edge in &self.edges {
|
||||
for end in [&edge.from, &edge.to] {
|
||||
if !ids.contains(end.as_str()) {
|
||||
return Err(format!("an edge names {end}, which is no node"));
|
||||
/// Walk `flow` for one event of type `event_type` and return the ops it
|
||||
/// emits. Assumes [`Flow::validate`] passed.
|
||||
pub fn evaluate(flow: &Flow, event_type: &str, fields: &Fields) -> Vec<Action> {
|
||||
let Some(root) = flow
|
||||
.nodes
|
||||
.iter()
|
||||
.find(|n| matches!(&n.kind, NodeKind::Source { events } if events.iter().any(|e| e == event_type)))
|
||||
else {
|
||||
return Vec::new();
|
||||
};
|
||||
let mut actions = Vec::new();
|
||||
let mut stack = vec![root.id.as_str()];
|
||||
while let Some(id) = stack.pop() {
|
||||
let Some(node) = flow.node(id) else { continue };
|
||||
match &node.kind {
|
||||
NodeKind::Source { .. } => stack.extend(flow.children(id)),
|
||||
NodeKind::Match { when } => {
|
||||
if when.iter().all(|c| holds(c, fields)) {
|
||||
stack.extend(flow.children(id));
|
||||
}
|
||||
}
|
||||
NodeKind::Todo(sink) => actions.extend(emit(flow, node, sink, fields)),
|
||||
}
|
||||
let roots: Vec<&Node> = self
|
||||
.nodes
|
||||
.iter()
|
||||
.filter(|n| matches!(n.kind, NodeKind::Source { .. }))
|
||||
.collect();
|
||||
let [root] = roots.as_slice() else {
|
||||
return Err(format!("{} source nodes, not one", roots.len()));
|
||||
};
|
||||
for node in &self.nodes {
|
||||
if matches!(node.kind, NodeKind::Todo(_)) && !self.children(&node.id).is_empty() {
|
||||
return Err(format!("sink {} has children", node.id));
|
||||
}
|
||||
}
|
||||
// From the one root, every node reached exactly once: a node reached
|
||||
// twice has a second parent or sits on a cycle, and one not reached is
|
||||
// cut off.
|
||||
let mut reached = 0;
|
||||
let mut stack = vec![(root.id.as_str(), 0)];
|
||||
while let Some((id, depth)) = stack.pop() {
|
||||
if depth > MAX_DEPTH {
|
||||
return Err(format!("node {id} is deeper than {MAX_DEPTH}"));
|
||||
}
|
||||
reached += 1;
|
||||
if reached > self.nodes.len() {
|
||||
return Err(
|
||||
"a node is reached twice: it has two parents or sits on a cycle".to_owned(),
|
||||
);
|
||||
}
|
||||
stack.extend(self.children(id).into_iter().map(|c| (c, depth + 1)));
|
||||
}
|
||||
if reached != self.nodes.len() {
|
||||
return Err(format!(
|
||||
"{} of {} nodes are not reachable from the source",
|
||||
self.nodes.len() - reached,
|
||||
self.nodes.len()
|
||||
));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
actions
|
||||
}
|
||||
|
||||
fn children(&self, id: &str) -> Vec<&str> {
|
||||
self.edges
|
||||
.iter()
|
||||
.filter(|e| e.from == id)
|
||||
.map(|e| e.to.as_str())
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn node(&self, id: &str) -> Option<&Node> {
|
||||
self.nodes.iter().find(|n| n.id == id)
|
||||
}
|
||||
|
||||
/// Walk the flow for one event of type `event_type` and return the ops it
|
||||
/// emits. Assumes [`Self::validate`] passed.
|
||||
pub fn evaluate(&self, event_type: &str, fields: &Fields) -> Vec<Action> {
|
||||
let Some(root) = self
|
||||
.nodes
|
||||
.iter()
|
||||
.find(|n| matches!(&n.kind, NodeKind::Source { events } if events.iter().any(|e| e == event_type)))
|
||||
else {
|
||||
return Vec::new();
|
||||
};
|
||||
let mut actions = Vec::new();
|
||||
let mut stack = vec![root.id.as_str()];
|
||||
while let Some(id) = stack.pop() {
|
||||
let Some(node) = self.node(id) else { continue };
|
||||
match &node.kind {
|
||||
NodeKind::Source { .. } => stack.extend(self.children(id)),
|
||||
NodeKind::Match { when } => {
|
||||
if when.iter().all(|c| holds(c, fields)) {
|
||||
stack.extend(self.children(id));
|
||||
}
|
||||
}
|
||||
NodeKind::Todo(sink) => actions.extend(self.emit(node, sink, fields)),
|
||||
}
|
||||
}
|
||||
actions
|
||||
}
|
||||
|
||||
fn emit(&self, node: &Node, sink: &TodoSink, fields: &Fields) -> Vec<Action> {
|
||||
let recipients = match fields.get(sink.to.as_str()) {
|
||||
fn emit(flow: &Flow, node: &Node, sink: &TodoSink, fields: &Fields) -> Vec<Action> {
|
||||
let recipients = match &sink.to {
|
||||
Recipients::Agent(agent) => vec![agent.clone()],
|
||||
Recipients::Field(name) => match fields.get(name.as_str()) {
|
||||
Some(Value::List(items)) => items.clone(),
|
||||
Some(Value::Text(one)) => vec![one.clone()],
|
||||
None => Vec::new(),
|
||||
},
|
||||
};
|
||||
let rendered = expand(&sink.key, fields).and_then(|key| {
|
||||
let summary = match sink.op {
|
||||
Op::Upsert => expand(&sink.text, fields)?,
|
||||
Op::Clear => String::new(),
|
||||
};
|
||||
let op = RouterTodoOp {
|
||||
op: sink.op,
|
||||
key: expand(&sink.key, fields),
|
||||
prefix: sink.op == Op::Clear && sink.prefix,
|
||||
summary: match sink.op {
|
||||
Op::Upsert => expand(&sink.text, fields),
|
||||
Op::Clear => String::new(),
|
||||
},
|
||||
source: format!("router:{}", self.id),
|
||||
};
|
||||
recipients
|
||||
.into_iter()
|
||||
.map(|agent| Action {
|
||||
flow: self.id.clone(),
|
||||
node: node.id.clone(),
|
||||
agent,
|
||||
op: op.clone(),
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
Ok((key, summary))
|
||||
});
|
||||
let (key, summary) = match rendered {
|
||||
Ok(done) => done,
|
||||
Err(why) => {
|
||||
tracing::warn!(flow = %flow.id, node = %node.id, %why, "notify router: a todo node names a placeholder the event lacks; nothing sent");
|
||||
return Vec::new();
|
||||
}
|
||||
};
|
||||
let op = RouterTodoOp {
|
||||
op: sink.op,
|
||||
key,
|
||||
prefix: sink.op == Op::Clear && sink.prefix,
|
||||
summary,
|
||||
source: format!("router:{}", flow.id),
|
||||
};
|
||||
recipients
|
||||
.into_iter()
|
||||
.map(|agent| Action {
|
||||
flow: flow.id.clone(),
|
||||
node: node.id.clone(),
|
||||
agent,
|
||||
op: op.clone(),
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn holds(condition: &Condition, fields: &Fields) -> bool {
|
||||
|
|
@ -224,9 +107,9 @@ fn holds(condition: &Condition, fields: &Fields) -> bool {
|
|||
}
|
||||
}
|
||||
|
||||
/// `template` with each `{field}` replaced by that text field's value. A list
|
||||
/// field is joined with `, `; an unknown name is left as written.
|
||||
pub fn expand(template: &str, fields: &Fields) -> String {
|
||||
/// `template` with each `{field}` replaced by that field's value, a list
|
||||
/// joined with `, `. A name that is no field is an error naming it.
|
||||
pub fn expand(template: &str, fields: &Fields) -> Result<String, String> {
|
||||
let mut out = String::with_capacity(template.len());
|
||||
let mut rest = template;
|
||||
while let Some(open) = rest.find('{') {
|
||||
|
|
@ -234,109 +117,24 @@ pub fn expand(template: &str, fields: &Fields) -> String {
|
|||
let after = &rest[open + 1..];
|
||||
let Some(close) = after.find('}') else {
|
||||
out.push_str(&rest[open..]);
|
||||
return out;
|
||||
return Ok(out);
|
||||
};
|
||||
let name = &after[..close];
|
||||
match fields.get(name) {
|
||||
Some(Value::Text(v)) => out.push_str(v),
|
||||
Some(Value::List(items)) => out.push_str(&items.join(", ")),
|
||||
None => {
|
||||
out.push('{');
|
||||
out.push_str(name);
|
||||
out.push('}');
|
||||
}
|
||||
None => return Err(format!("{{{name}}} is no event field")),
|
||||
}
|
||||
rest = &after[close + 1..];
|
||||
}
|
||||
out.push_str(rest);
|
||||
out
|
||||
}
|
||||
|
||||
/// The flows the router ships with.
|
||||
///
|
||||
/// - `assigned`: a newly assigned agent gets a todo for the issue or PR.
|
||||
/// - `unassigned`: an agent taken off it gets that todo rewritten to say so.
|
||||
/// Both write the same key, so the second replaces the first.
|
||||
/// - `closed`: closing it clears every router todo the router queued for it.
|
||||
pub fn builtins() -> Vec<Flow> {
|
||||
let assignment_key = "forge:{repo}#{number}:assignment";
|
||||
vec![
|
||||
chain(
|
||||
"assigned",
|
||||
&["issue_assign", "pull_request_assign"],
|
||||
"assigned",
|
||||
TodoSink {
|
||||
to: "assignees_added".to_owned(),
|
||||
op: Op::Upsert,
|
||||
key: assignment_key.to_owned(),
|
||||
prefix: false,
|
||||
text: "you are assigned to {kind} {repo}#{number}: {title} ({url})".to_owned(),
|
||||
},
|
||||
),
|
||||
chain(
|
||||
"unassigned",
|
||||
&["issue_assign", "pull_request_assign"],
|
||||
"unassigned",
|
||||
TodoSink {
|
||||
to: "assignees_removed".to_owned(),
|
||||
op: Op::Upsert,
|
||||
key: assignment_key.to_owned(),
|
||||
prefix: false,
|
||||
text: "you are not assigned to {kind} {repo}#{number} anymore: {title} ({url})"
|
||||
.to_owned(),
|
||||
},
|
||||
),
|
||||
chain(
|
||||
"closed",
|
||||
&["issues", "pull_request"],
|
||||
"closed",
|
||||
TodoSink {
|
||||
to: "todo_holders".to_owned(),
|
||||
op: Op::Clear,
|
||||
key: "forge:{repo}#{number}:".to_owned(),
|
||||
prefix: true,
|
||||
text: String::new(),
|
||||
},
|
||||
),
|
||||
]
|
||||
}
|
||||
|
||||
/// source(`events`) → match(`action` = `action`) → `sink`.
|
||||
fn chain(id: &str, events: &[&str], action: &str, sink: TodoSink) -> Flow {
|
||||
let node = |id: &str, kind| Node {
|
||||
id: id.to_owned(),
|
||||
kind,
|
||||
};
|
||||
let edge = |from: &str, to: &str| Edge {
|
||||
from: from.to_owned(),
|
||||
to: to.to_owned(),
|
||||
};
|
||||
Flow {
|
||||
id: id.to_owned(),
|
||||
nodes: vec![
|
||||
node(
|
||||
"source",
|
||||
NodeKind::Source {
|
||||
events: events.iter().map(|e| (*e).to_owned()).collect(),
|
||||
},
|
||||
),
|
||||
node(
|
||||
"match",
|
||||
NodeKind::Match {
|
||||
when: vec![Condition {
|
||||
field: "action".to_owned(),
|
||||
equals: action.to_owned(),
|
||||
}],
|
||||
},
|
||||
),
|
||||
node("sink", NodeKind::Todo(sink)),
|
||||
],
|
||||
edges: vec![edge("source", "match"), edge("match", "sink")],
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use swarm_wire::notify_flows::{Edge, builtins};
|
||||
|
||||
use super::*;
|
||||
|
||||
fn text(v: &str) -> Value {
|
||||
|
|
@ -368,16 +166,11 @@ mod tests {
|
|||
.expect("a built-in")
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn every_builtin_is_a_valid_tree() {
|
||||
for f in builtins() {
|
||||
assert_eq!(f.validate(), Ok(()), "{}", f.id);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn assigning_queues_a_todo_for_each_newly_assigned_agent_only() {
|
||||
let got = flow("assigned").evaluate(
|
||||
let assigned = flow("assigned");
|
||||
let got = evaluate(
|
||||
&assigned,
|
||||
"issue_assign",
|
||||
&event("assigned", &["atlas", "iris"], &[], &[]),
|
||||
);
|
||||
|
|
@ -392,21 +185,28 @@ mod tests {
|
|||
assert_eq!(got[0].op.source, "router:assigned");
|
||||
// The match gates: an unassign event reaches no assigned sink.
|
||||
assert!(
|
||||
flow("assigned")
|
||||
.evaluate("issue_assign", &event("unassigned", &["atlas"], &[], &[]))
|
||||
.is_empty()
|
||||
evaluate(
|
||||
&assigned,
|
||||
"issue_assign",
|
||||
&event("unassigned", &["atlas"], &[], &[])
|
||||
)
|
||||
.is_empty()
|
||||
);
|
||||
// The source gates: another event type starts nothing.
|
||||
assert!(
|
||||
flow("assigned")
|
||||
.evaluate("issues", &event("assigned", &["atlas"], &[], &[]))
|
||||
.is_empty()
|
||||
evaluate(
|
||||
&assigned,
|
||||
"issues",
|
||||
&event("assigned", &["atlas"], &[], &[])
|
||||
)
|
||||
.is_empty()
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unassigning_rewrites_the_same_key_for_the_removed_agent() {
|
||||
let got = flow("unassigned").evaluate(
|
||||
let got = evaluate(
|
||||
&flow("unassigned"),
|
||||
"pull_request_assign",
|
||||
&event("unassigned", &[], &["argus"], &[]),
|
||||
);
|
||||
|
|
@ -423,8 +223,12 @@ mod tests {
|
|||
|
||||
#[test]
|
||||
fn closing_clears_by_prefix_for_every_holder() {
|
||||
let got =
|
||||
flow("closed").evaluate("issues", &event("closed", &[], &[], &["atlas", "argus"]));
|
||||
let closed = flow("closed");
|
||||
let got = evaluate(
|
||||
&closed,
|
||||
"issues",
|
||||
&event("closed", &[], &[], &["atlas", "argus"]),
|
||||
);
|
||||
assert_eq!(got.len(), 2);
|
||||
for a in &got {
|
||||
assert_eq!(a.op.op, Op::Clear);
|
||||
|
|
@ -432,11 +236,7 @@ mod tests {
|
|||
assert_eq!(a.op.key, "forge:hyperhive/hyperhive#7:");
|
||||
assert_eq!(a.op.summary, "");
|
||||
}
|
||||
assert!(
|
||||
flow("closed")
|
||||
.evaluate("issues", &event("reopened", &[], &[], &["atlas"]))
|
||||
.is_empty()
|
||||
);
|
||||
assert!(evaluate(&closed, "issues", &event("reopened", &[], &[], &["atlas"])).is_empty());
|
||||
}
|
||||
|
||||
/// A match node with two children runs both: the branching the tree
|
||||
|
|
@ -450,7 +250,7 @@ mod tests {
|
|||
f.nodes.push(Node {
|
||||
id: "second".to_owned(),
|
||||
kind: NodeKind::Todo(TodoSink {
|
||||
to: "assignees_removed".to_owned(),
|
||||
to: Recipients::Field("assignees_removed".to_owned()),
|
||||
..sink
|
||||
}),
|
||||
});
|
||||
|
|
@ -459,7 +259,8 @@ mod tests {
|
|||
to: "second".to_owned(),
|
||||
});
|
||||
assert_eq!(f.validate(), Ok(()));
|
||||
let got = f.evaluate(
|
||||
let got = evaluate(
|
||||
&f,
|
||||
"issue_assign",
|
||||
&event("assigned", &["atlas"], &["argus"], &[]),
|
||||
);
|
||||
|
|
@ -471,48 +272,59 @@ mod tests {
|
|||
assert_eq!(nodes, [("second", "argus"), ("sink", "atlas")]);
|
||||
}
|
||||
|
||||
/// A custom flow's sink can name one agent, whatever the event holds.
|
||||
#[test]
|
||||
fn a_flow_that_is_not_a_tree_is_refused() {
|
||||
let edge = |from: &str, to: &str| Edge {
|
||||
from: from.to_owned(),
|
||||
to: to.to_owned(),
|
||||
};
|
||||
let mut cycle = flow("assigned");
|
||||
cycle.edges[0] = edge("sink", "match");
|
||||
cycle.edges.push(edge("match", "sink"));
|
||||
assert!(cycle.validate().is_err(), "a cycle");
|
||||
|
||||
let mut orphan = flow("assigned");
|
||||
orphan.edges.pop();
|
||||
assert!(orphan.validate().is_err(), "an unreachable sink");
|
||||
|
||||
let mut two_parents = flow("assigned");
|
||||
two_parents.edges.push(edge("source", "sink"));
|
||||
assert!(two_parents.validate().is_err(), "a sink with two parents");
|
||||
|
||||
let mut sink_child = flow("assigned");
|
||||
sink_child.nodes.push(Node {
|
||||
id: "m2".to_owned(),
|
||||
kind: NodeKind::Match { when: Vec::new() },
|
||||
fn a_sink_to_a_named_agent_reaches_it_once() {
|
||||
let mut f = flow("assigned");
|
||||
f.nodes[2].kind = NodeKind::Todo(TodoSink {
|
||||
to: Recipients::Agent("triage".to_owned()),
|
||||
op: Op::Upsert,
|
||||
key: "forge:{repo}#{number}:triage".to_owned(),
|
||||
prefix: false,
|
||||
text: "triage {repo}#{number}".to_owned(),
|
||||
});
|
||||
sink_child.edges.push(edge("sink", "m2"));
|
||||
assert!(sink_child.validate().is_err(), "a sink with a child");
|
||||
|
||||
let mut dangling = flow("assigned");
|
||||
dangling.edges.push(edge("match", "nowhere"));
|
||||
assert!(dangling.validate().is_err(), "an edge to no node");
|
||||
|
||||
let mut two_sources = flow("assigned");
|
||||
two_sources.nodes[1].kind = NodeKind::Source { events: Vec::new() };
|
||||
assert!(two_sources.validate().is_err(), "two sources");
|
||||
let got = evaluate(
|
||||
&f,
|
||||
"issue_assign",
|
||||
&event("assigned", &["atlas", "iris"], &[], &[]),
|
||||
);
|
||||
assert_eq!(got.len(), 1);
|
||||
assert_eq!(got[0].agent, "triage");
|
||||
assert_eq!(got[0].op.summary, "triage hyperhive/hyperhive#7");
|
||||
assert_eq!(got[0].op.source, "router:assigned");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_placeholder_expands_to_its_field_and_an_unknown_one_stays() {
|
||||
fn a_placeholder_expands_to_its_field_and_an_unknown_one_is_an_error() {
|
||||
let fields = event("assigned", &["a", "b"], &[], &[]);
|
||||
assert_eq!(
|
||||
expand("{repo}#{number} {assignees_added} {nope} {", &fields),
|
||||
"hyperhive/hyperhive#7 a, b {nope} {"
|
||||
expand("{repo}#{number} {assignees_added} {", &fields),
|
||||
Ok("hyperhive/hyperhive#7 a, b {".to_owned())
|
||||
);
|
||||
assert_eq!(
|
||||
expand("x {nope}", &fields),
|
||||
Err("{nope} is no event field".to_owned())
|
||||
);
|
||||
}
|
||||
|
||||
/// A sink naming a placeholder the event lacks sends nothing, rather
|
||||
/// than a todo under a literal `{name}` key.
|
||||
#[test]
|
||||
fn a_sink_with_an_unknown_placeholder_emits_nothing() {
|
||||
let mut f = flow("assigned");
|
||||
f.nodes[2].kind = NodeKind::Todo(TodoSink {
|
||||
to: Recipients::Agent("triage".to_owned()),
|
||||
op: Op::Upsert,
|
||||
key: "forge:{repo}#{numbr}:triage".to_owned(),
|
||||
prefix: false,
|
||||
text: "t".to_owned(),
|
||||
});
|
||||
let fields = event("assigned", &["atlas"], &[], &[]);
|
||||
assert!(evaluate(&f, "issue_assign", &fields).is_empty());
|
||||
// Control: spelled right, the same flow emits.
|
||||
if let NodeKind::Todo(s) = &mut f.nodes[2].kind {
|
||||
s.key = "forge:{repo}#{number}:triage".to_owned();
|
||||
}
|
||||
assert_eq!(evaluate(&f, "issue_assign", &fields).len(), 1);
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -4,7 +4,8 @@
|
|||
//! (`swarm_wire::forge_events`), runs every flow over each delivery
|
||||
//! ([`flow`], [`route`]), and queues the todo ops the flows emit on each
|
||||
//! recipient agent's subject of `ROUTER_TODOS` (`swarm_wire::router_todo`).
|
||||
//! Each agent's harness applies them to its own todo list.
|
||||
//! Each agent's harness applies them to its own todo list. The flows are the
|
||||
//! built-in ones, changed by the `notify-flows` document ([`stored`]).
|
||||
//!
|
||||
//! Per issue it keeps a snapshot in its state bucket ([`event::Snapshot`]):
|
||||
//! the assignees it last saw and the agents it queued a todo for. An event is
|
||||
|
|
@ -31,6 +32,7 @@ mod event;
|
|||
mod flow;
|
||||
mod route;
|
||||
mod settle;
|
||||
mod stored;
|
||||
|
||||
/// The cert-auth role this service logs in to the store as. `swarm-bao.nix`
|
||||
/// writes it, matching the leaf's subject.
|
||||
|
|
@ -83,17 +85,7 @@ async fn main() -> Result<()> {
|
|||
})
|
||||
.await?;
|
||||
|
||||
let flows: Vec<flow::Flow> = flow::builtins()
|
||||
.into_iter()
|
||||
.filter(|f| match f.validate() {
|
||||
Ok(()) => true,
|
||||
Err(reason) => {
|
||||
tracing::error!(flow = %f.id, %reason, "notify router: skipping a flow that is not a tree");
|
||||
false
|
||||
}
|
||||
})
|
||||
.collect();
|
||||
let mut router = Router::open(&queue, flows).await?;
|
||||
let mut router = Router::open(&queue).await?;
|
||||
tracing::info!(
|
||||
stream = forge_events::STREAM,
|
||||
"notify router: routing forge events"
|
||||
|
|
@ -113,12 +105,12 @@ struct Router {
|
|||
js: jetstream::Context,
|
||||
events: jetstream::consumer::Consumer<pull::Config>,
|
||||
state: kv::Store,
|
||||
flows: Vec<flow::Flow>,
|
||||
flows: stored::Flows,
|
||||
roster: Option<(Instant, BTreeSet<String>)>,
|
||||
}
|
||||
|
||||
impl Router {
|
||||
async fn open(queue: &async_nats::Client, flows: Vec<flow::Flow>) -> Result<Self> {
|
||||
async fn open(queue: &async_nats::Client) -> Result<Self> {
|
||||
let js = jetstream::new(queue.clone());
|
||||
let events = swarm_queue_client::forge_events::open_or_create(queue)
|
||||
.await
|
||||
|
|
@ -154,7 +146,7 @@ impl Router {
|
|||
js,
|
||||
events,
|
||||
state,
|
||||
flows,
|
||||
flows: stored::Flows::builtins(),
|
||||
roster: None,
|
||||
})
|
||||
}
|
||||
|
|
@ -193,8 +185,9 @@ impl Router {
|
|||
}),
|
||||
None => event::Snapshot::default(),
|
||||
};
|
||||
self.flows.refresh(&self.js).await;
|
||||
let roster = self.roster().await.map_err(Skip::Retry)?;
|
||||
let routed = route::route(&event_type, &delivery, &snapshot, &self.flows, |a| {
|
||||
let routed = route::route(&event_type, &delivery, &snapshot, &self.flows.active, |a| {
|
||||
roster.contains(a)
|
||||
});
|
||||
for action in &routed.actions {
|
||||
|
|
|
|||
|
|
@ -6,7 +6,9 @@ use std::collections::BTreeSet;
|
|||
use swarm_wire::router_todo::Op;
|
||||
|
||||
use crate::event::{Delivery, Snapshot, fields, is_assign};
|
||||
use crate::flow::{Action, Flow};
|
||||
use swarm_wire::notify_flows::Flow;
|
||||
|
||||
use crate::flow::{Action, evaluate};
|
||||
|
||||
/// What one event comes to.
|
||||
#[derive(Debug, PartialEq, Eq)]
|
||||
|
|
@ -30,7 +32,7 @@ pub fn route(
|
|||
let fields = fields(event_type, delivery, snapshot);
|
||||
let mut actions = Vec::new();
|
||||
let mut skipped = Vec::new();
|
||||
for action in flows.iter().flat_map(|f| f.evaluate(event_type, &fields)) {
|
||||
for action in flows.iter().flat_map(|f| evaluate(f, event_type, &fields)) {
|
||||
if is_agent(&action.agent) {
|
||||
actions.push(action);
|
||||
} else if !skipped.contains(&action.agent) {
|
||||
|
|
@ -68,7 +70,7 @@ pub fn route(
|
|||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::flow::builtins;
|
||||
use swarm_wire::notify_flows::builtins;
|
||||
|
||||
fn delivery(action: &str, assignees: &[&str]) -> Delivery {
|
||||
Delivery {
|
||||
|
|
|
|||
259
swarm-notify-router/src/stored.rs
Normal file
259
swarm-notify-router/src/stored.rs
Normal file
|
|
@ -0,0 +1,259 @@
|
|||
//! The flows the router runs: the built-ins, changed by the document in the
|
||||
//! `notify-flows` bucket.
|
||||
//!
|
||||
//! The router reads the key again every [`REFRESH`] and runs what
|
||||
//! [`Document::active`] says. What it does with each thing it can read:
|
||||
//!
|
||||
//! | the key | the router runs |
|
||||
//! | ------------------------------------ | ----------------------------------- |
|
||||
//! | absent, deleted, or no such bucket | the built-ins |
|
||||
//! | a document it reads | its built-ins and custom flows |
|
||||
//! | a document it cannot read | what it ran before |
|
||||
//! | the read itself failing | what it ran before |
|
||||
//!
|
||||
//! "Cannot read" is not JSON, not the document's shape, or a version this
|
||||
//! build does not know. Before any document has been read, "what it ran
|
||||
//! before" is the built-ins, so a bad document never leaves the router with
|
||||
//! no flows.
|
||||
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
use async_nats::jetstream::ErrorCode;
|
||||
use async_nats::jetstream::context::GetStreamErrorKind;
|
||||
use async_nats::jetstream::{self, kv};
|
||||
use swarm_wire::notify_flows::{self, Document, Flow};
|
||||
|
||||
/// How long a read of the key is reused.
|
||||
const REFRESH: Duration = Duration::from_secs(10);
|
||||
|
||||
/// What a read of the key came to.
|
||||
#[derive(Debug)]
|
||||
pub enum Read {
|
||||
/// No key, a deleted key, or no bucket.
|
||||
Absent,
|
||||
Document {
|
||||
revision: u64,
|
||||
bytes: Vec<u8>,
|
||||
},
|
||||
Failed(String),
|
||||
}
|
||||
|
||||
/// The flows in force, and which revision of the key they came from.
|
||||
pub struct Flows {
|
||||
pub active: Vec<Flow>,
|
||||
/// The revision `active` was read from; `None` for the built-ins alone.
|
||||
revision: Option<u64>,
|
||||
/// The last problem logged, so a standing one is logged once.
|
||||
problem: Option<String>,
|
||||
checked: Option<Instant>,
|
||||
}
|
||||
|
||||
impl Flows {
|
||||
/// The built-ins, which is what the router runs until it reads a document.
|
||||
pub fn builtins() -> Self {
|
||||
Self {
|
||||
active: Document::default().active().0,
|
||||
revision: None,
|
||||
problem: None,
|
||||
checked: None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Take what a read came to.
|
||||
pub fn apply(&mut self, read: Read) {
|
||||
match read {
|
||||
Read::Failed(why) => self.note(format!("reading the key failed: {why}")),
|
||||
Read::Absent => {
|
||||
if self.revision.take().is_some() {
|
||||
tracing::info!(
|
||||
"notify router: the flows document is gone; running the built-in flows"
|
||||
);
|
||||
}
|
||||
self.active = Document::default().active().0;
|
||||
self.problem = None;
|
||||
}
|
||||
Read::Document { revision, .. } if self.revision == Some(revision) => {}
|
||||
Read::Document { revision, bytes } => match Document::decode(&bytes) {
|
||||
Err(why) => {
|
||||
// The revision is not taken, so the next read looks at
|
||||
// the document again and the problem is logged once.
|
||||
self.note(format!("revision {revision} cannot be read: {why}"));
|
||||
}
|
||||
Ok(doc) => {
|
||||
let (active, skipped) = doc.active();
|
||||
for why in &skipped {
|
||||
tracing::error!(%why, "notify router: skipping a flow");
|
||||
}
|
||||
tracing::info!(
|
||||
revision,
|
||||
flows = active.len(),
|
||||
disabled = doc.disabled_builtins.len(),
|
||||
custom = doc.flows.len(),
|
||||
"notify router: running the flows document"
|
||||
);
|
||||
self.active = active;
|
||||
self.revision = Some(revision);
|
||||
self.problem = None;
|
||||
}
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
fn note(&mut self, problem: String) {
|
||||
if self.problem.as_ref() != Some(&problem) {
|
||||
tracing::error!(%problem, "notify router: keeping the flows it has");
|
||||
self.problem = Some(problem);
|
||||
}
|
||||
}
|
||||
|
||||
/// Read the key if the last read is older than [`REFRESH`].
|
||||
pub async fn refresh(&mut self, js: &jetstream::Context) {
|
||||
if self.checked.is_some_and(|at| at.elapsed() < REFRESH) {
|
||||
return;
|
||||
}
|
||||
self.checked = Some(Instant::now());
|
||||
self.apply(read(js).await);
|
||||
}
|
||||
}
|
||||
|
||||
async fn read(js: &jetstream::Context) -> Read {
|
||||
// A bucket that does not exist is the answer "no document". Any other
|
||||
// failure to look, such as a timeout, says nothing about the document.
|
||||
match js.get_stream(format!("KV_{}", notify_flows::BUCKET)).await {
|
||||
Ok(_) => {}
|
||||
Err(e)
|
||||
if matches!(e.kind(), GetStreamErrorKind::JetStream(j)
|
||||
if j.error_code() == ErrorCode::STREAM_NOT_FOUND) =>
|
||||
{
|
||||
return Read::Absent;
|
||||
}
|
||||
Err(e) => return Read::Failed(e.to_string()),
|
||||
}
|
||||
let store = match js.get_key_value(notify_flows::BUCKET).await {
|
||||
Ok(store) => store,
|
||||
Err(e) => return Read::Failed(e.to_string()),
|
||||
};
|
||||
match store.entry(notify_flows::KEY).await {
|
||||
Ok(Some(entry)) if entry.operation == kv::Operation::Put => Read::Document {
|
||||
revision: entry.revision,
|
||||
bytes: entry.value.to_vec(),
|
||||
},
|
||||
Ok(_) => Read::Absent,
|
||||
Err(e) => Read::Failed(e.to_string()),
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use swarm_wire::notify_flows::{Document, VERSION};
|
||||
|
||||
use super::*;
|
||||
|
||||
fn ids(f: &Flows) -> Vec<&str> {
|
||||
f.active.iter().map(|f| f.id.as_str()).collect()
|
||||
}
|
||||
|
||||
fn doc(disabled: &[&str], custom: &[&str]) -> Read {
|
||||
let mut flows = Vec::new();
|
||||
for id in custom {
|
||||
let mut f = notify_flows::builtins().remove(0);
|
||||
f.id = (*id).to_owned();
|
||||
flows.push(f);
|
||||
}
|
||||
let d = Document {
|
||||
version: VERSION,
|
||||
disabled_builtins: disabled.iter().map(|s| (*s).to_owned()).collect(),
|
||||
flows,
|
||||
};
|
||||
Read::Document {
|
||||
revision: 0,
|
||||
bytes: serde_json::to_vec(&d).unwrap(),
|
||||
}
|
||||
}
|
||||
|
||||
fn at(revision: u64, read: Read) -> Read {
|
||||
match read {
|
||||
Read::Document { bytes, .. } => Read::Document { revision, bytes },
|
||||
other => other,
|
||||
}
|
||||
}
|
||||
|
||||
fn junk(revision: u64, bytes: &[u8]) -> Read {
|
||||
Read::Document {
|
||||
revision,
|
||||
bytes: bytes.to_vec(),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn with_no_document_the_router_runs_the_builtins() {
|
||||
let mut f = Flows::builtins();
|
||||
assert_eq!(ids(&f), ["assigned", "unassigned", "closed"]);
|
||||
f.apply(Read::Absent);
|
||||
assert_eq!(ids(&f), ["assigned", "unassigned", "closed"]);
|
||||
// An empty document is the same.
|
||||
f.apply(at(1, doc(&[], &[])));
|
||||
assert_eq!(ids(&f), ["assigned", "unassigned", "closed"]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_document_switches_builtins_off_and_adds_custom_flows() {
|
||||
let mut f = Flows::builtins();
|
||||
f.apply(at(1, doc(&["unassigned"], &["triage"])));
|
||||
assert_eq!(ids(&f), ["assigned", "closed", "triage"]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_document_it_cannot_read_leaves_the_flows_as_they_were() {
|
||||
let mut f = Flows::builtins();
|
||||
f.apply(at(1, doc(&["closed"], &["triage"])));
|
||||
let before = ids(&f).join(",");
|
||||
assert_eq!(before, "assigned,unassigned,triage");
|
||||
for bad in [
|
||||
junk(2, b"not json"),
|
||||
junk(3, br#"{"version":2,"flows":[]}"#),
|
||||
junk(4, br#"{"flows":[{"id":"x"}]}"#),
|
||||
Read::Failed("timeout".to_owned()),
|
||||
] {
|
||||
f.apply(bad);
|
||||
assert_eq!(ids(&f).join(","), before);
|
||||
}
|
||||
// The next good revision replaces them: the control that the bad
|
||||
// ones were kept out by being bad, not by the loader being stuck.
|
||||
f.apply(at(5, doc(&[], &[])));
|
||||
assert_eq!(ids(&f), ["assigned", "unassigned", "closed"]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_bad_first_document_is_the_builtins_not_nothing() {
|
||||
let mut f = Flows::builtins();
|
||||
f.apply(junk(1, br#"{"version":9}"#));
|
||||
assert_eq!(ids(&f), ["assigned", "unassigned", "closed"]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_deleted_document_goes_back_to_the_builtins() {
|
||||
let mut f = Flows::builtins();
|
||||
f.apply(at(1, doc(&["closed"], &["triage"])));
|
||||
f.apply(Read::Absent);
|
||||
assert_eq!(ids(&f), ["assigned", "unassigned", "closed"]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_flow_that_is_not_valid_is_left_out_and_the_rest_run() {
|
||||
let mut f = Flows::builtins();
|
||||
let mut broken = notify_flows::builtins().remove(0);
|
||||
broken.id = "broken".to_owned();
|
||||
broken.edges.clear();
|
||||
let d = Document {
|
||||
version: VERSION,
|
||||
disabled_builtins: Vec::new(),
|
||||
flows: vec![broken],
|
||||
};
|
||||
f.apply(Read::Document {
|
||||
revision: 1,
|
||||
bytes: serde_json::to_vec(&d).unwrap(),
|
||||
});
|
||||
assert_eq!(ids(&f), ["assigned", "unassigned", "closed"]);
|
||||
}
|
||||
}
|
||||
|
|
@ -18,6 +18,7 @@ pub mod builder;
|
|||
pub mod claude_login;
|
||||
pub mod forge_events;
|
||||
pub mod notices;
|
||||
pub mod notify_flows;
|
||||
pub mod notify_router;
|
||||
pub mod review_classifier;
|
||||
pub mod router_todo;
|
||||
|
|
|
|||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Reference in a new issue