Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
fe578c57fd | ||
|
|
ada364a482 | ||
|
|
7795da44f8 | ||
|
|
0512862b3d | ||
|
|
e2e124755d | ||
|
|
bb08fac33c |
21 changed files with 2193 additions and 368 deletions
|
|
@ -64,21 +64,60 @@ The router holds no forge token and no webhook secret, and registers no hook.
|
||||||
|
|
||||||
A flow is a graph of nodes joined by edges: one `source` node naming the forge
|
A flow is a graph of nodes joined by edges: one `source` node naming the forge
|
||||||
event types it starts from, `match` nodes that pass when every condition
|
event types it starts from, `match` nodes that pass when every condition
|
||||||
holds, and `sink` nodes that emit one todo op per recipient. A match node can
|
holds, and `todo` nodes that emit one todo op per recipient. A match node can
|
||||||
have more than one child, and each one runs, so one event can reach several
|
have more than one child, and each one runs, so one event can reach several
|
||||||
sinks. A flow must be a tree: one source, one parent for every other node,
|
todos. A flow must be a tree: one source, one parent for every other node,
|
||||||
every node reachable, no children under a sink.
|
every node reachable, no children under a `todo`. The most is 64 nodes, 16
|
||||||
|
levels deep, and 100 custom flows. A `todo` node sends to the agents an event
|
||||||
|
field holds (`assignees_added`, `todo_holders`, …) or to one named agent.
|
||||||
|
|
||||||
The router ships the three flows above, each a source → match on `action` →
|
The router ships the three flows above, each a source → match on `action` →
|
||||||
todo sink.
|
todo. The code defines them (`swarm_wire::notify_flows::builtins`), so a
|
||||||
|
build that changes one reaches every swarm. A custom flow has the same type and
|
||||||
|
the same evaluator, so it can do anything a built-in does.
|
||||||
|
|
||||||
|
**The stored document.** swarm-controller keeps the rest in one document, the
|
||||||
|
`all` key of the `notify-flows` bucket: the ids of the built-in flows you
|
||||||
|
switched off, and the custom flows. Only the controller writes it, through
|
||||||
|
`GET` and `PUT /api/notify/flows`. A `PUT` replaces the whole document in one
|
||||||
|
compare-and-set. The controller answers 400 unless every flow passes the tree
|
||||||
|
check above and reads only what exists: a source event from `issues`,
|
||||||
|
`issue_assign`, `pull_request` or `pull_request_assign`, and condition and
|
||||||
|
recipient fields from the event's field list. The same goes for the `{field}`
|
||||||
|
placeholders in a todo's key and text. A `PUT` must carry the `revision` it read, `0` for no
|
||||||
|
document, and gets 409 when the stored revision is another. Swarm UI's
|
||||||
|
[`/notifications`](ui.md#notification-flows) page shows the document and
|
||||||
|
switches built-ins.
|
||||||
|
|
||||||
|
**What the router runs.** It reads the key at most every 10 seconds, when an
|
||||||
|
event arrives, and runs the built-ins you left on, then the custom flows that
|
||||||
|
are on and pass the tree check. It logs a custom flow that fails the check,
|
||||||
|
leaves it out, and runs the rest.
|
||||||
|
|
||||||
|
| the key | the router runs |
|
||||||
|
| -------------------------------------- | ----------------------- |
|
||||||
|
| absent, deleted, or its bucket missing | the built-in flows |
|
||||||
|
| a document it reads | the above |
|
||||||
|
| a document it can't read | the flows it ran before |
|
||||||
|
| a read that fails | the flows it ran before |
|
||||||
|
|
||||||
|
"Can't read" means not JSON, not a flows document, or a `version` other than 1.
|
||||||
|
Before the router has read any document, "before" means the built-ins, so a bad
|
||||||
|
document never leaves it with no flows. The controller refuses to overwrite a
|
||||||
|
document of a version it doesn't know.
|
||||||
|
|
||||||
|
**Deploying in any order.** The controller, the router and the UI can be on
|
||||||
|
different hosts. A controller that predates this leaves the key absent, and the
|
||||||
|
router runs the built-ins as before. A router that predates this ignores the
|
||||||
|
document, so the page and the API show a state the router isn't yet running.
|
||||||
|
|
||||||
## Credentials and grants
|
## Credentials and grants
|
||||||
|
|
||||||
| Principal | Grant |
|
| Principal | Grant |
|
||||||
| --------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
| --------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||||
| `swarm-notify-router`, in its container | store: `read` on `swarm/services/<router client id>/oidc/client`. Queue: its own consumer on `SWARM_EVENTS` (create, pull, ack), publish on `$SWARM.router.todo.*`, create `ROUTER_TODOS`, its own `notify-router-state` bucket, and `STREAM.INFO` on the agent-status bucket, which lists the agents |
|
| `swarm-notify-router`, in its container | store: `read` on `swarm/services/<router client id>/oidc/client`. Queue: its own consumer on `SWARM_EVENTS` (create, pull, ack), publish on `$SWARM.router.todo.*`, create `ROUTER_TODOS`, its own `notify-router-state` bucket, read on the `notify-flows` bucket, and `STREAM.INFO` on the agent-status bucket, which lists the agents |
|
||||||
| swarm-controller | publish on `$SWARM.events.forge.*` |
|
| swarm-controller | publish on `$SWARM.events.forge.*`; creates and writes the `notify-flows` bucket (every bucket, by its `$KV.*.>` grant) |
|
||||||
| an agent, under its own credential | create its own consumer `todo-<agent>` in the form whose filter is `$SWARM.router.todo.<agent>`, pull from it, and ack what it delivered |
|
| an agent, under its own credential | create its own consumer `todo-<agent>` in the form whose filter is `$SWARM.router.todo.<agent>`, pull from it, and ack what it delivered |
|
||||||
|
|
||||||
No other client may publish on `$SWARM.router.todo.*` or
|
No other client may publish on `$SWARM.router.todo.*` or
|
||||||
`$SWARM.events.forge.*`, so each has one writer. The agent's create subject
|
`$SWARM.events.forge.*`, so each has one writer. The agent's create subject
|
||||||
|
|
|
||||||
|
|
@ -19,6 +19,7 @@ domain, covers host-level detail for one hive.
|
||||||
| `/schedules` | the scheduled prompts: add, edit, pause, resume and remove |
|
| `/schedules` | the scheduled prompts: add, edit, pause, resume and remove |
|
||||||
| `/broadcast` | send one message to several agents at once, picked from a grid of their icons |
|
| `/broadcast` | send one message to several agents at once, picked from a grid of their icons |
|
||||||
| `/messages` | the swarm-wide message feed: every message between agents, and to them from the operator, reminders and compaction, with a history window and a filter on sender or recipient |
|
| `/messages` | the swarm-wide message feed: every message between agents, and to them from the operator, reminders and compaction, with a history window and a filter on sender or recipient |
|
||||||
|
| `/notifications` | the notification router's flows: a switch for each built-in, and every flow drawn as a tree |
|
||||||
| `/issues` | a cross-repo issue report |
|
| `/issues` | a cross-repo issue report |
|
||||||
| `/review` | the review classifier: its recent runs, a try-out on a real PR, and its settings |
|
| `/review` | the review classifier: its recent runs, a try-out on a real PR, and its settings |
|
||||||
| `/builds` | the swarm builder's recent builds of agent config PRs, with what nix printed when one failed |
|
| `/builds` | the swarm builder's recent builds of agent config PRs, with what nix printed when one failed |
|
||||||
|
|
@ -216,6 +217,22 @@ Only an operator can change a schedule here or with `swarmctl schedule`; an
|
||||||
agent has no way to add one. The page needs the controller's `/api/schedules`
|
agent has no way to add one. The page needs the controller's `/api/schedules`
|
||||||
routes.
|
routes.
|
||||||
|
|
||||||
|
### Notification flows
|
||||||
|
|
||||||
|
`/notifications` lists the [notification router's flows](notify-router.md#flows)
|
||||||
|
in two groups. Each **built-in** flow has an on/off switch; switching one off
|
||||||
|
stops the router running it from its next event, and switching it on again
|
||||||
|
restores it. Each **custom** flow shows its name and whether it's on. Every
|
||||||
|
flow draws as an indented tree: the forge events it starts from, the
|
||||||
|
conditions under it, and the todo each branch ends in.
|
||||||
|
|
||||||
|
The page edits nothing else. Add custom flows with `PUT /api/notify/flows`.
|
||||||
|
|
||||||
|
A switch sends the whole flows document with the revision the page read, `0` if
|
||||||
|
it read no document. The controller refuses the switch when the stored revision
|
||||||
|
is another, including when someone wrote a document after the page read none, and
|
||||||
|
the page shows why. Reload to switch against the current document.
|
||||||
|
|
||||||
### Sending a message to an agent
|
### Sending a message to an agent
|
||||||
|
|
||||||
Under an agent's terminal, in the preview and in its own tab, is a **message**
|
Under an agent's terminal, in the preview and in its own tab, is a **message**
|
||||||
|
|
|
||||||
|
|
@ -8,6 +8,7 @@ import { AlertsPage } from "./pages/AlertsPage.js";
|
||||||
import { BuildsPage } from "./pages/builds/BuildsPage.js";
|
import { BuildsPage } from "./pages/builds/BuildsPage.js";
|
||||||
import { BroadcastPage } from "./pages/broadcast/BroadcastPage.js";
|
import { BroadcastPage } from "./pages/broadcast/BroadcastPage.js";
|
||||||
import { MessagesPage } from "./pages/messages/MessagesPage.js";
|
import { MessagesPage } from "./pages/messages/MessagesPage.js";
|
||||||
|
import { NotificationsPage } from "./pages/notifications/NotificationsPage.js";
|
||||||
import { ComponentsPage } from "./pages/ComponentsPage.js";
|
import { ComponentsPage } from "./pages/ComponentsPage.js";
|
||||||
import { JobsPage } from "./pages/JobsPage.js";
|
import { JobsPage } from "./pages/JobsPage.js";
|
||||||
import { HivesPage } from "./pages/HivesPage.js";
|
import { HivesPage } from "./pages/HivesPage.js";
|
||||||
|
|
@ -43,6 +44,7 @@ export function App() {
|
||||||
<Route path="/builds" component={BuildsPage} />
|
<Route path="/builds" component={BuildsPage} />
|
||||||
<Route path="/broadcast" component={BroadcastPage} />
|
<Route path="/broadcast" component={BroadcastPage} />
|
||||||
<Route path="/messages" component={MessagesPage} />
|
<Route path="/messages" component={MessagesPage} />
|
||||||
|
<Route path="/notifications" component={NotificationsPage} />
|
||||||
<Route path="/issues" component={IssueReportPage} />
|
<Route path="/issues" component={IssueReportPage} />
|
||||||
<Route path="/review" component={ReviewPage} />
|
<Route path="/review" component={ReviewPage} />
|
||||||
<Route path="/components" component={ComponentsPage} />
|
<Route path="/components" component={ComponentsPage} />
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,95 @@
|
||||||
|
// Wire types of swarm-controller's `GET/PUT /api/notify/flows` and the small
|
||||||
|
// pure helpers the notifications page draws a flow with.
|
||||||
|
|
||||||
|
export type Recipients = { field: string } | { agent: string };
|
||||||
|
|
||||||
|
export interface Condition {
|
||||||
|
field: string;
|
||||||
|
equals: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
// A node as the API sends it: `kind` sits beside `id`.
|
||||||
|
export type FlowNode =
|
||||||
|
| { id: string; kind: "source"; events: string[] }
|
||||||
|
| { id: string; kind: "match"; when: Condition[] }
|
||||||
|
| {
|
||||||
|
id: string;
|
||||||
|
kind: "todo";
|
||||||
|
to: Recipients;
|
||||||
|
op: "upsert" | "clear";
|
||||||
|
key: string;
|
||||||
|
prefix: boolean;
|
||||||
|
text: string;
|
||||||
|
};
|
||||||
|
|
||||||
|
export interface Flow {
|
||||||
|
id: string;
|
||||||
|
name: string;
|
||||||
|
enabled: boolean;
|
||||||
|
nodes: FlowNode[];
|
||||||
|
edges: { from: string; to: string }[];
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface FlowsView {
|
||||||
|
revision: number;
|
||||||
|
disabled_builtins: string[];
|
||||||
|
builtins: Flow[];
|
||||||
|
flows: Flow[];
|
||||||
|
}
|
||||||
|
|
||||||
|
// One node of a flow with its depth below the source, in the order a reader
|
||||||
|
// walks the tree: a parent, then each child's subtree in edge order.
|
||||||
|
export interface TreeRow {
|
||||||
|
node: FlowNode;
|
||||||
|
depth: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function treeRows(flow: Flow): TreeRow[] {
|
||||||
|
const byId = new Map(flow.nodes.map((n) => [n.id, n]));
|
||||||
|
const root = flow.nodes.find((n) => n.kind === "source");
|
||||||
|
if (!root) return [];
|
||||||
|
const rows: TreeRow[] = [];
|
||||||
|
const seen = new Set<string>();
|
||||||
|
const walk = (id: string, depth: number) => {
|
||||||
|
const node = byId.get(id);
|
||||||
|
// The controller refuses a flow that is not a tree; a repeat here would
|
||||||
|
// mean a cycle, which `seen` stops from looping.
|
||||||
|
if (!node || seen.has(id)) return;
|
||||||
|
seen.add(id);
|
||||||
|
rows.push({ node, depth });
|
||||||
|
for (const e of flow.edges) if (e.from === id) walk(e.to, depth + 1);
|
||||||
|
};
|
||||||
|
walk(root.id, 0);
|
||||||
|
return rows;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function describeRecipients(to: Recipients): string {
|
||||||
|
return "field" in to ? `{${to.field}}` : to.agent;
|
||||||
|
}
|
||||||
|
|
||||||
|
// One line saying what a node is and does.
|
||||||
|
export function describeNode(node: FlowNode): string {
|
||||||
|
switch (node.kind) {
|
||||||
|
case "source":
|
||||||
|
return `forge ${node.events.join(", ")}`;
|
||||||
|
case "match":
|
||||||
|
return node.when.length === 0
|
||||||
|
? "match anything"
|
||||||
|
: node.when.map((c) => `${c.field} = ${c.equals}`).join(" and ");
|
||||||
|
case "todo":
|
||||||
|
return node.op === "clear"
|
||||||
|
? `clear todos ${node.key}${node.prefix ? "*" : ""} of ${describeRecipients(node.to)}`
|
||||||
|
: `todo to ${describeRecipients(node.to)}, key ${node.key}: ${node.text}`;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// `disabled` with built-in `id` switched on or off, without repeats and in a
|
||||||
|
// stable order.
|
||||||
|
export function withBuiltin(
|
||||||
|
disabled: string[],
|
||||||
|
id: string,
|
||||||
|
enabled: boolean,
|
||||||
|
): string[] {
|
||||||
|
const rest = disabled.filter((d) => d !== id);
|
||||||
|
return (enabled ? rest : [...rest, id]).sort();
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,46 @@
|
||||||
|
/* NotificationsPage — the flows drawn as indented trees. */
|
||||||
|
|
||||||
|
.notify-flows {
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
gap: 0.75em;
|
||||||
|
}
|
||||||
|
|
||||||
|
.notify-flow-head {
|
||||||
|
display: flex;
|
||||||
|
flex-wrap: wrap;
|
||||||
|
align-items: center;
|
||||||
|
justify-content: space-between;
|
||||||
|
gap: 0.5em;
|
||||||
|
}
|
||||||
|
|
||||||
|
.notify-flow-name {
|
||||||
|
font-weight: 600;
|
||||||
|
}
|
||||||
|
|
||||||
|
.notify-flow-tree {
|
||||||
|
margin: 0.4em 0 0;
|
||||||
|
padding: 0;
|
||||||
|
list-style: none;
|
||||||
|
font-size: 0.9em;
|
||||||
|
}
|
||||||
|
|
||||||
|
.notify-node {
|
||||||
|
display: flex;
|
||||||
|
gap: 0.5em;
|
||||||
|
padding: 0.1em 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.notify-node-kind {
|
||||||
|
flex: none;
|
||||||
|
min-width: 3.5em;
|
||||||
|
color: var(--muted);
|
||||||
|
}
|
||||||
|
|
||||||
|
.notify-node-text {
|
||||||
|
overflow-wrap: anywhere;
|
||||||
|
}
|
||||||
|
|
||||||
|
.notify-off {
|
||||||
|
opacity: 0.6;
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,144 @@
|
||||||
|
// <NotificationsPage> — the notification router's flows. Backed by
|
||||||
|
// swarm-controller's `GET/PUT /api/notify/flows`; the controller is the only
|
||||||
|
// writer of the stored document and the router reads it every few seconds.
|
||||||
|
//
|
||||||
|
// Read-only except for one switch per built-in flow. A flow draws as an
|
||||||
|
// indented tree, the shape the API validates it to. Custom flows are not
|
||||||
|
// editable here: they arrive through the API.
|
||||||
|
import { useState } from "preact/hooks";
|
||||||
|
import { ApiErrorPanel } from "@hive/shared/api-error-panel.js";
|
||||||
|
import { readApiError, type ProblemDetails } from "@hive/shared/api-error.js";
|
||||||
|
import { Card } from "../../ui/card/Card.js";
|
||||||
|
import { CheckboxField } from "../../ui/checkbox-field/CheckboxField.js";
|
||||||
|
import { Panel } from "../../ui/panel/Panel.js";
|
||||||
|
import {
|
||||||
|
RefreshIntervalPicker,
|
||||||
|
type RefreshIntervalMs,
|
||||||
|
} from "../../ui/refresh-interval/RefreshInterval.js";
|
||||||
|
import { fetchJson, usePolled } from "../../ui/polled/usePolled.js";
|
||||||
|
import {
|
||||||
|
describeNode,
|
||||||
|
treeRows,
|
||||||
|
withBuiltin,
|
||||||
|
type Flow,
|
||||||
|
type FlowsView,
|
||||||
|
} from "./FlowTypes.js";
|
||||||
|
import "./NotificationsPage.css";
|
||||||
|
|
||||||
|
const DEFAULT_INTERVAL_MS: RefreshIntervalMs = 30_000;
|
||||||
|
|
||||||
|
function FlowTree({ flow }: { flow: Flow }) {
|
||||||
|
return (
|
||||||
|
<ul class="notify-flow-tree">
|
||||||
|
{treeRows(flow).map(({ node, depth }) => (
|
||||||
|
<li
|
||||||
|
class="notify-node"
|
||||||
|
key={node.id}
|
||||||
|
style={{ paddingLeft: `${depth * 1.5}em` }}
|
||||||
|
>
|
||||||
|
<span class="notify-node-kind">{node.kind}</span>
|
||||||
|
<span class="notify-node-text">{describeNode(node)}</span>
|
||||||
|
</li>
|
||||||
|
))}
|
||||||
|
</ul>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function NotificationsPage() {
|
||||||
|
const [intervalMs, setIntervalMs] =
|
||||||
|
useState<RefreshIntervalMs>(DEFAULT_INTERVAL_MS);
|
||||||
|
const {
|
||||||
|
data: view,
|
||||||
|
error,
|
||||||
|
refresh,
|
||||||
|
} = usePolled(intervalMs, (signal) =>
|
||||||
|
fetchJson<FlowsView>("/api/notify/flows", signal),
|
||||||
|
);
|
||||||
|
const [pending, setPending] = useState(false);
|
||||||
|
const [actionError, setActionError] = useState<ProblemDetails | null>(null);
|
||||||
|
|
||||||
|
// Switch one built-in on or off: the whole document goes back with the
|
||||||
|
// revision this page read (0 for none), so a change made elsewhere since is refused
|
||||||
|
// rather than overwritten.
|
||||||
|
async function setBuiltin(id: string, enabled: boolean) {
|
||||||
|
if (!view) return;
|
||||||
|
setPending(true);
|
||||||
|
setActionError(null);
|
||||||
|
try {
|
||||||
|
const r = await fetch("/api/notify/flows", {
|
||||||
|
method: "PUT",
|
||||||
|
headers: { "content-type": "application/json" },
|
||||||
|
body: JSON.stringify({
|
||||||
|
revision: view.revision,
|
||||||
|
disabled_builtins: withBuiltin(view.disabled_builtins, id, enabled),
|
||||||
|
flows: view.flows,
|
||||||
|
}),
|
||||||
|
});
|
||||||
|
if (!r.ok) setActionError(await readApiError(r));
|
||||||
|
} catch (err) {
|
||||||
|
setActionError({ detail: String(err) });
|
||||||
|
} finally {
|
||||||
|
setPending(false);
|
||||||
|
refresh();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return (
|
||||||
|
<Panel
|
||||||
|
title="notifications"
|
||||||
|
icon="🔔"
|
||||||
|
actions={
|
||||||
|
<RefreshIntervalPicker
|
||||||
|
id="notifications-refresh"
|
||||||
|
value={intervalMs}
|
||||||
|
onChange={setIntervalMs}
|
||||||
|
/>
|
||||||
|
}
|
||||||
|
>
|
||||||
|
{error ? (
|
||||||
|
<ApiErrorPanel context="failed to load the flows" problem={error} />
|
||||||
|
) : null}
|
||||||
|
{actionError ? (
|
||||||
|
<ApiErrorPanel
|
||||||
|
context="the flows were not changed"
|
||||||
|
problem={actionError}
|
||||||
|
/>
|
||||||
|
) : null}
|
||||||
|
{!error && view === null ? <p>loading…</p> : null}
|
||||||
|
{view ? (
|
||||||
|
<div class="notify-flows">
|
||||||
|
<h3>built-in flows</h3>
|
||||||
|
{view.builtins.map((flow) => {
|
||||||
|
const on = !view.disabled_builtins.includes(flow.id);
|
||||||
|
return (
|
||||||
|
<Card key={flow.id} class={on ? "" : "notify-off"}>
|
||||||
|
<div class="notify-flow-head">
|
||||||
|
<span class="notify-flow-name">{flow.name}</span>
|
||||||
|
<CheckboxField
|
||||||
|
id={`builtin-${flow.id}`}
|
||||||
|
label={on ? "on" : "off"}
|
||||||
|
checked={on}
|
||||||
|
disabled={pending}
|
||||||
|
onChange={(checked) => setBuiltin(flow.id, checked)}
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
<FlowTree flow={flow} />
|
||||||
|
</Card>
|
||||||
|
);
|
||||||
|
})}
|
||||||
|
<h3>custom flows</h3>
|
||||||
|
{view.flows.length === 0 ? <p>no custom flows</p> : null}
|
||||||
|
{view.flows.map((flow) => (
|
||||||
|
<Card key={flow.id} class={flow.enabled ? "" : "notify-off"}>
|
||||||
|
<div class="notify-flow-head">
|
||||||
|
<span class="notify-flow-name">{flow.name}</span>
|
||||||
|
<span>{flow.enabled ? "on" : "off"}</span>
|
||||||
|
</div>
|
||||||
|
<FlowTree flow={flow} />
|
||||||
|
</Card>
|
||||||
|
))}
|
||||||
|
</div>
|
||||||
|
) : null}
|
||||||
|
</Panel>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
@ -56,6 +56,7 @@ const NAV_ITEMS: { href: string; label: string; accent: string }[] = [
|
||||||
{ href: "/schedules", label: "schedules", accent: "var(--cyan)" },
|
{ href: "/schedules", label: "schedules", accent: "var(--cyan)" },
|
||||||
{ href: "/broadcast", label: "broadcast", accent: "var(--blue)" },
|
{ href: "/broadcast", label: "broadcast", accent: "var(--blue)" },
|
||||||
{ href: "/messages", label: "messages", accent: "var(--purple)" },
|
{ href: "/messages", label: "messages", accent: "var(--purple)" },
|
||||||
|
{ href: "/notifications", label: "notifications", accent: "var(--yellow)" },
|
||||||
{ href: "/issues", label: "issues", accent: "var(--yellow)" },
|
{ href: "/issues", label: "issues", accent: "var(--yellow)" },
|
||||||
{ href: "/review", label: "review", accent: "var(--amber)" },
|
{ href: "/review", label: "review", accent: "var(--amber)" },
|
||||||
{ href: "/builds", label: "builds", accent: "var(--green)" },
|
{ href: "/builds", label: "builds", accent: "var(--green)" },
|
||||||
|
|
|
||||||
80
frontend/packages/swarm-ui/test/flow-types.test.mjs
Normal file
80
frontend/packages/swarm-ui/test/flow-types.test.mjs
Normal file
|
|
@ -0,0 +1,80 @@
|
||||||
|
// The notifications page's tree and wording helpers. Run by `npm test`.
|
||||||
|
import assert from "node:assert/strict";
|
||||||
|
import { test } from "node:test";
|
||||||
|
import {
|
||||||
|
describeNode,
|
||||||
|
treeRows,
|
||||||
|
withBuiltin,
|
||||||
|
} from "../src/pages/notifications/FlowTypes.ts";
|
||||||
|
|
||||||
|
const flow = {
|
||||||
|
id: "f",
|
||||||
|
name: "f",
|
||||||
|
enabled: true,
|
||||||
|
nodes: [
|
||||||
|
{ id: "s", kind: "source", events: ["issues", "issue_assign"] },
|
||||||
|
{ id: "m", kind: "match", when: [{ field: "action", equals: "closed" }] },
|
||||||
|
{ id: "m2", kind: "match", when: [] },
|
||||||
|
{
|
||||||
|
id: "t",
|
||||||
|
kind: "todo",
|
||||||
|
to: { field: "todo_holders" },
|
||||||
|
op: "clear",
|
||||||
|
key: "forge:{repo}#{number}:",
|
||||||
|
prefix: true,
|
||||||
|
text: "",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: "u",
|
||||||
|
kind: "todo",
|
||||||
|
to: { agent: "triage" },
|
||||||
|
op: "upsert",
|
||||||
|
key: "k",
|
||||||
|
prefix: false,
|
||||||
|
text: "look",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
edges: [
|
||||||
|
{ from: "s", to: "m" },
|
||||||
|
{ from: "m", to: "t" },
|
||||||
|
{ from: "s", to: "m2" },
|
||||||
|
{ from: "m2", to: "u" },
|
||||||
|
],
|
||||||
|
};
|
||||||
|
|
||||||
|
test("treeRows: each subtree follows its parent, with its depth", () => {
|
||||||
|
assert.deepEqual(
|
||||||
|
treeRows(flow).map((r) => [r.node.id, r.depth]),
|
||||||
|
[
|
||||||
|
["s", 0],
|
||||||
|
["m", 1],
|
||||||
|
["t", 2],
|
||||||
|
["m2", 1],
|
||||||
|
["u", 2],
|
||||||
|
],
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("treeRows: a cycle ends the walk and a flow with no source is empty", () => {
|
||||||
|
const cyclic = { ...flow, edges: [...flow.edges, { from: "t", to: "s" }] };
|
||||||
|
assert.equal(treeRows(cyclic).length, 5);
|
||||||
|
assert.deepEqual(treeRows({ ...flow, nodes: [] }), []);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("describeNode: says what each kind does", () => {
|
||||||
|
assert.equal(describeNode(flow.nodes[0]), "forge issues, issue_assign");
|
||||||
|
assert.equal(describeNode(flow.nodes[1]), "action = closed");
|
||||||
|
assert.equal(describeNode(flow.nodes[2]), "match anything");
|
||||||
|
assert.equal(
|
||||||
|
describeNode(flow.nodes[3]),
|
||||||
|
"clear todos forge:{repo}#{number}:* of {todo_holders}",
|
||||||
|
);
|
||||||
|
assert.equal(describeNode(flow.nodes[4]), "todo to triage, key k: look");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("withBuiltin: switching off adds once, switching on removes", () => {
|
||||||
|
assert.deepEqual(withBuiltin([], "b", false), ["b"]);
|
||||||
|
assert.deepEqual(withBuiltin(["b"], "b", false), ["b"]);
|
||||||
|
assert.deepEqual(withBuiltin(["c"], "a", false), ["a", "c"]);
|
||||||
|
assert.deepEqual(withBuiltin(["a", "b"], "a", true), ["b"]);
|
||||||
|
});
|
||||||
|
|
@ -82,6 +82,11 @@ The swarm's control plane. The swarm UI and `swarmctl` are its clients.
|
||||||
`review-classifier-settings` KV bucket, which this daemon alone reads and
|
`review-classifier-settings` KV bucket, which this daemon alone reads and
|
||||||
writes (`GET`/`PUT /api/review-classifier/settings`) →
|
writes (`GET`/`PUT /api/review-classifier/settings`) →
|
||||||
[`docs/swarm/review-classifier.md`](../docs/swarm/review-classifier.md#swarm-level-settings).
|
[`docs/swarm/review-classifier.md`](../docs/swarm/review-classifier.md#swarm-level-settings).
|
||||||
|
- **Notification flows** — which of the notification router's built-in flows are
|
||||||
|
switched off, and the custom flows, as one document in the `notify-flows`
|
||||||
|
KV bucket, which this daemon alone writes (`GET`/`PUT /api/notify/flows`);
|
||||||
|
the router reads it →
|
||||||
|
[`docs/swarm/notify-router.md`](../docs/swarm/notify-router.md#flows).
|
||||||
- **Operator profiles** — each operator's name and the pronouns on their
|
- **Operator profiles** — each operator's name and the pronouns on their
|
||||||
forge profile, for every member of authelia's operator group, as one list
|
forge profile, for every member of authelia's operator group, as one list
|
||||||
under the one key of the `operator-profiles` KV bucket. This daemon alone
|
under the one key of the `operator-profiles` KV bucket. This daemon alone
|
||||||
|
|
|
||||||
|
|
@ -2113,6 +2113,14 @@ mod tests {
|
||||||
events(DeliveryKind::PullRequest),
|
events(DeliveryKind::PullRequest),
|
||||||
["pull_request", "pull_request_assign"]
|
["pull_request", "pull_request_assign"]
|
||||||
);
|
);
|
||||||
|
// The events a stored flow's source may name are the ones the hooks
|
||||||
|
// deliver to the router.
|
||||||
|
let mut routed = events(DeliveryKind::Issues);
|
||||||
|
routed.extend(events(DeliveryKind::PullRequest));
|
||||||
|
routed.sort_unstable();
|
||||||
|
let mut sources = swarm_wire::notify_flows::SOURCE_EVENTS.to_vec();
|
||||||
|
sources.sort_unstable();
|
||||||
|
assert_eq!(routed, sources);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
|
|
|
||||||
|
|
@ -50,6 +50,7 @@ mod jobq;
|
||||||
mod linked_accounts;
|
mod linked_accounts;
|
||||||
mod matrix_account;
|
mod matrix_account;
|
||||||
mod message_stream;
|
mod message_stream;
|
||||||
|
mod notify_flows;
|
||||||
mod otel_http_client;
|
mod otel_http_client;
|
||||||
mod pulls;
|
mod pulls;
|
||||||
mod queue_identity;
|
mod queue_identity;
|
||||||
|
|
|
||||||
446
swarm-controller/src/notify_flows.rs
Normal file
446
swarm-controller/src/notify_flows.rs
Normal file
|
|
@ -0,0 +1,446 @@
|
||||||
|
//! `GET` and `PUT /api/notify/flows`: which of the notification router's
|
||||||
|
//! built-in flows are switched off, and the custom flows.
|
||||||
|
//!
|
||||||
|
//! The flows are one document under one key ([`notify_flows::KEY`] in
|
||||||
|
//! [`notify_flows::BUCKET`]), which only this daemon writes, as the schedule
|
||||||
|
//! definitions are. A `PUT` replaces the whole document in one
|
||||||
|
//! compare-and-set, after [`Document::check`]. The router reads the key
|
||||||
|
//! every few seconds, so a change applies to the next event with no restart.
|
||||||
|
//! No document means the built-in flows alone.
|
||||||
|
//!
|
||||||
|
//! The built-in flows are defined in code, in `swarm_wire::notify_flows`, and
|
||||||
|
//! the document stores only their disabled ids, so a `GET` shows the ones
|
||||||
|
//! this build knows.
|
||||||
|
|
||||||
|
use anyhow::Context as _;
|
||||||
|
use async_nats::jetstream::kv::Store;
|
||||||
|
use axum::Json;
|
||||||
|
use axum::extract::State;
|
||||||
|
use axum::extract::rejection::JsonRejection;
|
||||||
|
use axum::http::StatusCode;
|
||||||
|
use problem_details::ProblemDetails;
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
use swarm_wire::notify_flows::{self, DecodeError, Document, Flow};
|
||||||
|
use utoipa::ToSchema;
|
||||||
|
|
||||||
|
use crate::routes::{AppState, error_problem, wanted_error_status};
|
||||||
|
use crate::schedules::{queue, read_with, refused, write_revision};
|
||||||
|
|
||||||
|
/// How many times a losing writer re-reads and re-applies, as in
|
||||||
|
/// `wanted::MAX_ATTEMPTS`.
|
||||||
|
const MAX_ATTEMPTS: usize = 5;
|
||||||
|
|
||||||
|
/// What `GET` and a successful `PUT` answer.
|
||||||
|
#[derive(Debug, Clone, PartialEq, Serialize, ToSchema)]
|
||||||
|
pub(crate) struct FlowsView {
|
||||||
|
/// The revision of the stored document, to send back in a `PUT`; `0` when
|
||||||
|
/// none is stored.
|
||||||
|
revision: u64,
|
||||||
|
/// The ids of the built-in flows that are switched off.
|
||||||
|
disabled_builtins: Vec<String>,
|
||||||
|
/// The built-in flows this build ships, all of them, switched off or not.
|
||||||
|
builtins: Vec<Flow>,
|
||||||
|
/// The custom flows.
|
||||||
|
flows: Vec<Flow>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The body of a `PUT`: the whole document.
|
||||||
|
#[derive(Debug, Clone, Deserialize, ToSchema)]
|
||||||
|
#[serde(deny_unknown_fields)]
|
||||||
|
pub(crate) struct PutFlows {
|
||||||
|
/// The `revision` the client read, `0` for "no document stored". The
|
||||||
|
/// `PUT` is refused when the stored revision is another, which includes a
|
||||||
|
/// document written since a client read `0`. A KV revision is never `0`.
|
||||||
|
revision: u64,
|
||||||
|
/// The ids of the built-in flows to switch off.
|
||||||
|
#[serde(default)]
|
||||||
|
disabled_builtins: Vec<String>,
|
||||||
|
/// The custom flows.
|
||||||
|
#[serde(default)]
|
||||||
|
flows: Vec<Flow>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl FlowsView {
|
||||||
|
fn of(doc: Document, revision: u64) -> Self {
|
||||||
|
Self {
|
||||||
|
revision,
|
||||||
|
disabled_builtins: doc.disabled_builtins,
|
||||||
|
builtins: notify_flows::builtins(),
|
||||||
|
flows: doc.flows,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What the key holds.
|
||||||
|
enum Stored {
|
||||||
|
/// No key, or a deleted one.
|
||||||
|
Nothing,
|
||||||
|
Document(Document),
|
||||||
|
/// A document this build cannot read, with why.
|
||||||
|
Unreadable(DecodeError),
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The stored document, with the revision to write against.
|
||||||
|
async fn stored(store: &Store) -> anyhow::Result<(Stored, Option<u64>)> {
|
||||||
|
struct Read(Stored);
|
||||||
|
impl Default for Read {
|
||||||
|
fn default() -> Self {
|
||||||
|
Self(Stored::Nothing)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let (Read(stored), revision) = read_with(store, notify_flows::KEY, |bytes| {
|
||||||
|
Ok(Read(match Document::decode(bytes) {
|
||||||
|
Ok(doc) => Stored::Document(doc),
|
||||||
|
Err(e) => Stored::Unreadable(e),
|
||||||
|
}))
|
||||||
|
})
|
||||||
|
.await?;
|
||||||
|
Ok((stored, revision))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Open [`notify_flows::BUCKET`], creating it if nothing has yet.
|
||||||
|
async fn open(client: &async_nats::Client) -> anyhow::Result<Store> {
|
||||||
|
let js = async_nats::jetstream::new(client.clone());
|
||||||
|
if let Ok(store) = js.get_key_value(notify_flows::BUCKET).await {
|
||||||
|
return Ok(store);
|
||||||
|
}
|
||||||
|
js.create_key_value(async_nats::jetstream::kv::Config {
|
||||||
|
bucket: notify_flows::BUCKET.to_owned(),
|
||||||
|
description: "The notification router's flows: disabled built-ins and custom flows"
|
||||||
|
.to_owned(),
|
||||||
|
history: 1,
|
||||||
|
..Default::default()
|
||||||
|
})
|
||||||
|
.await
|
||||||
|
.with_context(|| format!("creating the {} bucket", notify_flows::BUCKET))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn queue_problem(e: &anyhow::Error) -> ProblemDetails {
|
||||||
|
tracing::warn!(error = %format!("{e:#}"), "notify flows: queue failure");
|
||||||
|
error_problem(wanted_error_status(e), &format!("{e:#}"))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Read the flows. Creates the `notify-flows` bucket when it is missing, as a
|
||||||
|
/// `PUT` does, so the router finds it empty rather than absent.
|
||||||
|
#[utoipa::path(
|
||||||
|
get,
|
||||||
|
path = "/api/notify/flows",
|
||||||
|
responses(
|
||||||
|
(status = 200, description = "the stored document, with the built-in flows this \
|
||||||
|
build ships; with none stored, no disabled built-ins and no custom flows",
|
||||||
|
body = FlowsView),
|
||||||
|
(status = 409, description = "the stored document is not one this build reads, \
|
||||||
|
such as a newer build's version (problem+json)", body = String),
|
||||||
|
(status = 503, description = "no swarm queue is configured on this host, or it is \
|
||||||
|
not connected (problem+json)", body = String),
|
||||||
|
(status = 500, description = "the flows could not be read (problem+json)",
|
||||||
|
body = String),
|
||||||
|
),
|
||||||
|
tag = "notify"
|
||||||
|
)]
|
||||||
|
pub(crate) async fn get_flows(
|
||||||
|
State(state): State<AppState>,
|
||||||
|
) -> Result<Json<FlowsView>, ProblemDetails> {
|
||||||
|
let client = queue(&state).map_err(refused)?;
|
||||||
|
let store = open(&client).await.map_err(|e| queue_problem(&e))?;
|
||||||
|
let (stored, revision) = stored(&store).await.map_err(|e| queue_problem(&e))?;
|
||||||
|
match stored {
|
||||||
|
Stored::Nothing => Ok(Json(FlowsView::of(
|
||||||
|
Document::default(),
|
||||||
|
revision.unwrap_or(0),
|
||||||
|
))),
|
||||||
|
Stored::Document(doc) => Ok(Json(FlowsView::of(doc, revision.unwrap_or(0)))),
|
||||||
|
Stored::Unreadable(e) => Err(unreadable(&e)),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn unreadable(why: &DecodeError) -> ProblemDetails {
|
||||||
|
error_problem(
|
||||||
|
StatusCode::CONFLICT,
|
||||||
|
&format!("the stored flows document is not one this build reads: {why}"),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Replace the flows.
|
||||||
|
#[utoipa::path(
|
||||||
|
put,
|
||||||
|
path = "/api/notify/flows",
|
||||||
|
request_body = PutFlows,
|
||||||
|
responses(
|
||||||
|
(status = 200, description = "the document as stored; the router applies it from \
|
||||||
|
its next event", body = FlowsView),
|
||||||
|
(status = 400, description = "not JSON or not this shape, or the document fails its \
|
||||||
|
check: a `disabled_builtins` entry that is no built-in, a custom flow whose id \
|
||||||
|
is another's or a built-in's, a flow that is not a tree (one `source`, one \
|
||||||
|
parent for every other node, every node reachable, no children under a \
|
||||||
|
`todo`), more than 64 nodes, 16 levels or 100 flows, an id that is not 1 to 64 \
|
||||||
|
letters, digits, `-` or `_`, a source event the router is not sent, a condition \
|
||||||
|
or recipient field that is no event field, or an upsert with no text \
|
||||||
|
(problem+json)", body = String),
|
||||||
|
(status = 409, description = "the stored document is not at `revision` (another write landed, or one now exists where the client read none), or the stored \
|
||||||
|
document is a version this build does not read, which a `PUT` would erase \
|
||||||
|
(problem+json)", body = String),
|
||||||
|
(status = 503, description = "no swarm queue is configured on this host, or it is \
|
||||||
|
not connected (problem+json)", body = String),
|
||||||
|
(status = 500, description = "the flows could not be written (problem+json)",
|
||||||
|
body = String),
|
||||||
|
),
|
||||||
|
tag = "notify"
|
||||||
|
)]
|
||||||
|
pub(crate) async fn put_flows(
|
||||||
|
State(state): State<AppState>,
|
||||||
|
body: Result<Json<PutFlows>, JsonRejection>,
|
||||||
|
) -> Result<Json<FlowsView>, ProblemDetails> {
|
||||||
|
let Json(put) = body.map_err(|e| error_problem(StatusCode::BAD_REQUEST, &e.body_text()))?;
|
||||||
|
let doc = Document {
|
||||||
|
version: notify_flows::VERSION,
|
||||||
|
disabled_builtins: put.disabled_builtins,
|
||||||
|
flows: put.flows,
|
||||||
|
};
|
||||||
|
doc.check()
|
||||||
|
.map_err(|why| error_problem(StatusCode::BAD_REQUEST, &why))?;
|
||||||
|
let client = queue(&state).map_err(refused)?;
|
||||||
|
let store = open(&client).await.map_err(|e| queue_problem(&e))?;
|
||||||
|
for _ in 0..MAX_ATTEMPTS {
|
||||||
|
let (current, revision) = stored(&store).await.map_err(|e| queue_problem(&e))?;
|
||||||
|
// An unreadable document is replaced only when it is not a version
|
||||||
|
// this build does not know: a newer build's document is kept for it.
|
||||||
|
if let Stored::Unreadable(why @ DecodeError::UnknownVersion(_)) = ¤t {
|
||||||
|
return Err(unreadable(why));
|
||||||
|
}
|
||||||
|
if put.revision != revision.unwrap_or(0) {
|
||||||
|
return Err(error_problem(
|
||||||
|
StatusCode::CONFLICT,
|
||||||
|
&format!(
|
||||||
|
"the flows are not at revision {}; read them again",
|
||||||
|
put.revision
|
||||||
|
),
|
||||||
|
));
|
||||||
|
}
|
||||||
|
if let Some(written) = write_revision(&store, notify_flows::KEY, &doc, revision)
|
||||||
|
.await
|
||||||
|
.map_err(|e| queue_problem(&e))?
|
||||||
|
{
|
||||||
|
tracing::info!(
|
||||||
|
disabled = ?doc.disabled_builtins,
|
||||||
|
custom = doc.flows.len(),
|
||||||
|
"set the notification router's flows"
|
||||||
|
);
|
||||||
|
return Ok(Json(FlowsView::of(doc, written)));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Err(queue_problem(&anyhow::anyhow!(
|
||||||
|
"gave up writing the notification flows after {MAX_ATTEMPTS} conflicting writes"
|
||||||
|
)))
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use swarm_wire::notify_flows::{Edge, Node, NodeKind, Recipients, TodoSink};
|
||||||
|
use swarm_wire::router_todo::Op;
|
||||||
|
|
||||||
|
use super::*;
|
||||||
|
use crate::test_nats::TestNats;
|
||||||
|
|
||||||
|
fn custom(id: &str) -> Flow {
|
||||||
|
Flow {
|
||||||
|
id: id.to_owned(),
|
||||||
|
name: "new to triage".to_owned(),
|
||||||
|
enabled: true,
|
||||||
|
nodes: vec![
|
||||||
|
Node {
|
||||||
|
id: "source".to_owned(),
|
||||||
|
kind: NodeKind::Source {
|
||||||
|
events: vec!["issues".to_owned()],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
Node {
|
||||||
|
id: "sink".to_owned(),
|
||||||
|
kind: NodeKind::Todo(TodoSink {
|
||||||
|
to: Recipients::Agent("triage".to_owned()),
|
||||||
|
op: Op::Upsert,
|
||||||
|
key: "forge:{repo}#{number}:triage".to_owned(),
|
||||||
|
prefix: false,
|
||||||
|
text: "triage {repo}#{number}".to_owned(),
|
||||||
|
}),
|
||||||
|
},
|
||||||
|
],
|
||||||
|
edges: vec![Edge {
|
||||||
|
from: "source".to_owned(),
|
||||||
|
to: "sink".to_owned(),
|
||||||
|
}],
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn body(revision: u64, disabled: &[&str], flows: Vec<Flow>) -> Json<PutFlows> {
|
||||||
|
Json(PutFlows {
|
||||||
|
revision,
|
||||||
|
disabled_builtins: disabled.iter().map(|s| (*s).to_owned()).collect(),
|
||||||
|
flows,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn the_flows_are_read_replaced_and_checked_on_save() {
|
||||||
|
let nats = TestNats::start();
|
||||||
|
let client = nats.client().await;
|
||||||
|
let state = crate::agent_limits::tests::state(&client);
|
||||||
|
|
||||||
|
// Nothing stored: the built-ins alone, none off, no revision.
|
||||||
|
let Json(view) = get_flows(State(state.clone())).await.expect("get");
|
||||||
|
assert_eq!(view.revision, 0);
|
||||||
|
assert!(view.disabled_builtins.is_empty() && view.flows.is_empty());
|
||||||
|
assert_eq!(view.builtins.len(), 3);
|
||||||
|
|
||||||
|
let Json(saved) = put_flows(
|
||||||
|
State(state.clone()),
|
||||||
|
Ok(body(0, &["unassigned"], vec![custom("triage")])),
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.expect("put");
|
||||||
|
let revision = saved.revision;
|
||||||
|
assert_ne!(revision, 0);
|
||||||
|
let Json(read) = get_flows(State(state.clone())).await.expect("get");
|
||||||
|
assert_eq!(read, saved);
|
||||||
|
assert_eq!(read.disabled_builtins, ["unassigned"]);
|
||||||
|
assert_eq!(read.flows[0].id, "triage");
|
||||||
|
|
||||||
|
// The stored bytes are the document the router decodes.
|
||||||
|
let store = open(&client).await.expect("bucket");
|
||||||
|
let bytes = store
|
||||||
|
.get(notify_flows::KEY)
|
||||||
|
.await
|
||||||
|
.expect("read")
|
||||||
|
.expect("stored");
|
||||||
|
let doc = Document::decode(&bytes).expect("the router reads it");
|
||||||
|
assert_eq!(doc.active().0.len(), 3, "2 built-ins and the custom flow");
|
||||||
|
|
||||||
|
// A refused save changes nothing: an orphan node, and a built-in that does not exist.
|
||||||
|
let mut orphan = custom("orphan");
|
||||||
|
orphan.edges.clear();
|
||||||
|
let refused = put_flows(State(state.clone()), Ok(body(revision, &[], vec![orphan])))
|
||||||
|
.await
|
||||||
|
.expect_err("an orphan node");
|
||||||
|
assert_eq!(refused.status, Some(StatusCode::BAD_REQUEST), "{refused:?}");
|
||||||
|
let unknown = put_flows(State(state.clone()), Ok(body(revision, &["nope"], vec![])))
|
||||||
|
.await
|
||||||
|
.expect_err("no such built-in");
|
||||||
|
assert_eq!(unknown.status, Some(StatusCode::BAD_REQUEST), "{unknown:?}");
|
||||||
|
let Json(after) = get_flows(State(state.clone())).await.expect("get");
|
||||||
|
assert_eq!(after, saved);
|
||||||
|
|
||||||
|
// A stale revision is refused; the current one lands.
|
||||||
|
let outdated = put_flows(State(state.clone()), Ok(body(revision + 100, &[], vec![])))
|
||||||
|
.await
|
||||||
|
.expect_err("stale");
|
||||||
|
assert_eq!(outdated.status, Some(StatusCode::CONFLICT), "{outdated:?}");
|
||||||
|
let Json(next) = put_flows(State(state.clone()), Ok(body(revision, &[], vec![])))
|
||||||
|
.await
|
||||||
|
.expect("current revision");
|
||||||
|
assert!(next.revision > revision);
|
||||||
|
assert!(next.flows.is_empty());
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A document of a version this build does not know is neither served
|
||||||
|
/// nor replaced, and a garbled one is replaced.
|
||||||
|
#[tokio::test]
|
||||||
|
async fn a_document_of_another_version_is_kept_and_a_garbled_one_is_replaced() {
|
||||||
|
let nats = TestNats::start();
|
||||||
|
let client = nats.client().await;
|
||||||
|
let state = crate::agent_limits::tests::state(&client);
|
||||||
|
let store = open(&client).await.expect("bucket");
|
||||||
|
|
||||||
|
store
|
||||||
|
.put(
|
||||||
|
notify_flows::KEY,
|
||||||
|
br#"{"version":2,"flows":[]}"#.to_vec().into(),
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.expect("seed");
|
||||||
|
let got = get_flows(State(state.clone()))
|
||||||
|
.await
|
||||||
|
.expect_err("version 2");
|
||||||
|
assert_eq!(got.status, Some(StatusCode::CONFLICT), "{got:?}");
|
||||||
|
let refused = put_flows(State(state.clone()), Ok(body(0, &[], vec![])))
|
||||||
|
.await
|
||||||
|
.expect_err("version 2 is kept");
|
||||||
|
assert_eq!(refused.status, Some(StatusCode::CONFLICT), "{refused:?}");
|
||||||
|
let kept = store
|
||||||
|
.get(notify_flows::KEY)
|
||||||
|
.await
|
||||||
|
.expect("read")
|
||||||
|
.expect("stored");
|
||||||
|
assert_eq!(&kept[..], br#"{"version":2,"flows":[]}"#);
|
||||||
|
|
||||||
|
// Control: garbage that names no version is replaced by a save.
|
||||||
|
let garbage = store
|
||||||
|
.put(notify_flows::KEY, b"garbage".to_vec().into())
|
||||||
|
.await
|
||||||
|
.expect("seed");
|
||||||
|
let _ = put_flows(State(state.clone()), Ok(body(garbage, &["closed"], vec![])))
|
||||||
|
.await
|
||||||
|
.expect("replaces garbage");
|
||||||
|
let Json(view) = get_flows(State(state)).await.expect("readable again");
|
||||||
|
assert_eq!(view.disabled_builtins, ["closed"]);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A page that read no document cannot erase one written since: its
|
||||||
|
/// `revision: 0` means "create", and the document exists by then.
|
||||||
|
#[tokio::test]
|
||||||
|
async fn a_page_that_read_nothing_cannot_erase_a_flow_written_after() {
|
||||||
|
let nats = TestNats::start();
|
||||||
|
let client = nats.client().await;
|
||||||
|
let state = crate::agent_limits::tests::state(&client);
|
||||||
|
|
||||||
|
let Json(page) = get_flows(State(state.clone())).await.expect("get");
|
||||||
|
assert_eq!(page.revision, 0);
|
||||||
|
// The API writes a custom flow while the page is open.
|
||||||
|
let Json(api) = put_flows(
|
||||||
|
State(state.clone()),
|
||||||
|
Ok(body(0, &[], vec![custom("triage")])),
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.expect("the API's write");
|
||||||
|
// The page's first switch carries what it read.
|
||||||
|
let erased = put_flows(
|
||||||
|
State(state.clone()),
|
||||||
|
Ok(body(page.revision, &["closed"], vec![])),
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.expect_err("the page read nothing, and a document exists");
|
||||||
|
assert_eq!(erased.status, Some(StatusCode::CONFLICT), "{erased:?}");
|
||||||
|
let Json(after) = get_flows(State(state.clone())).await.expect("get");
|
||||||
|
assert_eq!(after, api);
|
||||||
|
assert_eq!(after.flows[0].id, "triage");
|
||||||
|
|
||||||
|
// Control: re-read, the same switch lands and keeps the flow.
|
||||||
|
let Json(switched) = put_flows(
|
||||||
|
State(state),
|
||||||
|
Ok(body(after.revision, &["closed"], after.flows.clone())),
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.expect("the switch with the revision it read");
|
||||||
|
assert_eq!(switched.disabled_builtins, ["closed"]);
|
||||||
|
assert_eq!(switched.flows[0].id, "triage");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn a_template_naming_no_event_field_is_refused_on_save() {
|
||||||
|
let nats = TestNats::start();
|
||||||
|
let client = nats.client().await;
|
||||||
|
let state = crate::agent_limits::tests::state(&client);
|
||||||
|
|
||||||
|
let mut typo = custom("typo");
|
||||||
|
if let NodeKind::Todo(sink) = &mut typo.nodes[1].kind {
|
||||||
|
sink.key = "forge:{repo}#{numbr}:triage".to_owned();
|
||||||
|
}
|
||||||
|
let refused = put_flows(State(state.clone()), Ok(body(0, &[], vec![typo])))
|
||||||
|
.await
|
||||||
|
.expect_err("{numbr} is no field");
|
||||||
|
assert_eq!(refused.status, Some(StatusCode::BAD_REQUEST), "{refused:?}");
|
||||||
|
// Control: the same flow spelled right is stored.
|
||||||
|
let _ = put_flows(State(state), Ok(body(0, &[], vec![custom("typo")])))
|
||||||
|
.await
|
||||||
|
.expect("{number} is a field");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -31,8 +31,8 @@ use crate::{
|
||||||
agent_control, agent_icon, agent_input, agent_limits, agent_login, agent_permissions,
|
agent_control, agent_icon, agent_input, agent_limits, agent_login, agent_permissions,
|
||||||
agent_renewal, agent_state_stream, agent_status, agents_md, alerts, archetypes, auth, builder,
|
agent_renewal, agent_state_stream, agent_status, agents_md, alerts, archetypes, auth, builder,
|
||||||
claude_login, config_pr, forge, forge_account, github_account, inference, issue_report,
|
claude_login, config_pr, forge, forge_account, github_account, inference, issue_report,
|
||||||
linked_accounts, matrix_account, message_stream, pulls, queue_identity, review_classifier,
|
linked_accounts, matrix_account, message_stream, notify_flows, pulls, queue_identity,
|
||||||
schedules, status, subagent_term, term_stream, wanted, webhook,
|
review_classifier, schedules, status, subagent_term, term_stream, wanted, webhook,
|
||||||
};
|
};
|
||||||
|
|
||||||
/// Where the daemon binds, overridable via `SWARM_CONTROLLER_SOCKET`.
|
/// Where the daemon binds, overridable via `SWARM_CONTROLLER_SOCKET`.
|
||||||
|
|
@ -76,6 +76,7 @@ pub(crate) fn socket_path() -> PathBuf {
|
||||||
(name = "repos", description = "forge repo browsing (swarm-ui's issue-report page)"),
|
(name = "repos", description = "forge repo browsing (swarm-ui's issue-report page)"),
|
||||||
(name = "forge", description = "forge site admins (`swarmctl forge make-admin`)"),
|
(name = "forge", description = "forge site admins (`swarmctl forge make-admin`)"),
|
||||||
(name = "schedules", description = "scheduled prompts (`swarmctl schedule`)"),
|
(name = "schedules", description = "scheduled prompts (`swarmctl schedule`)"),
|
||||||
|
(name = "notify", description = "the notification router's flows: built-ins switched off, and custom flows"),
|
||||||
(name = "review-classifier", description = "trying review classifications against a PR (swarm-ui's try-out page), and the classifier's swarm-wide settings"),
|
(name = "review-classifier", description = "trying review classifications against a PR (swarm-ui's try-out page), and the classifier's swarm-wide settings"),
|
||||||
(name = "pulls", description = "open pull requests, for the try-out page's dropdowns"),
|
(name = "pulls", description = "open pull requests, for the try-out page's dropdowns"),
|
||||||
(name = "alerts", description = "the swarm Grafana's current alerts (swarm-ui's alerts page)"),
|
(name = "alerts", description = "the swarm Grafana's current alerts (swarm-ui's alerts page)"),
|
||||||
|
|
@ -635,6 +636,7 @@ pub(crate) fn build_app(state: AppState) -> axum::Router {
|
||||||
review_classifier::settings::get_settings,
|
review_classifier::settings::get_settings,
|
||||||
review_classifier::settings::put_settings
|
review_classifier::settings::put_settings
|
||||||
))
|
))
|
||||||
|
.routes(routes!(notify_flows::get_flows, notify_flows::put_flows))
|
||||||
.routes(routes!(webhook::post_webhook_forge))
|
.routes(routes!(webhook::post_webhook_forge))
|
||||||
.split_for_parts();
|
.split_for_parts();
|
||||||
// Just the JSON, not the UI — Swagger UI itself is nginx-hosted from
|
// Just the JSON, not the UI — Swagger UI itself is nginx-hosted from
|
||||||
|
|
|
||||||
|
|
@ -317,7 +317,7 @@ impl Policy {
|
||||||
|
|
||||||
/// What the notification router may do: drain its own consumer on the
|
/// What the notification router may do: drain its own consumer on the
|
||||||
/// forge events stream, queue todo ops for any agent, keep its own state
|
/// forge events stream, queue todo ops for any agent, keep its own state
|
||||||
/// bucket, and list which agents exist.
|
/// bucket, read the flows document, and list which agents exist.
|
||||||
///
|
///
|
||||||
/// - **Forge events.** Open or create the stream, create its one durable
|
/// - **Forge events.** Open or create the stream, create its one durable
|
||||||
/// in the extended form whose last tokens are the filter (the server
|
/// in the extended form whose last tokens are the filter (the server
|
||||||
|
|
@ -330,14 +330,18 @@ impl Policy {
|
||||||
/// subjects, which is what makes the router their sole writer.
|
/// subjects, which is what makes the router their sole writer.
|
||||||
/// - **Its state bucket.** Create, look up, read and write
|
/// - **Its state bucket.** Create, look up, read and write
|
||||||
/// `notify-router-state`, and no other bucket.
|
/// `notify-router-state`, and no other bucket.
|
||||||
|
/// - **The flows document.** `STREAM.INFO` and `DIRECT.GET` on
|
||||||
|
/// `notify-flows`, which it reads and never creates or writes: the
|
||||||
|
/// controller creates the bucket and is its one writer.
|
||||||
/// - **The roster.** `STREAM.INFO` on the agent-status bucket, which lists
|
/// - **The roster.** `STREAM.INFO` on the agent-status bucket, which lists
|
||||||
/// its keys (`<hive>.<agent>`) and writes none.
|
/// its keys (`<hive>.<agent>`) and writes none.
|
||||||
fn notify_router_permissions(client_id: &str) -> Permissions {
|
fn notify_router_permissions(client_id: &str) -> Permissions {
|
||||||
use swarm_wire::{forge_events, notify_router, router_todo};
|
use swarm_wire::{forge_events, notify_flows, notify_router, router_todo};
|
||||||
|
|
||||||
let events = forge_events::STREAM;
|
let events = forge_events::STREAM;
|
||||||
let consumer = forge_events::ROUTER_CONSUMER;
|
let consumer = forge_events::ROUTER_CONSUMER;
|
||||||
let state = format!("KV_{}", notify_router::STATE_BUCKET);
|
let state = format!("KV_{}", notify_router::STATE_BUCKET);
|
||||||
|
let flows = format!("KV_{}", notify_flows::BUCKET);
|
||||||
let mut publish = Self::jetstream_minimum().to_vec();
|
let mut publish = Self::jetstream_minimum().to_vec();
|
||||||
publish.extend([
|
publish.extend([
|
||||||
format!("$JS.API.STREAM.INFO.{events}"),
|
format!("$JS.API.STREAM.INFO.{events}"),
|
||||||
|
|
@ -355,6 +359,8 @@ impl Policy {
|
||||||
format!("$JS.API.STREAM.CREATE.{state}"),
|
format!("$JS.API.STREAM.CREATE.{state}"),
|
||||||
format!("$JS.API.DIRECT.GET.{state}.>"),
|
format!("$JS.API.DIRECT.GET.{state}.>"),
|
||||||
format!("$KV.{}.>", notify_router::STATE_BUCKET),
|
format!("$KV.{}.>", notify_router::STATE_BUCKET),
|
||||||
|
format!("$JS.API.STREAM.INFO.{flows}"),
|
||||||
|
format!("$JS.API.DIRECT.GET.{flows}.>"),
|
||||||
format!("$JS.API.STREAM.INFO.{}", Self::agent_status_stream()),
|
format!("$JS.API.STREAM.INFO.{}", Self::agent_status_stream()),
|
||||||
Self::inbox(client_id),
|
Self::inbox(client_id),
|
||||||
]);
|
]);
|
||||||
|
|
@ -1799,7 +1805,7 @@ mod tests {
|
||||||
#[test]
|
#[test]
|
||||||
fn the_notify_router_drains_its_own_consumer_and_writes_only_its_own_state() {
|
fn the_notify_router_drains_its_own_consumer_and_writes_only_its_own_state() {
|
||||||
use swarm_queue_client::client_inbox_prefix;
|
use swarm_queue_client::client_inbox_prefix;
|
||||||
use swarm_wire::{forge_events, notify_router, router_todo};
|
use swarm_wire::{forge_events, notify_flows, notify_router, router_todo};
|
||||||
|
|
||||||
let p = with_router()
|
let p = with_router()
|
||||||
.permissions(NOTIFY_ROUTER)
|
.permissions(NOTIFY_ROUTER)
|
||||||
|
|
@ -1807,6 +1813,7 @@ mod tests {
|
||||||
let may = |subject: &str| p.publish.iter().any(|g| covers(g, subject));
|
let may = |subject: &str| p.publish.iter().any(|g| covers(g, subject));
|
||||||
let events = forge_events::STREAM;
|
let events = forge_events::STREAM;
|
||||||
let state = notify_router::STATE_BUCKET;
|
let state = notify_router::STATE_BUCKET;
|
||||||
|
let flows = notify_flows::BUCKET;
|
||||||
for allowed in [
|
for allowed in [
|
||||||
format!(
|
format!(
|
||||||
"$JS.API.CONSUMER.CREATE.{events}.notify-router.{}",
|
"$JS.API.CONSUMER.CREATE.{events}.notify-router.{}",
|
||||||
|
|
@ -1820,6 +1827,12 @@ mod tests {
|
||||||
format!("$KV.{state}.42.5571"),
|
format!("$KV.{state}.42.5571"),
|
||||||
format!("$JS.API.DIRECT.GET.KV_{state}.$KV.{state}.42.5571"),
|
format!("$JS.API.DIRECT.GET.KV_{state}.$KV.{state}.42.5571"),
|
||||||
"$JS.API.STREAM.INFO.KV_agent-status".to_owned(),
|
"$JS.API.STREAM.INFO.KV_agent-status".to_owned(),
|
||||||
|
// The flows document: read it.
|
||||||
|
format!("$JS.API.STREAM.INFO.KV_{flows}"),
|
||||||
|
format!(
|
||||||
|
"$JS.API.DIRECT.GET.KV_{flows}.$KV.{flows}.{}",
|
||||||
|
notify_flows::KEY
|
||||||
|
),
|
||||||
] {
|
] {
|
||||||
assert!(may(&allowed), "the router may not reach {allowed}");
|
assert!(may(&allowed), "the router may not reach {allowed}");
|
||||||
}
|
}
|
||||||
|
|
@ -1838,6 +1851,10 @@ mod tests {
|
||||||
forge_events::subject("issues"),
|
forge_events::subject("issues"),
|
||||||
"$KV.agent-status.alpha.atlas".to_owned(),
|
"$KV.agent-status.alpha.atlas".to_owned(),
|
||||||
"$KV.agent-schedules.atlas".to_owned(),
|
"$KV.agent-schedules.atlas".to_owned(),
|
||||||
|
// The flows document: not write it, nor make the bucket.
|
||||||
|
format!("$KV.{flows}.{}", notify_flows::KEY),
|
||||||
|
format!("$JS.API.STREAM.CREATE.KV_{flows}"),
|
||||||
|
format!("$JS.API.STREAM.DELETE.KV_{flows}"),
|
||||||
swarm_queue_client::agent_input_subject("atlas"),
|
swarm_queue_client::agent_input_subject("atlas"),
|
||||||
"$JS.API.STREAM.CREATE.KV_agent-status".to_owned(),
|
"$JS.API.STREAM.CREATE.KV_agent-status".to_owned(),
|
||||||
] {
|
] {
|
||||||
|
|
|
||||||
|
|
@ -163,6 +163,22 @@ mod tests {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The names a stored flow may read are the ones `fields` produces.
|
||||||
|
#[test]
|
||||||
|
fn the_fields_are_the_ones_a_stored_flow_may_name() {
|
||||||
|
let d = Delivery::parse(&issue_assign("assigned", &["atlas"])).expect("parses");
|
||||||
|
for event in ["issues", "issue_assign"] {
|
||||||
|
let mut got: Vec<&str> = fields(event, &d, &Snapshot::default())
|
||||||
|
.keys()
|
||||||
|
.copied()
|
||||||
|
.collect();
|
||||||
|
let mut want = swarm_wire::notify_flows::FIELDS.to_vec();
|
||||||
|
got.sort_unstable();
|
||||||
|
want.sort_unstable();
|
||||||
|
assert_eq!(got, want, "{event}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn an_issue_delivery_reads_back_its_fields() {
|
fn an_issue_delivery_reads_back_its_fields() {
|
||||||
let d = Delivery::parse(&issue_assign("assigned", &["atlas", "iris"])).expect("parses");
|
let d = Delivery::parse(&issue_assign("assigned", &["atlas", "iris"])).expect("parses");
|
||||||
|
|
|
||||||
|
|
@ -1,26 +1,17 @@
|
||||||
//! Flows: what the router does with an event.
|
//! Running a flow over one event.
|
||||||
//!
|
//!
|
||||||
//! A flow is a graph of nodes joined by edges, rooted at one `source` node.
|
//! The flow types, their checks and the built-in flows are
|
||||||
//! A `match` node passes when every one of its conditions holds, and then
|
//! `swarm_wire::notify_flows`; the built-in and the custom flows are the same
|
||||||
//! each of its children runs, so one event can reach several branches. A
|
//! type and this walk runs both. A `match` node passes when every one of its
|
||||||
//! `sink` node is a leaf and emits one todo op per recipient.
|
//! conditions holds, and then each of its children runs, so one event can
|
||||||
//!
|
//! reach several branches. A `todo` node emits one op per recipient. The
|
||||||
//! Flows are trees: one source, one parent for every other node, every node
|
//! router runs only flows that pass [`Flow::validate`].
|
||||||
//! reachable from the source. [`Flow::validate`] checks that, and the router
|
|
||||||
//! runs only flows that pass.
|
|
||||||
//!
|
|
||||||
//! The built-in flows ([`builtins`]) are ordinary flows defined in code.
|
|
||||||
|
|
||||||
use std::collections::{BTreeMap, BTreeSet};
|
use std::collections::BTreeMap;
|
||||||
|
|
||||||
|
use swarm_wire::notify_flows::{Condition, Flow, Node, NodeKind, Recipients, TodoSink};
|
||||||
use swarm_wire::router_todo::{Op, RouterTodoOp};
|
use swarm_wire::router_todo::{Op, RouterTodoOp};
|
||||||
|
|
||||||
/// The most nodes one flow may have.
|
|
||||||
const MAX_NODES: usize = 64;
|
|
||||||
|
|
||||||
/// The deepest a node may sit below the source.
|
|
||||||
const MAX_DEPTH: usize = 16;
|
|
||||||
|
|
||||||
/// One event, as the fields a flow reads.
|
/// One event, as the fields a flow reads.
|
||||||
pub type Fields = BTreeMap<&'static str, Value>;
|
pub type Fields = BTreeMap<&'static str, Value>;
|
||||||
|
|
||||||
|
|
@ -31,56 +22,6 @@ pub enum Value {
|
||||||
List(Vec<String>),
|
List(Vec<String>),
|
||||||
}
|
}
|
||||||
|
|
||||||
#[derive(Clone, Debug)]
|
|
||||||
pub struct Flow {
|
|
||||||
pub id: String,
|
|
||||||
pub nodes: Vec<Node>,
|
|
||||||
pub edges: Vec<Edge>,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Clone, Debug)]
|
|
||||||
pub struct Node {
|
|
||||||
pub id: String,
|
|
||||||
pub kind: NodeKind,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// A directed edge, parent to child.
|
|
||||||
#[derive(Clone, Debug)]
|
|
||||||
pub struct Edge {
|
|
||||||
pub from: String,
|
|
||||||
pub to: String,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Clone, Debug)]
|
|
||||||
pub enum NodeKind {
|
|
||||||
/// The root: the forge event types the flow starts from.
|
|
||||||
Source { events: Vec<String> },
|
|
||||||
/// Passes when every condition holds.
|
|
||||||
Match { when: Vec<Condition> },
|
|
||||||
/// Emits a todo op to each recipient.
|
|
||||||
Todo(TodoSink),
|
|
||||||
}
|
|
||||||
|
|
||||||
/// `field` equals `value`. A list field holds when one of its items does.
|
|
||||||
#[derive(Clone, Debug)]
|
|
||||||
pub struct Condition {
|
|
||||||
pub field: String,
|
|
||||||
pub equals: String,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Clone, Debug)]
|
|
||||||
pub struct TodoSink {
|
|
||||||
/// The field naming the recipients, a list of agent names.
|
|
||||||
pub to: String,
|
|
||||||
pub op: Op,
|
|
||||||
/// The todo key, with `{field}` placeholders.
|
|
||||||
pub key: String,
|
|
||||||
/// For a clear: `key` is a prefix.
|
|
||||||
pub prefix: bool,
|
|
||||||
/// The todo text, with `{field}` placeholders. Unused for a clear.
|
|
||||||
pub text: String,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// One op the walk emitted, for one recipient.
|
/// One op the walk emitted, for one recipient.
|
||||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||||
pub struct Action {
|
pub struct Action {
|
||||||
|
|
@ -90,130 +31,72 @@ pub struct Action {
|
||||||
pub op: RouterTodoOp,
|
pub op: RouterTodoOp,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl Flow {
|
/// Walk `flow` for one event of type `event_type` and return the ops it
|
||||||
/// Check the flow is a tree within the caps, or say why not.
|
/// emits. Assumes [`Flow::validate`] passed.
|
||||||
pub fn validate(&self) -> Result<(), String> {
|
pub fn evaluate(flow: &Flow, event_type: &str, fields: &Fields) -> Vec<Action> {
|
||||||
if self.nodes.len() > MAX_NODES {
|
let Some(root) = flow
|
||||||
return Err(format!("{} nodes, more than {MAX_NODES}", self.nodes.len()));
|
.nodes
|
||||||
}
|
.iter()
|
||||||
let mut ids = BTreeSet::new();
|
.find(|n| matches!(&n.kind, NodeKind::Source { events } if events.iter().any(|e| e == event_type)))
|
||||||
for node in &self.nodes {
|
else {
|
||||||
if !ids.insert(node.id.as_str()) {
|
return Vec::new();
|
||||||
return Err(format!("node id {} is used twice", node.id));
|
};
|
||||||
}
|
let mut actions = Vec::new();
|
||||||
}
|
let mut stack = vec![root.id.as_str()];
|
||||||
for edge in &self.edges {
|
while let Some(id) = stack.pop() {
|
||||||
for end in [&edge.from, &edge.to] {
|
let Some(node) = flow.node(id) else { continue };
|
||||||
if !ids.contains(end.as_str()) {
|
match &node.kind {
|
||||||
return Err(format!("an edge names {end}, which is no node"));
|
NodeKind::Source { .. } => stack.extend(flow.children(id)),
|
||||||
|
NodeKind::Match { when } => {
|
||||||
|
if when.iter().all(|c| holds(c, fields)) {
|
||||||
|
stack.extend(flow.children(id));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
NodeKind::Todo(sink) => actions.extend(emit(flow, node, sink, fields)),
|
||||||
}
|
}
|
||||||
let roots: Vec<&Node> = self
|
|
||||||
.nodes
|
|
||||||
.iter()
|
|
||||||
.filter(|n| matches!(n.kind, NodeKind::Source { .. }))
|
|
||||||
.collect();
|
|
||||||
let [root] = roots.as_slice() else {
|
|
||||||
return Err(format!("{} source nodes, not one", roots.len()));
|
|
||||||
};
|
|
||||||
for node in &self.nodes {
|
|
||||||
if matches!(node.kind, NodeKind::Todo(_)) && !self.children(&node.id).is_empty() {
|
|
||||||
return Err(format!("sink {} has children", node.id));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
// From the one root, every node reached exactly once: a node reached
|
|
||||||
// twice has a second parent or sits on a cycle, and one not reached is
|
|
||||||
// cut off.
|
|
||||||
let mut reached = 0;
|
|
||||||
let mut stack = vec![(root.id.as_str(), 0)];
|
|
||||||
while let Some((id, depth)) = stack.pop() {
|
|
||||||
if depth > MAX_DEPTH {
|
|
||||||
return Err(format!("node {id} is deeper than {MAX_DEPTH}"));
|
|
||||||
}
|
|
||||||
reached += 1;
|
|
||||||
if reached > self.nodes.len() {
|
|
||||||
return Err(
|
|
||||||
"a node is reached twice: it has two parents or sits on a cycle".to_owned(),
|
|
||||||
);
|
|
||||||
}
|
|
||||||
stack.extend(self.children(id).into_iter().map(|c| (c, depth + 1)));
|
|
||||||
}
|
|
||||||
if reached != self.nodes.len() {
|
|
||||||
return Err(format!(
|
|
||||||
"{} of {} nodes are not reachable from the source",
|
|
||||||
self.nodes.len() - reached,
|
|
||||||
self.nodes.len()
|
|
||||||
));
|
|
||||||
}
|
|
||||||
Ok(())
|
|
||||||
}
|
}
|
||||||
|
actions
|
||||||
|
}
|
||||||
|
|
||||||
fn children(&self, id: &str) -> Vec<&str> {
|
fn emit(flow: &Flow, node: &Node, sink: &TodoSink, fields: &Fields) -> Vec<Action> {
|
||||||
self.edges
|
let recipients = match &sink.to {
|
||||||
.iter()
|
Recipients::Agent(agent) => vec![agent.clone()],
|
||||||
.filter(|e| e.from == id)
|
Recipients::Field(name) => match fields.get(name.as_str()) {
|
||||||
.map(|e| e.to.as_str())
|
|
||||||
.collect()
|
|
||||||
}
|
|
||||||
|
|
||||||
fn node(&self, id: &str) -> Option<&Node> {
|
|
||||||
self.nodes.iter().find(|n| n.id == id)
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Walk the flow for one event of type `event_type` and return the ops it
|
|
||||||
/// emits. Assumes [`Self::validate`] passed.
|
|
||||||
pub fn evaluate(&self, event_type: &str, fields: &Fields) -> Vec<Action> {
|
|
||||||
let Some(root) = self
|
|
||||||
.nodes
|
|
||||||
.iter()
|
|
||||||
.find(|n| matches!(&n.kind, NodeKind::Source { events } if events.iter().any(|e| e == event_type)))
|
|
||||||
else {
|
|
||||||
return Vec::new();
|
|
||||||
};
|
|
||||||
let mut actions = Vec::new();
|
|
||||||
let mut stack = vec![root.id.as_str()];
|
|
||||||
while let Some(id) = stack.pop() {
|
|
||||||
let Some(node) = self.node(id) else { continue };
|
|
||||||
match &node.kind {
|
|
||||||
NodeKind::Source { .. } => stack.extend(self.children(id)),
|
|
||||||
NodeKind::Match { when } => {
|
|
||||||
if when.iter().all(|c| holds(c, fields)) {
|
|
||||||
stack.extend(self.children(id));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
NodeKind::Todo(sink) => actions.extend(self.emit(node, sink, fields)),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
actions
|
|
||||||
}
|
|
||||||
|
|
||||||
fn emit(&self, node: &Node, sink: &TodoSink, fields: &Fields) -> Vec<Action> {
|
|
||||||
let recipients = match fields.get(sink.to.as_str()) {
|
|
||||||
Some(Value::List(items)) => items.clone(),
|
Some(Value::List(items)) => items.clone(),
|
||||||
Some(Value::Text(one)) => vec![one.clone()],
|
Some(Value::Text(one)) => vec![one.clone()],
|
||||||
None => Vec::new(),
|
None => Vec::new(),
|
||||||
|
},
|
||||||
|
};
|
||||||
|
let rendered = expand(&sink.key, fields).and_then(|key| {
|
||||||
|
let summary = match sink.op {
|
||||||
|
Op::Upsert => expand(&sink.text, fields)?,
|
||||||
|
Op::Clear => String::new(),
|
||||||
};
|
};
|
||||||
let op = RouterTodoOp {
|
Ok((key, summary))
|
||||||
op: sink.op,
|
});
|
||||||
key: expand(&sink.key, fields),
|
let (key, summary) = match rendered {
|
||||||
prefix: sink.op == Op::Clear && sink.prefix,
|
Ok(done) => done,
|
||||||
summary: match sink.op {
|
Err(why) => {
|
||||||
Op::Upsert => expand(&sink.text, fields),
|
tracing::warn!(flow = %flow.id, node = %node.id, %why, "notify router: a todo node names a placeholder the event lacks; nothing sent");
|
||||||
Op::Clear => String::new(),
|
return Vec::new();
|
||||||
},
|
}
|
||||||
source: format!("router:{}", self.id),
|
};
|
||||||
};
|
let op = RouterTodoOp {
|
||||||
recipients
|
op: sink.op,
|
||||||
.into_iter()
|
key,
|
||||||
.map(|agent| Action {
|
prefix: sink.op == Op::Clear && sink.prefix,
|
||||||
flow: self.id.clone(),
|
summary,
|
||||||
node: node.id.clone(),
|
source: format!("router:{}", flow.id),
|
||||||
agent,
|
};
|
||||||
op: op.clone(),
|
recipients
|
||||||
})
|
.into_iter()
|
||||||
.collect()
|
.map(|agent| Action {
|
||||||
}
|
flow: flow.id.clone(),
|
||||||
|
node: node.id.clone(),
|
||||||
|
agent,
|
||||||
|
op: op.clone(),
|
||||||
|
})
|
||||||
|
.collect()
|
||||||
}
|
}
|
||||||
|
|
||||||
fn holds(condition: &Condition, fields: &Fields) -> bool {
|
fn holds(condition: &Condition, fields: &Fields) -> bool {
|
||||||
|
|
@ -224,9 +107,9 @@ fn holds(condition: &Condition, fields: &Fields) -> bool {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// `template` with each `{field}` replaced by that text field's value. A list
|
/// `template` with each `{field}` replaced by that field's value, a list
|
||||||
/// field is joined with `, `; an unknown name is left as written.
|
/// joined with `, `. A name that is no field is an error naming it.
|
||||||
pub fn expand(template: &str, fields: &Fields) -> String {
|
pub fn expand(template: &str, fields: &Fields) -> Result<String, String> {
|
||||||
let mut out = String::with_capacity(template.len());
|
let mut out = String::with_capacity(template.len());
|
||||||
let mut rest = template;
|
let mut rest = template;
|
||||||
while let Some(open) = rest.find('{') {
|
while let Some(open) = rest.find('{') {
|
||||||
|
|
@ -234,109 +117,24 @@ pub fn expand(template: &str, fields: &Fields) -> String {
|
||||||
let after = &rest[open + 1..];
|
let after = &rest[open + 1..];
|
||||||
let Some(close) = after.find('}') else {
|
let Some(close) = after.find('}') else {
|
||||||
out.push_str(&rest[open..]);
|
out.push_str(&rest[open..]);
|
||||||
return out;
|
return Ok(out);
|
||||||
};
|
};
|
||||||
let name = &after[..close];
|
let name = &after[..close];
|
||||||
match fields.get(name) {
|
match fields.get(name) {
|
||||||
Some(Value::Text(v)) => out.push_str(v),
|
Some(Value::Text(v)) => out.push_str(v),
|
||||||
Some(Value::List(items)) => out.push_str(&items.join(", ")),
|
Some(Value::List(items)) => out.push_str(&items.join(", ")),
|
||||||
None => {
|
None => return Err(format!("{{{name}}} is no event field")),
|
||||||
out.push('{');
|
|
||||||
out.push_str(name);
|
|
||||||
out.push('}');
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
rest = &after[close + 1..];
|
rest = &after[close + 1..];
|
||||||
}
|
}
|
||||||
out.push_str(rest);
|
out.push_str(rest);
|
||||||
out
|
Ok(out)
|
||||||
}
|
|
||||||
|
|
||||||
/// The flows the router ships with.
|
|
||||||
///
|
|
||||||
/// - `assigned`: a newly assigned agent gets a todo for the issue or PR.
|
|
||||||
/// - `unassigned`: an agent taken off it gets that todo rewritten to say so.
|
|
||||||
/// Both write the same key, so the second replaces the first.
|
|
||||||
/// - `closed`: closing it clears every router todo the router queued for it.
|
|
||||||
pub fn builtins() -> Vec<Flow> {
|
|
||||||
let assignment_key = "forge:{repo}#{number}:assignment";
|
|
||||||
vec![
|
|
||||||
chain(
|
|
||||||
"assigned",
|
|
||||||
&["issue_assign", "pull_request_assign"],
|
|
||||||
"assigned",
|
|
||||||
TodoSink {
|
|
||||||
to: "assignees_added".to_owned(),
|
|
||||||
op: Op::Upsert,
|
|
||||||
key: assignment_key.to_owned(),
|
|
||||||
prefix: false,
|
|
||||||
text: "you are assigned to {kind} {repo}#{number}: {title} ({url})".to_owned(),
|
|
||||||
},
|
|
||||||
),
|
|
||||||
chain(
|
|
||||||
"unassigned",
|
|
||||||
&["issue_assign", "pull_request_assign"],
|
|
||||||
"unassigned",
|
|
||||||
TodoSink {
|
|
||||||
to: "assignees_removed".to_owned(),
|
|
||||||
op: Op::Upsert,
|
|
||||||
key: assignment_key.to_owned(),
|
|
||||||
prefix: false,
|
|
||||||
text: "you are not assigned to {kind} {repo}#{number} anymore: {title} ({url})"
|
|
||||||
.to_owned(),
|
|
||||||
},
|
|
||||||
),
|
|
||||||
chain(
|
|
||||||
"closed",
|
|
||||||
&["issues", "pull_request"],
|
|
||||||
"closed",
|
|
||||||
TodoSink {
|
|
||||||
to: "todo_holders".to_owned(),
|
|
||||||
op: Op::Clear,
|
|
||||||
key: "forge:{repo}#{number}:".to_owned(),
|
|
||||||
prefix: true,
|
|
||||||
text: String::new(),
|
|
||||||
},
|
|
||||||
),
|
|
||||||
]
|
|
||||||
}
|
|
||||||
|
|
||||||
/// source(`events`) → match(`action` = `action`) → `sink`.
|
|
||||||
fn chain(id: &str, events: &[&str], action: &str, sink: TodoSink) -> Flow {
|
|
||||||
let node = |id: &str, kind| Node {
|
|
||||||
id: id.to_owned(),
|
|
||||||
kind,
|
|
||||||
};
|
|
||||||
let edge = |from: &str, to: &str| Edge {
|
|
||||||
from: from.to_owned(),
|
|
||||||
to: to.to_owned(),
|
|
||||||
};
|
|
||||||
Flow {
|
|
||||||
id: id.to_owned(),
|
|
||||||
nodes: vec![
|
|
||||||
node(
|
|
||||||
"source",
|
|
||||||
NodeKind::Source {
|
|
||||||
events: events.iter().map(|e| (*e).to_owned()).collect(),
|
|
||||||
},
|
|
||||||
),
|
|
||||||
node(
|
|
||||||
"match",
|
|
||||||
NodeKind::Match {
|
|
||||||
when: vec![Condition {
|
|
||||||
field: "action".to_owned(),
|
|
||||||
equals: action.to_owned(),
|
|
||||||
}],
|
|
||||||
},
|
|
||||||
),
|
|
||||||
node("sink", NodeKind::Todo(sink)),
|
|
||||||
],
|
|
||||||
edges: vec![edge("source", "match"), edge("match", "sink")],
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
mod tests {
|
mod tests {
|
||||||
|
use swarm_wire::notify_flows::{Edge, builtins};
|
||||||
|
|
||||||
use super::*;
|
use super::*;
|
||||||
|
|
||||||
fn text(v: &str) -> Value {
|
fn text(v: &str) -> Value {
|
||||||
|
|
@ -368,16 +166,11 @@ mod tests {
|
||||||
.expect("a built-in")
|
.expect("a built-in")
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn every_builtin_is_a_valid_tree() {
|
|
||||||
for f in builtins() {
|
|
||||||
assert_eq!(f.validate(), Ok(()), "{}", f.id);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn assigning_queues_a_todo_for_each_newly_assigned_agent_only() {
|
fn assigning_queues_a_todo_for_each_newly_assigned_agent_only() {
|
||||||
let got = flow("assigned").evaluate(
|
let assigned = flow("assigned");
|
||||||
|
let got = evaluate(
|
||||||
|
&assigned,
|
||||||
"issue_assign",
|
"issue_assign",
|
||||||
&event("assigned", &["atlas", "iris"], &[], &[]),
|
&event("assigned", &["atlas", "iris"], &[], &[]),
|
||||||
);
|
);
|
||||||
|
|
@ -392,21 +185,28 @@ mod tests {
|
||||||
assert_eq!(got[0].op.source, "router:assigned");
|
assert_eq!(got[0].op.source, "router:assigned");
|
||||||
// The match gates: an unassign event reaches no assigned sink.
|
// The match gates: an unassign event reaches no assigned sink.
|
||||||
assert!(
|
assert!(
|
||||||
flow("assigned")
|
evaluate(
|
||||||
.evaluate("issue_assign", &event("unassigned", &["atlas"], &[], &[]))
|
&assigned,
|
||||||
.is_empty()
|
"issue_assign",
|
||||||
|
&event("unassigned", &["atlas"], &[], &[])
|
||||||
|
)
|
||||||
|
.is_empty()
|
||||||
);
|
);
|
||||||
// The source gates: another event type starts nothing.
|
// The source gates: another event type starts nothing.
|
||||||
assert!(
|
assert!(
|
||||||
flow("assigned")
|
evaluate(
|
||||||
.evaluate("issues", &event("assigned", &["atlas"], &[], &[]))
|
&assigned,
|
||||||
.is_empty()
|
"issues",
|
||||||
|
&event("assigned", &["atlas"], &[], &[])
|
||||||
|
)
|
||||||
|
.is_empty()
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn unassigning_rewrites_the_same_key_for_the_removed_agent() {
|
fn unassigning_rewrites_the_same_key_for_the_removed_agent() {
|
||||||
let got = flow("unassigned").evaluate(
|
let got = evaluate(
|
||||||
|
&flow("unassigned"),
|
||||||
"pull_request_assign",
|
"pull_request_assign",
|
||||||
&event("unassigned", &[], &["argus"], &[]),
|
&event("unassigned", &[], &["argus"], &[]),
|
||||||
);
|
);
|
||||||
|
|
@ -423,8 +223,12 @@ mod tests {
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn closing_clears_by_prefix_for_every_holder() {
|
fn closing_clears_by_prefix_for_every_holder() {
|
||||||
let got =
|
let closed = flow("closed");
|
||||||
flow("closed").evaluate("issues", &event("closed", &[], &[], &["atlas", "argus"]));
|
let got = evaluate(
|
||||||
|
&closed,
|
||||||
|
"issues",
|
||||||
|
&event("closed", &[], &[], &["atlas", "argus"]),
|
||||||
|
);
|
||||||
assert_eq!(got.len(), 2);
|
assert_eq!(got.len(), 2);
|
||||||
for a in &got {
|
for a in &got {
|
||||||
assert_eq!(a.op.op, Op::Clear);
|
assert_eq!(a.op.op, Op::Clear);
|
||||||
|
|
@ -432,11 +236,7 @@ mod tests {
|
||||||
assert_eq!(a.op.key, "forge:hyperhive/hyperhive#7:");
|
assert_eq!(a.op.key, "forge:hyperhive/hyperhive#7:");
|
||||||
assert_eq!(a.op.summary, "");
|
assert_eq!(a.op.summary, "");
|
||||||
}
|
}
|
||||||
assert!(
|
assert!(evaluate(&closed, "issues", &event("reopened", &[], &[], &["atlas"])).is_empty());
|
||||||
flow("closed")
|
|
||||||
.evaluate("issues", &event("reopened", &[], &[], &["atlas"]))
|
|
||||||
.is_empty()
|
|
||||||
);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/// A match node with two children runs both: the branching the tree
|
/// A match node with two children runs both: the branching the tree
|
||||||
|
|
@ -450,7 +250,7 @@ mod tests {
|
||||||
f.nodes.push(Node {
|
f.nodes.push(Node {
|
||||||
id: "second".to_owned(),
|
id: "second".to_owned(),
|
||||||
kind: NodeKind::Todo(TodoSink {
|
kind: NodeKind::Todo(TodoSink {
|
||||||
to: "assignees_removed".to_owned(),
|
to: Recipients::Field("assignees_removed".to_owned()),
|
||||||
..sink
|
..sink
|
||||||
}),
|
}),
|
||||||
});
|
});
|
||||||
|
|
@ -459,7 +259,8 @@ mod tests {
|
||||||
to: "second".to_owned(),
|
to: "second".to_owned(),
|
||||||
});
|
});
|
||||||
assert_eq!(f.validate(), Ok(()));
|
assert_eq!(f.validate(), Ok(()));
|
||||||
let got = f.evaluate(
|
let got = evaluate(
|
||||||
|
&f,
|
||||||
"issue_assign",
|
"issue_assign",
|
||||||
&event("assigned", &["atlas"], &["argus"], &[]),
|
&event("assigned", &["atlas"], &["argus"], &[]),
|
||||||
);
|
);
|
||||||
|
|
@ -471,48 +272,59 @@ mod tests {
|
||||||
assert_eq!(nodes, [("second", "argus"), ("sink", "atlas")]);
|
assert_eq!(nodes, [("second", "argus"), ("sink", "atlas")]);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// A custom flow's sink can name one agent, whatever the event holds.
|
||||||
#[test]
|
#[test]
|
||||||
fn a_flow_that_is_not_a_tree_is_refused() {
|
fn a_sink_to_a_named_agent_reaches_it_once() {
|
||||||
let edge = |from: &str, to: &str| Edge {
|
let mut f = flow("assigned");
|
||||||
from: from.to_owned(),
|
f.nodes[2].kind = NodeKind::Todo(TodoSink {
|
||||||
to: to.to_owned(),
|
to: Recipients::Agent("triage".to_owned()),
|
||||||
};
|
op: Op::Upsert,
|
||||||
let mut cycle = flow("assigned");
|
key: "forge:{repo}#{number}:triage".to_owned(),
|
||||||
cycle.edges[0] = edge("sink", "match");
|
prefix: false,
|
||||||
cycle.edges.push(edge("match", "sink"));
|
text: "triage {repo}#{number}".to_owned(),
|
||||||
assert!(cycle.validate().is_err(), "a cycle");
|
|
||||||
|
|
||||||
let mut orphan = flow("assigned");
|
|
||||||
orphan.edges.pop();
|
|
||||||
assert!(orphan.validate().is_err(), "an unreachable sink");
|
|
||||||
|
|
||||||
let mut two_parents = flow("assigned");
|
|
||||||
two_parents.edges.push(edge("source", "sink"));
|
|
||||||
assert!(two_parents.validate().is_err(), "a sink with two parents");
|
|
||||||
|
|
||||||
let mut sink_child = flow("assigned");
|
|
||||||
sink_child.nodes.push(Node {
|
|
||||||
id: "m2".to_owned(),
|
|
||||||
kind: NodeKind::Match { when: Vec::new() },
|
|
||||||
});
|
});
|
||||||
sink_child.edges.push(edge("sink", "m2"));
|
let got = evaluate(
|
||||||
assert!(sink_child.validate().is_err(), "a sink with a child");
|
&f,
|
||||||
|
"issue_assign",
|
||||||
let mut dangling = flow("assigned");
|
&event("assigned", &["atlas", "iris"], &[], &[]),
|
||||||
dangling.edges.push(edge("match", "nowhere"));
|
);
|
||||||
assert!(dangling.validate().is_err(), "an edge to no node");
|
assert_eq!(got.len(), 1);
|
||||||
|
assert_eq!(got[0].agent, "triage");
|
||||||
let mut two_sources = flow("assigned");
|
assert_eq!(got[0].op.summary, "triage hyperhive/hyperhive#7");
|
||||||
two_sources.nodes[1].kind = NodeKind::Source { events: Vec::new() };
|
assert_eq!(got[0].op.source, "router:assigned");
|
||||||
assert!(two_sources.validate().is_err(), "two sources");
|
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn a_placeholder_expands_to_its_field_and_an_unknown_one_stays() {
|
fn a_placeholder_expands_to_its_field_and_an_unknown_one_is_an_error() {
|
||||||
let fields = event("assigned", &["a", "b"], &[], &[]);
|
let fields = event("assigned", &["a", "b"], &[], &[]);
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
expand("{repo}#{number} {assignees_added} {nope} {", &fields),
|
expand("{repo}#{number} {assignees_added} {", &fields),
|
||||||
"hyperhive/hyperhive#7 a, b {nope} {"
|
Ok("hyperhive/hyperhive#7 a, b {".to_owned())
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
expand("x {nope}", &fields),
|
||||||
|
Err("{nope} is no event field".to_owned())
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// A sink naming a placeholder the event lacks sends nothing, rather
|
||||||
|
/// than a todo under a literal `{name}` key.
|
||||||
|
#[test]
|
||||||
|
fn a_sink_with_an_unknown_placeholder_emits_nothing() {
|
||||||
|
let mut f = flow("assigned");
|
||||||
|
f.nodes[2].kind = NodeKind::Todo(TodoSink {
|
||||||
|
to: Recipients::Agent("triage".to_owned()),
|
||||||
|
op: Op::Upsert,
|
||||||
|
key: "forge:{repo}#{numbr}:triage".to_owned(),
|
||||||
|
prefix: false,
|
||||||
|
text: "t".to_owned(),
|
||||||
|
});
|
||||||
|
let fields = event("assigned", &["atlas"], &[], &[]);
|
||||||
|
assert!(evaluate(&f, "issue_assign", &fields).is_empty());
|
||||||
|
// Control: spelled right, the same flow emits.
|
||||||
|
if let NodeKind::Todo(s) = &mut f.nodes[2].kind {
|
||||||
|
s.key = "forge:{repo}#{number}:triage".to_owned();
|
||||||
|
}
|
||||||
|
assert_eq!(evaluate(&f, "issue_assign", &fields).len(), 1);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -4,7 +4,8 @@
|
||||||
//! (`swarm_wire::forge_events`), runs every flow over each delivery
|
//! (`swarm_wire::forge_events`), runs every flow over each delivery
|
||||||
//! ([`flow`], [`route`]), and queues the todo ops the flows emit on each
|
//! ([`flow`], [`route`]), and queues the todo ops the flows emit on each
|
||||||
//! recipient agent's subject of `ROUTER_TODOS` (`swarm_wire::router_todo`).
|
//! recipient agent's subject of `ROUTER_TODOS` (`swarm_wire::router_todo`).
|
||||||
//! Each agent's harness applies them to its own todo list.
|
//! Each agent's harness applies them to its own todo list. The flows are the
|
||||||
|
//! built-in ones, changed by the `notify-flows` document ([`stored`]).
|
||||||
//!
|
//!
|
||||||
//! Per issue it keeps a snapshot in its state bucket ([`event::Snapshot`]):
|
//! Per issue it keeps a snapshot in its state bucket ([`event::Snapshot`]):
|
||||||
//! the assignees it last saw and the agents it queued a todo for. An event is
|
//! the assignees it last saw and the agents it queued a todo for. An event is
|
||||||
|
|
@ -31,6 +32,7 @@ mod event;
|
||||||
mod flow;
|
mod flow;
|
||||||
mod route;
|
mod route;
|
||||||
mod settle;
|
mod settle;
|
||||||
|
mod stored;
|
||||||
|
|
||||||
/// The cert-auth role this service logs in to the store as. `swarm-bao.nix`
|
/// The cert-auth role this service logs in to the store as. `swarm-bao.nix`
|
||||||
/// writes it, matching the leaf's subject.
|
/// writes it, matching the leaf's subject.
|
||||||
|
|
@ -83,17 +85,7 @@ async fn main() -> Result<()> {
|
||||||
})
|
})
|
||||||
.await?;
|
.await?;
|
||||||
|
|
||||||
let flows: Vec<flow::Flow> = flow::builtins()
|
let mut router = Router::open(&queue).await?;
|
||||||
.into_iter()
|
|
||||||
.filter(|f| match f.validate() {
|
|
||||||
Ok(()) => true,
|
|
||||||
Err(reason) => {
|
|
||||||
tracing::error!(flow = %f.id, %reason, "notify router: skipping a flow that is not a tree");
|
|
||||||
false
|
|
||||||
}
|
|
||||||
})
|
|
||||||
.collect();
|
|
||||||
let mut router = Router::open(&queue, flows).await?;
|
|
||||||
tracing::info!(
|
tracing::info!(
|
||||||
stream = forge_events::STREAM,
|
stream = forge_events::STREAM,
|
||||||
"notify router: routing forge events"
|
"notify router: routing forge events"
|
||||||
|
|
@ -113,12 +105,12 @@ struct Router {
|
||||||
js: jetstream::Context,
|
js: jetstream::Context,
|
||||||
events: jetstream::consumer::Consumer<pull::Config>,
|
events: jetstream::consumer::Consumer<pull::Config>,
|
||||||
state: kv::Store,
|
state: kv::Store,
|
||||||
flows: Vec<flow::Flow>,
|
flows: stored::Flows,
|
||||||
roster: Option<(Instant, BTreeSet<String>)>,
|
roster: Option<(Instant, BTreeSet<String>)>,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl Router {
|
impl Router {
|
||||||
async fn open(queue: &async_nats::Client, flows: Vec<flow::Flow>) -> Result<Self> {
|
async fn open(queue: &async_nats::Client) -> Result<Self> {
|
||||||
let js = jetstream::new(queue.clone());
|
let js = jetstream::new(queue.clone());
|
||||||
let events = swarm_queue_client::forge_events::open_or_create(queue)
|
let events = swarm_queue_client::forge_events::open_or_create(queue)
|
||||||
.await
|
.await
|
||||||
|
|
@ -154,7 +146,7 @@ impl Router {
|
||||||
js,
|
js,
|
||||||
events,
|
events,
|
||||||
state,
|
state,
|
||||||
flows,
|
flows: stored::Flows::builtins(),
|
||||||
roster: None,
|
roster: None,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
@ -193,8 +185,9 @@ impl Router {
|
||||||
}),
|
}),
|
||||||
None => event::Snapshot::default(),
|
None => event::Snapshot::default(),
|
||||||
};
|
};
|
||||||
|
self.flows.refresh(&self.js).await;
|
||||||
let roster = self.roster().await.map_err(Skip::Retry)?;
|
let roster = self.roster().await.map_err(Skip::Retry)?;
|
||||||
let routed = route::route(&event_type, &delivery, &snapshot, &self.flows, |a| {
|
let routed = route::route(&event_type, &delivery, &snapshot, &self.flows.active, |a| {
|
||||||
roster.contains(a)
|
roster.contains(a)
|
||||||
});
|
});
|
||||||
for action in &routed.actions {
|
for action in &routed.actions {
|
||||||
|
|
|
||||||
|
|
@ -6,7 +6,9 @@ use std::collections::BTreeSet;
|
||||||
use swarm_wire::router_todo::Op;
|
use swarm_wire::router_todo::Op;
|
||||||
|
|
||||||
use crate::event::{Delivery, Snapshot, fields, is_assign};
|
use crate::event::{Delivery, Snapshot, fields, is_assign};
|
||||||
use crate::flow::{Action, Flow};
|
use swarm_wire::notify_flows::Flow;
|
||||||
|
|
||||||
|
use crate::flow::{Action, evaluate};
|
||||||
|
|
||||||
/// What one event comes to.
|
/// What one event comes to.
|
||||||
#[derive(Debug, PartialEq, Eq)]
|
#[derive(Debug, PartialEq, Eq)]
|
||||||
|
|
@ -30,7 +32,7 @@ pub fn route(
|
||||||
let fields = fields(event_type, delivery, snapshot);
|
let fields = fields(event_type, delivery, snapshot);
|
||||||
let mut actions = Vec::new();
|
let mut actions = Vec::new();
|
||||||
let mut skipped = Vec::new();
|
let mut skipped = Vec::new();
|
||||||
for action in flows.iter().flat_map(|f| f.evaluate(event_type, &fields)) {
|
for action in flows.iter().flat_map(|f| evaluate(f, event_type, &fields)) {
|
||||||
if is_agent(&action.agent) {
|
if is_agent(&action.agent) {
|
||||||
actions.push(action);
|
actions.push(action);
|
||||||
} else if !skipped.contains(&action.agent) {
|
} else if !skipped.contains(&action.agent) {
|
||||||
|
|
@ -68,7 +70,7 @@ pub fn route(
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
mod tests {
|
mod tests {
|
||||||
use super::*;
|
use super::*;
|
||||||
use crate::flow::builtins;
|
use swarm_wire::notify_flows::builtins;
|
||||||
|
|
||||||
fn delivery(action: &str, assignees: &[&str]) -> Delivery {
|
fn delivery(action: &str, assignees: &[&str]) -> Delivery {
|
||||||
Delivery {
|
Delivery {
|
||||||
|
|
|
||||||
259
swarm-notify-router/src/stored.rs
Normal file
259
swarm-notify-router/src/stored.rs
Normal file
|
|
@ -0,0 +1,259 @@
|
||||||
|
//! The flows the router runs: the built-ins, changed by the document in the
|
||||||
|
//! `notify-flows` bucket.
|
||||||
|
//!
|
||||||
|
//! The router reads the key again every [`REFRESH`] and runs what
|
||||||
|
//! [`Document::active`] says. What it does with each thing it can read:
|
||||||
|
//!
|
||||||
|
//! | the key | the router runs |
|
||||||
|
//! | ------------------------------------ | ----------------------------------- |
|
||||||
|
//! | absent, deleted, or no such bucket | the built-ins |
|
||||||
|
//! | a document it reads | its built-ins and custom flows |
|
||||||
|
//! | a document it cannot read | what it ran before |
|
||||||
|
//! | the read itself failing | what it ran before |
|
||||||
|
//!
|
||||||
|
//! "Cannot read" is not JSON, not the document's shape, or a version this
|
||||||
|
//! build does not know. Before any document has been read, "what it ran
|
||||||
|
//! before" is the built-ins, so a bad document never leaves the router with
|
||||||
|
//! no flows.
|
||||||
|
|
||||||
|
use std::time::{Duration, Instant};
|
||||||
|
|
||||||
|
use async_nats::jetstream::ErrorCode;
|
||||||
|
use async_nats::jetstream::context::GetStreamErrorKind;
|
||||||
|
use async_nats::jetstream::{self, kv};
|
||||||
|
use swarm_wire::notify_flows::{self, Document, Flow};
|
||||||
|
|
||||||
|
/// How long a read of the key is reused.
|
||||||
|
const REFRESH: Duration = Duration::from_secs(10);
|
||||||
|
|
||||||
|
/// What a read of the key came to.
|
||||||
|
#[derive(Debug)]
|
||||||
|
pub enum Read {
|
||||||
|
/// No key, a deleted key, or no bucket.
|
||||||
|
Absent,
|
||||||
|
Document {
|
||||||
|
revision: u64,
|
||||||
|
bytes: Vec<u8>,
|
||||||
|
},
|
||||||
|
Failed(String),
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The flows in force, and which revision of the key they came from.
|
||||||
|
pub struct Flows {
|
||||||
|
pub active: Vec<Flow>,
|
||||||
|
/// The revision `active` was read from; `None` for the built-ins alone.
|
||||||
|
revision: Option<u64>,
|
||||||
|
/// The last problem logged, so a standing one is logged once.
|
||||||
|
problem: Option<String>,
|
||||||
|
checked: Option<Instant>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Flows {
|
||||||
|
/// The built-ins, which is what the router runs until it reads a document.
|
||||||
|
pub fn builtins() -> Self {
|
||||||
|
Self {
|
||||||
|
active: Document::default().active().0,
|
||||||
|
revision: None,
|
||||||
|
problem: None,
|
||||||
|
checked: None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Take what a read came to.
|
||||||
|
pub fn apply(&mut self, read: Read) {
|
||||||
|
match read {
|
||||||
|
Read::Failed(why) => self.note(format!("reading the key failed: {why}")),
|
||||||
|
Read::Absent => {
|
||||||
|
if self.revision.take().is_some() {
|
||||||
|
tracing::info!(
|
||||||
|
"notify router: the flows document is gone; running the built-in flows"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
self.active = Document::default().active().0;
|
||||||
|
self.problem = None;
|
||||||
|
}
|
||||||
|
Read::Document { revision, .. } if self.revision == Some(revision) => {}
|
||||||
|
Read::Document { revision, bytes } => match Document::decode(&bytes) {
|
||||||
|
Err(why) => {
|
||||||
|
// The revision is not taken, so the next read looks at
|
||||||
|
// the document again and the problem is logged once.
|
||||||
|
self.note(format!("revision {revision} cannot be read: {why}"));
|
||||||
|
}
|
||||||
|
Ok(doc) => {
|
||||||
|
let (active, skipped) = doc.active();
|
||||||
|
for why in &skipped {
|
||||||
|
tracing::error!(%why, "notify router: skipping a flow");
|
||||||
|
}
|
||||||
|
tracing::info!(
|
||||||
|
revision,
|
||||||
|
flows = active.len(),
|
||||||
|
disabled = doc.disabled_builtins.len(),
|
||||||
|
custom = doc.flows.len(),
|
||||||
|
"notify router: running the flows document"
|
||||||
|
);
|
||||||
|
self.active = active;
|
||||||
|
self.revision = Some(revision);
|
||||||
|
self.problem = None;
|
||||||
|
}
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn note(&mut self, problem: String) {
|
||||||
|
if self.problem.as_ref() != Some(&problem) {
|
||||||
|
tracing::error!(%problem, "notify router: keeping the flows it has");
|
||||||
|
self.problem = Some(problem);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Read the key if the last read is older than [`REFRESH`].
|
||||||
|
pub async fn refresh(&mut self, js: &jetstream::Context) {
|
||||||
|
if self.checked.is_some_and(|at| at.elapsed() < REFRESH) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
self.checked = Some(Instant::now());
|
||||||
|
self.apply(read(js).await);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn read(js: &jetstream::Context) -> Read {
|
||||||
|
// A bucket that does not exist is the answer "no document". Any other
|
||||||
|
// failure to look, such as a timeout, says nothing about the document.
|
||||||
|
match js.get_stream(format!("KV_{}", notify_flows::BUCKET)).await {
|
||||||
|
Ok(_) => {}
|
||||||
|
Err(e)
|
||||||
|
if matches!(e.kind(), GetStreamErrorKind::JetStream(j)
|
||||||
|
if j.error_code() == ErrorCode::STREAM_NOT_FOUND) =>
|
||||||
|
{
|
||||||
|
return Read::Absent;
|
||||||
|
}
|
||||||
|
Err(e) => return Read::Failed(e.to_string()),
|
||||||
|
}
|
||||||
|
let store = match js.get_key_value(notify_flows::BUCKET).await {
|
||||||
|
Ok(store) => store,
|
||||||
|
Err(e) => return Read::Failed(e.to_string()),
|
||||||
|
};
|
||||||
|
match store.entry(notify_flows::KEY).await {
|
||||||
|
Ok(Some(entry)) if entry.operation == kv::Operation::Put => Read::Document {
|
||||||
|
revision: entry.revision,
|
||||||
|
bytes: entry.value.to_vec(),
|
||||||
|
},
|
||||||
|
Ok(_) => Read::Absent,
|
||||||
|
Err(e) => Read::Failed(e.to_string()),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use swarm_wire::notify_flows::{Document, VERSION};
|
||||||
|
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
fn ids(f: &Flows) -> Vec<&str> {
|
||||||
|
f.active.iter().map(|f| f.id.as_str()).collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn doc(disabled: &[&str], custom: &[&str]) -> Read {
|
||||||
|
let mut flows = Vec::new();
|
||||||
|
for id in custom {
|
||||||
|
let mut f = notify_flows::builtins().remove(0);
|
||||||
|
f.id = (*id).to_owned();
|
||||||
|
flows.push(f);
|
||||||
|
}
|
||||||
|
let d = Document {
|
||||||
|
version: VERSION,
|
||||||
|
disabled_builtins: disabled.iter().map(|s| (*s).to_owned()).collect(),
|
||||||
|
flows,
|
||||||
|
};
|
||||||
|
Read::Document {
|
||||||
|
revision: 0,
|
||||||
|
bytes: serde_json::to_vec(&d).unwrap(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn at(revision: u64, read: Read) -> Read {
|
||||||
|
match read {
|
||||||
|
Read::Document { bytes, .. } => Read::Document { revision, bytes },
|
||||||
|
other => other,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn junk(revision: u64, bytes: &[u8]) -> Read {
|
||||||
|
Read::Document {
|
||||||
|
revision,
|
||||||
|
bytes: bytes.to_vec(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn with_no_document_the_router_runs_the_builtins() {
|
||||||
|
let mut f = Flows::builtins();
|
||||||
|
assert_eq!(ids(&f), ["assigned", "unassigned", "closed"]);
|
||||||
|
f.apply(Read::Absent);
|
||||||
|
assert_eq!(ids(&f), ["assigned", "unassigned", "closed"]);
|
||||||
|
// An empty document is the same.
|
||||||
|
f.apply(at(1, doc(&[], &[])));
|
||||||
|
assert_eq!(ids(&f), ["assigned", "unassigned", "closed"]);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_document_switches_builtins_off_and_adds_custom_flows() {
|
||||||
|
let mut f = Flows::builtins();
|
||||||
|
f.apply(at(1, doc(&["unassigned"], &["triage"])));
|
||||||
|
assert_eq!(ids(&f), ["assigned", "closed", "triage"]);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_document_it_cannot_read_leaves_the_flows_as_they_were() {
|
||||||
|
let mut f = Flows::builtins();
|
||||||
|
f.apply(at(1, doc(&["closed"], &["triage"])));
|
||||||
|
let before = ids(&f).join(",");
|
||||||
|
assert_eq!(before, "assigned,unassigned,triage");
|
||||||
|
for bad in [
|
||||||
|
junk(2, b"not json"),
|
||||||
|
junk(3, br#"{"version":2,"flows":[]}"#),
|
||||||
|
junk(4, br#"{"flows":[{"id":"x"}]}"#),
|
||||||
|
Read::Failed("timeout".to_owned()),
|
||||||
|
] {
|
||||||
|
f.apply(bad);
|
||||||
|
assert_eq!(ids(&f).join(","), before);
|
||||||
|
}
|
||||||
|
// The next good revision replaces them: the control that the bad
|
||||||
|
// ones were kept out by being bad, not by the loader being stuck.
|
||||||
|
f.apply(at(5, doc(&[], &[])));
|
||||||
|
assert_eq!(ids(&f), ["assigned", "unassigned", "closed"]);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_bad_first_document_is_the_builtins_not_nothing() {
|
||||||
|
let mut f = Flows::builtins();
|
||||||
|
f.apply(junk(1, br#"{"version":9}"#));
|
||||||
|
assert_eq!(ids(&f), ["assigned", "unassigned", "closed"]);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_deleted_document_goes_back_to_the_builtins() {
|
||||||
|
let mut f = Flows::builtins();
|
||||||
|
f.apply(at(1, doc(&["closed"], &["triage"])));
|
||||||
|
f.apply(Read::Absent);
|
||||||
|
assert_eq!(ids(&f), ["assigned", "unassigned", "closed"]);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_flow_that_is_not_valid_is_left_out_and_the_rest_run() {
|
||||||
|
let mut f = Flows::builtins();
|
||||||
|
let mut broken = notify_flows::builtins().remove(0);
|
||||||
|
broken.id = "broken".to_owned();
|
||||||
|
broken.edges.clear();
|
||||||
|
let d = Document {
|
||||||
|
version: VERSION,
|
||||||
|
disabled_builtins: Vec::new(),
|
||||||
|
flows: vec![broken],
|
||||||
|
};
|
||||||
|
f.apply(Read::Document {
|
||||||
|
revision: 1,
|
||||||
|
bytes: serde_json::to_vec(&d).unwrap(),
|
||||||
|
});
|
||||||
|
assert_eq!(ids(&f), ["assigned", "unassigned", "closed"]);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -18,6 +18,7 @@ pub mod builder;
|
||||||
pub mod claude_login;
|
pub mod claude_login;
|
||||||
pub mod forge_events;
|
pub mod forge_events;
|
||||||
pub mod notices;
|
pub mod notices;
|
||||||
|
pub mod notify_flows;
|
||||||
pub mod notify_router;
|
pub mod notify_router;
|
||||||
pub mod review_classifier;
|
pub mod review_classifier;
|
||||||
pub mod router_todo;
|
pub mod router_todo;
|
||||||
|
|
|
||||||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Reference in a new issue