29aa242201
After a rollback, a record a newer build wrote (a new enum variant, a changed required field) stopped decoding, and the strict `schedules::read` turned that into an error on every read and write of the key until someone deleted it by hand. - agent_limits / inference (one document per agent): an undecodable document reads as unset, with a WARN naming the bucket, key and error. The PUT's "equal to stored, skip the write" shortcut no longer applies to an undecodable document, so a PUT of the defaults replaces it too. - schedule definitions (one document, every schedule): salvaged per definition, as the review-classifier run list is. An undecodable definition is kept as its raw JSON, hidden from the list, edits and projections, written back by every commit after the readable ones, and its id is never minted again. A document with no list of schedules stays an error: reading it as empty would empty every agent's key. - per-agent agent-schedules: an undecodable key reads as no schedules and is rewritten on the next projection, even when the agent has none. `schedules::read` is replaced by `read_or_default`; `wanted` keeps its own strict read. Closes #5193
152 lines
7.8 KiB
Markdown
152 lines
7.8 KiB
Markdown
# Scheduled prompts
|
|
|
|
A scheduled prompt hands one or more agents the same text at set times: once,
|
|
or every interval from a start time. Each prompt has a name, its title. The
|
|
swarm keeps every prompt, and each agent it names fires it in its own
|
|
container, so a fire needs neither the hive nor the swarm controller to be up
|
|
at that moment, only the agent.
|
|
|
|
Operators manage them with `swarmctl schedule` on the swarm controller's
|
|
host. Agents can't create, change or remove one. For a one-off wake an agent
|
|
sets itself, it uses `remind`, which stays inside its container.
|
|
|
|
## Managing them
|
|
|
|
```console
|
|
# swarmctl schedule add --name standup --agent scribe --agent iris \
|
|
--body "post the standup summary" --at 2026-10-05T09:00:00+02:00 --every 1d
|
|
added schedule kqzpmwab
|
|
kqzpmwab "standup" every 1d from 2026-10-05T07:00:00Z next 2026-10-05T07:00:00Z agents: iris, scribe | post the standup summary
|
|
|
|
# swarmctl schedule list
|
|
# swarmctl schedule list --agent scribe
|
|
# swarmctl schedule edit kqzpmwab --body "post the standup summary to the team room"
|
|
# swarmctl schedule edit kqzpmwab --add-agent atlas --remove-agent iris
|
|
# swarmctl schedule pause kqzpmwab
|
|
# swarmctl schedule resume kqzpmwab
|
|
# swarmctl schedule rm kqzpmwab
|
|
```
|
|
|
|
- `--agent` names an agent to fire the prompt in; repeat it for more. Each
|
|
must be an agent some hive's wanted state declares.
|
|
- `--at` takes an RFC 3339 time with its offset, or unix seconds. Without it
|
|
the first fire is now. A time in the past fires its latest past time once,
|
|
as soon as each agent reads the prompt.
|
|
- `--every` takes a count and a unit, `s`, `m`, `h` or `d`. Without it the
|
|
prompt fires once.
|
|
- `schedule edit` changes only the flags you pass and keeps the id. `--once`
|
|
turns a recurring prompt into a one-shot. `--add-agent` and
|
|
`--remove-agent` change who gets it; removing the last agent fails, so
|
|
remove the prompt instead.
|
|
- `schedule list` shows one row per prompt: id, name, cadence, start, next
|
|
fire time, agents, and the first line of the body.
|
|
|
|
The full flag reference is [`swarmctl-cli.md`](../tools/swarmctl-cli.md).
|
|
|
|
## When a prompt fires
|
|
|
|
A prompt's fire times are its start time plus whole intervals. Each agent
|
|
works them out from the prompt itself, so:
|
|
|
|
- **Editing the text or the name doesn't move them.** Neither does changing
|
|
the agents. Only a new `--at` or `--every` does.
|
|
- **An agent that was down or disconnected fires once.** When it's back, it
|
|
fires the latest time it missed and then carries on at the next one. The
|
|
earlier missed times don't fire.
|
|
- **A restart refires nothing.** Each agent records the last time it fired
|
|
for each prompt in its state database, and fires a time only when it's
|
|
later than that. An agent removed from a prompt and added back keeps that
|
|
record, so it doesn't refire a time it already fired.
|
|
- **A one-shot fires once**, even though it stays in `schedule list` until
|
|
you remove it. Moving its `--at` to a later time fires it again then.
|
|
- **A paused prompt doesn't fire**, and the times that pass while it's paused
|
|
don't fire when it resumes. An agent marks those times as passed while it
|
|
watches the paused prompt; if the agent was down for the whole pause, it
|
|
fires the latest time it missed once, as above.
|
|
|
|
A fire reaches the agent as a `schedule` loose end with the prompt's text.
|
|
The agent's turn loop wakes for it like any other loose end. If the agent
|
|
hasn't dealt with the previous fire yet, the new fire replaces it rather
|
|
than adding a second one; once the agent dismissed the previous one, the
|
|
next fire opens a new one.
|
|
|
|
Each agent reads its prompts every 10 seconds, so a fire can land up to 10
|
|
seconds after its time. A change reaches each agent within the same window,
|
|
or within a minute if the controller's first attempt to pass it on failed.
|
|
|
|
## When a prompt doesn't fire
|
|
|
|
- **The agent isn't connected to the swarm queue.** It reads its prompts with
|
|
its own swarm-queue credential. A harness that can't connect retries for
|
|
about five minutes, then exits and comes back; it reads no prompts until a
|
|
connect succeeds. See
|
|
[Giving the agents the queue too](README.md#giving-the-agents-the-queue-too).
|
|
- **The agent hasn't read its prompts since it started.** It fires from the
|
|
last list it read; until the first read succeeds it fires nothing. A read
|
|
that fails later keeps the previous list.
|
|
|
|
<details><summary>Implementation</summary>
|
|
|
|
swarm-controller is the only client that writes either of the two KV buckets
|
|
involved.
|
|
|
|
`schedule-definitions` holds every prompt as one JSON document under the key
|
|
`all`. Agents hold no grant on this bucket. Each change re-reads the key and
|
|
writes against its revision, so a prompt exists for every one of its agents or for
|
|
none of them, and two changes at once don't lose one. The controller leaves a
|
|
prompt it can't decode, such as one a newer build wrote, out of the list and
|
|
out of every agent's key, with a warning in its log. It keeps the prompt in
|
|
the document as it was, after the others, and won't reuse its id. A document
|
|
that isn't an object with a list of `schedules` is an error.
|
|
|
|
`agent-schedules` holds one key per agent: the prompts naming that agent,
|
|
without their names or other agents. A key the controller can't decode counts
|
|
as no prompts, with a warning in its log, and the next rewrite replaces it.
|
|
|
|
```json
|
|
{
|
|
"schedules": [
|
|
{
|
|
"id": "kqzpmwab",
|
|
"body": "post the standup summary",
|
|
"anchor_unix": 1791183600,
|
|
"interval_seconds": 86400,
|
|
"paused": false
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
`interval_seconds` is absent for a one-shot. After each change the controller
|
|
rewrites the key of every agent the change named before or after, and once a
|
|
minute it rewrites every agent's key, so a rewrite that failed converges. Ids
|
|
are 8 lowercase letters, the same in every agent's key, never reused, and so
|
|
never equal to a hive schedule's numeric id.
|
|
|
|
`swarmctl schedule` calls these routes over the controller's socket:
|
|
|
|
| method | path | body |
|
|
| -------- | --------------------- | ------------------------------------------------------------------------------------------------------------------------- |
|
|
| `GET` | `/api/schedules` | —; answers every prompt, sorted by name then id |
|
|
| `POST` | `/api/schedules` | `name`, `agents`, `body`, and optional `anchor_unix`, `interval_seconds`, `paused`; answers 201 with the prompt |
|
|
| `PATCH` | `/api/schedules/<id>` | any of `name`, `body`, `anchor_unix`, `interval_seconds` (`null` for a one-shot), `paused`, `add_agents`, `remove_agents` |
|
|
| `DELETE` | `/api/schedules/<id>` | —; answers 204 |
|
|
|
|
Each prompt they answer with has `id`, `name`, `agents` (sorted), `body`,
|
|
`anchor_unix`, `interval_seconds` (`null` for a one-shot), `paused`, and
|
|
`next_fire_unix`: the next time on the prompt's grid, as of the moment the
|
|
controller answers, `null` when paused or a one-shot whose time has passed.
|
|
|
|
An agent's own queue credential grants exactly one subject in this bucket:
|
|
`$JS.API.DIRECT.GET.KV_agent-schedules.$KV.agent-schedules.<agent>`,
|
|
the direct read of its own key. It can't read another agent's key, open or
|
|
watch the bucket, or write to it. A KV watch would need a consumer, and a
|
|
consumer create subject doesn't name a key, so granting one would let the
|
|
agent read every key; the agent reads its key every 10 seconds instead.
|
|
|
|
The harness (`hive-agent`'s `schedule_timer`) keeps the last fired time per
|
|
prompt id in the `schedule_fired_slots` table of its state database, and
|
|
pushes each fire as the todo `schedule`/`schedule:<id>` with source
|
|
`scheduled`, reopening an acknowledged one.
|
|
|
|
</details>
|