0f39bdaaa4
Account minting, knowledge sync, the archetype refresh, the matrix appservice flow and the SSO secret delivery move to pages under 4-implementation/swarm. Each handbook page keeps what an operator does or sees, with a pointer.
90 lines
4.5 KiB
Markdown
90 lines
4.5 KiB
Markdown
# Knowledge repo: sync, change notice and state
|
|
|
|
How a hive keeps its clone of `internal/knowledge` current, tells agents when it
|
|
changes, and mounts it into each container. The operator's view is
|
|
[the knowledge page](../../2-handbook/swarm/knowledge.md).
|
|
|
|
## Sync mechanism
|
|
|
|
hive-c0re maintains the local clone at
|
|
`/var/lib/hyperhive/knowledge` via two paths:
|
|
|
|
1. **Swarm event** — the swarm controller holds the single push hook on
|
|
`internal/knowledge` (see
|
|
[swarm-wide forge webhooks](controller.md#swarm-wide-forge-webhooks)). On any push to main, including merge commits, it sends an
|
|
event to every hive over the swarm queue and each hive runs `git
|
|
pull`, so agents see the new content on their next turn.
|
|
|
|
A hive that's offline when the swarm controller sends the event
|
|
doesn't get it on reconnect — the periodic pull below is what closes that gap. One
|
|
hive briefly showing older `/knowledge` content than another is
|
|
expected, and resolves by itself within the fallback interval.
|
|
|
|
**don't add a per-hive hook.** A webhook has exactly one target
|
|
URL, so a second registration against the same repo doesn't add a
|
|
recipient — it takes delivery away from whoever registered first.
|
|
Earlier versions had each hive register its own, and later ones
|
|
removed that leftover at startup. A hive upgraded straight past those
|
|
versions still holds its old hook: delete any hook on the repo that
|
|
points at a hive's own `/webhook/knowledge`.
|
|
|
|
2. **Periodic pull** — a background task in `hive-c0re::main`
|
|
queues a pull on a fixed cadence as a fallback (webhook missed, c0re
|
|
restarted between pushes). The pull is best-effort — a failure
|
|
shows as a failed `KnowledgePull` node on the job queue, counts toward
|
|
the `knowledge_pull` hive warning, and doesn't affect the rest of the daemon.
|
|
|
|
A failed pull of either kind, or the one at boot, also queues a retry after 5 s,
|
|
doubling per failure in a row up to 5 min, until a pull succeeds. The
|
|
`knowledge_pull` hive warning appears after seven failures in a row, about five
|
|
minutes in, so a forge still starting at boot raises nothing.
|
|
|
|
Both paths, and the one pull at boot, queue the same `KnowledgePull` job
|
|
node. It holds the working tree for its duration, so two pulls never run
|
|
over each other — each trigger queues its own pass, and a pass that lands
|
|
while another is already queued or running waits its turn, since the
|
|
one ahead of it may have fetched before the push. The node runs
|
|
`knowledge::pull()`, which also handles the change notice below.
|
|
|
|
### Change notice
|
|
|
|
When a pull actually moves the local clone's `HEAD` (a real change,
|
|
not a no-op — for example the periodic pull finding nothing new), hive-c0re
|
|
broadcasts a short notice to every currently registered agent's inbox:
|
|
sender `system`, body `[system] /knowledge updated:` followed by a
|
|
`git diff --stat <old>..<new>` summary of what changed (or a generic
|
|
"see the repo" fallback if computing the diff itself fails). This is
|
|
the same broadcast mechanism used for other hive-wide notices — inbox
|
|
message only, no forced wake, and it carries the standard "this was a
|
|
broadcast" hint. hive-c0re logs a diff or per-agent send failure
|
|
without blocking the pull itself.
|
|
|
|
On a hive with no local clone yet, `knowledge::pull()` takes the clone
|
|
instead of pulling, so every hive gets `/knowledge` from its first pull,
|
|
whether or not it runs the forge. It authenticates with the hive's forge
|
|
read token when it has one (see
|
|
[forge-accounts.md](forge-accounts.md#how-a-hive-reads-the-forge)) and clones the public
|
|
repo anonymously otherwise.
|
|
|
|
## State
|
|
|
|
<!-- vale write-good.Passive = NO -->
|
|
|
|
- **Host clone**: `/var/lib/hyperhive/knowledge` — persists across
|
|
hive-c0re restarts and agent destroy/recreate. Deleted only by
|
|
manual operator action.
|
|
- **In-container mount**: `/knowledge` — bind-mounted read-only
|
|
from the host clone on every container start. Gone when container
|
|
is stopped; reappears on next start with the current clone state.
|
|
<!-- vale write-good.Passive = YES -->
|
|
|
|
The mount deliberately **excludes `.git`**: hive-priv overlays an empty
|
|
tmpfs at `/knowledge/.git`, so agents see the documents, not the branches or
|
|
history. An older clone also kept a token in `.git/config`; `knowledge::pull()`
|
|
rewrites that remote to the plain URL.
|
|
|
|
hive-priv adds the overlay only when the host clone has a `.git`. Before the
|
|
first clone succeeds, the agent starts with an empty `/knowledge` and no
|
|
overlay, because nspawn can't create the mount point inside the read-only
|
|
bind. A container that started in that window sees the `.git` once the clone
|
|
lands, until its next start.
|