remove what no longer has a reader: build-log store, dashboard sent/delivered events, graph resource dep
This commit is contained in:
atlas
mara
parent
32f5b74ba4
commit
49266e4410
5 changed files with 204 additions and 1054 deletions
|
|
@ -142,45 +142,20 @@ hive-c0re reads this file on each `build_all` sweep (~10s) via
|
|||
sentinel files (`hyperhive-rate-limited`, `hyperhive-needs-login`) when the
|
||||
JSON is absent.
|
||||
|
||||
### `/var/lib/hyperhive/db/build_logs.sqlite` (host)
|
||||
### Container-operation logs (host)
|
||||
|
||||
Full stdout + stderr capture for every `nixos-container` / `nix
|
||||
build` invocation the lifecycle layer fires. One row per invocation;
|
||||
the row accumulates lines as the child runs.
|
||||
|
||||
Capturing the full stream (rather than a short tail buffer) matters
|
||||
because real eval errors routinely run long — "tried alternatives"
|
||||
blocks alone are often 30+ lines — so a truncated tail would cut off
|
||||
the actual failure and leave only the host journal holding the
|
||||
complete output. This table holds the entire log; no HTTP route serves
|
||||
it.
|
||||
|
||||
Three indices:
|
||||
|
||||
- `(agent, started_at)` — per-agent lookup by recency.
|
||||
- `(status, finished_at)` — backs the retention sweep that runs
|
||||
as part of the existing hourly vacuum.
|
||||
- `(node_id)` — added by a later migration so a build log row can be
|
||||
looked up by the job-queue node it belongs to (a `hive_jobq` node is
|
||||
immutable after insert, so hive-c0re records the link on the log row
|
||||
instead); `node_id` is nullable.
|
||||
|
||||
Writes are best-effort: `append_stdout` / `append_stderr` / `finish`
|
||||
log a warning on sqlite error and let the build continue. A failed
|
||||
log row never blocks a rebuild.
|
||||
|
||||
Each operation also logs to the host journal, target `nixos-container`, and the
|
||||
swarm log store collects those lines. Every line carries `build_id` (this
|
||||
table's row id; empty without the store), `agent`, `kind` and `cmdline`. A
|
||||
hive-c0re keeps no database of `nixos-container` / `nix build` output. Each
|
||||
operation logs to the host journal, target `nixos-container`, and the swarm log
|
||||
store collects those lines. Every line carries `agent`, `kind` and `cmdline`. A
|
||||
`start` line (`event=start`) opens the build, one line follows per line of
|
||||
output, and a `finish` line (`event=finish`) closes it with `status` (`ok` or
|
||||
`fail`) and `runtime_secs`. hive-c0re logs a nix warning or error at warn, and a failed
|
||||
finish too.
|
||||
`fail`) and `runtime_secs`. hive-c0re logs a nix warning or error at warn, and a
|
||||
failed finish too. A failed `start` also logs its error and the container
|
||||
unit's journal tail.
|
||||
|
||||
A row is open (`finished_at IS NULL`) only while its child runs. At startup
|
||||
`Coordinator::open` closes every row still open as `fail` and appends a
|
||||
line saying hive-c0re stopped: the previous process died mid-build, and
|
||||
the retention sweep never deletes open rows.
|
||||
An earlier hive-c0re stored the same output in
|
||||
`/var/lib/hyperhive/db/build_logs.sqlite`. Nothing reads or writes that file now, and
|
||||
hive-c0re leaves it on disk untouched.
|
||||
|
||||
### `/harness/hyperhive-model` (per agent)
|
||||
|
||||
|
|
|
|||
|
|
@ -25,8 +25,7 @@ from the same basic nodes.
|
|||
### Two levels: DAG and node
|
||||
|
||||
The **DAG** is the unit of cancel and the
|
||||
dashboard group; the **node** is the unit of scheduling / execution /
|
||||
build-log. Deps are intra-DAG edges only (`AfterOk` by default:
|
||||
dashboard group; the **node** is the unit of scheduling / execution. Deps are intra-DAG edges only (`AfterOk` by default:
|
||||
the dep must succeed, a failed/cancelled dep cancels the dependent —
|
||||
cancel-downstream). Cross-DAG ordering comes from the per-agent lease,
|
||||
never from edges between DAGs.
|
||||
|
|
@ -264,13 +263,11 @@ That endpoint serves the graph generically (`hive-jobq-wire`): every node
|
|||
carries `id`, `parent`, `deps`, `state`, `label` (the node kind's own wire
|
||||
string — the kind _is_ the phase label, there is no separate sub-step
|
||||
string) and free-form `data` for what only some kinds have (`agent`,
|
||||
`inputs`, `build_log_id`). Group roots ride as ordinary
|
||||
`inputs`). Group roots ride as ordinary
|
||||
nodes, so a group's state is just the root's own `state`.
|
||||
|
||||
No group-level `agent` exists — agent is per-node, so one group can
|
||||
span agents; consumers derive a group's agents from its nodes. Build logs
|
||||
are likewise **per-node**: the dashboard renders the node tree and keys the
|
||||
live-log panel off the running node.
|
||||
span agents; consumers derive a group's agents from its nodes.
|
||||
|
||||
The event carries no payload by design: shipping a typed projection of
|
||||
the whole queue in the event itself would be a second rendering of the
|
||||
|
|
|
|||
|
|
@ -32,7 +32,7 @@ known operations; there is no arbitrary command pass-through:
|
|||
| ------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `StartContainer` / `StopContainer` | `nixos-container start/stop <name>` |
|
||||
| `KillContainer` | `machinectl kill <machine> --signal=SIGKILL` (`nixos-container` has no kill verb) |
|
||||
| `StartFailureLog` | `journalctl _SYSTEMD_UNIT=container@h-<name>.service --since @<t> -n 30 -o cat --no-pager`, fixed argv, 5 s timeout, output capped at 8 KiB; hive-priv clamps `<t>` to the last hour. hive-c0re appends the lines to the failed start's build log, which only the dashboard serves. |
|
||||
| `StartFailureLog` | `journalctl _SYSTEMD_UNIT=container@h-<name>.service --since @<t> -n 30 -o cat --no-pager`, fixed argv, 5 s timeout, output capped at 8 KiB; hive-priv clamps `<t>` to the last hour. hive-c0re logs the lines under the failed start's `nixos-container` tag. |
|
||||
| `CreateContainer` / `UpdateContainer` | `nixos-container create <name> --system-path <toplevel>`; for an update, `nix-env -p <profile> --set <toplevel>` and a reload of the unit. `<toplevel>` must already be in the store with a signature by the swarm cache's key: hive-priv first runs `nix store copy-sigs --store daemon` for that path from the cache named in `/run/hyperhive-nix/cache.conf`, with the netrc beside it, which imports the cache's signature for a path that was already in the store (on a failure it logs a warning and the check goes on); it then runs `nix store verify --no-contents --sigs-needed 1` with `trusted-public-keys` set to the key in root-owned `/run/hyperhive-priv/attic/public-key`, and refuses a path that's absent or that the key didn't sign. hive-priv builds nothing. The check doesn't bind the path to the agent whose pull produced it: hive-c0re could already pull any swarm-signed path and apply it to any agent, so the check bounds the same risk (one agent gets another's closure), not an arbitrary-path one |
|
||||
| `RealiseClosure` | Kept for one release, for an older hive-c0re that still asks hive-priv to download. Downloads nothing: answers `Ok` when the path is in the store and signed by the swarm key (what `CreateContainer` needs), else `not available: <reason>`. The path must be `/nix/store/<32 chars>-<name>` and the agent name valid |
|
||||
| `RefreshCacheFacts` | `systemctl start hive-priv-cache-facts.service`, waited for up to 60 s. The request names no unit and has no field. hive-c0re asks for it when the fragment is missing or lacks a setting |
|
||||
|
|
|
|||
|
|
@ -433,19 +433,14 @@ below — some endpoints aren't in it yet.
|
|||
`{ "marked": N }`. Agent name validated against
|
||||
`[a-z0-9_-]`, 1-63 chars; 400 on bad input.
|
||||
|
||||
- `GET /api/dashboard/stream` — unified live event channel:
|
||||
broker `sent` / `delivered`, plus the mutation events listed
|
||||
below. <!-- vale write-good.Passive = YES -->
|
||||
- `GET /api/dashboard/stream` — unified live event channel: the
|
||||
mutation events listed below. <!-- vale write-good.Passive = YES -->
|
||||
|
||||
## Dashboard event channel
|
||||
|
||||
Wire vocabulary on `/api/dashboard/stream` (kind tag is in the JSON
|
||||
payload):
|
||||
|
||||
- `sent` / `delivered` — broker traffic, mirrored from the
|
||||
intra-process channel by a forwarder task. Both carry `id: i64`
|
||||
(the broker row id) and `in_reply_to: Option<i64>` for thread
|
||||
rendering.
|
||||
- `transient_set` (name, transient_kind, since_unix) /
|
||||
`transient_cleared` (name) — lifecycle action spinners. The
|
||||
client ticks the elapsed-seconds badge off `since_unix`
|
||||
|
|
|
|||
|
|
@ -52,9 +52,8 @@
|
|||
domain would exhaust this budget on a live hive; dispatching
|
||||
by `kind` on the client is a one-liner. Per-domain splits are
|
||||
reserved for high-volume sub-streams most consumers skip (none
|
||||
exist yet). The broker's intra-process channel stays separate
|
||||
from the dashboard channel to avoid coupling `recv_blocking_batch`
|
||||
(hot path inside the harness turn loop) to presentation concerns.
|
||||
exist yet). Broker messages don't ride it: the broker's intra-process
|
||||
channel only wakes `recv_blocking_batch`.
|
||||
**SSE multiplexing**: the dashboard uses a
|
||||
`SharedWorker` (`stream-worker.js`) to hold one upstream
|
||||
`EventSource` per URL. All same-origin tabs share this worker
|
||||
|
|
|
|||
Loading…
Reference in a new issue