Compare commits
14 changed files with 28 additions and 325 deletions
11
README.md
11
README.md
|
|
@ -91,17 +91,6 @@ hive-c0re will then:
|
||||||
- auto-create the manager container (`hm1nd`) if missing,
|
- auto-create the manager container (`hm1nd`) if missing,
|
||||||
- auto-rebuild any managed container whose hyperhive rev is stale.
|
- auto-rebuild any managed container whose hyperhive rev is stale.
|
||||||
|
|
||||||
`claude-code` is unfree; hyperhive does not auto-allow it for you.
|
|
||||||
Add to your host config:
|
|
||||||
|
|
||||||
```nix
|
|
||||||
nixpkgs.config.allowUnfreePredicate =
|
|
||||||
pkg: builtins.elem (nixpkgs.lib.getName pkg) [ "claude-code" ];
|
|
||||||
```
|
|
||||||
|
|
||||||
(or `nixpkgs.config.allowUnfree = true`, your call). Each per-agent
|
|
||||||
container inherits this through the same nixpkgs evaluation.
|
|
||||||
|
|
||||||
## Build / deploy
|
## Build / deploy
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
|
|
|
||||||
27
TODO.md
27
TODO.md
|
|
@ -3,17 +3,6 @@
|
||||||
Pick anything from here when relevant. Cross-cutting design notes live in
|
Pick anything from here when relevant. Cross-cutting design notes live in
|
||||||
[CLAUDE.md](CLAUDE.md); high-level project intro in [README.md](README.md).
|
[CLAUDE.md](CLAUDE.md); high-level project intro in [README.md](README.md).
|
||||||
|
|
||||||
## Permissions / policy
|
|
||||||
|
|
||||||
- **Per-agent send allow-list.** Today any agent can `send` to any
|
|
||||||
other recipient (peer, manager, operator). Add a per-agent
|
|
||||||
policy that constrains the `to` field — declared in `agent.nix`,
|
|
||||||
e.g. `hyperhive.allowedRecipients = [ "manager" "alice" ]`.
|
|
||||||
Broker rejects with an `Err { message }` when the policy denies.
|
|
||||||
Default: unrestricted (back-compat). The manager can still
|
|
||||||
always send anywhere. Useful for sandboxing untrusted sub-agents
|
|
||||||
so they can only talk to the manager, not other sub-agents.
|
|
||||||
|
|
||||||
## Security
|
## Security
|
||||||
|
|
||||||
- **Unprivileged containers (userns mapping).** Today the nspawn container
|
- **Unprivileged containers (userns mapping).** Today the nspawn container
|
||||||
|
|
@ -42,8 +31,16 @@ Pick anything from here when relevant. Cross-cutting design notes live in
|
||||||
derived from the same config so the operator stays in control of
|
derived from the same config so the operator stays in control of
|
||||||
what's exposed.
|
what's exposed.
|
||||||
|
|
||||||
## UI / UX
|
## Per-agent settings
|
||||||
|
|
||||||
|
- **Model override persistence.** `/model <name>` already switches
|
||||||
|
the model at runtime via `Bus::set_model`; the chip on the agent
|
||||||
|
page reflects the current value. Override is in-memory only and
|
||||||
|
resets on harness restart — by design for now, but consider
|
||||||
|
optional persistence (`/state/model` file?) so an operator-set
|
||||||
|
model survives a rebuild.
|
||||||
|
|
||||||
|
## UI / UX
|
||||||
|
|
||||||
- **Terminal: `/model` slash command.** Operator-typeable model
|
- **Terminal: `/model` slash command.** Operator-typeable model
|
||||||
override from the terminal. Depends on the model-override work
|
override from the terminal. Depends on the model-override work
|
||||||
|
|
@ -102,3 +99,9 @@ Pick anything from here when relevant. Cross-cutting design notes live in
|
||||||
that takes the existing notes + a "compact this" prompt and rewrites
|
that takes the existing notes + a "compact this" prompt and rewrites
|
||||||
them in place. Add when the notes start bloating.
|
them in place. Add when the notes start bloating.
|
||||||
|
|
||||||
|
## Lifecycle / reliability
|
||||||
|
|
||||||
|
- **Container crash events.** Watch `container@*.service` via D-Bus, push
|
||||||
|
`HelperEvent::ContainerCrash` to the manager's inbox so the manager can
|
||||||
|
react (restart, escalate, etc.).
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -115,10 +115,6 @@ regular claude turn so the manager can react. Variants
|
||||||
- `Killed { agent }` — admin `HostRequest::Kill` + dashboard
|
- `Killed { agent }` — admin `HostRequest::Kill` + dashboard
|
||||||
`/kill` + manager `Kill` MCP tool.
|
`/kill` + manager `Kill` MCP tool.
|
||||||
- `Destroyed { agent }` — `actions::destroy`.
|
- `Destroyed { agent }` — `actions::destroy`.
|
||||||
- `ContainerCrash { agent, note }` — `crash_watch`: a previously-
|
|
||||||
running container went away with no operator-initiated transient
|
|
||||||
state (Stopping / Restarting / Destroying / Rebuilding). Manager
|
|
||||||
can `start` it again or escalate.
|
|
||||||
- `OperatorAnswered { id, question, answer }` — dashboard
|
- `OperatorAnswered { id, question, answer }` — dashboard
|
||||||
`/answer-question/{id}` after the operator submits the answer
|
`/answer-question/{id}` after the operator submits the answer
|
||||||
form.
|
form.
|
||||||
|
|
|
||||||
|
|
@ -53,15 +53,11 @@ socket without needing a clean reinstall.
|
||||||
|
|
||||||
## `claude-code` is unfree
|
## `claude-code` is unfree
|
||||||
|
|
||||||
The flake pins it to **nixpkgs-unstable** via
|
`harness-base.nix` allow-list's it specifically. The flake pins it to
|
||||||
`overlays.claude-unstable` (stable lags too far). The overlay
|
**nixpkgs-unstable** via `overlays.claude-unstable` (stable lags too
|
||||||
imports unstable inheriting the user's `nixpkgs.config`, so the
|
far). The overlay imports unstable with its own
|
||||||
operator must opt in by setting `allowUnfree = true` (or an
|
`allowUnfreePredicate` so the access inside the overlay doesn't
|
||||||
`allowUnfreePredicate` that whitelists `claude-code`) on their host
|
itself trip.
|
||||||
config. hyperhive deliberately does NOT auto-allow — silent unfree
|
|
||||||
bypass would be sketchy, and the error message is clear enough that
|
|
||||||
the operator can fix it once and forget about it. Same on the
|
|
||||||
per-agent containers (they inherit through the same nixpkgs).
|
|
||||||
|
|
||||||
## Claude credentials are per-agent
|
## Claude credentials are per-agent
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -67,14 +67,9 @@
|
||||||
claude-unstable =
|
claude-unstable =
|
||||||
final: prev:
|
final: prev:
|
||||||
let
|
let
|
||||||
# Inherit the *user's* nixpkgs config so allowUnfree (or an
|
|
||||||
# `allowUnfreePredicate` they set on their flake) propagates
|
|
||||||
# into the unstable import. hyperhive does not silently
|
|
||||||
# bypass the unfree gate — if the operator hasn't opted in,
|
|
||||||
# this overlay's `claude-code` access fails honestly.
|
|
||||||
unstable = import nixpkgs-unstable {
|
unstable = import nixpkgs-unstable {
|
||||||
inherit (prev.stdenv.hostPlatform) system;
|
inherit (prev.stdenv.hostPlatform) system;
|
||||||
config = prev.config;
|
config.allowUnfreePredicate = pkg: builtins.elem (prev.lib.getName pkg) [ "claude-code" ];
|
||||||
};
|
};
|
||||||
in
|
in
|
||||||
{
|
{
|
||||||
|
|
|
||||||
|
|
@ -26,7 +26,7 @@ You're the policy gate between sub-agents and the operator's approval queue —
|
||||||
|
|
||||||
Two ways to talk to the operator: `send(to: "operator", ...)` for fire-and-forget status / pointers (surfaces in the operator inbox), or `ask_operator(question, options?)` when you need a decision. `ask_operator` is non-blocking — it queues the question and returns an id immediately; the answer arrives on a future turn as an `operator_answered` system event. Prefer `ask_operator` over an open-ended `send` for anything you actually need to wait on.
|
Two ways to talk to the operator: `send(to: "operator", ...)` for fire-and-forget status / pointers (surfaces in the operator inbox), or `ask_operator(question, options?)` when you need a decision. `ask_operator` is non-blocking — it queues the question and returns an id immediately; the answer arrives on a future turn as an `operator_answered` system event. Prefer `ask_operator` over an open-ended `send` for anything you actually need to wait on.
|
||||||
|
|
||||||
Messages from sender `system` are hyperhive helper events (JSON body, `event` field discriminates): `approval_resolved`, `spawned`, `rebuilt`, `killed`, `destroyed`, `container_crash`, `operator_answered`. Use these to react to lifecycle changes — e.g. greet a freshly-spawned agent, retry a failed rebuild, restart an agent whose container crashed, or pick up the operator's answer to a question you previously asked.
|
Messages from sender `system` are hyperhive helper events (JSON body, `event` field discriminates): `approval_resolved`, `spawned`, `rebuilt`, `killed`, `destroyed`, `operator_answered`. Use these to react to lifecycle changes — e.g. greet a freshly-spawned agent, retry a failed rebuild, or pick up the operator's answer to a question you previously asked.
|
||||||
|
|
||||||
Durable knowledge:
|
Durable knowledge:
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -24,36 +24,6 @@ const HISTORY_CAPACITY: usize = 2000;
|
||||||
/// `HYPERHIVE_EVENTS_DB` env var (used in tests and one-shot tools).
|
/// `HYPERHIVE_EVENTS_DB` env var (used in tests and one-shot tools).
|
||||||
const DEFAULT_EVENTS_DB: &str = "/state/hyperhive-events.sqlite";
|
const DEFAULT_EVENTS_DB: &str = "/state/hyperhive-events.sqlite";
|
||||||
|
|
||||||
/// Persisted model name file. Same lifecycle as the events db —
|
|
||||||
/// survives destroy/recreate, gone on purge. Empty / missing file
|
|
||||||
/// falls back to `DEFAULT_MODEL`.
|
|
||||||
const DEFAULT_MODEL_FILE: &str = "/state/hyperhive-model";
|
|
||||||
|
|
||||||
/// Path to the persisted model file. Overridable via
|
|
||||||
/// `HYPERHIVE_MODEL_FILE` for dev / tests.
|
|
||||||
fn model_file_path() -> PathBuf {
|
|
||||||
std::env::var_os("HYPERHIVE_MODEL_FILE")
|
|
||||||
.map_or_else(|| PathBuf::from(DEFAULT_MODEL_FILE), PathBuf::from)
|
|
||||||
}
|
|
||||||
|
|
||||||
fn load_model() -> Option<String> {
|
|
||||||
let s = std::fs::read_to_string(model_file_path()).ok()?;
|
|
||||||
let name = s.trim();
|
|
||||||
if name.is_empty() {
|
|
||||||
None
|
|
||||||
} else {
|
|
||||||
Some(name.to_owned())
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
fn persist_model(name: &str) -> std::io::Result<()> {
|
|
||||||
let path = model_file_path();
|
|
||||||
if let Some(parent) = path.parent() {
|
|
||||||
let _ = std::fs::create_dir_all(parent);
|
|
||||||
}
|
|
||||||
std::fs::write(path, format!("{name}\n"))
|
|
||||||
}
|
|
||||||
|
|
||||||
fn now_unix() -> i64 {
|
fn now_unix() -> i64 {
|
||||||
std::time::SystemTime::now()
|
std::time::SystemTime::now()
|
||||||
.duration_since(std::time::UNIX_EPOCH)
|
.duration_since(std::time::UNIX_EPOCH)
|
||||||
|
|
@ -207,12 +177,11 @@ impl Bus {
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
let (tx, _) = broadcast::channel(CHANNEL_CAPACITY);
|
let (tx, _) = broadcast::channel(CHANNEL_CAPACITY);
|
||||||
let initial_model = load_model().unwrap_or_else(|| DEFAULT_MODEL.to_owned());
|
|
||||||
Self {
|
Self {
|
||||||
tx: Arc::new(tx),
|
tx: Arc::new(tx),
|
||||||
store,
|
store,
|
||||||
state: Arc::new(Mutex::new((TurnState::Idle, now_unix()))),
|
state: Arc::new(Mutex::new((TurnState::Idle, now_unix()))),
|
||||||
model: Arc::new(Mutex::new(initial_model)),
|
model: Arc::new(Mutex::new(DEFAULT_MODEL.to_owned())),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -224,16 +193,9 @@ impl Bus {
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Switch the model for future turns. The current turn (if any)
|
/// Switch the model for future turns. The current turn (if any)
|
||||||
/// keeps the model it was already running. Persisted to
|
/// keeps the model it was already running.
|
||||||
/// `/state/hyperhive-model` so the override survives harness
|
|
||||||
/// restart and container rebuild (gone on `--purge`, matching
|
|
||||||
/// every other piece of agent state).
|
|
||||||
pub fn set_model(&self, name: impl Into<String>) {
|
pub fn set_model(&self, name: impl Into<String>) {
|
||||||
let value: String = name.into();
|
*self.model.lock().unwrap() = name.into();
|
||||||
self.model.lock().unwrap().clone_from(&value);
|
|
||||||
if let Err(e) = persist_model(&value) {
|
|
||||||
tracing::warn!(error = ?e, "model: persist failed");
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Update the harness's authoritative turn-loop state. Records
|
/// Update the harness's authoritative turn-loop state. Records
|
||||||
|
|
|
||||||
|
|
@ -34,113 +34,6 @@
|
||||||
return f;
|
return f;
|
||||||
};
|
};
|
||||||
|
|
||||||
// ─── browser notifications ──────────────────────────────────────────────
|
|
||||||
// Fires OS notifications on three operator-bound signals:
|
|
||||||
// - new approval landed in the queue
|
|
||||||
// - new operator question queued (ask_operator)
|
|
||||||
// - broker message sent `to: "operator"`
|
|
||||||
// permission grant is per-browser; a localStorage "muted" toggle lets
|
|
||||||
// the operator silence without revoking. Secure-context only (HTTPS /
|
|
||||||
// localhost) — on other origins the API is unavailable and we hide
|
|
||||||
// the controls.
|
|
||||||
const NOTIF = (() => {
|
|
||||||
const supported = typeof Notification !== 'undefined';
|
|
||||||
const MUTED_KEY = 'hyperhive.notify.muted';
|
|
||||||
const isMuted = () => localStorage.getItem(MUTED_KEY) === '1';
|
|
||||||
const setMuted = (v) => v
|
|
||||||
? localStorage.setItem(MUTED_KEY, '1')
|
|
||||||
: localStorage.removeItem(MUTED_KEY);
|
|
||||||
function renderControls() {
|
|
||||||
const enable = $('notif-enable');
|
|
||||||
const mute = $('notif-mute');
|
|
||||||
const unmute = $('notif-unmute');
|
|
||||||
const status = $('notif-status');
|
|
||||||
if (!enable || !mute || !unmute || !status) return;
|
|
||||||
if (!supported) {
|
|
||||||
enable.hidden = mute.hidden = unmute.hidden = true;
|
|
||||||
status.hidden = false;
|
|
||||||
status.textContent = 'notifications unsupported in this browser';
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
const perm = Notification.permission;
|
|
||||||
enable.hidden = perm === 'granted';
|
|
||||||
mute.hidden = perm !== 'granted' || isMuted();
|
|
||||||
unmute.hidden = perm !== 'granted' || !isMuted();
|
|
||||||
status.hidden = perm !== 'denied';
|
|
||||||
if (perm === 'denied') status.textContent = 'notifications blocked — grant in site settings';
|
|
||||||
}
|
|
||||||
function bind() {
|
|
||||||
const enable = $('notif-enable');
|
|
||||||
const mute = $('notif-mute');
|
|
||||||
const unmute = $('notif-unmute');
|
|
||||||
if (!supported || !enable || !mute || !unmute) return;
|
|
||||||
enable.addEventListener('click', async () => {
|
|
||||||
await Notification.requestPermission();
|
|
||||||
renderControls();
|
|
||||||
});
|
|
||||||
mute.addEventListener('click', () => { setMuted(true); renderControls(); });
|
|
||||||
unmute.addEventListener('click', () => { setMuted(false); renderControls(); });
|
|
||||||
renderControls();
|
|
||||||
}
|
|
||||||
function show(title, body) {
|
|
||||||
if (!supported || Notification.permission !== 'granted' || isMuted()) return;
|
|
||||||
try {
|
|
||||||
const n = new Notification(title, {
|
|
||||||
body,
|
|
||||||
tag: 'hyperhive', // collapse rapid bursts
|
|
||||||
icon: '/static/dashboard.css', // any same-origin asset works as a favicon stand-in
|
|
||||||
});
|
|
||||||
n.onclick = () => { window.focus(); n.close(); };
|
|
||||||
} catch (err) {
|
|
||||||
console.warn('notification show failed', err);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return { bind, show, renderControls };
|
|
||||||
})();
|
|
||||||
|
|
||||||
// Track which items we've already notified about so a re-render
|
|
||||||
// doesn't re-fire for the same row. Keyed by stable ids; reset only
|
|
||||||
// when the page reloads.
|
|
||||||
const seenApprovals = new Set();
|
|
||||||
const seenQuestions = new Set();
|
|
||||||
const seenInboxIds = new Set();
|
|
||||||
let seededNotify = false;
|
|
||||||
|
|
||||||
function notifyDeltas(s) {
|
|
||||||
const approvals = s.approvals || [];
|
|
||||||
const questions = s.questions || [];
|
|
||||||
const inbox = s.operator_inbox || [];
|
|
||||||
if (!seededNotify) {
|
|
||||||
// First render after page load — fill the "seen" sets without
|
|
||||||
// firing notifications. We only want to notify on NEW items
|
|
||||||
// that arrived while the page is open.
|
|
||||||
for (const a of approvals) seenApprovals.add(a.id);
|
|
||||||
for (const q of questions) seenQuestions.add(q.id);
|
|
||||||
for (const m of inbox) seenInboxIds.add(m.id);
|
|
||||||
seededNotify = true;
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
for (const a of approvals) {
|
|
||||||
if (seenApprovals.has(a.id)) continue;
|
|
||||||
seenApprovals.add(a.id);
|
|
||||||
const verb = a.kind === 'spawn' ? 'spawn approval' : 'config commit';
|
|
||||||
NOTIF.show('◆ approval #' + a.id, `${verb} for ${a.agent}`);
|
|
||||||
}
|
|
||||||
for (const q of questions) {
|
|
||||||
if (seenQuestions.has(q.id)) continue;
|
|
||||||
seenQuestions.add(q.id);
|
|
||||||
NOTIF.show('◆ manager asks', q.question.slice(0, 120));
|
|
||||||
}
|
|
||||||
// operator_inbox: only notify on truly new ids — sse already
|
|
||||||
// handles single-message notifications, but if the operator
|
|
||||||
// missed an SSE event (page reloaded), this catches up.
|
|
||||||
for (const m of inbox) {
|
|
||||||
if (seenInboxIds.has(m.id)) continue;
|
|
||||||
seenInboxIds.add(m.id);
|
|
||||||
// suppress here; SSE path handles the live notification.
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// ─── async forms ────────────────────────────────────────────────────────
|
// ─── async forms ────────────────────────────────────────────────────────
|
||||||
document.addEventListener('submit', async (e) => {
|
document.addEventListener('submit', async (e) => {
|
||||||
const f = e.target;
|
const f = e.target;
|
||||||
|
|
@ -584,7 +477,6 @@
|
||||||
renderQuestions(s);
|
renderQuestions(s);
|
||||||
renderInbox(s);
|
renderInbox(s);
|
||||||
renderApprovals(s);
|
renderApprovals(s);
|
||||||
notifyDeltas(s);
|
|
||||||
// Auto-refresh: fast (2s) while a spawn or a per-container
|
// Auto-refresh: fast (2s) while a spawn or a per-container
|
||||||
// action is in flight, otherwise heartbeat (5s) so newly-queued
|
// action is in flight, otherwise heartbeat (5s) so newly-queued
|
||||||
// approvals from the manager show up without the operator
|
// approvals from the manager show up without the operator
|
||||||
|
|
@ -601,7 +493,6 @@
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
refreshState();
|
refreshState();
|
||||||
NOTIF.bind();
|
|
||||||
|
|
||||||
// ─── message flow SSE ───────────────────────────────────────────────────
|
// ─── message flow SSE ───────────────────────────────────────────────────
|
||||||
(() => {
|
(() => {
|
||||||
|
|
@ -626,12 +517,8 @@
|
||||||
let m;
|
let m;
|
||||||
try { m = JSON.parse(e.data); } catch { return; }
|
try { m = JSON.parse(e.data); } catch { return; }
|
||||||
pulseBanner();
|
pulseBanner();
|
||||||
// Live-update the inbox when claude sends to operator + ping
|
// Live-update the inbox when claude sends to operator.
|
||||||
// the OS notification center.
|
if (m.kind === 'sent' && m.to === 'operator') refreshState();
|
||||||
if (m.kind === 'sent' && m.to === 'operator') {
|
|
||||||
refreshState();
|
|
||||||
NOTIF.show('◆ ' + m.from + ' → operator', String(m.body || '').slice(0, 200));
|
|
||||||
}
|
|
||||||
const row = document.createElement('div');
|
const row = document.createElement('div');
|
||||||
row.className = 'msgrow ' + m.kind;
|
row.className = 'msgrow ' + m.kind;
|
||||||
const kind = m.kind === 'sent' ? '→' : '✓';
|
const kind = m.kind === 'sent' ? '→' : '✓';
|
||||||
|
|
|
||||||
|
|
@ -190,32 +190,6 @@ a:hover {
|
||||||
word-break: normal;
|
word-break: normal;
|
||||||
}
|
}
|
||||||
|
|
||||||
/* Notification controls — sit between the banner and the
|
|
||||||
containers section. Hidden by JS when notifications are
|
|
||||||
unsupported, denied, or already in the right state. */
|
|
||||||
.notif-row {
|
|
||||||
display: flex;
|
|
||||||
gap: 0.5em;
|
|
||||||
align-items: center;
|
|
||||||
margin: 0.5em 0;
|
|
||||||
font-size: 0.85em;
|
|
||||||
}
|
|
||||||
.btn-notif {
|
|
||||||
font-family: inherit;
|
|
||||||
font-size: 0.85em;
|
|
||||||
background: transparent;
|
|
||||||
color: var(--cyan);
|
|
||||||
border: 1px solid var(--cyan);
|
|
||||||
padding: 0.2em 0.7em;
|
|
||||||
border-radius: 999px;
|
|
||||||
cursor: pointer;
|
|
||||||
text-shadow: 0 0 4px currentColor;
|
|
||||||
}
|
|
||||||
.btn-notif:hover {
|
|
||||||
background: rgba(137, 220, 235, 0.1);
|
|
||||||
box-shadow: 0 0 10px -2px currentColor;
|
|
||||||
}
|
|
||||||
|
|
||||||
.pending-state {
|
.pending-state {
|
||||||
color: var(--amber);
|
color: var(--amber);
|
||||||
font-size: 0.85em;
|
font-size: 0.85em;
|
||||||
|
|
|
||||||
|
|
@ -10,13 +10,6 @@
|
||||||
░▒▓█▓▒░ HYPERHIVE ░▒▓█▓▒░ HIVE-C0RE ░▒▓█▓▒░ WE ARE THE WIRED ░▒▓█▓▒░
|
░▒▓█▓▒░ HYPERHIVE ░▒▓█▓▒░ HIVE-C0RE ░▒▓█▓▒░ WE ARE THE WIRED ░▒▓█▓▒░
|
||||||
</pre>
|
</pre>
|
||||||
|
|
||||||
<div id="notif-row" class="notif-row">
|
|
||||||
<button type="button" id="notif-enable" class="btn btn-notif" hidden>🔔 enable notifications</button>
|
|
||||||
<button type="button" id="notif-mute" class="btn btn-notif" hidden>🔕 mute</button>
|
|
||||||
<button type="button" id="notif-unmute" class="btn btn-notif" hidden>🔔 unmute</button>
|
|
||||||
<span id="notif-status" class="meta" hidden></span>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<h2>◆ C0NTAINERS ◆</h2>
|
<h2>◆ C0NTAINERS ◆</h2>
|
||||||
<div class="divider">══════════════════════════════════════════════════════════════</div>
|
<div class="divider">══════════════════════════════════════════════════════════════</div>
|
||||||
<div id="containers-section">
|
<div id="containers-section">
|
||||||
|
|
|
||||||
|
|
@ -1,72 +0,0 @@
|
||||||
//! Container crash watcher. Polls every managed container's running
|
|
||||||
//! state on a fixed interval; when a previously-running container is
|
|
||||||
//! suddenly stopped AND no operator-initiated transient (`Stopping`,
|
|
||||||
//! `Restarting`, `Destroying`) was set, fire `HelperEvent::ContainerCrash`
|
|
||||||
//! into the manager's inbox. The manager can then react — usually
|
|
||||||
//! a `start` or a config rebuild.
|
|
||||||
//!
|
|
||||||
//! D-Bus subscription would be lower-latency, but polling is far
|
|
||||||
//! simpler and the failure modes are honest (a crash discovered 10s
|
|
||||||
//! late is fine for our scale).
|
|
||||||
|
|
||||||
use std::collections::HashSet;
|
|
||||||
use std::sync::Arc;
|
|
||||||
use std::time::Duration;
|
|
||||||
|
|
||||||
use crate::coordinator::{Coordinator, TransientKind};
|
|
||||||
use crate::lifecycle::{self, AGENT_PREFIX, MANAGER_NAME};
|
|
||||||
|
|
||||||
const POLL_INTERVAL: Duration = Duration::from_secs(10);
|
|
||||||
|
|
||||||
pub fn spawn(coord: Arc<Coordinator>) {
|
|
||||||
tokio::spawn(async move {
|
|
||||||
// Seed the running-set from the first poll so we don't emit a
|
|
||||||
// crash for every agent on startup. First tick fills it; only
|
|
||||||
// running→stopped transitions across subsequent ticks count.
|
|
||||||
let mut prev_running: HashSet<String> = HashSet::new();
|
|
||||||
let mut seeded = false;
|
|
||||||
loop {
|
|
||||||
let raw = lifecycle::list().await.unwrap_or_default();
|
|
||||||
let mut current_running = HashSet::new();
|
|
||||||
for c in &raw {
|
|
||||||
let logical = if c == MANAGER_NAME {
|
|
||||||
MANAGER_NAME.to_owned()
|
|
||||||
} else if let Some(n) = c.strip_prefix(AGENT_PREFIX) {
|
|
||||||
n.to_owned()
|
|
||||||
} else {
|
|
||||||
continue;
|
|
||||||
};
|
|
||||||
if lifecycle::is_running(&logical).await {
|
|
||||||
current_running.insert(logical);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if seeded {
|
|
||||||
let transients = coord.transient_snapshot();
|
|
||||||
for stopped in prev_running.difference(¤t_running) {
|
|
||||||
let deliberate = transients.get(stopped).is_some_and(|st| {
|
|
||||||
matches!(
|
|
||||||
st.kind,
|
|
||||||
TransientKind::Stopping
|
|
||||||
| TransientKind::Restarting
|
|
||||||
| TransientKind::Destroying
|
|
||||||
| TransientKind::Rebuilding
|
|
||||||
)
|
|
||||||
});
|
|
||||||
if deliberate {
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
tracing::warn!(agent = %stopped, "container crash detected");
|
|
||||||
coord.notify_manager(&hive_sh4re::HelperEvent::ContainerCrash {
|
|
||||||
agent: stopped.clone(),
|
|
||||||
note: Some("container stopped without an operator action".into()),
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
prev_running = current_running;
|
|
||||||
seeded = true;
|
|
||||||
|
|
||||||
tokio::time::sleep(POLL_INTERVAL).await;
|
|
||||||
}
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
@ -12,7 +12,6 @@ mod auto_update;
|
||||||
mod broker;
|
mod broker;
|
||||||
mod client;
|
mod client;
|
||||||
mod coordinator;
|
mod coordinator;
|
||||||
mod crash_watch;
|
|
||||||
mod dashboard;
|
mod dashboard;
|
||||||
mod events_vacuum;
|
mod events_vacuum;
|
||||||
mod lifecycle;
|
mod lifecycle;
|
||||||
|
|
@ -131,10 +130,6 @@ async fn main() -> Result<()> {
|
||||||
// Per-agent events.sqlite vacuum: host-side so the harness
|
// Per-agent events.sqlite vacuum: host-side so the harness
|
||||||
// doesn't need any retention wiring of its own.
|
// doesn't need any retention wiring of its own.
|
||||||
events_vacuum::spawn(coord.clone());
|
events_vacuum::spawn(coord.clone());
|
||||||
// Container crash watcher: emits HelperEvent::ContainerCrash
|
|
||||||
// when a previously-running container goes away without an
|
|
||||||
// operator-initiated transient state.
|
|
||||||
crash_watch::spawn(coord.clone());
|
|
||||||
let dash_coord = coord.clone();
|
let dash_coord = coord.clone();
|
||||||
tokio::spawn(async move {
|
tokio::spawn(async move {
|
||||||
if let Err(e) = dashboard::serve(dashboard_port, dash_coord).await {
|
if let Err(e) = dashboard::serve(dashboard_port, dash_coord).await {
|
||||||
|
|
|
||||||
|
|
@ -259,16 +259,6 @@ pub enum HelperEvent {
|
||||||
/// A sub-agent's container was torn down (container removed; state
|
/// A sub-agent's container was torn down (container removed; state
|
||||||
/// dirs preserved per `destroy` semantics).
|
/// dirs preserved per `destroy` semantics).
|
||||||
Destroyed { agent: String },
|
Destroyed { agent: String },
|
||||||
/// Container exited without an operator-initiated stop. Fired by
|
|
||||||
/// the crash watcher when an agent's container transitions from
|
|
||||||
/// running → stopped and no `Stopping` / `Restarting` /
|
|
||||||
/// `Destroying` transient was set, so the operator (or the
|
|
||||||
/// manager) knows it crashed rather than was killed on purpose.
|
|
||||||
ContainerCrash {
|
|
||||||
agent: String,
|
|
||||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
|
||||||
note: Option<String>,
|
|
||||||
},
|
|
||||||
/// The operator answered a question that was queued via
|
/// The operator answered a question that was queued via
|
||||||
/// `AskOperator`. `id` matches the `QuestionQueued.id` returned to the
|
/// `AskOperator`. `id` matches the `QuestionQueued.id` returned to the
|
||||||
/// asker; `question` echoes the original prompt so the manager can
|
/// asker; `question` echoes the original prompt so the manager can
|
||||||
|
|
|
||||||
|
|
@ -7,12 +7,7 @@
|
||||||
|
|
||||||
boot.isNspawnContainer = true;
|
boot.isNspawnContainer = true;
|
||||||
|
|
||||||
# `claude-code` is unfree. hyperhive intentionally does NOT auto-allow
|
nixpkgs.config.allowUnfreePredicate = pkg: builtins.elem (pkgs.lib.getName pkg) [ "claude-code" ];
|
||||||
# it — the operator opts in by setting
|
|
||||||
# `nixpkgs.config.allowUnfreePredicate` (or `allowUnfree = true`) in
|
|
||||||
# their own host config / agent.nix. Without that, the per-agent
|
|
||||||
# build fails on this package and the operator sees an honest "this
|
|
||||||
# is unfree, are you sure?" error.
|
|
||||||
|
|
||||||
environment.systemPackages = with pkgs; [
|
environment.systemPackages = with pkgs; [
|
||||||
hyperhive
|
hyperhive
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue