feat: group host-side /var/lib/hyperhive state into db/ forge/ matrix/ run/ subdirs with startup migration

This commit is contained in:
damocles 2026-06-05 22:48:10 +02:00 committed by mara
commit 14c7b0d406
15 changed files with 261 additions and 51 deletions

View file

@ -117,7 +117,7 @@ are retried automatically on subsequent poll ticks via
## Agent port map (`agent-ports.json`) ## Agent port map (`agent-ports.json`)
`/var/lib/hyperhive/agent-ports.json` is a flat JSON object keyed by `/var/lib/hyperhive/run/agent-ports.json` is a flat JSON object keyed by
logical agent name → TCP web port: logical agent name → TCP web port:
```json ```json

View file

@ -132,7 +132,7 @@ Matrix client that groups all agent-to-agent + operator rooms in one
place. place.
**State**: the room ID is persisted to **State**: the room ID is persisted to
`/var/lib/hyperhive/matrix-space-room-id` (mode `0600`, root-owned). `/var/lib/hyperhive/matrix/space-room-id` (mode `0600`, root-owned).
This path is **outside** every agent state dir and is NOT deleted by This path is **outside** every agent state dir and is NOT deleted by
`nixos-container destroy --purge` — the Space survives full agent `nixos-container destroy --purge` — the Space survives full agent
purges and is reused on re-provision. purges and is reused on re-provision.

View file

@ -4,7 +4,7 @@ Where state lives, what survives what, and how it's bounded.
## Three sqlite databases ## Three sqlite databases
### `/var/lib/hyperhive/broker.sqlite` (host) ### `/var/lib/hyperhive/db/broker.sqlite` (host)
Six tables, all in one file — four queues plus the schedule Six tables, all in one file — four queues plus the schedule
header/targets split: header/targets split:
@ -127,7 +127,7 @@ sentinel files (`hyperhive-rate-limited`, `hyperhive-needs-login`) if the
JSON is absent, so existing containers keep working through the transition JSON is absent, so existing containers keep working through the transition
window before their next rebuild. window before their next rebuild.
### `/var/lib/hyperhive/build_logs.sqlite` (host) ### `/var/lib/hyperhive/db/build_logs.sqlite` (host)
Full stdout + stderr capture for every `nixos-container` / `nix Full stdout + stderr capture for every `nixos-container` / `nix
build` invocation the lifecycle layer fires. One row per invocation; build` invocation the lifecycle layer fires. One row per invocation;

View file

@ -83,8 +83,8 @@ matrix-tuwunel user provisioning. Manual entry point to the same idempotent flow
* `create-user` — Create or refresh the matrix account + access token for `<name>` * `create-user` — Create or refresh the matrix account + access token for `<name>`
* `sync-admin` — Provision (or re-provision) the hive system admin matrix account (`@hive:<server>`). hive-c0re runs this automatically on startup before the agent sweep so the account is the first registered user — Conduit/tuwunel grants admin rights to the first user. Run manually to recover a missing admin token file * `sync-admin` — Provision (or re-provision) the hive system admin matrix account (`@hive:<server>`). hive-c0re runs this automatically on startup before the agent sweep so the account is the first registered user — Conduit/tuwunel grants admin rights to the first user. Run manually to recover a missing admin token file
* `promote-user` — Promote a matrix user to homeserver admin via the admin API. Uses the hive system admin token at `/var/lib/hyperhive/matrix-admin-token`. The `server_name` is discovered automatically from the running homeserver * `promote-user` — Promote a matrix user to homeserver admin via the admin API. Uses the hive system admin token at `/var/lib/hyperhive/matrix/admin-token`. The `server_name` is discovered automatically from the running homeserver
* `reset-password` — Reset a matrix user's password via the admin API and persist the new password to `/var/lib/hyperhive/matrix-creds/<name>-password` so the next `ensure_user_for` (or `create-user`) can re-login * `reset-password` — Reset a matrix user's password via the admin API and persist the new password to `/var/lib/hyperhive/matrix/creds/<name>-password` so the next `ensure_user_for` (or `create-user`) can re-login
* `invite` — Invite a matrix user to the hive Space (default) or a specific room. Uses the hive admin token; the admin account must be a member of the target room with invite power (it owns the hive Space). Idempotent — already-member / already-invited is a no-op * `invite` — Invite a matrix user to the hive Space (default) or a specific room. Uses the hive admin token; the admin account must be a member of the target room with invite power (it owns the hive Space). Idempotent — already-member / already-invited is a no-op
@ -122,7 +122,7 @@ Provision (or re-provision) the hive system admin matrix account (`@hive:<server
## `hivectl matrix promote-user` ## `hivectl matrix promote-user`
Promote a matrix user to homeserver admin via the admin API. Uses the hive system admin token at `/var/lib/hyperhive/matrix-admin-token`. The `server_name` is discovered automatically from the running homeserver Promote a matrix user to homeserver admin via the admin API. Uses the hive system admin token at `/var/lib/hyperhive/matrix/admin-token`. The `server_name` is discovered automatically from the running homeserver
**Usage:** `hivectl matrix promote-user <NAME>` **Usage:** `hivectl matrix promote-user <NAME>`
@ -134,7 +134,7 @@ Promote a matrix user to homeserver admin via the admin API. Uses the hive syste
## `hivectl matrix reset-password` ## `hivectl matrix reset-password`
Reset a matrix user's password via the admin API and persist the new password to `/var/lib/hyperhive/matrix-creds/<name>-password` so the next `ensure_user_for` (or `create-user`) can re-login. Reset a matrix user's password via the admin API and persist the new password to `/var/lib/hyperhive/matrix/creds/<name>-password` so the next `ensure_user_for` (or `create-user`) can re-login.
After this command succeeds, run `hivectl matrix create-user <name>` to mint a fresh access token for the agent. After this command succeeds, run `hivectl matrix create-user <name>` to mint a fresh access token for the agent.

View file

@ -1,4 +1,4 @@
//! `/var/lib/hyperhive/agent-ports.json` writer — flat map of //! `/var/lib/hyperhive/run/agent-ports.json` writer — flat map of
//! agent name → TCP web port. Written alongside `agents.conf` on //! agent name → TCP web port. Written alongside `agents.conf` on
//! every topology change. JSON shape, port derivation (FNV-1a hash), //! every topology change. JSON shape, port derivation (FNV-1a hash),
//! atomicity, and manager exclusion: `docs/gateway.md::Agent port map`. //! atomicity, and manager exclusion: `docs/gateway.md::Agent port map`.
@ -10,11 +10,9 @@ use anyhow::{Context, Result};
use crate::lifecycle::{self, MANAGER_NAME}; use crate::lifecycle::{self, MANAGER_NAME};
const HOST_PORTS_PATH: &str = "/var/lib/hyperhive/agent-ports.json";
#[must_use] #[must_use]
pub fn host_ports_path() -> PathBuf { pub fn host_ports_path() -> PathBuf {
PathBuf::from(HOST_PORTS_PATH) crate::paths::agent_ports_file()
} }
/// Compute the agent-port map for the given logical agent names. /// Compute the agent-port map for the given logical agent names.
@ -34,7 +32,7 @@ pub fn build_map(names: &[String]) -> BTreeMap<String, u16> {
} }
/// Render the map as pretty-printed JSON. Pretty so a human peek at /// Render the map as pretty-printed JSON. Pretty so a human peek at
/// `cat /var/lib/hyperhive/agent-ports.json` shows one row per agent /// `cat /var/lib/hyperhive/run/agent-ports.json` shows one row per agent
/// — keeps the file readable without a separate jq step. /// — keeps the file readable without a separate jq step.
fn render(map: &BTreeMap<String, u16>) -> String { fn render(map: &BTreeMap<String, u16>) -> String {
// BTreeMap → serde_json::to_string_pretty preserves key order, // BTreeMap → serde_json::to_string_pretty preserves key order,
@ -44,7 +42,7 @@ fn render(map: &BTreeMap<String, u16>) -> String {
} }
/// Atomically write the JSON for `names` to /// Atomically write the JSON for `names` to
/// `/var/lib/hyperhive/agent-ports.json`. Writes via a sibling /// `/var/lib/hyperhive/run/agent-ports.json`. Writes via a sibling
/// `<path>.tmp` + rename so a crashing process never leaves a /// `<path>.tmp` + rename so a crashing process never leaves a
/// partial file behind that the gateway worker would fail to parse. /// partial file behind that the gateway worker would fail to parse.
/// ///

View file

@ -1,4 +1,4 @@
//! `/var/lib/hyperhive/agent-sockets.json` writer. Sibling to //! `/var/lib/hyperhive/run/agent-sockets.json` writer. Sibling to
//! `agent_ports.rs`; same atomic `<path>.tmp` + `rename()` shape so //! `agent_ports.rs`; same atomic `<path>.tmp` + `rename()` shape so
//! the gateway's nginx worker never reads a partial file. Includes //! the gateway's nginx worker never reads a partial file. Includes
//! manager and sub-agents so the gateway can route //! manager and sub-agents so the gateway can route
@ -13,8 +13,6 @@ use std::path::{Path, PathBuf};
use anyhow::{Context, Result}; use anyhow::{Context, Result};
const HOST_SOCKETS_PATH: &str = "/var/lib/hyperhive/agent-sockets.json";
/// Host-side parent directory holding per-agent socket subdirs. The /// Host-side parent directory holding per-agent socket subdirs. The
/// gateway container bind-mounts this whole tree (read-only) so it /// gateway container bind-mounts this whole tree (read-only) so it
/// can `proxy_pass` to any agent. Each agent's container bind-mounts /// can `proxy_pass` to any agent. Each agent's container bind-mounts
@ -43,7 +41,7 @@ const READY_MARKER_LEGACY: &str = ".bound";
#[must_use] #[must_use]
pub fn host_sockets_path() -> PathBuf { pub fn host_sockets_path() -> PathBuf {
PathBuf::from(HOST_SOCKETS_PATH) crate::paths::agent_sockets_file()
} }
/// Per-agent socket subdir on the host. Lifecycle pre-creates this /// Per-agent socket subdir on the host. Lifecycle pre-creates this
@ -111,7 +109,7 @@ pub fn ready_marker_for(name: &str) -> PathBuf {
} }
/// Render the map as pretty-printed JSON. Pretty so a human peek at /// Render the map as pretty-printed JSON. Pretty so a human peek at
/// `cat /var/lib/hyperhive/agent-sockets.json` shows one row per agent /// `cat /var/lib/hyperhive/run/agent-sockets.json` shows one row per agent
/// — keeps the file readable without a separate jq step (mirrors /// — keeps the file readable without a separate jq step (mirrors
/// `agent_ports::render`). /// `agent_ports::render`).
fn render(map: &BTreeMap<String, PathBuf>) -> String { fn render(map: &BTreeMap<String, PathBuf>) -> String {
@ -128,7 +126,7 @@ fn render(map: &BTreeMap<String, PathBuf>) -> String {
} }
/// Atomically write the JSON for `names` to /// Atomically write the JSON for `names` to
/// `/var/lib/hyperhive/agent-sockets.json`. Writes via a sibling /// `/var/lib/hyperhive/run/agent-sockets.json`. Writes via a sibling
/// `<path>.tmp` + rename so a crashing process never leaves a /// `<path>.tmp` + rename so a crashing process never leaves a
/// partial file behind that the gateway worker would fail to parse. /// partial file behind that the gateway worker would fail to parse.
/// ///

View file

@ -198,14 +198,14 @@ enum MatrixCmd {
SyncAdmin, SyncAdmin,
/// Promote a matrix user to homeserver admin via the admin API. /// Promote a matrix user to homeserver admin via the admin API.
/// Uses the hive system admin token at /// Uses the hive system admin token at
/// `/var/lib/hyperhive/matrix-admin-token`. The `server_name` is /// `/var/lib/hyperhive/matrix/admin-token`. The `server_name` is
/// discovered automatically from the running homeserver. /// discovered automatically from the running homeserver.
PromoteUser { PromoteUser {
/// Matrix localpart of the user to promote (e.g. `argus`). /// Matrix localpart of the user to promote (e.g. `argus`).
name: String, name: String,
}, },
/// Reset a matrix user's password via the admin API and persist the /// Reset a matrix user's password via the admin API and persist the
/// new password to `/var/lib/hyperhive/matrix-creds/<name>-password` /// new password to `/var/lib/hyperhive/matrix/creds/<name>-password`
/// so the next `ensure_user_for` (or `create-user`) can re-login. /// so the next `ensure_user_for` (or `create-user`) can re-login.
/// ///
/// After this command succeeds, run `hivectl matrix create-user /// After this command succeeds, run `hivectl matrix create-user
@ -655,7 +655,7 @@ async fn matrix_reset_password(name: &str) -> Result<()> {
.await .await
.with_context(|| format!("matrix reset-password {name}"))?; .with_context(|| format!("matrix reset-password {name}"))?;
// Password is persisted by reset_user_password. // Password is persisted by reset_user_password.
let pw_path = PathBuf::from("/var/lib/hyperhive/matrix-creds").join(format!("{name}-password")); let pw_path = hive_c0re::paths::matrix_creds_dir().join(format!("{name}-password"));
println!("matrix: password for @{name}:{server_name} reset"); println!("matrix: password for @{name}:{server_name} reset");
println!("password persisted at: {}", pw_path.display()); println!("password persisted at: {}", pw_path.display());
println!("next: hivectl matrix create-user {name} # mints a fresh access token"); println!("next: hivectl matrix create-user {name} # mints a fresh access token");

View file

@ -1,7 +1,7 @@
//! Sqlite-backed full build-log capture — stdout + stderr per //! Sqlite-backed full build-log capture — stdout + stderr per
//! `nixos-container` / `nix build` invocation, accumulated live. //! `nixos-container` / `nix build` invocation, accumulated live.
//! Schema, indices, retention, and the rationale for replacing //! Schema, indices, retention, and the rationale for replacing
//! the old ring buffer: `docs/persistence.md::/var/lib/hyperhive/build_logs.sqlite`. //! the old ring buffer: `docs/persistence.md::/var/lib/hyperhive/db/build_logs.sqlite`.
use std::path::Path; use std::path::Path;
use std::sync::{Arc, Mutex, OnceLock}; use std::sync::{Arc, Mutex, OnceLock};

View file

@ -20,20 +20,11 @@ const TOKEN_NAME_PREFIX: &str = "hyperhive";
/// itself to push the meta repo + drive admin API calls (org /// itself to push the meta repo + drive admin API calls (org
/// creation, future webhook setup, etc.). Root-only. /// creation, future webhook setup, etc.). Root-only.
const CORE_TOKEN_PATH: &str = "/var/lib/hyperhive/forge-core-token"; const CORE_TOKEN_PATH: &str = "/var/lib/hyperhive/forge-core-token";
/// Marker that records whether `ensure_core_avatar` has successfully // Forge provisioning markers (`forge/core-avatar-set`,
/// uploaded the hyperhive logo as `core`'s avatar. One-shot: the // `forge/agent-configs-avatar-set`, `forge/email-aligned-<name>`) live
/// upload runs once, the marker is written, subsequent startups skip // in `crate::paths` — one-shot guards: the upload/align runs once, the
/// the call. Delete to force re-upload. // marker is written, subsequent startups skip. Delete one to force its
const CORE_AVATAR_MARKER: &str = "/var/lib/hyperhive/forge-core-avatar-set"; // step to re-run.
/// Sibling marker for the `agent-configs` org avatar. Same one-shot
/// semantics — delete to force the upload to re-run.
const CONFIG_ORG_AVATAR_MARKER: &str = "/var/lib/hyperhive/forge-agent-configs-avatar-set";
/// Per-agent marker written once the account email has been aligned to
/// `{name}@hyperhive`. Skips the `PATCH /api/v1/admin/users/{name}`
/// call on every subsequent `sync_agent` tick — that PATCH was resetting
/// Forgejo's `use_custom_avatar` flag and clobbering the avatar uploaded
/// by `forge-avatar-sync`. Delete to force re-alignment.
const EMAIL_ALIGNED_MARKER_PREFIX: &str = "/var/lib/hyperhive/forge-email-aligned-";
// Avatar PNGs are loaded at runtime from // Avatar PNGs are loaded at runtime from
// `$HIVE_ASSETS_DIR/branding/{hyperhive,agent-configs}.png` via the // `$HIVE_ASSETS_DIR/branding/{hyperhive,agent-configs}.png` via the
// helpers in `hive_sh4re::assets`. The `agent-configs.png` is // helpers in `hive_sh4re::assets`. The `agent-configs.png` is
@ -221,8 +212,8 @@ async fn change_user_password(name: &str, password: &str) -> Result<()> {
/// `login_name` (required by Forgejo's `EditUserOption` validator) and /// `login_name` (required by Forgejo's `EditUserOption` validator) and
/// `source_id = 0` (local auth, the default for users hive-c0re creates). /// `source_id = 0` (local auth, the default for users hive-c0re creates).
async fn ensure_user_email(name: &str) { async fn ensure_user_email(name: &str) {
let marker = format!("{EMAIL_ALIGNED_MARKER_PREFIX}{name}"); let marker = crate::paths::forge_email_aligned_marker(name);
if std::path::Path::new(&marker).exists() { if marker.exists() {
return; return;
} }
let Some(token) = core_token() else { let Some(token) = core_token() else {
@ -236,6 +227,9 @@ async fn ensure_user_email(name: &str) {
let url = format!("{FORGE_HTTP}/api/v1/admin/users/{name}"); let url = format!("{FORGE_HTTP}/api/v1/admin/users/{name}");
match forge_http(reqwest::Method::PATCH, &url, &token, &body).await { match forge_http(reqwest::Method::PATCH, &url, &token, &body).await {
Ok(status) if status.is_success() => { Ok(status) if status.is_success() => {
if let Some(parent) = marker.parent() {
std::fs::create_dir_all(parent).ok();
}
std::fs::write(&marker, "").ok(); std::fs::write(&marker, "").ok();
tracing::info!(%name, %email, "forge: user email aligned"); tracing::info!(%name, %email, "forge: user email aligned");
} }
@ -361,7 +355,7 @@ pub async fn provision_user_token(name: &str, password: Option<&str>) -> Result<
/// — any non-2xx is logged at the caller; the project runs fine /// — any non-2xx is logged at the caller; the project runs fine
/// with the default hash identicon. /// with the default hash identicon.
async fn ensure_core_avatar(token: &str) -> Result<()> { async fn ensure_core_avatar(token: &str) -> Result<()> {
let marker = std::path::Path::new(CORE_AVATAR_MARKER); let marker = crate::paths::forge_core_avatar_marker();
if marker.exists() { if marker.exists() {
return Ok(()); return Ok(());
} }
@ -392,7 +386,7 @@ async fn ensure_core_avatar(token: &str) -> Result<()> {
/// endpoint is `POST /api/v1/orgs/{org}/avatar` with a base64-PNG /// endpoint is `POST /api/v1/orgs/{org}/avatar` with a base64-PNG
/// JSON body — same shape as the admin user endpoint above. /// JSON body — same shape as the admin user endpoint above.
async fn ensure_config_org_avatar(token: &str) -> Result<()> { async fn ensure_config_org_avatar(token: &str) -> Result<()> {
let marker = std::path::Path::new(CONFIG_ORG_AVATAR_MARKER); let marker = crate::paths::forge_config_org_avatar_marker();
if marker.exists() { if marker.exists() {
return Ok(()); return Ok(());
} }

View file

@ -42,6 +42,7 @@ pub mod matrix;
pub mod meta; pub mod meta;
pub mod migrate; pub mod migrate;
pub mod operator_questions; pub mod operator_questions;
pub mod paths;
pub mod priv_client; pub mod priv_client;
pub mod questions; pub mod questions;
pub mod rebuild_queue; pub mod rebuild_queue;

View file

@ -54,7 +54,7 @@ enum Cmd {
#[arg(long, default_value = "")] #[arg(long, default_value = "")]
nixpkgs_unstable_flake: String, nixpkgs_unstable_flake: String,
/// Path to the sqlite message store. /// Path to the sqlite message store.
#[arg(long, default_value = "/var/lib/hyperhive/broker.sqlite")] #[arg(long, default_value = hive_c0re::paths::BROKER_DB)]
db: PathBuf, db: PathBuf,
/// Dashboard HTTP port. /// Dashboard HTTP port.
#[arg(long, default_value_t = 7000)] #[arg(long, default_value_t = 7000)]
@ -223,6 +223,11 @@ async fn cmd_serve(
agent_memory_max: String, agent_memory_max: String,
socket: &std::path::Path, socket: &std::path::Path,
) -> Result<()> { ) -> Result<()> {
// Move any host-side state still at the legacy flat layout into its
// subdir (`db/`, `forge/`, `matrix/`, `run/`) BEFORE opening the
// broker db — the broker + build-logs dbs are among the relocated
// files. Idempotent; a no-op once migrated.
hive_c0re::paths::relocate_legacy_state();
let cwt: std::collections::HashMap<String, u64> = serde_json::from_str(&context_window_tokens) let cwt: std::collections::HashMap<String, u64> = serde_json::from_str(&context_window_tokens)
.context("--context-window-tokens: invalid JSON")?; .context("--context-window-tokens: invalid JSON")?;
let coord = Arc::new(Coordinator::open( let coord = Arc::new(Coordinator::open(

View file

@ -55,7 +55,7 @@ pub const HIVE_SPACE_NAME: &str = "hive";
/// purgeable path — not deleted by `destroy --purge` on any agent. /// purgeable path — not deleted by `destroy --purge` on any agent.
#[must_use] #[must_use]
pub fn admin_token_path() -> PathBuf { pub fn admin_token_path() -> PathBuf {
PathBuf::from("/var/lib/hyperhive/matrix-admin-token") crate::paths::matrix_admin_token()
} }
/// Token file inside the agent's bind-mounted state dir (visible as /// Token file inside the agent's bind-mounted state dir (visible as
@ -68,7 +68,7 @@ fn token_path(name: &str) -> PathBuf {
/// purgeable `agent_state_root` tree so it survives `destroy --purge` /// purgeable `agent_state_root` tree so it survives `destroy --purge`
/// and allows re-login recovery when the same agent name is re-spawned. /// and allows re-login recovery when the same agent name is re-spawned.
/// ///
/// Path: `/var/lib/hyperhive/matrix-creds/<name>-password` /// Path: `/var/lib/hyperhive/matrix/creds/<name>-password`
/// ///
/// The token file lives inside the agent's bind-mounted state dir (under /// The token file lives inside the agent's bind-mounted state dir (under
/// `agent_notes_dir`) so the agent container can read it; the password /// `agent_notes_dir`) so the agent container can read it; the password
@ -76,7 +76,7 @@ fn token_path(name: &str) -> PathBuf {
/// the access token exclusively) and belongs with other hive-c0re /// the access token exclusively) and belongs with other hive-c0re
/// credential state, not inside the purgeable per-agent tree. /// credential state, not inside the purgeable per-agent tree.
fn password_path(name: &str) -> PathBuf { fn password_path(name: &str) -> PathBuf {
PathBuf::from("/var/lib/hyperhive/matrix-creds").join(format!("{name}-password")) crate::paths::matrix_creds_dir().join(format!("{name}-password"))
} }
/// Legacy password path (inside the old purgeable `agent_notes_dir`). /// Legacy password path (inside the old purgeable `agent_notes_dir`).
@ -92,7 +92,7 @@ fn legacy_password_path(name: &str) -> PathBuf {
/// Outside every purgeable path — not deleted by `destroy --purge`. /// Outside every purgeable path — not deleted by `destroy --purge`.
#[must_use] #[must_use]
pub fn hive_space_room_id_path() -> PathBuf { pub fn hive_space_room_id_path() -> PathBuf {
PathBuf::from("/var/lib/hyperhive/matrix-space-room-id") crate::paths::matrix_space_room_id()
} }
/// Probe whether `hive-matrix` exists as a nixos-container. Cheap — /// Probe whether `hive-matrix` exists as a nixos-container. Cheap —
@ -661,7 +661,7 @@ pub async fn ensure_user_for(
// Password file missing — attempt auto-recovery via admin API. // Password file missing — attempt auto-recovery via admin API.
// This covers the case where agent state dirs were wiped but the // This covers the case where agent state dirs were wiped but the
// homeserver still has the accounts. Requires the hive admin // homeserver still has the accounts. Requires the hive admin
// token at /var/lib/hyperhive/matrix-admin-token. // token at /var/lib/hyperhive/matrix/admin-token.
tracing::info!( tracing::info!(
%name, %name,
"matrix: stored password missing, attempting admin-API auto-recovery" "matrix: stored password missing, attempting admin-API auto-recovery"

View file

@ -101,7 +101,7 @@ pub async fn sync_agents(hive: &HiveEnv, agents: &[AgentSpec]) -> Result<()> {
crate::topology::reconcile(&agent_names) crate::topology::reconcile(&agent_names)
.with_context(|| format!("reconcile {}", crate::topology::topology_path().display()))?; .with_context(|| format!("reconcile {}", crate::topology::topology_path().display()))?;
// Refresh /var/lib/hyperhive/agent-ports.json so the hive-gateway // Refresh /var/lib/hyperhive/run/agent-ports.json so the hive-gateway
// nginx sees the new agent set. The file is the single source of // nginx sees the new agent set. The file is the single source of
// truth for which agents the gateway proxies to, since the // truth for which agents the gateway proxies to, since the
// gateway container lives in system config and can't be rebuilt // gateway container lives in system config and can't be rebuilt
@ -115,7 +115,7 @@ pub async fn sync_agents(hive: &HiveEnv, agents: &[AgentSpec]) -> Result<()> {
tracing::warn!(error = ?e, "agent_ports::write failed (non-fatal)"); tracing::warn!(error = ?e, "agent_ports::write failed (non-fatal)");
} }
// Refresh /var/lib/hyperhive/agent-sockets.json — sibling to the // Refresh /var/lib/hyperhive/run/agent-sockets.json — sibling to the
// ports map, drives the gateway's unix-socket upstreams once // ports map, drives the gateway's unix-socket upstreams once
// agents opt in to `HIVE_WEB_SOCKET`. Coexists with the TCP-port // agents opt in to `HIVE_WEB_SOCKET`. Coexists with the TCP-port
// map during the transition: the gateway picks the socket // map during the transition: the gateway picks the socket

214
hive-c0re/src/paths.rs Normal file
View file

@ -0,0 +1,214 @@
//! Central host-side state paths under `/var/lib/hyperhive`.
//!
//! Historically these were flat string literals scattered across many
//! modules (`broker.sqlite`, `matrix-admin-token`, `agent-ports.json`,
//! …) directly under the state root. This module groups the **strictly
//! host-side** ones (read/written by hive-c0re alone, no nix-module or
//! container coupling) into subdirs: `db/`, `forge/`, `matrix/`, `run/`.
//!
//! Nix-coupled paths (`forge-core-token`, `matrix-register-token`,
//! `gateway/`, `meta/`, `agents/`) are intentionally **not** moved here
//! — they cross into nix modules / bind mounts and are tracked
//! separately so the Rust path and the nix default can move in lockstep.
//!
//! [`relocate_legacy_state`] moves any file still at the old flat
//! location into its new subdir on startup, before the broker db is
//! opened.
use std::path::{Path, PathBuf};
/// Root of all hive-c0re persistent state.
pub const STATE_ROOT: &str = "/var/lib/hyperhive";
/// Default broker db path (`db/broker.sqlite`). Exposed as a `&str` for
/// the `--broker-db` clap `default_value`; `build_logs.sqlite` is placed
/// alongside it (the build-logs store keys off the broker db's parent).
pub const BROKER_DB: &str = "/var/lib/hyperhive/db/broker.sqlite";
#[must_use]
pub fn state_root() -> PathBuf {
PathBuf::from(STATE_ROOT)
}
/// `db/` — sqlite databases (broker, build logs).
#[must_use]
pub fn db_dir() -> PathBuf {
state_root().join("db")
}
/// `forge/` — hive-c0re's own forge provisioning markers.
#[must_use]
pub fn forge_dir() -> PathBuf {
state_root().join("forge")
}
/// `forge/core-avatar-set` — marker: core account avatar uploaded.
#[must_use]
pub fn forge_core_avatar_marker() -> PathBuf {
forge_dir().join("core-avatar-set")
}
/// `forge/agent-configs-avatar-set` — marker: agent-configs org avatar set.
#[must_use]
pub fn forge_config_org_avatar_marker() -> PathBuf {
forge_dir().join("agent-configs-avatar-set")
}
/// `forge/email-aligned-<name>` — marker: `<name>`'s forge email aligned.
#[must_use]
pub fn forge_email_aligned_marker(name: &str) -> PathBuf {
forge_dir().join(format!("email-aligned-{name}"))
}
/// `matrix/` — host-side matrix provisioning state (admin token, hive
/// Space room id, per-agent password creds). The shared registration
/// token is bind-mounted into the tuwunel container via nix and stays
/// at its own path (tracked separately).
#[must_use]
pub fn matrix_dir() -> PathBuf {
state_root().join("matrix")
}
/// `matrix/admin-token` — hive system admin access token.
#[must_use]
pub fn matrix_admin_token() -> PathBuf {
matrix_dir().join("admin-token")
}
/// `matrix/space-room-id` — persisted hive Space room id.
#[must_use]
pub fn matrix_space_room_id() -> PathBuf {
matrix_dir().join("space-room-id")
}
/// `matrix/creds/` — per-agent throwaway matrix passwords (survive
/// `destroy --purge`; agents auth by token, this is recovery only).
#[must_use]
pub fn matrix_creds_dir() -> PathBuf {
matrix_dir().join("creds")
}
/// `run/` — runtime maps hive-c0re regenerates on every meta sync.
#[must_use]
pub fn run_dir() -> PathBuf {
state_root().join("run")
}
/// `run/agent-ports.json` — name→port map the gateway routing reads.
#[must_use]
pub fn agent_ports_file() -> PathBuf {
run_dir().join("agent-ports.json")
}
/// `run/agent-sockets.json` — name→socket-path map for UDS upstreams.
#[must_use]
pub fn agent_sockets_file() -> PathBuf {
run_dir().join("agent-sockets.json")
}
/// Move any host-side state file still at its legacy flat location
/// (directly under [`STATE_ROOT`]) into its new subdir. Idempotent and
/// rename-based: a move only happens when the old path exists and the
/// new one doesn't, so re-runs are no-ops.
///
/// Must run **before** the broker db is opened (the broker + build-logs
/// dbs are relocated here). Safe because those dbs use rollback-journal
/// mode (no `-wal`/`-shm` sidecars after a clean shutdown), and a rename
/// within the same filesystem is atomic.
pub fn relocate_legacy_state() {
let root = state_root();
let moves: [(&str, PathBuf); 8] = [
("broker.sqlite", db_dir().join("broker.sqlite")),
("build_logs.sqlite", db_dir().join("build_logs.sqlite")),
("forge-core-avatar-set", forge_core_avatar_marker()),
(
"forge-agent-configs-avatar-set",
forge_config_org_avatar_marker(),
),
("matrix-admin-token", matrix_admin_token()),
("matrix-space-room-id", matrix_space_room_id()),
("matrix-creds", matrix_creds_dir()),
("agent-sockets.json", agent_sockets_file()),
];
for (old_rel, new) in &moves {
move_if_legacy(&root.join(old_rel), new);
}
// agent-ports.json handled here too (kept out of the array only to
// keep the fixed-size literal tidy).
move_if_legacy(&root.join("agent-ports.json"), &agent_ports_file());
// `forge-email-aligned-<name>` markers: glob the flat root.
if let Ok(rd) = std::fs::read_dir(&root) {
for ent in rd.flatten() {
if let Some(name) = ent
.file_name()
.to_str()
.and_then(|s| s.strip_prefix("forge-email-aligned-"))
{
move_if_legacy(&ent.path(), &forge_email_aligned_marker(name));
}
}
}
}
/// Rename `old` → `new` when `old` exists and `new` doesn't, creating
/// `new`'s parent dir first. Logs on success / failure; never panics
/// (a failed relocate must not take the daemon down — worst case the
/// owning module recreates fresh state at the new path).
fn move_if_legacy(old: &Path, new: &Path) {
if !old.exists() || new.exists() {
return;
}
if let Some(parent) = new.parent() {
let _ = std::fs::create_dir_all(parent);
}
match std::fs::rename(old, new) {
Ok(()) => tracing::info!(
from = %old.display(),
to = %new.display(),
"relocate: moved legacy state into subdir"
),
Err(e) => tracing::warn!(
from = %old.display(),
to = %new.display(),
error = ?e,
"relocate: rename failed; owning module will recreate at the new path"
),
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::fs;
#[test]
fn move_if_legacy_moves_then_is_idempotent() {
let tmp = tempfile::tempdir().unwrap();
let old = tmp.path().join("flat-file");
let new = tmp.path().join("sub/dir/new-file");
fs::write(&old, b"payload").unwrap();
move_if_legacy(&old, &new);
assert!(!old.exists(), "old should be gone after move");
assert_eq!(fs::read(&new).unwrap(), b"payload");
// Re-run with old absent → no-op, new untouched.
move_if_legacy(&old, &new);
assert_eq!(fs::read(&new).unwrap(), b"payload");
}
#[test]
fn move_if_legacy_skips_when_new_exists() {
let tmp = tempfile::tempdir().unwrap();
let old = tmp.path().join("flat");
let new = tmp.path().join("sub/new");
fs::write(&old, b"OLD").unwrap();
fs::create_dir_all(new.parent().unwrap()).unwrap();
fs::write(&new, b"NEW").unwrap();
// New already present → must NOT overwrite, old left in place.
move_if_legacy(&old, &new);
assert_eq!(fs::read(&new).unwrap(), b"NEW");
assert!(old.exists(), "old left untouched when new exists");
}
}

View file

@ -4,7 +4,7 @@
//! `interval_seconds` and re-arm `next_fire_at` on delivery; //! `interval_seconds` and re-arm `next_fire_at` on delivery;
//! one-shots are reaped. //! one-shots are reaped.
//! //!
//! Schema + retention: `docs/persistence.md::/var/lib/hyperhive/broker.sqlite` //! Schema + retention: `docs/persistence.md::/var/lib/hyperhive/db/broker.sqlite`
//! (the `scheduled_prompts` / `scheduled_prompt_targets` table bullets). //! (the `scheduled_prompts` / `scheduled_prompt_targets` table bullets).
//! Submit paths (operator-direct vs `ApprovalKind::SchedulePrompt`, //! Submit paths (operator-direct vs `ApprovalKind::SchedulePrompt`,
//! plus why even agent-self schedules go through approval): //! plus why even agent-self schedules go through approval):