diff --git a/docs/integrations/forge.md b/docs/integrations/forge.md index 472fd6cc..f39dbaea 100644 --- a/docs/integrations/forge.md +++ b/docs/integrations/forge.md @@ -69,14 +69,11 @@ forge container: - seeding `proposed` from `agent-configs/` on first deploy - cloning and pulling `/knowledge` -The builds read as the hive's account too. nix fetches the config inputs and -`internal/meta` with `git`, and `git` gets its credential from -`git-credential-hive-forge`, which reads `/run/hyperhive/forge-hive-token`. -hive-c0re writes that file (account on the first line, token on the second) -from the hive's store token at each boot and each `internal/meta` read, when its -content differs. A hive whose store holds no token has no such file, fetches -nothing private through nix and fails its builds: they have no other flake than -`internal/meta`. This also holds on a hive with no forge container. +The hive builds nothing, so no nix process reads the forge as the hive's +account. hive-c0re hands the token to each `git` command it runs +(`http.extraHeader`) and keeps no copy on disk. A hive whose store holds no token +reads nothing private from the forge. This also holds on a hive with no forge +container. Roll a change to any of this out store host first (its granter writes the controller's grant on `swarm/hives/+/forge/token`), then the controller host, diff --git a/docs/scheduler/coordinator.md b/docs/scheduler/coordinator.md index 7b5b3dff..2966f16f 100644 --- a/docs/scheduler/coordinator.md +++ b/docs/scheduler/coordinator.md @@ -72,7 +72,6 @@ Cheap — no build slot: | `SetWanted` | write the durable power intent (`wanted = Up`/`Offline`) as the head node of a power-op DAG. Takes the agent lease even though it's a store write, so the intent write and the tail `Reconcile` are atomic per-agent — two racing power ops can't clobber each other's intent before either reconciles | | `EmitRebuilt` | tail of a rebuild — emit the agent's `Rebuilt` manager event (ok/fail per outcome, nothing on cancel). One node per agent _and_ per outcome | | `WriteDropin` | `set_nspawn_flags` + `set_resource_limits` + daemon-reload | -| `ForgeSweep` | one-shot boot-time publish of the hive's forge read token for nix (`forge::ensure_all`) as a first-class node, so it shows as real work on the dashboard instead of running invisibly in a bare `tokio::spawn`. Agentless | | `KnowledgePull` | `/knowledge` pull (`knowledge::pull`): at boot (commits that landed while `hive-c0re` was down), on the swarm knowledge-changed event, and hourly as a fallback. Holds `Resource::KnowledgeTree` (capacity 1), so two pulls never overlap on the working tree; each trigger queues its own pass, which waits for the resource if one is already live. Agentless | | `WantedPull` | pull of the agent set the swarm controller declares for this hive (`wanted::pull`), converging the agents it names. Runs at boot, waiting up to a minute in total for the queue connection, and again after every attach of the wanted-state watch (which retries with a backoff when the attach fails or the stream ends); the deploy event (`swarm_status`) and the watch are the fast path, this repairs a missed one. A first deploy already queued or running for an agent isn't queued again. Agentless | diff --git a/docs/swarm/credentials-reference.md b/docs/swarm/credentials-reference.md index 7dd943f5..c75ee094 100644 --- a/docs/swarm/credentials-reference.md +++ b/docs/swarm/credentials-reference.md @@ -156,7 +156,7 @@ pieces fit, and the reasoning behind them, is in [`credentials.md`](credentials. | the cache's public key and pull token, for hive-priv | `swarm-controller` copies them to `swarm/hives//attic/{public-key,pull-token}` (see the cache-facts row above) | `hive-priv-cache-facts` (root) reads those two under the hive's own store identity every ten minutes and writes `/run/hyperhive-priv/attic/public-key` and `netrc` (tmpfs, root, each `0600`, directory `0700`); the cache's address comes from `services.hyperhive.swarm.attic.domain` and `cache`, not from the store. A store that holds no facts, or that doesn't answer, leaves the files as they stand, and a hive that never got them has `hive-priv` answer `not available`. | | the forge's copy of its OIDC secret | `swarm-bao-forge-oidc.service` reads it out of the swarm secret store under the forge publisher's leaf, wherever authelia runs | `/run/hive-forge-oidc/.secret` on the host (tmpfs), bound into the forge container, handed to `forgejo-sso-source` through `LoadCredential` | | the homeserver's copy of its OIDC secret | `swarm-bao-matrix-oidc.service` reads it out of the swarm secret store under matrix-ctl's leaf, wherever authelia runs | `/run/hive-matrix-oidc/.secret` on the host (tmpfs), bound into the homeserver container, handed to tuwunel through `LoadCredential` | -| this hive's forge read token | `swarm-controller`, into the secret store at `swarm/hives//forge/token`, for the `hive-` account; a pass at start and every five minutes re-mints a missing or stale token | `hive-c0re` reads it under the hive's own store identity, and copies it (account on the first line, token on the second) to `/run/hyperhive/forge-hive-token` (tmpfs, `0600` `hive-core`) for nix's `git`, which `hive-priv` also runs. Each `internal/meta` read and each boot rewrites the copy when it differs; where they differ after a rotation, a build that starts before the next read fails to authenticate, and a retry reads the new token | +| this hive's forge read token | `swarm-controller`, into the secret store at `swarm/hives//forge/token`, for the `hive-` account; a pass at start and every five minutes re-mints a missing or stale token | `hive-c0re` reads it under the hive's own store identity for each forge read, and hands it to the `git` commands it runs. It keeps no copy on disk | | squid's users file | `hive-c0re` (`workers::squid_users`), every 30 seconds, from the proxy credentials of its agents and the self-check | `services.hyperhive.deploy.hive-controller.squid.usersFile` (`/var/lib/hive-squid/users`, `hive-core:squid`, `0640`): `:` lines, SHA-512 crypt hashes only, renamed into place when it changed. `unrestricted` beside it lists the users whose agent holds the swarm's `unrestricted_internet` flag, names only; it lists every agent the hive never read as off `systemd-tmpfiles` creates it empty so squid's helper never sees a missing file. A hive whose agents have no credential yet (an older controller) has an empty file: every login fails and the self-check reports `ok=false`; an agent without one sends none, so it goes direct | | the squid self-check's password | `swarm-controller` mints `swarm/hives//squid-selfcheck`; `squid-selfcheck-credential` (root, every 10 minutes) reads it as the hive | `/run/credstore/hive-squid.selfcheck` (tmpfs, root, `0400`), imported by `squid-selfcheck.service` as a systemd credential and handed to `curl` on its stdin. Both self-check users (`selfcheck.squid` and the always-unrestricted `selfcheck-unrestricted.squid`) use it, so whoever holds it has unrestricted egress through this squid Missing, or different from the users file: the self-check logs `credential=absent` or a 407 and fails, and nothing else breaks | diff --git a/docs/trust-boundary/security.md b/docs/trust-boundary/security.md index ebe65029..fa131ec3 100644 --- a/docs/trust-boundary/security.md +++ b/docs/trust-boundary/security.md @@ -252,34 +252,34 @@ for every operation that genuinely requires root. **Narrow interface** — `PrivRequest` variants map 1:1 to specific known operations; there is no arbitrary command pass-through: -| Operation | What it runs | -| ------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `StartContainer` / `StopContainer` | `nixos-container start/stop ` | -| `KillContainer` | `machinectl kill --signal=SIGKILL` (`nixos-container` has no kill verb) | -| `CreateContainer` / `UpdateContainer` | `nixos-container create --system-path `; for an update, `nix-env -p --set ` and a reload of the unit. `` must be the store path the last successful `RealiseClosure` returned for that agent (in memory, per agent), so hive-c0re can't name a path never realised for the agent; hive-priv builds nothing | -| `RealiseClosure` | `nix-store --realise --max-jobs 0` against the swarm cache, with the host's configured cache address, and the public key and netrc from root-owned `/run/hyperhive-priv/attic/`, as per-call settings that replace the host's substituters and trusted keys and require a signature. The path must be `/nix/store/<32 chars>-` and the agent name valid. It downloads and never builds | -| `DestroyContainer` | `nixos-container destroy ` | -| `ListContainers` | `nixos-container list` | -| `ReloadGatewayNginx` | `systemctl reload/start/reset-failed nginx` (host unit; the unit name is hard-coded, not a parameter) | -| `WriteNspawnFlags` | write `/etc/nixos-containers/.conf` (bind-mount list, network isolation vars, and `--ephemeral` after the root's salvage). hive-priv first copies a stopped container's root to the agent's `work/`, then renames it aside (it deletes nothing), following no link the container could have planted | -| `EnsureAgentSocketDir` | `mkdirat` `/run/hive-agent/` `0751 root:root` under an `O_NOFOLLOW` parent fd; leaves an existing directory alone, refuses anything else | -| `WriteResourceLimits` | write `CPUQuota=`/`MemoryMax=`/`CPUWeight=`/`IOWeight=` systemd drop-in for agent container, with an `ExecStartPre=` that runs `hive-priv check-rootfs ` (no-follow check of the root paths the start script touches; see persistence.md), and a GC root on the build it names | -| `RemoveServiceDropin` | remove `container@.service.d/` drop-in on destroy, and the GC root of the hive-priv build it named | -| `DaemonReload` | `systemctl daemon-reload` | -| `ControlInfraContainer` | `systemctl nginx.service` for the gateway — the `InfraContainer` enum is the allowlist, and serde rejects unknown names at the wire boundary (`hive-c0re`, the forge and the matrix have no variant, so no request can name them) | -| `SyncAgentTmpfiles` | unlink `/etc/tmpfiles.d/hyperhive-agents.conf` and return `Ok` | -| `SetAgentPaused` | create / remove the `//harness/paused` marker that parks an agent's turn loop | -| `EnsureAgentSubvolume` | `btrfs subvolume create /` for a new agent — no-op when the path exists or the filesystem isn't btrfs | -| `UpgradeAgentSubvolume` | migrate an existing plain state dir into a subvolume: create, `cp -a --reflink=auto`, atomic swap. Operator opt-in, and the caller stops the agent first | -| `DeleteAgentSubvolume` | `btrfs subvolume delete /` — purge path only, never a plain destroy | -| `EnsureBtrfsQuota` | `btrfs quota enable `. Operator opt-in — enabling forces a full rescan | -| `ReadSubvolumeUsage` | `btrfs qgroup show -f --raw /` | -| `SetSubvolumeQuota` | `btrfs qgroup limit /` | -| `MeasureAgentDisk` | `du -s -x -b -c -P` over `//state`, `//work` and `/var/lib/nixos-containers/h-`, killed after 10 min. hive-priv derives the paths from the name; the request carries none | -| `SnapshotAgentSubvolume` | `btrfs subvolume snapshot -r ` — the frozen point-in-time copy a migration streams from | -| `DeleteAgentSnapshot` | `btrfs subvolume delete ` | -| `SendAgentSnapshotToFile` | `btrfs send [-p ] ` into a bare filename under `MIGRATE_STAGING_ROOT`; refuses to overwrite an existing export | -| `SendAgentSnapshotToFd` | `btrfs send [-p ] ` into a **file descriptor passed with the request** (`SCM_RIGHTS`). hive-c0re connects to the peer and hands over the connected socket, so hive-priv never learns an address or protocol. hive-priv requires exactly one descriptor here, and refuses one that arrives alongside any other operation | +| Operation | What it runs | +| ------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `StartContainer` / `StopContainer` | `nixos-container start/stop ` | +| `KillContainer` | `machinectl kill --signal=SIGKILL` (`nixos-container` has no kill verb) | +| `CreateContainer` / `UpdateContainer` | `nixos-container create --system-path `; for an update, `nix-env -p --set ` and a reload of the unit. `` must be the store path the last successful `RealiseClosure` returned for that agent (in memory, per agent), so hive-c0re can't name a path never realised for the agent; hive-priv builds nothing. The binding doesn't mean hive-c0re can't choose the closure: it can still ask hive-priv to download any swarm-signed path for any agent and then apply it. The swarm key and the cache bound that to a cross-assignment risk (one agent gets another's closure), not an arbitrary-path one | +| `RealiseClosure` | `nix-store --realise --max-jobs 0` against the swarm cache, with the host's configured cache address, and the public key and netrc from root-owned `/run/hyperhive-priv/attic/`, as per-call settings that replace the host's substituters and trusted keys and require a signature. The path must be `/nix/store/<32 chars>-` and the agent name valid. It downloads and never builds | +| `DestroyContainer` | `nixos-container destroy ` | +| `ListContainers` | `nixos-container list` | +| `ReloadGatewayNginx` | `systemctl reload/start/reset-failed nginx` (host unit; the unit name is hard-coded, not a parameter) | +| `WriteNspawnFlags` | write `/etc/nixos-containers/.conf` (bind-mount list, network isolation vars, and `--ephemeral` after the root's salvage). hive-priv first copies a stopped container's root to the agent's `work/`, then renames it aside (it deletes nothing), following no link the container could have planted | +| `EnsureAgentSocketDir` | `mkdirat` `/run/hive-agent/` `0751 root:root` under an `O_NOFOLLOW` parent fd; leaves an existing directory alone, refuses anything else | +| `WriteResourceLimits` | write `CPUQuota=`/`MemoryMax=`/`CPUWeight=`/`IOWeight=` systemd drop-in for agent container, with an `ExecStartPre=` that runs `hive-priv check-rootfs ` (no-follow check of the root paths the start script touches; see persistence.md), and a GC root on the build it names | +| `RemoveServiceDropin` | remove `container@.service.d/` drop-in on destroy, and the GC root of the hive-priv build it named | +| `DaemonReload` | `systemctl daemon-reload` | +| `ControlInfraContainer` | `systemctl nginx.service` for the gateway — the `InfraContainer` enum is the allowlist, and serde rejects unknown names at the wire boundary (`hive-c0re`, the forge and the matrix have no variant, so no request can name them) | +| `SyncAgentTmpfiles` | unlink `/etc/tmpfiles.d/hyperhive-agents.conf` and return `Ok` | +| `SetAgentPaused` | create / remove the `//harness/paused` marker that parks an agent's turn loop | +| `EnsureAgentSubvolume` | `btrfs subvolume create /` for a new agent — no-op when the path exists or the filesystem isn't btrfs | +| `UpgradeAgentSubvolume` | migrate an existing plain state dir into a subvolume: create, `cp -a --reflink=auto`, atomic swap. Operator opt-in, and the caller stops the agent first | +| `DeleteAgentSubvolume` | `btrfs subvolume delete /` — purge path only, never a plain destroy | +| `EnsureBtrfsQuota` | `btrfs quota enable `. Operator opt-in — enabling forces a full rescan | +| `ReadSubvolumeUsage` | `btrfs qgroup show -f --raw /` | +| `SetSubvolumeQuota` | `btrfs qgroup limit /` | +| `MeasureAgentDisk` | `du -s -x -b -c -P` over `//state`, `//work` and `/var/lib/nixos-containers/h-`, killed after 10 min. hive-priv derives the paths from the name; the request carries none | +| `SnapshotAgentSubvolume` | `btrfs subvolume snapshot -r ` — the frozen point-in-time copy a migration streams from | +| `DeleteAgentSnapshot` | `btrfs subvolume delete ` | +| `SendAgentSnapshotToFile` | `btrfs send [-p ] ` into a bare filename under `MIGRATE_STAGING_ROOT`; refuses to overwrite an existing export | +| `SendAgentSnapshotToFd` | `btrfs send [-p ] ` into a **file descriptor passed with the request** (`SCM_RIGHTS`). hive-c0re connects to the peer and hands over the connected socket, so hive-priv never learns an address or protocol. hive-priv requires exactly one descriptor here, and refuses one that arrives alongside any other operation | **Closures from the swarm cache** — `RealiseClosure` is how a hive's rebuild takes the closure the swarm built instead of building it. Root-owned sources diff --git a/hive-c0re/src/forge/mod.rs b/hive-c0re/src/forge/mod.rs index a65c9fd8..1a063e98 100644 --- a/hive-c0re/src/forge/mod.rs +++ b/hive-c0re/src/forge/mod.rs @@ -1,5 +1,5 @@ //! The hive's forge access: reads of the forge go through [`reader_token`] on -//! every hive, and [`ensure_all`] publishes that token for nix's `git`. The +//! every hive. The //! swarm-wide orgs, teams and repos, and each agent's read grant on //! `internal/docs`, are swarm-controller's. Full design: //! `docs/integrations/forge.md`. @@ -10,7 +10,7 @@ mod reconcile; mod repos; pub(crate) use internal_meta::internal_meta_source; -pub use reader::{publish_nix_credential, reader_token}; +pub use reader::reader_token; pub use reconcile::{fetch_forge_main, reconcile_config_apply, reconcile_config_status}; pub use repos::{clone_config_into_proposed, fetch_config_main_into_applied}; @@ -145,16 +145,6 @@ where .map_err(|_| anyhow::anyhow!("{op}: no answer from the forge within {API_TIMEOUT:?}")) } -/// Publish this hive's forge read token for nix's `git` at startup, so a build -/// that starts before any `internal/meta` read still has a credential. Every -/// hive runs it: the token is the hive's own, in the store, whether or not a -/// forge container is local. -pub async fn ensure_all() { - if publish_nix_credential().await { - tracing::debug!("forge: nix's forge credential is current"); - } -} - #[cfg(test)] mod tests { use super::{API_TIMEOUT, bounded, git_url_with_base}; diff --git a/hive-c0re/src/forge/reader.rs b/hive-c0re/src/forge/reader.rs index 149ea67f..0bc243fa 100644 --- a/hive-c0re/src/forge/reader.rs +++ b/hive-c0re/src/forge/reader.rs @@ -1,14 +1,6 @@ //! The credential this hive reads the forge with: its own read-only account, //! whose token `swarm-controller` mints into the store at //! `swarm/hives//forge/token`. -//! -//! nix's `git` cannot reach the store, so [`ReaderToken::publish_for_nix`] -//! copies the token to a runtime file that `git-credential-hive-forge` -//! (`nix/host-modules/lib/hive-forge-git.nix`) reads. - -use std::path::Path; - -use anyhow::Result; use super::basic_auth_header; @@ -39,30 +31,6 @@ impl ReaderToken { pub(crate) fn api(&self) -> anyhow::Result { super::api(&self.value) } - - /// Copy this token to the file nix's `git` authenticates from - /// ([`crate::paths::FORGE_HIVE_TOKEN`]). Whether the file now holds it; a - /// failed write is logged and leaves the file as it was. - pub(crate) fn publish_for_nix(&self) -> bool { - let path = Path::new(crate::paths::FORGE_HIVE_TOKEN); - match self.write_credential(path) { - Ok(()) => true, - Err(e) => { - tracing::warn!(path = %path.display(), error = ?e, "forge: could not write the nix forge credential"); - false - } - } - } - - /// Login on the first line, token on the second. Written only when the - /// content differs, so a build's resolve is not a disk write. - fn write_credential(&self, path: &Path) -> Result<()> { - let content = format!("{}\n{}\n", self.login, self.value); - if std::fs::read(path).is_ok_and(|have| have == content.as_bytes()) { - return Ok(()); - } - crate::secret_file::write_secret_0600(path, content.as_bytes()) - } } /// The token to read the forge with, or `None` when the store holds none for @@ -77,16 +45,6 @@ pub async fn reader_token() -> Option { token } -/// Publish this hive's read token for nix; see [`ReaderToken::publish_for_nix`]. -/// Whether the file holds a token. -pub async fn publish_nix_credential() -> bool { - let Some(token) = reader_token().await else { - tracing::debug!("forge: no hive read token in the store; nix has no forge credential"); - return false; - }; - token.publish_for_nix() -} - /// This hive's token from the store, under the hive's own identity. /// /// `None`, never an error, for every way it can come up empty — no hive name, @@ -131,8 +89,6 @@ async fn stored_reader_token() -> Option { #[cfg(test)] mod tests { use super::*; - use std::os::unix::fs::PermissionsExt as _; - fn stored() -> ReaderToken { ReaderToken { login: "hive-alpha".to_owned(), @@ -140,73 +96,6 @@ mod tests { } } - fn mode(path: &Path) -> u32 { - std::fs::metadata(path).expect("stat").permissions().mode() & 0o777 - } - - #[test] - fn the_nix_credential_is_the_hives_login_then_its_token_at_0600() { - let dir = tempfile::tempdir().expect("tempdir"); - let path = dir.path().join("forge-hive-token"); - stored().write_credential(&path).expect("written"); - assert_eq!( - std::fs::read_to_string(&path).expect("read"), - "hive-alpha\nfrom-the-store\n" - ); - assert_eq!(mode(&path), 0o600); - } - - #[test] - fn a_rotated_token_replaces_the_nix_credential() { - let dir = tempfile::tempdir().expect("tempdir"); - let path = dir.path().join("forge-hive-token"); - stored().write_credential(&path).expect("first"); - let rotated = ReaderToken { - login: "hive-alpha".to_owned(), - value: "rotated".to_owned(), - }; - rotated.write_credential(&path).expect("second"); - assert_eq!( - std::fs::read_to_string(&path).expect("read"), - "hive-alpha\nrotated\n" - ); - assert_eq!(mode(&path), 0o600); - } - - #[test] - fn an_unchanged_credential_is_not_rewritten() { - let dir = tempfile::tempdir().expect("tempdir"); - let path = dir.path().join("forge-hive-token"); - stored().write_credential(&path).expect("first"); - // A rewrite goes through a sibling temp file and a rename, which - // gives the file a new inode. - let before = std::fs::metadata(&path).expect("stat"); - stored().write_credential(&path).expect("second"); - let after = std::fs::metadata(&path).expect("stat"); - assert_eq!( - std::os::unix::fs::MetadataExt::ino(&before), - std::os::unix::fs::MetadataExt::ino(&after) - ); - } - - /// `hive-forge-git.nix`'s credential helper names this path as a literal. - #[test] - fn the_nix_credential_lives_on_the_runtime_tmpfs_where_the_helper_reads_it() { - assert_eq!( - crate::paths::FORGE_HIVE_TOKEN, - "/run/hyperhive/forge-hive-token" - ); - assert!(crate::paths::FORGE_HIVE_TOKEN.starts_with(crate::paths::RUNTIME_ROOT)); - } - - #[test] - fn a_path_that_cannot_be_written_is_an_error_not_a_panic() { - let dir = tempfile::tempdir().expect("tempdir"); - let file = dir.path().join("a-file"); - std::fs::write(&file, "").expect("file"); - assert!(stored().write_credential(&file.join("sub")).is_err()); - } - #[test] fn the_header_names_the_tokens_own_account() { use base64::Engine as _; diff --git a/hive-c0re/src/job_queue/exec.rs b/hive-c0re/src/job_queue/exec.rs index 2de5dd9d..39f0994b 100644 --- a/hive-c0re/src/job_queue/exec.rs +++ b/hive-c0re/src/job_queue/exec.rs @@ -99,8 +99,6 @@ pub(super) trait Host: Sync { async fn delete_state_subvolume(&self, name: &str) -> Result<()>; /// The persistent directories a purge removes. fn purge_dirs(&self, name: &str) -> Vec; - /// Publish the hive's forge credential for nix. - async fn forge_ensure_all(&self); /// One `/knowledge` pull. async fn knowledge_pull(&self, coord: &Arc) -> Result<()>; /// One swarm wanted-state pull. @@ -228,10 +226,6 @@ impl Host for RealHost { .collect() } - async fn forge_ensure_all(&self) { - crate::forge::ensure_all().await; - } - async fn knowledge_pull(&self, coord: &Arc) -> Result<()> { crate::workers::knowledge::pull(coord).await } @@ -332,23 +326,12 @@ async fn dispatch( // `Finishing` so the nodes under it start. What they declare stays held // until their whole subtree settles. NodeKind::AgentWindow { .. } => Ok(()), - NodeKind::ForgeSweep => run_forge_sweep(host).await, NodeKind::KnowledgePull => run_knowledge_pull(host, coord, KNOWLEDGE_PULL_DEADLINE).await, NodeKind::WantedPull => run_wanted_pull(host, coord).await, }; (builder, result) } -/// Boot-time publish of the hive's forge credential for nix, as a DAG node — -/// see [`NodeKind::ForgeSweep`]. `forge::ensure_all` logs its own outcome and -/// is best-effort (a build re-publishes the credential before it fetches), so -/// this wrapper has nothing left to report; it exists to make the step a -/// visible unit of work. -async fn run_forge_sweep(host: &impl Host) -> Result<()> { - host.forge_ensure_all().await; - Ok(()) -} - /// `/knowledge` pull as a DAG node — see [`NodeKind::KnowledgePull`]. Every /// pass reports here, whoever submitted it: as the node's own error on the /// dashboard, and into the debounced warning. diff --git a/hive-c0re/src/job_queue/exec/tests.rs b/hive-c0re/src/job_queue/exec/tests.rs index 6d4e2920..a95f947e 100644 --- a/hive-c0re/src/job_queue/exec/tests.rs +++ b/hive-c0re/src/job_queue/exec/tests.rs @@ -12,9 +12,9 @@ use hive_jobq::scheduler::Scheduler; use super::{ Host, dispatch, run_create, run_destroy_bookkeeping, run_destroy_container, run_drain, - run_emit_rebuilt, run_forge_sweep, run_knowledge_pull, run_meta_sync, run_pause_drain, - run_pause_signal, run_prebuild, run_provision, run_purge_state, run_rebuild_bookkeeping, - run_reconcile, run_set_wanted, run_signal, run_start, run_stop, run_stop_for_update, run_swap, + run_emit_rebuilt, run_knowledge_pull, run_meta_sync, run_pause_drain, run_pause_signal, + run_prebuild, run_provision, run_purge_state, run_rebuild_bookkeeping, run_reconcile, + run_set_wanted, run_signal, run_start, run_stop, run_stop_for_update, run_swap, run_wanted_pull, run_write_dropin, }; use crate::coordinator::{AgentPaths, Coordinator, HiveEnv}; @@ -254,10 +254,6 @@ impl Host for FakeHost { self.purge.clone() } - async fn forge_ensure_all(&self) { - self.record("forge_ensure_all".to_owned()); - } - async fn knowledge_pull(&self, _coord: &Arc) -> anyhow::Result<()> { if self.hang { std::future::pending::<()>().await; @@ -1131,13 +1127,6 @@ async fn emit_rebuilt_reports_success_and_failure_under_one_key() { // --- Sweeps and pulls --------------------------------------------------------- -#[tokio::test] -async fn forge_sweep_runs_once_and_never_fails() { - let host = FakeHost::up(); - run_forge_sweep(&host).await.expect("ok"); - assert_eq!(host.calls(), ["forge_ensure_all"]); -} - #[tokio::test] async fn knowledge_pull_passes_the_result_through_and_times_out() { let (_dir, coord) = coordinator(); @@ -1305,7 +1294,6 @@ async fn dispatch_routes_each_host_backed_kind_to_its_executor() { async fn dispatch_routes_the_agentless_and_hostless_kinds() { let host = FakeHost::up(); for kind in [ - NodeKind::ForgeSweep, NodeKind::WantedPull, NodeKind::KnowledgePull, alice(|agent| NodeKind::AgentWindow { agent }), @@ -1325,7 +1313,7 @@ async fn dispatch_routes_the_agentless_and_hostless_kinds() { ); } let calls = host.calls(); - for want in ["forge_ensure_all", "wanted_pull", "knowledge_pull"] { + for want in ["wanted_pull", "knowledge_pull"] { assert!(calls.iter().any(|c| c == want), "{want} in {calls:?}"); } } diff --git a/hive-c0re/src/job_queue/model.rs b/hive-c0re/src/job_queue/model.rs index 8eb0fe0d..6c3e67e5 100644 --- a/hive-c0re/src/job_queue/model.rs +++ b/hive-c0re/src/job_queue/model.rs @@ -96,9 +96,6 @@ pub enum NodeKind { /// Write the agent's durable power intent (`wanted = Up` when `up`, /// else `Offline`) as the head node of a power-op DAG. SetWanted { agent: String, up: bool }, - /// One-shot boot-time publish of the hive's forge read token for nix - /// (`forge::ensure_all`). Agentless. - ForgeSweep, /// `/knowledge` pull (`knowledge::pull`): at boot, on the swarm /// knowledge-changed event, and hourly. Agentless. KnowledgePull, @@ -165,7 +162,7 @@ impl NodeKind { | NodeKind::AgentWindow { agent } | NodeKind::EmitRebuilt { agent, .. } | NodeKind::SetWanted { agent, .. } => agent, - NodeKind::ForgeSweep | NodeKind::KnowledgePull | NodeKind::WantedPull => "", + NodeKind::KnowledgePull | NodeKind::WantedPull => "", } } diff --git a/hive-c0re/src/lifecycle/git.rs b/hive-c0re/src/lifecycle/git.rs index 7d97cfef..c2cbb406 100644 --- a/hive-c0re/src/lifecycle/git.rs +++ b/hive-c0re/src/lifecycle/git.rs @@ -10,9 +10,8 @@ use anyhow::{Context, Result, bail}; use tokio::io::{AsyncRead, AsyncReadExt}; use tokio::process::Command; -/// Deadline for a child that talks to a remote (forge fetch/push, `nix flake -/// update`/`lock`): a healthy one takes seconds, so ten minutes only ever -/// fires on a hang. +/// Deadline for a child that talks to a remote (forge fetch/push): a healthy +/// one takes seconds, so ten minutes only ever fires on a hang. pub const FETCH_DEADLINE: Duration = Duration::from_mins(10); /// Deadline for a git child that touches only the local repo. @@ -21,7 +20,7 @@ pub const GIT_LOCAL_DEADLINE: Duration = Duration::from_mins(2); /// Run `cmd` to completion and capture its output, failing once `deadline` /// passes. `what` names the command in the error. The child leads its own /// process group and the whole group is `SIGKILL`ed on timeout, so a -/// grandchild (a `git` or credential helper under `nix`) dies with it instead +/// grandchild (a `git` or credential helper it spawned) dies with it instead /// of outliving the lock. pub async fn run_with_deadline( cmd: &mut Command, diff --git a/hive-c0re/src/main.rs b/hive-c0re/src/main.rs index c8568b9d..231900f9 100644 --- a/hive-c0re/src/main.rs +++ b/hive-c0re/src/main.rs @@ -259,10 +259,6 @@ async fn cmd_serve( tracing::warn!(error = ?e, "auto-update task failed"); } }); - // Forge credential publish: a `NodeKind::ForgeSweep` DAG node (see - // `workers::auto_update::submit_startup_sweep_nodes`), submitted - // unconditionally on every boot — moved off a bare `tokio::spawn` so it - // shows as real work on the dashboard. // Knowledge periodic pull: hourly fallback in case a knowledge event from // the swarm is missed (e.g. hive-c0re was down when it was sent). Sleeps // before its first pull: the boot `NodeKind::KnowledgePull` DAG node diff --git a/hive-c0re/src/paths.rs b/hive-c0re/src/paths.rs index 246619a8..11aed6b1 100644 --- a/hive-c0re/src/paths.rs +++ b/hive-c0re/src/paths.rs @@ -196,13 +196,6 @@ pub fn gateway_agents_conf() -> PathBuf { gateway_dir().join("agents.conf") } -/// `forge-hive-token` — this hive's read-only forge token (login on the first -/// line, token on the second), copied from the swarm secret store for nix's -/// `git`. Tmpfs: re-published on every boot. -// nix: read by `git-credential-hive-forge`; the same literal is in -// `nix/host-modules/lib/hive-forge-git.nix` — must match. -pub const FORGE_HIVE_TOKEN: &str = "/run/hyperhive/forge-hive-token"; - /// `attic/` — the swarm cache's pull credentials for the probe: `netrc`, `substituter` /// and `public-key`, each `0600`, copied from the swarm secret store. Tmpfs: /// re-published on every boot. diff --git a/hive-c0re/src/stats/warnings.rs b/hive-c0re/src/stats/warnings.rs index dca7cf8f..066f331b 100644 --- a/hive-c0re/src/stats/warnings.rs +++ b/hive-c0re/src/stats/warnings.rs @@ -172,9 +172,8 @@ pub fn snapshot() -> Vec { /// Raise a warning for a one-shot boot-time step that has no periodic /// retry to observe a later success and clear the warning via -/// [`WarningGuard`] drop — e.g. a step inside `forge::ensure_all()`, -/// which `tokio::spawn`s once at hive-c0re startup and never runs again -/// this process. Intentionally **leaks** the guard for the life of the +/// [`WarningGuard`] drop — e.g. a step that runs once at hive-c0re startup +/// and never again this process. Intentionally **leaks** the guard for the life of the /// process: the warning clears the next time hive-c0re restarts (a fresh /// process starts with an empty registry) and re-runs the step, which is /// exactly when a config/environment fix would take effect anyway. diff --git a/hive-c0re/src/workers/auto_update.rs b/hive-c0re/src/workers/auto_update.rs index 43cd54a5..0807b7f5 100644 --- a/hive-c0re/src/workers/auto_update.rs +++ b/hive-c0re/src/workers/auto_update.rs @@ -196,13 +196,13 @@ fn boot_action(wanted: Option, fresh: bool, running: bool) } } -/// Submit the boot-time forge/knowledge/wanted-state sweeps as DAG -/// nodes — `ForgeSweep`, `KnowledgePull`, `WantedPull`. Unlike +/// Submit the boot-time knowledge and wanted-state sweeps as DAG +/// nodes — `KnowledgePull`, `WantedPull`. Unlike /// [`submit_boot_tree`] this runs on **every** boot, quiet or not: these /// aren't config-drift work, they're startup housekeeping that always needs /// to happen, and the point of moving them here is exactly so they show up /// as real work on the dashboard instead of an invisible `tokio::spawn` that -/// only surfaces on failure. Three independent, build-slot- and lease-exempt +/// only surfaces on failure. Independent, build-slot- and lease-exempt /// roots — no dependency edges between them, matching the existing /// `Reconcile`-root pattern in [`boot_nodes`]. `KnowledgePull` holds its /// sweep's own resource, so it queues behind any pass of the same sweep @@ -211,7 +211,6 @@ fn submit_startup_sweep_nodes(coord: &Arc) { use crate::job_queue::NodeKind; if let Err(e) = coord.job_queue.insert_job(|b| { - let _ = b.node(NodeKind::ForgeSweep); let _ = crate::job_queue::templates::knowledge_pull(b); let _ = b.node(NodeKind::WantedPull); Vec::new() diff --git a/nix/host-modules/hive-c0re/default.nix b/nix/host-modules/hive-c0re/default.nix index 6cd79d13..7441493f 100644 --- a/nix/host-modules/hive-c0re/default.nix +++ b/nix/host-modules/hive-c0re/default.nix @@ -35,26 +35,18 @@ let gatewayCfg = config.services.hyperhive.gateway; }; - forgeGit = import ../lib/hive-forge-git.nix { - inherit pkgs; - forgeDomain = config.services.hyperhive.swarm.forge.domain; - }; - # Privsep splits ownership across users, so git/libgit2's dubious- # ownership guard trips on legitimate cross-user reads: hive-c0re # (hive-core) fetches the agent-owned proposed-config repos. The process is # trusted and can already read the files; this gitconfig only satisfies the # ownership guard. libgit2 honours the literal `*` (mid-path globs aren't # supported, so per-agent repos can't be listed); in practice this process - # only ever touches hyperhive's own repos. - # - # The forge sections (`forgeGit.config`) answer for the `git+http` forge - # fetches nix makes: the agent config inputs and `internal/meta`. hive-c0re's - # own forge reads use `forge::reader_token()` instead. + # only ever touches hyperhive's own repos. Its forge reads authenticate with + # `forge::reader_token()`, handed to git per command, so no credential helper + # is configured. safeDirGitconfig = pkgs.writeText "hyperhive-safe-gitconfig" '' [safe] directory = * - ${forgeGit.config} [http] # Abort a stalled fetch instead of blocking startup indefinitely. If a # git http transfer from the forge (e.g. filling an applied mirror) @@ -251,9 +243,6 @@ in after = [ "hive-c0re.socket" ]; path = [ pkgs.git - # `git-credential-hive-forge` on PATH so git finds it for git+http - # fetches from the forge (helper = hive-forge). - forgeGit.credentialHelper "/run/current-system/sw" ]; environment = import ./environment.nix { inherit lib config pkgs; }; diff --git a/nix/host-modules/hive-c0re/environment.nix b/nix/host-modules/hive-c0re/environment.nix index bad8fc36..1d4976b3 100644 --- a/nix/host-modules/hive-c0re/environment.nix +++ b/nix/host-modules/hive-c0re/environment.nix @@ -47,8 +47,8 @@ in HOME = "/var/lib/hyperhive"; HYPERHIVE_GIT = "${pkgs.git}/bin/git"; # Never let git block on an interactive credential prompt. hive-core is a - # TTY-less system user, so a prompt (e.g. the forge credential helper - # returns nothing because the forge isn't reachable yet on cold boot) + # TTY-less system user, so a prompt (e.g. git asking for credentials + # because the forge isn't reachable yet on cold boot) # would hang forever and freeze the daemon during startup. With this set, # git fails fast instead of prompting. Paired with the git http # low-speed abort in `safeDirGitconfig` (bounds a stalled transfer) and diff --git a/nix/host-modules/lib/hive-forge-git.nix b/nix/host-modules/lib/hive-forge-git.nix deleted file mode 100644 index d85d31f1..00000000 --- a/nix/host-modules/lib/hive-forge-git.nix +++ /dev/null @@ -1,45 +0,0 @@ -# Git settings for the host processes that hand nix a forge URL: hive-c0re -# (prebuilds as `hive-core`) and hive-priv (builds as root, with a HOME of its -# own), each through its own `$HOME/.gitconfig`. nix runs `git` for a `git+…` -# input, so git's config is the only credential and URL plumbing it has. -# -# - `credential`: the forge host answers through `git-credential-hive-forge`, -# which reads the hive's own forge account and token from the file hive-c0re -# publishes. An empty `helper =` first resets the list, so no /etc/gitconfig -# helper (e.g. `oauth`, which waits on a browser) runs ahead of ours. -# - `url`: the swarm's `internal/meta` lock names its inputs by the forge's -# `https` URL, which only the builder's credential helper answers for. They -# are fetched over the `http` URL the helper above is scoped to. -# -# Pure function — NOT a NixOS module (don't add it to the host-modules -# aggregator). Call it from a module's `let`: -# -# forgeGit = import ./lib/hive-forge-git.nix { inherit pkgs; forgeDomain = …; }; -# # then put `forgeGit.credentialHelper` on the unit's `path` and -# # `forgeGit.config` into its gitconfig. -{ pkgs, forgeDomain }: -{ - # Answers only `get`. The file (`/run/hyperhive/forge-hive-token`, hive-c0re's - # `paths::FORGE_HIVE_TOKEN`) holds the account on its first line and the - # token on its second; it is read on every invocation, so it never holds a - # stale copy and survives token rotation. Prints nothing where the file is - # absent or incomplete. - credentialHelper = pkgs.writeShellScriptBin "git-credential-hive-forge" '' - [ "''${1:-}" = "get" ] || exit 0 - [ -r /run/hyperhive/forge-hive-token ] || exit 0 - { read -r login; read -r token; } < /run/hyperhive/forge-hive-token || true - [ -n "$login" ] && [ -n "$token" ] || exit 0 - printf 'username=%s\n' "$login" - printf 'password=%s\n' "$token" - ''; - - config = '' - [credential] - helper = - [credential "http://${forgeDomain}"] - helper = - helper = hive-forge - [url "http://${forgeDomain}/"] - insteadOf = https://${forgeDomain}/ - ''; -} diff --git a/nix/module-eval/core-toggle.nix b/nix/module-eval/core-toggle.nix index 3c6bb9a5..745260b9 100644 --- a/nix/module-eval/core-toggle.nix +++ b/nix/module-eval/core-toggle.nix @@ -234,39 +234,25 @@ let && env shared.controllerAlone == { }; } { - # hive-c0re carries the credential helper and the https-to-http rewrite as - # git config. hive-priv builds nothing and fetches nothing from the forge, - # so it carries neither and has no forge URL. The hive-c0re arm is the - # control that the lookups find a helper and a forge URL where there is - # one, so the absences on hive-priv and the controller are real. - name = "hive-c0re answers the forge's git fetches and rewrites its https inputs to http; hive-priv has neither"; + # No host process hands nix a forge URL any more, so none carries a git + # credential helper for the forge. The hive-c0re arm is the control: its + # `git` and its forge URL are found by the same lookups, so the absences + # are real. + name = "no host unit carries the forge credential helper, and hive-priv has no forge URL"; ok = let - forge = allLocal.services.hyperhive.swarm.forge.domain; - git = import ../host-modules/lib/hive-forge-git.nix { - inherit pkgs; - forgeDomain = forge; - }; - helperOn = + pathNames = unit: - lib.any ( - p: builtins.isAttrs p && (p.name or "") == "git-credential-hive-forge" - ) allLocal.systemd.services.${unit}.path; + map (p: if builtins.isAttrs p then p.name or "" else "") allLocal.systemd.services.${unit}.path; env = unit: allLocal.systemd.services.${unit}.environment; in - helperOn "hive-c0re" - && !(helperOn "hive-priv") - # The helper answers from the file hive-c0re publishes the hive's own - # token to. The first arm is the control for the second: the lookup - # finds the helper's text, so the absent `core` strings are really absent. - && lib.hasInfix "/run/hyperhive/forge-hive-token" git.credentialHelper.text - && !(lib.hasInfix "forge-core-token" git.credentialHelper.text) - && !(lib.hasInfix "username=core" git.credentialHelper.text) - && !(lib.hasInfix "username = core" git.config) - && !(helperOn "swarm-controller") - && lib.hasInfix "[credential \"http://${forge}\"]\nhelper =\nhelper = hive-forge" git.config - && lib.hasInfix "[url \"http://${forge}/\"]\ninsteadOf = https://${forge}/" git.config + lib.any (lib.hasPrefix "git-2") (pathNames "hive-c0re") && (env "hive-c0re") ? HIVE_FORGE_URL + && lib.all (unit: !(builtins.elem "git-credential-hive-forge" (pathNames unit))) [ + "hive-c0re" + "hive-priv" + "swarm-controller" + ] && !((env "hive-priv") ? HIVE_PRIV_FORGE_URL); } {