remove the hive-c0re forge-credential chain nothing reads any more
With no nix build on the hive, the boot publish of the forge token for nix has no reader. Removes the ForgeSweep node, the token file, the git credential helper and hive-c0re's forge gitconfig sections.
This commit is contained in:
atlas
parent
6b800600e2
commit
221c3f35a1
18 changed files with 69 additions and 310 deletions
|
|
@ -69,14 +69,11 @@ forge container:
|
|||
- seeding `proposed` from `agent-configs/<agent>` on first deploy
|
||||
- cloning and pulling `/knowledge`
|
||||
|
||||
The builds read as the hive's account too. nix fetches the config inputs and
|
||||
`internal/meta` with `git`, and `git` gets its credential from
|
||||
`git-credential-hive-forge`, which reads `/run/hyperhive/forge-hive-token`.
|
||||
hive-c0re writes that file (account on the first line, token on the second)
|
||||
from the hive's store token at each boot and each `internal/meta` read, when its
|
||||
content differs. A hive whose store holds no token has no such file, fetches
|
||||
nothing private through nix and fails its builds: they have no other flake than
|
||||
`internal/meta`. This also holds on a hive with no forge container.
|
||||
The hive builds nothing, so no nix process reads the forge as the hive's
|
||||
account. hive-c0re hands the token to each `git` command it runs
|
||||
(`http.extraHeader`) and keeps no copy on disk. A hive whose store holds no token
|
||||
reads nothing private from the forge. This also holds on a hive with no forge
|
||||
container.
|
||||
|
||||
Roll a change to any of this out store host first (its granter writes the
|
||||
controller's grant on `swarm/hives/+/forge/token`), then the controller host,
|
||||
|
|
|
|||
|
|
@ -72,7 +72,6 @@ Cheap — no build slot:
|
|||
| `SetWanted` | write the durable power intent (`wanted = Up`/`Offline`) as the head node of a power-op DAG. Takes the agent lease even though it's a store write, so the intent write and the tail `Reconcile` are atomic per-agent — two racing power ops can't clobber each other's intent before either reconciles |
|
||||
| `EmitRebuilt` | tail of a rebuild — emit the agent's `Rebuilt` manager event (ok/fail per outcome, nothing on cancel). One node per agent _and_ per outcome |
|
||||
| `WriteDropin` | `set_nspawn_flags` + `set_resource_limits` + daemon-reload |
|
||||
| `ForgeSweep` | one-shot boot-time publish of the hive's forge read token for nix (`forge::ensure_all`) as a first-class node, so it shows as real work on the dashboard instead of running invisibly in a bare `tokio::spawn`. Agentless |
|
||||
| `KnowledgePull` | `/knowledge` pull (`knowledge::pull`): at boot (commits that landed while `hive-c0re` was down), on the swarm knowledge-changed event, and hourly as a fallback. Holds `Resource::KnowledgeTree` (capacity 1), so two pulls never overlap on the working tree; each trigger queues its own pass, which waits for the resource if one is already live. Agentless |
|
||||
| `WantedPull` | pull of the agent set the swarm controller declares for this hive (`wanted::pull`), converging the agents it names. Runs at boot, waiting up to a minute in total for the queue connection, and again after every attach of the wanted-state watch (which retries with a backoff when the attach fails or the stream ends); the deploy event (`swarm_status`) and the watch are the fast path, this repairs a missed one. A first deploy already queued or running for an agent isn't queued again. Agentless |
|
||||
|
||||
|
|
|
|||
|
|
@ -156,7 +156,7 @@ pieces fit, and the reasoning behind them, is in [`credentials.md`](credentials.
|
|||
| the cache's public key and pull token, for hive-priv | `swarm-controller` copies them to `swarm/hives/<hive>/attic/{public-key,pull-token}` (see the cache-facts row above) | `hive-priv-cache-facts` (root) reads those two under the hive's own store identity every ten minutes and writes `/run/hyperhive-priv/attic/public-key` and `netrc` (tmpfs, root, each `0600`, directory `0700`); the cache's address comes from `services.hyperhive.swarm.attic.domain` and `cache`, not from the store. A store that holds no facts, or that doesn't answer, leaves the files as they stand, and a hive that never got them has `hive-priv` answer `not available`. |
|
||||
| the forge's copy of its OIDC secret | `swarm-bao-forge-oidc.service` reads it out of the swarm secret store under the forge publisher's leaf, wherever authelia runs | `/run/hive-forge-oidc/<id>.secret` on the host (tmpfs), bound into the forge container, handed to `forgejo-sso-source` through `LoadCredential` |
|
||||
| the homeserver's copy of its OIDC secret | `swarm-bao-matrix-oidc.service` reads it out of the swarm secret store under matrix-ctl's leaf, wherever authelia runs | `/run/hive-matrix-oidc/<id>.secret` on the host (tmpfs), bound into the homeserver container, handed to tuwunel through `LoadCredential` |
|
||||
| this hive's forge read token | `swarm-controller`, into the secret store at `swarm/hives/<hive>/forge/token`, for the `hive-<hive>` account; a pass at start and every five minutes re-mints a missing or stale token | `hive-c0re` reads it under the hive's own store identity, and copies it (account on the first line, token on the second) to `/run/hyperhive/forge-hive-token` (tmpfs, `0600` `hive-core`) for nix's `git`, which `hive-priv` also runs. Each `internal/meta` read and each boot rewrites the copy when it differs; where they differ after a rotation, a build that starts before the next read fails to authenticate, and a retry reads the new token |
|
||||
| this hive's forge read token | `swarm-controller`, into the secret store at `swarm/hives/<hive>/forge/token`, for the `hive-<hive>` account; a pass at start and every five minutes re-mints a missing or stale token | `hive-c0re` reads it under the hive's own store identity for each forge read, and hands it to the `git` commands it runs. It keeps no copy on disk |
|
||||
| squid's users file | `hive-c0re` (`workers::squid_users`), every 30 seconds, from the proxy credentials of its agents and the self-check | `services.hyperhive.deploy.hive-controller.squid.usersFile` (`/var/lib/hive-squid/users`, `hive-core:squid`, `0640`): `<user>:<hash>` lines, SHA-512 crypt hashes only, renamed into place when it changed. `unrestricted` beside it lists the users whose agent holds the swarm's `unrestricted_internet` flag, names only; it lists every agent the hive never read as off `systemd-tmpfiles` creates it empty so squid's helper never sees a missing file. A hive whose agents have no credential yet (an older controller) has an empty file: every login fails and the self-check reports `ok=false`; an agent without one sends none, so it goes direct |
|
||||
| the squid self-check's password | `swarm-controller` mints `swarm/hives/<hive>/squid-selfcheck`; `squid-selfcheck-credential` (root, every 10 minutes) reads it as the hive | `/run/credstore/hive-squid.selfcheck` (tmpfs, root, `0400`), imported by `squid-selfcheck.service` as a systemd credential and handed to `curl` on its stdin. Both self-check users (`selfcheck.squid` and the always-unrestricted `selfcheck-unrestricted.squid`) use it, so whoever holds it has unrestricted egress through this squid Missing, or different from the users file: the self-check logs `credential=absent` or a 407 and fails, and nothing else breaks |
|
||||
|
||||
|
|
|
|||
|
|
@ -253,10 +253,10 @@ for every operation that genuinely requires root.
|
|||
known operations; there is no arbitrary command pass-through:
|
||||
|
||||
| Operation | What it runs |
|
||||
| ------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| ------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `StartContainer` / `StopContainer` | `nixos-container start/stop <name>` |
|
||||
| `KillContainer` | `machinectl kill <machine> --signal=SIGKILL` (`nixos-container` has no kill verb) |
|
||||
| `CreateContainer` / `UpdateContainer` | `nixos-container create <name> --system-path <toplevel>`; for an update, `nix-env -p <profile> --set <toplevel>` and a reload of the unit. `<toplevel>` must be the store path the last successful `RealiseClosure` returned for that agent (in memory, per agent), so hive-c0re can't name a path never realised for the agent; hive-priv builds nothing |
|
||||
| `CreateContainer` / `UpdateContainer` | `nixos-container create <name> --system-path <toplevel>`; for an update, `nix-env -p <profile> --set <toplevel>` and a reload of the unit. `<toplevel>` must be the store path the last successful `RealiseClosure` returned for that agent (in memory, per agent), so hive-c0re can't name a path never realised for the agent; hive-priv builds nothing. The binding doesn't mean hive-c0re can't choose the closure: it can still ask hive-priv to download any swarm-signed path for any agent and then apply it. The swarm key and the cache bound that to a cross-assignment risk (one agent gets another's closure), not an arbitrary-path one |
|
||||
| `RealiseClosure` | `nix-store --realise <store_path> --max-jobs 0` against the swarm cache, with the host's configured cache address, and the public key and netrc from root-owned `/run/hyperhive-priv/attic/`, as per-call settings that replace the host's substituters and trusted keys and require a signature. The path must be `/nix/store/<32 chars>-<name>` and the agent name valid. It downloads and never builds |
|
||||
| `DestroyContainer` | `nixos-container destroy <name>` |
|
||||
| `ListContainers` | `nixos-container list` |
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
//! The hive's forge access: reads of the forge go through [`reader_token`] on
|
||||
//! every hive, and [`ensure_all`] publishes that token for nix's `git`. The
|
||||
//! every hive. The
|
||||
//! swarm-wide orgs, teams and repos, and each agent's read grant on
|
||||
//! `internal/docs`, are swarm-controller's. Full design:
|
||||
//! `docs/integrations/forge.md`.
|
||||
|
|
@ -10,7 +10,7 @@ mod reconcile;
|
|||
mod repos;
|
||||
|
||||
pub(crate) use internal_meta::internal_meta_source;
|
||||
pub use reader::{publish_nix_credential, reader_token};
|
||||
pub use reader::reader_token;
|
||||
pub use reconcile::{fetch_forge_main, reconcile_config_apply, reconcile_config_status};
|
||||
pub use repos::{clone_config_into_proposed, fetch_config_main_into_applied};
|
||||
|
||||
|
|
@ -145,16 +145,6 @@ where
|
|||
.map_err(|_| anyhow::anyhow!("{op}: no answer from the forge within {API_TIMEOUT:?}"))
|
||||
}
|
||||
|
||||
/// Publish this hive's forge read token for nix's `git` at startup, so a build
|
||||
/// that starts before any `internal/meta` read still has a credential. Every
|
||||
/// hive runs it: the token is the hive's own, in the store, whether or not a
|
||||
/// forge container is local.
|
||||
pub async fn ensure_all() {
|
||||
if publish_nix_credential().await {
|
||||
tracing::debug!("forge: nix's forge credential is current");
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::{API_TIMEOUT, bounded, git_url_with_base};
|
||||
|
|
|
|||
|
|
@ -1,14 +1,6 @@
|
|||
//! The credential this hive reads the forge with: its own read-only account,
|
||||
//! whose token `swarm-controller` mints into the store at
|
||||
//! `swarm/hives/<hive>/forge/token`.
|
||||
//!
|
||||
//! nix's `git` cannot reach the store, so [`ReaderToken::publish_for_nix`]
|
||||
//! copies the token to a runtime file that `git-credential-hive-forge`
|
||||
//! (`nix/host-modules/lib/hive-forge-git.nix`) reads.
|
||||
|
||||
use std::path::Path;
|
||||
|
||||
use anyhow::Result;
|
||||
|
||||
use super::basic_auth_header;
|
||||
|
||||
|
|
@ -39,30 +31,6 @@ impl ReaderToken {
|
|||
pub(crate) fn api(&self) -> anyhow::Result<forgejo_api::Forgejo> {
|
||||
super::api(&self.value)
|
||||
}
|
||||
|
||||
/// Copy this token to the file nix's `git` authenticates from
|
||||
/// ([`crate::paths::FORGE_HIVE_TOKEN`]). Whether the file now holds it; a
|
||||
/// failed write is logged and leaves the file as it was.
|
||||
pub(crate) fn publish_for_nix(&self) -> bool {
|
||||
let path = Path::new(crate::paths::FORGE_HIVE_TOKEN);
|
||||
match self.write_credential(path) {
|
||||
Ok(()) => true,
|
||||
Err(e) => {
|
||||
tracing::warn!(path = %path.display(), error = ?e, "forge: could not write the nix forge credential");
|
||||
false
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Login on the first line, token on the second. Written only when the
|
||||
/// content differs, so a build's resolve is not a disk write.
|
||||
fn write_credential(&self, path: &Path) -> Result<()> {
|
||||
let content = format!("{}\n{}\n", self.login, self.value);
|
||||
if std::fs::read(path).is_ok_and(|have| have == content.as_bytes()) {
|
||||
return Ok(());
|
||||
}
|
||||
crate::secret_file::write_secret_0600(path, content.as_bytes())
|
||||
}
|
||||
}
|
||||
|
||||
/// The token to read the forge with, or `None` when the store holds none for
|
||||
|
|
@ -77,16 +45,6 @@ pub async fn reader_token() -> Option<ReaderToken> {
|
|||
token
|
||||
}
|
||||
|
||||
/// Publish this hive's read token for nix; see [`ReaderToken::publish_for_nix`].
|
||||
/// Whether the file holds a token.
|
||||
pub async fn publish_nix_credential() -> bool {
|
||||
let Some(token) = reader_token().await else {
|
||||
tracing::debug!("forge: no hive read token in the store; nix has no forge credential");
|
||||
return false;
|
||||
};
|
||||
token.publish_for_nix()
|
||||
}
|
||||
|
||||
/// This hive's token from the store, under the hive's own identity.
|
||||
///
|
||||
/// `None`, never an error, for every way it can come up empty — no hive name,
|
||||
|
|
@ -131,8 +89,6 @@ async fn stored_reader_token() -> Option<ReaderToken> {
|
|||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use std::os::unix::fs::PermissionsExt as _;
|
||||
|
||||
fn stored() -> ReaderToken {
|
||||
ReaderToken {
|
||||
login: "hive-alpha".to_owned(),
|
||||
|
|
@ -140,73 +96,6 @@ mod tests {
|
|||
}
|
||||
}
|
||||
|
||||
fn mode(path: &Path) -> u32 {
|
||||
std::fs::metadata(path).expect("stat").permissions().mode() & 0o777
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_nix_credential_is_the_hives_login_then_its_token_at_0600() {
|
||||
let dir = tempfile::tempdir().expect("tempdir");
|
||||
let path = dir.path().join("forge-hive-token");
|
||||
stored().write_credential(&path).expect("written");
|
||||
assert_eq!(
|
||||
std::fs::read_to_string(&path).expect("read"),
|
||||
"hive-alpha\nfrom-the-store\n"
|
||||
);
|
||||
assert_eq!(mode(&path), 0o600);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_rotated_token_replaces_the_nix_credential() {
|
||||
let dir = tempfile::tempdir().expect("tempdir");
|
||||
let path = dir.path().join("forge-hive-token");
|
||||
stored().write_credential(&path).expect("first");
|
||||
let rotated = ReaderToken {
|
||||
login: "hive-alpha".to_owned(),
|
||||
value: "rotated".to_owned(),
|
||||
};
|
||||
rotated.write_credential(&path).expect("second");
|
||||
assert_eq!(
|
||||
std::fs::read_to_string(&path).expect("read"),
|
||||
"hive-alpha\nrotated\n"
|
||||
);
|
||||
assert_eq!(mode(&path), 0o600);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_unchanged_credential_is_not_rewritten() {
|
||||
let dir = tempfile::tempdir().expect("tempdir");
|
||||
let path = dir.path().join("forge-hive-token");
|
||||
stored().write_credential(&path).expect("first");
|
||||
// A rewrite goes through a sibling temp file and a rename, which
|
||||
// gives the file a new inode.
|
||||
let before = std::fs::metadata(&path).expect("stat");
|
||||
stored().write_credential(&path).expect("second");
|
||||
let after = std::fs::metadata(&path).expect("stat");
|
||||
assert_eq!(
|
||||
std::os::unix::fs::MetadataExt::ino(&before),
|
||||
std::os::unix::fs::MetadataExt::ino(&after)
|
||||
);
|
||||
}
|
||||
|
||||
/// `hive-forge-git.nix`'s credential helper names this path as a literal.
|
||||
#[test]
|
||||
fn the_nix_credential_lives_on_the_runtime_tmpfs_where_the_helper_reads_it() {
|
||||
assert_eq!(
|
||||
crate::paths::FORGE_HIVE_TOKEN,
|
||||
"/run/hyperhive/forge-hive-token"
|
||||
);
|
||||
assert!(crate::paths::FORGE_HIVE_TOKEN.starts_with(crate::paths::RUNTIME_ROOT));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_path_that_cannot_be_written_is_an_error_not_a_panic() {
|
||||
let dir = tempfile::tempdir().expect("tempdir");
|
||||
let file = dir.path().join("a-file");
|
||||
std::fs::write(&file, "").expect("file");
|
||||
assert!(stored().write_credential(&file.join("sub")).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_header_names_the_tokens_own_account() {
|
||||
use base64::Engine as _;
|
||||
|
|
|
|||
|
|
@ -99,8 +99,6 @@ pub(super) trait Host: Sync {
|
|||
async fn delete_state_subvolume(&self, name: &str) -> Result<()>;
|
||||
/// The persistent directories a purge removes.
|
||||
fn purge_dirs(&self, name: &str) -> Vec<PathBuf>;
|
||||
/// Publish the hive's forge credential for nix.
|
||||
async fn forge_ensure_all(&self);
|
||||
/// One `/knowledge` pull.
|
||||
async fn knowledge_pull(&self, coord: &Arc<Coordinator>) -> Result<()>;
|
||||
/// One swarm wanted-state pull.
|
||||
|
|
@ -228,10 +226,6 @@ impl Host for RealHost {
|
|||
.collect()
|
||||
}
|
||||
|
||||
async fn forge_ensure_all(&self) {
|
||||
crate::forge::ensure_all().await;
|
||||
}
|
||||
|
||||
async fn knowledge_pull(&self, coord: &Arc<Coordinator>) -> Result<()> {
|
||||
crate::workers::knowledge::pull(coord).await
|
||||
}
|
||||
|
|
@ -332,23 +326,12 @@ async fn dispatch(
|
|||
// `Finishing` so the nodes under it start. What they declare stays held
|
||||
// until their whole subtree settles.
|
||||
NodeKind::AgentWindow { .. } => Ok(()),
|
||||
NodeKind::ForgeSweep => run_forge_sweep(host).await,
|
||||
NodeKind::KnowledgePull => run_knowledge_pull(host, coord, KNOWLEDGE_PULL_DEADLINE).await,
|
||||
NodeKind::WantedPull => run_wanted_pull(host, coord).await,
|
||||
};
|
||||
(builder, result)
|
||||
}
|
||||
|
||||
/// Boot-time publish of the hive's forge credential for nix, as a DAG node —
|
||||
/// see [`NodeKind::ForgeSweep`]. `forge::ensure_all` logs its own outcome and
|
||||
/// is best-effort (a build re-publishes the credential before it fetches), so
|
||||
/// this wrapper has nothing left to report; it exists to make the step a
|
||||
/// visible unit of work.
|
||||
async fn run_forge_sweep(host: &impl Host) -> Result<()> {
|
||||
host.forge_ensure_all().await;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// `/knowledge` pull as a DAG node — see [`NodeKind::KnowledgePull`]. Every
|
||||
/// pass reports here, whoever submitted it: as the node's own error on the
|
||||
/// dashboard, and into the debounced warning.
|
||||
|
|
|
|||
|
|
@ -12,9 +12,9 @@ use hive_jobq::scheduler::Scheduler;
|
|||
|
||||
use super::{
|
||||
Host, dispatch, run_create, run_destroy_bookkeeping, run_destroy_container, run_drain,
|
||||
run_emit_rebuilt, run_forge_sweep, run_knowledge_pull, run_meta_sync, run_pause_drain,
|
||||
run_pause_signal, run_prebuild, run_provision, run_purge_state, run_rebuild_bookkeeping,
|
||||
run_reconcile, run_set_wanted, run_signal, run_start, run_stop, run_stop_for_update, run_swap,
|
||||
run_emit_rebuilt, run_knowledge_pull, run_meta_sync, run_pause_drain, run_pause_signal,
|
||||
run_prebuild, run_provision, run_purge_state, run_rebuild_bookkeeping, run_reconcile,
|
||||
run_set_wanted, run_signal, run_start, run_stop, run_stop_for_update, run_swap,
|
||||
run_wanted_pull, run_write_dropin,
|
||||
};
|
||||
use crate::coordinator::{AgentPaths, Coordinator, HiveEnv};
|
||||
|
|
@ -254,10 +254,6 @@ impl Host for FakeHost {
|
|||
self.purge.clone()
|
||||
}
|
||||
|
||||
async fn forge_ensure_all(&self) {
|
||||
self.record("forge_ensure_all".to_owned());
|
||||
}
|
||||
|
||||
async fn knowledge_pull(&self, _coord: &Arc<Coordinator>) -> anyhow::Result<()> {
|
||||
if self.hang {
|
||||
std::future::pending::<()>().await;
|
||||
|
|
@ -1131,13 +1127,6 @@ async fn emit_rebuilt_reports_success_and_failure_under_one_key() {
|
|||
|
||||
// --- Sweeps and pulls ---------------------------------------------------------
|
||||
|
||||
#[tokio::test]
|
||||
async fn forge_sweep_runs_once_and_never_fails() {
|
||||
let host = FakeHost::up();
|
||||
run_forge_sweep(&host).await.expect("ok");
|
||||
assert_eq!(host.calls(), ["forge_ensure_all"]);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn knowledge_pull_passes_the_result_through_and_times_out() {
|
||||
let (_dir, coord) = coordinator();
|
||||
|
|
@ -1305,7 +1294,6 @@ async fn dispatch_routes_each_host_backed_kind_to_its_executor() {
|
|||
async fn dispatch_routes_the_agentless_and_hostless_kinds() {
|
||||
let host = FakeHost::up();
|
||||
for kind in [
|
||||
NodeKind::ForgeSweep,
|
||||
NodeKind::WantedPull,
|
||||
NodeKind::KnowledgePull,
|
||||
alice(|agent| NodeKind::AgentWindow { agent }),
|
||||
|
|
@ -1325,7 +1313,7 @@ async fn dispatch_routes_the_agentless_and_hostless_kinds() {
|
|||
);
|
||||
}
|
||||
let calls = host.calls();
|
||||
for want in ["forge_ensure_all", "wanted_pull", "knowledge_pull"] {
|
||||
for want in ["wanted_pull", "knowledge_pull"] {
|
||||
assert!(calls.iter().any(|c| c == want), "{want} in {calls:?}");
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -96,9 +96,6 @@ pub enum NodeKind {
|
|||
/// Write the agent's durable power intent (`wanted = Up` when `up`,
|
||||
/// else `Offline`) as the head node of a power-op DAG.
|
||||
SetWanted { agent: String, up: bool },
|
||||
/// One-shot boot-time publish of the hive's forge read token for nix
|
||||
/// (`forge::ensure_all`). Agentless.
|
||||
ForgeSweep,
|
||||
/// `/knowledge` pull (`knowledge::pull`): at boot, on the swarm
|
||||
/// knowledge-changed event, and hourly. Agentless.
|
||||
KnowledgePull,
|
||||
|
|
@ -165,7 +162,7 @@ impl NodeKind {
|
|||
| NodeKind::AgentWindow { agent }
|
||||
| NodeKind::EmitRebuilt { agent, .. }
|
||||
| NodeKind::SetWanted { agent, .. } => agent,
|
||||
NodeKind::ForgeSweep | NodeKind::KnowledgePull | NodeKind::WantedPull => "",
|
||||
NodeKind::KnowledgePull | NodeKind::WantedPull => "",
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -10,9 +10,8 @@ use anyhow::{Context, Result, bail};
|
|||
use tokio::io::{AsyncRead, AsyncReadExt};
|
||||
use tokio::process::Command;
|
||||
|
||||
/// Deadline for a child that talks to a remote (forge fetch/push, `nix flake
|
||||
/// update`/`lock`): a healthy one takes seconds, so ten minutes only ever
|
||||
/// fires on a hang.
|
||||
/// Deadline for a child that talks to a remote (forge fetch/push): a healthy
|
||||
/// one takes seconds, so ten minutes only ever fires on a hang.
|
||||
pub const FETCH_DEADLINE: Duration = Duration::from_mins(10);
|
||||
|
||||
/// Deadline for a git child that touches only the local repo.
|
||||
|
|
@ -21,7 +20,7 @@ pub const GIT_LOCAL_DEADLINE: Duration = Duration::from_mins(2);
|
|||
/// Run `cmd` to completion and capture its output, failing once `deadline`
|
||||
/// passes. `what` names the command in the error. The child leads its own
|
||||
/// process group and the whole group is `SIGKILL`ed on timeout, so a
|
||||
/// grandchild (a `git` or credential helper under `nix`) dies with it instead
|
||||
/// grandchild (a `git` or credential helper it spawned) dies with it instead
|
||||
/// of outliving the lock.
|
||||
pub async fn run_with_deadline(
|
||||
cmd: &mut Command,
|
||||
|
|
|
|||
|
|
@ -259,10 +259,6 @@ async fn cmd_serve(
|
|||
tracing::warn!(error = ?e, "auto-update task failed");
|
||||
}
|
||||
});
|
||||
// Forge credential publish: a `NodeKind::ForgeSweep` DAG node (see
|
||||
// `workers::auto_update::submit_startup_sweep_nodes`), submitted
|
||||
// unconditionally on every boot — moved off a bare `tokio::spawn` so it
|
||||
// shows as real work on the dashboard.
|
||||
// Knowledge periodic pull: hourly fallback in case a knowledge event from
|
||||
// the swarm is missed (e.g. hive-c0re was down when it was sent). Sleeps
|
||||
// before its first pull: the boot `NodeKind::KnowledgePull` DAG node
|
||||
|
|
|
|||
|
|
@ -196,13 +196,6 @@ pub fn gateway_agents_conf() -> PathBuf {
|
|||
gateway_dir().join("agents.conf")
|
||||
}
|
||||
|
||||
/// `forge-hive-token` — this hive's read-only forge token (login on the first
|
||||
/// line, token on the second), copied from the swarm secret store for nix's
|
||||
/// `git`. Tmpfs: re-published on every boot.
|
||||
// nix: read by `git-credential-hive-forge`; the same literal is in
|
||||
// `nix/host-modules/lib/hive-forge-git.nix` — must match.
|
||||
pub const FORGE_HIVE_TOKEN: &str = "/run/hyperhive/forge-hive-token";
|
||||
|
||||
/// `attic/` — the swarm cache's pull credentials for the probe: `netrc`, `substituter`
|
||||
/// and `public-key`, each `0600`, copied from the swarm secret store. Tmpfs:
|
||||
/// re-published on every boot.
|
||||
|
|
|
|||
|
|
@ -172,9 +172,8 @@ pub fn snapshot() -> Vec<ServerWarning> {
|
|||
|
||||
/// Raise a warning for a one-shot boot-time step that has no periodic
|
||||
/// retry to observe a later success and clear the warning via
|
||||
/// [`WarningGuard`] drop — e.g. a step inside `forge::ensure_all()`,
|
||||
/// which `tokio::spawn`s once at hive-c0re startup and never runs again
|
||||
/// this process. Intentionally **leaks** the guard for the life of the
|
||||
/// [`WarningGuard`] drop — e.g. a step that runs once at hive-c0re startup
|
||||
/// and never again this process. Intentionally **leaks** the guard for the life of the
|
||||
/// process: the warning clears the next time hive-c0re restarts (a fresh
|
||||
/// process starts with an empty registry) and re-runs the step, which is
|
||||
/// exactly when a config/environment fix would take effect anyway.
|
||||
|
|
|
|||
|
|
@ -196,13 +196,13 @@ fn boot_action(wanted: Option<crate::power::Wanted>, fresh: bool, running: bool)
|
|||
}
|
||||
}
|
||||
|
||||
/// Submit the boot-time forge/knowledge/wanted-state sweeps as DAG
|
||||
/// nodes — `ForgeSweep`, `KnowledgePull`, `WantedPull`. Unlike
|
||||
/// Submit the boot-time knowledge and wanted-state sweeps as DAG
|
||||
/// nodes — `KnowledgePull`, `WantedPull`. Unlike
|
||||
/// [`submit_boot_tree`] this runs on **every** boot, quiet or not: these
|
||||
/// aren't config-drift work, they're startup housekeeping that always needs
|
||||
/// to happen, and the point of moving them here is exactly so they show up
|
||||
/// as real work on the dashboard instead of an invisible `tokio::spawn` that
|
||||
/// only surfaces on failure. Three independent, build-slot- and lease-exempt
|
||||
/// only surfaces on failure. Independent, build-slot- and lease-exempt
|
||||
/// roots — no dependency edges between them, matching the existing
|
||||
/// `Reconcile`-root pattern in [`boot_nodes`]. `KnowledgePull` holds its
|
||||
/// sweep's own resource, so it queues behind any pass of the same sweep
|
||||
|
|
@ -211,7 +211,6 @@ fn submit_startup_sweep_nodes(coord: &Arc<Coordinator>) {
|
|||
use crate::job_queue::NodeKind;
|
||||
|
||||
if let Err(e) = coord.job_queue.insert_job(|b| {
|
||||
let _ = b.node(NodeKind::ForgeSweep);
|
||||
let _ = crate::job_queue::templates::knowledge_pull(b);
|
||||
let _ = b.node(NodeKind::WantedPull);
|
||||
Vec::new()
|
||||
|
|
|
|||
|
|
@ -35,26 +35,18 @@ let
|
|||
gatewayCfg = config.services.hyperhive.gateway;
|
||||
};
|
||||
|
||||
forgeGit = import ../lib/hive-forge-git.nix {
|
||||
inherit pkgs;
|
||||
forgeDomain = config.services.hyperhive.swarm.forge.domain;
|
||||
};
|
||||
|
||||
# Privsep splits ownership across users, so git/libgit2's dubious-
|
||||
# ownership guard trips on legitimate cross-user reads: hive-c0re
|
||||
# (hive-core) fetches the agent-owned proposed-config repos. The process is
|
||||
# trusted and can already read the files; this gitconfig only satisfies the
|
||||
# ownership guard. libgit2 honours the literal `*` (mid-path globs aren't
|
||||
# supported, so per-agent repos can't be listed); in practice this process
|
||||
# only ever touches hyperhive's own repos.
|
||||
#
|
||||
# The forge sections (`forgeGit.config`) answer for the `git+http` forge
|
||||
# fetches nix makes: the agent config inputs and `internal/meta`. hive-c0re's
|
||||
# own forge reads use `forge::reader_token()` instead.
|
||||
# only ever touches hyperhive's own repos. Its forge reads authenticate with
|
||||
# `forge::reader_token()`, handed to git per command, so no credential helper
|
||||
# is configured.
|
||||
safeDirGitconfig = pkgs.writeText "hyperhive-safe-gitconfig" ''
|
||||
[safe]
|
||||
directory = *
|
||||
${forgeGit.config}
|
||||
[http]
|
||||
# Abort a stalled fetch instead of blocking startup indefinitely. If a
|
||||
# git http transfer from the forge (e.g. filling an applied mirror)
|
||||
|
|
@ -251,9 +243,6 @@ in
|
|||
after = [ "hive-c0re.socket" ];
|
||||
path = [
|
||||
pkgs.git
|
||||
# `git-credential-hive-forge` on PATH so git finds it for git+http
|
||||
# fetches from the forge (helper = hive-forge).
|
||||
forgeGit.credentialHelper
|
||||
"/run/current-system/sw"
|
||||
];
|
||||
environment = import ./environment.nix { inherit lib config pkgs; };
|
||||
|
|
|
|||
|
|
@ -47,8 +47,8 @@ in
|
|||
HOME = "/var/lib/hyperhive";
|
||||
HYPERHIVE_GIT = "${pkgs.git}/bin/git";
|
||||
# Never let git block on an interactive credential prompt. hive-core is a
|
||||
# TTY-less system user, so a prompt (e.g. the forge credential helper
|
||||
# returns nothing because the forge isn't reachable yet on cold boot)
|
||||
# TTY-less system user, so a prompt (e.g. git asking for credentials
|
||||
# because the forge isn't reachable yet on cold boot)
|
||||
# would hang forever and freeze the daemon during startup. With this set,
|
||||
# git fails fast instead of prompting. Paired with the git http
|
||||
# low-speed abort in `safeDirGitconfig` (bounds a stalled transfer) and
|
||||
|
|
|
|||
|
|
@ -1,45 +0,0 @@
|
|||
# Git settings for the host processes that hand nix a forge URL: hive-c0re
|
||||
# (prebuilds as `hive-core`) and hive-priv (builds as root, with a HOME of its
|
||||
# own), each through its own `$HOME/.gitconfig`. nix runs `git` for a `git+…`
|
||||
# input, so git's config is the only credential and URL plumbing it has.
|
||||
#
|
||||
# - `credential`: the forge host answers through `git-credential-hive-forge`,
|
||||
# which reads the hive's own forge account and token from the file hive-c0re
|
||||
# publishes. An empty `helper =` first resets the list, so no /etc/gitconfig
|
||||
# helper (e.g. `oauth`, which waits on a browser) runs ahead of ours.
|
||||
# - `url`: the swarm's `internal/meta` lock names its inputs by the forge's
|
||||
# `https` URL, which only the builder's credential helper answers for. They
|
||||
# are fetched over the `http` URL the helper above is scoped to.
|
||||
#
|
||||
# Pure function — NOT a NixOS module (don't add it to the host-modules
|
||||
# aggregator). Call it from a module's `let`:
|
||||
#
|
||||
# forgeGit = import ./lib/hive-forge-git.nix { inherit pkgs; forgeDomain = …; };
|
||||
# # then put `forgeGit.credentialHelper` on the unit's `path` and
|
||||
# # `forgeGit.config` into its gitconfig.
|
||||
{ pkgs, forgeDomain }:
|
||||
{
|
||||
# Answers only `get`. The file (`/run/hyperhive/forge-hive-token`, hive-c0re's
|
||||
# `paths::FORGE_HIVE_TOKEN`) holds the account on its first line and the
|
||||
# token on its second; it is read on every invocation, so it never holds a
|
||||
# stale copy and survives token rotation. Prints nothing where the file is
|
||||
# absent or incomplete.
|
||||
credentialHelper = pkgs.writeShellScriptBin "git-credential-hive-forge" ''
|
||||
[ "''${1:-}" = "get" ] || exit 0
|
||||
[ -r /run/hyperhive/forge-hive-token ] || exit 0
|
||||
{ read -r login; read -r token; } < /run/hyperhive/forge-hive-token || true
|
||||
[ -n "$login" ] && [ -n "$token" ] || exit 0
|
||||
printf 'username=%s\n' "$login"
|
||||
printf 'password=%s\n' "$token"
|
||||
'';
|
||||
|
||||
config = ''
|
||||
[credential]
|
||||
helper =
|
||||
[credential "http://${forgeDomain}"]
|
||||
helper =
|
||||
helper = hive-forge
|
||||
[url "http://${forgeDomain}/"]
|
||||
insteadOf = https://${forgeDomain}/
|
||||
'';
|
||||
}
|
||||
|
|
@ -234,39 +234,25 @@ let
|
|||
&& env shared.controllerAlone == { };
|
||||
}
|
||||
{
|
||||
# hive-c0re carries the credential helper and the https-to-http rewrite as
|
||||
# git config. hive-priv builds nothing and fetches nothing from the forge,
|
||||
# so it carries neither and has no forge URL. The hive-c0re arm is the
|
||||
# control that the lookups find a helper and a forge URL where there is
|
||||
# one, so the absences on hive-priv and the controller are real.
|
||||
name = "hive-c0re answers the forge's git fetches and rewrites its https inputs to http; hive-priv has neither";
|
||||
# No host process hands nix a forge URL any more, so none carries a git
|
||||
# credential helper for the forge. The hive-c0re arm is the control: its
|
||||
# `git` and its forge URL are found by the same lookups, so the absences
|
||||
# are real.
|
||||
name = "no host unit carries the forge credential helper, and hive-priv has no forge URL";
|
||||
ok =
|
||||
let
|
||||
forge = allLocal.services.hyperhive.swarm.forge.domain;
|
||||
git = import ../host-modules/lib/hive-forge-git.nix {
|
||||
inherit pkgs;
|
||||
forgeDomain = forge;
|
||||
};
|
||||
helperOn =
|
||||
pathNames =
|
||||
unit:
|
||||
lib.any (
|
||||
p: builtins.isAttrs p && (p.name or "") == "git-credential-hive-forge"
|
||||
) allLocal.systemd.services.${unit}.path;
|
||||
map (p: if builtins.isAttrs p then p.name or "" else "") allLocal.systemd.services.${unit}.path;
|
||||
env = unit: allLocal.systemd.services.${unit}.environment;
|
||||
in
|
||||
helperOn "hive-c0re"
|
||||
&& !(helperOn "hive-priv")
|
||||
# The helper answers from the file hive-c0re publishes the hive's own
|
||||
# token to. The first arm is the control for the second: the lookup
|
||||
# finds the helper's text, so the absent `core` strings are really absent.
|
||||
&& lib.hasInfix "/run/hyperhive/forge-hive-token" git.credentialHelper.text
|
||||
&& !(lib.hasInfix "forge-core-token" git.credentialHelper.text)
|
||||
&& !(lib.hasInfix "username=core" git.credentialHelper.text)
|
||||
&& !(lib.hasInfix "username = core" git.config)
|
||||
&& !(helperOn "swarm-controller")
|
||||
&& lib.hasInfix "[credential \"http://${forge}\"]\nhelper =\nhelper = hive-forge" git.config
|
||||
&& lib.hasInfix "[url \"http://${forge}/\"]\ninsteadOf = https://${forge}/" git.config
|
||||
lib.any (lib.hasPrefix "git-2") (pathNames "hive-c0re")
|
||||
&& (env "hive-c0re") ? HIVE_FORGE_URL
|
||||
&& lib.all (unit: !(builtins.elem "git-credential-hive-forge" (pathNames unit))) [
|
||||
"hive-c0re"
|
||||
"hive-priv"
|
||||
"swarm-controller"
|
||||
]
|
||||
&& !((env "hive-priv") ? HIVE_PRIV_FORGE_URL);
|
||||
}
|
||||
{
|
||||
|
|
|
|||
Loading…
Reference in a new issue