diff --git a/docs/networking/network.md b/docs/networking/network.md index a288374c..1efcf70a 100644 --- a/docs/networking/network.md +++ b/docs/networking/network.md @@ -263,7 +263,7 @@ wiring is runtime: - the `hyperhive-isolated-dns` oneshot (`nix/agent-modules/network.nix`), gated on that marker, rewrites `/etc/resolv.conf` to `nameserver ` at boot. It's ordered `before` the harness (`hive-ag3nt`), the matrix daemon, and - `tea-login` so the resolver is correct before the first DNS lookup. + the forge-token fetch so the resolver is correct before the first DNS lookup. **Why isolation is safe**: hive-c0re's control-plane sockets are unix domain sockets bind-mounted into containers, not network listeners — see diff --git a/docs/process/conventions.md b/docs/process/conventions.md index efd6bf10..7db2237e 100644 --- a/docs/process/conventions.md +++ b/docs/process/conventions.md @@ -502,13 +502,9 @@ sparingly) or build just the suspect check, for example `nix build ## Best-effort oneshot services The harness ships a family of one-shot systemd services that -configure agent-side surfaces from values hive-c0re writes into -the state dir at provisioning time: +configure agent-side surfaces from values delivered to the agent at +provisioning time: -- `tea-login` — writes `~/.config/tea/config.yml` from the - `forge-token` written by `hive-c0re::forge::ensure_user_for`, - so `tea repos create` / `tea pulls create` work without - interactive prompts. - `forge-avatar-sync` — uploads `services.hyperhive.agent.icon` SVG to the agent's Forgejo profile, so the icon shows up on commits / PRs / issue comments. @@ -538,13 +534,10 @@ Shape contract — every one of these: by the `.path` watchers that re-fire on token appearance (see `docs/agent-lifecycle/persistence.md::Matrix per-agent daemon`). -The artefact lives under the agent user's home where applicable -(`~/.config/tea/config.yml`) and is chown'd to that user, but the -service itself stays root-owned so the bootstrap ordering doesn't -need a user-existence check before each fire. +The service stays root-owned so the bootstrap ordering doesn't need a +user-existence check before each fire. This pattern keeps the rebuild path resilient: any failure inside -these services degrades the corresponding surface (no tea config, -no avatar) but never blocks the container from coming up. The +these services degrades the corresponding surface (no avatar) but never blocks the container from coming up. The operator notices through `journalctl -u ` rather than a broken switch-to-configuration. diff --git a/docs/process/gotchas.md b/docs/process/gotchas.md index 6401176e..4571624c 100644 --- a/docs/process/gotchas.md +++ b/docs/process/gotchas.md @@ -448,7 +448,7 @@ connects to the compositor at `127.0.0.1:`. must never block on weston signalling readiness. A misconfigured weston degrades to a `Restart=on-failure` loop visible in `journalctl`, it doesn't abort the `nixos-container update`. - Same reasoning as the `tea-login` unit in `nix/agent-modules/forge.nix`. + Same reasoning as the best-effort oneshots in `docs/process/conventions.md`. - **`[core] idle-time=0`**: disables weston's 300-second idle timeout. Without it the VNC desktop fades to black and desktop-shell shows its click-to-unlock screen — useless for an diff --git a/docs/turn-loop/config.md b/docs/turn-loop/config.md index 50549231..e50213da 100644 --- a/docs/turn-loop/config.md +++ b/docs/turn-loop/config.md @@ -137,11 +137,9 @@ services.hyperhive.agent.forge.url = "http://forge.example:3000"; # default: nu services.hyperhive.agent.matrix.url = "https://matrix.example"; # default: null ``` -**`services.hyperhive.agent.forge.url`** — base URL of the Forgejo instance. Used by -a one-shot boot unit (`tea-login`) that writes `~/.config/tea/config.yml` -directly from the agent's `forge-token`, so `tea` and `hive-forge` -work without an interactive auth step. The unit is a no-op when -`forge-token` is absent. Override when the agent should connect to a +**`services.hyperhive.agent.forge.url`** — base URL of the Forgejo instance. It scopes +the git credential helper to this forge and is where `forge-avatar-sync` +uploads the agent's icon. Override when the agent should connect to a Forgejo on a different host or port (for example a swarm peer's forge). Validated: must be an `http://` or `https://` URL, or `null`. @@ -149,7 +147,7 @@ Validated: must be an `http://` or `https://` URL, or `null`. loopback default would only ever be correct when the forge shares the agent's network namespace, and inside a container `localhost` is the agent itself, so the default was a value that built fine and then talked -to the wrong machine. With `null` the `tea-login` and `forge-avatar-sync` +to the wrong machine. With `null` the credential helper and `forge-avatar-sync` units aren't generated at all: an absent integration rather than a misdirected one. You don't normally set this — hive-c0re renders the host's real forge URL into every agent, and refuses to write a meta diff --git a/hive-agent/src/web_ui/state.rs b/hive-agent/src/web_ui/state.rs index f39e3095..a701e4a5 100644 --- a/hive-agent/src/web_ui/state.rs +++ b/hive-agent/src/web_ui/state.rs @@ -351,7 +351,12 @@ fn agent_links(label: &str, gui_enabled: bool) -> Vec { }); } - if crate::paths::state_dir().join("forge-token").is_file() { + // Either copy of the forge token: the one fetched from the swarm secret + // store (`HIVE_FORGE_TOKEN_FILE`), or the state-dir file the hive used + // to write. + let fetched_token = std::env::var_os("HIVE_FORGE_TOKEN_FILE") + .is_some_and(|p| std::path::Path::new(&p).is_file()); + if fetched_token || crate::paths::state_dir().join("forge-token").is_file() { links.push(AgentLink { url: format!("/{label}"), icon: "🔨".to_owned(), diff --git a/hive-c0re/src/meta.rs b/hive-c0re/src/meta.rs index 3f26dd4b..e510ab40 100644 --- a/hive-c0re/src/meta.rs +++ b/hive-c0re/src/meta.rs @@ -642,7 +642,7 @@ const FORWARDED_VARS: &[&str] = &[ /// Map of forwarded env var -> the agent option carrying the same value. /// /// Both exist because they're consumed at different times: the option is baked -/// into scripts at build time (tea-login bakes `FORGE_URL` from it), the env +/// into scripts at build time (forge-avatar-sync bakes `FORGE_URL` from it), the env /// var is read at runtime. Setting only one leaves the other on its default, /// which is how a hive ends up with two disagreeing answers for one value. /// @@ -1242,7 +1242,7 @@ where # resolve the agent's durable state dir without hard-coding it. # `environment.variables` only writes /etc/environment (login # shells); `systemd.globalEnvironment` is the analogue for - # systemd units so tea-login / forge-avatar-sync / + # systemd units so forge-avatar-sync / # hive-matrix-daemon etc. can read `$HYPERHIVE_STATE_DIR` # without each service having to redeclare it. environment.variables = { @@ -1259,7 +1259,7 @@ where // Forwarded vars (HIVE_FORGE_URL etc.) also go into globalEnvironment, // not just the harness service env below, so EVERY service + shell in // the container inherits them — crucially the bash-task runner (where - // `hive-forge` + `git` actually run), plus the matrix daemon, tea-login + // `hive-forge` + `git` actually run), plus the matrix daemon, forge-avatar-sync // and interactive shells. Scoped to the harness service alone they were // invisible to bash tasks: harmless in shared netns (the localhost // default works) but broken under isolation, where the in-cluster @@ -2081,7 +2081,7 @@ mod tests { fn render_flake_sets_service_url_options_from_forwarded_env() { // The forwarded env vars must ALSO become option assignments, because // the two are consumed at different times: the option is baked into - // scripts at build time (tea-login's FORGE_URL), the env var is read at + // scripts at build time (forge-avatar-sync's FORGE_URL), the env var is read at // runtime. Emitting only the env var leaves the option on its default, // which is how a hive ends up with two disagreeing answers for the same // URL. diff --git a/hive-forge-notify/src/bin/hive-forge-notify/main.rs b/hive-forge-notify/src/bin/hive-forge-notify/main.rs index b1786c87..f9717ed6 100644 --- a/hive-forge-notify/src/bin/hive-forge-notify/main.rs +++ b/hive-forge-notify/src/bin/hive-forge-notify/main.rs @@ -37,23 +37,17 @@ async fn forgejo_loop(state_dir: String, socket: std::path::PathBuf) { } }; - let token_path = format!("{state_dir}/forge-token"); - // Retry reading the token to handle races where hive-priv provisions - // it after the harness starts, or where a parent-container chown - // briefly makes the file unreadable. - let token = { + let token_paths = hive_forge_notify::forge_token_paths(&state_dir); + // Retry reading the token to handle races where the agent's fetch of + // it from the swarm secret store lands after this unit starts, or where + // a parent-container chown briefly makes the file unreadable. + let mut token = { let mut attempts = 0u32; loop { - match tokio::fs::read_to_string(&token_path).await { - Ok(t) => { - let t = t.trim().to_owned(); - if !t.is_empty() { - break t; - } - debug!("forge_notify: empty forge token at {token_path}"); - } - Err(e) => debug!("forge_notify: cannot read token at {token_path}: {e}"), + if let Some(t) = hive_forge_notify::read_first_token(&token_paths) { + break t; } + debug!(?token_paths, "forge_notify: no forge token yet"); attempts += 1; if attempts >= TOKEN_RETRY_MAX { debug!( @@ -65,7 +59,7 @@ async fn forgejo_loop(state_dir: String, socket: std::path::PathBuf) { } }; - let Some(source) = ForgejoSource::new(&forge_url, &token) else { + let Some(mut source) = ForgejoSource::new(&forge_url, &token) else { return; }; @@ -111,6 +105,18 @@ async fn forgejo_loop(state_dir: String, socket: std::path::PathBuf) { loop { interval.tick().await; + // The swarm rotates the token when it goes stale, and the agent's + // fetch replaces the file; pick the new one up rather than polling + // with a revoked token for the rest of this process's life. + if let Some(t) = hive_forge_notify::read_first_token(&token_paths) + && t != token + && let Some(s) = ForgejoSource::new(&forge_url, &t) + { + info!("forge_notify: forge token changed on disk; using the new one"); + token = t; + source = s; + own_login.clear(); + } if own_login.is_empty() { own_login = resolve_own_login(&client, &source).await; if !own_login.is_empty() { diff --git a/hive-forge-notify/src/lib.rs b/hive-forge-notify/src/lib.rs index 95f1cf2d..702ba922 100644 --- a/hive-forge-notify/src/lib.rs +++ b/hive-forge-notify/src/lib.rs @@ -70,3 +70,76 @@ pub fn init_tracing() { pub fn state_dir() -> String { std::env::var("HYPERHIVE_STATE_DIR").unwrap_or_default() } + +/// Where this agent's forge token is read from, first match wins: +/// `HIVE_FORGE_TOKEN_FILE` (the copy the agent fetched from the swarm secret +/// store, set by `nix/agent-modules/forge-token.nix`), then +/// `/forge-token` (the file the hive used to write, still the only +/// copy on an agent without a store identity). +#[must_use] +pub fn forge_token_paths(state_dir: &str) -> Vec { + let mut paths = Vec::with_capacity(2); + if let Ok(fetched) = std::env::var("HIVE_FORGE_TOKEN_FILE") + && !fetched.is_empty() + { + paths.push(std::path::PathBuf::from(fetched)); + } + paths.push(std::path::Path::new(state_dir).join("forge-token")); + paths +} + +/// The first non-empty token among `paths`, trimmed. `None` when none of +/// them holds one. +#[must_use] +pub fn read_first_token(paths: &[std::path::PathBuf]) -> Option { + paths.iter().find_map(|p| { + let t = std::fs::read_to_string(p).ok()?; + let t = t.trim(); + (!t.is_empty()).then(|| t.to_owned()) + }) +} + +#[cfg(test)] +mod token_tests { + use super::read_first_token; + + fn scratch(tag: &str) -> std::path::PathBuf { + let ts = std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .map_or(0, |d| d.as_nanos()); + let dir = std::env::temp_dir().join(format!("hive-forge-notify-token-{tag}-{ts}")); + std::fs::create_dir_all(&dir).expect("create scratch dir"); + dir + } + + #[test] + fn the_fetched_token_wins_over_the_state_file() { + let dir = scratch("wins"); + let (fetched, state) = (dir.join("fetched"), dir.join("forge-token")); + std::fs::write(&fetched, "new\n").expect("write"); + std::fs::write(&state, "old\n").expect("write"); + assert_eq!(read_first_token(&[fetched, state]).as_deref(), Some("new")); + let _ = std::fs::remove_dir_all(&dir); + } + + #[test] + fn a_missing_or_empty_fetched_token_falls_back_to_the_state_file() { + let dir = scratch("fallback"); + let (fetched, state) = (dir.join("fetched"), dir.join("forge-token")); + std::fs::write(&state, "old\n").expect("write"); + assert_eq!( + read_first_token(&[fetched.clone(), state.clone()]).as_deref(), + Some("old") + ); + std::fs::write(&fetched, "\n").expect("write"); + assert_eq!(read_first_token(&[fetched, state]).as_deref(), Some("old")); + let _ = std::fs::remove_dir_all(&dir); + } + + #[test] + fn no_token_anywhere_is_none() { + let dir = scratch("none"); + assert_eq!(read_first_token(&[dir.join("a"), dir.join("b")]), None); + let _ = std::fs::remove_dir_all(&dir); + } +} diff --git a/hive-forge/src/client.rs b/hive-forge/src/client.rs index a33f87e1..cfe86bdb 100644 --- a/hive-forge/src/client.rs +++ b/hive-forge/src/client.rs @@ -1,5 +1,6 @@ -//! App-level Forgejo client wrapper. Identity is the per-agent token -//! under `${HYPERHIVE_STATE_DIR}/forge-token`. REST calls go through +//! App-level Forgejo client wrapper. Identity is the per-agent token at +//! `$HIVE_FORGE_TOKEN_FILE`, else `${HYPERHIVE_STATE_DIR}/forge-token` +//! (see `read_token`). REST calls go through //! the typed [`forgejo_api::sync::Forgejo`] client (exposed via //! [`Client::api`]); a minimal raw `reqwest` client remains for the //! few *web-router* routes Forgejo does not serve under `/api/v1/` @@ -462,9 +463,20 @@ fn state_dir() -> PathBuf { } } -/// Locate and read the forge token. Falls back to `$PWD/forge-token` -/// when `HYPERHIVE_STATE_DIR` isn't set, matching the bash helper. +/// Locate and read the forge token: `HIVE_FORGE_TOKEN_FILE` first (the +/// copy the agent fetched from the swarm secret store, set by +/// `nix/agent-modules/forge-token.nix`) when it names a non-empty file, +/// otherwise `/forge-token`, the file the hive used to write. +/// The state dir falls back to `$PWD` when `HYPERHIVE_STATE_DIR` isn't +/// set, matching the bash helper. fn read_token() -> Result { + if let Ok(fetched) = std::env::var("HIVE_FORGE_TOKEN_FILE") + && !fetched.is_empty() + && let Ok(raw) = std::fs::read_to_string(&fetched) + && !raw.trim().is_empty() + { + return Ok(raw.trim().to_owned()); + } let path = state_dir().join("forge-token"); let raw = std::fs::read_to_string(&path) .with_context(|| format!("hive-forge: no forge-token at {}", path.display()))?; diff --git a/hive-forge/tests/credential_helper.rs b/hive-forge/tests/credential_helper.rs index e15947a4..77936bb6 100644 --- a/hive-forge/tests/credential_helper.rs +++ b/hive-forge/tests/credential_helper.rs @@ -29,7 +29,23 @@ fn scratch_state_dir(tag: &str) -> PathBuf { /// `state_dir`, feeding `request` as the git credential-protocol /// request body on stdin. Returns `(exit success, stdout, stderr)`. fn run_get(base_url: &str, state_dir: &Path, request: &str) -> (bool, String, String) { - let mut child = Command::new(env!("CARGO_BIN_EXE_hive-forge")) + run_get_with(base_url, state_dir, None, request) +} + +/// [`run_get`], with `HIVE_FORGE_TOKEN_FILE` set to `fetched` when given and +/// removed otherwise, so the caller's own environment never leaks in. +fn run_get_with( + base_url: &str, + state_dir: &Path, + fetched: Option<&Path>, + request: &str, +) -> (bool, String, String) { + let mut cmd = Command::new(env!("CARGO_BIN_EXE_hive-forge")); + match fetched { + Some(p) => cmd.env("HIVE_FORGE_TOKEN_FILE", p), + None => cmd.env_remove("HIVE_FORGE_TOKEN_FILE"), + }; + let mut child = cmd .arg("credential-helper") .arg("get") .env("HIVE_FORGE_URL", base_url) @@ -111,3 +127,46 @@ fn credential_helper_get_still_works_with_no_host_line() { ); let _ = std::fs::remove_dir_all(&dir); } + +/// The token the agent fetched from the swarm secret store wins over the +/// state-dir file the hive used to write, so a rotated token takes effect +/// even while the old file is still on disk. +#[test] +fn the_fetched_token_wins_over_the_state_file() { + let dir = scratch_state_dir("fetched"); + let fetched = dir.join("fetched-token"); + std::fs::write(&fetched, "fetched-token-value\n").expect("seed fetched token"); + let (ok, stdout, stderr) = run_get_with( + "http://forge.internal.example", + &dir, + Some(&fetched), + "protocol=https\nhost=forge.internal.example\n", + ); + assert!(ok, "stderr={stderr}"); + assert!( + stdout.contains("password=fetched-token-value"), + "stdout={stdout}" + ); + assert!(!stdout.contains(TOKEN), "stdout={stdout}"); + let _ = std::fs::remove_dir_all(&dir); +} + +/// An agent whose token was not minted yet has the variable set and no file +/// behind it; it keeps working off the state-dir file. +#[test] +fn a_fetched_token_that_is_not_there_falls_back_to_the_state_file() { + let dir = scratch_state_dir("fallback"); + let missing = dir.join("never-fetched"); + let (ok, stdout, stderr) = run_get_with( + "http://forge.internal.example", + &dir, + Some(&missing), + "protocol=https\nhost=forge.internal.example\n", + ); + assert!(ok, "stderr={stderr}"); + assert!( + stdout.contains(&format!("password={TOKEN}")), + "stdout={stdout}" + ); + let _ = std::fs::remove_dir_all(&dir); +} diff --git a/nix/agent-modules/default.nix b/nix/agent-modules/default.nix index a244ad02..68a93b9f 100644 --- a/nix/agent-modules/default.nix +++ b/nix/agent-modules/default.nix @@ -46,6 +46,7 @@ in ./dashboard-links.nix ./docs.nix ./forge.nix + ./forge-token.nix ./frontend.nix ./github.nix ./logs.nix diff --git a/nix/agent-modules/forge-token.nix b/nix/agent-modules/forge-token.nix new file mode 100644 index 00000000..58ca6a94 --- /dev/null +++ b/nix/agent-modules/forge-token.nix @@ -0,0 +1,182 @@ +# This agent's own forge access token, fetched from the swarm secret store by +# the agent itself. +# +# `swarm-controller` mints the token with the forge's admin API and writes it +# to `swarm/agents//forge-token` +# (`swarm_secret_client::forge::agent_token_path`); no hive is in that chain. +# This unit logs in to the store with the certificate ./bao.nix already proves +# it can log in with, and reads its own path. The shape is ./queue-identity.nix's, +# for the same reason: a hive handing the token over would be the hive reading +# a secret on the agent's behalf. +# +# Unlike the queue credential this one rotates: the controller replaces the +# token when the forge's copy stops matching the stored one. So the unit is +# re-run by a timer, and it swaps the file in by rename, only when the value +# changed, so a reader never sees half a token and a watcher on the file +# (forge-avatar-sync.path) fires only on a real change. +# +# Consumers read `services.hyperhive.agent.forge.tokenFile` first and fall back +# to `/forge-token`, the file the hive wrote before this existed. +{ + pkgs, + lib, + config, + ... +}: +let + cfg = config.services.hyperhive.agent.bao; + + # The name `swarm-controller` minted the token under — see ./queue-identity.nix. + agentName = config.services.hyperhive.agent.user.name; + + # The same three ids ./bao.nix and ./queue-identity.nix load. + certCredential = "hive-agent-bao-cert"; + keyCredential = "hive-agent-bao-key"; + serverCaCredential = "hive-agent-bao-server-ca"; + + unitName = "hive-agent-forge-token"; + + # The nix half of `swarm_secret_client::forge::agent_token_path` plus + # `path::MOUNT`. + tokenPath = "secret/swarm/agents/${agentName}/forge-token"; + + runtimeDir = unitName; + tokenFile = "/run/${runtimeDir}/token"; + # Beside the token, so the rename that replaces it stays in one directory. + stagingFile = "/run/${runtimeDir}/token.new"; + + # The store's address is the whole switch, as in ./bao.nix and + # ./queue-identity.nix. + configured = cfg.addr != null; +in +{ + options.services.hyperhive.agent.forge.tokenFile = lib.mkOption { + type = lib.types.str; + readOnly = true; + default = tokenFile; + description = '' + Path this agent's own forge token is fetched to. Read-only: it is a + fact about where `${unitName}.service` writes, not a knob. + + 🩸 A PATH and never a value. The file is `0400` to the agent user. + + The file exists only once the swarm has minted a token for this agent + and the agent has a store identity to fetch it with. Until then every + consumer falls back to `$HYPERHIVE_STATE_DIR/forge-token`. + ''; + }; + + config = lib.mkIf configured { + # Every unit and the bash-task runner (where `hive-forge` and `git` run) + # resolve the token through this, the same way they find + # `$HYPERHIVE_STATE_DIR`. A path, never the value. + systemd.globalEnvironment.HIVE_FORGE_TOKEN_FILE = tokenFile; + environment.variables.HIVE_FORGE_TOKEN_FILE = tokenFile; + + systemd.services.${unitName} = { + description = "fetch this agent's own forge token from the secret store"; + after = [ + "network.target" + # Ordering only, for the reason ./queue-identity.nix gives. + "hive-agent-bao-identity.service" + ]; + before = [ "hive-forge-notify.service" ]; + wantedBy = [ "multi-user.target" ]; + path = [ + pkgs.openbao + pkgs.coreutils + pkgs.diffutils + ]; + startLimitBurst = 4; + startLimitIntervalSec = 300; + serviceConfig = { + Type = "oneshot"; + # Not `RemainAfterExit`, unlike the queue fetch: the timer below has to + # be able to start this unit again, and an active unit cannot be + # started. `RuntimeDirectoryPreserve` is what keeps the directory, and + # the token in it, alive between runs instead. + RemainAfterExit = false; + TimeoutStartSec = 30; + Restart = "on-failure"; + RestartSec = 15; + User = agentName; + Group = agentName; + RuntimeDirectory = runtimeDir; + RuntimeDirectoryMode = "0700"; + RuntimeDirectoryPreserve = "yes"; + UMask = "0377"; + LoadCredential = [ + certCredential + keyCredential + serverCaCredential + ]; + }; + environment = { + BAO_ADDR = cfg.addr; + BAO_CLIENT_CERT = "%d/${certCredential}"; + BAO_CLIENT_KEY = "%d/${keyCredential}"; + }; + script = '' + set -euo pipefail + + # No identity delivered: ./bao.nix's check reports that; saying it + # twice adds nothing. + for id in ${lib.escapeShellArg certCredential} ${lib.escapeShellArg keyCredential}; do + if [ ! -s "$CREDENTIALS_DIRECTORY/$id" ]; then + echo "this agent has no store identity, so it cannot fetch its own forge token." >&2 + exit 0 + fi + done + + if [ -s "$CREDENTIALS_DIRECTORY/${serverCaCredential}" ]; then + export BAO_CACERT="$CREDENTIALS_DIRECTORY/${serverCaCredential}" + fi + + err="$(mktemp)" + trap 'rm -f "$err" ${lib.escapeShellArg stagingFile}' EXIT + + if ! BAO_TOKEN="$(bao login -method=cert -token-only 2>"$err")"; then + echo "this agent's certificate was refused by the swarm secret store at $BAO_ADDR." >&2 + if [ -s "$err" ]; then cat "$err" >&2; fi + exit 1 + fi + export BAO_TOKEN + + # 🩸 Degrades rather than fails, for the reason ./queue-identity.nix + # gives: the policy stanza that let the login read `bao-mtls` covers + # this path too, so a refusal here is a token not minted yet. The + # file already in place, if any, is kept: a store that is briefly + # unreachable must not take a working token away. + # + # ⚠️ Written by redirect, never echoed: the field is the secret. The + # staging file is removed first so the redirect creates it — a file + # `UMask=0377` left behind is `0400` and could not be reopened for + # writing. + rm -f ${lib.escapeShellArg stagingFile} + if ! bao kv get -field=value ${lib.escapeShellArg tokenPath} > ${lib.escapeShellArg stagingFile} 2>"$err"; then + echo "no forge token at ${tokenPath} yet; consumers keep using the state-dir token if there is one." >&2 + if [ -s "$err" ]; then cat "$err" >&2; fi + exit 0 + fi + + if cmp -s ${lib.escapeShellArg stagingFile} ${lib.escapeShellArg tokenFile}; then + echo "this agent's forge token at ${tokenPath} is unchanged." + exit 0 + fi + mv -f ${lib.escapeShellArg stagingFile} ${lib.escapeShellArg tokenFile} + echo "fetched this agent's forge token from ${tokenPath}." + ''; + }; + + # The controller re-checks every agent's token every five minutes; this + # picks up a rotation within about ten more. + systemd.timers.${unitName} = { + description = "re-fetch this agent's forge token from the secret store"; + wantedBy = [ "timers.target" ]; + timerConfig = { + OnUnitInactiveSec = "10min"; + RandomizedDelaySec = "1min"; + }; + }; + }; +} diff --git a/nix/agent-modules/forge.nix b/nix/agent-modules/forge.nix index 694b479e..19dc9649 100644 --- a/nix/agent-modules/forge.nix +++ b/nix/agent-modules/forge.nix @@ -1,6 +1,6 @@ -# In-container forge (Forgejo) integration: the `tea` CLI login -# oneshot, the `hive-forge` verb CLI on PATH, and the icon → forge -# avatar sync. +# In-container forge (Forgejo) integration: the `hive-forge` verb CLI on +# PATH, the git credential helper, the notification poller, and the icon → +# forge avatar sync. The token itself is fetched by ./forge-token.nix. { pkgs, lib, @@ -9,7 +9,18 @@ }: let userName = config.services.hyperhive.agent.user.name; - homeDir = "/home/${userName}"; + # The token the agent fetched from the swarm secret store + # (./forge-token.nix), then the state-dir file the hive used to write, + # which is still the only copy for an agent without a store identity. + # Both are PATHS; each reader below takes the first that holds a token. + fetchedTokenFile = config.services.hyperhive.agent.forge.tokenFile; + stateTokenFile = "/agents/${userName}/state/forge-token"; + pickTokenFile = '' + TOKEN_FILE= + for f in ${lib.escapeShellArg fetchedTokenFile} ${lib.escapeShellArg stateTokenFile}; do + if [ -s "$f" ] && [ -r "$f" ]; then TOKEN_FILE="$f"; break; fi + done + ''; # Same 512×512 rasterization of the agent icon the matrix avatar # sync uses (./matrix.nix — identical derivation, same store path). # Only forced when an icon is configured AND a forge is (the avatar-sync @@ -20,7 +31,7 @@ let # git credential helper for the hive forge --- the exact shape # `./github.nix` uses for github.com, for the same two reasons: the token - # is read from the agent's state file AT INVOCATION (so a re-issued token + # is read from its file AT INVOCATION (so a re-issued token # takes effect with no rebuild), and the token PATH is baked in at build # time rather than read from the environment, because claude's Bash tool # runs `bash -c` in a minimal env that never sources `/etc/set-environment`. @@ -32,8 +43,8 @@ let gitCredHelper = pkgs.writeShellScriptBin "git-credential-hive-forge" '' # git credential-helper protocol: only `get` needs an answer. [ "''${1:-}" = "get" ] || exit 0 - TOKEN_FILE="/agents/${userName}/state/forge-token" - [ -r "$TOKEN_FILE" ] || exit 0 + ${pickTokenFile} + [ -n "$TOKEN_FILE" ] || exit 0 printf 'username=%s\n' ${lib.escapeShellArg userName} printf 'password=%s\n' "$(cat "$TOKEN_FILE")" ''; @@ -44,20 +55,16 @@ in default = null; example = "http://forge.internal:3000"; description = '' - Base URL of the hyperhive-managed Forgejo. Used at container - boot by a oneshot systemd unit that calls - `tea login add --url --token "$(cat $HYPERHIVE_STATE_DIR/forge-token)"` - (= `/agents//state/forge-token`) so the agent's claude can - shell out to `tea` without an extra auth dance. No-op when the - forge-token file is missing (i.e. hive-forge isn't running on - the host). + Base URL of the hyperhive-managed Forgejo. Scopes the git + credential helper to this forge, and is where the avatar sync + uploads the agent's icon. **`null` means "no forge", not "guess one".** There is deliberately no loopback default: the forge may run on a different host from the agents, and inside an agent's network namespace `localhost` reaches the agent rather than the forge, so a default would be a value that builds fine and then talks to the wrong machine. - When this is `null` the tea-login and avatar-sync units are not + When this is `null` the credential helper and avatar-sync units are not generated at all --- an absent integration, never a misdirected one. @@ -88,10 +95,6 @@ in ]; environment.systemPackages = [ - # tea: gitea/forgejo CLI client. Configured at boot by the - # tea-login oneshot below if /state/forge-token is present, so - # claude can `tea repos create`, `tea pulls create`, etc. - pkgs.tea # hive-forge : CLI wrapping common Forgejo REST API operations # (view, pr, issue, comment, assign, close, labels, branches, etc.). # The per-bin split package — narrow closure, no hivectl/wireguard. @@ -178,94 +181,22 @@ in }; }; - # One-shot: tea config.yml from the seeded forge token. Shape - # contract (always exit 0, no set -e, skip-silently, re-runnable): - # docs/process/conventions.md::Best-effort oneshot services. - # Not generated at all when no forge is configured: an absent - # integration rather than one pointed at a guessed address. - systemd.services.tea-login = lib.mkIf (config.services.hyperhive.agent.forge.url != null) { - description = "configure tea CLI from hive-forge token (best-effort)"; - wantedBy = [ "multi-user.target" ]; - after = [ "local-fs.target" ]; - serviceConfig = { - Type = "oneshot"; - RemainAfterExit = true; - # Pin the journal identity (else it's the `script` store-path wrapper). - SyslogIdentifier = "tea-login"; - }; - path = [ - pkgs.curl - pkgs.jq - pkgs.coreutils - ]; - environment.HOME_DIR = homeDir; - environment.AGENT_USER = userName; - script = '' - # No `set -e`: best-effort posture (see docs pointer above). - FORGE_URL=${lib.escapeShellArg config.services.hyperhive.agent.forge.url} - # $HYPERHIVE_STATE_DIR is system-wide via the meta flake. - TOKEN_FILE="$HYPERHIVE_STATE_DIR/forge-token" - if [ ! -f "$TOKEN_FILE" ]; then - echo "tea-login: no forge-token at $TOKEN_FILE; skipping" - exit 0 - fi - TOKEN=$(cat "$TOKEN_FILE") - # Resolve the agent username from the forge API. - USER=$(curl -sf --max-time 5 \ - -H "Authorization: token $TOKEN" \ - "$FORGE_URL/api/v1/user" \ - | jq -r '.login // empty' 2>/dev/null || true) - if [ -z "$USER" ]; then - echo "tea-login: could not resolve username from forge API; skipping" - exit 0 - fi - # Config under the agent user's home, chown'd to them; - # service stays root-owned (see docs pointer above). - CONFIG="$HOME_DIR/.config/tea/config.yml" - mkdir -p "$(dirname "$CONFIG")" || true - cat > "$CONFIG" << EOF - logins: - - name: forge - url: $FORGE_URL - token: $TOKEN - default: true - ssh_host: "" - ssh_key: "" - insecure: false - ssh_agent: false - user: $USER - preferences: - editor: false - flag_defaults: - remote: "" - EOF - chown -R "$AGENT_USER:$AGENT_USER" "$HOME_DIR/.config" 2>/dev/null || true - echo "tea-login: configured for $FORGE_URL as $USER (config at $CONFIG)" - ''; - }; - - # Path-trigger sibling: re-fires forge-avatar-sync when - # `/forge-token` is written. On first agent deployment the - # container boots before hive-c0re has provisioned the forge-token, so - # the service fires too early and exits with "no forge-token found". - # Without this path unit, RemainAfterExit=true would prevent systemd - # from ever re-running the service. See - # docs/agent-lifecycle/persistence.md::forge-avatar-sync. + # Path-trigger sibling: re-fires forge-avatar-sync when the fetched + # forge token (./forge-token.nix) appears or is replaced. On first + # agent deployment the container can boot before the swarm has minted + # the token, so the service fires too early and exits with "no + # forge-token found". Without this path unit, nothing would ever + # re-run it. See docs/agent-lifecycle/persistence.md::forge-avatar-sync. # # PathChanged=, not PathExists=: a PathExists= condition that already # holds re-activates the unit immediately every time the path unit # re-arms, and a oneshot re-arms it by deactivating — so it loops until # systemd's start limit stops it. PathChanged= does not fire on an - # already-present path, and hive-priv writes this file in place - # (write_state_file_nofollow: O_TRUNC, no rename), so close-after-write - # still triggers it. Same directive and same reason as - # swarm-controller.nix's queue-credential watcher. + # already-present path, and does fire on the rename the fetch unit + # swaps a new token in with. The fetch renames only when the value + # changed, so its timer does not re-upload the avatar every tick. # - # ⚠️ This agent's own token, not a glob over `/agents/*/`. Every agent's - # state dir is visible from inside every container, so a wildcard here - # watches paths this unit has no business reacting to. - # The service reads `$HYPERHIVE_STATE_DIR/forge-token`; this is the same - # file, spelled the way `tea-login` above already spells it. + # ⚠️ This agent's own token, not a glob. # # ⚠️ Gated on the SAME condition as the service it triggers, not just on # the icon: a `.path` unit whose `Unit=` does not exist is a unit pulled @@ -278,7 +209,7 @@ in { description = "trigger forge-avatar-sync when forge-token appears"; wantedBy = [ "multi-user.target" ]; - pathConfig.PathChanged = "/agents/${userName}/state/forge-token"; + pathConfig.PathChanged = fetchedTokenFile; }; # One-shot: services.hyperhive.agent.icon → Forgejo profile avatar. Shape contract: @@ -295,7 +226,7 @@ in { description = "sync agent icon to Forgejo user avatar (best-effort)"; wantedBy = [ "multi-user.target" ]; - after = [ "tea-login.service" ]; + after = [ "hive-agent-forge-token.service" ]; serviceConfig = { Type = "oneshot"; RemainAfterExit = false; @@ -309,10 +240,8 @@ in ]; script = '' FORGE_URL=${lib.escapeShellArg config.services.hyperhive.agent.forge.url} - # $HYPERHIVE_STATE_DIR is set system-wide by the meta flake - # (systemd.globalEnvironment) to `/agents//state`. - TOKEN_FILE="$HYPERHIVE_STATE_DIR/forge-token" - if [ ! -f "$TOKEN_FILE" ]; then + ${pickTokenFile} + if [ -z "$TOKEN_FILE" ]; then echo "forge-avatar-sync: no forge-token found; skipping" exit 0 fi diff --git a/nix/agent-modules/network.nix b/nix/agent-modules/network.nix index 64adf66b..bf9f9d6a 100644 --- a/nix/agent-modules/network.nix +++ b/nix/agent-modules/network.nix @@ -45,7 +45,7 @@ # `/etc/hyperhive-bridge-dns` (containing the gateway IP) since # isolation is always on; the oneshot reads it and rewrites # resolv.conf on every boot. Ordered before the first DNS consumer - # (tea-login) and the network targets so name resolution works for + # (the forge-token fetch) and the network targets so name resolution works for # the very first turn. systemd.services.hyperhive-isolated-dns = { description = "point resolv.conf at the hive bridge resolver (isolated containers)"; @@ -61,7 +61,7 @@ # is a harmless no-op when matrix is disabled (the unit is absent). before = [ "network-online.target" - "tea-login.service" + "hive-agent-forge-token.service" "hive-agent.service" "hive-matrix-daemon.service" ]; diff --git a/swarmctl/src/agent.rs b/swarmctl/src/agent.rs index 7b6811d1..e6b57bf5 100644 --- a/swarmctl/src/agent.rs +++ b/swarmctl/src/agent.rs @@ -1,5 +1,5 @@ -//! `swarmctl agent create` and `swarmctl agent mint-identity` — queue work -//! on the swarm-controller's job graph. +//! `swarmctl agent create`, `swarmctl agent mint-identity` and `swarmctl agent +//! mint-forge-token` — queue work on the swarm-controller's job graph. //! //! Each POSTs and returns as soon as the work is *inserted*. Both verbs //! print the queued node id and stop: creation's last node only *publishes* @@ -64,6 +64,12 @@ struct MintIdentityResponse { node_id: u64, } +/// Success body of `POST /api/agents/{name}/forge-token`, which takes no body. +#[derive(Deserialize)] +struct MintForgeTokenResponse { + node_id: u64, +} + /// Run `swarmctl agent create`. /// /// Synchronous on purpose: every other verb in this crate is, and this is @@ -137,6 +143,33 @@ pub(crate) fn mint_identity(socket: &Path, name: &str, hive: &str) -> Result<()> Ok(()) } +/// Run `swarmctl agent mint-forge-token`. +/// +/// Synchronous for the same reason [`create`] is, and built on the same +/// round trip. No `--hive`: the token's store path has no hive in it. +pub(crate) fn mint_forge_token(socket: &Path, name: &str) -> Result<()> { + let name = parse_ident(name, "agent name")?; + + let rt = tokio::runtime::Builder::new_current_thread() + .enable_io() + .build() + .context("starting a tokio runtime for the controller request")?; + let resp: MintForgeTokenResponse = rt.block_on(post( + socket, + &format!("/api/agents/{name}/forge-token"), + &serde_json::json!({}), + "mint-forge-token", + ))?; + + println!("queued: job node {}", resp.node_id); + println!( + "agent {name:?}'s forge token will be checked once the job graph runs, and minted \ + only if it is missing or stale; `swarmctl` does not wait for it. The agent picks a \ + new token up on its next fetch" + ); + Ok(()) +} + /// One `POST /api/agents` round trip over the controller's unix socket. async fn post_create(socket: &Path, name: &str, hive: &str) -> Result { post( diff --git a/swarmctl/src/main.rs b/swarmctl/src/main.rs index 78eef948..7832e335 100644 --- a/swarmctl/src/main.rs +++ b/swarmctl/src/main.rs @@ -184,6 +184,15 @@ enum AgentVerb { /// Queues and returns, the same way `agent create` does — watch the /// swarm UI's job view for the outcome. MintIdentity(AgentMintIdentityArgs), + /// Check one agent's forge token, and mint it if it's missing or stale. + /// + /// swarm-controller does this for every agent with a store identity at + /// start and every five minutes; this is for when waiting isn't an + /// option. It leaves a current token alone. + /// + /// Queues and returns, the same way `agent create` does — watch the + /// swarm UI's job view for the outcome. + MintForgeToken(AgentMintForgeTokenArgs), } #[derive(Args)] @@ -210,6 +219,19 @@ struct AgentCreateArgs { controller_socket: Option, } +#[derive(Args)] +struct AgentMintForgeTokenArgs { + /// Name of an agent that already exists. + name: String, + /// swarm-controller's unix socket. + /// + /// Supplied by the nix module that installs this binary, from the same + /// `socketPath` option the daemon binds; falls back to + /// `SWARM_CONTROLLER_SOCKET`. + #[arg(long, value_name = "PATH")] + controller_socket: Option, +} + #[derive(Args)] struct AgentMintIdentityArgs { /// Name of an agent that already exists. @@ -308,6 +330,13 @@ fn main() -> Result<()> { let socket = path_from(args.controller_socket, "SWARM_CONTROLLER_SOCKET")?; agent::mint_identity(&socket, &args.name, &args.hive) } + // Same socket-resolution reasoning as `Create` above. + Verb::Agent { + command: AgentVerb::MintForgeToken(args), + } => { + let socket = path_from(args.controller_socket, "SWARM_CONTROLLER_SOCKET")?; + agent::mint_forge_token(&socket, &args.name) + } // Resolved lazily, inside the one arm that actually touches the // deployment env vars — see the `MarkdownDocs` doc comment above // for why an unconditional resolve up front would be wrong. @@ -697,6 +726,31 @@ mod tests { ); } + #[test] + fn the_forge_token_verb_takes_an_agent_and_no_hive() { + let cli = Cli::try_parse_from(["swarmctl", "agent", "mint-forge-token", "scribe"]) + .expect("the minimal form parses"); + let Verb::Agent { + command: AgentVerb::MintForgeToken(args), + } = cli.command + else { + panic!("expected `agent mint-forge-token`"); + }; + assert_eq!(args.name, "scribe"); + assert!( + Cli::try_parse_from([ + "swarmctl", + "agent", + "mint-forge-token", + "scribe", + "--hive", + "a" + ]) + .is_err(), + "the token has no hive, so the verb must not take one" + ); + } + #[test] fn parses_authelia_hash_output() { let out = "Random Password: hunter2\nDigest: $argon2id$v=19$m=65536$abc\n";