Compare commits

...
Author SHA1 Message Date
atlas
87f8e936d5 refactor(#2502): thread forge_base param into render_flake, drop the localhost fallback
Per operator review (#2547): remove the never-reached branch. render_flake
read HIVE_FORGE_URL inline with an unwrap_or_else(localhost:3000) fallback
that can't be hit — the daemon always sets HIVE_FORGE_URL (environment.nix)
— and the localhost URL wouldn't match the domain-scoped credential helper
anyway. Thread forge_base as an explicit param instead: the caller
sync_agents passes forge::forge_http_base(), tests pass it explicitly. This
removes the dead branch AND the hidden env-read, and drops the racy env
set_var from the forge-url test. render_flake is pure/param-driven again.
2026-07-17 01:48:27 +02:00
atlas
7df9f5d024 docs(#2502): meta flake input is the forge config repo, deploy overrides to local applied 2026-07-17 01:48:27 +02:00
atlas
27a099bb5b feat(#2502): re-lock deploy + rebuild from local applied, not forge
The persistent agent-<n> input url in the meta flake is now the forge
config repo, but the deploy + manual-rebuild re-lock must not follow it:
prepare_deploy and lock_update_for_rebuild override the input back to the
local applied mirror (git+file://<applied_dir>, main head) via
--override-input, which writes the applied rev into flake.lock.

This keeps the forge as the declared, reviewable source while:
- surviving forge unreachability (rebuilds fire on crash-restart and meta
  bumps too, not just config PRs -- coupling every rebuild to forge would
  be a regression),
- deploying exactly the reviewed head applied/<n>/main was fast-forwarded
  to (no TOCTOU on a newer forge head merging mid-deploy),
- reusing verify_commit's local-override pattern so verify and deploy eval
  the same source.

New applied_override_url helper + unit test.
2026-07-17 01:48:27 +02:00
atlas
b806aa253e feat(#2502): git credential helper for hive-core forge fetches
hive-core now fetches each agent's config as a forge-hosted flake input
(git+http://<forge>/agent-configs/<n>.git). Add a git credential helper
(git-credential-hive-forge) that reads the live forge-core admin token on
every invocation and authenticates as the forge core user, wired via the
[credential] stanza in hive-core's $HOME/.gitconfig + on the service PATH.
Reading the token file live means zero stale copies and no resync on
rotation; no token ever lands in a flake URL or lock. Mirrors the existing
github.nix credential-helper pattern.
2026-07-17 01:48:27 +02:00
atlas
a5870c5ddf wip(#2502): render agent config input from forge repo (meta.rs + test) 2026-07-17 01:48:27 +02:00
4 changed files with 173 additions and 6 deletions

View file

@ -213,13 +213,31 @@ transient.
The hive-c0re-owned repo at `/var/lib/hyperhive/meta/` The hive-c0re-owned repo at `/var/lib/hyperhive/meta/`
declares one flake input per agent (`agent-<n>.url = declares one flake input per agent (`agent-<n>.url =
"git+file:///var/lib/hyperhive/applied/<n>"`) and one "git+http://<forge>/agent-configs/<n>.git"`) and one
`nixosConfigurations.<n>` output per agent. Each output wraps `nixosConfigurations.<n>` output per agent. Each output wraps
`inputs.agent-<n>.nixosModules.default` with the identity + `inputs.agent-<n>.nixosModules.default` with the identity +
`HIVE_PORT` / `HIVE_LABEL` / `HIVE_DASHBOARD_PORT` injection `HIVE_PORT` / `HIVE_LABEL` / `HIVE_DASHBOARD_PORT` injection
module that `setup_applied` used to generate inline. module that `setup_applied` used to generate inline.
Containers run against `--flake /var/lib/hyperhive/meta#<n>`. Containers run against `--flake /var/lib/hyperhive/meta#<n>`.
The declared input url is the agent's **forge config repo** (the
same `agent-configs/<n>` the config-PR flow lands approved changes
on), so the meta flake references a reviewable, reproducible source
rather than a local checkout. hive-c0re authenticates that
`git+http` fetch via a git credential helper that reads the live
forge-core token — no token in the url or the lock. The deploy and
manual-rebuild paths, however, do **not** re-lock from the forge:
they `--override-input agent-<n>
git+file:///var/lib/hyperhive/applied/<n>`, locking the exact config
that `verify_commit` gated and `applied/<n>/main` was
fast-forwarded to. That keeps a deploy/rebuild reproducible and
independent of forge reachability — rebuilds fire on crash-restart
and meta bumps, not just config PRs — while the declared url stays
the forge. `sync_agents` re-renders + re-locks the persistent input;
a plain `nix flake lock` leaves an existing applied override in
place (it only re-locks when the declared url itself changes), so
the forge-declared / applied-deployed split is stable.
Per-deploy lock flow (two-phase, owned by Per-deploy lock flow (two-phase, owned by
`actions::run_merge_config_pr``deploy_applied_target` `actions::run_merge_config_pr``deploy_applied_target`
`meta::{prepare,finalize,abort}_deploy`): `meta::{prepare,finalize,abort}_deploy`):

View file

@ -64,6 +64,11 @@ Two things live in the `agent-configs` Forgejo organization:
approved history, the `failed/<id>` tag records the divergence). approved history, the `failed/<id>` tag records the divergence).
Repos stay private, so an agent can't read another Repos stay private, so an agent can't read another
agent's config. (Agents remain read-only collaborators on `core/meta`.) agent's config. (Agents remain read-only collaborators on `core/meta`.)
hive-c0re also references this repo as the agent's **persistent meta
flake input** (`agent-<n>.url = git+http://<forge>/agent-configs/<n>.git`;
see [approvals.md § Meta flake](approvals.md)), fetching it as the `core`
user via a git credential helper that reads the live forge-core token —
so the config lives on the forge, not a hand-synced local checkout.
- The dashboard links each container's "config" anchor to this - The dashboard links each container's "config" anchor to this
config repo, so operators can click straight from the SW4RM tab into config repo, so operators can click straight from the SW4RM tab into
the rendered repo without an extra `git` step. the rendered repo without an extra `git` step.

View file

@ -74,6 +74,7 @@ pub async fn sync_agents(hive: &HiveEnv, agents: &[AgentSpec]) -> Result<()> {
&hive.operator_pronouns, &hive.operator_pronouns,
&hive.context_window_tokens, &hive.context_window_tokens,
agents, agents,
crate::forge::forge_http_base(),
); );
let flake_path = dir.join("flake.nix"); let flake_path = dir.join("flake.nix");
let on_disk = std::fs::read_to_string(&flake_path).unwrap_or_default(); let on_disk = std::fs::read_to_string(&flake_path).unwrap_or_default();
@ -238,7 +239,29 @@ pub async fn prepare_deploy(name: &str) -> Result<()> {
let _guard = META_LOCK.lock().await; let _guard = META_LOCK.lock().await;
let dir = crate::paths::meta_root(); let dir = crate::paths::meta_root();
let input = format!("agent-{name}"); let input = format!("agent-{name}");
nix_logged(&dir, &["flake", "update", &input], name, "prepare-deploy").await?; // Re-lock the agent input against the LOCAL applied mirror, not the
// persistent forge URL declared in the meta flake (see the `## Meta flake`
// note in docs/approvals.md): the deploy must build the exact reviewed
// config that `verify_commit` gated and `applied/<n>/main` was
// fast-forwarded to, and it must keep working when the forge is
// unreachable (rebuilds fire on crash-restart / meta bumps too, not just
// config PRs). `--override-input` writes the applied rev into the lock;
// the forge URL stays the declared, reviewable source of truth.
let applied = applied_override_url(&crate::paths::applied_dir(name));
nix_logged(
&dir,
&[
"flake",
"update",
&input,
"--override-input",
&input,
&applied,
],
name,
"prepare-deploy",
)
.await?;
// Stage the new lock — git+file://'s dirty-tree fetcher reads // Stage the new lock — git+file://'s dirty-tree fetcher reads
// index entries, so the upcoming nixos-container update sees the // index entries, so the upcoming nixos-container update sees the
// bumped rev without a commit yet. // bumped rev without a commit yet.
@ -281,7 +304,22 @@ pub async fn lock_update_for_rebuild(name: &str) -> Result<()> {
let _guard = META_LOCK.lock().await; let _guard = META_LOCK.lock().await;
let dir = crate::paths::meta_root(); let dir = crate::paths::meta_root();
let input = format!("agent-{name}"); let input = format!("agent-{name}");
nix(&dir, &["flake", "update", &input]).await?; // Re-lock from the local applied mirror, not the persistent forge URL —
// same rationale as `prepare_deploy`: build exactly `applied/<n>/main` and
// stay reproducible when the forge is unreachable.
let applied = applied_override_url(&crate::paths::applied_dir(name));
nix(
&dir,
&[
"flake",
"update",
&input,
"--override-input",
&input,
&applied,
],
)
.await?;
if !paths_dirty(&dir, &["flake.lock"]).await? { if !paths_dirty(&dir, &["flake.lock"]).await? {
return Ok(()); return Ok(());
} }
@ -301,6 +339,17 @@ fn agent_input_override(applied_dir: &Path, sha: &str) -> String {
format!("git+file://{}?rev={sha}", applied_dir.display()) format!("git+file://{}?rev={sha}", applied_dir.display())
} }
/// `--override-input` URL re-locking an agent's config input against its
/// LOCAL applied mirror (`git+file://<applied_dir>`, current `main` head)
/// instead of the persistent forge URL declared in the meta flake. The
/// deploy + rebuild paths re-lock from here: the applied tree was already
/// fast-forwarded to the reviewed head, so this builds exactly that config
/// and stays reproducible when the forge is unreachable. No `?rev` — `main`
/// head is the reviewed head at deploy time. Pure so it's unit-testable.
fn applied_override_url(applied_dir: &Path) -> String {
format!("git+file://{}", applied_dir.display())
}
/// Non-mutating "would this commit apply?" verify for the PR-based config /// Non-mutating "would this commit apply?" verify for the PR-based config
/// flow. Evaluates the agent's nixos configuration with its meta /// flow. Evaluates the agent's nixos configuration with its meta
/// input overridden to the exact `sha`, WITHOUT moving `applied/main` or /// input overridden to the exact `sha`, WITHOUT moving `applied/main` or
@ -586,6 +635,11 @@ pub async fn bulk_commit_topology(
Ok(changed) Ok(changed)
} }
#[allow(
clippy::too_many_arguments,
reason = "many genuine flake inputs (source flakes, port, pronouns, tokens, \
agents, forge base); a params struct would just move the same fields"
)]
fn render_flake( fn render_flake(
hyperhive_flake: &str, hyperhive_flake: &str,
docs_flake: &str, docs_flake: &str,
@ -594,6 +648,7 @@ fn render_flake(
operator_pronouns: &str, operator_pronouns: &str,
context_window_tokens: &std::collections::HashMap<String, u64>, context_window_tokens: &std::collections::HashMap<String, u64>,
agents: &[AgentSpec], agents: &[AgentSpec],
forge_base: &str,
) -> String { ) -> String {
render_flake_with_lookup( render_flake_with_lookup(
hyperhive_flake, hyperhive_flake,
@ -603,6 +658,7 @@ fn render_flake(
operator_pronouns, operator_pronouns,
context_window_tokens, context_window_tokens,
agents, agents,
forge_base,
agent_canonical_inputs, agent_canonical_inputs,
) )
} }
@ -879,6 +935,7 @@ fn render_flake_with_lookup<F>(
operator_pronouns: &str, operator_pronouns: &str,
context_window_tokens: &std::collections::HashMap<String, u64>, context_window_tokens: &std::collections::HashMap<String, u64>,
agents: &[AgentSpec], agents: &[AgentSpec],
forge_base: &str,
lookup: F, lookup: F,
) -> String ) -> String
where where
@ -923,12 +980,19 @@ where
let _ = writeln!(out, " hyperhive-docs.url = \"{docs_flake}\";"); let _ = writeln!(out, " hyperhive-docs.url = \"{docs_flake}\";");
out.push_str(" hyperhive-docs.flake = false;\n"); out.push_str(" hyperhive-docs.flake = false;\n");
} }
// Each agent's *persistent* config input is its canonical repo on the
// forge (`git+{forge_base}/agent-configs/<name>.git`, `forge_base` supplied
// by the caller from `HIVE_FORGE_URL`), authenticated by hive-core's git
// credential helper (which reads the live `forge-core-token` — no token in
// the URL or lock). The deploy re-lock + `verify_commit` eval keep pinning
// the local `applied/<name>` override (`agent_input_override`), so a deploy
// never does a network fetch — only the persistent input tracks the forge.
for spec in agents { for spec in agents {
let _ = writeln!( let _ = writeln!(
out, out,
" agent-{}.url = \"git+file://{}\";", " agent-{name}.url = \"git+{forge_base}/{org}/{name}.git\";",
spec.name, name = spec.name,
crate::paths::applied_dir(&spec.name).display(), org = crate::forge::CONFIG_ORG,
); );
// For each canonical input the agent declares in its own // For each canonical input the agent declares in its own
// `flake.nix` (detected by reading its applied `flake.lock`), // `flake.nix` (detected by reading its applied `flake.lock`),
@ -1450,6 +1514,20 @@ mod tests {
); );
} }
#[test]
fn applied_override_url_targets_local_mirror_main_head() {
// Deploy + rebuild re-lock against the local applied mirror's `main`
// head (no `?rev`), never the persistent forge URL — so a rebuild
// survives forge unreachability and builds the fast-forwarded config.
let p = Path::new("/var/lib/hyperhive/agents/iris/applied");
let url = applied_override_url(p);
assert_eq!(url, "git+file:///var/lib/hyperhive/agents/iris/applied");
assert!(
!url.contains("?rev="),
"must lock main head, not a pinned rev: {url}"
);
}
#[test] #[test]
fn render_flake_uses_explicit_nixpkgs_url_when_provided() { fn render_flake_uses_explicit_nixpkgs_url_when_provided() {
let out = render_flake( let out = render_flake(
@ -1460,6 +1538,7 @@ mod tests {
"she/her", "she/her",
&std::collections::HashMap::new(), &std::collections::HashMap::new(),
&[sample_spec("alice", false, 9001)], &[sample_spec("alice", false, 9001)],
"http://forge.test",
); );
// nixpkgs is a top-level input with an explicit URL; hyperhive // nixpkgs is a top-level input with an explicit URL; hyperhive
// follows it. // follows it.
@ -1503,6 +1582,7 @@ mod tests {
"she/her", "she/her",
&std::collections::HashMap::new(), &std::collections::HashMap::new(),
&[sample_spec("alice", false, 9001)], &[sample_spec("alice", false, 9001)],
"http://forge.test",
); );
assert!( assert!(
!out.contains("hyperhive-docs"), !out.contains("hyperhive-docs"),
@ -1522,6 +1602,7 @@ mod tests {
"she/her", "she/her",
&std::collections::HashMap::new(), &std::collections::HashMap::new(),
&[sample_spec("alice", false, 9001)], &[sample_spec("alice", false, 9001)],
"http://forge.test",
); );
assert!( assert!(
out.contains("nixpkgs.follows = \"hyperhive/nixpkgs\""), out.contains("nixpkgs.follows = \"hyperhive/nixpkgs\""),
@ -1555,6 +1636,7 @@ mod tests {
sample_spec("bitburner", false, 9002), sample_spec("bitburner", false, 9002),
sample_spec("dmatrix", false, 9003), sample_spec("dmatrix", false, 9003),
], ],
"http://forge.test",
lookup, lookup,
); );
// bitburner declares nixpkgs → follows emitted. // bitburner declares nixpkgs → follows emitted.
@ -1583,6 +1665,7 @@ mod tests {
"she/her", "she/her",
&std::collections::HashMap::new(), &std::collections::HashMap::new(),
&[sample_spec("alice", false, 9001)], &[sample_spec("alice", false, 9001)],
"http://forge.test",
|_| Vec::new(), |_| Vec::new(),
); );
// No agent-side follows when the lookup reports nothing // No agent-side follows when the lookup reports nothing
@ -1617,6 +1700,7 @@ mod tests {
"she/her", "she/her",
&std::collections::HashMap::new(), &std::collections::HashMap::new(),
&[sample_spec("alice", false, 9001)], &[sample_spec("alice", false, 9001)],
"http://forge.test",
); );
unsafe { unsafe {
std::env::remove_var("HIVE_FORGE_URL"); std::env::remove_var("HIVE_FORGE_URL");
@ -1639,6 +1723,36 @@ mod tests {
); );
} }
#[test]
fn render_flake_agent_input_points_at_forge_config_repo() {
// The persistent agent config input must reference the canonical
// repo on the forge (git+http, org `agent-configs`), NOT the local
// `applied/<n>` checkout — that's what lets the config live on the
// forge instead of a hand-synced local copy. Auth is out-of-band via
// hive-core's git credential helper, so no creds appear in the URL.
// `forge_base` is an explicit param now, so no env mutation is needed.
let out = render_flake(
"github:example/hyperhive",
"path:/nix/store/bbbb-hyperhive-docs-source",
"path:/nix/store/aaaa-nixpkgs-source",
8000,
"she/her",
&std::collections::HashMap::new(),
&[sample_spec("alice", false, 9001)],
"http://forge.example.test",
);
assert!(
out.contains(
"agent-alice.url = \"git+http://forge.example.test/agent-configs/alice.git\""
),
"expected the agent input to point at the forge config repo:\n{out}"
);
assert!(
!out.contains("agent-alice.url = \"git+file://"),
"the local applied/<n> path must no longer be the persistent input:\n{out}"
);
}
#[test] #[test]
fn render_flake_embeds_hive_ca_when_signalled() { fn render_flake_embeds_hive_ca_when_signalled() {
// When hive-tls.nix signals a self-signed hive CA via // When hive-tls.nix signals a self-signed hive CA via
@ -1665,6 +1779,7 @@ mod tests {
"she/her", "she/her",
&std::collections::HashMap::new(), &std::collections::HashMap::new(),
&[sample_spec("alice", false, 9001)], &[sample_spec("alice", false, 9001)],
"http://forge.test",
) )
}; };
@ -1747,6 +1862,7 @@ mod tests {
"she/her", "she/her",
&std::collections::HashMap::new(), &std::collections::HashMap::new(),
&[sample_spec("alice", false, 9001)], &[sample_spec("alice", false, 9001)],
"http://forge.test",
) )
}; };
unsafe { unsafe {
@ -1831,6 +1947,7 @@ mod tests {
"she/her", "she/her",
&std::collections::HashMap::new(), &std::collections::HashMap::new(),
&[sample_spec("alice", false, 9001)], &[sample_spec("alice", false, 9001)],
"http://forge.test",
) )
}; };
unsafe { unsafe {

View file

@ -21,9 +21,33 @@ let
# only satisfies the ownership guard. libgit2 honours the literal `*` # only satisfies the ownership guard. libgit2 honours the literal `*`
# (mid-path globs aren't supported, so per-agent repos can't be listed); # (mid-path globs aren't supported, so per-agent repos can't be listed);
# in practice these processes only ever touch hyperhive's own repos. # in practice these processes only ever touch hyperhive's own repos.
#
# The agent config inputs now live on the forge
# (`git+http://${forge.domain}/agent-configs/<n>.git`, see meta.rs render),
# so hive-core's `nix flake update` of those inputs is an authenticated
# `git+http` fetch. The `[credential]` stanza points git at the `hive-forge`
# helper below (scoped to the forge host) so the fetch authenticates as the
# forge `core` user with no token in any URL or lock.
safeDirGitconfig = pkgs.writeText "hyperhive-safe-gitconfig" '' safeDirGitconfig = pkgs.writeText "hyperhive-safe-gitconfig" ''
[safe] [safe]
directory = * directory = *
[credential "http://${config.services.hyperhive.forge.domain}"]
helper = hive-forge
username = core
'';
# git credential helper for hive-core's authenticated fetches of the
# agent-config repos on the forge. Reads the live forge-core admin token
# (`/var/lib/hyperhive/forge-core-token` — paths.rs `FORGE_CORE_TOKEN`) on
# every invocation, so it never holds a stale copy and survives token
# rotation. Scoped to the forge host by the `[credential]` stanza above;
# implements the git credential-helper protocol (only `get` answers).
forgeCredHelper = pkgs.writeShellScriptBin "git-credential-hive-forge" ''
[ "''${1:-}" = "get" ] || exit 0
if [ -r /var/lib/hyperhive/forge-core-token ]; then
printf 'username=core\n'
printf 'password=%s\n' "$(cat /var/lib/hyperhive/forge-core-token)"
fi
''; '';
# The `hive-c0re serve` config JSON. Keys are snake_case to match the # The `hive-c0re serve` config JSON. Keys are snake_case to match the
@ -116,6 +140,9 @@ in
after = [ "hive-c0re.socket" ]; after = [ "hive-c0re.socket" ];
path = [ path = [
pkgs.git pkgs.git
# `git-credential-hive-forge` on PATH so git finds it when nix fetches
# the forge-hosted agent-config inputs (helper = hive-forge).
forgeCredHelper
"/run/current-system/sw" "/run/current-system/sw"
]; ];
environment = import ./environment.nix { inherit lib config pkgs; }; environment = import ./environment.nix { inherit lib config pkgs; };