diff --git a/docs/networking/network.md b/docs/networking/network.md index 9bfff2bfd..74ff49a98 100644 --- a/docs/networking/network.md +++ b/docs/networking/network.md @@ -45,7 +45,7 @@ that touches it. | container | netns | IPv4 | listens / reached via | | ------------------------- | ----------------------- | -------------- | ----------------------------------------------------------------------------------------------- | | gateway (host) | host | host addresses | nginx `:80`/`:443` (every vhost); dnsmasq `bridgeIp:53` + DHCP `:67` on the bridge | -| egress proxy (host) | host | `bridgeIp` | squid `bridgeIp:3128`; agents with a credential send HTTP and HTTPS through it | +| egress proxy (host) | host | `bridgeIp` | squid `bridgeIp:3128`; no agent uses it yet | | `hive-forge` | host (shared) | host addresses | forgejo `:3000` http, `:2222` git-ssh; fronted by the `forge.` vhost | | `hive-matrix` | private, veth on bridge | DHCP pool | tuwunel on UDS `/run/hive-matrix/tuwunel.sock` → the matrix vhost; out to store, SSO, peers | | `swarm-ci-runner` | private, veth on bridge | DHCP pool | outbound only (runner → forge); no inbound surface | @@ -212,11 +212,9 @@ DNS/gateway), so only expose services safe for any agent to reach. Every hive host runs squid in the host netns, listening on `:3128` (`nix/host-modules/hive-squid.nix`). The port is open on the bridge interface -only. An agent sends its outbound HTTP and HTTPS through it once its hive says -the proxy is ready and it has fetched its proxy credential (see _Agents and the -proxy_ below). Any other agent goes straight out through the NAT, as before, -and so does any program that ignores the proxy variables: nothing blocks direct -egress yet. +only. **No agent uses it yet**: none sends a credential, and agent traffic +still goes straight out through the NAT. The proxy goes first so you can check +its metrics and logs before anything reroutes. Squid filters on the destination hostname: a `CONNECT` target or a plain-HTTP host. It never decrypts TLS, so it holds no signing key. It caches nothing. @@ -235,42 +233,6 @@ host. It never decrypts TLS, so it holds no signing key. It caches nothing. A request without a valid credential gets a 407, once the destination checks below have passed it. -- **Agents and the proxy.** An agent uses the proxy only when its hive says - this host's squid is ready for it. `hive-c0re` puts two facts in every - container's `/etc/hyperhive-hive-facts`, only on a hive that holds a store - identity: `HIVE_SQUID_PROXY` (`services.hyperhive.squid.listen`, the bridge IP - and port) and `HIVE_EGRESS_NO_PROXY` (the direct hosts below). A hive that - predates them writes neither, and its agents go direct. - `hive-agent-proxy-credential.service` (`nix/agent-modules/egress-proxy.nix`) - runs in each agent's container, as the agent user, at start and every 10 - minutes. With both facts present it logs in to the secret store with the - agent's own certificate, reads `swarm/agents//proxy-credential` and - writes `/run/hive-agent-proxy-credential/proxy.env` (`0600`) with - `HTTPS_PROXY`, `HTTP_PROXY`, their lower-case forms, `NODE_USE_ENV_PROXY=1` - and `NO_PROXY`/`no_proxy`. The proxy URL is - `http://:@:3128`; Claude Code, Node, curl, git - and reqwest send the userinfo as `Proxy-Authorization`. The harness, its bash - tool, the subagents, the MCP servers and the pollers load the file with - `EnvironmentFile=-` when they start. Until the facts and the credential both - exist nothing writes the file, so those units have no proxy variable and go - direct; a failed read keeps the file already in place. Nothing restarts on - deploy: an agent takes the proxy at its next start, which is its next rebuild - or restart. The credential is in the environment of every process those units - start, so `env` in the agent's shell prints it. Login shells - (`nixos-container root-login`, SSH) don't load the file. -- **Direct destinations.** `HIVE_EGRESS_NO_PROXY` is the hosts this agent must - reach without the proxy, and `NO_PROXY` is `localhost,127.0.0.1,::1` followed - by that list. `hive-c0re` computes it from the host's options: the bridge IP, - the hive's domain (`services.hyperhive.domain`), the secret store and queue - names (`services.hyperhive.swarm.bao.domain`, `swarm.nats.domain`), - `services.hyperhive.swarm.serviceDomains` (the swarm UI and controller apex, - the forge, matrix, attic, logs, metrics and otel names) and every name the - gateway answers with the bridge IP (`services.hyperhive.gateway.localNames`). - Squid refuses every destination that resolves to the bridge, and the hive's - resolver answers the bridge address for those names, so they never go through - the proxy. A bare name matches itself and its subdomains. Node's `NO_PROXY` - has no CIDR support, so the list holds the bridge as an address. - - **Two lists.** The credential decides the policy. `hive-c0re` writes a second file beside the users file, `services.hyperhive.squid.unrestrictedFile` (`/var/lib/hive-squid/unrestricted`, `hive-core:squid`, `0640`): the names of diff --git a/docs/process/conventions.md b/docs/process/conventions.md index c876f2869..61bd31410 100644 --- a/docs/process/conventions.md +++ b/docs/process/conventions.md @@ -43,10 +43,10 @@ agent's build, and neither do the hive's other per-host values: the dashboard port (`HIVE_DASHBOARD_PORT`), the pinned claude-code (`HYPERHIVE_CLAUDE_CODE_PATH`), extra OTEL labels (`HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES`) and the queue's address -(`HIVE_AGENT_NATS_URL`, from `deploy.hive-controller.queue.agentNatsUrl`) and the egress proxy's address and direct hosts (`HIVE_SQUID_PROXY`, `HIVE_EGRESS_NO_PROXY`, see [the egress proxy](../networking/network.md#egress-proxy-squid)). `hive-priv` writes them to +(`HIVE_AGENT_NATS_URL`, from `deploy.hive-controller.queue.agentNatsUrl`). `hive-priv` writes them to `/etc/hyperhive-hive-facts` before every container start, leaving out any the hive doesn't set; the agent then uses its own default. The agent's environment -generator (`nix/agent-modules/hive-facts.nix`) passes all but the two egress facts, which only the proxy fetch reads, to every unit, +generator (`nix/agent-modules/hive-facts.nix`) passes them to every unit, including `hivectl choom` sessions, and `/etc/set-environment` exports them in login shells that no unit starts (`nixos-container root-login`). None is part of an agent's closure, and an agent started on another hive reports that hive. @@ -323,7 +323,7 @@ that value. Read it with `swarm_queue_client::agent_permissions::unrestricted`. The hive reads it for the agent's squid policy: [the egress proxy](../networking/network.md#egress-proxy-squid) lets an unrestricted agent's credential reach any destination, and holds the -others to the allow-list. An agent sends its credential once it has fetched it, from its next start. +others to the allow-list. No agent sends a credential yet. **How a change applies** — - The agent's harness reads its own key every 10 seconds and builds every diff --git a/docs/swarm/credentials-reference.md b/docs/swarm/credentials-reference.md index 57d3134ea..77657ec67 100644 --- a/docs/swarm/credentials-reference.md +++ b/docs/swarm/credentials-reference.md @@ -36,7 +36,7 @@ the rest of the cell says how. | `swarm/agents//claude-credentials` | `swarm-claude-login`, when you log the agent in from its card; one version per login | the agent, while it waits for a login | ❌ log in again | ✅ every 30 seconds while waiting | | `swarm/hives//matrix/sender-token` | `swarm-controller`, every 5 minutes | `swarm-controller` and the hive | ✅ when missing or rejected by the homeserver | ✅ the hive re-reads on every sweep | | `swarm/hives//forge/token` | `swarm-controller`, every 5 minutes | `swarm-controller` and the hive | ✅ when missing or no longer matching the forge | ✅ read on every forge call | -| `swarm/agents//proxy-credential` | `swarm-controller`: 32 random bytes, at start and every 5 minutes for every agent that holds a store identity | the hive (`hive-c0re`) and the agent, which fetches it | ❌ never replaced: written only while the path holds nothing | ✅ `hive-c0re` re-reads it every 30 seconds | +| `swarm/agents//proxy-credential` | `swarm-controller`: 32 random bytes, at start and every 5 minutes for every agent that holds a store identity | the hive (`hive-c0re`); the agent can read it, nothing sends it yet | ❌ never replaced: written only while the path holds nothing | ✅ `hive-c0re` re-reads it every 30 seconds | | `swarm/hives//squid-selfcheck` | `swarm-controller`, in the same pass as the agents' credentials | the hive: `hive-c0re` and `squid-selfcheck-credential` (root) | ❌ never replaced: written only while the path holds nothing | ✅ `hive-c0re` every 30 seconds, the root unit every 10 minutes | | `swarm/hives//oidc/client` | authelia, first boot; `swarm-secret-publish` copies it in | the hive | ❌ minted once; `swarm-secret-publish` republishes a re-mint | ✅ re-read when authelia refuses it | | `swarm/services//oidc/client` | authelia | that service | must be stated | must be stated | @@ -130,7 +130,6 @@ pieces fit, and the reasoning behind them, is in [`credentials.md`](credentials. | the swarm controller's forge token | the forge's `forgejo-swarm-controller-account`, in-container, once | the swarm secret store at `swarm/controller/swarm-controller/forge/token`, written by the mint under the forge publisher's leaf, which the forge container loads as a credential; the controller holds it in memory only. `/var/lib/forgejo/swarm-controller-token-name` in the forge container holds its name, the minter's record that it has minted | nothing to place beyond the controller's store leaf. A forge host without the store needs the publisher's leaf placed (`deploy.bao.forgePublisherClientCertFile`) | | the swarm controller's Grafana token | `swarm-bao-grafana-controller-token.service`, **on the host that runs Grafana**, as Grafana's built-in admin over its socket: it creates the `swarm-controller` service account as a Viewer (and sets it back to Viewer on every run), mints a token for it and writes it to the swarm secret store at `swarm/controller/swarm-controller/grafana/token` under Grafana's leaf. Re-run every ten minutes: it keeps a stored token Grafana accepts, and replaces one Grafana refuses with 401 (or a missing one), deleting every other token of that account first | the swarm secret store only; the controller reads it for `GET /api/alerts` and holds it in memory, re-reading the store once when Grafana answers 401 | nothing to place: Grafana's leaf is already required to run Grafana. Until the unit has run, `/api/alerts` answers 503 naming the empty path | | an agent's API-key environment (`ANTHROPIC_API_KEY`, `ANTHROPIC_BASE_URL`, an ACP provider key) | `hive-agent-backend-env.service`, in the agent's container, on an agent with a store that uses `useApiKey` or the `acp` runtime. When the store holds nothing at `swarm/agents//backend-env` and `backendEnvironmentFile` is a non-empty file, it creates the leaf with the file's exact bytes and `-cas=0`, under the agent's own leaf, which may create that path and not update it. It never rotates a value and never rewrites or deletes the file. To set or replace a key, use **set the API-key environment** in the swarm UI (`PUT /api/hives/{hive}/agents/{agent}/backend-env`): `swarm-controller` overwrites the leaf with its own grant, and the store's value then wins over the file. Its **remove** (`DELETE` on the same path) deletes the leaf, and an agent whose file has content then creates it from that file again | the swarm secret store at that path; the unit writes it to `/run/hive-agent-backend-env/env` (tmpfs, `0600`, root), and `hive-agent` and `hive-subagent-daemon` load that after `backendEnvironmentFile` as `EnvironmentFile=` | the same unit, same path. A variable in the store wins over the same one in the file. With the store down at a start, the file alone carries the agent, as it did before the store held a copy; with an older hive build the agent ignores the leaf and keeps reading the file | -| an agent's proxy settings (`HTTPS_PROXY`, `HTTP_PROXY`, `NO_PROXY`, the credential in the proxy URL) | `hive-agent-proxy-credential.service`, in the agent's container, as the agent user, at start and every 10 minutes: it reads `swarm/agents//proxy-credential` under the agent's own leaf and builds the file from it. It writes nothing while the path holds nothing or the hive's facts name no squid | `/run/hive-agent-proxy-credential/proxy.env` (tmpfs, `0600`, the agent user); the harness, its bash tool, the subagents, the MCP servers and the pollers load it as `EnvironmentFile=-` and read it when they start | With no credential, or on a hive that writes no `HIVE_SQUID_PROXY` and `HIVE_EGRESS_NO_PROXY` fact, nothing writes the file and a unit has no proxy variable and reaches the internet directly. A failed read keeps the file already in place. A unit started before the file appeared goes direct until its next start. The credential is also in the environment of every process those units start, so `env` in the agent's shell prints it | | an agent's claude login (`.credentials.json`) | `swarm-claude-login.service`, in its container, when an operator completes `claude auth login` from the agent's card in the swarm UI; it stores the file as a new version at `swarm/agents//claude-credentials` under its own leaf, which may create and update that path and read nothing under an agent | the swarm secret store at that path; the agent's `hive-agent` reads it under the agent's own leaf every 30 seconds while it waits for a login and writes it to `~/.claude/.credentials.json` | the same read, co-located or not; nothing for an operator to place beyond the agent's store leaf. When the store's copy and the agent's differ the KV version decides: the agent installs only a version newer than the last it installed (`claude-credentials-version` under the harness dir), because claude rotates the refresh token in its own copy and the stored one goes stale. The agent ignores a stale or equal version, and a relogin writes a newer one — [claude-login.md](claude-login.md) | | the queue responder's copy of its OIDC secret | the responder reads it out of the swarm secret store itself as it starts, under its own leaf, and again when authelia refuses it | the daemon's memory only; never on disk | same read, wherever authelia runs. Nothing for an operator to place beyond the responder's store leaf (`deploy.nats.authBaoClientCertFile`) | | the queue container's journal forwarder's copy of the queue's OIDC secret | `swarm-journal-forwarder-oidc.service` inside the `swarm-nats` container reads it out of the swarm secret store under the responder's store leaf, and again every 10 minutes | `/run/swarm-journal-forwarder-oidc/client-secret` on the container's tmpfs, root `0400`, never on persistent disk; `LoadCredential` hands it to the collector | same unit, same path. After a reboot the collector restarts until the file exists, then resumes from its saved journal cursor. Nothing for an operator to place beyond the responder's store leaf (`deploy.nats.authBaoClientCertFile`) | @@ -157,7 +156,7 @@ pieces fit, and the reasoning behind them, is in [`credentials.md`](credentials. | the forge's copy of its OIDC secret | `swarm-bao-forge-oidc.service` reads it out of the swarm secret store under the forge publisher's leaf, wherever authelia runs | `/run/hive-forge-oidc/.secret` on the host (tmpfs), bound into the forge container, handed to `forgejo-sso-source` through `LoadCredential` | | the homeserver's copy of its OIDC secret | `swarm-bao-matrix-oidc.service` reads it out of the swarm secret store under matrix-ctl's leaf, wherever authelia runs | `/run/hive-matrix-oidc/.secret` on the host (tmpfs), bound into the homeserver container, handed to tuwunel through `LoadCredential` | | this hive's forge read token | `swarm-controller`, into the secret store at `swarm/hives//forge/token`, for the `hive-` account; a pass at start and every five minutes re-mints a missing or stale token | `hive-c0re` reads it under the hive's own store identity, and copies it (account on the first line, token on the second) to `/run/hyperhive/forge-hive-token` (tmpfs, `0600` `hive-core`) for nix's `git`, which `hive-priv` also runs. Each `internal/meta` read and each boot rewrites the copy when it differs; where they differ after a rotation, a build that starts before the next read fails to authenticate, and a retry reads the new token | -| squid's users file | `hive-c0re` (`workers::squid_users`), every 30 seconds, from the proxy credentials of its agents and the self-check | `services.hyperhive.squid.usersFile` (`/var/lib/hive-squid/users`, `hive-core:squid`, `0640`): `:` lines, SHA-512 crypt hashes only, renamed into place when it changed. `unrestricted` beside it lists the users whose agent holds the swarm's `unrestricted_internet` flag, names only; it lists every agent the hive never read as off `systemd-tmpfiles` creates it empty so squid's helper never sees a missing file. A hive whose agents have no credential yet (an older controller) has an empty file: every login fails and the self-check reports `ok=false`; an agent without one sends none, so it goes direct | +| squid's users file | `hive-c0re` (`workers::squid_users`), every 30 seconds, from the proxy credentials of its agents and the self-check | `services.hyperhive.squid.usersFile` (`/var/lib/hive-squid/users`, `hive-core:squid`, `0640`): `:` lines, SHA-512 crypt hashes only, renamed into place when it changed. `unrestricted` beside it lists the users whose agent holds the swarm's `unrestricted_internet` flag, names only; it lists every agent the hive never read as off `systemd-tmpfiles` creates it empty so squid's helper never sees a missing file. A hive whose agents have no credential yet (an older controller) has an empty file: every login fails and the self-check reports `ok=false`; no agent sends a credential, so no traffic changes | | the squid self-check's password | `swarm-controller` mints `swarm/hives//squid-selfcheck`; `squid-selfcheck-credential` (root, every 10 minutes) reads it as the hive | `/run/credstore/hive-squid.selfcheck` (tmpfs, root, `0400`), imported by `squid-selfcheck.service` as a systemd credential and handed to `curl` on its stdin. Both self-check users (`selfcheck.squid` and the always-unrestricted `selfcheck-unrestricted.squid`) use it, so whoever holds it has unrestricted egress through this squid Missing, or different from the users file: the self-check logs `credential=absent` or a 407 and fails, and nothing else breaks | ### Store identities, one per principal diff --git a/docs/swarm/ui.md b/docs/swarm/ui.md index 66676fd7a..d468c5b84 100644 --- a/docs/swarm/ui.md +++ b/docs/swarm/ui.md @@ -569,9 +569,8 @@ the swarm never set the flag, which counts as on. The **restrict** or permission list along with the flag, because a `PUT` without the list resets a stored list to the default. An agent with no stored list sends the flag alone. The hive's egress proxy applies the flag to each agent's proxy credential (see -[the egress proxy](../networking/network.md#egress-proxy-squid)). An agent sends its -credential from its next start after it has fetched it, so a flag change applies -to an agent's traffic from then on. +[the egress proxy](../networking/network.md#egress-proxy-squid)); no agent sends +one yet, so the flag changes no traffic today. A saved change isn't instant. The hive reads the caps about every 30 seconds, and a running agent takes a changed cap at its next spawn or rebuild, so the diff --git a/docs/trust-boundary/security.md b/docs/trust-boundary/security.md index 06e7b1c2d..e44ec02d3 100644 --- a/docs/trust-boundary/security.md +++ b/docs/trust-boundary/security.md @@ -209,11 +209,6 @@ token policy bounds file reads; network isolation bounds network reach. nixbld users can't read it. The swarm-minted token lives under `/run/hive-agent-forge-token/` (`0700` directory, `0400` file), outside the state dir. -- `/run/hive-agent-proxy-credential/proxy.env` — the agent's egress-proxy - credential inside its proxy URL (`0700` directory, `0600` file, tmpfs). Units - load it as an environment file, so the credential is also in the environment of - every process they start. It authenticates this agent to the hive's squid and - nothing else. **Policy**: all credential files written to agent state directories MUST be mode `0600` or stricter. Don't create world-readable secret files in agent state dirs. diff --git a/hive-c0re/src/lifecycle/mod.rs b/hive-c0re/src/lifecycle/mod.rs index beb680609..fbe481f1e 100644 --- a/hive-c0re/src/lifecycle/mod.rs +++ b/hive-c0re/src/lifecycle/mod.rs @@ -141,8 +141,8 @@ pub fn network_isolation_from_env() -> Result } /// The hive's facts every container is handed at runtime, so that no agent -/// build carries a value that differs between hosts. The name, domain, egress -/// proxy and its direct hosts, OTEL resource labels and queue address come from hive-c0re's own unit environment +/// build carries a value that differs between hosts. The name, domain, OTEL +/// resource labels and queue address come from hive-c0re's own unit environment /// (`nix/host-modules/hive-c0re/environment.nix`), where an unset or empty /// variable is `None`; the dashboard port and pinned claude from `hive`. fn hive_facts(hive: &HiveEnv) -> hive_priv_sock::HiveFacts { @@ -162,8 +162,6 @@ fn hive_facts_from( claude_code_path: hive.claude_code_path.clone(), otel_extra_resource_attributes: read("HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES"), agent_nats_url: read("HIVE_AGENT_NATS_URL"), - squid_proxy: read("HIVE_SQUID_PROXY"), - egress_no_proxy: read("HIVE_EGRESS_NO_PROXY"), } } diff --git a/hive-c0re/src/lifecycle/tests.rs b/hive-c0re/src/lifecycle/tests.rs index 9a7ec33af..d638cdb22 100644 --- a/hive-c0re/src/lifecycle/tests.rs +++ b/hive-c0re/src/lifecycle/tests.rs @@ -541,8 +541,6 @@ fn hive_facts_carry_the_per_host_values() { let facts = hive_facts_from(&hive, |name| match name { "HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES" => Some("team=a".to_owned()), "HIVE_AGENT_NATS_URL" => Some("tls://nats.t.local:4222".to_owned()), - "HIVE_SQUID_PROXY" => Some("10.42.0.1:3128".to_owned()), - "HIVE_EGRESS_NO_PROXY" => Some("10.42.0.1,t.local".to_owned()), _ => None, }); assert_eq!(facts.dashboard_port, Some(7123)); @@ -558,8 +556,6 @@ fn hive_facts_carry_the_per_host_values() { facts.agent_nats_url.as_deref(), Some("tls://nats.t.local:4222") ); - assert_eq!(facts.squid_proxy.as_deref(), Some("10.42.0.1:3128")); - assert_eq!(facts.egress_no_proxy.as_deref(), Some("10.42.0.1,t.local")); } /// Unset and empty both leave a fact out, so the agent falls back to its @@ -572,7 +568,5 @@ fn hive_facts_leave_unset_values_out() { assert_eq!(facts.claude_code_path, None); assert_eq!(facts.otel_extra_resource_attributes, None); assert_eq!(facts.agent_nats_url, None); - assert_eq!(facts.squid_proxy, None); - assert_eq!(facts.egress_no_proxy, None); assert_eq!(facts.name, None); } diff --git a/hive-priv-sock/src/lib.rs b/hive-priv-sock/src/lib.rs index 9524b49f9..4b02dfc7e 100644 --- a/hive-priv-sock/src/lib.rs +++ b/hive-priv-sock/src/lib.rs @@ -243,14 +243,6 @@ pub struct HiveFacts { /// `HIVE_AGENT_NATS_URL`: where the agent reaches the swarm queue, from /// `deploy.hive-controller.queue.agentNatsUrl`. pub agent_nats_url: Option, - /// `HIVE_SQUID_PROXY`: `:` of this host's squid, which - /// checks each agent's proxy credential and applies the agent's - /// `unrestricted_internet` flag. Set only by a hive that carries that - /// policy; an agent sends nothing through a proxy without it. - pub squid_proxy: Option, - /// `HIVE_EGRESS_NO_PROXY`: the comma-separated hosts an agent must reach - /// directly and never through the squid above. - pub egress_no_proxy: Option, } /// Wall-clock bound hive-priv puts on one [`PrivRequest::MeasureAgentDisk`] diff --git a/hive-priv/src/container_config.rs b/hive-priv/src/container_config.rs index 6b79966e3..d9281702f 100644 --- a/hive-priv/src/container_config.rs +++ b/hive-priv/src/container_config.rs @@ -409,20 +409,6 @@ fn render_hive_facts(facts: &HiveFacts) -> Result { 253, &|b: u8| dns(b) || matches!(b, b':' | b'/'), ), - // `:`. - ( - "HIVE_SQUID_PROXY", - facts.squid_proxy.as_ref(), - 21, - &|b: u8| b.is_ascii_digit() || matches!(b, b'.' | b':'), - ), - // `host,host`: bare names and addresses. - ( - "HIVE_EGRESS_NO_PROXY", - facts.egress_no_proxy.as_ref(), - 4096, - &|b: u8| dns(b) || b == b',', - ), ] { let Some(value) = value else { continue }; if value.is_empty() || value.len() > max_len || !value.bytes().all(allowed) { @@ -817,59 +803,6 @@ mod tests { ); } - /// The egress facts: the proxy as `:`, and the hosts that skip it. - #[test] - fn hive_facts_render_the_egress_proxy() { - let facts = HiveFacts { - squid_proxy: Some("10.42.0.1:3128".to_owned()), - egress_no_proxy: Some("10.42.0.1,pr1ma.example.test,forge.example.test".to_owned()), - ..HiveFacts::default() - }; - assert_eq!( - render_hive_facts(&facts).unwrap(), - "HIVE_SQUID_PROXY=10.42.0.1:3128\n\ - HIVE_EGRESS_NO_PROXY=10.42.0.1,pr1ma.example.test,forge.example.test\n" - ); - } - - /// Neither egress fact may carry a second assignment, a space, a quote or a - /// URL: the agent builds its proxy URL and `NO_PROXY` from them. - #[test] - fn hive_facts_refuse_bad_egress_values() { - for bad in [ - "", - "10.42.0.1:3128\nHIVE_AGENT_NATS_URL=x", - "http://10.42.0.1:3128", - "10.42.0.1:3128 ", - "u:p@10.42.0.1:3128", - ] { - let facts = HiveFacts { - squid_proxy: Some(bad.to_owned()), - ..HiveFacts::default() - }; - assert!( - render_hive_facts(&facts).is_err(), - "{bad:?} must be refused" - ); - } - for bad in [ - "", - "a.test,b.test\nPATH=/x", - "a.test b.test", - "\"a.test\"", - "$(x)", - ] { - let facts = HiveFacts { - egress_no_proxy: Some(bad.to_owned()), - ..HiveFacts::default() - }; - assert!( - render_hive_facts(&facts).is_err(), - "{bad:?} must be refused" - ); - } - } - /// The claude path is a top-level store path or nothing, and neither /// value may carry generator syntax. #[test] diff --git a/nix/agent-modules/default.nix b/nix/agent-modules/default.nix index 9682b8b88..ca73c8d72 100644 --- a/nix/agent-modules/default.nix +++ b/nix/agent-modules/default.nix @@ -71,7 +71,6 @@ in ./claude-settings.nix ./dashboard-links.nix ./docs.nix - ./egress-proxy.nix ./forge.nix ./forge-accounts.nix ./forge-token.nix diff --git a/nix/agent-modules/egress-proxy.nix b/nix/agent-modules/egress-proxy.nix deleted file mode 100644 index d40d485de..000000000 --- a/nix/agent-modules/egress-proxy.nix +++ /dev/null @@ -1,240 +0,0 @@ -# This agent's own egress-proxy credential, fetched from the swarm secret store -# by the agent itself, and the proxy environment built from it. -# -# `swarm-controller` mints the credential at -# `swarm/agents//proxy-credential` -# (`swarm_secret_client::proxy::agent_credential_path`); the hive's squid checks -# it against the hash `hive-c0re` renders. This unit logs in with the -# certificate ./bao.nix keeps current, reads the agent's own path, and writes -# `proxy.env` for the units below to load as an `EnvironmentFile=`: -# -# - `HTTPS_PROXY` and `HTTP_PROXY`, upper and lower case, with the credential in -# the URL's userinfo, which Claude Code, Node, curl, git and reqwest send as -# `Proxy-Authorization`; -# - `NODE_USE_ENV_PROXY=1`, without which Node's global `fetch` ignores them; -# - `NO_PROXY` and `no_proxy`. -# -# The file is written only when the credential AND the hive's facts are both -# there. `HIVE_SQUID_PROXY` (the proxy's address) and `HIVE_EGRESS_NO_PROXY` (the -# hosts that skip it) are written by a hive whose squid checks credentials and -# applies each agent's flag; on any other hive the agent has no proxy variable -# and reaches the internet directly. The units load the file with a leading `-`, -# so an agent without it is unchanged. A failed read keeps the file already in -# place, and units read it when they start, so an agent takes the proxy at its -# next start. -# -# `NO_PROXY` is the loopback names and the hive's list as it is: squid refuses -# every destination that resolves to the bridge, and the hive's resolver answers -# the bridge address for the swarm's services. Node has no CIDR support, so the -# list holds the bridge as an address. -{ - pkgs, - lib, - config, - ... -}: -let - cfg = config.services.hyperhive.agent.bao; - agentCfg = config.services.hyperhive.agent; - - # The name `swarm-controller` minted the credential under — see ./queue-identity.nix. - agentName = agentCfg.user.name; - - # The server CA id ./bao.nix loads. - serverCaCredential = "hive-agent-bao-server-ca"; - - unitName = "hive-agent-proxy-credential"; - - # The nix half of `swarm_secret_client::proxy::agent_credential_path` plus - # `path::MOUNT`. - credentialPath = "secret/swarm/agents/${agentName}/proxy-credential"; - - runtimeDir = unitName; - envFile = "/run/${runtimeDir}/proxy.env"; - # Beside the file, so the rename that replaces it stays in one directory. - stagingFile = "/run/${runtimeDir}/proxy.env.new"; - secretFile = "/run/${runtimeDir}/secret.new"; - # bao's stderr, in the unit's own `0700` directory rather than `/tmp`. - errFile = "/run/${runtimeDir}/bao.err"; - - # Written by `hive-priv` into every container (./hive-facts.nix). The two - # egress facts are read here and nowhere else: the generator there exports - # only the keys it names, so neither reaches a unit's environment. - factsFile = "/etc/hyperhive-hive-facts"; - - loopback = [ - "localhost" - "127.0.0.1" - "::1" - ]; - - # The store's address is the whole switch, as in ./bao.nix and - # ./forge-token.nix. - configured = cfg.addr != null; - - storeRetry = import ../host-modules/lib/store-retry.nix { }; - - # Units that make outbound calls: the harness and the claude it starts, the - # bash tool and the subagents it runs, MCP servers, and the pollers. They - # exist under the same conditions as their definitions, because a unit that - # is not defined would be generated inert. - consumers = [ - "hive-agent" - "hive-bash-daemon" - "hive-subagent-daemon" - "hive-mcp-http" - "hive-forge-notify" - ] - ++ lib.optional (agentCfg.matrixAccounts != { }) "hive-matrix-daemon" - ++ lib.optional agentCfg.github.enable "hive-github-notify" - ++ lib.optional (agentCfg.icon != null && agentCfg.forge.url != null) "forge-avatar-sync"; -in -{ - config = lib.mkIf configured { - systemd.services = { - ${unitName} = { - description = "fetch this agent's egress-proxy credential from the secret store"; - after = [ - "network.target" - # Ordering only: that unit reports a store this agent cannot reach as - # itself. - "hive-agent-bao-identity.service" - ]; - wantedBy = [ "multi-user.target" ]; - path = [ - pkgs.openbao - pkgs.coreutils - pkgs.diffutils - ]; - # ../host-modules/lib/store-retry.nix. - inherit (storeRetry) startLimitBurst startLimitIntervalSec; - serviceConfig = storeRetry.serviceConfig // { - Type = "oneshot"; - # Not `RemainAfterExit`: the timer below has to be able to start this - # unit again. `RuntimeDirectoryPreserve` keeps the file between runs. - RemainAfterExit = false; - TimeoutStartSec = 30; - User = agentName; - Group = agentName; - RuntimeDirectory = runtimeDir; - RuntimeDirectoryMode = "0700"; - RuntimeDirectoryPreserve = "yes"; - UMask = "0177"; - LoadCredential = [ serverCaCredential ]; - }; - environment = { - BAO_ADDR = cfg.addr; - BAO_CLIENT_CERT = cfg.clientCertFile; - BAO_CLIENT_KEY = cfg.clientKeyFile; - }; - script = '' - set -euo pipefail - - # No identity delivered: ./bao.nix's check reports that. - for f in "$BAO_CLIENT_CERT" "$BAO_CLIENT_KEY"; do - if [ ! -s "$f" ]; then - echo "this agent has no store identity, so it cannot fetch its egress-proxy credential." >&2 - exit 0 - fi - if [ ! -r "$f" ]; then - echo "cannot read $f, so this agent cannot present its store identity." >&2 - exit 1 - fi - done - - # The hive says its squid checks each agent's credential and applies - # its flag by writing both facts. Without them (a hive that predates - # them) there is no proxy to name, and the store is not asked. - proxy= direct= - if [ -r ${factsFile} ]; then - while IFS='=' read -r key value || [ -n "$key" ]; do - case "$key" in - HIVE_SQUID_PROXY) proxy=$value ;; - HIVE_EGRESS_NO_PROXY) direct=$value ;; - esac - done < ${factsFile} - fi - if [[ ! "$proxy" =~ ^[0-9]+(\.[0-9]+){3}:[0-9]+$ || ! "$direct" =~ ^[A-Za-z0-9.,_-]+$ ]]; then - echo "this hive's facts name no squid proxy, so this agent stays direct." >&2 - exit 0 - fi - - if [ -s "$CREDENTIALS_DIRECTORY/${serverCaCredential}" ]; then - export BAO_CACERT="$CREDENTIALS_DIRECTORY/${serverCaCredential}" - fi - - err=${lib.escapeShellArg errFile} - trap 'rm -f "$err" ${lib.escapeShellArg stagingFile} ${lib.escapeShellArg secretFile}' EXIT - - # 🩸 Every exit below that is not a rename leaves ${envFile} as it - # was: a store that is briefly unreachable, or has no credential for - # this agent yet, must not take a working proxy away, and must not - # invent one. - rm -f "$err" - if ! BAO_TOKEN="$(bao login -method=cert -token-only 2>"$err")"; then - echo "could not log in to the swarm secret store at $BAO_ADDR; the proxy settings in place, if any, are kept:" >&2 - if [ -s "$err" ]; then cat "$err" >&2; fi - exit 1 - fi - export BAO_TOKEN - - # ⚠️ Written by redirect, never echoed: the field is the secret. - rm -f "$err" ${lib.escapeShellArg secretFile} - if ! bao kv get -field=value ${lib.escapeShellArg credentialPath} > ${lib.escapeShellArg secretFile} 2>"$err"; then - echo "no egress-proxy credential at ${credentialPath} yet; this agent sends no proxy variables until there is one." >&2 - if [ -s "$err" ]; then cat "$err" >&2; fi - exit 0 - fi - secret="$(<${lib.escapeShellArg secretFile})" - - # The controller mints base64url, which needs no escaping in a URL's - # userinfo or an env file. Anything else is not that secret. - if [[ ! "$secret" =~ ^[A-Za-z0-9_-]+$ ]]; then - echo "the value at ${credentialPath} is not a base64url secret, so no proxy variables are written." >&2 - exit 1 - fi - - no_proxy=${lib.escapeShellArg (lib.concatStringsSep "," loopback)},$direct - url="http://${agentName}:$secret@$proxy" - - rm -f ${lib.escapeShellArg stagingFile} - { - printf 'HTTPS_PROXY=%s\n' "$url" - printf 'HTTP_PROXY=%s\n' "$url" - printf 'https_proxy=%s\n' "$url" - printf 'http_proxy=%s\n' "$url" - printf 'NODE_USE_ENV_PROXY=1\n' - printf 'NO_PROXY=%s\n' "$no_proxy" - printf 'no_proxy=%s\n' "$no_proxy" - } > ${lib.escapeShellArg stagingFile} - - if cmp -s ${lib.escapeShellArg stagingFile} ${lib.escapeShellArg envFile}; then - echo "this agent's proxy settings are unchanged." - exit 0 - fi - mv -f ${lib.escapeShellArg stagingFile} ${lib.escapeShellArg envFile} - echo "wrote this agent's proxy settings from ${credentialPath}." - ''; - }; - } - // lib.genAttrs consumers (_: { - # `wants`, not `requires`: a store outage must not stop the harness, which - # then starts direct. `after` waits for one attempt. - wants = [ "${unitName}.service" ]; - after = [ "${unitName}.service" ]; - # A list, which merges with the entries other modules give a unit. - serviceConfig.EnvironmentFile = [ "-${envFile}" ]; - }); - - # The controller re-checks every agent's credential every five minutes; the - # fetch picks one up within about ten more. - systemd.timers.${unitName} = { - description = "re-fetch this agent's egress-proxy credential from the secret store"; - wantedBy = [ "timers.target" ]; - timerConfig = { - OnUnitInactiveSec = "10min"; - RandomizedDelaySec = "1min"; - }; - }; - }; -} diff --git a/nix/agent-modules/hive-facts.nix b/nix/agent-modules/hive-facts.nix index db6dc1b75..e9010ebf0 100644 --- a/nix/agent-modules/hive-facts.nix +++ b/nix/agent-modules/hive-facts.nix @@ -12,8 +12,6 @@ # `OTEL_RESOURCE_ATTRIBUTES` by ./otel.nix (no extra labels when absent). # - `HIVE_AGENT_NATS_URL`, where the harness reaches the swarm queue (it fails # at boot when absent). -# - `HIVE_SQUID_PROXY` and `HIVE_EGRESS_NO_PROXY`: the squid address and the hosts -# that skip it, read only by ./egress-proxy.nix; the generator drops them. # # Any of them may be absent: a hive-c0re older than the agent's build writes # fewer. Two readers hand the lines on: diff --git a/nix/checks.nix b/nix/checks.nix index 6ff29e1de..1d194d25a 100644 --- a/nix/checks.nix +++ b/nix/checks.nix @@ -183,8 +183,7 @@ in inherit (pkgs) lib; }; # Executes what the two `module-eval-agent-*-bao` suites above only - # evaluate: the rendered scripts of the forge-token, identity and - # proxy-credential fetch units, against a stub `bao`. + # evaluate: both fetch units' rendered scripts, against a stub `bao`. script-test-agent-bao-fetch = import ./script-tests/agent-bao-fetch.nix { inherit pkgs self nixosSystem; inherit (pkgs) lib; diff --git a/nix/host-modules/hive-c0re/environment.nix b/nix/host-modules/hive-c0re/environment.nix index 867245ef7..d342f9fb1 100644 --- a/nix/host-modules/hive-c0re/environment.nix +++ b/nix/host-modules/hive-c0re/environment.nix @@ -15,30 +15,6 @@ let # sharing a host with the store. Deriving this from # `deploy.bao.enable` would be the co-location assumption itself. haveBaoClientIdentity = baoDeploy.clientCertFile != null && baoDeploy.clientKeyFile != null; - - # What an agent reaches directly and never through squid. Squid refuses every - # destination that resolves to the bridge, and the hive's resolver answers the - # bridge address for the gateway's names, so each of these must skip it. A bare - # name matches itself and its subdomains in curl, Node and reqwest. - egressDirectHosts = lib.unique ( - lib.filter (n: n != null && n != "") ( - [ - # Node's NO_PROXY has no CIDR support, so the bridge is listed as the address. - config.services.hyperhive.network.bridgeIp - # The hive's own zone, which dnsmasq answers with the bridge IP. - config.services.hyperhive.domain - # The mTLS store and the queue: not in `serviceDomains`. - config.services.hyperhive.swarm.bao.domain - config.services.hyperhive.swarm.nats.domain - ] - # The swarm's apex (the swarm UI and the controller) and its forge, matrix, - # attic, logs, metrics and otel services, wherever they sit. The store and - # queue names are not in it, hence their lines above. - ++ config.services.hyperhive.swarm.serviceDomains - # Every name this host's gateway answers with the bridge IP. - ++ config.services.hyperhive.gateway.localNames - ) - ); in { # nix (the prebuild `nix build`, flake-check, and meta eval) writes @@ -301,13 +277,6 @@ in # store under the identity above. HIVE_SQUID_USERS_FILE = config.services.hyperhive.squid.usersFile; HIVE_SQUID_UNRESTRICTED_FILE = config.services.hyperhive.squid.unrestrictedFile; - # The egress proxy facts every container is handed (`HiveFacts` in - # `hive-priv-sock`). They sit with the users file above because that is what - # makes squid's credential check and per-agent policy real on this host: an - # agent sends nothing through the proxy unless it reads both. Absent on a - # hive that predates them, so its agents go direct. - HIVE_SQUID_PROXY = config.services.hyperhive.squid.listen; - HIVE_EGRESS_NO_PROXY = lib.concatStringsSep "," egressDirectHosts; } // lib.optionalAttrs (haveBaoClientIdentity && baoDeploy.serverCaFile != null) { # Absent means the system trust store — right for a deployment with a real diff --git a/nix/host-modules/hive-squid.nix b/nix/host-modules/hive-squid.nix index 0875d9ee4..7f1e107ed 100644 --- a/nix/host-modules/hive-squid.nix +++ b/nix/host-modules/hive-squid.nix @@ -4,9 +4,9 @@ # must carry a per-agent proxy credential, checked against the users file # hive-c0re renders. A credential's agent either has the swarm's # `unrestricted_internet` flag, and may then reach any destination the deny -# rules leave, or reaches only the listed names. An agent sends its credential -# once it has fetched it (../agent-modules/egress-proxy.nix); the exporter, -# journal forwarding and the self-check below show the proxy works. +# rules leave, or reaches only the listed names. No agent sends a credential +# yet; the exporter, journal forwarding and the self-check below are how a +# deploy shows it works. { pkgs, lib, @@ -19,7 +19,6 @@ let netCfg = hyperhiveCfg.network; port = 3128; - listen = "${netCfg.bridgeIp}:${toString port}"; proxyUrl = "http://${netCfg.bridgeIp}:${toString port}"; exporterListen = "127.0.0.1:9301"; denyCanary = "deny-canary.invalid"; @@ -68,7 +67,7 @@ let # port, and every other user to the listed names. The `dst` denies cost an # unlisted name a lookup on the host's resolver before it is refused. squidConf = '' - http_port ${listen} + http_port ${netCfg.bridgeIp}:${toString port} pid_filename /run/squid/squid.pid cache_log stdio:/var/log/squid/cache.log coredump_dir /var/cache/squid @@ -198,19 +197,6 @@ let in { options.services.hyperhive.squid = { - listen = lib.mkOption { - type = lib.types.str; - default = listen; - readOnly = true; - defaultText = lib.literalMD "`:3128`"; - description = '' - The address squid listens on, as `:`. Read-only. - hive-c0re hands it to every container as the `HIVE_SQUID_PROXY` fact, - which is what tells an agent that this host's squid checks its proxy - credential and applies its `unrestricted_internet` flag. - ''; - }; - allowedDomains = lib.mkOption { type = lib.types.listOf lib.types.str; default = hyperhiveCfg.swarm.serviceDomains; diff --git a/nix/module-eval/agent-forge-bao.nix b/nix/module-eval/agent-forge-bao.nix index 04fe2cb0c..8f51ebc8a 100644 --- a/nix/module-eval/agent-forge-bao.nix +++ b/nix/module-eval/agent-forge-bao.nix @@ -3,9 +3,7 @@ # not executes"). # # The agent side of the swarm-minted forge token: ../agent-modules/forge-token.nix -# fetches it, and ../agent-modules/forge.nix's readers find it. The same fixtures -# cover ../agent-modules/egress-proxy.nix, the agent's proxy credential and the -# environment built from it. +# fetches it, and ../agent-modules/forge.nix's readers find it. { pkgs, lib, @@ -33,8 +31,6 @@ let agentForgeBao = agentWith { services.hyperhive.agent.bao.addr = baoAddr; services.hyperhive.agent.forge.url = forgeUrl; - # A second `EnvironmentFile=` on the harness, which the proxy file must join. - services.hyperhive.agent.backendEnvironmentFile = "/var/lib/t/backend.env"; services.hyperhive.agent.icon = pkgs.emptyFile; }; @@ -46,22 +42,8 @@ let }; fetchUnit = machine: machine.systemd.services.hive-agent-forge-token; - proxyUnit = machine: machine.systemd.services.hive-agent-proxy-credential; accountsUnit = machine: machine.systemd.services.hive-agent-forge-accounts; tokenFile = machine: machine.services.hyperhive.agent.forge.tokenFile; - - proxyEnvEntry = "-/run/hive-agent-proxy-credential/proxy.env"; - # The units that make outbound calls and so load the proxy file. This fixture - # has an icon, a forge and GitHub on, and no matrix accounts. - proxyConsumers = [ - "hive-agent" - "hive-bash-daemon" - "hive-subagent-daemon" - "hive-mcp-http" - "hive-forge-notify" - "hive-github-notify" - "forge-avatar-sync" - ]; in let cases = [ @@ -256,137 +238,6 @@ let name = "the account fetch re-runs every two minutes"; ok = agentForgeBao.systemd.timers.hive-agent-forge-accounts.timerConfig.OnUnitInactiveSec == "2min"; } - { - # ../agent-modules/egress-proxy.nix. Presence arm: the unit and its timer - # exist, and the agent is told nothing until a credential is fetched. - name = "an agent with a store address fetches its egress-proxy credential, and one without does not"; - ok = - agentForgeBao.systemd.services ? hive-agent-proxy-credential - && agentForgeBao.systemd.timers ? hive-agent-proxy-credential - && !(agentForgeNoBao.systemd.services ? hive-agent-proxy-credential) - && !(agentForgeNoBao.systemd.timers ? hive-agent-proxy-credential); - } - { - # The nix half of `swarm_secret_client::proxy::agent_credential_path` - # plus the mount. - name = "the proxy fetch reads the agent's own proxy-credential path, by redirect, and never echoes it"; - ok = - let - name = agentForgeBao.services.hyperhive.agent.user.name; - s = (proxyUnit agentForgeBao).script; - in - lib.hasInfix "secret/swarm/agents/${name}/proxy-credential" s - && lib.hasInfix "-field=value" s - && lib.hasInfix "> ${lib.escapeShellArg "/run/hive-agent-proxy-credential/secret.new"}" s - && !(lib.hasInfix "echo \"$(bao" s) - && !(lib.hasInfix "echo $(bao" s) - && !(lib.hasInfix "echo \"$secret" s) - && !(lib.hasInfix "printf '%s\\n' \"$secret" s); - } - { - name = "the proxy fetch runs as the agent with its own store identity and keeps its file between runs"; - ok = - let - u = proxyUnit agentForgeBao; - c = u.serviceConfig; - in - c.User == agentForgeBao.services.hyperhive.agent.user.name - && c.UMask == "0177" - && !c.RemainAfterExit - && c.RuntimeDirectory == "hive-agent-proxy-credential" - && c.RuntimeDirectoryMode == "0700" - && c.RuntimeDirectoryPreserve == "yes" - && u.environment.BAO_ADDR == baoAddr - && u.environment.BAO_CLIENT_CERT == "/run/hive-agent-bao-identity/current/cert"; - } - { - # Presence: every outbound unit loads the file, and the harness keeps the - # entry it already had. Absence: an agent with no store loads none. - name = "every outbound unit loads the proxy file optionally, beside its other environment files"; - ok = - let - svc = agentForgeBao.systemd.services; - files = c: svc.${c}.serviceConfig.EnvironmentFile or [ ]; - in - lib.all ( - c: - lib.elem proxyEnvEntry (files c) - && lib.elem "hive-agent-proxy-credential.service" svc.${c}.after - && lib.elem "hive-agent-proxy-credential.service" svc.${c}.wants - && !(lib.elem "hive-agent-proxy-credential.service" (svc.${c}.requires or [ ])) - ) proxyConsumers - && lib.elem "-/var/lib/t/backend.env" (files "hive-agent") - && !(lib.any - ( - c: - lib.elem proxyEnvEntry (agentForgeNoBao.systemd.services.${c}.serviceConfig.EnvironmentFile or [ ]) - ) - [ - "hive-agent" - "hive-bash-daemon" - "hive-forge-notify" - ] - ); - } - { - # The credential is in a file only the fetch writes. Nothing static (a - # unit environment, the global one, the login shells) carries a proxy - # variable, so an agent without a fetched credential goes direct. - name = "no proxy variable is set where the credential is not"; - ok = - let - names = [ - "HTTPS_PROXY" - "HTTP_PROXY" - "https_proxy" - "http_proxy" - "NO_PROXY" - "no_proxy" - "NODE_USE_ENV_PROXY" - ]; - anywhere = - m: n: - m.systemd.globalEnvironment ? ${n} - || m.environment.variables ? ${n} - || lib.any (s: s.environment ? ${n}) (lib.attrValues m.systemd.services); - in - !(lib.any (anywhere agentForgeBao) names) && !(lib.any (anywhere agentForgeNoBao) names); - } - { - # The hive's facts are the only source of the proxy address and of the - # hosts that skip it, and they are read before the store is asked: a hive - # without them gets no login and no file. The loopback names stay local. - name = "the proxy file takes the proxy and the direct hosts from the hive's facts, before it asks the store"; - ok = - let - s = (proxyUnit agentForgeBao).script; - at = needle: lib.stringLength (lib.head (lib.splitString needle s)); - in - lib.hasInfix "HIVE_SQUID_PROXY) proxy=$value" s - && lib.hasInfix "HIVE_EGRESS_NO_PROXY) direct=$value" s - && lib.hasInfix "no_proxy=localhost,127.0.0.1,::1,$direct" s - && lib.hasInfix "/etc/hyperhive-hive-facts" s - && at "HIVE_SQUID_PROXY)" < at "bao login" - && !(lib.hasInfix "hyperhive-bridge-dns" s); - } - { - name = "the proxy file names every variable the clients read, and the credential only in the URL's userinfo"; - ok = - let - s = (proxyUnit agentForgeBao).script; - name = agentForgeBao.services.hyperhive.agent.user.name; - in - lib.all (v: lib.hasInfix "printf '${v}=%s\\n'" s) [ - "HTTPS_PROXY" - "HTTP_PROXY" - "https_proxy" - "http_proxy" - "NO_PROXY" - "no_proxy" - ] - && lib.hasInfix "NODE_USE_ENV_PROXY=1" s - && lib.hasInfix "url=\"http://${name}:$secret@$proxy\"" s; - } ]; in runGroup "agent-forge-bao" cases diff --git a/nix/module-eval/agent-runtime.nix b/nix/module-eval/agent-runtime.nix index 308f4b4f7..8f9c00937 100644 --- a/nix/module-eval/agent-runtime.nix +++ b/nix/module-eval/agent-runtime.nix @@ -167,15 +167,11 @@ let "-${backendEnv}" "-/run/hive-agent-backend-env/env" ]; - # The proxy file is loaded too (./egress-proxy.nix); its own cases - # cover it, and the order that matters here is the backend's. - backendFiles = - u: lib.remove "-/run/hive-agent-proxy-credential/proxy.env" u.serviceConfig.EnvironmentFile; in - backendFiles (harness claudeApiKeyBao) == want - && backendFiles (subagent claudeApiKeyBao) == want - && backendFiles (harness acpBao) == [ "-/run/hive-agent-backend-env/env" ] - && backendFiles (subagent acpBao) == [ "-/run/hive-agent-backend-env/env" ]; + (harness claudeApiKeyBao).serviceConfig.EnvironmentFile == want + && (subagent claudeApiKeyBao).serviceConfig.EnvironmentFile == want + && (harness acpBao).serviceConfig.EnvironmentFile == [ "-/run/hive-agent-backend-env/env" ] + && (subagent acpBao).serviceConfig.EnvironmentFile == [ "-/run/hive-agent-backend-env/env" ]; } { # The case above is the presence control: the same lookup finds the diff --git a/nix/module-eval/hive-otel.nix b/nix/module-eval/hive-otel.nix index 6bf1e61f6..942945944 100644 --- a/nix/module-eval/hive-otel.nix +++ b/nix/module-eval/hive-otel.nix @@ -622,68 +622,6 @@ let builtins.elem 3128 fw.interfaces.${hiveOtel.services.hyperhive.network.bridgeName}.allowedTCPPorts && !(builtins.elem 3128 fw.allowedTCPPorts); } - { - # The facts that tell an agent this host's squid checks its credential and - # applies its flag. Presence arm: a hive holding a store identity writes - # both, and the direct hosts carry the names the brief's destinations - # resolve under. Absence arm: a hive with no identity renders no users - # file, so it announces no proxy. - name = "hive-c0re hands agents the squid address and the hosts that skip it, only with a store identity"; - ok = - let - e = hiveOtelAllLocal.systemd.services.hive-c0re.environment; - s = hiveOtelAllLocal.services.hyperhive; - hosts = lib.splitString "," e.HIVE_EGRESS_NO_PROXY; - in - e.HIVE_SQUID_PROXY == "${s.network.bridgeIp}:3128" - && e.HIVE_SQUID_PROXY == s.squid.listen - && lib.all (h: lib.elem h hosts) ( - [ - s.network.bridgeIp - s.swarm.domain - s.swarm.bao.domain - s.swarm.nats.domain - s.swarm.attic.domain - s.swarm.forge.domain - s.swarm.matrix.gatewayHost - s.swarm.victorialogs.domain - s.swarm.victoriametrics.domain - s.swarm.otel.domain - s.otel.collector.host - s.domain - ] - ++ s.gateway.localNames - ) - && !(lib.elem "example.com" hosts) - && lib.length hosts == lib.length (lib.unique hosts) - && !(hiveOtelBare.systemd.services.hive-c0re.environment ? HIVE_SQUID_PROXY) - && !(hiveOtelBare.systemd.services.hive-c0re.environment ? HIVE_EGRESS_NO_PROXY); - } - { - # The store and the queue are in no list the gateway builds, so a hive whose - # store is elsewhere gets them from their own options. Control: this - # fixture runs no store, so the gateway does not name it. - name = "a hive without a local store still lists the store and the queue as direct hosts"; - ok = - let - s = hiveOtelLeaf.services.hyperhive; - hosts = lib.splitString "," hiveOtelLeaf.systemd.services.hive-c0re.environment.HIVE_EGRESS_NO_PROXY; - in - !(lib.elem s.swarm.bao.domain s.gateway.localNames) - && lib.elem s.swarm.bao.domain hosts - && lib.elem s.swarm.nats.domain hosts; - } - { - # Squid's listener and the fact are one value. - name = "the squid fact is the address squid binds"; - ok = - let - s = hiveOtelAllLocal.services.hyperhive; - in - lib.elem "http_port ${s.squid.listen}" ( - lib.splitString "\n" hiveOtelAllLocal.services.squid.configText - ); - } ]; in runGroup "hive-otel" cases diff --git a/nix/script-tests/agent-bao-fetch.nix b/nix/script-tests/agent-bao-fetch.nix index 65d7901ef..8e4a93123 100644 --- a/nix/script-tests/agent-bao-fetch.nix +++ b/nix/script-tests/agent-bao-fetch.nix @@ -1,7 +1,7 @@ # `checks.script-test-agent-bao-fetch` — runs the agent units that log in to -# the swarm secret store and fetch a secret, ../agent-modules/forge-token.nix, -# ../agent-modules/bao.nix and ../agent-modules/egress-proxy.nix, against a stub -# `bao`. The cases are in ./agent-bao-fetch.sh. +# the swarm secret store and fetch a secret, ../agent-modules/forge-token.nix +# and ../agent-modules/bao.nix, against a stub `bao`. The cases are in +# ./agent-bao-fetch.sh. # # What runs is the unit's rendered `ExecStart`, on the unit's own `PATH` with # the stub in front. The one edit is the unit's `/run//` prefix, moved @@ -77,7 +77,6 @@ pkgs.runCommand "hyperhive-script-test-agent-bao-fetch" ( unitEnv "FORGE" "hive-agent-forge-token" // unitEnv "IDENT" "hive-agent-bao-identity" - // unitEnv "PROXY" "hive-agent-proxy-credential" // { FAKE_BAO_BIN = "${fakeBao}/bin"; } diff --git a/nix/script-tests/agent-bao-fetch.sh b/nix/script-tests/agent-bao-fetch.sh index 4082e71ca..74ad8476a 100644 --- a/nix/script-tests/agent-bao-fetch.sh +++ b/nix/script-tests/agent-bao-fetch.sh @@ -1,5 +1,5 @@ # Cases for ./agent-bao-fetch.nix. Each case gets a fresh runtime directory and -# credentials directory, runs one unit's script under the unit's own `UMask=` +# credentials directory, runs one unit's script under the unit's `UMask=0377` # with a clean environment, and asserts on the exit code, the output, the # files left behind and the `bao` calls made. set -uo pipefail @@ -48,13 +48,7 @@ setup() { client_cert=$creds/hive-agent-bao-cert client_key=$creds/hive-agent-bao-key fi : >"$log" - # The proxy fetch reads the egress facts the hive writes into a - # container, moved under the build directory like /run. - mkdir "$dir/etc" - printf 'HIVE_DASHBOARD_PORT=7000\nHIVE_SQUID_PROXY=10.42.0.1:3128\nHIVE_EGRESS_NO_PROXY=10.42.0.1,t.hive,bao.t.local\n' >"$dir/etc/hyperhive-hive-facts" - umask_value=0377 - if [ "$prefix" = PROXY ]; then umask_value=0177; fi - if ! sed -e "s|/run/$unit/|$rt/|g" -e "s|/etc/hyperhive-|$dir/etc/hyperhive-|g" "${!script_var}" >"$dir/script"; then + if ! sed "s|/run/$unit/|$rt/|g" "${!script_var}" >"$dir/script"; then echo "cannot read the rendered script for $unit" >&2 exit 1 fi @@ -74,7 +68,7 @@ setup() { go() { count=$((count + 1)) ( - umask "$umask_value" + umask 0377 cd "$dir" || exit 99 exec env -i \ PATH="$dir/bin:$FAKE_BAO_BIN:$unit_path" \ @@ -388,177 +382,6 @@ for n in 1 2 3; do expect_pair cert2 key2 done -# The proxy fetch writes the agent's proxy variables, credential included, to -# one file the units load as an `EnvironmentFile=`. Nothing here may print the -# credential, so every case checks the output for it. -proxy_url=http://a1:the-secret@10.42.0.1:3128 -proxy_no=localhost,127.0.0.1,::1,10.42.0.1,t.hive,bao.t.local -proxy_path=secret/swarm/agents/a1/proxy-credential -proxy_kv_call="kv get -field=value $proxy_path" -proxy_env() { - printf 'HTTPS_PROXY=%s\nHTTP_PROXY=%s\nhttps_proxy=%s\nhttp_proxy=%s\nNODE_USE_ENV_PROXY=1\nNO_PROXY=%s\nno_proxy=%s\n' \ - "$proxy_url" "$proxy_url" "$proxy_url" "$proxy_url" "$1" "$1" -} - -expect_secret_unprinted() { - if grep -qF -e the-secret -e s.fake-token "$dir/out" "$dir/err"; then - fail "a secret reached the unit's output: $(cat "$dir/out" "$dir/err")" - fi -} - -setup PROXY "no store identity delivered" -: >"$creds/hive-agent-bao-cert" -go -expect_rc 0 -expect_err "has no store identity" -expect_calls -expect_no_file "$rt/proxy.env" - -setup PROXY "a first fetch writes the variables 0600, and prints no secret" -go -expect_rc 0 -expect_out "wrote this agent's proxy settings from $proxy_path" -expect_calls "$login_call" "$proxy_kv_call" -expect_file "$rt/proxy.env" "$(proxy_env "$proxy_no")" -if [ "$(stat -c %a "$rt/proxy.env")" != 600 ]; then fail "proxy.env is mode $(stat -c %a "$rt/proxy.env"), expected 600"; fi -expect_no_file "$rt/proxy.env.new" -expect_no_file "$rt/secret.new" -expect_no_file "$rt/bao.err" -expect_secret_unprinted - -setup PROXY "no credential minted yet writes nothing" -kv_rc=2 -kv_err="No value found at secret/data/swarm/agents/a1" -go -expect_rc 0 -expect_err "no egress-proxy credential at $proxy_path yet" -expect_calls "$login_call" "$proxy_kv_call" -expect_no_file "$rt/proxy.env" -expect_no_file "$rt/proxy.env.new" -expect_no_file "$rt/secret.new" - -setup PROXY "no credential minted yet leaves the file in place" -proxy_env old.example >"$rt/proxy.env" -chmod 0600 "$rt/proxy.env" -kv_rc=2 -go -expect_rc 0 -expect_file "$rt/proxy.env" "$(proxy_env old.example)" - -setup PROXY "a store that refuses the login leaves the file in place" -proxy_env old.example >"$rt/proxy.env" -login_rc=2 -login_err=$'Error authenticating: Error making API request.\n\nCode: 503. Errors:\n\n* Vault is sealed\n' -go -expect_rc 1 -expect_err "could not log in to the swarm secret store" -expect_err "Vault is sealed" -expect_calls "$login_call" -expect_file "$rt/proxy.env" "$(proxy_env old.example)" - -setup PROXY "a store that does not answer leaves the file in place" -proxy_env old.example >"$rt/proxy.env" -login_rc=2 -login_err="Error authenticating: Put \"$bao_addr/v1/auth/cert/login\": dial tcp: lookup bao.t.local: no such host" -go -expect_rc 1 -expect_file "$rt/proxy.env" "$(proxy_env old.example)" - -setup PROXY "0600 files already in place do not block the fetch" -printf stale >"$rt/proxy.env.new" -printf stale >"$rt/secret.new" -printf stale >"$rt/bao.err" -go -expect_rc 0 -expect_no_err "Permission denied" -expect_file "$rt/proxy.env" "$(proxy_env "$proxy_no")" - -setup PROXY "an unchanged credential leaves the file alone" -proxy_env "$proxy_no" >"$rt/proxy.env" -before=$(stat -c %i "$rt/proxy.env") -go -expect_rc 0 -expect_out "are unchanged" -if [ "$(stat -c %i "$rt/proxy.env")" != "$before" ]; then fail "the unchanged file was replaced"; fi -expect_no_file "$rt/proxy.env.new" - -setup PROXY "a changed credential replaces the file whole" -proxy_env old.example >"$rt/proxy.env" -go -expect_rc 0 -expect_file "$rt/proxy.env" "$(proxy_env "$proxy_no")" - -setup PROXY "a value that is not a base64url secret is refused and the file kept" -proxy_env old.example >"$rt/proxy.env" -kv_out=$'abc\nNO_PROXY=*' -go -expect_rc 1 -expect_err "is not a base64url secret" -expect_file "$rt/proxy.env" "$(proxy_env old.example)" -if grep -qF abc "$dir/out" "$dir/err"; then fail "the refused value was printed"; fi - -setup PROXY "an empty value is refused" -kv_out= -go -expect_rc 1 -expect_err "is not a base64url secret" -expect_no_file "$rt/proxy.env" - -# A hive that predates the facts, or whose squid does not announce itself, gets -# no proxy: the store is not asked and nothing is written. -no_fact() { - expect_rc 0 - expect_err "this hive's facts name no squid proxy" - expect_calls - expect_no_file "$rt/proxy.env" - expect_secret_unprinted -} - -setup PROXY "a hive that writes no egress facts means no proxy" -printf 'HIVE_DASHBOARD_PORT=7000\nHYPERHIVE_HIVE_DOMAIN=t.hive\n' >"$dir/etc/hyperhive-hive-facts" -go -no_fact - -setup PROXY "no facts file means no proxy" -rm "$dir/etc/hyperhive-hive-facts" -go -no_fact - -setup PROXY "a proxy fact without the direct hosts means no proxy" -printf 'HIVE_SQUID_PROXY=10.42.0.1:3128\n' >"$dir/etc/hyperhive-hive-facts" -go -no_fact - -setup PROXY "direct hosts without a proxy fact mean no proxy" -printf 'HIVE_EGRESS_NO_PROXY=10.42.0.1,t.hive\n' >"$dir/etc/hyperhive-hive-facts" -go -no_fact - -for bad in 'not-an-address' 'http://10.42.0.1:3128' '10.42.0.1' 'proxy.t.local:3128'; do - setup PROXY "a proxy fact that is not an address and a port is refused: $bad" - printf 'HIVE_SQUID_PROXY=%s\nHIVE_EGRESS_NO_PROXY=10.42.0.1\n' "$bad" >"$dir/etc/hyperhive-hive-facts" - go - no_fact -done - -setup PROXY "direct hosts that are not host names are refused" -printf 'HIVE_SQUID_PROXY=10.42.0.1:3128\nHIVE_EGRESS_NO_PROXY=a b,c\n' >"$dir/etc/hyperhive-hive-facts" -go -no_fact - -setup PROXY "no facts leave the file in place" -proxy_env old.example >"$rt/proxy.env" -rm "$dir/etc/hyperhive-hive-facts" -go -expect_rc 0 -expect_file "$rt/proxy.env" "$(proxy_env old.example)" - -setup PROXY "the direct hosts are used as the hive wrote them, after the loopback names" -printf 'HIVE_SQUID_PROXY=10.9.8.7:3129\nHIVE_EGRESS_NO_PROXY=10.9.8.7,x.t.hive,y.t.hive\n' >"$dir/etc/hyperhive-hive-facts" -go -expect_rc 0 -expect_file "$rt/proxy.env" "$(proxy_url=http://a1:the-secret@10.9.8.7:3129 proxy_env localhost,127.0.0.1,::1,10.9.8.7,x.t.hive,y.t.hive)" - if [ "$failures" -ne 0 ]; then echo "script-test-agent-bao-fetch: $failures failed assertions over $count runs" >&2 exit 1 diff --git a/nix/script-tests/agent-hive-facts.sh b/nix/script-tests/agent-hive-facts.sh index 636a8be7d..2d1b3517f 100644 --- a/nix/script-tests/agent-hive-facts.sh +++ b/nix/script-tests/agent-hive-facts.sh @@ -40,7 +40,7 @@ check() { for sh in $SHELLS; do # Only what the snippet exported reaches the child's environment. No PATH: # the snippet must not need one. - got=$(env -i "$sh" -c ". '$dir/profile'; exec '$envbin'" | grep -E '^(HYPERHIVE_|HIVE_DASHBOARD_PORT=|HIVE_AGENT_NATS_URL=|HIVE_SQUID_PROXY=|HIVE_EGRESS_NO_PROXY=|CLAUDE_|LD_PRELOAD=|__hyperhive)' | sort) + got=$(env -i "$sh" -c ". '$dir/profile'; exec '$envbin'" | grep -E '^(HYPERHIVE_|HIVE_DASHBOARD_PORT=|HIVE_AGENT_NATS_URL=|CLAUDE_|LD_PRELOAD=|__hyperhive)' | sort) if [ "$got" != "$(printf '%s\n' "$want" | sort)" ]; then fail "$sh: a child of the sourcing shell saw:"$'\n'"$got" fi @@ -68,13 +68,6 @@ HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES=team=a,env=prod HIVE_AGENT_NATS_URL=tls://nats.example.test:4222 CLAUDE_REMOTE_CONTROL_SESSION_NAME_PREFIX=unknown-a1" -# The egress facts are for ./egress-proxy.nix's fetch, which reads the file -# itself: a unit with `HIVE_SQUID_PROXY` in its environment would hold a proxy -# address with no credential to go with it. -printf 'HYPERHIVE_HIVE_NAME=pr1ma\nHIVE_SQUID_PROXY=10.42.0.1:3128\nHIVE_EGRESS_NO_PROXY=10.42.0.1,pr1ma.example.test\n' >"$facts" -check "the egress facts stay out of the environment" "HYPERHIVE_HIVE_NAME=pr1ma -CLAUDE_REMOTE_CONTROL_SESSION_NAME_PREFIX=pr1ma-a1" - : >"$facts" check "an empty file (no facts)" "CLAUDE_REMOTE_CONTROL_SESSION_NAME_PREFIX=unknown-a1" diff --git a/nix/vm-tests/hive-squid.nix b/nix/vm-tests/hive-squid.nix index 7824985f0..735bf3d02 100644 --- a/nix/vm-tests/hive-squid.nix +++ b/nix/vm-tests/hive-squid.nix @@ -1,7 +1,5 @@ # `checks.vm-hive-squid`: runs ../host-modules/hive-squid.nix in a VM, which -# the module-eval suite can only read, and checks that an agent with the proxy -# variables of nix/agent-modules/egress-proxy.nix reaches squid and skips it -# where it should. Three nodes on two networks: +# the module-eval suite can only read. Three nodes on two networks: # # agent (vlan 1) --> proxy (vlan 1 = the stand-in bridge, vlan 2 = uplink) --> web (vlan 2) # @@ -172,8 +170,6 @@ pkgs.testers.runNixOSTest { value = [ p.name ]; }) privateNets ); - # The gateway: nginx answers on the bridge, which agents reach directly. - networking.firewall.interfaces.eth1.allowedTCPPorts = [ 443 ]; # Renders the users file the way hive-c0re does. environment.systemPackages = [ pkgs.mkpasswd ]; services.hyperhive.squid.selfcheck.unrestrictedCanary = "other.t.local"; @@ -215,8 +211,6 @@ pkgs.testers.runNixOSTest { { lib, ... }: { virtualisation.vlans = [ 1 ]; - # The bridge listener, by the name the proxy resolves to the same address. - networking.hosts.${bridgeIp} = [ "bridge.t.local" ]; networking.interfaces.eth1.ipv4.addresses = lib.mkForce [ { address = "192.168.1.3"; @@ -363,45 +357,6 @@ pkgs.testers.runNixOSTest { "http://${bridgeIp}:3128/squid-internal-mgr/counters) = 403" ) - # The variables an agent's `proxy.env` carries (nix/agent-modules/egress-proxy.nix), - # as a curl on the agent reads them. - def agent_env(cred, no_proxy): - url = f"http://{cred}@${bridgeIp}:3128" - pairs = { - "HTTPS_PROXY": url, "HTTP_PROXY": url, "https_proxy": url, "http_proxy": url, - "NO_PROXY": no_proxy, "no_proxy": no_proxy, - } - return " ".join(f"{k}='{v}'" for k, v in pairs.items()) - - def agent_get(cred, no_proxy, url): - # The status of the answer, whoever gave it; 000 when none came. - return agent.succeed( - f"env {agent_env(cred, no_proxy)} curl -sS -m 15 -o /dev/null -w '%{{http_code}}' {url} 2>/dev/null || true" - ).strip() - - def squid_saw(pattern): - return re.search(pattern, proxy.succeed("journalctl -u squid.service -o cat")) - - with subtest("an agent with the proxy variables reaches an allowed name through squid and a swarm-internal name directly"): - NO_PROXY = "localhost,127.0.0.1,::1,bridge.t.local,${bridgeIp}" - # Plain HTTP to the listeners' port 443, so the answer carries a status. - assert agent_get(ATLAS, NO_PROXY, "http://forge.t.local:443/") not in ("000", "403", "407") - assert squid_saw(r"192\.168\.1\.3 TCP_\w+/\d+ .* GET http://forge\.t\.local:443/ atlas "), "through squid, as atlas" - assert agent_get(ATLAS, NO_PROXY, "http://bridge.t.local:443/") not in ("000", "403", "407") - assert not squid_saw(r"GET http://bridge\.t\.local:443/ "), "a swarm-internal name never reached squid" - # Control: without the exemption the same request goes to squid, which refuses the bridge. - assert agent_get(ATLAS, "localhost", "http://bridge.t.local:443/") == "403" - assert squid_saw(r"192\.168\.1\.3 TCP_DENIED/403 .* GET http://bridge\.t\.local:443/ "), "control: squid refuses it" - # CONNECT takes the same path: HTTPS_PROXY names the proxy for https URLs. - assert agent.succeed( - f"env {agent_env(ATLAS, NO_PROXY)} curl -sS -m 15 -o /dev/null -w '%{{http_connect}}' https://forge.t.local/ 2>/dev/null || true" - ).strip() == "200" - - with subtest("an agent without the proxy variables goes direct"): - assert agent.succeed( - "curl -sS -m 15 -o /dev/null -w '%{http_code}' http://bridge.t.local:443/ 2>/dev/null || true" - ).strip() not in ("000", "403", "407") - with subtest("the self-check passes with its credential"): proxy.succeed("install -d -m 0700 /run/credstore") proxy.succeed(f"printf '%s\\n' {PW['selfcheck.squid']} > /run/credstore/hive-squid.selfcheck")